Você pode criar um novo dataframe da coluna
page_view_count e join
coluna _id . Último sort_index
:df1 = pd.DataFrame([x for x in df['page_view_count']]).join(df['_id'])
df1 = df1.sort_index(1)
print df1
_id comments detailed-rating detailed-rating2 \
0 568a8c25cac4991645c287ac NaN 2 1
1 568cd22e9e82dfc166d7dff1 1 1 1
2 568e5a38b4a797c664143dda NaN 1 1
3 568e5a561ae56e09656bfb99 NaN 1 NaN
4 568df45a177e30c6487d3600 1 1 1
main-rating thank-you
0 2 NaN
1 1 1
2 1 NaN
3 1 NaN
4 1 1
EDITAR:
Ainda há problemas com
NaN com join.Solution é substituir NaN para esvaziar o dictionary por fillna
e, em seguida, crie Dataframe :import pandas as pd
import numpy as np
df = pd.DataFrame([[1, {'name':'Jack','email':'abc'} ],
[2, np.nan],
[3, {'name':'Ram','email':'xyz'} ],
], columns=['_id','page_view_count'])
print df[df['page_view_count'].isnull()].index
#Int64Index([1], dtype='int64')
print pd.Series([{}], index=df[df['page_view_count'].isnull()].index , name='page_view_count')
#1 {}
#Name: page_view_count, dtype: object
df['page_view_count'] = df['page_view_count'].fillna(pd.Series([{}], index=df[df['page_view_count'].isnull()].index , name='page_view_count'))
print df
# _id page_view_count
#0 1 {u'name': u'Jack', u'email': u'abc'}
#1 2 {}
#2 3 {u'name': u'Ram', u'email': u'xyz'}
df1 = pd.DataFrame([x for x in df['page_view_count']]).join(df['_id'], how='right')
df1 = df1.sort_index(1)
print df1
# _id email name
#0 1 abc Jack
#1 2 NaN NaN
#2 3 xyz Ram
