problem faced when getting json keys from an array of jsons in a hive

Changing the value passed to split function has worked for me. Please check once:

get_json_object  (spark_event,concat('$.Stage Infos[',e.i,'].Stage ID'))    as stage_id,
get_json_object  (spark_event,concat('$.Stage Infos[',e.i,'].Stage Attempt ID'))   as atmpt_id,
get_json_object  (spark_event,concat('$.Stage Infos[',e.i,'].Stage Name'))    as stage_name,
get_json_object  (spark_event,concat('$.Stage Infos[',e.i,'].Number of Tasks'))   as number_tasks from    json_example t
lateral view    json_tuple  (spark_event,'Event','Job ID','Submission Time') j as event,job_id,submission_time
lateral view    posexplode  (split(get_json_object (spark_event,'$.Stage Infos[*]'),'},')) e as i,x;

event   job_id  submission_time stage_id    atmpt_id    stage_name  number_tasks
Mohan   1   18032203    2   0   Stage_name  1394
Mohan   1   18032203    3   0   Stage_name2 1894

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top