Do full-outer-join with pandas.merge_asof

This seems simple enough yet no direct solution. There’s an option to merge again to bring in the missing rows:

# enumerate the rows of `df2` to later identify which are missing
df2 = df2.reset_index().assign(idx=np.arange(df2.shape[0]))
(pd.merge_asof(df1.reset_index(), 
               df2[['Time','idx']], 
              on='Time',
              direction='nearest', 
              tolerance=pd.Timedelta('0.3s'))
  .merge(df2, on='idx', how='outer')                        # merge back on row number
  .assign(Time=lambda x: x['Time_x'].fillna(x['Time_y']))   # fill the time
  .set_index(['Time'])                                      # set index back
  .drop(['Time_x','Time_y','idx'], axis=1)
  .sort_index()
)

                               Value1  Value2
Time                                         
2020-07-17 14:25:03.535906075   108.0   222.0
2020-07-17 14:25:04.545104980     NaN   150.0
2020-07-17 14:25:05.457247019   110.0     NaN
2020-07-17 14:25:07.467777014   126.0    60.0

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top