can i create a datetime64ns variable or use the betweentime function?

You need DatetimeIndex from column ts, one possible solution is use DatetimeIndex or convert column to index and select .index:

df['ts'] = pd.to_datetime(df['ts'])
df = df.iloc[pd.DatetimeIndex(df['ts']).indexer_between_time(datetime.time(8,20,0),
                                                             datetime.time(8,27,0))]

df['ts'] = pd.to_datetime(df['ts'])
df = df.iloc[df.set_index('ts').index.indexer_between_time(datetime.time(8,20,0),
                                                           datetime.time(8,27,0))]

print (df)
                        ts  value
15 2019-10-18 08:23:26.375     40
16 2019-10-18 08:23:26.527     49
17 2019-10-18 08:23:26.725     48

But simplier solution is if use DatetimeIndex – convert column ts by DataFrame.set_index and then use DataFrame.between_time:

df['ts'] = pd.to_datetime(df['ts'])
df = df.set_index('ts').between_time(datetime.time(8,20,0), datetime.time(8,27,0))
print (df)
                         value
ts                            
2019-10-18 08:23:26.375     40
2019-10-18 08:23:26.527     49
2019-10-18 08:23:26.725     48