Split words from datraframe by space to rows while duplicating the info from other columns ( python,pandas)

You need to split and explode:

df2 = (df
 .assign(comments=df['comments'].str.split())
 .explode('comments')
)

output:

   r_id        start    comments
0      1  2021-01-01           i
0      1  2021-01-01          am
0      1  2021-01-01         the
0      1  2021-01-01        text
0      1  2021-01-01        that
0      1  2021-01-01       needs
0      1  2021-01-01   splitting
0      1  2021-01-01          by
0      1  2021-01-01       space
0      1  2021-01-01          to
0      1  2021-01-01        rows
1      2  2021-01-02       hello
1      2  2021-01-02       hello