how can i not change the file extension while changing a file name?

You can use pathlib.Path objects. It has name and suffix attributes, and a rename method:

import re
from pathlib import Path
for file in Path(r'C:\tmp').glob('*'):
    if not file.is_file():
        continue
    new_name = re.sub('\d','', file.stem) + file.suffix
    file.rename(file.parent/new_name)

The parent attribute gives the folder the file belongs to and the is_file method is used to check that we are handing a regular file (and not a folder). New Path objects are created easily with the / operator (full new filepath is file.parent / new_name).

The re.sub() is used to replace the numbers (\d means a digit) in the old filename stem part.