matplotlib how to change figure size but NOT plot size

You can achieve a constant axes sizes by addind the axes manually. In my code example I introduce an scale factor sc which determines the ratio of figure and axes size.

import matplotlib.pyplot as plt

gr = (1 + np.sqrt(5))/2
sc = 2

fig_w = 3 * gr * sc
fig_h = 3 * sc

fig =  plt.figure(figsize=(fig_w, fig_h))

panel_width = 1/sc
panel_height = 1/sc
off = (1 - 1/sc) / 2

ax = fig.add_axes([off, off, panel_width, panel_height])
ax.plot([-1, -4.5, 16, 23, 15, 59])