I would seriously consider using swig instead of pybind11 for example. It’s just peace of mind. If you don’t want to use swig directly, you can at least see what swig does to wrap up features like proxy objects.
CLICK HERE to find out more related problems solutions.