python pickle string in multiprocessing

think what it’s doing is trying to pickle the ProcessPoolExecutor because the function you’re trying to run in the pool is a bound method. The object it is bound to is a Sample that includes a reference to your ProcessPoolExecutor. If I create the ProcessPoolExecutor as a top-level object not in the class, your code runs fine.

Python serializes (pickles, by default) objects moved back and forth between the process spaces rather than relying on any kind of shared memory that might be possible between two processes. So you have to be careful what your function has access to that Python has to bring along into the other process your function will run in.

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top