poop infinite loop parallel processing does not stop

This is a simplified version of your code and the way you are using multiprocessing.Pool seems correct. The program can hang if the code in the subprocesses is malfunctioning. Are you sure that your pd.read_sql code exits with a good status?

import datetime
from multiprocessing.pool import Pool
from os import getpid

yms = range(1, 10000)


def minc(ym):
    print('MINC %s %s %s\n' % (ym, getpid(), str(datetime.datetime.now())))

    return ym


def parallelized():
    pool = Pool(processes = 8)
    df = pool.map(minc, yms)
    print("FINISHED")
    assert list(yms) == df

    # you don't really have to close/join but it is still a good idea
    # https://stackoverflow.com/a/38271957/598057
    pool.close()
    pool.join()

parallelized()

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top