输出:(Win和Linux下输出相似)
main process id 5236 starts at 0.000002s start assigning tasks all tasks assigned; wait for son processes to finish son process id 8724 starts working on TaskNo.1 at 0.030557s with parameter 1 son process id 14584 starts working on TaskNo.2 at 0.037581s with parameter 1 son process id 10028 starts working on TaskNo.3 at 0.041210s with parameter 1 son process id 14584 ends working on TaskNo.2 at 1.042662s son process id 8724 ends working on TaskNo.1 at 1.040211s son process id 14584 starts working on TaskNo.4 at 1.044109s with parameter 1 son process id 10028 ends working on TaskNo.3 at 1.054017s son process id 14584 ends working on TaskNo.4 at 2.055515s all tasks finished at 2.214534s main process id 5236 ends at 2.214884s当使用apply_async(“异步调用”)添加任务时,主进程在子进程执行任务期间会继续运行;如果用apply(“同步调用”)添加任务,则主进程会暂停(“阻塞”)直到该任务完成。一般使用apply_async而不是apply。
进程池中的进程复制 from multiprocessing import Pool import os, time all_tasks_on_this_son_process = [] def task(duration, base_time, task_name): global all_tasks_on_this_son_process pid = os.getpid() print(f'son process id {pid} starts working on {task_name} at {"%.6f" % (time.perf_counter()-base_time)}s with parameter {duration}, this process already executed',all_tasks_on_this_son_process) time.sleep(duration) print(f'son process id {pid} ends working on {task_name} at {"%.6f" % (time.perf_counter()-base_time)}s') all_tasks_on_this_son_process += [task_name] if __name__ == '__main__': pid = os.getpid() base_time = time.perf_counter() print(f'main process id {pid} starts at {"%.6f" % (time.perf_counter()-base_time)}s') pool = Pool(3) print('start assigning tasks') for i in range(4): pool.apply_async(task, args=(1, base_time, "TaskNo."+str(i+1))) pool.close() print('all tasks assigned; wait for son processes to finish') pool.join() print(f'all tasks finished at {"%.6f" % (time.perf_counter()-base_time)}s') print(f'main process id {pid} ends at {"%.6f" % (time.perf_counter()-base_time)}s') print('gu?')Windows下输出:
main process id 6116 starts at 0.000001s start assigning tasks all tasks assigned; wait for son processes to finish gu? gu? gu? son process id 16028 starts working on TaskNo.1 at 0.037577s with parameter 1, this process already executed [] son process id 11696 starts working on TaskNo.2 at 0.041393s with parameter 1, this process already executed [] son process id 5400 starts working on TaskNo.3 at 0.038409s with parameter 1, this process already executed [] son process id 11696 ends working on TaskNo.2 at 1.041521s son process id 16028 ends working on TaskNo.1 at 1.038722s son process id 11696 starts working on TaskNo.4 at 1.042543s with parameter 1, this process already executed ['TaskNo.2'] son process id 5400 ends working on TaskNo.3 at 1.052573s son process id 11696 ends working on TaskNo.4 at 2.053483s all tasks finished at 2.167447s main process id 6116 ends at 2.167904s gu?在Windows下,池中的每个线程会在(且仅在)它分配到的的第一个任务将要开始执行时,运行一遍父进程的代码以构建运行环境。一个进程在前一个任务中对运行环境的改变,会原样体现在下一个任务的运行环境里。(即接受新任务的时候会直接继续使用上一个任务遗留下的运行环境)
Linux下输出:
main process id 691 starts at 0.000001s all tasks assigned; wait for son processes to finish son process id 692 starts working on TaskNo.1 at 0.104757s with parameter 1, this process already executed [] son process id 693 starts working on TaskNo.2 at 0.104879s with parameter 1, this process already executed [] son process id 694 starts working on TaskNo.3 at 0.105440s with parameter 1, this process already executed [] son process id 692 ends working on TaskNo.1 at 1.106427s son process id 693 ends working on TaskNo.2 at 1.106426s son process id 694 ends working on TaskNo.3 at 1.107157s son process id 692 starts working on TaskNo.4 at 1.107560s with parameter 1, this process already executed ['TaskNo.1'] son process id 692 ends working on TaskNo.4 at 2.110033s all tasks finished at 2.117158s main process id 691 ends at 2.117452s gu?在Linux下,池中的每个线程会在(且仅在)它的第一个任务将要开始执行时,从父进程将运行环境完整复制一遍。一个进程在前一个任务中对运行环境的改变,会原样体现在下一个任务的运行环境里。(即接受新任务的时候会直接继续使用上一个任务遗留下的运行环境)
from multiprocessing import Pool import os, time all_tasks_on_this_son_process = [] def init(init_name): global all_tasks_on_this_son_process all_tasks_on_this_son_process += [init_name] def task(duration, base_time, task_name): global all_tasks_on_this_son_process pid = os.getpid() print(f'son process id {pid} starts working on {task_name} at {"%.6f" % (time.perf_counter()-base_time)}s with parameter {duration}, this process already executed',all_tasks_on_this_son_process) time.sleep(duration) print(f'son process id {pid} ends working on {task_name} at {"%.6f" % (time.perf_counter()-base_time)}s') all_tasks_on_this_son_process += [task_name] if __name__ == '__main__': pid = os.getpid() base_time = time.perf_counter() print(f'main process id {pid} starts at {"%.6f" % (time.perf_counter()-base_time)}s') pool = Pool(3, initializer=init, initargs=('init',)) # look here print('start assigning tasks') for i in range(4): pool.apply_async(task, args=(1, base_time, "TaskNo."+str(i+1))) pool.close() print('all tasks assigned; wait for son processes to finish') pool.join() print(f'all tasks finished at {"%.6f" % (time.perf_counter()-base_time)}s') print(f'main process id {pid} ends at {"%.6f" % (time.perf_counter()-base_time)}s')