multiprocessing多進程模塊

1 基本理解

　　python不支持真多線程，沒法在單進程中調用多核cpu。對於cpu密集型任務，可使用多進程。python會調用OS原生多進程，運行在多核上，以此提升運行速度。

 

2 基本實現

import  multiprocessing
import  time

def test(n):
    print('this is the num %s process' %n)
    time.sleep(100)

if __name__ == '__main__':
    for i in range(10):
        mp = multiprocessing.Process(target=test,args=(i,))
        mp.start()
        #mp.join()
#和threading模塊使用方式基本一致
#調用10個進程，系統就會同步運行10個進程
#若是在主循環中使用join方法，就會等待子進程運行結束後再往下走。

　　

3 父子進程間關係

import  multiprocessing
import  time,os

def test(n):
    print('this is the num %s process,pid is %s parent pid is %s' %(n,os.getpid(),os.getppid()))
    time.sleep(100)

if __name__ == '__main__':
    print('the pid is %s, the parent pid is %s' %(os.getpid(),os.getppid()))
    for i in range(10):
        mp = multiprocessing.Process(target=test,args=(i,))
        mp.start()

　　

[root@yhzk01 scripts]# python3 mp.py 
the pid is 128430, the parent pid is 102584
this is the num 0 process,pid is 128431 parent pid is 128430
this is the num 1 process,pid is 128432 parent pid is 128430
this is the num 2 process,pid is 128433 parent pid is 128430
this is the num 8 process,pid is 128439 parent pid is 128430
this is the num 5 process,pid is 128436 parent pid is 128430
this is the num 6 process,pid is 128437 parent pid is 128430
this is the num 3 process,pid is 128434 parent pid is 128430
this is the num 7 process,pid is 128438 parent pid is 128430
this is the num 4 process,pid is 128435 parent pid is 128430
this is the num 9 process,pid is 128440 parent pid is 128430

[root@yhzk01 ~]# ps aux |grep mp.py
root     128430  0.2  0.8 141552  8148 pts/0    S+   15:26   0:00 python3 mp.py
root     128431  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128432  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128433  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128434  0.0  0.6 141552  6412 pts/0    S+   15:26   0:00 python3 mp.py
root     128435  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128436  0.0  0.6 141552  6416 pts/0    S+   15:26   0:00 python3 mp.py
root     128437  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128438  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128439  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128440  0.0  0.6 141552  6420 pts/0    S+   15:26   0:00 python3 mp.py
root     128444  0.0  0.0 112660   984 pts/1    S+   15:26   0:00 grep --color=auto mp.py

[root@yhzk01 ~]# ps aux |grep 102584
root     102584  0.0  0.2 116308  2640 pts/0    Ss   May30   0:00 -bash
root     128442  0.0  0.0 112660   976 pts/1    R+   15:26   0:00 grep --color=auto 102584

#可見，bash進程啓動子進程mp.py。mp.py再啓動多個子進程。

　　

4 class Queue()

　　線程共享內存數據，進程的內存是獨立的。

　　在queue模塊中，有class Queue，但這個隊列只是在線程間通訊的。

　　在mulitprocessing模塊中，也有class Queue，能夠提供父進程與子進程或者子進程之間的數據交互。