python10

 1、多進程multiprocessing

       multiprocessing包是Python中的多進程管理包。與threading.Thread相似，它能夠利用multiprocessing.Process對象來建立一個進程。該Process對象與Thread對象的用法相同，也有start(), run(), join()的方法。

 1 import multiprocessing,threading  2 import time  3  4 def thread_run():  5 print(threading.get_ident())  6 def run(name):  7 time.sleep(2)  8 print('Hello ',name)  9 t = threading.Thread(target=thread_run) 10  t.start() 11 12 13 if __name__== '__main__': 14 for i in range(10): 15 p = multiprocessing.Process(target=run, args=('bob_%s'%i,)) 16 p.start()

運行結果：

 1 Hello bob_0  2 1144  3 Hello bob_8  4 1268  5 Hello bob_4  6 4360  7 Hello bob_2  8 768  9 Hello bob_6 10 5308 11 Hello bob_5 12 Hello bob_1 13 6076 14 5088 15 Hello bob_9 16 6104 17 Hello bob_3 18 5196 19 Hello bob_7 20 748

2、進程池

      若是要建立多個進程，可使用進程池，啓用進程池須要使用Pool庫，使用指令pool=Pool()可自動調用全部CPU，

map()函數至關於一個循環，將參數2中的列表元素逐次灌入參數1的函數中。

 1 from multiprocessing import Pool  2  3 def squre(num):  4 return num ** 2  5  6  7 if __name__ == '__main__':  8 numbers = [0,1,2,3,4,5]  9 pool = Pool(processes=5)#進程池中最多能放入5個進程 10 print(pool.map(squre,numbers))

 運行結果：

[0, 1, 4, 9, 16, 25]

 Pool還有如下經常使用的方法:

• apply_async(func, args)從進程池中取出一個進程執行func，args爲func的參數。它將返回一個AsyncResult的對象，咱們能夠調用get()方法以得到結果。

• close() 關閉進程池，再也不建立新的進程

• join() wait()進程池中的所有進程。可是必須對Pool先調用close()方法才能join.

 1 from multiprocessing import Process, Pool,freeze_support  2 import time  3 import os  4  5 def Foo(i):  6 time.sleep(2)  7 print("in process",os.getpid())  8 return i + 100  9 10 def Bar(arg): 11 print('-->exec done:', arg,os.getpid()) 12 13 if __name__ == '__main__': 14 #freeze_support() 15 pool = Pool(processes=3) 16 print("主進程",os.getpid()) 17 for i in range(10): 18 pool.apply_async(func=Foo, args=(i,), callback=Bar) #callback=回調 19 20 print('end') 21  pool.close() 22 pool.join() 

 運行結果：

 1 主進程 5660
 2 end  3 in process 7048  4 -->exec done: 100 5660  5 in process 3396  6 -->exec done: 101 5660  7 in process 6728  8 -->exec done: 102 5660  9 in process 7048 10 -->exec done: 103 5660 11 in process 3396 12 -->exec done: 104 5660 13 in process 6728 14 -->exec done: 105 5660 15 in process 7048 16 -->exec done: 106 5660 17 in process 3396 18 -->exec done: 107 5660 19 in process 6728 20 -->exec done: 108 5660 21 in process 7048 22 -->exec done: 109 5660

除了主進程，其它結果是三個一組執行的，由於進程池中每次最多有三個進程。

3、進程通訊

      進程間通訊經常使用兩種方法：Queue和pipe,Queue能夠用在多個進程間實現通訊，pipe用在兩個進程間通訊。

 1 import os  2 import multiprocessing  3 import time  4 #==================  5 # input worker  6 def inputQ(queue):  7 info = str(os.getpid()) + '(put):' + str(time.time())  8  queue.put(info)  9 10 # output worker 11 def outputQ(queue,lock): 12 info = queue.get() 13  lock.acquire() 14 print (str(os.getpid()) + '(get):' + info) 15  lock.release() 16 17 if __name__ == '__main__': 18 record1 = [] # store input processes 19 record2 = [] # store output processes 20 lock = multiprocessing.Lock() # To prevent messy print 21 queue = multiprocessing.Queue(3) 22 23 24 for i in range(10): 25 process = multiprocessing.Process(target=inputQ, args=(queue,)) 26  process.start() 27  record1.append(process) 28 29 30 for i in range(10): 31 process = multiprocessing.Process(target=outputQ, args=(queue, lock)) 32  process.start() 33  record2.append(process) 34 35 for p in record1: 36  p.join() 37 38  queue.close() 39 40 for p in record2: 41 p.join()

運行結果：

 1 4004(get):4556(put):1476337412.4875286
 2 512(get):5088(put):1476337412.6345284  3 8828(get):7828(put):1476337412.7965286  4 8372(get):1032(put):1476337412.8185284  5 7740(get):1496(put):1476337412.9205284  6 4176(get):632(put):1476337412.9855285  7 5828(get):8508(put):1476337412.9595284  8 4236(get):9204(put):1476337412.9925284  9 7632(get):8956(put):1476337413.2055285 10 6376(get):4160(put):1476337413.0705285

      Pipe能夠是單向(half-duplex)，也能夠是雙向(duplex)。咱們經過mutiprocessing.Pipe(duplex=False)建立單向管道 (默認爲雙向)。一個進程從PIPE一端輸入對象，而後被PIPE另外一端的進程接收，單向管道只容許管道一端的進程輸入，而雙向管道則容許從兩端輸入。

 1 from multiprocessing import Process, Pipe  2  3  4 def f(conn):  5 conn.send([42, None, 'hello from child'])  6 conn.send([42, None, 'hello from child2'])  7 print("from parent:",conn.recv())  8  conn.close()  9 10 if __name__ == '__main__': 11 parent_conn, child_conn = Pipe() 12 p = Process(target=f, args=(child_conn,)) 13  p.start() 14 print(parent_conn.recv()) # prints "[42, None, 'hello']" 15 print(parent_conn.recv()) # prints "[42, None, 'hello']" 16 parent_conn.send("chupi可好") # prints "[42, None, 'hello']" 17 p.join()

運行結果：

1 [42, None, 'hello from child'] 2 [42, None, 'hello from child2'] 3 from parent: chupi可好