Python學習-day10

 1、多進程multiprocessing

       multiprocessing包是Python中的多進程管理包。與threading.Thread相似，它能夠利用multiprocessing.Process對象來建立一個進程。該Process對象與Thread對象的用法相同，也有start(), run(), join()的方法。

 1 import multiprocessing,threading
 2 import time
 3 
 4 def thread_run():
 5     print(threading.get_ident())
 6 def run(name):
 7     time.sleep(2)
 8     print('Hello ',name)
 9     t = threading.Thread(target=thread_run)
10     t.start()
11 
12 
13 if __name__== '__main__':
14     for i in range(10):
15         p = multiprocessing.Process(target=run, args=('bob_%s'%i,))
16         p.start()

View Code

運行結果：

 1 Hello  bob_0
 2 1144
 3 Hello  bob_8
 4 1268
 5 Hello  bob_4
 6 4360
 7 Hello  bob_2
 8 768
 9 Hello  bob_6
10 5308
11 Hello  bob_5
12 Hello  bob_1
13 6076
14 5088
15 Hello  bob_9
16 6104
17 Hello  bob_3
18 5196
19 Hello  bob_7
20 748

View Code

2、進程池

      若是要建立多個進程，可使用進程池，啓用進程池須要使用Pool庫，使用指令pool=Pool()可自動調用全部CPU，

map()函數至關於一個循環，將參數2中的列表元素逐次灌入參數1的函數中。

 1 from multiprocessing import Pool
 2 
 3 def squre(num):
 4     return num ** 2
 5 
 6 
 7 if __name__ == '__main__':
 8     numbers = [0,1,2,3,4,5]
 9     pool = Pool(processes=5)#進程池中最多能放入5個進程
10     print(pool.map(squre,numbers))

View Code

 運行結果：

[0, 1, 4, 9, 16, 25]

 Pool還有如下經常使用的方法:

• apply_async(func, args)從進程池中取出一個進程執行func，args爲func的參數。它將返回一個AsyncResult的對象，咱們能夠調用get()方法以得到結果。

• close() 關閉進程池，再也不建立新的進程

• join() wait()進程池中的所有進程。可是必須對Pool先調用close()方法才能join.

 1 from  multiprocessing import Process, Pool,freeze_support
 2 import time
 3 import os
 4 
 5 def Foo(i):
 6     time.sleep(2)
 7     print("in process",os.getpid())
 8     return i + 100
 9 
10 def Bar(arg):
11     print('-->exec done:', arg,os.getpid())
12 
13 if __name__ == '__main__':
14     #freeze_support()
15     pool = Pool(processes=3)
16     print("主進程",os.getpid())
17     for i in range(10):
18         pool.apply_async(func=Foo, args=(i,), callback=Bar) #callback=回調
19         
20     print('end')
21     pool.close()
22     pool.join() 

View Code

 運行結果：

 1 主進程 5660
 2 end
 3 in process 7048
 4 -->exec done: 100 5660
 5 in process 3396
 6 -->exec done: 101 5660
 7 in process 6728
 8 -->exec done: 102 5660
 9 in process 7048
10 -->exec done: 103 5660
11 in process 3396
12 -->exec done: 104 5660
13 in process 6728
14 -->exec done: 105 5660
15 in process 7048
16 -->exec done: 106 5660
17 in process 3396
18 -->exec done: 107 5660
19 in process 6728
20 -->exec done: 108 5660
21 in process 7048
22 -->exec done: 109 5660

View Code

除了主進程，其它結果是三個一組執行的，由於進程池中每次最多有三個進程。

3、進程通訊

      進程間通訊經常使用兩種方法：Queue和pipe,Queue能夠用在多個進程間實現通訊，pipe用在兩個進程間通訊。

 1 import os
 2 import multiprocessing
 3 import time
 4 #==================
 5 # input worker
 6 def inputQ(queue):
 7     info = str(os.getpid()) + '(put):' + str(time.time())
 8     queue.put(info)
 9 
10 # output worker
11 def outputQ(queue,lock):
12     info = queue.get()
13     lock.acquire()
14     print (str(os.getpid()) + '(get):' + info)
15     lock.release()
16 
17 if __name__ == '__main__':
18     record1 = []  # store input processes
19     record2 = []  # store output processes
20     lock = multiprocessing.Lock()  # To prevent messy print
21     queue = multiprocessing.Queue(3)
22 
23     
24     for i in range(10):
25         process = multiprocessing.Process(target=inputQ, args=(queue,))
26         process.start()
27         record1.append(process)
28 
29    
30     for i in range(10):
31         process = multiprocessing.Process(target=outputQ, args=(queue, lock))
32         process.start()
33         record2.append(process)
34 
35     for p in record1:
36         p.join()
37 
38     queue.close()  
39 
40     for p in record2:
41         p.join()

View Code

運行結果：

 1 4004(get):4556(put):1476337412.4875286
 2 512(get):5088(put):1476337412.6345284
 3 8828(get):7828(put):1476337412.7965286
 4 8372(get):1032(put):1476337412.8185284
 5 7740(get):1496(put):1476337412.9205284
 6 4176(get):632(put):1476337412.9855285
 7 5828(get):8508(put):1476337412.9595284
 8 4236(get):9204(put):1476337412.9925284
 9 7632(get):8956(put):1476337413.2055285
10 6376(get):4160(put):1476337413.0705285

      Pipe能夠是單向(half-duplex)，也能夠是雙向(duplex)。咱們經過mutiprocessing.Pipe(duplex=False)建立單向管道 (默認爲雙向)。一個進程從PIPE一端輸入對象，而後被PIPE另外一端的進程接收，單向管道只容許管道一端的進程輸入，而雙向管道則容許從兩端輸入。

 1 from multiprocessing import Process, Pipe
 2 
 3 
 4 def f(conn):
 5     conn.send([42, None, 'hello from child'])
 6     conn.send([42, None, 'hello from child2'])
 7     print("from parent:",conn.recv())
 8     conn.close()
 9 
10 if __name__ == '__main__':
11     parent_conn, child_conn = Pipe()
12     p = Process(target=f, args=(child_conn,))
13     p.start()
14     print(parent_conn.recv())  # prints "[42, None, 'hello']"
15     print(parent_conn.recv())  # prints "[42, None, 'hello']"
16     parent_conn.send("chupi可好") # prints "[42, None, 'hello']"
17     p.join()

View Code

運行結果：

1 [42, None, 'hello from child']
2 [42, None, 'hello from child2']
3 from parent: chupi可好