python 併發編程 協程 gevent模塊

 

 

一 gevent模塊

gevent應用場景：

單線程下，多個任務，io密集型程序

安裝

pip3 install gevent

 

Gevent 是一個第三方庫，能夠輕鬆經過gevent實現併發同步或異步編程，在gevent中用到的主要模式是Greenlet, 它是以C擴展模塊形式接入Python的輕量級協程。 Greenlet所有運行在主程序操做系統進程的內部，但它們被協做式地調度。

gevent能夠檢測io，實現遇到io自動切換另一個任務

 

#用法
g1=gevent.spawn(func,1,2,3,x=4,y=5)
建立一個協程對象g1，spawn括號內第一個參數是函數名，如eat，後面能夠有多個參數，能夠是位置實參或關鍵字實參，都是傳給函數eat的 g2=gevent.spawn(func2) g1.join() #等待g1結束
 g2.join() #等待g2結束

#或者上述兩步合做一步：gevent.joinall([g1,g2])
 g1.value#拿到func1的返回值

 

遇到io馬上 切換到另一個任務，這是使用gevent.sleep 本身產生的io操做

import gevent
import time

def eat(name):
    print("%s:eat 1" %name)
    gevent.sleep(3)
    print("%s:eat 2" %name)

def play(name):
    print("%s:play 1" % name)
    gevent.sleep(4)
    print("%s:play 2" % name)


g1 = gevent.spawn(eat,"mike")
g2 = gevent.spawn(play,"mike")

start_time = time.time()
g1.join()
g2.join()

end_time = time.time()
print(end_time-start_time)

'''
mike:eat 1
mike:play 1
mike:eat 2
mike:play 2
4.0012290477752686
'''

 

 

而time.sleep(2)或其餘的阻塞,gevent是不能直接識別的，遇到io不切換另一個任務

gevent模塊 只能識別本身模擬的io操做，其餘io操做沒法識別

import gevent
import time

def eat(name):
    print("%s:eat 1" %name)
    time.sleep(3)
    print("%s:eat 2" %name)

def play(name):
    print("%s:play 1" % name)
    time.sleep(4)
    print("%s:play 2" % name)


g1 = gevent.spawn(eat,"mike")
g2 = gevent.spawn(play,"mike")

start_time = time.time()
g1.join()
g2.join()

end_time = time.time()
print(end_time-start_time)

'''
mike:eat 1
mike:eat 2
mike:play 1
mike:play 2
7.0004003047943115
'''

 

須要用下面一行代碼,打補丁,就能夠識別了，

from gevent import monkey;monkey.patch_all()必須放到被打補丁者的前面，如time，socket模塊以前

或者咱們乾脆記憶成：要用gevent，須要將from gevent import monkey;monkey.patch_all()放到文件的開頭

 

把monkey.pathch_all() 下面全部代碼的涉及到io阻塞操做都打個標記，變成非阻塞操做，讓gevent能夠識別

這樣就能夠檢測到io操做

from gevent import monkey;monkey.patch_all()
import gevent
import time

def eat(name):
    print("%s:eat 1" %name)
    time.sleep(3)
    print("%s:eat 2" %name)

def play(name):
    print("%s:play 1" % name)
    time.sleep(4)
    print("%s:play 2" % name)


g1 = gevent.spawn(eat,"mike")
g2 = gevent.spawn(play,"mike")

start_time = time.time()
g1.join()
g2.join()

end_time = time.time()
print(end_time-start_time)

'''
mike:eat 1
mike:play 1
mike:eat 2
mike:play 2
4.032230854034424
'''

 

join()  主線程等待任務運行完後才銷燬

joinall()  等待多個任務 用列表存聽任務

from gevent import monkey;monkey.patch_all()
import gevent
import time

def eat(name):
    print("%s:eat 1" %name)
    time.sleep(3)
    print("%s:eat 2" %name)

def play(name):
    print("%s:play 1" % name)
    time.sleep(4)
    print("%s:play 2" % name)


g1 = gevent.spawn(eat,"mike")
g2 = gevent.spawn(play,"mike")

gevent.joinall([g1,g2])