python小工具:用python操做HP的Quality Center (二)----- 用異步方式提升速度

上接第一篇 http://www.cnblogs.com/sdet/p/6874631.htmlhtml

 

在python中,很簡單地能把http請求經過異步的方式發送,如下代碼在python 3.6.0上運行經過,python

import asyncio
import requests

async def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = await future1
    response2 = await future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

按這個思路,把我上次的python 上傳QC的工具移到python 3.6上,作簡單改寫後,上傳文件速度變得飛快。服務器

原來傳1次10個文件,我用時485秒,Time: 485.037000 secondsdom

如今僅用時75秒,日誌以下:異步

[2017-06-02 12:45:21,714] post http://xxxx.qc.com:80/qcbin/authentication-point/authenticate
[2017-06-02 12:45:22,401] headers = {'Authorization': 'Basic xxxxxxxxxx'}
[2017-06-02 12:45:22,401] 200 OK
[2017-06-02 12:45:22,401] --------------------
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894980/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894983/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894982/attachments
[2017-06-02 12:45:22,416] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894981/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894986/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894987/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894984/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894985/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894988/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894979/attachments
[2017-06-02 12:45:51,835] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:51,835] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:51,835] 200 OK
[2017-06-02 12:45:51,835] --------------------
[2017-06-02 12:45:59,504] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,504] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,504] 200 OK
[2017-06-02 12:45:59,504] --------------------
[2017-06-02 12:45:59,520] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,520] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,520] 200 OK
[2017-06-02 12:45:59,520] --------------------
[2017-06-02 12:46:03,244] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:03,244] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:03,244] 200 OK
[2017-06-02 12:46:03,244] --------------------
[2017-06-02 12:46:05,059] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:05,059] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:05,059] 200 OK
[2017-06-02 12:46:05,059] --------------------
[2017-06-02 12:46:08,068] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:08,068] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:08,068] 200 OK
[2017-06-02 12:46:08,068] --------------------
[2017-06-02 12:46:10,090] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:10,090] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:10,090] 200 OK
[2017-06-02 12:46:10,090] --------------------
[2017-06-02 12:46:13,878] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:13,878] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:13,878] 200 OK
[2017-06-02 12:46:13,878] --------------------
[2017-06-02 12:46:14,299] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:14,299] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:14,299] 200 OK
[2017-06-02 12:46:14,299] --------------------
[2017-06-02 12:46:36,965] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:36,965] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:36,965] 200 OK
[2017-06-02 12:46:36,965] --------------------
TIME:  75.2970232963562

 從日誌上看,10個上傳請求,在1秒鐘內從本地發起。並在30秒後開始陸續收到QC服務器的響應。async

 

改寫的思路:ide

 

原來的寫法:工具

把10個post請求組裝好,在一個for循環中一個一個依次發送。oop

 

如今的寫法:post

先把10個post請求組裝好,交給asyncio的event loop,而後await這些請求。

 response1 = await future1
 response2 = await future2

 這兩句執行的時候,python使用異步的方式,不會阻塞住程序等第一個響應回來,

 而是會直接往下走,把全部請求都發出去。再等服務器一個一個迴應。

 

 因而這個小工具的代碼邏輯就從:

  執行一段多是計算也多是IO的邏輯------》若是是IO的話就等IO完成------》執行一段多是計算也多是IO的邏輯------》這樣繼續循環下去

 變成了:

  執行一段計算--------》異步批量執行一些IO---------》等這一批IO完成----》執行下一段計算--------》異步批量執行一些IO--------》這樣繼續循環下去

 

 小結完畢,繼續學習。。。

相關文章
相關標籤/搜索