python小工具：用python操做HP的Quality Center （二）----- 用異步方式提升速度

上接第一篇 http://www.cnblogs.com/sdet/p/6874631.html

 

在python中,很簡單地能把http請求經過異步的方式發送，如下代碼在python 3.6.0上運行經過，

import asyncio
import requests

async def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = await future1
    response2 = await future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

按這個思路，把我上次的python 上傳QC的工具移到python 3.6上，作簡單改寫後，上傳文件速度變得飛快。

原來傳1次10個文件，我用時485秒，Time: 485.037000 seconds

如今僅用時75秒，日誌以下：

[2017-06-02 12:45:21,714] post http://xxxx.qc.com:80/qcbin/authentication-point/authenticate
[2017-06-02 12:45:22,401] headers = {'Authorization': 'Basic xxxxxxxxxx'}
[2017-06-02 12:45:22,401] 200 OK
[2017-06-02 12:45:22,401] --------------------
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894980/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894983/attachments
[2017-06-02 12:45:22,401] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894982/attachments
[2017-06-02 12:45:22,416] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894981/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894986/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894987/attachments
[2017-06-02 12:45:22,494] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894984/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894985/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894988/attachments
[2017-06-02 12:45:22,572] post http://xxxx.qc.com:80/qcbin/rest/domains/xxxx_domain/projects/xxxx_project/test-instances/894979/attachments
[2017-06-02 12:45:51,835] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:51,835] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:51,835] 200 OK
[2017-06-02 12:45:51,835] --------------------
[2017-06-02 12:45:59,504] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,504] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,504] 200 OK
[2017-06-02 12:45:59,504] --------------------
[2017-06-02 12:45:59,520] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:45:59,520] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:45:59,520] 200 OK
[2017-06-02 12:45:59,520] --------------------
[2017-06-02 12:46:03,244] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:03,244] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:03,244] 200 OK
[2017-06-02 12:46:03,244] --------------------
[2017-06-02 12:46:05,059] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:05,059] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:05,059] 200 OK
[2017-06-02 12:46:05,059] --------------------
[2017-06-02 12:46:08,068] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:08,068] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:08,068] 200 OK
[2017-06-02 12:46:08,068] --------------------
[2017-06-02 12:46:10,090] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:10,090] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:10,090] 200 OK
[2017-06-02 12:46:10,090] --------------------
[2017-06-02 12:46:13,878] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:13,878] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:13,878] 200 OK
[2017-06-02 12:46:13,878] --------------------
[2017-06-02 12:46:14,299] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:14,299] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:14,299] 200 OK
[2017-06-02 12:46:14,299] --------------------
[2017-06-02 12:46:36,965] data = {'filename': ('', 'xxxx_file_name'), 'override-existing-attachment': ('', 'y')}
[2017-06-02 12:46:36,965] files = {'file': <_io.BufferedReader name='xxxx_file_name'>}
[2017-06-02 12:46:36,965] 200 OK
[2017-06-02 12:46:36,965] --------------------
TIME:  75.2970232963562

 從日誌上看，10個上傳請求，在1秒鐘內從本地發起。並在30秒後開始陸續收到QC服務器的響應。

 

改寫的思路：

 

原來的寫法:

把10個post請求組裝好，在一個for循環中一個一個依次發送。

 

如今的寫法：

先把10個post請求組裝好，交給asyncio的event loop，而後await這些請求。

 response1 = await future1
 response2 = await future2

 這兩句執行的時候，python使用異步的方式，不會阻塞住程序等第一個響應回來，

 而是會直接往下走，把全部請求都發出去。再等服務器一個一個迴應。

 

 因而這個小工具的代碼邏輯就從：

  執行一段多是計算也多是IO的邏輯------》若是是IO的話就等IO完成------》執行一段多是計算也多是IO的邏輯------》這樣繼續循環下去

 變成了：

  執行一段計算--------》異步批量執行一些IO---------》等這一批IO完成----》執行下一段計算--------》異步批量執行一些IO--------》這樣繼續循環下去

 

 小結完畢，繼續學習。。。