最近網上面試看到了有關這方面的問題,因爲近兩個月這些庫或多或少都用過,如今根據本身的經驗和網上介紹來總結一下。html
httplibpython
實現了HTTP和HTTPS的客戶端協議,通常不直接使用,在python更高層的封裝模塊中(urllib,urllib2)使用了它的http實現。因此效率要比這兩個庫高一點。http://www.cnblogs.com/beiyeren/p/4046139.html面試
#-*- coding:utf-8 –*- from urlparse import urlparse import httplib from hds.utils import get_logger import json from exception import HTTPException class HttpClienttMonitor(object): def __init__(self, logger=None): self.logger = logger or get_logger("HttpClienttMonitor") def send_request_with_body(self, url, content, method, token): self.logger.info("url: "+url) self.logger.info(method+ ": "+ str(content)) assert len(url) != 0,"url is empty" assert len(token) !=0,"token is empty" o = urlparse(url) content = json.dumps(content).encode('utf-8') headers = {"Content-Type":"application/json"} headers["Authorization"] = token httpClient = httplib.HTTPConnection(o.hostname, o.port) try: httpClient.set_debuglevel(0) httpClient.request(method, o.path+"?"+o.query, content, headers) response = httpClient.getresponse() if not response: return "No response from httpClient" self.logger.info("status: "+str(response.status)) self.logger.info("reason: " + response.reason) return response.status except Exception as e: raise HTTPException(e) finally: if httpClient: httpClient.close() class HTTP_REQUEST_METHOD(): get = "GET" post = "POST" delete = "DELETE" put = "PUT" patch = "PATCH"
urllib2
can accept a Request
object to set the headers for a URL request,urllib
accepts only a URL. That means, you cannot masquerade your User Agent string etc.urllib2能夠接受一個Request類的實例來設置URL請求的headers,urllib僅能夠接受URL。這意味着,你不能夠假裝你的User Agent字符串等。編程
urllib
provides the urlencode
method which is used for the generation of GET query strings, urllib2
doesn't have such a function. This is one of the reasons why urllib
is often used along with urllib2
.urllib提供urlencode方法用來GET查詢字符串的產生,而urllib2沒有。這是爲什麼urllib常和urllib2一塊兒使用的緣由。json
pycurl瀏覽器
關於python網絡編程,使用urllib與服務器通訊時,客戶端的數據是添加到Head裏面經過URL,發送到服務器端,urllib包實現客戶端上傳文件時,會出現死掉的狀態(其實是超時設置問題)。
Pycurl包是一個libcurl的Python接口,它是由C語言編寫的。與urllib相比,它的速度要快不少。
Libcurl 是一個支持FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE 和 LDAP的客戶端URL傳輸庫.libcurl也支持HTTPS認證,HTTP POST,HTTP PUT,FTP上傳,代理,Cookies,基自己份驗證,FTP文件斷點繼傳,HTTP代理通道等等。服務器
class CurlTest(object): def post(self): provisionContent["metadata"] = metadata b = StringIO.StringIO() c = pycurl.Curl() c.setopt(pycurl.VERBOSE, 1) c.setopt(pycurl.URL, remoteServiceURL) c.setopt(pycurl.HTTPHEADER, ["Accept: application/json", "Content-Type: application/json" ])
c.setopt(pycurl.WRITEFUNCTION, b.write) c.setopt(pycurl.FOLLOWLOCATION, 1) c.setopt(pycurl.MAXREDIRS, 5) c.setopt(c.POSTFIELDS, json.dumps(provisionContent)) c.perform() print c.getinfo(c.HTTP_CODE), c.getinfo(c.EFFECTIVE_URL), c.POSTFIELDS print b.getvalue() self.response = eval(b.getvalue()) b.close() c.close()
其中c.setopt(pycurl.WRITEFUNCTION, b.write) #把StringIO的寫函數註冊到pycurl的WRITEFUNCTION中,即pycurl全部獲取的內容都寫入到StringIO中,若是沒有這一句,pycurl就會把全部的內容在默認的輸出器中輸出網絡
pycurl經常使用操做以下:app
pycurl.Curl() #建立一個pycurl對象的方法
pycurl.Curl(pycurl.URL, http://www.google.com.hk) #設置要訪問的URL
pycurl.Curl().setopt(pycurl.MAXREDIRS, 5) #設置最大重定向次數
pycurl.Curl().setopt(pycurl.CONNECTTIMEOUT, 60)
pycurl.Curl().setopt(pycurl.TIMEOUT, 300) #鏈接超時設置
pycurl.Curl().setopt(pycurl.USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)") #模擬瀏覽器
pycurl.Curl().perform() #服務器端返回的信息
pycurl.Curl().getinfo(pycurl.HTTP_CODE) #查看HTTP的狀態 相似urllib中status屬性
pycurl.NAMELOOKUP_TIME 域名解析時間
pycurl.CONNECT_TIME 遠程服務器鏈接時間
pycurl.PRETRANSFER_TIME 鏈接上後到開始傳輸時的時間
pycurl.STARTTRANSFER_TIME 接收到第一個字節的時間
pycurl.TOTAL_TIME 上一請求總的時間
pycurl.REDIRECT_TIME 若是存在轉向的話,花費的時間
pycurl.EFFECTIVE_URL
pycurl.HTTP_CODE HTTP 響應代碼
pycurl.REDIRECT_COUNT 重定向的次數
pycurl.SIZE_UPLOAD 上傳的數據大小
pycurl.SIZE_DOWNLOAD 下載的數據大小
pycurl.SPEED_UPLOAD 上傳速度
pycurl.HEADER_SIZE 頭部大小
pycurl.REQUEST_SIZE 請求大小
pycurl.CONTENT_LENGTH_DOWNLOAD 下載內容長度
pycurl.CONTENT_LENGTH_UPLOAD 上傳內容長度
pycurl.CONTENT_TYPE 內容的類型
pycurl.RESPONSE_CODE 響應代碼
pycurl.SPEED_DOWNLOAD 下載速度
pycurl.SSL_VERIFYRESULT
pycurl.INFO_FILETIME 文件的時間信息
pycurl.HTTP_CONNECTCODE HTTP 鏈接代碼
pycurl.HTTPAUTH_AVAIL
pycurl.PROXYAUTH_AVAIL
pycurl.OS_ERRNO
pycurl.NUM_CONNECTS
pycurl.SSL_ENGINES
pycurl.INFO_COOKIELIST
pycurl.LASTSOCKET
pycurl.FTP_ENTRY_PATHcurl