httplib urllib urllib2 pycurl 比較

最近網上面試看到了有關這方面的問題,因爲近兩個月這些庫或多或少都用過,如今根據本身的經驗和網上介紹來總結一下。html

httplibpython

實現了HTTP和HTTPS的客戶端協議,通常不直接使用,在python更高層的封裝模塊中(urllib,urllib2)使用了它的http實現。因此效率要比這兩個庫高一點。http://www.cnblogs.com/beiyeren/p/4046139.html面試

 

#-*- coding:utf-8 –*-
from urlparse import urlparse
import httplib
from hds.utils import get_logger
import json
from exception import HTTPException



class HttpClienttMonitor(object):
    def __init__(self, logger=None):
        self.logger = logger or get_logger("HttpClienttMonitor")

    def send_request_with_body(self, url, content, method, token):
        self.logger.info("url: "+url)
        self.logger.info(method+ ": "+ str(content))
        assert len(url) != 0,"url is empty"
        assert len(token) !=0,"token is empty"
        o = urlparse(url)
        content = json.dumps(content).encode('utf-8')
        headers = {"Content-Type":"application/json"}
        headers["Authorization"] = token
        httpClient = httplib.HTTPConnection(o.hostname, o.port)

        try:
            httpClient.set_debuglevel(0)
            httpClient.request(method, o.path+"?"+o.query, content, headers)
            response = httpClient.getresponse()
            if not response:
                return "No response from httpClient"
            self.logger.info("status: "+str(response.status))
            self.logger.info("reason: " + response.reason)
            return response.status
        except Exception as e:
            raise HTTPException(e)
        finally:
            if httpClient:
                httpClient.close()


class HTTP_REQUEST_METHOD():
    get = "GET"
    post = "POST"
    delete = "DELETE"
    put = "PUT"
    patch = "PATCH"
  urllib2urllib各有各自的特色:
  • urllib2 can accept a Request object to set the headers for a URL request,urllib accepts only a URL. That means, you cannot masquerade your User Agent string etc.

urllib2能夠接受一個Request類的實例來設置URL請求的headers,urllib僅能夠接受URL。這意味着,你不能夠假裝你的User Agent字符串等。編程

  • urllib provides the urlencode method which is used for the generation of GET query strings, urllib2 doesn't have such a function. This is one of the reasons why urllib is often used along with urllib2.

urllib提供urlencode方法用來GET查詢字符串的產生,而urllib2沒有。這是爲什麼urllib常和urllib2一塊兒使用的緣由。json

 

pycurl瀏覽器

關於python網絡編程,使用urllib與服務器通訊時,客戶端的數據是添加到Head裏面經過URL,發送到服務器端,urllib包實現客戶端上傳文件時,會出現死掉的狀態(其實是超時設置問題)。
Pycurl包是一個libcurl的Python接口,它是由C語言編寫的。與urllib相比,它的速度要快不少。
Libcurl 是一個支持FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE 和 LDAP的客戶端URL傳輸庫.libcurl也支持HTTPS認證,HTTP POST,HTTP PUT,FTP上傳,代理,Cookies,基自己份驗證,FTP文件斷點繼傳,HTTP代理通道等等。服務器

class CurlTest(object):

    def post(self):
        provisionContent["metadata"] = metadata

        b = StringIO.StringIO()
        c = pycurl.Curl()
        c.setopt(pycurl.VERBOSE, 1)
        c.setopt(pycurl.URL, remoteServiceURL)
        c.setopt(pycurl.HTTPHEADER, ["Accept: application/json",
                                     "Content-Type: application/json"
                                     ])
     c.setopt(pycurl.WRITEFUNCTION, b.write) c.setopt(pycurl.FOLLOWLOCATION,
1) c.setopt(pycurl.MAXREDIRS, 5) c.setopt(c.POSTFIELDS, json.dumps(provisionContent)) c.perform() print c.getinfo(c.HTTP_CODE), c.getinfo(c.EFFECTIVE_URL), c.POSTFIELDS print b.getvalue() self.response = eval(b.getvalue()) b.close() c.close()

其中c.setopt(pycurl.WRITEFUNCTION, b.write) #把StringIO的寫函數註冊到pycurl的WRITEFUNCTION中,即pycurl全部獲取的內容都寫入到StringIO中,若是沒有這一句,pycurl就會把全部的內容在默認的輸出器中輸出網絡

pycurl經常使用操做以下:app

pycurl.Curl() #建立一個pycurl對象的方法
pycurl.Curl(pycurl.URL, http://www.google.com.hk) #設置要訪問的URL
pycurl.Curl().setopt(pycurl.MAXREDIRS, 5) #設置最大重定向次數
pycurl.Curl().setopt(pycurl.CONNECTTIMEOUT, 60)
pycurl.Curl().setopt(pycurl.TIMEOUT, 300) #鏈接超時設置
pycurl.Curl().setopt(pycurl.USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)") #模擬瀏覽器
pycurl.Curl().perform() #服務器端返回的信息
pycurl.Curl().getinfo(pycurl.HTTP_CODE) #查看HTTP的狀態 相似urllib中status屬性
pycurl.NAMELOOKUP_TIME 域名解析時間
pycurl.CONNECT_TIME 遠程服務器鏈接時間
pycurl.PRETRANSFER_TIME 鏈接上後到開始傳輸時的時間
pycurl.STARTTRANSFER_TIME 接收到第一個字節的時間
pycurl.TOTAL_TIME 上一請求總的時間
pycurl.REDIRECT_TIME 若是存在轉向的話,花費的時間
pycurl.EFFECTIVE_URL
pycurl.HTTP_CODE HTTP 響應代碼
pycurl.REDIRECT_COUNT 重定向的次數
pycurl.SIZE_UPLOAD 上傳的數據大小
pycurl.SIZE_DOWNLOAD 下載的數據大小
pycurl.SPEED_UPLOAD 上傳速度
pycurl.HEADER_SIZE 頭部大小
pycurl.REQUEST_SIZE 請求大小
pycurl.CONTENT_LENGTH_DOWNLOAD 下載內容長度
pycurl.CONTENT_LENGTH_UPLOAD 上傳內容長度
pycurl.CONTENT_TYPE 內容的類型
pycurl.RESPONSE_CODE 響應代碼
pycurl.SPEED_DOWNLOAD 下載速度
pycurl.SSL_VERIFYRESULT
pycurl.INFO_FILETIME 文件的時間信息
pycurl.HTTP_CONNECTCODE HTTP 鏈接代碼
pycurl.HTTPAUTH_AVAIL
pycurl.PROXYAUTH_AVAIL
pycurl.OS_ERRNO
pycurl.NUM_CONNECTS
pycurl.SSL_ENGINES
pycurl.INFO_COOKIELIST
pycurl.LASTSOCKET
pycurl.FTP_ENTRY_PATHcurl

相關文章
相關標籤/搜索