requests源碼框架淺析

本文主要是對 requests 的 for human 結構的部分進行簡單分析，對於裏面具體的功能實現（好比cookies如何存儲，http相關對接）沒有深刻研究。

1 對於requests主要模塊說明：

1.1 __init__.py: 寫入了requests的各類方法，能夠直接調用

1.2 api.py: 定義了requests 的請求的各類方法（get, post, put, delete...)

1.3 session.py: 主要定義了Session類，裏面封裝這各類session的具體請求方法(Session.request()..)

1.4 model.py: 定義了各類模型（Request, Response)

1.5 utils.py: 定義了中途須要用的各類方法

 

2 requests的具體請求過程

html = requests.get('https://www.baidu.com', headers=headers).text

html2 = requests.Session().get('https://www.xxx.com', coookies=cookies).text

2.1 requests or session

無論是 requests.get 仍是 requests.Session.get 他都會先去 調用調用 session.request(method=method, url=url, **kwargs) 方法

    with sessions.Session() as session:
        return session.request(method=method, url=url, **kwargs)

***此處補一句，  定義方法時候傳入**kwargs這樣類型的參數，很方便後面對象或方法接收參數。***

2.2 session.request 過程

因此如今咱們來看 session.request 這個方法

    def request(self, method, url,
        params=None,
        data=None,
        headers=None,
        cookies=None,
        files=None,
        auth=None,
        timeout=None,
        allow_redirects=True,
        proxies=None,
        hooks=None,
        stream=None,
        verify=None,
        cert=None,
        json=None):
        """Constructs a :class:`Request <Request>`, prepares it and sends it.
        Returns :class:`Response <Response>` object.

..."""
        # Create the Request.
        req = Request(
            method = method.upper(),
            url = url,
            headers = headers,
            files = files,
            data = data or {},
            json = json,
            params = params or {},
            auth = auth,
            cookies = cookies,
            hooks = hooks,
        )
        prep = self.prepare_request(req)

        proxies = proxies or {}

        settings = self.merge_environment_settings(
            prep.url, proxies, stream, verify, cert
        )

        # Send the request.
        send_kwargs = {
            'timeout': timeout,
            'allow_redirects': allow_redirects,
        }
        send_kwargs.update(settings)
        resp = self.send(prep, **send_kwargs)

        return resp

2.3 Request類

首先將接收到的各類參數（headers, cookies,...）傳入Request類進行封裝

class Request(RequestHooksMixin):

    def __init__(self, method=None, url=None, headers=None, files=None,
        data=None, params=None, auth=None, cookies=None, hooks=None, json=None):

        # Default empty dicts for dict params.
        data = [] if data is None else data
        files = [] if files is None else files
        headers = {} if headers is None else headers
        params = {} if params is None else params
        hooks = {} if hooks is None else hooks

        self.hooks = default_hooks()
        for (k, v) in list(hooks.items()):
            self.register_hook(event=k, hook=v)

        self.method = method
        self.url = url
        self.headers = headers
        self.files = files
        self.data = data
        self.json = json
        self.params = params
        self.auth = auth
        self.cookies = cookies

    # ...

2.4 對request和傳入的參數進行處理

而後將封裝好的對象傳入 prepare_request(self, request) 方法

    def prepare_request(self, request):

        cookies = request.cookies or {}

        # Bootstrap CookieJar.
        if not isinstance(cookies, cookielib.CookieJar):
            cookies = cookiejar_from_dict(cookies)

        # Merge with session cookies
        merged_cookies = merge_cookies(
            merge_cookies(RequestsCookieJar(), self.cookies), cookies)

        # Set environment's basic authentication if not explicitly set.
        auth = request.auth
        if self.trust_env and not auth and not self.auth:
            auth = get_netrc_auth(request.url)

        p = PreparedRequest()
        p.prepare(
            method=request.method.upper(),
            url=request.url,
            files=request.files,
            data=request.data,
            json=request.json,
            headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict),
            params=merge_setting(request.params, self.params),
            auth=merge_setting(auth, self.auth),
            cookies=merged_cookies,
            hooks=merge_hooks(request.hooks, self.hooks),
        )
        return p

該方法主要是對傳入個各類參數（headers, cookies,...）進行相應的處理以便後續調用，

這裏他先將cookies進行了處理，而後對全部傳入的參數進行相應處理，調用PrepareRequest.prepare 方法進行處理

    def prepare(self, method=None, url=None, headers=None, files=None,
        data=None, params=None, auth=None, cookies=None, hooks=None, json=None):
        """Prepares the entire request with the given parameters."""

        self.prepare_method(method)
        self.prepare_url(url, params)
        self.prepare_headers(headers)
        self.prepare_cookies(cookies)
        self.prepare_body(data, files, json)
        self.prepare_auth(auth, url)

        # Note that prepare_auth must be last to enable authentication schemes
        # such as OAuth to work on a fully prepared request.

        # This MUST go after prepare_auth. Authenticators could add a hook
        self.prepare_hooks(hooks)

能夠看出，每一個具體的請求參數都有對應的參數進行處理。

2.5 發送獲得response

request請求對象設計好後，send(request)，獲得 response