介紹下3種獲取cookie的方法。cookie
(1)藉助handlerapp
這種方法也是網上介紹最多的一種方法,可是用起來比較麻post
from http import cookiejar
from urllib import request class Craw(): def __init__(self): self.url = '' self.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) ' \ 'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.162 Safari/537.36' self.headers['Content-Type'] = 'application/x-www-form-urlencoded' def getCookies(self): cookie = cookiejar.CookieJar() handler = request.HTTPCookieProcessor(cookie) opener = request.build_opener(handler) response = opener.open(self.url) cookieValue = '' for item in cookie: cookieValue += item.name + '=' + item.value + ';' self.headers['Cookie'] = cookieValue response = requests.get(url=self.url) def getVerificationCode(self):
img_url = '' imgResponse = requests.get(url=img_url,headers = self.headers) #直接使用headers便可 base64_jpg = base64.b64encode(imgResponse.content) return base64_jpg
(2)使用response headers的set_cookie ui
import requests
import re
class Crawler():
def getCookie(self):
response = requests.post(self.url)
set_cookie = response.headers['Set-Cookie']
array = re.split('[;,]',set_cookie)
cookieValue = ''
for arr in array:
if arr.find('DZSW_SESSIONID') >= 0 or arr.find('bl0gm1HBTB') >= 0:
cookieValue += arr + ';'
(3)使用response的cookies屬性獲取url
只寫getCookies方法,代碼以下:spa
import requests class Crawler(): def getCookie(self): response = requests.get(self.url) cookie_value = '' for key,value in response.cookies.items(): cookie_value += key + '=' + value + ';' self.headers['Cookie'] = cookie_value
另外還有個疑問,就是response.cookies返回的是RequestsCookieJar對象,看到書上有這樣的代碼code
import requests class Crawl(): def __init__(self): self.url = '' self.headers = {} self.cookieJar = requests.cookies.RequestsCookieJar() def getCookie() cookies = 'a=a1;b=b1' # 字符串 for cookie in cookies.split(';'): key,value = cookie.split('=,1) self.cookieJar.set(key,value) r = requests.get(self.url,cookies = self.cookieJar,headers = self.headers)
爲何不能夠直接使用response.cookies賦值給最後請求的cookies。已嘗試,結果報錯,最後仍是使用的方法2。
但願熟悉的大神看到能夠給予解答。orm