1.工欲善其事必先利其器,fiddler安裝python
https://www.telerik.com/fiddlerandroid
2.安裝exe(無腦下一步)json
3.安裝成功後配置fiddler(由於啓動fiddler時連接不能登陸)api
(1)開啓fiddler----->Tools----->Options----->Actions----->Export root,此時桌面就會出現fiddler證書。安全
(2)(Firefox)進入網頁,點擊選項,搜索證書,查看證書,導入證書(FiddlerRoot.cer)所有勾上。網絡
(Google Chrome)點擊設置,進入高級,管理證書,同上導入證書。app
(3)App,首先要設置同一網絡下,設置,手動代理IP,輸入本機IP,端口:8888,同時找到系統安全,安裝上面說到的證書。ui
至此,你們就能夠用fiddler抓包了,上一段簡單的代碼(抓取wawayaya閱讀第一頁圖片)url
重中之重:若是要運行下列代碼,記得信任證書,fiddler----->Tools----->Options----->Actions------>Trust root(否則圖片會加載失敗)3d
#-*- coding: UTF-8 -*- from urllib.request import urlretrieve import requests import os def book_imgs_download(books_url,header): req = requests.get(url = books_url, headers = header).json() a = req["retinfo"] book_num = len(a['libList']) print(book_num) print('一共有%d個種類' % book_num) book_images_path = 'books_images' for each_book in a['libList']: book_photo_url = each_book['imageEn'] book_name = each_book['cname'] + '.jpg' filename = book_images_path + '/' + book_name if book_images_path not in os.listdir(): os.makedirs(book_images_path) urlretrieve(url = book_photo_url, filename = filename) if __name__ == '__main__': headers = {'Accept-Charset': 'UTF-8', 'Accept-Encoding': 'gzip', 'User-Agent': 'Dalvik/2.1.0 (Linux; U; Android 6.0.1; Redmi 4 MIUI/V8.5.3.0.MBECNED)', 'X-Requested-With': 'XMLHttpRequest', 'Content-type': 'application/x-www-form-urlencoded', 'Connection': 'Keep-Alive', 'Host': 'duba.wawayaya.com'} books_url = "http://api3-joyreader.wawayaya.com/api/server/book/getLibHomeList?deviceType=phone&client_lang=zh&appVer=3.8.6&userId=610378&uuid=android_fe087c86fc37533980072e3aa76c244e&platform=android&osVer=23&mac=02:00:00:00:00:00&token=9bdebc55f4f3465def7f3dabd0a0727c&sig=009ff5f1&countryCode=CN&appId=2209410&app_ver=3.8.6&imei=863968035553662×tamp=1513081106655" # req = requests.get(url = heros_url, headers = headers).json() # a = req["retinfo"] # b = a["libList"] # c = b[1] # print(c) book_imgs_download(books_url, headers)
注意:連接失效較快,大約幾分鐘就會失效,這時就要從新獲取連接。並且要注意把headers的東西改爲本身的(
'User-Agent,
'Host
)