在爬蟲爬取網站的時候,常常遇到一些反爬蟲技術,好比:javascript
對於js加密
通過加密傳輸的就是密文,可是加密函數或者過程必定是在瀏覽器完成,
也就是必定會把js代碼暴露給使用者
經過閱讀加密算法,就能夠模擬出加密過程,從而達到破解
怎樣判斷網站有沒有使用js加密,很簡單,例若有道在線翻譯 html
「salt」: 「1523100789519」, 「sign」: 「b8a55a436686cd8973fa46514ccedbe」,
# 破解js加密,版本2 ''' 經過在js文件中查找salt或者sign,能夠找到 1.能夠找到這個計算salt的公式 r = "" + ((new Date).getTime() + parseInt(10 * Math.random(), 10)) 2.sign:n.md5("fanyideskweb" + t + r + "ebSeFb%=XZ%T[KZ)c(sy!"); md5 一共須要四個參數,第一個和第四個都是固定值得字符串,第三個是所謂的salt, 第二個參數是輸入的須要翻譯的單詞
Python學習交流羣:857662006 ''' from urllib import request, parse def getSalt(): ''' salt的公式r = "" + ((new Date).getTime() + parseInt(10 * Math.random(), 10)) 把它翻譯成python代碼 ''' import time, random salt = int(time.time()*1000) + random.randint(0, 10) return salt def getMD5(v): import hashlib md5 = hashlib.md5() md5.update(v.encode('utf-8')) sign = md5.hexdigest() return sign def getSign(key, salt): sign = "fanyideskweb" + key + str(salt) + "ebSeFb%=XZ%T[KZ)c(sy!" sign = getMD5(sign) return sign def youdao(key): # url從http://fanyi.youdao.com輸入詞彙右鍵檢查獲得 url = "http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=true" salt = getSalt() # data從右鍵檢查FormData獲得 data = { "i": key, "from": "AUTO", "to": "AUTO", "smartresult": "dict", "client": "fanyideskweb", "salt": str(salt), "sign": getSign(key, salt), "doctype": "json", "version": "2.1", "keyform": "fanyi.web", "action": "FY_BY_REALTIME", "typoResult": "false" } print(data) # 對data進行編碼,由於參數data須要bytes格式 data = parse.urlencode(data).encode() # headers從右鍵檢查Request Headers獲得 headers = { "Accept": "application/json, text/javascript, */*; q=0.01", "Accept-Language": "zh-CN,zh;q=0.9", "Connection": "keep-alive", "Content-Length": len(data), "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8", "Cookie": "OUTFOX_SEARCH_USER_ID=685021846@10.168.8.76; OUTFOX_SEARCH_USER_ID_NCOO=366356259.5731474; _ntes_nnid=1f61e8bddac5e72660c6d06445559ffb,1535033370622; JSESSIONID=aaaVeQTI9KXfqfVBNsXvw; ___rl__test__cookies=1535204044230", "Host": "fanyi.youdao.com", "Origin": "http://fanyi.youdao.com", "Referer": "http://fanyi.youdao.com/", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.75 Safari/537.36", "X-Requested-With": "XMLHttpRequest" } req = request.Request(url=url, data=data, headers=headers) rsp = request.urlopen(req) html = rsp.read().decode() print(html) if __name__ == '__main__': youdao("girl")
返回翻譯後的值,纔算是成功java
按照步驟,熟悉流程最重要python
---------------------
原文:https://blog.csdn.net/qq_40147863/article/details/82079649
git