python爬取喜馬拉雅音頻，突破xm-sign校驗反爬（爬蟲）

 

目錄

1、python執行js代碼生成xm-sign

1.1獲取喜馬拉雅服務器時間戳

1.2生成xm-sign

2、根據須要爬取的專輯Id獲取音頻連接

3、爬取音頻並保存

4、測試

很久沒寫博客了，寫個玩玩

1、python執行js代碼生成xm-sign

1.1獲取喜馬拉雅服務器時間戳

js生成xm-sign的函數須要用到這個時間戳，直接上代碼啦

'''爬取喜馬拉雅服務器系統時間戳，用於生成xm-sign'''
def getxmtime(): headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36', 'Accept': 'text/html,application/xhtml+ xml,application/xml;q = 0.9,image/webp,image/apng,*/*;q=0.8, application/signe-exchange;v = b3', 'Host': 'www.ximalaya.com' } url="https://www.ximalaya.com/revision/time" response = requests.get(url, headers=headers) # print(response)
    html = response.text return html

 

1.2生成xm-sign

這一步須要用到的js代碼請到這裏(CSDN)或者這裏(藍奏雲)下載，以後將其與python代碼放在同一路徑下

'''生成xm-sign'''
def exec_js(): #獲取喜馬拉雅系統時間戳
    time = getxmtime() #讀取同一路徑下的js文件
    with open('xmSign.js', encoding='utf-8') as f: js = f.read() # 經過compile命令轉成一個js對象
    docjs = execjs.compile(js) # 調用js的function
    res = docjs.call('python',time) #res = docjs.call('getnow')
    return res

 

2、根據須要爬取的專輯Id獲取音頻連接

def spider_list(albumId): all_list = [] headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36', 'Accept':'text/html,application/xhtml+ xml,application/xml;q = 0.9,image/webp,image/apng,*/*;q=0.8, application/signe-exchange;v = b3', 'Host': 'www.ximalaya.com' } sign = exec_js() #加入xm-sign到header中
    headers['xm-sign'] = sign # 音頻地址
    data_url = 'https://www.ximalaya.com/revision/play/album?albumId={}&pageNum=1&sort=-1&pageSize=30'.format(albumId) # print(data_url)
    response = requests.get(data_url, headers=headers) py_dict = json.loads(response.content.decode()) # print(py_dict)
    book_list = py_dict['data']['tracksAudioPlay'] for book in book_list: # 獲取每段音頻的名稱和地址
        list = {} list['name'] = book['trackName'] list['bookName'] = book['albumName'] list['src'] = book['src'] # 打印list
        # print(list)
 all_list.append(list) # 返回字典
    return all_list

 

3、爬取音頻並保存

調用第二步中的函數獲取到連接，進行爬取保存

def spider_songs(list): '''保存音頻（字典）'''
    for i in list: dir = "ximalaya/{}/".format(i['bookName']) if not os.path.exists(dir): print("建立目錄:.%s" % dir) os.makedirs(dir) i['name'] = i['name'].replace("?", "").replace('"', "") # 在目錄下建立一個喜馬拉雅的文件夾
        with open(r'{}/{}.m4a'.format(dir, i['name']), 'ab')as f: r = requests.get(i['src']) print("正在下載:{}...".format(i['name']), end="") f.write(r.content) print("\t下載完成！") def spider_ximalaya(albumId): all_list = spider_list(albumId) spider_songs(all_list)

 

4、測試

if __name__ =='__main__':
    spider_ximalaya(12642314)