[python爬蟲]把js轉化成json

有一個優秀的庫能夠使用demjson

示範連接
http://fcd.5173.com/commondat...

若是咱們要把這個變成字典，須要先把他轉成json，再從json變成dict。

經過demjson能夠一步到位

import requests
import demjson

url = 'http://fcd.5173.com/commondata/Category.aspx?type=area&cache=&id=20c8bbc1b9794fc98bd96859624d4769&jsoncallback=callarea'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36',
}

resposne = requests.get(url, headers=headers)

for item in demjson.decode(resposne.text[9:-1]):
    id = item.get('id')
    name = item.get('name')
    print(id, name)

獲得以下結果

直接經過json.loads是行不通的

import requests
import json

url = 'http://fcd.5173.com/commondata/Category.aspx?type=area&cache=&id=20c8bbc1b9794fc98bd96859624d4769&jsoncallback=callarea'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36',
}

resposne = requests.get(url, headers=headers)

print(json.loads(resposne.text[9:-1]))