【轉】python3 urllib.request 網絡請求操做

python3 urllib.request 網絡請求操做

基本的網絡請求示例

''' Created on 2014年4月22日 @author: dev.keke@gmail.com ''' import urllib.request #請求百度網頁 resu = urllib.request.urlopen('http://www.baidu.com', data = None, timeout = 10) print(resu.read(300)) #指定編碼請求 with urllib.request.urlopen('http://www.baidu.com') as resu: print(resu.read(300).decode('GBK')) #指定編碼請求 f = urllib.request.urlopen('http://www.baidu.com') print(f.read(100).decode('utf-8'))

 發送數據請求，CGI程序處理

>>> import urllib.request >>> req = urllib.request.Request(url='https://localhost/cgi-bin/test.cgi', ... data=b'This data is passed to stdin of the CGI') >>> f = urllib.request.urlopen(req) >>> print(f.read().decode('utf-8')) Got Data: "This data is passed to stdin of the CGI"

PUT請求

import urllib.request DATA=b'some data' req = urllib.request.Request(url='http://localhost:8080', data=DATA,method='PUT') f = urllib.request.urlopen(req) print(f.status) print(f.reason)

基本的HTTP驗證，登陸請求

import urllib.request # Create an OpenerDirector with support for Basic HTTP Authentication... auth_handler = urllib.request.HTTPBasicAuthHandler() auth_handler.add_password(realm='PDQ Application', uri='https://mahler:8092/site-updates.py', user='klem', passwd='kadidd!ehopper') opener = urllib.request.build_opener(auth_handler) # ...and install it globally so it can be used with urlopen. urllib.request.install_opener(opener) urllib.request.urlopen('http://www.example.com/login.html')

支持代理方式驗證請求

proxy_handler = urllib.request.ProxyHandler({'http': 'http://www.example.com:3128/'}) proxy_auth_handler = urllib.request.ProxyBasicAuthHandler() proxy_auth_handler.add_password('realm', 'host', 'username', 'password') opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler) # This time, rather than install the OpenerDirector, we use it directly: opener.open('http://www.example.com/login.html')

添加 http headers

import urllib.request req = urllib.request.Request('http://www.example.com/') req.add_header('Referer', 'http://www.python.org/') r = urllib.request.urlopen(req)

添加 user-agent

import urllib.request opener = urllib.request.build_opener() opener.addheaders = [('User-agent', 'Mozilla/5.0')] opener.open('http://www.example.com/')

帶參數的GET 請求

>>> import urllib.request >>> import urllib.parse >>> params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) >>> f = urllib.request.urlopen("http://www.musi-cal.com/cgi-bin/query?%s" % params) >>> print(f.read().decode('utf-8'))

帶參數的POST請求

>>> import urllib.request >>> import urllib.parse >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) >>> data = data.encode('utf-8') >>> request = urllib.request.Request("http://requestb.in/xrbl82xr") >>> # adding charset parameter to the Content-Type header. >>> request.add_header("Content-Type","application/x-www-form-urlencoded;charset=utf-8") >>> f = urllib.request.urlopen(request, data) >>> print(f.read().decode('utf-8'))

指定代理方式請求

>>> import urllib.request >>> proxies = {'http': 'http://proxy.example.com:8080/'} >>> opener = urllib.request.FancyURLopener(proxies) >>> f = opener.open("http://www.python.org") >>> f.read().decode('utf-8')

無添加代理

>>> import urllib.request >>> opener = urllib.request.FancyURLopener({}) >>> f = opener.open("http://www.python.org/") >>> f.read().decode('utf-8')

http://www.cnblogs.com/cocoajin/p/3679821.html