什么是Urllib
Urllib是Python3內置的HTTP請求庫
| 常用模塊 | 釋義 |
|---|---|
| urllib.request | 請求模塊 |
| urllib.error | 異常處理模塊 |
| urllib.parse | url解析模塊 |
| urllib.robotparser | robots.txt解析模塊 |
例1:發(fā)送最基本的get請求,獲取html代碼---request.urlopen()
from urllib import request
response = request.urlopen('http://www.baidu.com')#發(fā)送請求,獲取響應內容
result = response.read()#提取響應的字節(jié)流
print(result.decode('utf-8'))#將字節(jié)流轉換為'utf-8',并打印
例2:post請求數(shù)據data(需要bytes類型)---parse.urlencode()
from urllib import request,parse
data = bytes(parse.urlencode({'word':'hi'}),encoding='utf-8')
response = request.urlopen('http://httpbin.org/post',data=data)
print(response.read())
例3:創(chuàng)造請求頭,增加headers---request.Request()
from urllib import request,parse
url = 'http://httpbin.org/post'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36",
}
dict = {
'name':'Hello',
}
data = bytes(parse.urlencode(dict),encoding='utf-8')
req = request.Request(url=url,data=data,headers=headers,method='POST')
response = request.urlopen(req)
print(response.read().decode('utf-8'))
