1.使用requests
要通過(guò)GET訪問(wèn)一個(gè)頁(yè)面,只需要幾行代碼:
import requests
r = requests.get('https://www.douban.com/')#豆瓣首頁(yè)
requestCode = r.status_code
print('請(qǐng)求狀態(tài)值:',requestCode)
requestData = r.text
# print('請(qǐng)求 Data',requestData);
2.對(duì)于帶參數(shù)的 URL,傳入一個(gè) dict 作為 params 參數(shù)
r1 = requests.get('https://www.douban.com/search',params={'q':'python','cat':'1001'})
requestsUrl = r1.url
print('請(qǐng)求的 url:',requestsUrl)
#requests自動(dòng)檢測(cè)編碼族操,可以使用 encoding 屬性來(lái)查看:
requestsEncode = r.encoding
print('requests自動(dòng)檢測(cè)編碼:',requestsEncode)
# 無(wú)論響應(yīng)是文本還是二進(jìn)制內(nèi)容柑晒,我們都可以用 content 屬性獲得 bytes 對(duì)象
requestsBytes = r.content
# print('requests bytes 對(duì)象:',requestsBytes)
3.requests的方便在于對(duì)于特定類型的響應(yīng),例如 JSON,可以直接獲取
r2 = requests.get('https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20%3D%202151330&format=json')
requestsJson = r2.json()
# print('requestsJson:',requestsJson)
4.需要傳入HTTP Header時(shí)辫继,我們傳入一個(gè)dict作為headers參數(shù):
r3 = requests.get('https://www.douban.com/',headers={'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit'})
requestsText = r3.text
# print('requestsText:',requestsText)
5.要發(fā)送POST請(qǐng)求惯殊,只需要把get()方法變成post(),然后傳入data參數(shù)作為POST請(qǐng)求的數(shù)據(jù):
r4 = requests.post('https://accounts.douban.com/login', data={'form_email': 'abc@example.com', 'form_password': '123456'})
requestsR4 = r4.text
# print('requestsR4:',requestsR4)
requests默認(rèn)使用application/x-www-form-urlencoded對(duì)POST數(shù)據(jù)編碼昌粤。如果要傳遞JSON數(shù)據(jù)既绕,可以直接傳入json參數(shù):
params = {'key':'value'}
r5 = requests.post(url,json=params)#內(nèi)部自動(dòng)序列化 JOSN
# 類似的,上傳文件需要更復(fù)雜的編碼格式涮坐,但是requests把它簡(jiǎn)化成files參數(shù):
upload_files = {'file':open('report.xls','rb')}
r6 = requests.post(url,file=upload_files)
在讀取文件時(shí)凄贩,注意務(wù)必使用'rb'即二進(jìn)制模式讀取,這樣獲取的bytes長(zhǎng)度才是文件的長(zhǎng)度袱讹。
把post()方法替換為put()怎炊,delete()等,就可以以PUT或DELETE方式請(qǐng)求資源廓译。
除了能輕松獲取響應(yīng)內(nèi)容外评肆,requests對(duì)獲取HTTP響應(yīng)的其他信息也非常簡(jiǎn)單。例如非区,獲取響應(yīng)頭:
requestsHeaders = r4.headers
print('requestsHeaders:',requestsHeaders)
requests對(duì)Cookie做了特殊處理瓜挽,使得我們不必解析Cookie就可以輕松獲取指定的Cookie:
requestsCookies = r3.cookies['ts']
print('requestsCookies:',requestsCookies)
要在請(qǐng)求中傳入Cookie,只需準(zhǔn)備一個(gè)dict傳入cookies參數(shù):
url = 'https://www.douban.com/'
cs = {'token':'12345','status':'working'}
r6 = requests.get(url,cookies=cs)
print('requestsr6:',r6.content)
最后征绸,要指定超時(shí)久橙,傳入以秒為單位的timeout參數(shù):
r7 = requests.get(url,timeout=2.5)
print('requestsr7:',r7.text)
6.獲取請(qǐng)求頭
rd = requests.get('https://www.douban.com/search', params={'q':'python', 'cat':'1001'})
print(rd.status_code, rd.url)
h = rd.headers
print('練習(xí)一:獲取請(qǐng)求頭')
for k, v in h.items():
print(k,': ', v)