一亿遂、問題
在編寫跨站腳本漏洞輔助驗證腳本時遇到一個問題波岛,當使用 Requests 模塊發(fā)起請求材彪,默認會對 URL 進行一次編碼冠息,比如下面的代碼:
proxy = {"http": "http://127.0.0.1:8080"}
response = requests.get(res, proxies=proxy).text
使用 Burp Suite 抓包可以看到請求 URL 被 URL 編碼了一次:
二、原因
request.get 是通過構造 session 模塊中的Session類荠瘪,并調用其request方法完成請求夯巷。Session類的request方法先根據(jù)傳入的參數(shù)(如URL等)構造一個Request類,并調用self.prepare_request方法預準備哀墓,最后調用self.send發(fā)送最終的請求趁餐。
URL的編碼過程便在self.prepare_request方法中:
# session.py
def prepare_request(self, request):
# 此部分代碼已忽略
p = PreparedRequest()
p.prepare(method=request.method.upper(), url=request.url, files=request.files,
data=request.data, json=request.json,
headers=merge_setting(request.headers, self.headers, dict_class=CaseInsensitiveDict),
params=merge_setting(request.params, self.params), auth=merge_setting(auth, self.auth),
cookies=merged_cookies,hooks=merge_hooks(request.hooks, self.hooks),
)
return p
# model.py
class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
def prepare(self, method=None, url=None, headers=None, files=None,
data=None, params=None, auth=None, cookies=None, hooks=None, json=None):
"""Prepares the entire request with the given parameters."""
self.prepare_method(method)
self.prepare_url(url, params)
self.prepare_headers(headers)
self.prepare_cookies(cookies)
self.prepare_body(data, files, json)
self.prepare_auth(auth, url)
self.prepare_hooks(hooks)
def prepare_url(self, url, params):
# 這里requests庫做了很多努力
url = requote_uri(urlunparse([scheme, netloc, path, None, query, fragment]))
self.url = url
最終我們傳進去的URL就被prepare_url函數(shù)進行了字符串編碼替換,回到Session類篮绰,調用resp = self.send(prep, **send_kwargs)發(fā)送請求后雷,prep即是上述PreparedRequest對象。
三吠各、解決方法
由于URL在PreparedRequest類中轉碼后傳到Session類的send方法中發(fā)送最終的請求臀突,只要寫一個類繼承Session并重寫send方法,在里面進行反編碼贾漏,將PreparedRequest對象的url屬性值改了候学。代碼如下:
#python2
import urllib
import requests
class NoQuoteSession(requests.Session):
def send(self, prep, **send_kwargs):
table = {
urllib.quote('{'): '{',
urllib.quote('}'): '}',
urllib.quote(':'): ':',
urllib.quote(','): ',',
urllib.quote('<'): '<',
urllib.quote('>'): '>',
}
for old, new in table.items():
prep.url = prep.url.replace(old, new)
return super(NoQuoteSeesion, self).send(prep, **send_kwargs)
s = NoQuoteSeesion()
res = s.get(url)
python3 中urllib.quote()變?yōu)?urllib.parse.quote():
#python3
class NoQuoteSession(requests.Session):
def send(self, prep, **send_kwargs):
table = {
urllib.parse.quote('{'): '{',
urllib.parse.quote('}'): '}',
urllib.parse.quote(':'): ':',
urllib.parse.quote(','): ',',
urllib.parse.quote('<'): '<',
urllib.parse.quote('>'): '>',
}
for old, new in table.items():
prep.url = prep.url.replace(old, new)
return super().send(prep, **send_kwargs)
2019 10 17 更新
python 3 中此方法沒用,解碼之后再發(fā)送請求之前纵散,url.py 中的 url()函數(shù)還會 urlencode梳码,所以發(fā)送的請求還是會編碼。
解決方法是使用 urllib 來發(fā)送請求:
with request.urlopen(url) as response:
data = response.read()
return data.decode('utf-8')