眾所周知，使用常規(guī)方法爬取淘寶的難度是很高的，所以使用selenium+瀏覽器幾乎成了爬取淘寶最理想的方法补鼻。然而現(xiàn)在淘寶搜索需要用戶登錄才可以，據(jù)說是為了收集匹配用戶信息。

taobao.gif

而每次使用selenium時(shí)（+chrome）风范，chromedriver會(huì)新打開一個(gè)瀏覽器窗口咨跌，所以添加cookie便顯得很必要了。
這里看一下selenium的add_cookie方法

20190105143721122.png

與普通的在headers里添加{'Cookies':' '}不一樣的是硼婿，此方法需要按照cookie的name,value,path,domain格式逐個(gè)cookie添加锌半，如下圖

淘寶cookies格式

淘寶的cookie很多，所以需要逐項(xiàng)添加
而手動(dòng)復(fù)制的話很不方便寇漫，所以這里使用到一個(gè)chrome插件
EditThisCookie

EditThisCookie

它有個(gè)導(dǎo)出功能刊殉，當(dāng)你登錄完淘寶后點(diǎn)擊導(dǎo)出便會(huì)得到一個(gè)list格式的字符串

cookies=
[
{
    "domain": ".baidu.com",
    "expirationDate": 1563853177.468827,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BAIDUID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "C0E2BCF5DCEB680B2453F78AC4CD0956:FG=1",
    "id": 1
},
{
    "domain": ".baidu.com",
    "expirationDate": 1546756058.11283,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BDORZ",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "FFFB88E999055A3F8A630C64834BD6D0",
    "id": 2
},
{
    "domain": ".baidu.com",
    "expirationDate": 3648534764.317173,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BIDUPSID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "CC0841B75FADCE2D56D45D685391FED0",
    "id": 3
},
{
    "domain": ".baidu.com",
    "hostOnly": false,
    "httpOnly": false,
    "name": "H_PS_PSSID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": true,
    "storeId": "0",
    "value": "",
    "id": 4
},
{
    "domain": ".baidu.com",
    "expirationDate": 2493441539.633233,
    "hostOnly": false,
    "httpOnly": false,
    "name": "MCITY",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "-%3A",
    "id": 5
},
{
    "domain": ".baidu.com",
    "expirationDate": 3648534764.317246,
    "hostOnly": false,
    "httpOnly": false,
    "name": "PSTM",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "1501051117",
    "id": 6
},
{
    "domain": ".www.baidu.com",
    "expirationDate": 2492662056,
    "hostOnly": false,
    "httpOnly": false,
    "name": "bdime",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "0",
    "id": 7
}
]

稍加修改就可以作為python的list來導(dǎo)入cookie了

for item in cookies:
    driver.add_cookie(item)

之后使用selenium就可以正常搜索了

淘寶搜索