簡單的Python爬蟲應(yīng)用_學(xué)習(xí)筆記003

從京東網(wǎng)站抓取_007

`接上篇內(nèi)容 從京東網(wǎng)站抓取_006

組織成列表的形式

if __name__ == '__main__':
    bose_info = get_content_from_jd('Bose', '1', '3')  
    bose_content = BeautifulSoup(bose_info)
    bose_div = bose_content.find_all('div', class_ = 'lh-wrap') 

    bose_id = [i.strong['class'][0] for i in bose_div]

    info_results = [] #初始化一個空的列表用來存放抓取的商品信息

    for item in bose_div:
        data = {} #初始化一個空的字典用來存放信息
        data['url'] = item.find('div', class_ = 'p-img').a['href']#商品鏈接
        data['img'] = item.find('img')['data-lazyload']#商品圖片
        data['title'] = item.find('div', class_ = 'p-name').get_text().strip()#商品標(biāo)題
        data['id'] = item.find('div', class_ = 'p-price').strong['class'][0]#商品id
        info_results.append(data)

    print info_results[0] #通過下標(biāo)可查詢?nèi)我庖粭l商品信息

返回內(nèi)容

{
'url': 'http://item.jd.com/1150767.html', 
'id': 'J_1150767', 
'img': 'http://img12.360buyimg.com/n7/jfs/t160/263/2138317114/91577/2ad0b9a5/53c64152Nfa21b93d.jpg', 
'title': u'Bose SoundLink Mini\u84dd\u7259\u626c\u58f0\u5668 \u84dd\u7259\u97f3\u7bb1'
}

商品價格的抓取
京東網(wǎng)站價格通過GET命令獲取集歇，通過chrome調(diào)試工具先獲取價格網(wǎng)址

Json.PNG

http://p.3.cn/prices/mgets?skuids=J_1150767,J_1225287,J_1150768,J_1237665,J_1439929797,J_1150689,J_1225340,J_1264417,J_1153768,J_1253203,J_1150696,J_1150765,J_1237667,J_1150699,J_1253201,J_1253221,J_1253226,J_1253232,J_1237663,J_1237666,J_1253123,J_1253126,J_1222097,J_1150701,J_1150735,J_1230490,J_1217252,J_1237670,J_1255249,J_1150760&area=1_72_2799_0&type=1

此網(wǎng)址由三個參數(shù)組成skuid,area,type

構(gòu)建函數(shù)獲取商品價格
函數(shù)接受所有商品ID桶略，之后構(gòu)建如上鏈接，獲取商品價格

import json
def get_price(all_id):
  params = {

            'skuids':','.join(all_id),
            'area':'1_72_2799_0',
            'type':'1'
  }
  html_price = urllib.urlencode(params)
  html_price_content = urllib.urlopen('http://p.3.cn/prices/mgets?'+html_price)
  price_content = html_price_content.read()
  html_price_content.close()
  return json.loads(price_content)

修改主程序增加商品價格字段

if __name__ == '__main__':
    bose_info = get_content_from_jd('Bose', '1', '3')  
    bose_content = BeautifulSoup(bose_info)
    bose_div = bose_content.find_all('div', class_ = 'lh-wrap') 
    # <strong class="J_1150767"></strong>
    bose_id = [i.strong['class'][0] for i in bose_div]

    bose_price = get_price(bose_id)

    info_results = []

    for item in bose_div:
        data = {}

        data['url'] = item.find('div', class_ = 'p-img').a['href']
        data['img'] = item.find('img')['data-lazyload']
        data['title'] = item.find('div', class_ = 'p-name').get_text().strip()
        data['id'] = item.find('div', class_ = 'p-price').strong['class'][0]
        data['price'] = filter(lambda price: price['id'] == data['id'], bose_price)[0]['p']

        info_results.append(data)
    for i in info_results:
        print json.dumps(i, encoding="gb2312", ensure_ascii=False)

返回結(jié)果

{"url": "http://item.jd.com/1150767.html", "price": "1880.00", "id": "J_1150767", "img": "http://img12.360buyimg.com/n7/jfs/t160/263/2138317114/91577/2ad0b9a5/53c64152Nfa21b93d.jpg", "title": "Bose SoundLink Mini藍(lán)牙揚(yáng)聲器 藍(lán)牙音箱"}
{"url": "http://item.jd.com/1225287.html", "price": "2680.00", "id": "J_1225287", "img": "http://img12.360buyimg.com/n7/jfs/t382/299/358410988/102067/28bc7fa/5417ef6eN1a471400.jpg", "title": "Bose QuietComfort25有源消噪耳機(jī)-黑色  QC25"}
{"url": "http://item.jd.com/1150768.html", "price": "2600.00", "id": "J_1150768", "img": "http://img13.360buyimg.com/n7/jfs/t208/86/2195379170/121054/373a1635/53c6414cNa684fbeb.jpg", "title": "Bose SoundLink 藍(lán)牙揚(yáng)聲器III 藍(lán)牙音箱"}
{"url": "http://item.jd.com/1237665.html", "price": "999.00", "id": "J_1237665", "img": "http://img10.360buyimg.com/n7/jfs/t340/338/1222059663/138599/83529a01/5434b619Nd24a3d4b.jpg", "title": "Bose SoundLink Colour藍(lán)牙揚(yáng)聲器-紅色"}
{"url": "http://item.jd.com/1439929797.html", "price": "138.00", "id": "J_1439929797", "img": "http://img12.360buyimg.com/n7/jfs/t700/310/356730772/324161/83da6250/54a3bc7eNfa25365f.jpg", "title": "山水D11迷你音響便攜式插卡收音機(jī)老人帶mp3音樂播放器外放小音箱 紅色"}
{"url": "http://item.jd.com/1150689.html", "price": "1390.00", "id": "J_1150689", "img": "http://img14.360buyimg.com/n7/jfs/t208/317/2213636018/63579/6e4b116f/53c748efN7baca4a7.jpg", "title": "Bose SoundTrue 耳罩式耳機(jī)-薄荷黑"}
{"url": "http://item.jd.com/1225340.html", "price": "2680.00", "id": "J_1225340", "img": "http://img10.360buyimg.com/n7/jfs/t223/352/2430825655/67306/a9994f48/5417ef44N0c7a8927.jpg", "title": "Bose QuietComfort25有源消噪耳機(jī)-白色  QC25"}
{"url": "http://item.jd.com/1264417.html", "price": "230.00", "id": "J_1264417", "img": "http://img12.360buyimg.com/n7/jfs/t517/72/112548631/56020/8bda8115/544ef80cN9a010b1f.jpg", "title": "Bose SoundLink Mini藍(lán)牙揚(yáng)聲器封套-橙色 藍(lán)牙音箱配件"}
{"url": "http://item.jd.com/1153768.html", "price": "2900.00", "id": "J_1153768", "img": "http://img13.360buyimg.com/n7/jfs/t166/272/2180604273/69161/fb60368e/53c6403dNee2eb466.jpg", "title": "Bose QC3有源消噪耳機(jī)"}
{"url": "http://item.jd.com/1253203.html", "price": "860.00", "id": "J_1253203", "img": "http://img13.360buyimg.com/n7/jfs/t322/59/1919666393/48470/e8feae8a/5444db09N9675a5b6.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-Audio白色"}
{"url": "http://item.jd.com/1150696.html", "price": "1390.00", "id": "J_1150696", "img": "http://img11.360buyimg.com/n7/jfs/t148/316/2168034496/55985/33491bc7/53c748f6N18f6bba2.jpg", "title": "Bose SoundTrue 耳罩式耳機(jī)-白色"}
{"url": "http://item.jd.com/1150765.html", "price": "1190.00", "id": "J_1150765", "img": "http://img10.360buyimg.com/n7/jfs/t301/274/605583376/53605/b339d875/541a76b2N5ccb18e3.jpg", "title": "Bose SoundTrue 貼耳式耳機(jī)-薄荷綠"}
{"url": "http://item.jd.com/1237667.html", "price": "999.00", "id": "J_1237667", "img": "http://img12.360buyimg.com/n7/jfs/t289/182/1218390339/163662/9fe085a4/5434b621Naedfdec8.jpg", "title": "Bose SoundLink Colour藍(lán)牙揚(yáng)聲器-藍(lán)色"}
{"url": "http://item.jd.com/1150699.html", "price": "1100.00", "id": "J_1150699", "img": "http://img14.360buyimg.com/n7/jfs/t181/31/2177504157/57468/1c4635a/53c748feN97183aa9.jpg", "title": "BOSE FreeStyle耳塞式耳機(jī)-靛藍(lán)色"}
{"url": "http://item.jd.com/1253201.html", "price": "860.00", "id": "J_1253201", "img": "http://img11.360buyimg.com/n7/jfs/t346/156/1911266815/54489/7c8e3997/5444db0dNa623d2b8.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-Audio黑色"}
{"url": "http://item.jd.com/1253221.html", "price": "1100.00", "id": "J_1253221", "img": "http://img11.360buyimg.com/n7/jfs/t289/140/1935678963/56537/582956b2/5444db00N572caa4a.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-MFI黑色"}
{"url": "http://item.jd.com/1253226.html", "price": "1100.00", "id": "J_1253226", "img": "http://img11.360buyimg.com/n7/jfs/t277/71/1930074315/50436/5acf672a/5444dafcN0c4791d5.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-MFI白色"}
{"url": "http://item.jd.com/1253232.html", "price": "1100.00", "id": "J_1253232", "img": "http://img12.360buyimg.com/n7/jfs/t328/196/1930225986/61739/bbec7731/5444daf9Nddbecfb6.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-MFI紅色"}
{"url": "http://item.jd.com/1237663.html", "price": "999.00", "id": "J_1237663", "img": "http://img13.360buyimg.com/n7/jfs/t319/242/1280480196/152009/5fe7709a/5434b615Nbcfa5d7c.jpg", "title": "Bose SoundLink Colour藍(lán)牙揚(yáng)聲器-白色"}
{"url": "http://item.jd.com/1237666.html", "price": "999.00", "id": "J_1237666", "img": "http://img11.360buyimg.com/n7/jfs/t304/242/1241674581/146981/c06eaa1f/5434b61dN9c7696b3.jpg", "title": "Bose SoundLink Colour藍(lán)牙揚(yáng)聲器-黑色"}
{"url": "http://item.jd.com/1253123.html", "price": "1280.00", "id": "J_1253123", "img": "http://img13.360buyimg.com/n7/jfs/t292/198/1897986217/53974/f562c7e1/5444db17Ne54d0602.jpg", "title": "Bose SoundSport 耳塞式運(yùn)動耳機(jī)-MFI藍(lán)色"}
{"url": "http://item.jd.com/1253126.html", "price": "1280.00", "id": "J_1253126", "img": "http://img11.360buyimg.com/n7/jfs/t292/189/1898519967/51313/51a8a428/5444db10Nf3ff2fd3.jpg", "title": "Bose SoundSport 耳塞式運(yùn)動耳機(jī)-MFI綠色"}
{"url": "http://item.jd.com/1222097.html", "price": "1300.00", "id": "J_1222097", "img": "http://img12.360buyimg.com/n7/jfs/t361/202/251727987/123343/f26572be/5412c577N18d44163.jpg", "title": "Bose SoundDock XT 揚(yáng)聲器-黃色"}
{"url": "http://item.jd.com/1150701.html", "price": "1100.00", "id": "J_1150701", "img": "http://img11.360buyimg.com/n7/jfs/t190/337/2171347628/71312/aaee683/53c74901N1f1f18c6.jpg", "title": "BOSE FreeStyle耳塞式耳機(jī)-冰藍(lán)色"}
{"url": "http://item.jd.com/1150735.html", "price": "2100.00", "id": "J_1150735", "img": "http://img10.360buyimg.com/n7/jfs/t145/287/2185253825/90804/2249f972/53c748dfN2ddf2080.jpg", "title": "Bose AE2w藍(lán)牙 音樂耳機(jī)"}
{"url": "http://item.jd.com/1230490.html", "price": "3900.00", "id": "J_1230490", "img": "http://img10.360buyimg.com/n7/jfs/t271/122/797716279/40566/938f086d/54250485Nd234341a.jpg", "title": "Bose Solo 15 電視音響"}
{"url": "http://item.jd.com/1217252.html", "price": "280.00", "id": "J_1217252", "img": "http://img12.360buyimg.com/n7/jfs/t391/286/362695776/68919/aa56f3ad/5417d812N5f28a56b.jpg", "title": "Bose SoundLink 藍(lán)牙揚(yáng)聲器III封套-灰色"}
{"url": "http://item.jd.com/1237670.html", "price": "200.00", "id": "J_1237670", "img": "http://img10.360buyimg.com/n7/jfs/t289/169/1495591883/259103/d0ae681e/543ceff6N239ff5f6.jpg", "title": "Bose SoundLink Colour藍(lán)牙揚(yáng)聲器 便攜包"}
{"url": "http://item.jd.com/1255249.html", "price": "988.00", "id": "J_1255249", "img": "http://img14.360buyimg.com/n7/jfs/t280/299/1915466812/56681/67fa4c6b/5444dae9Na3c39a8c.jpg", "title": "Bose SoundTrue 耳塞式耳機(jī)-SMSG黑色"}
{"url": "http://item.jd.com/1150760.html", "price": "1190.00", "id": "J_1150760", "img": "http://img10.360buyimg.com/n7/jfs/t172/238/2201679276/52515/cab70722/53c748baN815e7a20.jpg", "title": "Bose SoundTrue 貼耳式耳機(jī)-白色"}

最后編輯于：2017.11.27 00:34:01

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者

人面猴
序言：七十年代末诲宇，一起剝皮案震驚了整個濱河市际歼，隨后出現(xiàn)的幾起案子，更是在濱河造成了極大的恐慌焕窝，老刑警劉巖蹬挺，帶你破解...
沈念sama閱讀 217,406評論 6贊 503
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件，死亡現(xiàn)場離奇詭異它掂，居然都是意外死亡，警方通過查閱死者的電腦和手機(jī)溯泣，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 92,732評論 3贊 393
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進(jìn)店門虐秋，熙熙樓的掌柜王于貴愁眉苦臉地迎上來，“玉大人垃沦，你說我怎么就攤上這事客给。” “怎么了肢簿？”我有些...
開封第一講書人閱讀 163,711評論 0贊 353
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵靶剑，是天一觀的道長。經(jīng)常有香客問我池充，道長桩引，這世上最難降的妖魔是什么？我笑而不...
開封第一講書人閱讀 58,380評論 1贊 293
?港島之戀（遺憾婚禮）
正文為了忘掉前任收夸，我火速辦了婚禮坑匠，結(jié)果婚禮上，老公的妹妹穿的比我還像新娘卧惜。我一直安慰自己厘灼，他們只是感情好，可當(dāng)我...
茶點故事閱讀 67,432評論 6贊 392
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布咽瓷。她就那樣靜靜地躺著设凹，像睡著了一般。火紅的嫁衣襯著肌膚如雪茅姜。梳的紋絲不亂的頭發(fā)上闪朱，一...
開封第一講書人閱讀 51,301評論 1贊 301
城市分裂傳說
那天，我揣著相機(jī)與錄音，去河邊找鬼监透。笑死桶错，一個胖子當(dāng)著我的面吹牛，可吹牛的內(nèi)容都是我干的院刁。我是一名探鬼主播，決...
沈念sama閱讀 40,145評論 3贊 418
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼粪狼，長吁一口氣：“原來是場噩夢啊……” “哼！你這毒婦竟也來了再榄？” 一聲冷哼從身側(cè)響起，我...
開封第一講書人閱讀 39,008評論 0贊 276
萬榮殺人案實錄
序言：老撾萬榮一對情侶失蹤困鸥，失蹤者是張志新（化名）和其女友劉穎，沒想到半個月后疾就，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體，經(jīng)...
沈念sama閱讀 45,443評論 1贊 314
?護(hù)林員之死
正文獨居荒郊野嶺守林人離奇死亡猬腰，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點故事閱讀 37,649評論 3贊 334
?白月光啟示錄
正文我和宋清朗相戀三年，在試婚紗的時候發(fā)現(xiàn)自己被綠了姑荷。大學(xué)時的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片。...
茶點故事閱讀 39,795評論 1贊 347
活死人
序言：一個原本活蹦亂跳的男人離奇死亡鼠冕，死狀恐怖添寺，靈堂內(nèi)的尸體忽然破棺而出，到底是詐尸還是另有隱情供鸠，我是刑警寧澤畦贸，帶...
沈念sama閱讀 35,501評論 5贊 345
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布，位于F島的核電站楞捂，受9級特大地震影響薄坏，放射性物質(zhì)發(fā)生泄漏。R本人自食惡果不足惜寨闹，卻給世界環(huán)境...
茶點故事閱讀 41,119評論 3贊 328
男人毒藥：我在死后第九天來索命
文/蒙蒙一胶坠、第九天我趴在偏房一處隱蔽的房頂上張望。院中可真熱鬧繁堡，春花似錦沈善、人聲如沸乡数。這莊子的主人今日做“春日...
開封第一講書人閱讀 31,731評論 0贊 22
一樁弒父案闻牡，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽净赴。三九已至，卻和暖如春罩润，著一層夾襖步出監(jiān)牢的瞬間玖翅，已是汗流浹背。一陣腳步聲響...
開封第一講書人閱讀 32,865評論 1贊 269
情欲美人皮
我被黑心中介騙來泰國打工割以，沒想到剛下飛機(jī)就差點兒被人妖公主榨干…… 1. 我叫王不留金度，地道東北人。一個月前我還...
沈念sama閱讀 47,899評論 2贊 370
代替公主和親
正文我出身青樓严沥，卻偏偏與公主長得像猜极，于是被迫代替她去往敵國和親。傳聞我的和親對象是個殘疾皇子消玄，可洞房花燭夜當(dāng)晚...
茶點故事閱讀 44,724評論 2贊 354

簡單的Python爬蟲應(yīng)用_學(xué)習(xí)筆記003

從京東網(wǎng)站抓取_007

推薦閱讀更多精彩內(nèi)容