一丶介紹
獲取某個api的數(shù)據(jù),定制化格式提取;
http://restapi.amap.com/v3/place/text?&keywords=&types=120201&city=hangzhou&output=json&offset=50&page=1&key=4254ccdb6f119ac71f046022bbe73bfc&extensions=all
比如這個鏈接,我想從page 1 - >10抓取數(shù)據(jù);
提取里面JSON的name,tel,address等數(shù)據(jù)進(jìn)行定制化抓取;
二丶
代碼
#!/usr/bin/python
#-*- coding: utf-8 -*-
#encoding=utf-8
import json
import requests
import os
import urllib2,urllib
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
def wjxRead(urlStr):
response = urllib2.urlopen(urlStr).read()
s = json.loads(response)
poisArray = s['pois']
content = ""
for index,value in enumerate(poisArray):
if isinstance(value["tel"],unicode):
tmpContent = "------>\nname : " + value["name"] + "\n" + "address : " + value["address"] + "\n" + "tel : " + value["tel"] + "\n"
content = content + tmpContent
else:
tmpContent = "------>\nname : " + value["name"] + "\n" + "address : " + value["address"] + "\n" + "tel : " + \n
content = content + tmpContent
return content
def saveFile(content):
file = open("wjx.txt","wb")
file.write(content)
file.close()
wjxContent = ""
for x in xrange(1,10):
urlStr = 'http://restapi.amap.com/v3/place/text?&keywords=&types=120201&city=hangzhou&output=json&offset=50&page=' + str(x) +'&key=4254ccdb6f119ac71f046022bbe73bfc&extensions=all'
wjxContent = wjxContent + wjxRead(urlStr)
saveFile(wjxContent)
三丶總結(jié)
沒有考慮并發(fā)
