環(huán)境:Mac OS X Yosemite 10.10.3
安裝Scrapy
學(xué)習(xí)Python爬蟲必須要使用的框架Scrapy厅翔,話不多說。
打開終端執(zhí)行命令:
sudo easy_install pip
pip 和 easy_install 都是 Python 的框架管理命令搀突,pip 是對 easy_install的升級刀闷。
然后終端執(zhí)行命令安裝 Scrapy:
sudo pip install Scrapy
如果執(zhí)行成功,那么 Scrapy 就安裝成功了仰迁,但往往事與愿違甸昏,你很有可能遇到如下錯誤:
/private/tmp/pip-build-9RYtLC/lxml/src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
#include "libxml/xmlversion.h"
^
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-9RYtLC/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-544HZx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-9RYtLC/lxml
解決方法有如下幾種:
1、終端執(zhí)行命令安裝或更新命令行開發(fā)工具:
xcode-select --install
2徐许、配置路徑:C_INCLUDE_PATH
C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2/libxml:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include
3施蜜、參照官網(wǎng)使用如下命令安裝Scrapy
STATIC_DEPS=true pip install lxml
一般此三個方法就可解決錯誤成功安裝Scrapy,如果還是失敗雌隅,參考 StackOverflow上的一個帖子
安裝PIL
PIL是Python的圖形處理庫翻默,在學(xué)習(xí)爬蟲的時候可以用來處理驗(yàn)證碼缸沃。
終端輸入命令:
sudo pip install pil
恩,出錯:
/Library/Python/2.7/site-packages/pip-6.1.1-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:79: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Could not find a version that satisfies the requirement pil (from versions: )
Some externally hosted files were ignored as access to them may be unreliable (use --allow-external pil to allow).
No matching distribution found for pil
不過提示了添加參數(shù) --allow-external pil
好修械,改一下命令重新執(zhí)行:
sudo pip install PIL --allow-external PIL
好的趾牧,開始安裝了,哎肯污?好像又出錯了G痰ァ!蹦渣!
_imagingft.c:73:10: fatal error: 'freetype/fterrors.h' file not found
#include <freetype/fterrors.h>
^
1 error generated.
error: Setup script exited with error: command 'cc' failed with exit status 1
提示沒找到 freetype/fterrors.h
文件哄芜,百度怎么解決,很多文章的解決辦法是執(zhí)行命令:ln -s /usr/local/include/freetype2 /usr/local/include/freetype
然后柬唯,試了认臊,不行。
從Finder來到目錄 usr/local/include
下权逗,咦美尸?好像有目錄freetype2,但是么有freetype斟薇,那么...可以復(fù)制一個freetype2的副本再改名freetype不行嗎师坎?恩,然后我就這樣干了堪滨。然后在終端重新執(zhí)行安裝PIL的命令:
sudo pip install PIL --allow-external PIL
然后就安裝成功了~~
安裝BeautifulSoup
首先胯陋,官網(wǎng)下載最新的包beautifulsoup4 4.3.2
,然后解壓縮袱箱,從終端進(jìn)入該目錄遏乔。
終端執(zhí)行
sudo python setup.py install
好,安裝成功发笔。
Beautifulsoup的官方文檔
補(bǔ)充:
easy_install使用方法:
安裝:easy_install PackageName
刪除:easy_install -m PackageName
更新:easy_install -U PackageName
pip使用方法:
安裝:pip install PackageName
刪除:pip uninstall PackageName
更新:pip install -U PackageName
搜索:pip search PackageName