背景
最近想搞一個個人項目，需要用到爬蟲吮龄，所以接觸了scrapy。
環(huán)境參數(shù)
- OS環(huán)境
CentOS6.8 x86_64- OS默認python版本
python2.6
安裝問題
在默認python版本環(huán)境（python2.6）下安裝了pip母债，通過pip install scrapy 結果運行的時候出現(xiàn)一下錯誤:
提示 scrapy1.4.0 依賴python2.7版本
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# python
Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scrapy
Scrapy 1.4.0 requires Python 2.7
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]#
簡單尝抖，我們去官網(wǎng)看下，果然是的衙熔。
最新版（1.4.0）的scrapy需要python2.7及以上或python3.3以上的版本才支持搅荞。
解決方法
怎么解決？來痢甘，我們有兩種方法解決這個問題茉贡。
方法一: 安裝的時候選擇版本
大不了我們不安裝最新版本，安裝舊版本不就行了嗎腔丧。
驗了一下scrapy 1.0.0 及以上都依賴python2.7。
[root@iZwz9e75q2nzsxqdr0ll5yZ distribute-0.6.10]# pip install scrapy==0.9.0
You are using pip version 7.1.0, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting scrapy==0.9.0
Downloading http://mirrors.aliyun.com/pypi/packages/1a/70/fc8948afda8349cfd6dbd099da59248fdfde1ee658af63fc04643a928db3/Scrapy-0.9.tar.gz (771kB)
100% |████████████████████████████████| 774kB 2.2MB/s
Requirement already satisfied (use --upgrade to upgrade): Twisted>=2.5 in
...（省略）...
/usr/lib/python2.6/site-packages (from Automat>=0.3.0->Twisted>=2.5->scrapy==0.9.0)
Installing collected packages: scrapy
Running setup.py install for scrapy
Successfully installed scrapy-0.9
安裝成功悔据，但是還是要import確認一下
[root@iZwz9e75q2nzsxqdr0ll5yZ distribute-0.6.10]# python
Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scrapy
/usr/lib64/python2.6/site-packages/cryptography/__init__.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. The next version of cryptography will drop support for Python 2.6
DeprecationWarning
>>>
OK科汗，沒問題绷雏。
只是有個提示怖亭，python2.6 快成為棄兒了坤检，叫我們趕緊升級。
/usr/lib64/python2.6/site-packages/cryptography/init.py:26: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. The next version of cryptography will drop support for Python 2.6
DeprecationWarning
方法二: 升級python到2.7版本
我個人使用軟件包倾芝，一般情況下都會選擇最新版箭跳，我認為最新版若不是最好也是較好的，除非官方提示特殊情況谱姓。
來，那我們就看看升級python到2.7然后再安裝scrapy1.4.0（最新版）的方法路翻。
- 安裝python2.7
2.7.14 是python2 的最新版
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# https://www.python.org/ftp/python/2.7.14/Python-2.7.14.tgz
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# tar -zxvf Python-2.7.14.tgz && cd Python-2.7.14 && ./configure && make all && make install
...（省略）...
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# cd /usr/bin/; rm -f python && ln -s /usr/local/bin/python2.7 python
（如果不創(chuàng)建軟鏈接的話茄靠，運行python2.7的時候要用python2.7命令，腳本頭要寫 #!/usr/bin/python2.7）
[root@iZwz9e75q2nzsxqdr0ll5yZ bin]# ll python
lrwxrwxrwx 1 root root 24 11月 23 14:42 python -> /usr/local/bin/python2.7
注：1. 要把 /usr/bin/yum 中的 #!/usr/bin/python 改成 #!/usr/bin/python2.6
- 安裝 setuptools 最新版
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# wget -O setuptools-37.0.0.zip https://pypi.python.org/packages/7c/cb/bdfbb0b6a56459d5461768de824d4f40ec4c4c778f3a8fb0b84c25f03b68/setuptools-37.0.0.zip#md5=f905ca70d2db37b7284c0f6314ab6814
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# unzip setuptools-37.0.0.zip && cd setuptools-37.0.0
[root@iZwz9e75q2nzsxqdr0ll5yZ setuptools-37.0.0]# python setup.py install
(或者python2.7 setup.py install)
注：安裝成功之后账嚎，會看到以下目錄
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# ls -d /usr/local/lib/python2.7/site-packages/setuptools*
/usr/local/lib/python2.7/site-packages/setuptools-37.0.0-py2.7.egg /usr/local/lib/python2.7/site-packages/setuptools.pth
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]#
- 安裝pip最新版
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# wget -O pip-9.0.1.tar.gz https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# tar -zxvf pip-9.0.1.tar.gz && cd pip-9.0.1
[root@iZwz9e75q2nzsxqdr0ll5yZ pip-9.0.1]# python setup.py install
(或者python2.7 setup.py install)
注：安裝成功之后郭蕉，會看到以下目錄
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# ls -d /usr/local/lib/python2.7/site-packages/pip*
/usr/local/lib/python2.7/site-packages/pip-9.0.1-py2.7.egg
- 安裝scrapy
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# pip install scrapy
Collecting Twisted>=13.1.0 (from scrapy)
Could not find a version that satisfies the requirement Twisted>=13.1.0 (from scrapy) (from versions: )
No matching distribution found for Twisted>=13.1.0 (from scrapy)
wget https://twistedmatrix.com/Releases/Twisted/17.9/Twisted-17.9.0.tar.bz2
以上錯誤表示scrapy依賴Twisted 13.1.0及以上的版本喂江，但是pip 找不到這個版本（Twisted是用Python實現(xiàn)的基于事件驅動的網(wǎng)絡引擎框架），只能手動下載這個版本的安裝包來安裝了涨岁。
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# wget -O Twisted-17.9.0.tar.bz2 https://pypi.python.org/packages/a2/37/298f9547606c45d75aa9792369302cc63aa4bbcf7b5f607560180dd099d2/Twisted-17.9.0.tar.bz2#md5=6dbedb918f0c7288a4c670f59393ecf8
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# tar -jxvf Twisted-17.9.0.tar.bz2 && cd Twisted-17.9.0
[root@iZwz9e75q2nzsxqdr0ll5yZ Twisted-17.9.0]# python setup.py install
(或者python2.7 setup.py install)
然后再安裝 scrapy
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# /usr/local/bin/pip install scrapy
Collecting scrapy
Downloading http://mirrors.aliyun.com/pypi/packages/a8/96/3affe11cf53a5d2105536919113d5b453479038bb486f7387f4ce4a3b83f/Scrapy-1.4.0-py2.py3-none-any.whl (248kB)
100% |████████████████████████████████| 256kB 2.1MB/s
Requirement already satisfied: queuelib in /usr/local/lib/python2.7/site-packages (from scrapy)
...（省略）...
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.9->pyOpenSSL->scrapy)
Installing collected packages: scrapy
Successfully installed scrapy-1.4.0
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]#
驗證import吉嚣，很完美，沒有任何告警
[root@iZwz9e75q2nzsxqdr0ll5yZ ~]# python
Python 2.7.14 (default, Nov 23 2017, 13:28:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scrapy
>>>
