Install scrapy1.3.2 in win7 and win7scrapy1.3.2

Install scrapy1.3.2 in win7 and win7scrapy1.3.2

At the beginning, I learned about crawlers and searched the internet. Currently, scrapy is the most suitable choice.

Install scrapy first.

As mentioned in many blogs, scrapy currently does not support python3 very well. However, you cannot leave it alone.

Use the latest version of anaconda 4.3.2 First.

 

Import scrapy error!

Directly install scrapy on pip,

The installation process is okay, or an error is reported. After several iterations, I finally stuck in the lack of c ++ tools.

Step 1 upgrade. netframework to 4.6

Step 2 download and install visualcppbuildtools_full.exe. The file is not big, only 3 m, and the installation process needs to be networked. It took me an hour to install it.

Step 3: pip install scrapy. Pass

Step 4 import scrapy

Step 5: Practice crawlers. it's okay for a small case. An error will be reported for a large and complete case. What's 'float' not a iterator... get bored?

The support is still insufficient.

 

Check scrapy version

Go to the root directory and run the command line to check scrapy version.

The version is 1.1.1.

. Go to the scrapy website to find the latest version. Found to be 1.3.2. The display can support python3 (a lot worse .)

Go to pip

Https://pypi.python.org/pypi/pip

Search for scrapy. There is 1.3.2 above. (I don't understand why pip install scrapy didn't automatically install this version)

 

Download down Scrapy-1.3.2-py2.py3-none-any.whl

Install pip directly

Pip install Scrapy-1.3.2-py2.py3-none-any.whl

The process shows that 1.1.1 is uninstalled and 1.3.2 is successfully installed.

Command Line Test

Scrapy fetch http://www.tjcac.gov.cn/xxgk/tzgg/201701/t20170120_46683.html (find a web page)

This is the last part. It is displayed as successful.

 

Finally, scrapy is now able to better support python3.