Install python3.6.1 and scrapy and win7python3.6.1 in win7

Install python3.6.1 and scrapy and win7python3.6.1 in win7

--- Restore content start ---

The first blog record the process and problems of self-learning python.

First download python3.6.1 and required materials

Baidu: https://pan.baidu.com/s/1geOEp6z

Password: 1fuw

The file list is as follows:

Install python-3.6.1-amd64.exe first.

After the installation is complete, Enter cmd and enter python

The installation is successful.

Install the scrapy dependency plug-in below.

Run pip install xxxx to install the following files:

Install lxml-3.7.3-cp36-cp36m-win_amd64.whl

Install zope. interface-4.3.3-cp36-cp36m-win_amd64.whl
Install pywin32-221-cp36-cp36m-win_amd64.whl
Install Twisted-17.1.0-cp36-cp36m-win_amd64.whl
Install OpenSSL Through pip: pip install pyOpenSSL

 

To verify whether the scrapy dependency is successfully installed after installation:

Run the cmd command to go To the python console.

● Run import lxml. If no error is reported, lxml is successfully installed.
● Run import twisted. If no error is reported, the twisted installation is successful.
● Run import OpenSSL. If no error is reported, it indicates that OpenSSL is successfully installed.
● Run import zope. interface. If no error is reported, zope. interface is successfully installed.

Install scrapy

pip install scrapy==1.1.0rc3 

After the installation is complete, verify whether the installation is successful. Run the following command in cmd:

scrapy version

Return

Scrapy 1.1.0rc3
The installation is successful.
Then install Microsoft Visual Studio, which is installed in the 2015 Professional Edition. Select the programming environment for installation.
After successful creation, we can create a crawler project for testing.
The following uses crawling Baidu's title as an example:
Enter the console to output scrapy startproject baidu

Create a crawler File

Use pyCharm to enter the project and modify the Code as follows:

Items. py

import scrapyclass BaiduItem(scrapy.Item):    # define the fields for your item here like:    # name = scrapy.Field()    title = scrapy.Field()

First. py

import scrapyfrom baidu.items import BaiduItemclass FirstSpider(scrapy.Spider):    name = 'first'    allowed_domains = ['baidu.com']    start_urls = ['http://www.baidu.com/']    def parse(self, response):        item = BaiduItem()        item["title"] = response.xpath('/html/head/title/text()').extract()        print(item["title"])

Settings. py modify the following code and do not comply with the robots Protocol

ROBOTSTXT_OBEY = False

Cmd run the following command to return -- nolog, indicating no log is printed.

All environments have been installed successfully.

If an error is reported during execution

TypeError: 'float' object is not iterable

This is because the scrapy version is too low. We recommend that you manually download the high version to your local device for installation. The Network Disk information shared above contains the following files.

Go to cmd, enter the File Save directory, and execute pip install Scrapy-1.4.0-py2.py3-none-any.whl

If success is returned, OK is returned.

I am a beginner in python. I wrote my blog for the first time. If anything is inaccurate, you are welcome to discuss it together.