Experience in installing Scrapy from scratch, and experience in installing Scrapy from scratch

Source: Internet
Author: User

Experience in installing Scrapy from scratch, and experience in installing Scrapy from scratch

Scrapy is a python-based Web Crawler framework that crawls information from the network. It is a good way to obtain data. So I want to install it.

Go to its official website and go to the Installation Introduction page.

Https://docs.scrapy.org/en/latest/intro/install.html

There are three installation methods: one is from pip, the other is from source code, and the other is from conda

Based on previous knowledge, pip is already a package management tool integrated in python. It is simple and clear, and ignores a sentence on the official website introducing the interface.

Note that sometimes this may require solving compilation issues for some Scrapy dependencies depending on your operating system

Many errors are reported in the compilation phase.

Then I gave up. Compiling from the source code, like pip, is also a bunch of compilation errors.

No way, go to the conda and download a miniconda, more than 60 MB. The results of a careful study are no longer satisfactory.

Python may also notice that its package needs to be compiled after it is downloaded. If it is compiled, it depends on the environment configuration of its own OS, which often causes errors.

Miniconda is an integrated environment with python installed. It is equivalent to downloading and installing miniconda, that is, downloading the basic python core program, then, you can use the conda command to download the conda compiled package for Function Extension. That is, the scrapy package and its dependent lxml, twisted, and other compiled half-dead packages are already compiled. Then download and use it directly.

Https://conda.io/docs/install/quick.html

Https://conda.io/miniconda.html

 

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.