Scrapy Introduction and Environment installation Scrapy Introduction and Environment installation what is Scrapy?
An open source and collaborative framework for extracting the data of need from websites. In a fast, simple, yet extensible way.--scrapy Home Page
- Scrapy is a fast web crawler crawl framework developed by Python for crawling web sites and extracting structured data from pages. Scrapy can be used for data mining, monitoring and automated testing in a wide range of applications.
Official website: scrapy.org
How to install Scrapy?
- Installing lxml
- Installing Zope.interface
- Installing twisted
- Installing PYOPENSSL
- Installing Pywin32
- Install Scrapy (
pip install scrapy
)
Scrapy itself is based on some underlying library files, so you need to install some of the underlying library files. In other words, in addition to the last step, if the other steps are installed, it is not necessary to install again.
About the installation of the Scrapy framework