First, check and update the Python environment
The default MacBook is a python2.7 version, so we don't think it's new enough to go to the official web site to find the MAC installation package and then reinstall it.
The code is as follows |
Copy Code |
Official Download Website: https://www.python.org/download
|
Because of the compatibility support for some environments, I'm not going to upgrade to the python3.4 version or the 2.7 version.
Second, install the PIP Toolkit
This must be installed and is based on the Python Management toolkit.
The code is as follows |
Copy Code |
wget https://bootstrap.pypa.io/get-pip.py
|
If we are unable to execute the wget command with wget, we need to first check the 2 methods of the MacBook System environment installation wget-Traditional and homebrew installation to install wget.
The code is as follows |
Copy Code |
sudo python get-pip.py
|
Then execute the install command.
Third, modify the PIP toolkit source
Since the PIP package needs some source, we need to replace it with a domestic source so that it is faster to install.
The code is as follows |
Copy Code |
CD ~
mkdir. Pip
Touch pip.conf
|
Open the pip.conf with the editor and drop the following script
The code is as follows |
Copy Code |
[Global]
Index-url = Http://pypi.v2ex.com/simple
|
Here we use the V2ex address.
Iv. Installation of Pillow/pil modules
If you need to deal with the picture later, or also the Pillow/pil module installed.
The code is as follows |
Copy Code |
sudo pip install-u Pillow
|
V. Installation of Scrapy Crawler framework
The code is as follows |
Copy Code |
sudo pip install Scrapy
|
Crawler frame installation Direct input can be, remember to enter Sudo, execute half a day and finally a bunch of red error hint script.
To summarize, so far, we can carry on the crawler in the MacBook Drill, because in the local walkthrough is convenient, direct to the VPS operation file efficiency is not high, so whether it is Mac or Windows, are installed and debugging locally.