Python installation and scrapy installation under Mac

Mac comes with tools such as Python and Pip, but when using install scrapy, there are some errors, because there are some core directories (such as/library) that do not have operational permissions on the operating system, Mac has some of its own permissions control program (non-sudo chmod can change), So simply reinstall Python so that the newly installed Python will be in the/usr directory, not the system's own/library. /usr below we have any permissions to operate.

1, I use homebrew method to install, of course, can also directly login to the Python official website download installation, personal installation preferences.

2. Execute brew Install Python

After the installation is complete, the file is/usr/local/opt/python/libexec/bin

After installation here, Pip is also automatically installed, execute PIP2 on it, if the PIP is executed, the Mac comes with Python inside the PIP.

For convenience, set up aliases:

Alias python2= '/usr/local/opt/python/libexec/bin/python '

3, the installation of picture processing module, this step is I see someone else's post installed, I also installed. actually does not install also does not matter.

PIP2 Install-u Pillow, at this time, installation always error, readtime out. This is because the source of the PIP is foreign, so the network often has problems. At this time, with the domestic image can be, here is the source of Tsinghua University, is said to be 5 minutes synchronization, so the basic and foreign Python source.

CD ~; mkdir. Pip; Touch pip.conf; Vim pip.conf;

Enter as follows, save exit: Wq

[Global] Index-url=https://pypi.tuna.tsinghua.edu.cn/simple [Install] trusted-host=pypi.tuna.tsinghua.edu.cn

Execute the command again PIP2 install-u Pillow is ready. Very fast.

4, Installation Scrapy

PIP2 Install Scrapy

After you change the source, the installation is fast.

From then on, you can use the Scrapy framework to do the crawler.

If you need to do login interactions, it is best to use mechanize and selenium to simulate the behavior of your browser. If you've ever played the Webrowser controls inside Microsoft, you know they're very useful. I used Webrowser to achieve bulk login, fill out forms and get relevant information, very useful. Now it seems that the original is a reptile, at that time completely did not know. No matter how powerful the technology is, there is a simple and effective idea, and I always think so.

This article is from "A cup of Happy tea" blog, please be sure to keep this source http://ggwhsd.blog.51cto.com/13135051/1955993

Python installation and scrapy installation under Mac