How Python virtual machine installation is configured scrapy for MAC

Date: February 21, 2018

Because of the time problem, the following notes are roughly recorded. Only as a personal note

Installing Virtualenv and Virtualenvwrapper

How to install the details below this article also has an introduction, including how to use the switch virtual machine is also introduced

http://blog.csdn.net/xshalk/article/details/47301339

Where to find the. bash_profile file

Open the directory with the shortcut key command+shift+g input in the Finder ~/
Again shortcut key: shift+comman+. Show hidden files to find this file
After you have modified the save, and then run the source ~/.bash_profile on the terminal command line, the file is executed

Later in the installation process encountered some problems, and then through Baidu, the following this can solve the problem of Python path

Https://www.jianshu.com/p/2999e258cbf9

In addition, create a virtual environment that does not require any packages mkvirtualenv--no-site-packages venv

Next step is: Workon virtual environment, PIP3 install a series of packages,

The CD then enters the directory where you want to create the project,

Then CD into the project root directory to create a crawler py file,

Note that there is a small error here: scrapy genspider name URL, where the URL does not require "http://".

Then use Pycharm to open the project, and remember to re-select the virtual environment configuration, directly select the front Workon selected virtual environment.

Then a debugging tip, create a new main.py file in the scrapy.cfg sibling directory

Then the code looks like this:

1 #with this file to debug, you can improve the development efficiency2 3  fromScrapy.cmdlineImportExecute4 5 ImportSYS6 ImportOS7 8 " "9 set up a catalog of projectsTen Os.path.abspath (__file__) Gets the directory of the current file One  A Os.path.dirname (full path to file) Gets the parent directory of the current file -  - Print Test the Print (Os.path.abspath (__file__)) - Print (Os.path.dirname (Os.path.abspath (__file__))) -  - " " +  - #Print (Os.path.abspath (__file__)) + #Print (Os.path.dirname (Os.path.abspath (__file__))) ASys.path.append (Os.path.dirname (Os.path.abspath (__file__))) at  -  - #path = Os.path.dirname (Os.path.abspath (__file__)) + "/articlespider" - #sys.path.append (path) - #print (Sys.path) -  in " "how to execute a command--query the stock name code list" " - #Execute ("Scrapy crawl quote_eastmoney_com". Split ()) to  + " "how to execute a command--query the stock name code list" " -Execute"scrapy Crawl DATA_EASTMONEY_COM_HSGTCG". Split ())

How virtual machine installation for Python is configured scrapy for MAC