Python crawler (i)

It took four days to write a simple reptile program in Python. The entire process is divided into two parts: the installation of the tool and the implementation of the program

This article does not tell the details of the implementation of the problem encountered, but to start some pre-preparatory

The first part (installation of the tool)

download and installation of development tools MySql + mysqlworkbench +pycharm (can be tested)

Part II (Program implementation part)

Mac OS X Python installation mysqldb module

Before the entire program is started, the Python module that the crawler needs is mainly the MySQLdb module

Installing this module also took a bit of time, with installation steps such as the following:

1. Download the MySQLdb module at the following URL:

http://sourceforge.net/projects/mysql-python/    

2, in Mac OS X directly double-click Unzip , enter the installation folder in the terminal, run the python setup.py build

Assumption appears Sh:mysql_config:command not found such as error, can solve the method:

When you install the MAC version of MYSQL by default, the Mysql_config file is generated under the/usr/local/mysql/bin folder.

So the solution to download the installation is: Locate the Site.cfg file in the Mysql-python installation package, open it, and find the following:

# The path to Mysql_config.

# only use this if mysql_config are not on your PATH, or you have some weird

# setup that requires it.

# mysql_config =/usr/local/bin/mysql_config

Remove the last sentence first and change to:

Mysql_config =/usr/local/mysql/bin/mysql_config

　　

Then run:

$ python setup.py Install

reason:image not found error may occur, resolve reason:image not found errors such as the following:

The workaround is to run at the terminal:

$ sudo ln-s/usr/local/mysql/lib/libmysqlclient.18.dylib/usr/lib/libmysqlclient.18.dylib

$ sudo ln-s/usr/local/mysql/lib/usr/local/mysql/lib/mysql

sudo ln-s/usr/local/mysql/lib/libmysqlclient.18.dylib/usr/lib/libmysqlclient.18.dylib

Note When installing these things under Mac, you need to know how to use vim when it comes to configuration file changes

Mac OS X Python installation mysqldb module is a great hardship, finally in the terminal and PYCHARM environment can apply this module, and can link operation database.

Detailed implementation of the program to be continued Python crawler (ii)

Blog:

Http://www.cnblogs.com/ifantastic/archive/2013/04/13/3017677.html

http://www.codeif.com/post/1073/

Python crawler (i)