Golang Crawler Implementation (i)

This is a creation in Article, where the information may have evolved or changed.

First, the Environment preparation

1, compiler, IntelliJ idea.

Download Address:https://d1opms6zj7jotq.cloudfront.net/idea/ideaIC-2016.1.2b.exe

After downloading, install directly. The editor after installation does not support Golang. You need to add plugins that support Golang, there are two ways to add plug-ins.

The first type:

(1) Click File->setttings->plugins->browse Repositories;

(2) Search go, find, click Install, restart the compiler;

The second type:

(1) Download the source code of the Golang plugin. Download Address: Https://codeload.github.com/go-lang-plugin-org/go-lang-idea-plugin/zip/master

(2) Compile the source code, add the compiled tools to the compiler. Specific method:http://blog.csdn.net/slug302/article/details/16991323

2, Golang compilation environment.

Download Address: Http://pc2-dx1.newasp.net/soft/yh/go.windows-amd64.zip

There are two ways to install a compilation environment.

The first type:

Direct Officer Network download EXE file, one-click installation.

The second type:

This type of installation is source code installation.

After downloading the source code, you need to set environment variables. The file path after extracting the download is: D:\go.

Add Goroot with a value of D:\go. You also need to add%goroot%\bin to the value of path.

To detect if the installation was successful:

1. Open cmd and enter echo%PATH% to see if the environment variable has been added. There is no need to restart the system.

2. Enter Go version. See if there is any information output.

Either way, you need to add a gopath.

Gopath is an important environment variable that the GO command relies on. It is linked to the working directory. Frankly, it's your work space. When there are multiple workspaces, add them directly, separated by semicolons. The Gopath working directory has three subdirectories:

-SRC store source code (e.g.,. Go, etc.)

-PKG store the compiled files (for example:. a)

-bin hosting an executable environment built after compilation

When you create a new application or add a third-party library, a code package, it is stored under SRC.

3, MySQL.

Download Address: Http://cdn.mysql.com/Downloads/MySQL-5.7/mysql-5.7.12-winx64.msi

There are two different ways to install MySQL.

The first type: direct download. Mis file, one-click installation.

The second type:. zip package installation.

(1) After downloading the source code, after decompression, you need to change the environment variables. Add to Path: D:\mysql5.7\bin (the path of the extracted file);

(2) Modify the configuration file My-default.ini.

#basedir = = = Basedir =d:\mysql5.7
#datadir = = = DataDir =d:\mysql5.7\data

(3) Open cmd, input: mysqld-install;

(4) net start MySQL;

(5) Mysql-u root-p. After the loss will be prompted to enter a password, password is empty, directly press ENTER;

(6) Use MySQL = Update user set Password=password (' 123456 ') where user = ' root '; (change root password)

(7) Restart MySQL. net stop mysql = net start mysql;

4. Install MySQL Driver

Golang operation MySQL, requires MySQL driver. MySQL Drive source download: Https://github.com/go-sql-driver/mysql

(1) After downloading the source code, copy the source code to Gopath's working directory src;

(2) Open the CMD,CD to the place driving source;

(3) Go install; Open the Gopath directory pkg, you can see the generated. A file, which is the MySQL driver file.

(4) Write code to test whether the database can be connected. The code is as follows:

Package Mainimport ("Database/sql" "Fmt" _ "reptiles/src/mysql-driver"//Generate driver file directory name) type User struct {ID                 Intusername, password String}var sqldata map[interface{}]interface{}func main () {var u userdb, err: = SQL. Open ("MySQL", "root:123456@tcp (127.0.0.1:3306)/splider?charset=utf8") Check (ERR)//Insert Data stmt, err: = db. Prepare ("INSERT t_test SET s_username=?,s_pwd=?") Check (ERR) res, err: = stmt. Exec ("Xiaowei", "Xiaowei") check (ERR) ID, err: = Res. Lastinsertid () Check (err) fmt. PRINTLN (ID)//query data rows, err: = db. Query ("SELECT * from T_test") Check (err) fmt. Println (rows. Columns ()) UserInfo: = Make (map[interface{}]interface{}) for rows. Next () {err: = rows. Scan (&u.id, &u.username, &u.password) Check (err) userinfo[u.id] = u}fmt. Println (userinfo)}func check (err error) {if err! = Nil {fmt. PRINTLN (ERR)}}