Use shell paging to read 6 million + MySQL data scripts

shell-mysql
(1) Script background:
Because to be on Linux, remote read MySQL table data, and then do a certain cleaning, upload data to the Hadoop cluster, use Java to write it, feel too troublesome, have to develop on win, also have to play into jar package, upload to Linux, if there is a problem, Also have to repeat this, very inconvenient, then use the shell to write a bar, and do not need any JDBC driver package, only need to install a MySQL client on Linux, with a line of yum command can be done, so it took a little time, encapsulated a small script

(2) Function Introduction:
A small script that reads data from MySQL tables remotely using shell scripts directly under Linux, with the measured read 6 million + data efficiency similar to JDBC

(3) Script Description:
There are three main script components
1,page.sh This is a master script, which defines the conditions for paging, and you can see
2,f.sh a small package script, which will use SED to remove the table header some information
3,port.sh the execution script for paged read data

(4) How to use?  
Use is very simple, you need to change the query field inside the page.sh, and the number of paged query, the default is 10000, and then execute SH page.sh databaseName tablename incoming database name and table name can be

GitHub Address: Https://github.com/qindongliang/shell-mysql

Use shell paging to read 6 million + MySQL data scripts