Mysql efficient data import Method

The mysql efficient data import method simulates tests on the current network. A test environment needs to be set up to import hundreds of millions of data to the database. Make a simple record of the problem. Data: a database, 2000 tables, 2000 table data, 30 columns per record, 86388670 records per table on average, a total of records. MACHINE: linux 64 bit 8g memory 4 Core IntelX3320@2.5GHz basic ideas:

Mysql-utest-pxxxx-s-e "source xxxx. SQL"

 

Put the load statement in xxxx. SQL. load Syntax:

Load data [low_priority] [local] infile 'file_name.txt '[replace | ignore]

 

If the data is imported in a serial mode, it will be slow. Consider parallel mode, but there may be conflicts. Solution: Clear the table before loading.

Truncate table xxxx or delete from table xxxx;

 

Or use the replace or ignore keyword.