Performance analysis of BULK insert data into MySQL

MySQL BULK INSERT Database implementation statement performance analysis

Suppose our table structure is as follows

The code is as follows
CREATE TABLE Example ( example_id INT not NULL, Name VARCHAR (not NULL), Value VARCHAR (not NULL), Other_value VARCHAR (not NULL) )

Normally a single inserted SQL statement would be written like this:

The code is as follows
INSERT into example (example_id, name, value, Other_value) VALUES (+, ' Name 1 ', ' Value 1 ', ' other 1 ');

MySQL allows us to insert data in bulk in an SQL statement, as in the following SQL statement:

The code is as follows
INSERT into example (example_id, name, value, Other_value) VALUES (+, ' Name 1 ', ' Value 1 ', ' other 1 '), (101, ' Name 2 ', ' Value 2 ', ' other 2 '), (102, ' Name 3 ', ' Value 3 ', ' other 3 '), (103, ' Name 4 ', ' Value 4 ', ' other 4 ');

If we insert columns in the same order as the columns in the table, you can omit the definition of the column name, as in SQL

The code is as follows
INSERT into example VALUES (+, ' Name 1 ', ' Value 1 ', ' other 1 '), (101, ' Name 2 ', ' Value 2 ', ' other 2 '), (102, ' Name 3 ', ' Value 3 ', ' other 3 '), (103, ' Name 4 ', ' Value 4 ', ' other 4 ');

There seems to be nothing wrong with this, let me use the SQL statement optimization tips, the following will be tested separately, the goal is to insert an empty data table 200W data

The first method: use INSERT INTO to insert, the code is as follows:

The code is as follows
$params = Array (' value ' = ' 50 '); Set_time_limit (0); echo Date ("H:i:s"); for ($i =0; $i <2000000; $i + +) { $connect _mysql->insert ($params); }; echo Date ("H:i:s");

The last display is: 23:25:05-01:32:05 that took more than 2 hours!

The second method: Use transaction Commit, BULK Insert database (under 10W per submission) Last display time consumed is: 22:56:13 23:04:00, altogether 8 minutes 13 seconds, the code is as follows:

The code is as follows
echo Date ("H:i:s"); $connect _mysql->query (' BEGIN '); $params = Array (' value ' = ' 50 '); for ($i =0; $i <2000000; $i + +) { $connect _mysql->insert ($params); if ($i%100000==0) { $connect _mysql->query (' COMMIT '); $connect _mysql->query (' BEGIN '); } } $connect _mysql->query (' COMMIT '); echo Date ("H:i:s");

The third approach: using optimized SQL statements: Splicing SQL statements, using INSERT into table () values (), (), (), () and then inserting them once, if the string is too long,

You need to configure MySQL to run on the MySQL command line: Set global max_allowed_packet = 2*1024*1024*10; consumption time: 11:24:06 11:25:06;

Inserting 200W test data only took 1 minutes! The code is as follows:

The code is as follows
$sql = "INSERT into twenty_million (value) values"; for ($i =0; $i <2000000; $i + +) { $sql. = "(' 50 '),"; }; $sql = substr ($sql, 0,strlen ($sql)-1); $connect _mysql->query ($sql);

Finally, when inserting large quantities of data, the first method is undoubtedly the worst, while the second method is widely used in practical applications, and the third method is suitable for inserting test data or other low requirements, and the speed is really fast.

Performance analysis of BULK insert data into MySQL