MySQL big data warehouse receiving (single table) _ MySQL

MySQL big data warehouse receiving (single table) bitsCN.com

MySQL big data warehouse receiving (single table)

Recently, I have been studying data warehouse receiving. for the INSERT command, a small amount of data can be satisfied, while for a large amount of data to be inserted into the same table, the speed is very slow and obviously does not meet the requirements. So I found it from the internet. MYSQL has a LOAD DATA INFILE command. Here is a brief description of the role of load data infile: it mainly records the files into tables in batches, the speed is very fast. However, operations can only be performed on one table. that is to say, all records are in one table, which has certain limitations.

MYSQL connection string:

                 m_pConnection->Open("DRIVER={MySQL ODBC 5.1 Driver}; SERVER=localhost; DATABASE=test;","root","123456",adModeUnknown);

{MySQL ODBC 5.1 Driver} is the ODBC Driver of MySql. it is downloaded on the official website and version 5.1 is

SERVER is the IP address of the mysql computer, if it is a local localhost.

DATABASE is the DATABASE to be connected.

In the Open method, "root" indicates the database username "123456" indicates the user password.

Test Procedure: 1. create a database test

2. execute the following database script in this database

                              CREATE TABLE T_TEST(                                                      F_ID BIGINT PRIMARY KEY,                                                      F_NAME VARCHAR(32),                                                      F_SEX SMALLINT,                                                      F_BIRTHDAY DATETIME,                                                      F_CONTENT VARCHAR(512));

3. make sure that the T_TEST table is empty during program execution.

4. the test result is that pieces of data can be stored in the database in about 1 s.

BitsCN.com