The second method for batch writing PHP Data to MySQL

Previously, the sqlbulkcopy function was used in an out-of-the-box system to batch write data to SQL Server. The results are very good, which is much more efficient than the traditional insert into function. As a result, I had the idea of "Block write" when writing PHP to MySQL in batches. I don't know much about PhP, But I just worked with my Master Amin to solve the problem of slow batch Writing of big data.

When writing data to mysql, you can use the loader to generate the data as a TXT text file, and then import the data to MySQL for writing. For analysis, the entire operation involves two steps:

1. generate data into a text on the disk. (First write to disk)

2. Import the text to the target database. (The second write to the disk)CodeFragment:

  $ SQL  = "   load data infile'   $ filepath   'into table  "  .   $ table  .   "  Character Set GBK fields terminated by ': | 'Lines terminated by', '('id', 'name', 'age ');   " ; 
   $ db  ->   excnonquery (  $ SQL  );  

As you can see, there are two disk operations in total. We know that at present, the biggest bottleneck of a computer is a disk, so here we have performed two operations on the "bottleneck", which must result in a slow, endless slow-wait. The document has been found on msdn. We can see that. Net has a memorystream function, which is a stream operation that writes information to the memory. This method reminds me that if the generated operation can be executed in the memory, the overall behavior will be much faster.

However, the problem is that the data is generated in the memory, but there is no way to know the written memory address, and data cannot be read... So this method appeared. I called it the second method for batch writing PHP Data to MySQL. Why is it called? Because I don't know much about PhP, maybe there is a better way. If you don't want to close the interface, you can simply confess: Use ramdisk (virtual hard disk software) to virtualize a memory disk and place the first operation in the memory disk for execution. Let's take a look:

Let's take a look at this "disk performance ":

Using the red line box, one is the average reading speed, and the other is the sudden reading speed, which is amazing. Any hard disk cannot reach the transmission rate. That is, this disk solves the problem of generating 0.1 billion pieces of data and then writing it to the database for a short time. Let's call it the second method ~