Flood Log Inbound

Mass log Storage
The log directory has 10 log files, each file is compressed about 60M or so, the file suffix is. gz, such as a.gz, b.gz, etc., the contents of the file is id=2112112,email=xxx@163.com, etc.
id=2112112,email=xxx@163.com, wait for the other,
id=2112112,email=xxx@163.com, wait for the other,
id=2112112,email=xxx@163.com, wait for the other,
id=2112112,email=xxx@163.com, wait for the other,
id=2112112,email=xxx@163.com, wait for the other,
id=2112112,email=xxx@163.com, wait for the other,

Now you want to insert all the contents of each file in this directory into the database, the database table, is through the e-mail table, about Log_1,log_2, until the log_1000 of the sub-table, ask the next detailed solution, For example, how to ensure that each file in a very fast time to put into storage, making script execution more efficient
Put a piece of code first

 Error_reporting (E_all & ~e_notice);
Receive parameters
$mysql _host = XX. Xx. Xx. XX;
$mysql _user = XXX;
$mysql _pass = XX;
$mysql _port = 3306;
$mysql _db = ' Test ';
$table _pre = ' log_ ';
$gz _log_file = a.gz;
Script Execution Log
$exec _log = '/data_log/record.txt ';
File_put_contents ($exec _log, ' *****************************************start*********************************** '." \ r \ n ", file_append);
File_put_contents ($exec _log, ' param is mysql_host= '. $mysql _host. ' mysql_user= '. $mysql _user. ' mysql_pass= '. $mysql _ Pass. ' mysql_port= '. $mysql _port. ' mysql_db= '. $mysql _db. ' table_pre= '. $table _pre. ' gz_log_file= '. $gz _log_file. ' Start_time= '. Date ("Y-m-d h:i:s"). " \ r \ n ", file_append);
Read Log Inbound
$z _handle = Gzopen ($gz _log_file, ' R ');
$time _start = Microtime_float ();
$mysql _value_ary = Array ();
Link Database
$conn = mysql_connect ("$mysql _host: $mysql _port", $mysql _user, $mysql _pass);
if (! $conn) {
File_put_contents ($exec _log, ' Could not connect database error, error= '. Mysql_error (). " \ r \ n ", file_append);
Exit
}
$selec _db = mysql_select_db ($mysql _db);
if (! $selec _db) {
File_put_contents ($exec _log, ' Select Database error, database= '. $mysql _db. " \ r \ n ", file_append);
Exit
}
while (!gzeof ($z _handle)) {
$each _gz_line = gzgets ($z _handle, 4096);
$line _to_array = explode ("\ t", $each _gz_line);
Filtering Invalid Logs
if (!empty ($line _to_array[3]) &&!empty ($line _to_array[2]) &&!empty ($line _to_array[4])) {
$insert _value = "('". $line _to_array[3]. "', '". $line _to_array[2]. "', '". $line _to_array[1]. "', '". $line _to_array[4]. "' , ' ". $line _to_array[0]." ') ";
$insert _sql = "INSERT INTO $table _name (uid,email,ip,ctime) values $insert _value";
$table _id = ABS (CRC32 ($line _to_array[2])% 1000);
$table _name = $table _pre. $table _id;
$result = mysql_query ($insert _sql);
if (! $result) {
If an error is inserted, the log is logged
File_put_contents ($exec _log, ' table_name= '. $table _name. ' email= '. $line _to_array[2]. " \ r \ n ", file_append);
}
}
}
$time _end = Microtime_float ();
$diff = $time _end-$time _start;
File_put_contents ($exec _log, ' success to inserts Database,log_file is '. $gz _log_file. ' Time-consuming is= '. $diff. " s \ r \ n ", file_append);
File_put_contents ($exec _log, ' *******************************************end*********************************** '." \ r \ n ", file_append);
Gzclose ($z _handle);

The above code executes, very slowly, unbearable, please Daniel Help massive log? Log analysis? scripting? Efficiency

------Solution--------------------
The table type is modified to: InnoDB, then implemented with a transaction,
If not, change the load file.

