PHP batch processing large data long time processing

Author: Lin Mu

In order to prevent malicious reprint, put this article link http://blog.csdn.net/wolinxuebin/article/details/7722424 may have errors in this article, will be modified in time, please refer to the link above to view the latest updates.

Recently in doing a thing, written in PHP, because you want to read the name of the picture in the database, and then access the corresponding picture, read the picture size and resolution, to save the database. The amount of data is now processed in 5,000, and the default maximum PHP execution time is 30 seconds if you don't add any settings.

So in 30 seconds, I can only handle 700, just started manually, 700, 700 of the implementation of the. Later the feeling is not good, future data volume will be over 10W, this will be exhausting me.

Online search, there are three kinds of processing methods.

First Kind

Very straightforward, since you default 30 seconds, then I will set your default time to a long point, or directly without restrictions.

Add in PHP, PHP ini_set (' max_execution_time ', ' 100 '), function on it, 100 for 100 seconds, if you set 0, then there is no limit. PS: I used the combination of Zend, set to 0, is not without restrictions, but is about 120 of the way, my solution is to set this value of so big

More information on this approach is available in the Web site http://hi.baidu.com/shouji163/item/a604ddf3534aec2a84d2784a.

Second Kind

I like, is the use of PHP and batch processing of the combination.

@REM <?php
@REM = = '
@SET phpcli= ' D:\Program files\zend\zend studio-8.0.1\plugins\ Org.zend.php.debug.debugger.win32.x86_5.3.18.v20110322\resources\php53\php.exe "
@REM PHPCLI Set the path of your Php.exe
@%phpcli%%0
@goto: EOF
@REM ';? >
<?php
include ' C:\\Program files\\zend\\apache2\\htdocs\\pic_gather\\pic_operate.php ';
echo "\rfinish";
Ob_flush ();
EXEC ("pause");

From Forum: http://bbs.chinaunix.net/thread-527114-1-1.html

is a batch command, so it took me 1 hours to look at the next batch command. Translate the above program roughly.

@: Do not display the following instructions, if removed, instructions will be printed in the command window

REM is a comment for a batch, followed by a comment, not executed. Unlike @, @ just commands don't print and don't represent annotations.

The program starts to execute, first the bat runs

@SET phpcli= "D:\Program files\zend\zend studio-8.0.1\plugins\org.zend.php.debug.debugger.win32.x86_5.3.18. V20110322\resources\php53\php.exe "

Set the value of the PHPCLI

@%phpcli%%0

%phpcli% represents the path you set above, which means execute php.exe, and then% 0 represents the path to the. bat file, which means running the. bat file with Php.exe.

The above phenomenon will appear, PHP does not think REM is a comment ah, just think of the ordinary characters.

Will print the @rem first, and then

<?php
@REM = = '
@SET phpcli= ' D:\Program files\zend\zend studio-8.0.1\plugins\ Org.zend.php.debug.debugger.win32.x86_5.3.18.v20110322\resources\php53\php.exe "
@REM PHPCLI Set the path of your Php.exe
@%phpcli%%0
@goto: EOF
@REM ';? >

Here, Php.exe can know, to express the above meaning. In PHP @ can recognize No. This represents a suppression error, then is the content of the string "" with REM = =, then there is the @ inhibition error. Get through it safely.

into the next paragraph

<?php
include ' C:\\Program files\\zend\\apache2\\htdocs\\pic_gather\\pic_operate.php ';
echo "\rfinish";
Ob_flush ();
EXEC ("pause");

This is the program you want to execute. I don't want to write it in here, so just include it so it's easy to make. bat a template.

Your php.exe is gone, but it's not over yet, just a batch order.

@%phpcli%%0

Then in order not to let the batch command not know the language of PHP, use the

@goto: EOF

To end the program.

Some people ask why not just like this:

@SET phpcli= "D:\Program files\zend\zend studio-8.0.1\plugins\org.zend.php.debug.debugger.win32.x86_5.3.18. V20110322\resources\php53\php.exe "
@REM phpcli
@%phpcli%%0
@goto: EOF

<?php
    include ' c:\\ Program files\\zend\\apache2\\htdocs\\pic_gather\\pic_operate.php ';
echo "\rfinish";
Ob_flush ();
EXEC ("pause");

It's not so complicated, I tried it,

The cmd command line was found to appear:

@SET phpcli= "D:\Program files\zend\zend Studio-8.0.1\plugins\org.zend.php.debu
G.debugger.win32.x86_5.3.18.v20110322\resources\php53\php.exe "
@REM PHPCLI
@%phpcli%%0
@goto: EOF

The original author of the code wanted to avoid such a situation and print out too much junk information.

So this is the original author's incisive.

Third Kind

This if you, just update the string information inside, such as you set class, if class= ' natural scenery ', then put the image in front of Pic_gather/nature/image. So there is no need to use the above complex things, directly with the SQL language, the concat () function has such a function.

For example: UPDATE ' Game_3533_com_lingsheng ' SET ' Music ' =concat (' ring_gather/funny/', ' music ') WHERE ' class ' = ' funny ringtone '