PHP Program execution timeout when batch image processing

This batch processing is because the project itself after several outsourcing, the picture is stored everywhere, and there is no specific thumbnail and uniform size of the picture, this time in the original text mixed data extracted from the image, stored in the specified directory, crop size and generate the corresponding thumbnail, and the results are stored in the database.

In the process of batch processing, it is unavoidable to encounter the program execution time is too long, resulting in 504 errors, such as adjusting the server configuration is not a very good solution, after all, the production environment dynamic knife ...

But I'm lazy, and I don't want to do the paging request processing for a single extracted image ^_^

By the way, up to one processing of the extracted images of the data reached 3000 sheets ~

Below to get to the point, our main goal is to extend the program execution time as much as possible so that the captured pictures are caught and processed to finish writing to the database.

Our environment is based on Nginx PHP5.6, first of all the code to pay attention to, one is put on Set_time_limit (0), which avoids adjusting php.ini, of course, some basic such as the maximum memory and so on or to tune, here do not introduce; Another must be set Curl Curlopt_timeout (I use the Curl grab remote picture) Considering that the picture is in a known number of servers, time-out can not catch must be the problem of simply give up, if not set this timeout, the runtime will continue to give you an error.

Next is the server, the first is the Nginx program execution time:

Fastcgi_connect_timeout 900s;

Fastcgi_send_timeout 900s;

Fastcgi_read_timeout 900s;

This configuration in nginx.conf, there is a change in value, no append on, the time according to their own situation settings, adjusted after the restart service.

MySQL will also report a timeout error, modify My.ini Add or modify the configuration:

wait_timeout=2880000

interactive_timeout=2880000

max_allowed_packet=100m

After adjusting to restart MySQL in order to take effect, after the above settings, basically clear the situation of the times wrong, my program ran 2 days 3 nights, collect and generate a total of 86G pictures.

Again, this operation is not very recommended, especially in the production environment, will bring many problems, the most important, after completing the bulk task, restore the above settings ~