Php batch downloads remote files to the server. I put the address of the remote files in the database. now I retrieve the remote files from the database and then download them to the server. Is there any efficient method? Currently, there are 10 million data records. ------ Solution ------------------ The simplest method is the principle of pseudo-multithreading in php. 1. write a download script, use file_get_contents2, and write a script to call the download, multiple php downloads are called to batch download remote files to the server.
I put the address of the remote file in the database. now I can retrieve it from the database and download it to the server. Is there any efficient way? Currently, there are 10 million data records.
------ Solution --------------------
The simplest method
It is actually the principle of php pseudo-multithreading.
1. write a download script using file_get_contents.
2. write a script to call the download. In this example, multiple download steps are called. you can use file_get_contents to submit a url to the download script.
It should be noted that the number of threads is a matter of saving the number of threads to the database, which is better controlled.
