Php batch downloads remote files to the server. how can this problem be solved?
Source: Internet
Author: User
Php batch downloads remote files to the server. I put the address of the remote files in the database. now I retrieve the remote files from the database and then download them to the server. Is there any efficient method? Currently, there are 10 million data records ., The simplest method is the principles of php's pseudo-multithreading. 1. write a download script, use file_get_contents2, and write a php batch download call to download remote files to the server.
I put the address of the remote file in the database. now I can retrieve it from the database and download it to the server. Is there any efficient way? Currently, there are 10 million data records.
------ Solution --------------------
The simplest method
It is actually the principle of php pseudo-multithreading.
1. write a download script using file_get_contents.
2. write a script to call the download. In this example, multiple download steps are called. you can use file_get_contents to submit a url to the download script.
It should be noted that the number of threads is a matter of saving the number of threads to the database, which is better controlled.
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.