Using Python to write a browser-like downloader, this is a very simple

Before learning python, want to download an online file, you can only call the browser, find the file download link, put the address in.

Now after learning Python, only to find that the original download a thing is very simple.

Give me an example. Let's look at this picture and its URL.

Here is the next small series of Python learning Group: 483546416, whether you are small white or Daniel, small series I welcome, do not regularly share dry goods, including small series of their own collation of a 2017 of the latest Python materials and 0 basic introductory tutorials, welcome beginners and advanced in the small partners. In the not busy time I will give everybody to dispel doubts.

Now I want to download this picture, in fact, is a very simple thing, just a few lines of code can be

Once I showed this code to the students to see, he said how so troublesome, directly right key to save the line. If I asked you to download 11 million images, would you be able to save them manually? One of the purposes of our programming is to let computers do trivial things for us.

Anyway, how do you write a downloader? Like the previous process, get the URL first, then use the requests module to download, and then save the file. So here's the problem, if we download the file is too large, such as I used to download the file on the Baidu network before, the effect is very good, a thread 100kb/s, open 20 threads, you can reach the 2m/s, very useful, but encountered a problem is that the file is too large, if the data, Now download and write to the file, the computer is not friendly, because this is the downloaded data will be stored in memory, soon the memory is not enough.

After looking for a long time did not find a good solution, how can you download the same side of the download side to write the file. And finally found a solution.

Just use the Iter_content method to get the data a little bit iteratively.

To see the complete code

DMS Download Access source