Wget download command

Wget is a command line tool, the fastest single-thread transmission tool in history. It is used to download websites/batch files and supports HTTP and FTP. Its task is to obtain the internet.

First, basically all Linux for personal computers have wget.

If we want to download all the files in a directory in FTP,

wget -r ftp://10.8.8.8/movie/

Wait! When I found something wrong, how did I get out a directory named 10.8.8.8 and check it out? It's a movie. Oh, wget recorded both the directory structure and the website title, don't you ?? No problem! For example

wget -r -nd ftp://10.8.8.8/movie/

No directories are returned, faint! Why? If you want to do so, let it do so. Otherwise, use

wget -r -nH ftp://10.8.8.8/movie/

Well? And movie? OK.

wget -r -nH --cut-dirs=1 ftp://10.8.8.8/movie/

Disconnected? Timeout? Add-C to continue

wget -c -r -nH --cut-dirs=1 ftp://10.8.8.8/movie/

We will download all the macro packages here.

wget -r -k http://www.ctan.org/tex-archive/macros/latex/

-K indicates converting the connection to a local connection. But now there are the same problems, so add what you need. In addition, you may not need to go down so many layers. For example, we will go to the second layer.

wget -r -l2 -k http://www.ctan.org/tex-archive/macros/latex/

The new problem is that the webpage has a sorting function, which is annoying because it has been repeated many times during download, you can use the-A and-R switches to control the download type and use wildcards.

wget -r -R ".htm\?*" -k http://www.ctan.org/tex-archive/macros/latex/

This time there is no such web page, right? -R indicates that a file of the matching type is rejected.-A indicates only the accepted file type, such as-"*. GIF will only download GIF images. If multiple GIF images are allowed or not, they can be used separately.

So what should we do if we are using a proxy server? Haha, I'm glad you chose wget. You can use its configuration file and environment variables to use proxies. We recommend that you use environment variables. For example, in bash, we can add the daily proxy to. bash_profile. This is a standard Linux Statement (many software programs use apt-Get, yum, and so on)

export http_proxy=http://10.20.30.40:8080

Then, proxy is enabled by default. If you need to temporarily disable it, you can use

wget --proxy=off -r -k http://www.ctan.org/tex-archive/macros/latex/

Of course, you can also write a. wgetrc file, which can be found in/usr/local/etc. There are detailed comments in it, so I won't say much about it.

When downloading a Web page, it is troublesome that some web pages are directed multiple times at the same time. To avoid multiple downloads, we use

wget -nc -r -k http://www.ctan.org/tex-archive/macros/latex/

This can be avoided. To avoid being directed to a non-http://www.ctan.org/tex-archive/macros/latex/ inner directory by some connections, we should also add

wget -nc -np -r -k http://www.ctan.org/tex-archive/macros/latex/

Avoid downloading files that are not in this directory, and avoid accessing different hosts. Of course, if you want this function to be available, you can use

wget -nc -np -H -r -k http://www.ctan.org/tex-archive/macros/latex/

So that we can span between multiple hosts, while-I and-x allow us to track only some directories or not some directories. If something you need in some HTML is not made by this type of thing, you have to use -- follow-tags and -- ignore-tags.

Hey, I have a file that is connected. What should I do? You only need

wget -i your.file

If yes, It is not complex.

wget -F -i your.file

Below is a supplement:

Wget-M ftp: // username: password @ IPaddress /*

-M is an option for site images. If you want to create a site image, use this option to automatically set other appropriate options for site images; it is equivalent to-r-N-l INF-Nr.

1 category: Download