Linux Wget Commands

Linux wget is a tool for downloading files, which is used at the command line. This is an essential tool for Linux users, especially for network administrators, who often download software or restore backups from remote servers to a local server. If we use a virtual host, we can only download from the remote server to our computer disk and then upload it to the server using the FTP tool. This is a waste of time and energy, there is no way to do. To a Linux VPS, it can be downloaded directly to the server without having to upload this step. The Wget tool is small but fully functional, it supports breakpoint download function, supports FTP and HTTP download, supports proxy server and is easy to set up. Below we explain how to use wget as an example.

1. Use wget to download individual files

The following example is to download a file from the network and save it in the current directory

1. wget Http://cn.wordpress.org/wordpress-3.1-zh_CN.zip

A progress bar is displayed during the download, including (Percent download complete, bytes already downloaded, current download speed, remaining download time).

2. Download with Wget-o and save it with a different file name

wget defaults to the last character that matches the "/" and the file name for dynamically linked downloads is usually incorrect.
Error: The following example downloads a file and saves it by name download.php?id=1080

1. wget https://www.centos.bz/download?id=1

Even if the downloaded file is in the zip format, it still takes the download.php?id=1080 command.
Correct: To solve this problem, we can use the parameter-o to specify a file name:

1. Wget-o Wordpress.zip https://www.centos.bz/download.php?id=1080

3, use wget–limit-rate speed limit download

When you execute wget, it will use all possible broadband downloads by default. But when you're ready to download a large file and you still need to download other files, it's necessary to limit the speed.

1. wget--limit-rate=300k Http://cn.wordpress.org/wordpress-3.1-zh_CN.zip

4. Use wget-c breakpoint to continue transmission

To restart the download of the interrupted file using wget-c:

1. Wget-c Http://cn.wordpress.org/wordpress-3.1-zh_CN.zip

It is very helpful for us to download a large file because of the interruption of network and other reasons, we can continue to download the file instead of downloading it again. You can use the-c parameter when you need to continue the interrupted download.

5, use wget-b background download

For downloading very large files, we can use the parameter-B to download the background.

1. Wget-b Http://cn.wordpress.org/wordpress-3.1-zh_CN.zip
2. Continuing in background, PID 1840.
3. Output'll is written to ' Wget-log '.

You can use the following command to view the download progress

1. Tail-f Wget-log

6. Disguise Agent name Download

Some websites may reject your download request by judging the proxy name as not a browser. But you can disguise it by –user-agent parameters.

1. wget--user-agent= "mozilla/5.0 (Windows; U Windows NT 6.1; En-US) applewebkit/534.16 (khtml, like Gecko) chrome/10.0.648.204 safari/534.16 "Download link

7. Use wget–spider test Download link

When you plan to do a timed download, you should test the download link at the scheduled time to see if it is valid. We can increase the –spider parameter to check.

1. wget--spider URL

If the download link is correct, it will show

1. wget--spider URL
2. Spider mode enabled. Check if remote file exists.
3. HTTP request sent, awaiting response ... OK
4. length:unspecified [text/html]
5. Remote file exists and could contain further links,
6. But recursion was disabled-not retrieving.

This ensures that the download can take place at the scheduled time, but when you give the wrong link, the following error will be displayed

1. wget--spider URL
2. Spider mode enabled. Check if remote file exists.
3. HTTP request sent, awaiting response ... 404 Not Found
4. Remote file does not exist--broken link!!!

You can use the spider parameter in the following situations:

• Check before scheduled download
• Interval detect whether a site is available
• Check for dead links on site pages

8. Use Wget–tries to increase the number of retries

It is also possible to fail if the network is having problems or downloading a large file. wget default retry 20 connection download file. If necessary, you can use –tries to increase the number of retries.

1. wget--tries=40 URL

9. Download multiple Files using Wget-i

First, save a copy of the download link file

1. Cat > Filelist.txt
2. Url1
3. Url2
4. Url3
5. Url4

Then use this file and parameters-I download

1. Wget-i filelist.txt

10. Using Wget–mirror Mirror website

The following example is to download the entire website to local.

1. wget--mirror-p--convert-links-p./local URL

• –miror: Account opening image download
• -P: Download all files for HTML page to display normal
• –convert-links: After download, convert cost to link
• -P./local: Save all files and directories to a locally specified directory

11, use wget–reject filter specified format download

You want to download a website, but you do not want to download pictures, you can use the following commands.

1. wget--reject=gif URL

12. Use Wget-o to save the download information to the log file

You do not want the download information to be displayed directly in the terminal but in a log file, you can use the following command:

1. Wget-o Download.log URL

13, use wget-q limit the total download file size

When you want to download more than 5M files and exit the download, you can use the following command:

1. Wget-q5m-i filelist.txt

Note: This parameter does not work for a single file download and is only valid for recursive downloads.

14. Use Wget-r-A to download the specified format file

You can use this feature in the following situations

• Download all pictures of a website
• Download all videos of a website
• Download all PDF files for a website

1. Wget-r-a.pdf URL

15, using wget FTP download

You can use wget to complete the download of the FTP link.
Using wget anonymous FTP download

1. wget Ftp-url

FTP download with wget user name and password authentication

1. wget--ftp-user=username--ftp-password=password URL

Linux Wget Commands