Download the file command wget in Linux

Wget is the most common download command in Linux. Generally, wget + space + URL path of the file to be downloaded

Example: # wget linuxsense.org/xxxx/xxx.tar.gz "> http://www.linuxsense.org/xxxx/xxx.tar.gz

To put it simply, the-C parameter is also very common. It can be resumable data transfer. If it is terminated accidentally, you can continue to use the command to download it.

Example: # wget-C http://www.linuxsense.org/xxxx/xxx.tar.gz

The following describes the usage of wget in detail:

Wget is a free tool for automatically downloading files from the network. It supports HTTP, https, and FTP protocols and can use HTTP proxy.

The so-called automatic download means that wget can be executed in the background after the user exits the system. This means that you can log on to the system, start a wget download task, and exit the system. wget will be executed in the background until the task is completed, compared with most other browsers, users need to participate in downloading a large amount of data, which saves a lot of trouble.

Wget allows you to track links on the HTML page and download them to create the local version of the remote server, completely recreating the directory structure of the original site. This is often called recursive download ". During recursive download, wget complies with the robot exclusion standard (/robots.txt). wget can convert the link to a local file while downloading to facilitate offline browsing.

Wget is very stable. It has strong adaptability to unstable networks with narrow bandwidth. if the download fails due to network reasons, wget keeps trying until the entire file is downloaded. If the server interrupts the download process, it will be again connected to the server to continue the download from the stopped place. This is very useful for downloading large files from servers with limited connection time.

Common wget usage

Wget format

Usage: wget [Option]… [Url]…

* Use wget as the site image:

Wget-r-p-NP-K http://dsec.pku.edu.cn /~ Usr_name/

# Or

Wget-M http://www.tldp.org/LDP/abs/html/

* Download a part of the downloaded files from an unstable network and during free time

Wget-T 0-W 31-C http://dsec.pku.edu.cn/BBC.avi-O down. log &

# Or read the list of files to be downloaded from filelist

Wget-T 0-W 31-C-B ftp://dsec.pku.edu.cn/linuxsoft-I filelist.txt-O down. log &

The above code can also be used for downloading when the network is relatively idle. My usage is: In mozillawill not easily download the urlchain to the memory, then paste it to the filelist.txt file, and execute the second code above before going out of the system at night.

* Download via proxy

Wget-y on-p-K https://sourceforge.net/projects/wvware/

The proxy can be set in the environment variable or wgetrc file.

# Set proxy in Environment Variables

Export proxy = http: // 211.90.168.94: 8080/

# In ~ /. Set proxy in wgetrc

Http_proxy = http://proxy.yoyodyne.com: 18023/

Ftp_proxy = http://proxy.yoyodyne.com: 18023/

Wget option category list

* Start

-V,-version: display the wget version and exit

-H,-help print syntax help

-B,-after the background is started, it is transferred to the background for execution.

-E,-Execute = command: Execute the command in the '. wgetrc' format. For the wgetrc format, see/etc/wgetrc or ~ /. Wgetrc

* Record and input files

-O,-output-file = file: Write the record to the file.

-A,-append-output = file: append the record to the file.

-D,-Debug print debugging output

-Q,-Quiet quiet mode (no output)

-V,-verbose mode (this is the default setting)

-NV,-non-verbose turn off the lengthy mode, but not the quiet mode

-I,-input-file = file: the URL that appears when the file is downloaded

-F,-force-HTML treats the input file as an HTML file

-B,-base = URL uses the URL as the prefix of the relative link in the file specified by the-f-I Parameter

-Sslcertfile = file: Optional client certificate

-Sslcertkey = Keyfile: Specifies the Keyfile of the client certificate.

-EGD-file = file: Specifies the file name of the EGD socket.

* Download

-Bind-address = address specifies the local address (host name or IP address, used when there are multiple local IP addresses or names)

-T,-tries = number indicates the maximum number of attempts (0 indicates no limit ).

-O-output-document = file: Write the document to the file.

-NC,-no-clobber do not overwrite existing files or use the. # prefix

-C,-continue, and then download the files that have not been downloaded

-Progress = type: set the process bar flag.

-N,-timestamping do not re-download the file except for non-newer than the local file

-S,-server-response Print Server Response

-Spider does not download anything.

-T,-Timeout = seconds: set the number of seconds for response timeout.

-W,-Wait = seconds: the interval between two attempts is seconds.

-Waitretry = seconds: Wait 1... Seconds seconds

-Random-Wait waits for 0 during download... 2 * Wait seconds

-Y,-proxy = On/Off open or close the proxy

-Q,-quota = Number sets the download capacity limit

-Limit-rate = Rate: Specifies the download rate.

* Directory

-Nd-no-Directories

-X,-force-directories force Directory Creation

-NH,-no-host-directories do not create the host directory

-P,-directory-Prefix = Prefix: save the file to the directory prefix /...

-Cut-dirs = Number ignore the remote directory of the number layer

* HTTP OPTIONS

-Http-user = User: Set the HTTP user name to user.

-Http-passwd = pass: Set the HTTP password to pass.

-C,-Cache = On/Off allow/Do Not Allow server-side data caching (generally allow ).

-E,-HTML-Extension: Save all text/html files with the. html Extension

-Ignore-length: Ignore the 'content-length' header.

-Header = string insert string in Headers

-Proxy-user = User: Set the proxy username to user.

-Proxy-passwd = pass: Set the proxy password to pass.

-Referer = the URL contains the 'Referer' header in the HTTP request.

-S,-save-headers Save the HTTP header to the file

-U,-User-Agent = agent sets the proxy name to agent instead of wget/version.

-No-http-keep-alive: Disable the HTTP activity Link (permanent link ).

-Cookies = off do not use cookies.

-Load-Cookies = file: loads cookies from the file before starting the session

-Save-Cookies = file: saves cookies to the file after the session ends.

* FTP options

-Nr,-Dont-Remove-listing does not remove the '. listing' file.

-G,-glob = On/Off enable or disable the globbing mechanism of the file name

-Passive-FTP uses passive transmission mode (default ).

-Active-FTP: Active Transmission Mode

-Retr-symlinks: recursively points a link to a file instead of a directory)

* Recursive download

-R,-recursive download-use with caution!

-L,-level = maximum recursive depth of number (inf or 0 indicates infinity ).

-Delete-after: Partial Deletion of objects after completion

-K,-convert-links converts non-relative links to relative links

-K,-backup-converted: Before converting file X, back up the file to X. orig.

-M,-mirror is equivalent to-r-N-l INF-Nr.

-P,-page-requisites download and display all images of HTML files

* Recursive download contains and does not contain (accept/reject)

-A,-accept = List a semicolon-separated list of accepted extensions

-R,-reject = List a semicolon-separated list of unacceptable extensions

-D,-domains = List the list of accepted domains separated by semicolons

-Exclude-domains = List a semicolon-separated list of unacceptable Domains

-Follow-FTP: Tracking FTP links in HTML documents

-Follow-tags = List a semicolon-separated list of HTML tags to be tracked

-G,-ignore-tags = List semicolon-separated list of ignored HTML tags

-H and-span-hosts are recursively transferred to the external host.

-L,-relative only traces relative links

-I,-include-directories = List list of permitted Directories

-X,-exclude-directories = List list of excluded Directories

-NP,-no-parent should not be traced back to the parent directory

Problem

During recursive download, when the directory contains Chinese characters, the local directory name created by wget will be processed using URL encoding rules. For example, "Skynet firewall" is saved as "% CC % EC % Cd % F8 % B7 % C0 % BB % F0 % C7 % BD", which makes reading inconvenient.