Analysis of the usage of wget in Linux system

Source: Internet
Author: User
Tags ftp html tags parent directory save file ftp protocol

Function Description: wget use HTTP and FTP protocol, support proxy server and breakpoint transmission, is based on the console's most powerful download tool.

Command format: wget [parameters]

Common parameters:

1. Startup options:

-V: Displays the version of wget

-H: Show wget usage instructions

-B: Run to the background after startup, log file is written in the current directory "Wget-log" file

-E : Executes a. WGETRC inside the instructions

2. log file and input file options:

-O : Writes the output of the command to the specified file

-A : writes the output of the command to the specified file in an append way

-D: Display debug information

-Q: Execute in quiet mode (no output)

-V: Output details

-NV: Turn off verbose output, but not quiet mode

-I : read URL from specified file

-F: Treat the input file as an HTML file (used concurrently with the-I parameter)

-B : Use with-F to prioritize URLs in the files specified by -I

3. Download options:

-T : When wget cannot establish a connection to the server, try the connection times (0 means no limit)

-O : Saves the downloaded file as a specified

-NC: Do not overwrite existing files

-C: Resume File

-N: Do not retrieve older files than local, download only updated files

-S: Show server response

-T : The number of seconds to set response timeout is

-W: Wait a second between two attempts

-Y: Connecting through a proxy server

-Q : Limit the total size of the download file to no more than , in bytes, you can use the k,m suffix

––limit-rate= : Limit download transfer rate, in bytes, you can use the k,m suffix

4. Directory Options:

-nd: Do not download directory structure, the files downloaded from the server all specified directories are stacked into the current directory

-X: Create a directory structure that is completely remote

-NH: Do not create a directory with the destination host domain name directory, the target host directory structure directly down to the current directory

-P : Save file to directory prefix/ ...

––cut-dirs= : Ignore layer remote directory

5. HTTP Options:

––http-user= : Specify users for HTTP user authentication

––http-passwd= : Specify user password for HTTP user authentication

––no-cache: Disable server-side data caching (allowed by default)

-c,–cache=on/off allows/does not allow server-side data caching (generally allowed).

-e,–html-extension saves all text/html documents with an. html extension

–ignore-length Ignore ' content-length ' header field

–header=string inserts string strings in headers

––proxy-user= : Specifies that the proxy user is

––proxy-passwd= : Specifies that the proxy user password is

-E: Save all documents of type Application/xhtml+xml or text/html with an. html extension

-U : Sets the name of the agent instead of Wget/version

–referer=url contains ' referer:url ' headers in HTTP requests

-s,–save-headers Save HTTP headers to file

-u,–user-agent=agent the name of the agent is agent instead of wget/version.

–no-http-keep-alive closes the HTTP activity link (forever link).

–cookies=off do not use cookies.

--load-cookies=file load cookies from file files before the session starts

–save-cookies=file to save cookies to file at the end of the session

6. FTP option:

––ftp-user= : Specify FTP user

––ftp-passwd= : Specify FTP user password

––no-remove-listing: Do not remove temporary. listing files

––no-glob: globbing mechanism to close file names (that is, do not use wildcard characters)

–PASSIVE-FTP uses the passive transfer mode (the default value).

–active-ftp Use active transfer mode

––NO-PASSIVE-FTP: Turn off the default passive transfer mode (that is, use active transfer mode)

––retr-symlinks: When recursion, point the link to a file (not a directory)

7. Use the recursive method to get the option:

-r: Open recursive download

-L : Specifies that the maximum recursive depth is (INF or 0 represents infinity)

––delete-after: Delete the downloaded local file

-K: Convert non-relative links to relative links

-K (uppercase), –backup-converted to back up the file X before converting it to X.orig

-M: Opens the appropriate option for mirroring, equivalent to-n-r-l inf––no-remove-listing

-P: To ensure that all elements used to display downloaded pages are downloaded, such as images, sounds, and style sheets used in Web pages

8. Allow and reject options for recursive methods:

- A: Specifies the extension file name to allow downloading (comma interval)

-R : extension file name in the specified prohibit download (with comma interval)

-D : Specify which domain name is allowed to download

––exclude-domains= : Specify a domain name that is prohibited from downloading

––FOLLOW-FTP: Tracking FTP links in HTML documents

––follow-tags= : Specifies a comma-delimited list of HTML tags to be tracked in

––ignore-tags= : Specifies a comma-delimited list of HTML tags that are not tracked in

-g,–ignore-tags=list semicolon-delimited list of ignored HTML tags

-H: Go to the external host when recursion

-L: Tracking only relative links

-I : Specify a list of directories that are allowed to be downloaded

-X : List of directories in the specified prohibit download

-NP: Do not go back to the parent directory, that is, download only the contents of the specified directory and subdirectories of the target site

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.