Wget download site files

Source: Internet
Author: User
Tags ftp file http authentication

A very simple command only requires:

wget -c -r -p -k -np [URL]

The meanings of the next parameter are explained below:

-C resumable download-r recursive download, You can traverse the entire site structure-P the page shows the required materials (images, CSS, JS, and so on) -K converts the connection from the downloaded webpage to a local connection (this function is awesome) -NP: only download files from the current site (otherwise, you may install the entire Internet on your hard disk)

The following is a wget help file:

[[Email protected] ~] # Wget -- helpgnu wget 1.12, a non-interactive network file download tool. Usage: wget [Option]... [url]... parameters required for the long option are also required when the short option is used. Start:-V, -- version displays the wget version information and exits. -H, -- help print this help. -B, -- Transfer background to the background after startup. -E, -- execute = command to run a '. wgetrc'-style command. Log in and enter the file-o, -- output-file = file to write the information to the file. -A, -- append-output = file: add the information to file. -D, -- debug prints a large amount of debugging information. -Q, -- Quiet quiet mode (no information output ). -V, -- verbose detailed output (this is the default value ). -NV, -- no-verbose disable detailed output, but do not enter quiet mode. -I, -- input-file = file: Download the URLs in the local or external file. -F, -- force-HTML treats the input file as an HTML file. -B, -- base = parse URL-related HTML input files (specified by the-I-f option ). Download:-T, -- tries = number. Set the number of retries to number (0 indicates no limit ). -- Retry-connrefused: retries even if the connection is rejected. -O, -- output-document = file: Write the document to file. -NC, -- no-clobber do not repeatedly download existing files. -C, -- continue continues to download some of the downloaded files. -- Progress = type: select the progress bar type. -N, -- timestamping only obtains new files than local files. -S, -- server-response: print the server response. -- Spider does not download any files. -T, -- timeout = seconds: Set all timeouts to seconds. -- DNS-Timeout = secs: Set DNS query timeout to secs seconds. -- Connect-Timeout = secs: Set the connection timeout to secs seconds. -- Read-Timeout = secs: Set the read timeout to secs seconds. -W, -- Wait = seconds wait interval is seconds. -- Waitretry = seconds waits for 1 .. seconds during the retry of file retrieval. -- Random-Wait waits for 0... 2 * Wait seconds for retrieval. -- No-Proxy: Disable proxy. -Q, -- quota = Number: Set the retrieval quota to number bytes. -- Bind-address = address: Address (host name or IP address) bound to the local host ). -- Limit-rate = rate: the download speed is limited to rate. -- No-DNS-Cache: Disable the DNS query cache. -- Restrict-file-names = OS-qualified file names contain the characters allowed by OS. -- Case sensitivity is ignored when ignore-case matches a file/directory. -4, -- the inet4-only is connected only to an IPv4 address. -6, -- the inet6-only is connected only to an IPv6 address. -- Prefer-family = family first connects to the address family of the specified protocol as IPV6, IPv4, or none. -- User = User: Set the FTP and HTTP usernames to user. -- Password = pass: set both the FTP and HTTP passwords to pass. -- Ask-Password: enter the password. -- No-Siri: Disable Siri support. -- Local-encoding = ENC Siri uses ENC as the local encoding. -- Remote-encoding = ENC uses ENC as the default remote encoding. Directory:-Nd, -- no-directories. -X, -- force-directories force Directory Creation. -NH, -- no-host-directories do not create a home directory. -- Protocol-directories use the Protocol name in the directory. -P, -- directory-Prefix = prefix to prefix/... save the file -- Cut-dirs = Number ignore the number of remote directory paths. HTTP option: -- http-user = User: Set the HTTP user name to user. -- Http-Password = pass: Set the HTTP password to pass. -- No-Cache does not cache data on the server. -- Default-page = Name: changes the response page (the response page is usually called response index.html "). -E, -- Adjust-extension saves html/CSS documents with appropriate extensions. -- Ignore-length ignores the 'content-length' area of the header. -- Header = string inserts a string in the header. -- Max-redirect: Maximum redirection allowed per page. -- Proxy-user = User: Use user as the proxy username. -- Proxy-Password = pass uses pass as the proxy password. -- Referer = the URL contains 'referer: url' in the HTTP request header '. -- Save-headers: Save the HTTP header to a file. -U, -- User-Agent = agent is identified as agent rather than wget/version. -- No-http-keep-alive disable HTTP keep-alive (permanent connection ). -- No-cookies do not use cookies. -- Load-Cookies = loads cookies from the file before the file session starts. -- Save-Cookies = save cookies to file after the file session ends. -- Keep-session-Cookies load and save SESSION (non-permanent) cookies. -- Post-Data = string: Use the POST method; send the string as the data. -- Post-file = file: Use post to send file content. -- The content-Disposition Header is allowed when the local file name is selected (the content-Disposition Header is still in the experiment ). -- Auth-no-challenge send basic HTTP authentication information without first waiting for the server's challenge. HTTPS (SSL/TLS) Options: -- secure-Protocol = Pr select the security protocol, which can be one of auto, SSLv2, SSLv3, or tlsv1. -- No-check-certificate do not verify the server certificate. -- Certificate = file client certificate file. -- Certificate-type = type client certificate type, PEM or der. -- Private-Key = file private key file. -- Private-key-type = type private key file type, PEM or der. -- Ca-certificate = file contains a set of Ca-authenticated files. -- Ca-directory = dir stores the directory of the hash list for CA authentication. -- Random-file = file contains a file that generates random ssl prng data. -- EGD-file = file is used to name an EGD socket file with random data. FTP option: -- FTP-user = User: Set the FTP user name to user. -- FTP-Password = pass: Set the FTP password to pass. -- No-Remove-listing: Do not delete the '. listing' file. -- No-glob does not use wildcards in the FTP file name. -- No-passive-FTP: Disable the "passive" transmission mode. -- When retr-symlinks recursive directory, get the linked file (rather than the directory ). Recursive download:-R, -- Recursive specifies recursive download. -L, -- level = maximum recursive depth of number (inf or 0 indicates unlimited, that is, all downloads ). -- Delete-after: After the download is complete, delete the local file. -K, -- convert-links enables the link in the downloaded HTML or CSS to point to a local file. -K, -- backup-converted backs up the file X to X. orig before converting the file X. -M, -- mirror-n-R-l inf -- no-Remove-listing. -P, -- page-requisites download all elements such as images used to display HTML pages. -- Strict-Comments: enables precise processing of HTML comments (SGML ). Recursive acceptance/rejection:-A, -- accept = List list of accepted extensions separated by commas. -R, -- reject = List comma-separated list of extensions to be rejected. -D, -- domains = List A list of acceptable domains separated by commas. -- Exclude-domains = List list of fields to be rejected separated by commas. -- Follow-FTP: tracks FTP links in HTML documents. -- Follow-tags = List the list of HTML tags of a comma-separated trace. -- Ignore-tags = List A List Of ignored HTML identifiers separated by commas. -H, -- span-hosts is switched to the external host recursively. -L, -- relative only traces links that have a relationship. -I, -- include-directories = List: List of allowed directories. -X, -- exclude-directories = List: list of excluded directories. -NP, -- no-parent is not traced back to the parent directory. Please send the error report or suggestions to the <bug-wget@gnu.org>.
Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.