Linux Curl Usage Detailed
Curl application, one can be directly through the command-line tools, the other is to use the Libcurl library to do the upper level of development. This article mainly summarizes the command-line tools of HTTP-related applications, especially HTTP downloads; The next story is based on the development of Libcurl Library.
Curl's command-line tools are very powerful, and the functionality of these data interactions is basically done through URLs, and let's take a look at Curl's flexible handling of multiple URLs that make our batch requirements very handy to use.
1, with {} to represent more than one URL
such as Http://site. {one,two,three}.com, as long as you write a different section in {}, you can represent 3 URLs.
2. Use [] to represent multiple URLs
such as ftp://ftp.numericals.com/file[1-100].txt,ftp://ftp.numericals.com/file[001-100].txt,ftp://ftp.letters.com/ File[a-z].txt, [] are arranged in the order of numbers or letters.
It can also indicate discontinuous permutations, such as Http://www.numericals.com/file[1-100:10].txt,http://www.letters.com/file[a-z:2].txt, ":" The numbers that follow indicate the number of steps in a linear order.
3, combined with the use of
such as http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html.
The basic statement for the Curl command-line tool is: curl [options] [URL ...], that is curl [option] [download address]. Therefore, if you want to download Baidu's homepage, the simplest way to start cmd, enter the Curl installation directory, input command: Curl http://www.baidu.com, enter, your Output window will immediately appear Baidu homepage HTML data.
For all boolean options, basically--option indicates support for this option,--no-option indicates that this option is disabled. Here's a summary of some of the more useful options.
1,-o/--output <file>
This is basically the most commonly used option for saving downloaded data to a file. such as Curl-o baidu.html http://www.baidu.com will be Baidu home page saved to baidu.html file. Standard output also displays downloaded statistics such as progress, number of bytes downloaded, download speed, and so on during the download process. If the output is specified as "-", the download data is output to the standard output.
If the output file is a multilevel directory, then the--create-dirs option will be matched to--create-dirs automatically create the output directory for the settings.
If you want to save to a local file name and the same as on the server, you can use the-o option directly (uppercase O) without specifying a local filename.
If you download multiple data files, you can use the multiple URL rules mentioned above. The download file name can also be customized in bulk, using "#". You can add "#" to the filename so that the # in the actual filename will be replaced by the current string in multiple URLs, such as Curl Http://{site,host}.host[1-5].com-o "#1_", #1将被 "site" or "host" instead, # 2 will be replaced by a number between 1-5, which is replaced by a string in the URL that is currently being processed.
In addition to downloading the data, you can add a "-#" option, using a progress bar instead of text to represent progress.
2.-c/--cookie-jar <file name> and-b/--cookie <name=data>
This is the two option to manipulate cookies, and many servers require cookie information. You can specify a file with the previous option to write cookies from other files or cookies from the server to a file, and the latter option to send cookie information to the server, either in the form of "name=data" or directly with a file name that holds the cookie.
3.-x/--proxy <proxyhost[:p ort]>
This option specifies the proxy and port for HTTP and, if no port is specified, the default is 1080. such as Curl-x 201.36.208.19:3128 http://curl.haxx.se/.
4,-u/--user <user:password> and-u/--proxy-user <user:password>
Login to some pages or FTP need to first authenticate, enter the username and password. Curl This option can directly handle this type of operation, with the specified account number and password for login authentication.
The following options specify the proxy username and password, so that you can use this proxy directly to access the Web page, such as Curl-u user:password-x 201.36.208.19:3128 http://curl.haxx.se/.
5,-a/--user-agent <agent string>
This option specifies the type of client that the server uses to determine the platform and browser information that the user applies. such as Curl-a "mozilla/4.0" (compatible; MSIE 6.0; Windows NT 5.0) "Http://www.clientinfo.com,
Indicates that the client is a IE6.0 running on a Windows 2000 system.
6,-d/--data <data>
This option is used to post to the HTTP server to send specific data, the most common is the submission of the form. such as curl-d "user=username&password=111" http://www.login.com, means to use account username and password 111 to send a login to the site application. Multiple data segments can be sent with a "&" connection, and if the data is preceded by a symbol "@", the data source is the file specified later, such as Curl-d password=@d:\pw.txthttp://www.login.com.
7,-c/--continue-at <offset>
The options provide a continuation of the breakpoint and are used in conjunction with the-o option. such as Curl-c-O Http://vfile.home.news.cn/music/public/vd05/200905/31/a8/MUfs052009053117155750a8be70.mp3. If offset is specified, it is resumed from the position of offset. If you do not specify offset, or if you are using "-C-" directly, curl will analyze where to begin the continuation of the transfer.
8,-r/--range <range>
This option specifies the scope of the download byte, which is often applied to block-download files. Range can be represented in a variety of ways, such as 100-500, specifying 400 bytes of data starting at 100, 500 for the last 500 bytes, 5000 for all bytes starting with the No. 5000 byte, and multiple byte blocks to be specified in the middle, separated by ",". Such as
Curl-r 0-1024000-o New_divide_1.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3 &
Curl-r 1024001-2048000-o New_divide_2.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3 &
Curl-r 2048001--o New_divide_3.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3
This will then download the target MP3 file into 3 files, and then use the DOS Copy command: Copy new_divide_1.mp3/b + new_divide_2.mp3/b + new_divide_3.mp3/b New_ Divide.mp3 can be merged into a complete MP3 file with 3 pieces of fragmented files.
9,-w/--write-out <format>
This option formats the data information for some user actions, with the formatting output characters in addition to the common escape characters "\ n", "\ r", "T", and "%". Where "@filename" represents the data in the output filename file, "@-" indicates that the output user writes the data to the standard input, the "%" identifier follows the defined keyword, then the corresponding curl data can be output, and the commonly used keywords are:
http_code--The response code returned by the last operation;
time_total--last operation time, the unit is seconds;
Time_connect--tcp the time consuming to connect to a remote host, in seconds;
The time_pretransfer--file actually starts the transmission before the interaction time consuming, the unit is second, possibly mainly includes some command processing and the protocol analysis time consuming;
The time elapsed before the first byte of the time_starttransfer--file begins to transmit, in seconds, including Time_pretransfer and the server to compute the result;
size_download--The total number of bytes downloaded;
Total number of bytes uploaded by size_upload--;
size_request--The total number of bytes sent to the HTTP request;
Speed_download--curl The average download speed of the successful download;
The average upload speed of Speed_upload--curl successfully uploaded;
For example: Curl-o page.html-s-W%{time_connect}:%{time_starttransfer}:%{time_total} http://curl.haxx.se/, where-s indicates quiet mode, which omits all status information.
、--connect-timeout <seconds> and-m/--max-time <seconds>
The former represents the maximum time, in seconds, of a allowed connection, which indicates the maximum time consuming, in seconds, of the entire operation allowed, which is useful for controlling the duration of a batch operation.
One 、--limit-rate <speed>,-y/--speed-limit <speed> and-y/--speed-time <time>
--limit-rate <speed> Specifies the maximum rate of data transfer, the unit is BYTES/S, or the ' k ' or ' k ' means kb/s, ' m ' or ' m ' means MB/s, ' g ' or ' G ' for GB, and the speed is the average transmission rate, Short-time spikes may exceed this value;
-y/--speed-limit <speed> Specify the minimum transfer rate, in bytes/s, if less than speed value, the transport abort, so the-y option is not set, the default time is 30;-y/--speed-time < time> Specifies the valid time period for the-y option, and if the-y option is not set, the default is 1.
、--max-filesize <bytes>
This option specifies the maximum length of the file to be downloaded, and if the bytes value is exceeded, the download does not start, and curl returns the exit code 63.
、--retry <num>,--retry-delay <seconds> and--retry-max-time <seconds>
When an error occurs during the transfer, such as a timeout, an FTP 5xx return code, or an HTTP 5xx return code, Curl will retry as set, and these options are related to retries. --retry <num> Set the number of retries,--retry-delay <seconds> set two retry intervals,--retry-max-time <seconds> set the maximum time interval of two retries. By default, curl does not retry, and if retried, the first interval is 1 seconds, followed by twice times each time until the interval reaches 10 minutes, followed by a 10-minute retry. If the values of these 3 options are reset, the setting value is executed.
14,-t/--upload-file <file>
This option is an upload command, such as uploading a file to an HTTP server: Curl-t D:\new_divide.mp3 http://www.uploadserver.com/path/; Uploading files to an FTP server: Curl-t D:\new_ Divide.mp3-u User:password ftp://upload_site:port/path/.
Curl defines a set of "EXIT codes" that identifies information about the error when it is present, ranging from 1-83 to a document that can be accessed from the Curl Library. These exit codes are of great help to us in analyzing errors and causes.
above a brief introduction of the use of Curl command-line tools for some simple HTTP and FTP applications, we can try to use more, will slowly discover the powerful function of curl, there are many unique functions to use very simple and convenient
