Linux Curl Usage Detailed

Source: Internet
Author: User
Tags response code

Linux Curl Usage Detailed

The application of curl, one can be directly through the command line tool, the other is the use of Libcurl library to do the upper layer of development. This article mainly summarizes the command-line tool HTTP-related applications, especially the HTTP download, the next one is based on the development of Libcurl Library.



The command-line tools of curl are very powerful, and the functions of these data interactions are mostly URL-based, so let's take a look at the flexible operation of curl on multiple URLs, which makes it easy to apply our batch requirements.

1. Use {} to represent multiple URLs
such as Http://site. {one,two,three}.com, you can represent 3 URLs as long as you write a different part in {}.

2. Use [] to represent multiple URLs
such as ftp://ftp.numericals.com/file[1-100].txt,ftp://ftp.numericals.com/file[001-100].txt,ftp://ftp.letters.com/ File[a-z].txt, [] in the Order of numbers or letters.
can also represent discontinuous permutations, such as Http://www.numericals.com/file[1-100:10].txt,http://www.letters.com/file[a-z:2].txt, ":" The numbers that follow indicate the number of steps in a linear arrangement.

3. Combined use
such as http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html.


The basic statements for the Curl command-line tool are: curl [options] [URL ...], which is curl [option] []. Therefore, if you want to download the homepage of Baidu, the simplest way to start cmd, into the Curl installation directory, input command: Curl http://www.baidu.com, enter, your Output window will immediately appear Baidu home page HTML data.
For all Boolean option, basically--option means that this option is supported,--no-option means disable this option. A few more useful options are summarized below.

1.-o/--output <file>
This is basically the most common option for saving downloaded data to a file. such as Curl-o baidu.html Http://www.baidu.com will Baidu home saved to the baidu.html file. The standard output also displays downloaded statistics, such as progress, download bytes, download speed, and so on during the download process. If the specified output is "-", the download data is output to standard output.
If the output file is a multilevel directory, then to mate with the--create-dirs option,--create-dirs will automatically create the output directory for the setting.
If you want to save to the local file name and the same on the server, you can directly use the-o option (capital O), without specifying a local file name.
If you download multiple data files, you can use the multi-URL rule mentioned above. The download file name can also be customized in bulk, using "#". You can add "#" to the file name so that # in the actual file name will be replaced by the current string in multiple URLs, such as Curl Http://{site,host}.host[1-5].com-o "#1_ # #", #1将被 "site" or "host" instead, # 2 will be replaced by a number between 1-5, which is the string in the URL currently being processed.
In addition to downloading the data, you can add a "-#" option, using a progress bar instead of text to indicate progress.

2.-c/--cookie-jar <file name> and-b/--cookie <name=data>
This is the two option to manipulate cookies, and many servers require cookie information. The previous option allows you to specify a file that writes cookies from other files or cookies from the server to the file, and the latter option to send cookie information to the server, either in the form of "name=data" or directly with a file name that holds the cookie.

3,-x/--proxy <proxyhost[:p ort]>
This option specifies the proxy and port for HTTP, and defaults to 1080 if you do not specify a port. such as Curl-x 201.36.208.19:3128 http://curl.haxx.se/.

4.-u/--user <user:password> and-u/--proxy-user <user:password>
Login to some pages or FTP requires authentication first, enter the user name and password. This option for curl can be used to directly handle this type of operation, with the specified account number and password for login authentication.
The following options specify the user name and password of the proxy so that the Web page can be accessed directly with the proxy, such as Curl-u user:password-x 201.36.208.19:3128 http://curl.haxx.se/.

5,-a/--user-agent <agent string>
This option allows you to specify the client type, which the server uses to determine the platform and browser information for the user. such as Curl-a "mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) "Http://www.clientinfo.com,
Indicates to the server that the client is a IE6.0 running on a Windows 2000 system.



6.-d/--data <data>
This option is used to send specific data to the HTTP server in post, and the most common is to submit the form. such as curl-d "user=username&password=111" http://www.login.com, means to use the account username and password 111 to send a login request to the site. Multiple data segments can be sent together with "&", if the data is preceded by a symbol "@", then the data source is the file specified later, such as curl-d [email protected]:\pw.txthttp://www.login.com.



7.-c/--continue-at <offset>
The re-entry provides the ability to continue the breakpoint and use it with the-o option. such as Curl-c-O Http://vfile.home.news.cn/music/public/vd05/200905/31/a8/MUfs052009053117155750a8be70.mp3. If offset is specified, the continuation is initiated from the position of offset. If offset is not specified, or "-C-" is used directly, curl parses the position itself to begin the continuation.



8.-r/--range <range>
This option specifies the range of download bytes and is often applied to chunked download files. Range is represented in a variety of ways, such as 100-500, specifying 400 bytes of data starting from 100, 500 for the last 500 bytes, 5000 for all bytes starting from the No. 5000 byte, and multiple byte blocks at the same time, separated by "," in the middle. Such as
Curl-r 0-1024000-o New_divide_1.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3 &
Curl-r 1024001-2048000-o New_divide_2.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3 &
Curl-r 2048001--o New_divide_3.mp3 http://vfile.home.news.cn/music/public/vd05/200905/31/a8/ Mufs052009053117155750a8be70.mp3
This will download the target MP3 file chunked to 3 files, then the dos Copy command: Copy new_divide_1.mp3/b + new_divide_2.mp3/b + new_divide_3.mp3/b New_ Divide.mp3 can merge 3 chunked files into a full mp3 file.

9.-w/--write-out <format>
This option formats the output of some user-operated data information, using the format output characters except "\ n", "\ r", "\ T" for these common escape characters, including "@" and "%". where "@filename" means data in the output filename file, "@-" means the output user writes the standard input data; "%" identifier followed by the definition of the keyword, you can output the corresponding curl data, commonly used keywords are:
http_code--The response code returned by the last operation;
time_total--Last operation time, unit is seconds;
Time_connect--tcp the time, in seconds, to connect the remote host;
The time_pretransfer--file actually starts the transfer before the interaction takes time, in seconds, and may include some command processing and protocol analysis time-consuming;
The time elapsed before the first byte of the time_starttransfer--file starts transmitting, in seconds, including Time_pretransfer and the server calculates the result;
size_download--total number of bytes downloaded;
size_upload--The total number of bytes uploaded;
size_request--The total number of bytes sent by the HTTP request;
Speed_download--curl average download speed for successful downloads;
Speed_upload--curl average upload speed for successful uploads;
Example: Curl-o page.html-s-W%{time_connect}:%{time_starttransfer}:%{time_total} http://curl.haxx.se/, where-s represents quiet mode, which omits all status information.

、--connect-timeout <seconds> and-m/--max-time <seconds>
The former represents the longest time, in seconds, of the allowable connection, and the latter represents the maximum time, in seconds, of the entire operation allowed, which is useful for controlling the length of a batch operation.

、--limit-rate <speed>,-y/--speed-limit <speed> and-y/--speed-time <time>
--limit-rate <speed> Specify the maximum data transfer rate, in units of bytes/s, can also be used ' k ' or ' k ' to denote kb/s, ' m ' or ' m ' represents MB/s, ' g ' or ' G ' for GB/s, the speed given is the average transfer rate, Short-time spikes may exceed this value;
-y/--speed-limit <speed> Specifies the minimum transfer rate, in bytes/s, if less than the speed value, the transfer is aborted, so the-y option is not set, and the default time is 30;-y/--speed-time < time> Specifies the valid time period for the-y option, which defaults to 1 if the-y option is not set.

、--max-filesize <bytes>
This option specifies the maximum length of the file to be downloaded, and if the bytes value is exceeded, the download does not start, and curl returns the exit code 63.

、--retry <num>,--retry-delay <seconds> and--retry-max-time <seconds>
When an error occurs during transmission, such as a timeout, an FTP 5xx return code, or an HTTP 5xx return code, Curl retries with the settings, and these options are all related to retry. --retry <num> Set the number of retries,--retry-delay <seconds> set the interval between two retries,--retry-max-time <seconds> set the maximum time for two retry intervals. By default, curl does not retry, and if you retry, the first interval is 1 seconds, then the last time interval is twice times, until the interval reaches 10 minutes, and then retries are taken at 10-minute intervals. If the values of these 3 options are reset, the setting value is executed.

14.-t/--upload-file <file>
This option is an upload command, such as uploading a file to an HTTP server: Curl-t D:\new_divide.mp3 http://www.uploadserver.com/path/; Uploading files to an FTP server: Curl-t D:\new_ Divide.mp3-u User:password ftp://upload_site:port/path/.


Curl defines a set of "EXIT CODES", which identifies the information in the event of an error, currently ranging from 1-83, to the documentation that the Curl library comes with. These exit codes are a great help for our analysis of errors and causes.


The above simple introduction of the use of the Curl Command line tool for some simple HTTP and FTP applications, you can try to use more, will slowly discover the powerful features of curl, there are many unique features used very simple and convenient

Linux Curl Usage detailed

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.