Turn: Curl in a detailed

Description of Use

The Curl command is a powerful network tool that can download files via HTTP, FTP, and so on, and can also upload files. In fact, curl is far more than the previous features, and you can get more information from the man Curl reading the manual page. Similar tools also have wget.

The Curl command uses the Libcurl library to implement, Libcurl library commonly used in the C program to handle HTTP requests, CURLPP is a C + + package Libcurl, these things can be used in crawling Web pages, network monitoring and other aspects of development, The Curl command can help resolve problems that are encountered during development.

Common parameters

There are many parameters to the Curl command, and here are just a list of those I used, especially in shell scripts.

-A: Feel free to specify your own browser information as claimed by this visit

-b/--cookie <name=string/file> Cookie string or file read location, use option to append the last cookie information to the HTTP request.

-c/--cookie-jar <file> Write cookies to this file after the operation is complete

-c/--continue-at <offset> Breakpoint Continuation

-d/--data <data> HTTP Post mode transfer data

-d/--dump-header <file> Write header information to this file

-f/--form <name=content> analog HTTP form submission data

The-v/--verbose lowercase v parameter, which is especially useful in debugging scripts, is used to print more information, including sent request information.

-m/--max-time <seconds> Specifies the maximum length of time to process

-h/--header

-s/--slient reduce the output of information, such as progress

--connect-timeout <seconds> Specifies the maximum length of time to attempt a connection

-x/--proxy <proxyhost[:p ort]> Specify proxy server address and port, port defaults to 1080

-t/--upload-file <file> Specify upload file path

-o/--output <file> Specify output file name

--retry <num> Specify the number of retries

-e/--referer <URL> Specify a reference address

-i/--head only returns header information, using head request

-u/--user <user[:p assword]> setting up the user and password for the server

-O: According to the file name on the server, automatically exists local

-r/--range <range> retrieving byte ranges from http/1.1 or FTP servers

-t/--upload-file <file> Uploading files

Using the example

1, crawl the page content into a file

[[email protected] mytest]# Curl-o home.html http://www.baidu.com--will be caught in Baidu home content to home.html

[[email protected] mytest] #curl-o #2_ #1.jpghttp://cgi2.tky.3web.ne.jp/~{a,b}/[001-201]. Jpg

Because the file name under A/b is 001,002...,201, downloaded files with the same name, so that the custom download down the filename, it becomes this: the original: a/001.jpg--> after download: 001-a.jpg Original: B/001.jpg--- > After download: 001-b.jpg

2, with-O (uppercase), the following URL to specific to a file, or not catch it down. You can also use regular to grab things.

[Email protected] mytest]# Curl-o http://www.baidu.com/img/bdlogo.gif

The results of the operation are as follows:

% total% Received% xferd Average speed Time Time current
Dload Upload Total spent
1575 1575 0 0 14940 0--:--:----:--:----:--:--1538k

A bdlogo.gif picture is generated in the current execution directory.

[Email protected] mytest]# Curl-o http://xxxxx/screen[1-10]. JPG--Download screen1.jpg~screen10.jpg

3, simulate form information, simulate login, save cookie Information

[Email protected] mytest]# curl-c/cookie_c.txt-f log=aaaa-f pwd=******http://www.xxxx.com/wp-login.php

4, simulating form information, simulating login, saving header information

[Email protected] mytest]# curl-d/cookie_d.txt-f log=aaaa-f pwd=******http://www.xxxx.com/wp-login.php

The cookie generated by-C (lowercase) is not the same as the cookie in-D.

5, use cookie file

[Email protected] mytest]# curl-b/cookie_c.txt http://www.XXXX.com/wp-admin

6, the breakpoint continues to pass,-C (uppercase)

[Email protected] mytest]# curl-c-o http://www.baidu.com/img/bdlogo.gif

7, transfer the data, it is best to use the login page test, because you pass the value of the past, curl back to grasp the data, you can see that you pass the value has no success

[Email protected] mytest]# curl-d log=aaaa http://www.XXXX.com/wp-login.php

8, show the fetch error, the following example, it is very clear that.

[Email protected] mytest]# CURL-FHTTP://WWW.XXXX.COM/ASDF

Curl: () The requested URL returned error:404

[Email protected] mytest]# CURLHTTP://WWW.XXXX.COM/ASDF

9, forged source address, some website will judge, request source address, prevent hotlinking.

[Email protected] mytest]# curl-ehttp://localhosthttp://www.xxxx.com/wp-login.php

10, when we often use curl to engage in people's things, people will put your IP to shield off, this time, we can use the agent

[Email protected] mytest]# curl-x 24.10.28.84:32779-o home.htmlhttp://www.xxxx.com

11, than the larger stuff that we can download in sections

[Email protected] mytest]# curl-r 0-100-o img.part1http://www.xxxx.com/wp-content/uploads/2010/09/compare_varnish.jpg

% total% Received% xferd Average speed Time Time current

Dload Upload Total spent

101 101 0 0 0--:--:----:---:----:--:--0

[Email protected] mytest]# curl-r 100-200-o img.part2http://www.xxxx.com/wp-ontent/uploads/2010/09/compare_varnish.jpg

% total% Received% xferd Average speed Time Time current

Dload Upload Total spent

100 101 100 101 0 0 57 0 0:00:01 0:00:01--:--:--0

[Email protected] mytest]# curl-r 200--o img.part3http://www.xxxx.com/wp-content/uploads/2010/09/compare_varnish.jpg

% total% Received% xferd Average speed Time Time current

Dload Upload Total spent

104k 104k 0 0 52793 0 0:00:02 0:00:02--:--:--88961

[[email protected] mytest]# ls |grep part | Xargs Du-sh

4.0K One.part1

112K Three.part3

4.0K Two.part2

When you use them, cat Ok,cat img.part* >img.jpg

12, download progress information is not displayed

[Email protected] mytest]# curl-s-o aaa.jpg http://www.baidu.com/img/bdlogo.gif

13, Show download progress bar

[[email protected] mytest]# curl-0 http://www.baidu.com/img/bdlogo.gif ( request by http1.0 protocol )

####################################################################### 100%

14. Download files via FTP

[[email protected] ~]$ curl-u Username: password-ohttp://www.xxxx.com/demo/curtain/bbstudy_files/style.css

% total% Received% xferd Average speed Time Time current

Dload Upload Total spent

101 1934 101 1934 0 0 3184 0--:--:----:--:----:--:--7136

[[email protected] ~]$ curl-u Username: password-o http://www.XXXX.com/demo/curtain/bbstudy_files/style.css

% total% Received% xferd Average speed Time Time current

Dload Upload Total spent

101 1934 101 1934 0 0 3184 0--:--:----:--:----:--:--7136

Or in the following way

[[email protected] ~]$ Curl-o ftp://user name: Password @ip:port/demo/curtain/bbstudy_files/style.css

[[email protected] ~]$ Curl-o ftp://user name: Password @ip:port/demo/curtain/bbstudy_files/style.css

15, upload via FTP

[[email protected] ~]$ curl-t test.sql ftp://user name: Password @ip:port/demo/curtain/bbstudy_files/

[[email protected] ~]$ curl-t test.sql ftp://user name: Password @ip:port/demo/curtain/bbstudy_files/

15, Analog Browser header

[Email protected] ~]$ curl-a "mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) "-X 123.45.67.89:1080-o page.html-d cookie0001.txthttp://www.www.baidu.com

16,put, GET, POST

For example curl-t LocalFile http://cgi2.tky.3web.ne.jp/~zz/abc.cgi, this time, the protocol used is the put method of HTTP
Just said put, naturally remembered the other several methos--get and post.
HTTP submits a form, more commonly used is the post mode and get mode
Get mode what option is not needed, just write the variable in the URL.
Like what:
Curl http://www.yahoo.com/login.cgi?user=nick&password=12345
The option for Post mode is-D
For example, curl-d "user=nick&password=12345" http://www.yahoo.com/login.cgi
is equivalent to issuing a login request to this site ~~~~~
In the end to use the Get mode or post mode, the opposite server to see the program settings.
It is important to note that file uploads on files in post mode, such as
<form method= "POST" enctype= "Multipar/form-data" action= "http://cgi2.tky.3web.ne.jp/~zz/up_file.cgi" >
<input Type=file name=upload>
<input type=submit name=nick value= "Go" >
</form>
Such an HTTP form, which we want to emulate with curl, is the syntax:
curl-f [email protected]-F Nick=go http://cgi2.tky.3web.ne.jp/~zz/up_file.cgi

Turn: Curl in a detailed