The Curl command is detailed

Command cases

1, read the page, the content of the Web page output

Curl http://www.linuxidc.com

2, save the page, download the file

Download Web page with page.html name: Curl–o page.html http://www.linuxidc.com

name the Web page with the file name on the server : Curl –O Http://cgi2.tky.3web.ne.jp/~zzh/screen1.JPG

Bulk Download file: Curl-o http://cgi2.tky.3web.ne.jp/~zzh/screen[1-10]. Jpg

3. Using a proxy server and its ports

Curl–x 127.0.0.1:8580–o page.html http://www.linuxidc.com

4. Use cookies to record session information

Cookie information is stored in Cookie1.txt: Curl–o page.html–d cookie1.txt http://www.linuxidc.com

Use the last cookie and generate a new cookie:curl–o page.html–d cookie2.txt-b cookie2.txt http://www.linuxidc.com

5, the breakpoint continues to pass

For example, download Screen1. JPG, suddenly dropped the line, you can start to continue: Curl-c-O Http://cgi2.tky.3wb.ne.jp/~zzh/screen1.JPG

Alternatively, you can use the- r option for chunked download

http://blog.csdn.net/cheng_fangang/article/details/8662250

• To send a POST request:
  • If transfer file: curl-f "[Email protected];type=text/plain" Localhost:8080/request_body
  • Normal POST request: curl-d "METHOD=SEARCHONE&MODULE=SELLER&USER_NAME=WB-LIQIU&NICKNAME=DD"-H "Host: Fmp.view.lz.taobao.com "" 10.235.160.141:8082/api.php "
• Simulate login, transfer cookies and parameters:
  • Login: curl-c./cookie_c.txt-f log=aaaa-f pwd=****** http://blog.51yip.com/wp-login.php
  • After logging in, you can use the-b parameter to carry cookie-b./cookie_c.txt http://www.***.com
• Forge Source Address, some website will judge, request source Address
  • CURL-E http://localhost http://www.sina.com.cn
• Fake cookies to prevent 302 jump pages from downloading
  
  • wget--cookies=on--load-cookies=cookie.txt--keep-session-cookies--save-cookies=cookie.txt/http detail.tmall.com/item.htm?id=25139372969
• Using the proxy cache
  • Curl-l-x127.0.0.1:13128 http://detail.tmall.com/item.htm?id=25823396605

Command Case explanation

First, view the source of the Web page

Directly after the Curl command add a URL, you can see the Web source. We take the URL www.sina.com as an example (select the URL, mainly because its page code is shorter)

Curl www.baidu.com

If you want to save this page, you can use the-o parameter, which is equivalent to using the wget command.

Curl-o [file name] Www.baidu.com

second, automatically jump to some of the URL is automatically jump. With the-l parameter, curl jumps to the new URL. Curl-l http://item.taobao.com/item.htm?id=25823396605 Type the above command, the result will automatically jump to http://detail.tmall.com/item.htm?id=25823396605

third, the display header information -I parameter can display the HTTP response header information, along with the Web page code. (The-i parameter is the header information that displays only HTTP response.) ）

Curl-i www.baidu.com

http/1.1 Okdate:fri, 05:39:57 gmtcontent-type:text/htmltransfer-encoding:chunkedconnection:keep-alive vary:accept-encodingset-cookie:baiduid=0f251a658e427ebb7cbeb0c3f4a70fae:fg=1; Expires=thu, 31-dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.comset-cookie:bdsvrtm=0; path=/set-cookie:h_ps_pssid=4104_5231_1445_5139_5225_5378_5368_4261_4760_5400; path=/; domain=.baidu.comp3p:cp= "OTI DSP COR IVA our IND COM" Expires:fri, 05:39:45 gmtcache-control:privateserver : bws/1.1bdpagetype:1bdqid:0xc3b306dca955703dbduserid:0<! DOCTYPE Html>

Iv. Display Communication Process The-v parameter can display the entire process of an HTTP communication, including port connections and HTTP request header information. Command: Curl-v www.baidu.com

* About-to-connect () to www.baidu.com port 80* Trying 115.239.210.26 ... connected* connected to www.baidu.com (115.239.210.26) port 80> get/http/1.1> user-agent:curl/7.15.5 (X86_64-redh AT-LINUX-GNU) libcurl/7.15.5 openssl/0.9.8b zlib/1.2.3 libidn/0.6.5> host:www.baidu.com> Accept: */*> < http/1.1 ok< Date:fri, 05:42:37 gmt< content-type:text/html< Con nection:keep-alive< vary:accept-encoding< set-cookie:baiduid=442ad49501ef253ae71f2baf3e0181fb:fg=1; Expires=thu, 31-dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com< set-cookie:bdsvrtm=0; path=/< set-cookie:h_ps_pssid=5228_1461_5187_5138_5225_5379_5368_4261_4760_5401_5286; path=/; domain=.baidu.com< p3p:cp= "OTI DSP COR IVA our IND com" < Expires:fri, 05:41:43 gmt< Cache-control : private< server:bws/1.1< bdpagetype:1< bdqid:0x906950d16fb1e95d< bduserid:0% total% Received Xfe  RdAverage Speed Time Time Dload Upload Total spent left Spee <! DOCTYPE html><!--STATUS ok-->
If you think the above information is not enough, then the following command can see a more detailed communication process: Curl--trace output.txt www.baidu.com or Curl--trace-ascii output.txt www.baidu.com After you run, open the Output.txt file to view.
Of course there are more ruthless, check the page jump process: curl-l-I--trace log http://item.taobao.com/item.htm?id=25823396605
http/1.1 301 Moved PermanentlyServer:TengineDate:Fri, 06:16:01 Gmtcontent-type:text/html;charset=gbkconten t-length:0connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Cont ent-language:zh-cnaccept-ranges:bytesx-varnish:1206961665via:1.1 varnishlocation:http://detail.tmall.com/ item.htm?id=25823396605&x-cache:misscache-control:max-age=3http/1.1 302 FoundServer:TengineDate:Fri, the Feb 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_bucketid:sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d1cache-control:http/1.1 302 Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260c Onnection:closeset-cookie: _tb_token_=ktbzewzfr6qy;domain=.taobao.com; path=/; httponlyset-cookie:cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9;domain=.taobao.com; path=/; Httponlyset-cookie: t=d768c73859b40e10ef81f7abd0824704;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=ktbzewzfr6qy&cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9&t= d768c73859b40e10ef81f7abd0824704&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d1&pacc=a_12nke89qhicayaubtovg==&opi=110.75.118.230&tmsc=1393568161483632http/1.1 302  Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo Our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=ktbzewzfr6qy;domain=.tmall.com;p ath=/set-cookie:cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9;domain=.tmall.com; path=/set-cookie:t=d768c73859b40e10ef81f7abd0824704;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=1http/1.1 302 FouNdServer:TengineDate:Fri, 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_ Bucketid:sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com% 2fitem.htm%3fid%3d25823396605%26%26tbpm%3d2cache-control:http/1.1 302 Founddate:fri, 2014 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:closeset-cookie: _tb_token_=ggu93fejkgt4;domain=. taobao.com; path=/; httponlyset-cookie:cookie2=ef3c440d74ff391de6b560da4ef8a5c9;domain=.taobao.com; path=/; httponlyset-cookie:t=187a71d8df58caac2c4e08d40245c31f;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=ggu93fejkgt4&cookie2=ef3c440d74ff391de6b560da4ef8a5c9&t= 187a71d8df58caac2c4e08d40245c31f&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d2&pacc=vo8f-vlyeypj6we3itx96q==&opi=110.75.118.230&tmsc=1393568161501736http/1.1 302 Founddate:fri, 2014 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM ST A PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=ggu93fejkgt4;domain=.tmall.com; path=/set-cookie:cookie2=ef3c440d74ff391de6b560da4ef8a5c9;domain=.tmall.com; path=/set-cookie:t=187a71d8df58caac2c4e08d40245c31f;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=2http/1.1 302 FoundServer: Tenginedate:fri, 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_bucketid: Sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com%2fitem.htm% 3fid%3d25823396605%26%26tbpm%3d3cache-control:http/1.1 302 Founddate:fri, 06:16:01 gmtcontent-type:text/ Htmlcontent-length:260connection:closeset-cookie: _TB_TOKEN_=UTA86POG6CWC;d omain=.taobao.com; path=/; httponlyset-cookie:cookie2=cd06dba05f2bf1124200861d0b8a151b;domain=.taobao.com; path=/; httponlyset-cookie:t=618388c77aff03aec08309747a82f440;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=uta86pog6cwc&cookie2=cd06dba05f2bf1124200861d0b8a151b&t= 618388c77aff03aec08309747a82f440&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d3&pacc=rptvnjcxdg4wtvfa0jiqoq==&opi=110.75.118.230&tmsc=1393568161510003http/1.1 302  Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo Our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=uta86pog6cwc;domain=.tmall.com;p ath=/set-cookie:cookie2=cd06dba05f2bf1124200861d0b8a151b;domain=.tmall.com; Path=/set-cookie:t=618388c77aff03aec08309747a82f440;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=3http/1.1 okserver:tenginedate: Fri, 06:16:01 Gmtcontent-type:text/html;charset=gbkconnection:keep-alivevary:accept-encodingat_bucketid : Sbucket_-1x-bucket-id: -1cache-control:max-age=1at_autype:4_103979994at_cat:item_50018599x-category:/cat/ 50008090at_nick:%e8%b6%8a%e5%ba%a6%e6%95%b0%e7%a0%81%e4%b8%93%e8%90%a5%e5%ba%97at_itemid:25823396605at_isb:1at _pgty:2at_cat:50018599at_brid:272676782at_prid:221236332at_autype:0_103979994at_auid: 25823396605content-language:zh-cnx-cache:miss Tcp_miss dirn:-2:-2via:wagbridge010207087016.cm3:8888
Send form information Send form information There are get and post two ways. The Get method is relatively simple, as long as the data is attached to the URL after the line: Curl Baidu.com/form.cgi?data=xxx
The Post method must separate the data from the URL, for example: curl-d "METHOD=SEARCHONE&MODULE=SELLER&USER_NAME=WB-LIQIU&NICKNAME=DD"-H "Host: Fmp.view.lz.taobao.com "" 10.235.160.141:8082/api.php "
vi. File Upload assumed file Upload form you can use Curl to upload files, command: Curl--form [email protected]--form Press=ok http://www.baidu.com
vii. referer field Sometimes you need to provide a referer field in the HTTP request header to indicate where you came from: Curl--referer http://www.baidu.com/http// Www.baidu.com
viii. User Agent field This field is used to represent the device information for the client. Depending on the field, the server sometimes returns pages of different formats, such as mobile and desktop versions, for different devices. IPhone4 's user agent is: "mozilla/5.0 (IPhone; U CPU iPhone os 4_0 like Mac os X; En-US) applewebkit/532.9 (khtml, like Gecko) version/4.0.5 mobile/8a293 safari/6531.22.7 ". Curl can emulate this: Curl--user-agent "[user agent]" [URL]
ix. Cookies Use the--cookie parameter, which allows curl to send cookies. Curl--cookie "Name=xxx" www.baidu.com the value of the specific cookie can be obtained from the Set-cookie field of the HTTP response header information. can also be used for example: Curl-c./cookie.txt http://www.baidu.com
10, adding header information sometimes need to add a header message in the HTTP request. The--header parameter can play this role. Curl--header "Xxx:xxxxxx" http://baidu.com
11, HTTP authentication Some domains require HTTP authentication, then curl needs to use the--user parameter: Curl--user Name:password baidu.com
12. Set Proxy Cache -x<ip:port>
Curl-l-x127.0.0.1:13128 http://detail.tmall.com/item.htm?id=25823396605
Http://www.cnblogs.com/liqiu/p/3200582.html
Curl Command Details (GO)