The Curl command is detailed

Source: Internet
Author: User
Tags http authentication send cookies

Command cases

1, read the page, the content of the Web page output

Curl http://www.linuxidc.com

2, save the page, download the file

Download Web page with page.html name: Curl–o page.html http://www.linuxidc.com

name the Web page with the file name on the server : Curl –O Http://cgi2.tky.3web.ne.jp/~zzh/screen1.JPG

Bulk Download file: Curl-o http://cgi2.tky.3web.ne.jp/~zzh/screen[1-10]. Jpg

3. Using a proxy server and its ports

Curl–x 127.0.0.1:8580–o page.html http://www.linuxidc.com

4. Use cookies to record session information

Cookie information is stored in Cookie1.txt: Curl–o page.html–d cookie1.txt http://www.linuxidc.com

Use the last cookie and generate a new cookie:curl–o page.html–d cookie2.txt-b cookie2.txt http://www.linuxidc.com

5, the breakpoint continues to pass

For example, download Screen1. JPG, suddenly dropped the line, you can start to continue: Curl-c-O Http://cgi2.tky.3wb.ne.jp/~zzh/screen1.JPG

Alternatively, you can use the- r option for chunked download

http://blog.csdn.net/cheng_fangang/article/details/8662250

    • To send a POST request:
      • If transfer file: curl-f "[Email protected];type=text/plain" Localhost:8080/request_body
      • Normal POST request: curl-d "METHOD=SEARCHONE&MODULE=SELLER&USER_NAME=WB-LIQIU&NICKNAME=DD"-H "Host: Fmp.view.lz.taobao.com "" 10.235.160.141:8082/api.php "
    • Simulate login, transfer cookies and parameters:
      • Login: curl-c./cookie_c.txt-f log=aaaa-f pwd=****** http://blog.51yip.com/wp-login.php
      • After logging in, you can use the-b parameter to carry cookie-b./cookie_c.txt http://www.***.com
    • Forge Source Address, some website will judge, request source Address
      • CURL-E http://localhost http://www.sina.com.cn
    • Fake cookies to prevent 302 jump pages from downloading
      • wget--cookies=on--load-cookies=cookie.txt--keep-session-cookies--save-cookies=cookie.txt/http detail.tmall.com/item.htm?id=25139372969
    • Using the proxy cache
      • Curl-l-x127.0.0.1:13128 http://detail.tmall.com/item.htm?id=25823396605
Command Case explanation

First, view the source of the Web page

Directly after the Curl command add a URL, you can see the Web source. We take the URL www.sina.com as an example (select the URL, mainly because its page code is shorter)

Curl www.baidu.com

If you want to save this page, you can use the-o parameter, which is equivalent to using the wget command.

Curl-o [file name] Www.baidu.com

second, automatically jump to some of the URL is automatically jump. With the-l parameter, curl jumps to the new URL. Curl-l http://item.taobao.com/item.htm?id=25823396605 Type the above command, the result will automatically jump to http://detail.tmall.com/item.htm?id=25823396605

third, the display header information -I parameter can display the HTTP response header information, along with the Web page code. (The-i parameter is the header information that displays only HTTP response.) )

Curl-i www.baidu.com
http/1.1 Okdate:fri, 05:39:57 gmtcontent-type:text/htmltransfer-encoding:chunkedconnection:keep-alive vary:accept-encodingset-cookie:baiduid=0f251a658e427ebb7cbeb0c3f4a70fae:fg=1; Expires=thu, 31-dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.comset-cookie:bdsvrtm=0; path=/set-cookie:h_ps_pssid=4104_5231_1445_5139_5225_5378_5368_4261_4760_5400; path=/; domain=.baidu.comp3p:cp= "OTI DSP COR IVA our IND COM" Expires:fri, 05:39:45 gmtcache-control:privateserver : bws/1.1bdpagetype:1bdqid:0xc3b306dca955703dbduserid:0<! DOCTYPE Html>

Iv. Display Communication Process The-v parameter can display the entire process of an HTTP communication, including port connections and HTTP request header information. Command: Curl-v www.baidu.com

* About-to-connect () to www.baidu.com port 80* Trying 115.239.210.26 ... connected* connected to www.baidu.com (115.239.210.26) port 80> get/http/1.1> user-agent:curl/7.15.5 (X86_64-redh AT-LINUX-GNU) libcurl/7.15.5 openssl/0.9.8b zlib/1.2.3 libidn/0.6.5> host:www.baidu.com> Accept: */*> < http/1.1 ok< Date:fri, 05:42:37 gmt< content-type:text/html< Con nection:keep-alive< vary:accept-encoding< set-cookie:baiduid=442ad49501ef253ae71f2baf3e0181fb:fg=1; Expires=thu, 31-dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com< set-cookie:bdsvrtm=0; path=/< set-cookie:h_ps_pssid=5228_1461_5187_5138_5225_5379_5368_4261_4760_5401_5286; path=/; domain=.baidu.com< p3p:cp= "OTI DSP COR IVA our IND com" < Expires:fri, 05:41:43 gmt< Cache-control : private< server:bws/1.1< bdpagetype:1< bdqid:0x906950d16fb1e95d< bduserid:0% total% Received Xfe  RdAverage Speed Time Time Dload Upload Total spent left Spee <! DOCTYPE html><!--STATUS ok-->

If you think the above information is not enough, then the following command can see a more detailed communication process: Curl--trace output.txt www.baidu.com or Curl--trace-ascii output.txt www.baidu.com After you run, open the Output.txt file to view.

Of course there are more ruthless, check the page jump process: curl-l-I--trace log http://item.taobao.com/item.htm?id=25823396605

http/1.1 301 Moved PermanentlyServer:TengineDate:Fri, 06:16:01 Gmtcontent-type:text/html;charset=gbkconten t-length:0connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Cont ent-language:zh-cnaccept-ranges:bytesx-varnish:1206961665via:1.1 varnishlocation:http://detail.tmall.com/ item.htm?id=25823396605&x-cache:misscache-control:max-age=3http/1.1 302 FoundServer:TengineDate:Fri, the Feb 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_bucketid:sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d1cache-control:http/1.1 302 Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260c Onnection:closeset-cookie: _tb_token_=ktbzewzfr6qy;domain=.taobao.com; path=/; httponlyset-cookie:cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9;domain=.taobao.com; path=/; Httponlyset-cookie: t=d768c73859b40e10ef81f7abd0824704;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=ktbzewzfr6qy&cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9&t= d768c73859b40e10ef81f7abd0824704&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d1&pacc=a_12nke89qhicayaubtovg==&opi=110.75.118.230&tmsc=1393568161483632http/1.1 302  Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo Our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=ktbzewzfr6qy;domain=.tmall.com;p ath=/set-cookie:cookie2=6c6bc65b9e9a5159cff5b3d0cae4dfd9;domain=.tmall.com; path=/set-cookie:t=d768c73859b40e10ef81f7abd0824704;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=1http/1.1 302 FouNdServer:TengineDate:Fri, 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_ Bucketid:sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com% 2fitem.htm%3fid%3d25823396605%26%26tbpm%3d2cache-control:http/1.1 302 Founddate:fri, 2014 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:closeset-cookie: _tb_token_=ggu93fejkgt4;domain=. taobao.com; path=/; httponlyset-cookie:cookie2=ef3c440d74ff391de6b560da4ef8a5c9;domain=.taobao.com; path=/; httponlyset-cookie:t=187a71d8df58caac2c4e08d40245c31f;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=ggu93fejkgt4&cookie2=ef3c440d74ff391de6b560da4ef8a5c9&t= 187a71d8df58caac2c4e08d40245c31f&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d2&pacc=vo8f-vlyeypj6we3itx96q==&opi=110.75.118.230&tmsc=1393568161501736http/1.1 302 Founddate:fri, 2014 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM ST A PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=ggu93fejkgt4;domain=.tmall.com; path=/set-cookie:cookie2=ef3c440d74ff391de6b560da4ef8a5c9;domain=.tmall.com; path=/set-cookie:t=187a71d8df58caac2c4e08d40245c31f;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=2http/1.1 302 FoundServer: Tenginedate:fri, 06:16:01 Gmtcontent-type:text/htmlcontent-length:260connection:keep-aliveat_bucketid: Sbucket_-1x-bucket-id: -1location:http://jump.taobao.com/jump?target=http%3a%2f%2fdetail.tmall.com%2fitem.htm% 3fid%3d25823396605%26%26tbpm%3d3cache-control:http/1.1 302 Founddate:fri, 06:16:01 gmtcontent-type:text/ Htmlcontent-length:260connection:closeset-cookie: _TB_TOKEN_=UTA86POG6CWC;d omain=.taobao.com; path=/; httponlyset-cookie:cookie2=cd06dba05f2bf1124200861d0b8a151b;domain=.taobao.com; path=/; httponlyset-cookie:t=618388c77aff03aec08309747a82f440;domain=.taobao.com; Expires=thu, 29-may-2014 06:16:01 GMT; path=/p3p:cp= ' CURa ADMa DEVa Psao psdo our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Location:http://pass.tmal l.com/add?_tb_token_=uta86pog6cwc&cookie2=cd06dba05f2bf1124200861d0b8a151b&t= 618388c77aff03aec08309747a82f440&target=http%3a%2f%2fdetail.tmall.com%2fitem.htm%3fid%3d25823396605%26% 26tbpm%3d3&pacc=rptvnjcxdg4wtvfa0jiqoq==&opi=110.75.118.230&tmsc=1393568161510003http/1.1 302  Founddate:fri, 06:16:01 gmtcontent-type:text/htmlcontent-length:260connection:closep3p:cp= ' CURa ADMa DEVa Psao psdo Our BUS UNI PUR INT DEM STA PRE COM NAV OTC NOI DSP COR ' Set-cookie: _tb_token_=uta86pog6cwc;domain=.tmall.com;p ath=/set-cookie:cookie2=cd06dba05f2bf1124200861d0b8a151b;domain=.tmall.com; Path=/set-cookie:t=618388c77aff03aec08309747a82f440;domain=.tmall.com; path=/location:http://detail.tmall.com/item.htm?id=25823396605&&tbpm=3http/1.1 okserver:tenginedate: Fri, 06:16:01 Gmtcontent-type:text/html;charset=gbkconnection:keep-alivevary:accept-encodingat_bucketid : Sbucket_-1x-bucket-id: -1cache-control:max-age=1at_autype:4_103979994at_cat:item_50018599x-category:/cat/ 50008090at_nick:%e8%b6%8a%e5%ba%a6%e6%95%b0%e7%a0%81%e4%b8%93%e8%90%a5%e5%ba%97at_itemid:25823396605at_isb:1at _pgty:2at_cat:50018599at_brid:272676782at_prid:221236332at_autype:0_103979994at_auid: 25823396605content-language:zh-cnx-cache:miss Tcp_miss dirn:-2:-2via:wagbridge010207087016.cm3:8888

Send form information Send form information There are get and post two ways. The Get method is relatively simple, as long as the data is attached to the URL after the line: Curl Baidu.com/form.cgi?data=xxx

The Post method must separate the data from the URL, for example: curl-d "METHOD=SEARCHONE&MODULE=SELLER&USER_NAME=WB-LIQIU&NICKNAME=DD"-H "Host: Fmp.view.lz.taobao.com "" 10.235.160.141:8082/api.php "

vi. File Upload assumed file Upload form you can use Curl to upload files, command: Curl--form [email protected]--form Press=ok http://www.baidu.com

vii. referer field Sometimes you need to provide a referer field in the HTTP request header to indicate where you came from: Curl--referer http://www.baidu.com/http// Www.baidu.com

viii. User Agent field This field is used to represent the device information for the client. Depending on the field, the server sometimes returns pages of different formats, such as mobile and desktop versions, for different devices. IPhone4 's user agent is: "mozilla/5.0 (IPhone; U CPU iPhone os 4_0 like Mac os X; En-US) applewebkit/532.9 (khtml, like Gecko) version/4.0.5 mobile/8a293 safari/6531.22.7 ". Curl can emulate this: Curl--user-agent "[user agent]" [URL]

ix. Cookies Use the--cookie parameter, which allows curl to send cookies. Curl--cookie "Name=xxx" www.baidu.com the value of the specific cookie can be obtained from the Set-cookie field of the HTTP response header information. can also be used for example: Curl-c./cookie.txt http://www.baidu.com

10, adding header information sometimes need to add a header message in the HTTP request. The--header parameter can play this role. Curl--header "Xxx:xxxxxx" http://baidu.com

11, HTTP authentication Some domains require HTTP authentication, then curl needs to use the--user parameter: Curl--user Name:password baidu.com

12. Set Proxy Cache -x<ip:port>

Curl-l-x127.0.0.1:13128 http://detail.tmall.com/item.htm?id=25823396605

Http://www.cnblogs.com/liqiu/p/3200582.html

Curl Command Details (GO)

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.