Linux Curl Command Parameters

First, LinuxCurlExamples of usage:1. Linux Curl Crawl Web page:

Crawl Baidu:

Curl http : //www.baidu.com

If found garbled, you can use iconv transcoding:

Curl http : //iframe.ip138.com/ic.asp|iconv-fgb2312

Iconv usage See: Handling text files with ICONV commands under Linux/unix system.

2. Linux Curl UsesAgent：

Linux Curl uses HTTP proxies to crawl pages:

Curl -x 111.95.243.36: http : //iframe.ip138.com/ic.asp|iconv-fgb2312

Curl -x 111.95.243.36:-U aiezu: Password http:// Www.baidu.com

To crawl a page using the socks agent:

Curl --socks4 202.113.65.229: 443 http://iframe.ip138.com/ic.asp|iconv-fgb2312

Curl --socks5 202.113.65.229: 443 http://iframe.ip138.com/ic.asp|iconv-fgb2312

The proxy server address can be obtained from the crawler agent.

3. Linux Curl ProcessingCookies

Receiving cookies:

Curl -c /tmp/Cookies http: //www.baidu.com #cookies保存到/tmp/cookies file

Send cookies:

Curl - b "KEY1=VAL1;KEY2=VAL2;" http://www.baidu.com #发送cookies文本

Curl - b /tmp/ Cookies HTTP://www.baidu.com #从文件中读取cookies

4. Linux Curl sends data:

The Linux curl get method submits data:

Curl - G - D "Name=value&name2=value2 " http://www.baidu.com

Linux Curl post Way to submit data:

Curl - D "name=value&name2=value2" http ://www.baidu.com #post数据

Curl -d a = B & C = D & txt @ / tmp / txt http : //www.baidu.com #post文件

To upload a file as a form:

Curl -F file = @/tmp/ Me. txt http : //www.aiezu.com

Equivalent to setting the form form method= "POST" and enctype= ' Multipart/form-data ' two properties.

5. Linux Curl HTTP Header processing:

To set HTTP request header information:

Curl - A "mozilla/5.0 firefox/21.0" http : //www.baidu.com #设置http请求头User-agent

Curl - e "http://pachong.org/" http: //www.baidu.com #设置http请求头Referer

Curl - H "connection:keep-alive \ user-agent:mozilla/5.0" HTTP://www.aiezu.com

To set HTTP response header processing:

Curl - I http://www.aiezu.com #仅仅返回header

Curl - D /tmp/Header http ://www.aiezu.com #将http header saved to/tmp/header file

6. Linux Curl Certification:

Curl - u aiezu:password http : //www.aiezu.com #用户名密码认证

Curl - E mycert. PEM HTTPS : //www.baidu.com #采用证书认证

6. Other:

Curl -# http://www.baidu.com #以 "#" Output progress bar

Curl - o /tmp/aiezu http ://www.baidu.com #保存http响应到/tmp/aiezu

Linux Curl Command Parameters