Linux curl Introduction)

Curl is the next powerful HTTP command line tool in Linux and has powerful functions.

1) Let's not talk about it. Let's start from here!

$ Curl http://www.linuxidc.com

After you press enter, the HTML of www.linuxidc.com is displayed on the screen ~

2) Well, is this the case if I want to save the read pages?

$ Curl http://www.linuxidc.com> page.html

Yes, of course, but you don't have to worry about it!

Use the built-in option of curl to save the HTTP result. Use this option:-o

$ Curl-O page.html http://www.linuxidc.com

In this way, you can see a download page progress indicator on the screen. When the Progress reaches 100%, it will naturally be OK.

3) What ?! Cannot access? It must be that your proxy has not been set.

When using curl, you can use this option to specify the proxy server used for HTTP access and its port:-x

$ Curl-x 123.45.67.89: 1080-O page.html http://www.linuxidc.com

4) it is annoying to visit some websites. He uses cookies to record session information.

Browsers like IE/nn can easily process cookie information, but what about our curl ?.....

Let's learn this option:-D <-this is to store the cookie information in the HTTP response to a special file.

$ Curl-x 123.45.67.89: 1080-O page.html-D cookie0001.txt http://www.linuxidc.com

In this case, when the page is stored in the same page as page.html, cookieinformation is also stored in cookie0001.txt.

5) So how can I continue to use the cookie information left by the last visit? You know, many websites rely on monitoring your cookie information to determine if you are not visiting their websites according to rules.

This time, we use this option to append the last cookie information to the HTTP request:-B

$ Curl-x 123.45.67.89: 1080-O page1.html-D cookie0002.txt-B cookie0001.txt http://www.linuxidc.com

In this way, we can simulate almost all the IE operations to access the webpage!

6) Wait a moment ~ I seem to have forgotten something ~

Right! Browser Information

Some annoying websites always use some specific browsers to access them. Sometimes, more importantly, some specific versions of NND must be used, where can I find these weird browsers for it !?

Fortunately, curl provides us with a useful option, which allows us to randomly specify the browser information we declared during this access:-

$ Curl-a "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"-x
1080-O page.html-D cookie0001.txt http://www.linuxidc.com

In this way, when the server receives an access request, it will think that you are an ie6.0 running on Windows 2000. Hey, maybe you are using an apple!

However, "Mozilla/4.73 [En] (X11; U; Linux 2.2; 15 i686" tells the other party that you are running Linux on a PC and use Netscape 4.73.

7) Another common restriction method on the server side is to check the Referer for HTTP access. For example, if you first visit the home page and then access the specified download page
The Referer address is the page address after the first successful access. In this way, as long as the server finds that the Referer address for a certain access to the download page is not the address of the home page, it can be determined that it is a thief
Connected ~

Hate ~ I want to steal the connection ~!!

Fortunately, curl provides us with the option to set Referer:-e

$ Curl-a "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"-x
123.45.67.89: 1080-e "mail.linuxidc.com"-O page.html-D cookie0001.txt

Http://www.linuxidc.com

In this way, you can lie to the other server. You clicked a link from mail.linuxidc.com.

8) It says something is missing! --- Use curl to download an object

As mentioned earlier, you can use-O to download a file from the download page. For example,

$ Curl-O 1.jpg http://cgi2.tky.3web.ne.jp /~ Zzh/screen1.jpg

Here we will teach you a new option:-O uppercase o, which is used as follows:

$ Curl-O http://cgi2.tky.3web.ne.jp /~ Zzh/screen1.jpg

In this way, the file name on the server is automatically stored locally!

Another one is better.

If screen1.jpg, screen2.jpg, screen3.jpg,..., and screen10.jpg need to be downloaded, can we write a script to complete these operations?

Don't do it!

In curl, you can write it like this:

$ Curl-O http://cgi2.tky.3web.ne.jp /~ Zzh/screen [1-10]. jpg

Hehao, amazing ?! ~

9) Let's continue to explain the download process!

$ Curl-O http://cgi2.tky.3web.ne.jp /~ {Zzh, Nick}/[001-201]. jpg

This download is

~ Zzh/001.jpg

~ Zzh/002.jpg

...

~ Zzh/201.jpg

~ Nick/001.jpg

~ Nick/002.jpg

...

~ Nick/201.jpg

Is it convenient enough? Hahaha

Why? I'm so happy.

Because the file names under zzh/Nick are both 001,002... and 201, the downloaded file has a duplicate name, and the previous files are overwritten later ~

It doesn't matter. We still have something worse!

$ Curl-O %2_%1.jpg http://cgi2.tky.3web.ne.jp /~ {Zzh, Nick}/[001-201]. jpg

-This is the download of the custom file name? -Right, huh, huh!

In this way, the downloaded file name is changed to the following :~ Zzh/001.jpg --> download: 001-zzh.jpg Original :~ Nick/001.jpg --> download: 001-nick.jpg

In this way, you are not afraid of duplicate names of files.

 

9) continue to download

On Windows, we usually use tools like flashget to help us download data in parallel in multiple parts and resume data transmission. Curl won't lose to anyone in these aspects.

For example, if we download screen1.jpg and suddenly drop the line, we can resume the download.

$ Curl-c-o http://cgi2.tky.3wb.ne.jp /~ Zzh/screen1.jpg

Of course, you don't need to use a flashget to download half of the files to fool me about the possibility of using half of the files downloaded from other software ~

For multipart download, we can use this option:-R

Example

For example, we have a http://cgi2.tky.3web.ne.jp /~ Zzh/zhao1.mp3 to download (Instructor Zhao's phone recitation), we can use this command:

$ Curl-R 0-10240-o "Zhao. Part1" http:/cgi2.tky.3web.ne.jp /~ Zzh/zhao1.mp3 &\

$ Curl-r 10241-20480-o "Zhao. Part1" http:/cgi2.tky.3web.ne.jp /~ Zzh/zhao1.mp3 &\

$ Curl-r 20481-40960-o "Zhao. Part1" http:/cgi2.tky.3web.ne.jp /~ Zzh/zhao1.mp3 &\

$ Curl-r 40961--o "Zhao. Part1" http:/cgi2.tky.3web.ne.jp /~ Zzh/zhao1.mp3

In this way, you can download them in multiple parts. However, you need to merge these broken files by yourself. If you use Unix or apple, use Cat Zhao. PART *> Zhao. if MP3 is used in windows, use copy/B.

All of the above are HTTP protocol downloads. In fact, FTP can also be used. Usage well,

$ Curl-u name: passwd ftp: // ip: Port/path/File

Or you are familiar

$ Curl ftp: // name: passwd @ IP: Port/path/File

10) after the download is completed, the next step is to talk about uploading. the upload option is-t.

For example, we can upload a file to FTP:

$ Curl-T localfile-u name: passwd ftp: // upload_site: Port/path/

Of course, you can also upload files to the HTTP server, for example

$ Curl-T localfile: http://cgi2.tky.3web.ne.jp /~ Zzh/ABC. cgi

Note: At this time, the protocol used is the http put method.

Speaking of put, hey, naturally, it reminds the old server of the other methods that haven't been mentioned yet! Neither get nor post can be forgotten.

When submitting a form over HTTP, the post mode and get mode are commonly used.

In get mode, no option is required. You only need to write the variable in the URL, for example:

$ Curl http://www.linuxidc.com/login.cgi? User = nickwolfe & Password = 12345

The option in post mode is-D.

For example,

$ Curl-d "user = nickwolfe & Password = 12345" http://www.linuxidc.com/login.cgi

It is equivalent to sending a login application to this site ~

Whether the get mode or post mode should be used depends onProgramSet.

Note that, in post mode, files are uploaded, such

<Form method = "Post" enctype = "multipar/form-Data" Action = "http://cgi2.tky.3web.ne.jp /~ Zzh/up_file.cgi ">

<Input type = file name = upload>

<Input type = submit name = Nick value = "go">

</Form>

To simulate such an HTTP form with curl, the syntax is as follows:

$ Curl-F upload = @ localfile-F Nick = go http://cgi2.tky.3web.ne.jp /~ Zzh/up_file.cgi

Luo luochao has said so much. In fact, curl still has many skills and usage, such as using a local certificate during HTTPS.

$ Curl-e localcert. pem https: // remote_server

For example, you can use curl to search the dictionary through the dict protocol ~

$ Curl dict: // dict.org/d:computer

 

 

This article from http://www.linuxidc.com/Linux/2008-01/10891.htm