Use of the Curl command

Recently the company and another company, the company will regularly send our company a batch of products useful to our company information data, and to be saved in the database for future use, of course, this task fell on me. Soon wrote the model (the data model used to hold the product information) and the interface function, the interface function is to post the data, after writing the interface function is of course the test, but did not test the post to receive the Data API (forgive an inexperienced person's ignorance ...) , but still let me find the test method, is not the remote post way to submit a form, many examples on the web, in this paste my code, in fact, it is not difficult to understand a few package methods, this code is still very well understood, the following

#coding =utf-8import sys, urllib2, urllib,jsonuser_agent =  ' Mozilla/4.0  ( compatible; msie 5.5; windows nt) ' headers = {  ' User-Agent '  : user _agent }info = [  {     "id": 554888,     "Code":  "85317",     "PartNo":  "xxx",     "origin":  "XXX" ,     "product_name":  "xxx",     "type":  "2"   },   {   "id": 554113,     "hs_code":  "8230099",      "PartNo":  "xxx",     "origin":  "xxx",     "product _name ": " xxx ",    " type ": " 2 "  }]page = {" Packetdata ": info} Page = json.dumps (page) url =  ' http://0.0.0.0:7000/api_recept_json_data/' data =  Urllib.urlencode (page) Req&nbsP;= urllib2. Request (url,data,headers) fd = urllib2.urlopen (req,data) Print fd.read ()

Then run this code, found that if the data format is correct, the receiving data is successful, and the database will generate corresponding records, then the test to fix the bug, increase the exception handling mechanism and so on, so that it can handle a variety of data during the transfer of the exception. Then I went to the manager to report, the demo, the manager saw me after the presentation, and then told me that the test code is a bit unprofessional, asked me to understand the curl command. Then find information on the Internet to learn, and found that it is really a powerful command, the individual feel that every web backend developers should have to master it.

The Curl command can be thought of as a command-line browser that makes a network request and then gets and submits the data, displayed on the standard output, and supports multiple protocols.

(1) View Web source

Directly after the Curl command add a URL, you can see the Web source. We use URL http://www.baidu.com as an example,

Curl http://www.baidu.com

If you want to save this page, you can use the-o parameter, Curl-o [filename] URL

If the file does not exist, it is created automatically, and if you want to get multiple files simultaneously, you can use the command

Curl-o [file name 1] url-o [filename 2] URL

-o parameter, saved to local using the default file name in the URL

(2) Auto jump

Some URLs are automatically redirected. With the-l parameter, curl jumps to the new URL.

For example, some URLs, only login to access some of the page content, this time with this parameter, will jump directly to the landing page, and then get the source of the landing page

(3) Display header information

The-I parameter can display the header information of the HTTP response, along with the page code.

-I parameter displays only the header information

(4) Display communication process

The-v parameter can display the entire process of an HTTP communication, including port connections and HTTP request header information.

If you think the above information is not enough, then the following command can see a more detailed communication process.

Curl--trace output.txt URL

Or

Curl--trace-ascii output.txt URL

After you run, open the Output.txt file to view.

(5) Send form information

Send form information with Get and post two methods. The Get method is relatively simple, as long as the data is attached to the URL behind the line.

Curl Tvbs.cc/form.cgi?data=xxx

The Post method must separate the data from the URL, and curl will use the--data parameter.

For example, the above examples can be ordered as follows

Curl--data "packetage=[{" id ": 554888," code ":" 85317 "," PartNo ":" XXX "," origin ":" XXX "," product_name ":" XXX "," type ":" 2 "},{" id ": 554113," Hs_code ":" 8230099 "," PartNo ":" XXX "," origin ":" XXX "," product_name ":" XXX "," type ":" 2 "}]" http/ 0.0.0.0:7000/api_recept_json_data/

If your data is not encoded by the form, you can also have curl encode it for you, with the parameter--data-urlencode.

If the interface requires authorization to access it requires the-u option to provide a user name and password for authorization

Curl-u Username:password URL

(6) File upload

You can upload files with curl:

Curl--form [email protected]--form Press=ok [URL]

(7) Referer Field

Sometimes you need to provide a referer field in the HTTP request header to indicate where you are jumping from.

Curl--referer URL

(8) User Agent Field

This field is used to represent the device information for the client. Depending on the field, the server sometimes returns pages of different formats, such as mobile and desktop versions, for different devices.

For example, the following user agent is

' Mozilla/4.0 (compatible; MSIE 5.5; Windows NT) '

Curl can simulate this:

Curl--user-agent "[user agent]" [URL]

(9) Cookies

Using the--cookie parameter, you can let curl send a cookie.

Curl--cookie "Name=xxx" URL

As for the value of the specific cookie, it can be obtained from the Set-cookie field of the HTTP response header information.

(10) Add header information

Sometimes it is necessary to add a header message to the HTTP request itself. The--header parameter can play this role.

Curl--header "Xxx:xxxxxx" URL

Xi. HTTP Authentication

Some domains require HTTP authentication, and curl needs to use the--user/-u parameter, as mentioned earlier.

Curl--use/-ur Name:password URL

Use of the Curl command