Basic tutorial on how to use the Request module to process HTTP requests in Node. js, node. jsrequest

Basic tutorial on how to use the Request module to process HTTP requests in Node. js, node. jsrequest

Here we will introduce a Node. js module -- request. With this module, http requests become super simple.

The Request is simple and supports both https and redirection.

var request = require ('request');
request ('http://www.google.com', function (error, response, body) {
 if (! error && response.statusCode == 200) {
 console.log (body) // print google homepage
}
})
flow:

Any response can be output to the file stream.

request ('http://google.com/doodle.png') .pipe (fs.createWriteStream ('doodle.png'))
Conversely, you can also pass the file to a PUT or POST request. If no header is provided, the file suffix name will be detected and the corresponding content-type will be set in the PUT request.

fs.createReadStream ('file.json') .pipe (request.put ('http://mysite.com/obj.json'))
The request can also be piped to yourself. In this case, the original content-type and content-length will be retained.

request.get ('http://google.com/img.png') .pipe (request.put ('http://mysite.com/img.png'))
Form:

The request supports application / x-www-form-urlencoded and multipart / form-data to realize form upload.

x-www-form-urlencoded is very simple:

request.post ('http://service.com/upload', {form: {key: 'value'}})
or:

request.post ('http://service.com/upload') .form ({key: 'value'})
Use multipart / form-data without worrying about setting up headers and other trivial matters, the request will help you solve it.

var r = request.post ('http://service.com/upload')
var form = r.form ()
form.append ('my_field', 'my_value')
form.append ('my_buffer', new Buffer ([1, 2, 3]))
form.append ('my_file', fs.createReadStream (path.join (__ dirname, 'doodle.png'))
form.append ('remote_file', request ('http://google.com/doodle.png'))
HTTP authentication:

request.get ('http://some.server.com/') .auth ('username', 'password', false);
or

request.get ('http://some.server.com/', {
 'auth': {
 'user': 'username',
 'pass': 'password',
 'sendImmediately': false
}
});
sendImmediately, true by default, sends a basic authentication header. After it is set to false, it will retry after receiving 401 (the server's 401 response must include WWW-Authenticate to specify the authentication method).

Digest authentication is supported when sendImmediately is true.

OAuth login:

// Twitter OAuth
var qs = require ('querystring')
 , oauth =
 {callback: 'http://mysite.com/callback/'
 , consumer_key: CONSUMER_KEY
 , consumer_secret: CONSUMER_SECRET
}
 , url = 'https://api.twitter.com/oauth/request_token'
;
request.post ({url: url, oauth: oauth}, function (e, r, body) {
 // Ideally, you would take the body in the response
 // and construct a URL that a user clicks on (like a sign in button).
 // The verifier is only available in the response after a user has
 // verified with twitter that they are authorizing your app.
 var access_token = qs.parse (body)
 , oauth =
 {consumer_key: CONSUMER_KEY
 , consumer_secret: CONSUMER_SECRET
 , token: access_token.oauth_token
 , verifier: access_token.oauth_verifier
}
 , url = 'https://api.twitter.com/oauth/access_token'
;
 request.post ({url: url, oauth: oauth}, function (e, r, body) {
 var perm_token = qs.parse (body)
 , oauth =
 {consumer_key: CONSUMER_KEY
 , consumer_secret: CONSUMER_SECRET
 , token: perm_token.oauth_token
 , token_secret: perm_token.oauth_token_secret
}
 , url = 'https://api.twitter.com/1/users/show.json?'
 , params =
 {screen_name: perm_token.screen_name
 , user_id: perm_token.user_id
}
;
 url + = qs.stringify (params)
 request.get ({url: url, oauth: oauth, json: true}, function (e, r, user) {
console.log (user)
})
})
})
Custom HTTP header

User-Agent and the like can be set in the options object. In the following example, we call the github API to find the number of collections and derived numbers of a warehouse. We used customized User-Agent and https.

var request = require ('request');

var options = {
 url: 'https://api.github.com/repos/mikeal/request',
 headers: {
 'User-Agent': 'request'
}
};

function callback (error, response, body) {
 if (! error && response.statusCode == 200) {
 var info = JSON.parse (body);
 console.log (info.stargazers_count + "Stars");
 console.log (info.forks_count + "Forks");
}
}

request (options, callback);

cookies:

By default, cookies are disabled. Set jar to true in defaults or options so that subsequent requests will use cookies.

var request = request.defaults ({jar: true})
request ('http://www.google.com', function () {
request ('http://images.google.com')
})
By creating a new instance of request.jar (), you can use a custom cookie instead of the request global cookie jar.

var j = request.jar ()
var request = request.defaults ({jar: j})
request ('http://www.google.com', function () {
request ('http://images.google.com')
})
or

var j = request.jar ()
var cookie = request.cookie ('your_cookie_here')
j.setCookie (cookie, uri, function (err, cookie) {})
request ({url: 'http://www.google.com', jar: j}, function () {
request ('http://images.google.com')
})
Note that setCookie requires at least three parameters, the last one is the callback function.

The pipe method of the request can be used to easily obtain the file stream of the picture

 var request = require ('request'),
 fs = require ('fs');
 
 request ('https://www.google.com.hk/images/srpr/logo3w.png') .pipe (fs.createWriteStream ('doodle.png'));
More usage methods and instructions can be read here: https://github.com/mikeal/request/

Examples

Here is a very simple example, used to grab the hotel query data of Qunar.com (to obtain the price ranking of each room type within a certain period of time):

  

 var request = require ('request'),
 fs = require ('fs');
 
 
 var reqUrl = 'http://hotel.qunar.com/price/detail.jsp?fromDate=2012-08-18&toDate=2012-08-19&cityurl=shanghai_city&HotelSEQ=shanghai_city_2856&cn=5';
 
 request ({uri: reqUrl}, function (err, response, body) {
 
 //console.log(response.statusCode);
 //console.log(response);
 
 // If the amount of data is relatively large, you need to store the returned data according to the date and hotel ID. If you get the data for comparison, read the file directly
 var filePath = __dirname + '/data/data.js';
 
 if (fs.exists (filePath)) {
  fs.unlinkSync (filePath);
 
  console.log ('Del file' + filePath);
 }
 
 fs.writeFile (filePath, body, 'utf8', function (err) {
  if (err) {
  throw err;
  }
 
  console.log ('Save' + filePath + 'ok ~');
 });
 
 console.log ('Fetch' + reqUrl + 'ok ~');
 });
 

This example originates from a friend who is in the hotel business and wants to know the competitiveness of the prices he offers to customers on the website:

1. If the price provided is too low, you will earn less money, so if your price is the lowest, you need to look at the second lowest, and then decide whether to adjust;

2. If the price provided is too high, then the search results will be relatively low, no customers come to book the hotel, the business will be gone

 

Because there are many hotel booking services, such as more than 2,000 hotels, if you rely on humans to query rankings one by one, it is more passive, and it is difficult to make it bigger, so I analyzed his needs and it is feasible A good real-time early warning system (of course the data will be automatically refreshed on the page every 5 to 10 minutes). Only in this way can we maximize profits, improve the efficiency of sales and customer departments, and accelerate the number of hotel cooperations and the expansion of the company's personnel:

1. Don't lose money, don't buy or sell at a loss;

2. If you find that the price provided is too low or too high, you need to support calling the platform's api interface and directly modify the price;

3. It has the function of automatically generating analysis reports to analyze the changes of competitors in price adjustment strategies;

Articles you may be interested in:
Node.js basic module http, webpage analysis tool cherrio realize crawler
Tutorial for implementing HTTP 206 content fragmentation using Node.js
How to use HTTP to upload files in Node.js
Instructions for using http.createClient method in node.js
Instructions for using http.response.addTrailers method in node.js
Instructions for using http.response.getHeader method in node.js
Instructions for using http.response.write method in node.js
Instructions for using http.response.removeHeader method in node.js
Instructions for using http.request.end method in node.js
Node.js generates HttpStatusCode auxiliary class and publishes it to npm