Combinestream and Multipart Form

Recently began to study HTTP, especially the multipart form, trying to figure out how he did it. In Nodejs, you can use Form-data to assemble a multipart form, and then use Http.request to send it out

 var  formData = require (' form-data ' );  var  http = require (' http '  = new   FormData (); F.append ( ' image ', Fs.createreadstream ('./test.jpg ') Span style= "color: #000000;" >)); var  options = Urlparse (URL); options.method  = ' Post ' ;options.headers  = f.getheaders (); var  req = http.request (options); F.pipe (req);  

Post-past form content can be accepted on the server (node can use multiparty middleware)

Form-data can pipe to req, indicating that he is at least a readable stream, after reviewing the code found that he inherited from Combine-stream. Continue to view Combine-stream found this is a duplex stream, the feature is to accept multiple readable stream to string them up and then merge them into a readable stream. Very interesting.

Understand what Combine-stream is for, and then understand how the multipart form is made up of the difference between the other forms

1. The Content-type in the header.
2. The contents of the body are delimited by the defined boundary.

Finally found that the use of combine-stream to assemble the multipart form is a good choice. Because stream is a resource abstraction, the goal is to make a trade-off between speed and system resource consumption (imagine that if there is no stream, sending any file requires the entire file to be put into memory; If the resource is not fully produced, such as live streaming, there is no abstraction for the stream)

The Req of HTTP is a writeable stream (res also), and all requests can be sent past req.write (or res.write). How to send the details, is the socket? And then TCP? Do not care, stream this layer of abstraction shielding the underlying system. We only use care to read from the readable stream and write it to the writeable stream.

So when we send the multipart form, we only use:

1. Write the header in the Req and pay attention to the special content-type.
2. Pipe file stream toward req.
3. Write the Boundray delimiter into the req.
4. Repeat 2, 3 steps. Until all files are sent.
5. Write the tail message to the Req.

Can be found in 2, 32 steps involves a lot of file streams, such as uploading multiple images to the server, then we will go to the Req pipe first file (and then add Boundray) wait for it to end after the pipe next picture stream. It is troublesome to repeat many times. It's better to string all these streams together into a whole stream, so that the repetitive logic of switching between streams is packaged, and it's not easy to make mistakes and be more concise and understandable.

Combinestream how to achieve it. As I just learned, the logic of Combinestream is simply to string up a string of readable streams, and one stream ends immediately for another, until all the added streams are finished. Then the first combinestream must be duplex. The transform stream in node is the perfect candidate.

1 varTransform = require (' stream '). Transform;2 varUtil = require (' util '));3 varassert = require (' Assert ');4 varFS = require (' FS ');5 6 functionCombinestream (options) {7Transform.call ( This, options);8 9      This. _streams = [];Ten      This. _currentstream =NULL; One  A      This. _prepare =function(){ -         varstream = This. _currentstream = This. _streams.shift (); -         if(stream) { the             if(typeofstream = = = ' String ') { -                  This. push (stream); -                  This. _prepare (); -                 return; +             } -Stream.pipe ( This, {end:false}); +Stream.on (' End ',function(){ A                  This. _prepare (); at}.bind ( This)); -Stream.on (' Error ',function(err) { - Console.error (err); -             }); -}Else { -              This. End (); in         } -     }; to } + util.inherits (Combinestream, Transform); -  theCombineStream.prototype.append =function(stream) { *      This. _streams.push (stream); $ }Panax Notoginseng  -Combinestream.prototype._transform =function(chunk, encoding, callback) { theCallbackNULL, chunk); + } A  theCombineStream.prototype.pipe =function(dest, options) { +      This. _prepare (); -Transform.prototype.pipe.call ( This, dest, options); $ } $  -Module.exports = Combinestream;

This enables a simple combinestream. Only a little code is added: The Append method adds a stream, where the string is adapted, and the _prepare is used to implement the logic of the flow switch.

Then you can try Combinestream to construct a multipart form and send it.

varCS =NewCombinestream (); Cs.append ('-----------------------------287032381131322\r\ncontent-disposition:form-data; Name= "image"; Filename= "Test.jpg" \r\ncontent-type:image/jpg\r\n\r\n '); Cs.append (Gmstream); Cs.append (' \ r \ n-----------------------------287032381131322--');varOptions =urlparse (URL); Options.method= ' Post '; Options.headers= {    ' Keep-alive ': 300,     ' Content-type ': ' Multipart/form-data; boundary=---------------------------287032381131322 ',    ' transfer-encoding ': ' chunked '};varreq =http.request (options); Cs.pipe (req); Req.on (' Error ',function(Err) {console.error (err);}); Req.on (' Response ',function(res) {//deal res here.});

Here I hardcode the separation information and tail information, there are a lot of \ r \ n can see that there is no system of processing methods are prone to error. This way, if you use multiparty middleware on the server side to parse the form, you can get the correct file upload content.

{image:    ' image ',       ' test.jpg ',       '/var/folders/02/ Pwvm1df51nsfvg373jf8ksg00000gn/t/hwm6h-88fqiemxyna1hd09ek.jpg ',       headers: [Object],       10988}] }

Conclusion:

Visible Form-data modules are basically working like this.

Combinestream and Multipart Form