HTTP File transfer

When the HTTP protocol is used for file transfer, the file content is generally placed in the message body. As a streaming protocol over TCP, the sending and receiving sides can stream and receive large files.

1. Determining the size of file transfers

The Content-length field of the message header represents the length of the file used to determine the end of the file for the receiving end.

2. Chunked encoding

The Content-length field cannot be set when the file size cannot be determined in advance. In this case, the file can be sent in multiple sections by means of a chunked transfer. In the block send mode, the head increases transfer-encoding:chunked, the presence of this head is not allowed to add content-length head, even if there is also ignored.

chunked mode, the message body chunked send, each piece of head to store data length, keep up with CRLF, then the specific data, block and block is also CRLF separated. When the length header is 0 o'clock, the end of the block is indicated.

3. Uploading files using Multipart/form-data

The original POST request message body is a URL-encoded form, formatted as Key=value, with & separated by different keys and value. When uploading a binary file, you can use the Multipart/form-data method.

In this way, the underlying request is still a POST request, the file content is placed in the body of the message, except that the value of the Content-type field is Multipart/form-data, and randomly selects a string as the delimiter (this delimiter is theoretically required to not appear in the file contents, Generally randomly selected strings appear in the body of the probability is very small, if the real will cause post failure, you need to initiate another request to re-select the random string), and then, each field is separated by a "--delimiter", the last "--delimiter--" indicates the end. Each field can contain the head and message body, and the contents of the header can include the file name, file path, and so on, or the binary contents of the file itself.

4. Breakpoint continuation and multi-threaded transmission

This is still the idea of using chunking: If the transmission is interrupted, the next step is to start again from where it was interrupted, avoiding the waste from scratch; In a multithreaded program, each thread is responsible for transferring a block of files, and then merging them back into the original file.

Chunked transfers need to determine the bounds of the block, which is used in the range field, which means that starting from a byte, such as range:bytes=100-, the request is from 100 bytes of the file to the end of the file, the return message is 206 Partial Content, The header field increases content-range:bytes100-199/200, which means that the file returns 100-199 bytes of content, and the file is 200 bytes.

When multithreading is transferred, each thread requests a different range in the file, which is then merged by the application when the transfer is complete.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

HTTP File transfer