Diagram: The core principle of HTTP range request, assisted breakpoint continuation, multi-threaded download

Map: by Charles Loyer

First, the order

Hi, everyone, I am the Incense ink shadow!

The HTTP protocol occupies an important place in the network knowledge, the most basic of the HTTP protocol is the request and the response message, and the message is composed of the header and the entity. Most HTTP protocols are used in a way that relies on the Header of different HTTP request/Response settings.

This series of "practical http" put aside the conventional Header explanation of the way, from the actual problem, to analyze the use of these HTTP protocols, in the end is to solve what problem? At the same time explain how it is designed and its implementation principle.

The HTTP protocol is a stateless "loose protocol" that does not record the state of different requests, and because it itself contains both ends (client and server), which are differentiated according to the request and response, most of it is just a suggestion, in fact, the bilateral can not comply with this recommendation.

"Here's a suggested retail price of $2 ..."

"Oh, don't accept suggestions!" ”

The text is the fifth in this series, the first four portals:

• Caching mechanisms for HTTP
• HTTP content entity encoding compression mechanism
• HTTP Transport encoding compression mechanism
• Cookie Details for HTTP

Let's introduce the range request for HTTP today. A range request is primarily a request or upload for a larger file, and you can manipulate only one segment of it.

A more common scenario is the continuation of the breakpoint/download, when the network situation is not good, you can disconnect, only continue to get some content. For example, downloading software on the Internet, has downloaded 95%, when the network is broken, if not support the scope of the request, it is only forced to start the download. However, if there is a scope request for the addition, only the last 5% of the resources should be downloaded to avoid re-download.

Another scenario is multi-threaded download, to large files, open multiple threads, each thread download one of them, and finally after the download is complete, the local stitching into a complete file, can be more efficient use of resources.

This is two more common scenarios, and let's look at the technical details supported by the HTTP protocol of the scope request.

Second, the range of HTTP request 2.1 whether the range request is supported

HTTP itself is a stateless "loose" protocol that supports scope requests only on http/1.1 (RFC2616) After many versions of iterations have passed. So if either side of the client or server is below http/1.1, we should not use the function of the scope request.

In http/1.1, it is clear that a response header is declared Access-Ranges to support a range request, and it has only one optional parameter bytes .

For example, given a MP4 response header, you can see that it is Accept-Ranges:bytes tagged and has this tag to identify the current resource support scope request.

2.2 Use Range Request

If you have determined that both sides support the scope request, we can use it when requesting the resource.

All files are ultimately bytes stored on disk or in memory, which can be split in bytes for the file to be manipulated. This only requires HTTP support to request the file from N to n+x in this range of resources, you can implement the scope of the request.

A Ranges request header is defined in http/1.1 to specify the scope of the requested entity. Its range value is 0 - Content-Length between, using - split:

For example, you have downloaded the contents of the bytes resources, and want to continue to download the contents of the resources, as long as the HTTP request header, increase Ranges:bytes=1000- it.

Range also has several different ways to limit the scope and can be flexibly customized as needed:

1. 500-1000: Specifies the start and end ranges, which are typically used for multi-threaded downloads.

2. 500-: Specifies the start interval, which is passed to the end. This is more suitable for the continuation of the breakpoint, or online playback and so on.

3. -500: No start interval, meaning only the content entity that requires the last bytes.

4. 100-300, 1000-3000: Specify more than one range, this way the use of the scene is very few, to understand the better.

The HTTP protocol is a bilateral negotiated protocol, and since the request header has been determined to be using Ranges, there is also a response header that needs to be used Content-Ragne to mark the response's entity content range.

Content-RangeThe format is also very clear, first marks its unit is bytes and then marks the current passed content entity range and total length.

Content-Range: bytes 100-999/1000

In this example, a content entity with a range of 100 to 999 is passed, and the total size of the resource file is bytes. And the HTTP response status code at this time is 206 Partial Content .

The HTTP 206 Partial Content Success Status Response code indicates that the request was successful and that the principal contains the requested data interval, which is specified at the request's Range header.

For explanations of the 206 status codes, refer to: https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Status/206

So a normal process should look like the following:

Note that the response header in each HTTP transaction here is included Content-Length , except that it contains the content entity length of the current range request response, not the full length of the resource.

Here is basically the correct flow of HTTP range requests, and then look at some special situations.

2.3 Resource Changes

When we download large-size resources in some download tools, occasionally pause and re-download, you may encounter it and start downloading again.

This appears to be an HTTP range request that is not valid, but it is not necessarily the case, probably because the requested resource has changed in the process of the request.

If you download the process, the download source resource file has changed, but the URL has not changed, at this time the length of the file may have changed (it is very easy to find), in extreme cases, even if there is no length changes, you continue to download, it is likely that after the final download is complete, The downloaded content cannot be stitched into the file we need.

If we need to download a resource from the server, be sure to prevent possible changes to this resource. As mentioned in the previous HTTP cache, in the HTTP protocol, the ETAG or last-modified can be used to identify whether the current resource changes.

• ETag: A validation token thumbprint of the current file that identifies the uniqueness of the file.
• Last-modified: Marks the time at which the current file was last modified.

In a range of HTTP requests, you can also use these two fields to differentiate between the resource of a segmented request, whether it has been modified, just in the request header, put it in If-Range the request packet header. If-Rangeuse ETag either or Last-Modified two of the parameters, and fill them in as they are.

At this point, if the two operations are the same resource file, will continue to return 206 status code, start the subsequent operation, the reverse will return 200 status code, indicating that the file has changed, to download from scratch.

It is necessary to pay attention to the If-Range need and Range use, otherwise it will be ignored by the server.

In addition, if a client requests a message header, the range is incorrectly populated, and a 416 status code is returned.

The HTTP 416 Range not satisfiable error status code means that the server cannot process the requested data range. The most common scenario is that the requested data interval is not within the file range, that is, the Range header value, although syntactically correct, is meaningless in semantics.

For 416 status codes, refer to: https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Status/416

Iii. examples of scope requests 3.1 play an adapter with Chrome

The concept described earlier, many of the technical points are actually described as a request fragment, and then we have a practical example to illustrate the specifics of the scope request.

In this example, I've looked for a video's playback address and played it directly in Chrome. After normal playback, then drag the video progress, then no action to automatically play for a period of time, look at the HTTP transaction messages.

A brief description of the situation, the natural play, will first want the URL of the resource to send a request, return 200 of the response code, you can determine the current resource support Accept-Ranges , the next will be used to Range send a range of requests, the resulting response code is 206, and return the corresponding scope of the entity content. Each time the progress is dragged, it will resend a range request and calculate the request scope according to the progress of the drag. There is no situation where resources are modified, so there is no re-request download.

Not one for the HTTP transaction, presumably abstracted the process, as shown in:

As you can see, a resource download actually contains many requests, and we need to look at it from a global perspective.

Iv. Summary of the scope request

Here we have explained the entire process of the HTTP range request.

and re-organize the key points:

1. HTTP range request, requires http/1.1 and above support, if the double end of a paragraph below this version, it is considered not supported.

2. Accept-Ranges Determine whether the scope request is supported by the response header.

3. Range Specify the byte range of the requested content entity by adding the request header to the request header.

4. in the response header, Content-Range identify the currently returned content entity range by using Content-length to identify the current returned content entity range length.

5. during the request process, it is possible If-Range to differentiate whether the resource file is changed or not, and its value comes from the ETag or last-modifled. If the resource file changes, the download process will be re-run.

With a flowchart, it's clearer.

All key technical points requested by this HTTP range have been explained clearly. Scope requests are used in scenarios such as: Breakpoint continuation, multi-threaded download, and most of the resources on the CDN are to support the scope of the request, specifically you can use in what scenario, it depends on your imagination.

What more ideas, welcome message discussion.

Public number back to growth " growth ", will be prepared by my study materials, can also reply to " Dabigatran ", learning progress together, you can also reply to " ask questions " and ask me questions.

Recommended reading:

Android P Adaptation Experience | Technology Entrepreneurship Selection Checklist | HTTP Transfer Encoding | What is consuming you? | HTTP Content Encoding | Schematic HTTP Cache | Chat About HTTP Cookies | Auxiliary Mode Combat | Accessibility Auxiliary Mode | Small Program Flex Layout | Good PR makes you more reliable | The way of password management

Diagram: The core principle of HTTP range request, assisted breakpoint continuation, multi-threaded download