HTTP 1.1 Learning Notes

Objective

Due to the limitations of HTTP 1 itself, it is not good for users to provide a good performance Web services. The HTTP1.1 standard REC2616 was formally released in June 1999, which clarified many ambiguities in previous versions, and added many important optimizations such as persistent connections, chunked encoding transmission, status code expansion, enhanced caching mechanisms, transmission coding, and request pipelines. This article is an individual in the study "Web Performance Authoritative guide", and looked up some of the documents written in an essay, only for reference and personal later review. The following is a comparison of some important improvements in the new version of http1.0. (This article was originally published in the company intranet, the original address: Tengyun --http 1.1 study notes )

1. Persistent connection

Each TCP connection requires three handshakes at the beginning of the establishment, a complete round trip between the client and the server, and at least one additional round trip if data transfer is required. Plus the server-side processing request time is the total time that can be obtained for each request. If each send request is sent by a new TCP connection, it will require a minimum of two full network round-trip times, as each TCP connection will need to be re-established three times.

So, can you reuse a TCP connection?

The answer is no doubt, HTTP1.1 added support for persistent connection, when sending the request again, you can directly use the last established TCP connection, so as to avoid the second TCP connection three times handshake, eliminate another TCP slow start round trip, greatly reducing the network delay. If you reuse a CTP connection, the more times you send an HTTP request, the greater the performance boost, because after the first three handshakes have been established, there is no need to spend extra time establishing the connection.

Note: Currently, all modern browsers try to persist HTTP connections if the server supports them. Using HTTP1.1, the default is persistent connection, if you use HTTP 1.0, you can explicitly use the Connection:keep-alive header declaration using persistent connection, you should also pay attention to the default behavior of some HTTP libraries and frameworks, they sometimes do not use persistent connection by default.

2.HTTP Pipe

Persistent HTTP allows us to reuse an existing connection to complete multiple requests, but these requests need to satisfy the FIFO queue order: Send the request-wait for the response, and then send the next request. http/1.1 allows multiple HTTP requests to be output at the same time via a socket without waiting for a corresponding response. The requestor then waits for their respective responses, which arrive in the order in which they were previously requested. (all requests remain in a FIFO queue, and after a request has been sent, it is not necessary to wait for the response of the request to be accepted, the next request can be sent again, and the server returns the response of those requests in the FIFO order).

In high-latency and multi-request scenarios, data transfer over the HTTP pipeline will have a greater performance boost and will save more time. After careful observation, It can be found that there are some limitations in the HTTP1.1, it strictly serial return response, it does not allow multiple data interleaving (multiplexing), can only wait for a response to fully return after the next response can be sent, regardless of whether the next response is earlier than the previous response to complete processing, this is called the first queue block. This leads to a very bad experience, if the first request has to be processed for a very long time, then the response is not returned immediately even after the server has processed it, but it is stored in the service's cache and waits for the first response to complete before it can be returned in FIFO order.

Because TCP is delivered strictly in order, losing a TCP packet blocks all high-numbered groupings, which can cause additional delays unless the missing packets are re-transmitted. Because multiplexing is not allowed in the HTTP1.1, the HTTP pipeline also brings some issues that cannot be overlooked:

• A slow response will block all subsequent requests;

• When the request is processed in parallel, the server needs to cache the processing result, it consumes the server resources, if a response is very large, it can easily form the attack surface of the server;

• A response failure may terminate the TCP connection, forcing the client to resend requests for subsequent resources, resulting in repeated processing;

• Intermediary agent exists in network environment, it is necessary to detect pipeline compatibility.

• If the intermediary agent does not support the pipeline, it may break the connection, or it may concatenate all requests together.

It is because of such or such problems, the application of HTTP pipeline technology is relatively limited, and there is no large area spread out, even if some browsers support it just as an advanced option. If you have strong control over both the client and the server, you can still use it, and it will bring a good performance boost. If you are using this technique, you need to be aware of the following:

• Ensure that the HTTP client supports the pipeline;

• Ensure that the HTTP server supports the pipeline;

• The application must handle the interrupted connection and recover;

• The application must handle the problem of the power of the interrupt request;

• The app must keep itself unaffected by the offending agent.

The best way to deploy HTTP pipelines in practice is to use HTTPS on both the client and server side, which eliminates the interference of intermediate agents that do not support pipelines.

3. Enhanced caching mechanism

In http/1.0, use the Expire header field to determine the fresh or stale of a resource and use a conditional request (conditional request) to determine whether the resource is still valid.
Related fields:

• Date: The time the server responded;

• Expires: Resource expiration time;

• Last-modified: Resource last modified time;

• If-modified-since: Used to verify that resources are out of date;

The approximate strategy is:

If expires is set for a time after date, the browser will not access the server until the time expires is marked, and the browser cache is used, if the expires is set either before date or after the browser time is expires. Then when the resource is accessed again, the browser sends the request to the server, but instead of pulling the data again, it asks the server if the resource is out of date, and when the method response the last last-modified time as if-modified-since, sends the request, The server compares the time and the current change time of the resource, or returns 304 if it has not changed, otherwise transmits the new file. In addition, the Pragma:no-cache header domain is also defined in http/1.0, which the client uses to indicate that the request resource cannot be fetched from the cache and must be fetched back to the source.

http/1.0, the if-modified-since header domain uses an absolute timestamp, accurate to the second, but the use of absolute time will bring the clock synchronization problem on different machines, the document update period is less than 1s, there will be problems.

In order to provide a better caching mechanism, HTTP1.1 has made a gradual improvement to the previous mechanism. The caching mechanism advocated by http/1.1 is to compare the hash value of the document, the content of the document, then the hash, and replace the absolute time with the relative time, so that some problems can be solved by using the absolute timestamp.

http/1.1 inherits http/1.0 so the related fields of http/1.0 are still valid, and these fields are reserved for compatibility with clients that only support http/1.0. http/1.1 server should not set the expiration policy with 1.0 contradiction, 1.1 of the server when there is no document hash value, you can also use If-modified-since to determine the document expires.
The following fields are added in HTTP1.1:

• Cache-control: Used to control the browser's caching behavior;

• ETag: The hash value of the document;

• If-none-match: Used to verify whether a resource has expired, that is, whether the document hash value changes.

In http/1.1, an ETag header domain is introduced for the reactivation mechanism, and its value etag can be used to uniquely describe a resource. The If-none-match header field can be used in the request message to match whether the entitytag of the resource has changed.

To make the caching mechanism more flexible, http/1.1 increases the Cache-control header domain (both request and response messages are available), which supports an extensible subset of instructions: for example, max-age directives support relative timestamps The private and No-store directives prohibit the object from being cached; no-transform prevents Proxy from making any behavior that alters the response; No-cache the browser cache, but is considered an expired cache; No-store Browser does not cache ; Max-age: Cache valid time periods, and so on.

If you want the browser to send requests every time, and also enable caching, then use Cache-control:no-cache, each time you access the image, the browser will verify the ETag.

The cache uses the keyword index for objects cached on disk, using the URL of the resource as the keyword in http/1.0. However, there may be different resources based on the same URL, and to differentiate them requires more information from the client, such as the Accept-language and Accept-charset header domains. To support this content negotiation mechanism, http/1.1 introduced a Vary header field in the response message that lists which header fields are required for content negotiation in the request message.

4. chunked Encoding Transmission

HTTP messages can contain entities of any length, usually using content-length to give a message end flag. However, for many dynamically generated responses, the size of the message can only be judged by buffering the complete message, but doing so increases the latency. If you do not use a long connection, you can also determine the end of a message by connecting the closed signal.

http/1.1 introduced the Chunke dtransfer-coding to solve the above problem, the sender divides the message into a number of arbitrary-sized chunks of data, each data block is sent with the length of the block, and finally a 0-length block as the message end of the flag. This method allows the sender to buffer only one fragment of the message, avoiding overloading the entire message.

In http/1.0, there is a CONTENT-MD5 header field to calculate that the header field needs to be buffered by the sender to complete the message. In http/1.1, a message with a chunked chunk is passed after the last block (0-length), and then a trailing (trailer), which contains one or more header fields that the sender calculates after passing all the blocks. The sender contains a trailer header field in the message that tells the receiver that the trailer is there.

5. Status Code expansion

A status code is an integer that attempts to understand and satisfy the requested three-digit number. Only 16 status response codes are defined in http/1.0, which is not specific enough to prompt for errors or warnings. http/1.1 introduces a warning header field that adds a description of the error or warning message.

The status codes that are included in the HTTP1.1 are roughly the following:

Status Code status information meaning

Message (1**)

Continue the initial request has been accepted and the customer should continue to send the remainder of the request. (HTTP 1.1 new)
101 Switching Protocols Server translates the client's request to another protocol (HTTP 1.1 new)

Success (2**)

All OK, the answer document for Get and post requests is followed.
The 201 Created Server has created the document, and the location header gives its URL.
202 Accepted has accepted the request, but the processing has not been completed.
203 Non-authoritative Information The document has returned normally, but some of the answer headers may be incorrect because a copy of the document (HTTP 1.1 new) is being used.
204 No Content There is no new document, the browser should continue to display the original document. This status code is useful if the user refreshes the page on a regular basis and the servlet can determine that the user's document is new enough.
205 Reset content is not new, but the browser should reset what it displays. Used to force the browser to clear the form input (HTTP 1.1 new).
The 206 Partial Content client sends a GET request with a range header that the server has completed (HTTP 1.1 new).

Redirect (3**)

Multiple Choices customer requested documents can be found in multiple locations that have been listed in the returned document. If the server wants to make a preference, it should be indicated in the location answer header.
301 Moved Permanently The document requested by the customer elsewhere, the new URL is given in the location header, and the browser should automatically access the new URL.
302 Found is similar to 301, but the new URL should be treated as a temporary replacement, not a permanent. Note that the corresponding status information in HTTP1.0 is "Moved temporatily".
When the status code appears, the browser can automatically access the new URL, so it is a useful status code.

Note that this status code can sometimes be used with 301 substitutions. For example, if the browser mistakenly requests Http://host/~user (the trailing slash is missing), some servers return 301, and some return 302.

Strictly speaking, we can only assume that the browser will automatically redirect only if the original request is get. See 307.
303 See other is similar to 301/302, except that if the original request is the Post,location header the specified redirect target document should be fetched via get (HTTP 1.1 new).
304 Not Modified client has buffered documents and issued a conditional request (typically providing a if-modified-since header indicating that the customer only wants to update the document than the specified date). The server tells the customer that the original buffered document can continue to be used.
305 Use proxy The document requested by the client should be extracted from the agent server indicated by the location header (HTTP 1.1 is new).
307 Temporary Redirect and 302 (Found) are the same. Many browsers incorrectly respond to a 302 response for redirection, even if the original request is post, even though it can actually be redirected only if the answer to the POST request is 303. For this reason, HTTP 1.1 has been added in 307 to allow for more cleanup of the region in several status codes: When a 303 response occurs, the browser can follow the redirected get and post requests, and if the 307 answer, the browser can only follow the redirect to the GET request. (HTTP 1.1 new)

Request Error (4**)

A syntax error has occurred in the request requests.
401 Unauthorized customer attempts to access password-protected pages without authorization. A www-authenticate header is included in the answer, and the browser displays the user name/Password dialog box, and then makes a request again after filling in the appropriate authorization header.
403 Forbidden resource is not available. The server understands the customer's request, but refuses to process it. This is usually caused by the permissions set on the file or directory on the server.
404 Not Found Could not find the resource at the specified location. This is also a common answer.
The 405 method not allowed request methods (GET, POST, HEAD, DELETE, PUT, trace, etc.) do not apply to the specified resource. (HTTP 1.1 new)
406 not acceptable the specified resource has been found, but its MIME type is incompatible with the client specified in the Accpet header (HTTP 1.1 new).
407 Proxy authentication Required is similar to 401, which means that the client must be authorized by the proxy server first. (HTTP 1.1 new)
408 Request Timeout The customer has not made any requests during the waiting time of the server license. Customers can repeat the same request at a later time. (HTTP 1.1 new)
409 Conflict is usually associated with a put request. The request cannot succeed because the request conflicts with the current state of the resource. (HTTP 1.1 new)
410 Gone The requested document is no longer available, and the server does not know which address to redirect to. It differs from 404 in that returning 407 means that the document has permanently left the specified location, and 404 indicates that the document is unavailable for unknown reasons. (HTTP 1.1 new)
411 Length Required The server cannot process the request unless the client sends a content-length header. (HTTP 1.1 new)
412 Precondition Failed Some of the prerequisites specified in the request header failed (HTTP 1.1 new).
413 Request Entity Too Large the size of the target document exceeds the size that the server is currently willing to handle. If the server thinks it can process the request later, it should provide a Retry-after header (HTTP 1.1 new).
414 Request Uri Too Long URI is too lengthy (HTTP 1.1 new).
416 requested range not satisfiable server does not meet the Range header specified by the customer in the request. (HTTP 1.1 new)

Server error (5**)

The Internal server Error server encountered unexpected conditions and was unable to complete the customer's request.
The 501 not implemented server does not support the functionality required to implement the request. For example, a customer sends a put request that is not supported by the server.
502 Bad Gateway server as a gateway or proxy, the server returned an illegal reply in order to complete the request to access the next server.
503 Service Unavailable Server failed to answer due to maintenance or heavy load. For example, a servlet might return 503 if the database connection pool is full. A retry-after header can be supplied when the server returns 503.
504 Gateway Timeout is used by a server acting as a proxy or gateway, indicating that it is not able to get answers from remote servers in a timely manner. (HTTP 1.1 new)
505 HTTP version not supported server does not support HTTP versions as specified in the request. (HTTP 1.1 new)

6.Host header Field

In HTTP1.0, each server is considered to be bound to a unique IP address, so the URL in the request message does not have a host name (hostname). However, with the development of virtual host technology, multiple virtual hosts (multi-homed Web Servers) can exist on a physical server, and they share an IP address. Because HTTP 1.0 does not support host Request header fields, Web browsers cannot use host header names to explicitly indicate which Web site to access on the server, so you cannot use a Web server to configure multiple virtual Web sites on the same IP address and port number. After adding the host Request Header field in HTTP 1.1, the Web browser can use the host header name to explicitly indicate which Web site to access on the server, which enables multiple virtual Web sites to be created on a single Web server with a different hostname on the same IP address and port number.

7. New Request method

Requests sent to the server by the client program can have different types, so that the server can handle different types of requests depending on the type of request. In HTTP1.0, the three most basic types of requests, get, post, and head are defined, and when the Get and post methods are used, the server eventually returns the resulting document to the client, the browser refreshes the display, and the head request is different. The head request communicates between the client and the server without returning a specific document, which simply communicates some internal data that does not affect the browsing process. Therefore, the head method is usually not used alone, but is complemented with other request methods. Some search engines use the automated search robot to use this method to obtain information about a Web page's logo, or to use this method to pass authentication information when authenticating securely.

In addition to the three most common access methods described above, more types of access methods are defined in HTTP1.1, as follows:

• PUT: Uploads its latest content to the specified resource location;

• Delete: The request server deletes the resource identified by the Request-uri;

• OPTIONS: Returns the HTTP request method that the server supports for a specific resource. You can also test the functionality of the server by sending a ' * ' request to the Web server;

• TRACE: Echo the request received by the server, mainly for testing or diagnostics;

• The content:http/1.1 protocol is reserved for proxy servers that can change connections to pipelines.

These methods are not commonly used, so most Web server software does not implement them. However, they are useful for specific situations, and if the server does not support the request method sent by the client, the server will return an error and immediately close the connection.

8. Bandwidth Optimization

In http/1.0, there are some wasted bandwidth phenomena, such as the client just need a part of an object, and the server will send the entire object, and such as the download of large files need to support the extension of the breakpoint, instead of having to re-download the full package after the disconnection. The Range header field is introduced in the request message in http/1.1, which allows only a portion of the resource to be requested. The Content-range header field in the response message declares the offset value and length of the returned part of the object. If the server returns the contents of the requested scope of the object accordingly, the response code is 206 (Partial content), which prevents the cache from mistaking the response for a complete object.

Another case is if the request message contains a larger entity content, but is not sure whether the server can receive the request (if there is permission), at this time if the request with the entity rushed out, if rejected also wasted bandwidth. http/1.1 added a new status code of (Continue). The client sends a lead-only domain request, and if the server rejects the request because of the permission, the Echo Response code 401 (unauthorized) is sent back, and if the server receives this request, the client can continue to send the full request with the entity with the Echo Response code 100. Note that the http/1.0 client does not support 100 response codes. However, you can have the client join the expect header domain in the request message and set its value to 100-continue.

A very effective way to conserve bandwidth resources is to compress the data to be transferred. Content-encoding is an end-to-end (end-to-end) encoding of the message, which may be an intrinsic format (such as a JPEG picture format) that the resource holds on the server, and a accept-encoding header field in the request message. It can tell the server how the client can decode the encoding. And Transfer-encoding is a segment-by-section (hop-by-hop) encoding, such as chunked encoding. Join the TE header domain in the request message to tell the server how to receive the transfer-coding.

9. Performance Optimization 9.1 Using multiple TCP connections

As already stated above, HTTP1. X does not support multiplexing, requests need to be queued at the client, and is prone to problems with the first block of the queue, and does not improve the data rates well. So, since you can't multiplex a single connection, can you open multiple connections for data transfer at the same time? The answer is yes, browser developers in order to solve this problem, the browser support client to open up to six connections, so that we can communicate more quickly. Everything is two-sided, while opening multiple connections will inevitably bring some optimization and problems, as follows:
Advantages:

• Clients can initiate multiple requests in parallel;

• The server can process multiple requests in parallel;

• The number of cumulative groups that can be sent for the first round trip is 6 times times the original;

Disadvantages:

• More sockets will take up more resources;

• The bandwidth of competing and sharing between parallel TCP streams;

• Processing multiple sockets, the implementation is more complex;

• Even with parallel TCP flows, the concurrency capabilities of the application are limited.

This way of opening multiple connections also brings some drawbacks, why is it so widely used now? Mainly by the following three reasons:

• As a stopgap for bypassing HTTP restrictions;

• As a workaround for bypassing TCP's low-initial congestion window;

• As a client bypasses an entitlement that cannot be scaled using TCP windows.

9.2 Domain partition

Since the HTTP1.1 protocol does not support multiplexing, forcing browser developers to introduce and maintain connection pooling in order to improve communication efficiency, each host can have 6 TCP streams. According to the HTTP archive statistics, the average page currently contains 90 or so of resources, if these resources are from the same host, even if you can open 6 TCP streams at the same time, will still lead to a noticeable queuing situation. We do not need to put all the resources on the same host, can be placed separately under different domain names, so that can increase the total number of TCP flows can be opened at the same time, can break through the browser's connection limit, achieve higher concurrency capability.

In theory, the more domain names are used, the stronger the parallel capability is. However, before sending a request for DNS resolution, different domain names need to be resolved separately, need to make additional DNS queries, if the number of domain names too many, will lead to a large number of additional parsing; the slow-start mechanism that exists in TCP connections can sometimes degrade performance , and each more socket requires the client and server to consume resources for maintenance, and worse, developers need to manually partition these resources into different domain names. The number of domain partitions is too large or too small to affect performance, but how to determine the optimal number of partitions is a good answer, because the number of resources in the page, the bandwidth available to the client connection, and the latency can affect the reasonableness of the number of partitions.

To determine the appropriate number of domain partitions, you can only use the most primitive way to start the test from the smallest partition, observe the effect of the number of partitions on the application, and then select the optimal value as the number of fixed partitions.

9.3 Connecting and merging

The quickest request is not to request, no matter what protocol or type of application, reduce the number of requests is always the best way to optimize. If each resource is essential, you might consider packing the resources together for a single network request. Connectivity and flattening technologies are content-centric application-layer optimizations that significantly improve performance by reducing network round-trip overhead. But these technologies also require additional processing, deployment, and coding, as well as additional complexity. Bundling multiple resources together can also put a burden on the cache, and some minor updates require resources to be re-requested and cached, and sometimes the current page does not require other resources in the file, which can affect the speed of the page execution.

While joining and merging technologies, while reducing network round-trip overhead and improving performance, it also increases the complexity of the application, resulting in problems such as caching, updating, execution speed, rendering of pages, and so on. Therefore, the use of this optimization, should be considered, to seek an optimal file packaging granularity.

9.4 Embedding Resources

Embedding resources is also a common optimization method, and embedding resources into documents can reduce the number of requests. The resources embedded in the page are especially small, rarely used, preferably once. In practice, one rule of thumb is to consider only embedding resources below 1-2kb, because resources smaller than this standard cause higher HTTP overhead than it does. However, if the embedded resource changes frequently, the invalid cache rate for the host document is raised. If you want to use small, individual files in your app, consider the following recommendations when considering embedding:

• If the file is small, and only the individual pages used, you can consider embedding;

• If the file is small but needs to be used on multiple pages, you should consider centralized packaging;

• If the file needs to be updated frequently, do not embed it;

• Minimize the protocol overhead by reducing the size of the HTTP cookie.

Resources:
The authoritative Guide to Web performance
"The difference between http/1.1 and http/1.0"
"HTTP 1.1 vs. HTTP 1.0"
Http://www.faqs.org/rfcs/rfc1945.html
Http://www.faqs.org/rfcs/rfc2616.html

HTTP 1.1 Learning Notes