Remarks: Vary response header in HTTP network protocol (1)

Remarks: Vary response header in HTTP network protocol (1)

Those who often capture packets and view HTTP requests should be familiar with the Vary response header field. What is its purpose? When you use the PageSpeed tool to check the page, you will often see the suggestion "Specify a Vary: Accept-Encoding header (please Specify a Vary: Accept-Encoding header)". Why? This article records some of my studies on Vary, including the answers to these questions.

HTTP content negotiation

To understand the role of Vary, you must first understand the HTTP content negotiation mechanism. Sometimes, the same URL can provide multiple different documents, which requires a mechanism to select the most appropriate version between the server and the client. This is content negotiation.

There are two ways to negotiate. One is that the server sends the list of available versions of the document to the client for selection. This can be achieved using the 300 Multiple Choices status code. This solution has many problems. First, there is one more network round-trip. Secondly, some versions of the same document on the server may be prepared for clients with certain technical characteristics, but ordinary users may not understand these details. For example, the server can usually output static resources to both compressed and uncompressed versions. The compressed version is obviously prepared to support the compressed client, but if you want to select a common user, it is very likely that the wrong version is selected.

Therefore, HTTP content negotiation usually uses another solution: the server automatically sends the most suitable version based on some fields in the request header sent by the client. The request header fields that can be used in this mechanism are divided into two types: The content negotiation special field (Accept field) and other fields.

The Accept field is shown in the following table:

Request Header field	Description	Response Header field
Accept	Notifies the server of the media type sent	Content-Type
Accept-Language	Tell the server which language to send	Content-Language
Accept-Charset	Tell the server which character set to send	Content-Type
Accept-Encoding	Inform the server of the compression method	Content-Encoding

For example, a client sends the following request headers:

Accept:*/*Accept-Encoding:gzip,deflate,sdchAccept-Language:zh-CN,en-US;q=0.8,en;q=0.6

It can accept resources of any MIME type. It supports resources compressed by gzip, deflate, or sdch. It can accept zh-CN, en-US, and en, and zh-CN has the highest weight (q is 0-1, the highest is 1, the lowest is 0, the default is 1). The server should first return the version with the language equal to zh-CN.

The browser's Response Header may be like this:

Content-Type: text/javascriptContent-Encoding: gzip

Indicates that the exact MIME type of this document is text/javascript. The document Content is compressed by gzip. The response header does not have the Content-Language field, it usually indicates that the Language of the returned version is exactly the one with the highest weight in the Request Header Accept-Language.

Sometimes, the above four Accept fields are not enough. For example, to output different content for a specific browser such as IE6, you need to use the User-Agent field in the request header. Similarly, the Cookie in the Request Header may be used by the server to output differentiated content.

Because one or more intermediate entities (such as the Cache Server) may exist between the client and the server, the most basic requirement of the cache service is to return the correct document to the user. If the server returns different content based on different User-agents, and the Cache Server caches the response of IE6 users and returns the response to users using other browsers, the problem will certainly occur.

Therefore, the HTTP protocol stipulates that if the content provided by the server depends on the request header field other than the "common Accept negotiation field" such as the User-Agent, the Response Header must contain the Vary Field, the Vary must contain the User-Agent. Similarly, if the server uses both the User-Agent and Cookie fields in the request header to generate content, the Vary Field in the response should look like this:

Vary: User-Agent, Cookie

That is to say, the Vary field is used to list a response field and tell the cache server how to cache and filter the appropriate version when the same URL corresponds to a document of different versions.

Cache service with bugs

Let's take a look at the "Specify a Vary: Accept-Encoding header" prompt in PageSpeed. According to the preceding instructions, Accept-Encoding is a special field for content negotiation, the server only needs to add the Content-Encoding field to the response header to specify the Content compression format. or, if the Content-Encoding is not output, it indicates that the Content has not been compressed. The Cache Server caches different Content based on different Content-Encoding and returns the most suitable version based on the Accept-Encoding field in the specific request.

However, some cache servers with bugs will ignore the Content-Encoding in the Response Header, which may return the cached compressed version to clients that do not support compression. There are two solutions to avoid this situation:

Set the Cache-Control field in the response header to private, telling the intermediate entity not to Cache it;

Add the Vary: Accept-Encoding response header to explicitly inform the cache server to cache different versions based on the content of the Accept-Encoding field;

We usually use the second method to better utilize the cache function of intermediate entities.

For static resources such as css and js, as long as the client supports gzip, the server should always enable it. To prevent the cache server with bugs from returning the wrong version to the user, the Vary should also be output: accept-Encoding.

Nginx and SPDY

Generally, the Web Server can help us with the above work. For Nginx, the following configuration can automatically add Vary: Accept-Encoding to the response with gzip enabled:

gzip_vary on;