Web Cache basics: Terms, HTTP headers, and cache policies

Web Cache basics: Terms, HTTP headers, and cache policies
Introduction

For visitors to your site, intelligent content caching is one of the most effective ways to improve user experience. Caching, or temporary storage of previous requests, is one of the core content delivery policies in HTTP. All components in the distribution path can cache content to accelerate subsequent requests, which is controlled by the cache policy declared for this content.

In this guide, we will discuss some basic concepts about Web content caching. This mainly includes how to select a cache policy to ensure that the cache on the internet can process your content correctly. We will talk about the benefits, side effects, and the best combination of performance and flexibility that different policies can bring.

What is cache )?

Cache is a term used to describe the behavior of storing reusable resources to accelerate subsequent requests. There are many different types of caches, each of which has its own characteristics. Application cache and memory cache are frequently used because of their acceleration of specific replies.

This Guide describes the different types of Web cache. Web Cache is a core feature of HTTP protocol. It can minimize network traffic and improve the overall system response speed perceived by users. Cache can be found at each layer during the transmission of content from the server to the browser.

The Web Cache caches the response of the corresponding HTTP request according to specific rules. Subsequent requests to the cached content can be directly satisfied by the cache instead of resending the request to the Web server.

Benefits

Effective caching technology can not only help users, but also help content providers. Cache provides the following benefits for content delivery:

• Reduce network overhead: content can be cached in many different places from the content provider to the content consumer network path. When the content is cached closer to the content consumer, the request will not consume additional network resources due to the existence of the cache.
• Speed up response: because it is not necessary to make a round-trip through the entire network, caching can make the content faster. The cache is placed closer to the user. For example, the browser cache makes the Content acquisition almost instantaneous.
• Speed up on the same hardware: for servers that store raw content, more performance can be squeezed out of the hardware by allowing radical cache policies. Content owners can use a powerful server on the distribution path to cope with the impact of specific content loads.
• When the network is interrupted, the content is still available: using a policy, the cache can ensure that the corresponding content is still available to users when the original server becomes unavailable.

Terms

When facing the cache, you may be unfamiliar with some frequently encountered terms. Some common terms are as follows:

• Original server: The original storage location of the content. If you are a Web server administrator, It is the machine you manage. It is responsible for replying to any content that cannot be obtained from the cache and setting cache policies for all content.
• Cache hit rate: the cache validity is measured based on the cache hit rate. It is the ratio of the number of requests that can obtain data from the cache to the number of all requests. A high cache hit rate means that a high proportion of data can be obtained from the cache. This is usually the result that most administrators want.
• Freshness: Indicates whether a cached item is suitable for returning to the client. The content in the cache is returned only when the cache policy specifies the freshness period.
• Expired content: Set expired content in the cache based on the Cache Policy's freshness. Expired content is marked as "obsolete ". In general, the expired content cannot be used to reply to client requests. You must request new content from the original server or at least verify that the cached content is still accurate.
• Verification: the expired content in the cache can be verified to verify whether it is valid to refresh the expiration time. The verification process includes contacting the original server to check whether the cached data still represents the latest version.
• Invalidation: the process of removing content from the cache based on the expiration date. This is required when the content has been changed on the original server. Expired content in the cache may cause problems on the client.

There are many other cache terms, but the above should help you start.

What can be cached?

Some specific content is easier to cache than others. For most websites, some content suitable for caching is as follows:

• Logo and trademark image
• Ordinary non-changing images (for example, navigation icons)
• CSS style sheets
• Common Javascript files
• Downloadable content
• Media files

These files tend to not change frequently, so caching them for a long time can benefit.

Note the following when caching some items:

• HTML page
• Will replace the changed image
• Frequently modified Javascript and CSS files
• Content that requires authenticated cookies to be accessed

Some content should never be cached:

• Resources related to sensitive information (bank data, etc)
• User-related and frequently-changed data

In addition to the general rules above, you usually need to specify rules to better cache different types of content. For example, if all logged-on users see the same website view, the page should be cached anywhere. If the logged-on user will see the user-specific view on the site within a period of time, you should let the user's browser cache the data instead of letting any intermediary node cache the view.

Web content cache location

Web content will be cached in many different locations throughout the delivery path:

• Browser cache: the Web browser maintains a small cache. Typically, a browser uses a policy to indicate the most important content to be cached. This may be because the user-related content may be requested again and the download cost is high.
• Intermediate cache Proxy: any server between the client and your infrastructure can cache some content as expected. These caches may be provided by the ISP (Network Service Provider) or other independent organizations.
• Reverse cache: your server infrastructure can implement your own cache for backend services. If the cache is implemented, the corresponding content can be returned at the location where the request is processed, instead of using the backend service for each request.

These locations can usually cache some corresponding content based on their own cache policies and the cache policies of the content source.

Cache Header

The cache policy depends on two different factors. The cached object must decide whether to cache acceptable content. It can only cache some content that can be cached, but cannot cache content that exceeds the limit.

The cache behavior is mainly determined by the cache policy, and the cache policy is set by the content owner. These policies are clearly expressed through specific HTTP headers.

After several changes in the HTTP protocol, there are some different headers for caching, and their complexity is different. The following lists the things you should pay attention:

• **Expires**: Although the scope of use is quite limitedExpiresThe header is very concise and clear. It usually sets a future time when the content will expire. At this time, any request for the same content should return to the original server. This header may be the most suitable for the rollback mode (fall back ).
• **Cache-Control**: This isExpiresIs a more modern replacement. It has been well supported and has more flexible implementation. In most casesExpiresBetter, but it is fine to set both values at the same time. We will discuss what you can set laterCache-Control.
• **ETag**:ETagUsed for cache verification. The source server can provide a uniqueETag. When a cache needs to verify whether the content is about to expire, it willETagSent back to the server. The source server either tells the cache that the content is consistent, or sends the updated content (with the newETag).
• Last-Modified: This header specifies the last modification time of the corresponding content. It may be used as part of a validation policy to ensure content freshness.
• **Content-Length**: Although not explicitly included in the cache,Content-LengthThe header is important when setting cache policies. If some software does not know the content size in advance to leave enough space, it will refuse to cache the content.
• **Vary**: The cache system usually uses the requested host and path as the key to store the resource. When determining whether a request contains the same content,VaryThe header can be used to remind the cache system to pay attention to another additional header. It is usually used to tell the cache system to pay the same attention.Accept-EncodingHeader so that the cache system can distinguish between compressed and uncompressed content.

Secret of the Vary Header

VaryThe header provides you with the ability to store different versions of the same content, at the cost of reducing the cache capacity.

In useAccept-Encoding, SetVaryThe header allows clear distinction between compressed and uncompressed content. This is important for browsers that cannot process compressed data. It ensures basic availability.VaryA typical value of isAccept-EncodingIt has only two to three optional values.

It looks likeUser-AgentThis header can be used to differentiate between mobile browsers and desktop browsers, so that your site can provide Differentiated Services. HoweverUser-AgentThe string is non-standard, and the result will cause the cache of many different versions of the same content to be saved in the intermediate cache, which will reduce the cache hit rate.VaryThe header should be used with caution, especially if you do not have the ability to standardize requests in the intermediate cache under your control (maybe, for example, you can control CDN ).

How Cache Control flag affects Cache

As mentioned aboveCache-ControlHow the headers are used in compliance with modern cache policy standards. You can use this header to set many different cache commands. Multiple different commands are separated by commas.

Some of the content cache policies you can useCache-ControlThe options are as follows:

• no-cache: This command indicates that all cached content must be re-verified when new requests arrive and then sent to the client. This command immediately marks the content as expired, but allows verification to be performed again to avoid re-downloading the entire content.
• no-store: This command indicates that the cached content cannot be cached in any way. It is suitable for setting when replying to sensitive information.
• public: It marks the content as public, which means it can be cached by the browser and any other intermediate node. In general, for requests that use HTTP authentication, the reply is markedprivate.publicThe flag will overwrite this setting.
• private: It marks the content as private. Private data can be cached by your browser,NoCached by any intermediate node. It is usually used for user-related data.
• max-age: This setting indicates the maximum lifetime of the cached content. After the maximum lifetime, it must be verified at the source server or downloaded again. In modern browsers, this option largely replacesExpiresThe browser also uses the header as the basis for determining the freshness of the content. The value of this option is measured in seconds. The maximum value is 31536000 seconds ).
• s-maxage: This option is very similarmax-ageIt specifies the time when the content can be cached. The difference is that this option is only valid in the cache of the intermediate node. With these two options, you can build more flexible cache policies.
• must-revalidate: It specifiesmax-age,s-maxageOrExpiresThe freshness information specified in the header must be strictly observed. This prevents cached data from being used in similar scenarios, such as network interruptions.
• proxy-revalidate: It serves the same purpose as the preceding options, but only applies to the intermediate proxy node. In this case, your browser can use expired content when the network is interrupted, but the intermediate cache content cannot be used for this purpose.
• no-transform: This option indicates that the cache cannot modify the received content for performance reasons under any circumstances. This means that the cache cannot compress and send the received content (the content of the compressed version is not received from the original server.

These options can be combined in different ways to obtain different cache behaviors. Some mutex values are as follows:

• no-cache,no-storeAs well as common cache behaviors specified by other options not mentioned earlier
• publicAndprivate

Ifno-storeAndno-cacheAre set, thenno-storeWill replaceno-cache. Reply to unauthorized requests,publicIs an implicit setting. Reply to authorized requests,privateOptions are hidden. They canCache-ControlSpecify the opposite options in the header to overwrite.

Develop a cache policy

Ideally, any content can be cached as much as possible, and your server only needs to occasionally provide some verification content. But this rarely happens in reality, so you should try to set some wise cache policies to balance the long-term cache and site change needs.

FAQs

In many cases, content should not be cached due to the way the content is generated (such as dynamically generated by each user) or the characteristics of the content (such as bank sensitive data. Another problem that many administrators may face when setting the cache is that the data in the external cache has not expired, but the new data version has been generated.

These are common problems that affect the cache performance and the accuracy of the data you provide. However, we can alleviate these problems by developing cache policies that anticipate these problems in advance.

General suggestions

Although your actual situation will guide you to select a cache policy, the following suggestions can help you get some reasonable decisions.

Before you worry about a specific header, there are some specific steps to help you increase your cache hit rate. Some suggestions are as follows:

• Create a specific folder for images, CSS, and shared content: place the content in a specific folder so that you can easily reference the content from any page on your site.
• Use the same URL to express the same content: Because the cache uses the Host Name and path in the content request as the key, you must ensure that the reference method of the content on all your pages is the same, the previous suggestion makes this easier.
• Use CSS image stitching as much as possible: For content such as icons and navigation, using CSS image stitching can reduce the number of requests required to render your page and allow a long time to cache the stitching.
• Localize host scripts and external resources whenever possible: If you use Javascript scripts and other external resources, if the upstream does not provide a suitable cache header, you should consider placing the content on your own server. You should pay attention to any upstream updates to update local copies.
• Collecting file summaries for cached content: static content such as CSS and Javascript files is usually suitable for collecting file summaries. This means that adding a unique identifier (usually the hash value of the file) to the file name can bypass the cache after the file is modified to ensure that the new content is retrieved again. There are many tools that can help you create file summaries and modify references in HTML documents.

For correctly selecting different headers for different files, the following content can be used as a general reference:

• Allow all caches to store general content: static content and non-user-related content should be cached on all nodes of the distribution chain. This allows the intermediate node to reply the content to multiple users.
• Allow the browser to cache user-related content: for each user's data, it is acceptable and beneficial to cache the data in the user's own browser. The cached content can be instantly read in the browser of the user, but it is not suitable for caching in any intermediate proxy node.
• Take time-sensitive content as a special case: If your data is time-sensitive, you should take the data as a special case for the above two references, to ensure that expired data is not used in critical situations. For example, if your site has a shopping cart, it should immediately reflect the items in the shopping cart. Based on the features of the content, you canCache-ControlUse in the headerno-cacheOrno-store.
• Always provide validators: The validators enable refresh expired content without re-downloading. SetETagAndLast-ModifiedThe header will allow the cache to verify the content to the original server and refresh the content freshness when the content is not modified to reduce the load.
• For supported content, set a long refresh period: to make better use of the cache, some supported content should be set for a long refresh period. This is usually suitable for rendering HTML page content by user requests such as images and CSS. Together with the file summary, setting an extended period of freshness will allow the cache to store these resources for a long time. If the resource changes, the modified file summary will invalidate the cached data and trigger the download of new content. At that time, the new supported content will continue to be cached.
• Set a short freshness period for the parent content: to make the previous mode work normally, the content of the container class should be set to a short freshness period, or set to not all cache. This is usually the HTML page used in other assistance content. This HTML page will be frequently downloaded so that it can respond to changes quickly. Therefore, supported content can be cached as much as possible.

The key is to achieve a balance. On the one hand, you can cache as much as possible, and on the other hand, you can reserve the opportunity to change the entire content when a change occurs in the future. Your site should also have:

• Content cached as much as possible
• Cache content with a short period of freshness can be re-Verified
• Content that is not cached at all

The purpose is to move the content to the first category (Cache as much as possible) and maintain an acceptable cache hit rate.

 

Conclusion

Taking the time to make sure your site uses the appropriate cache policy will have an important impact on your site. Cache allows you to reduce bandwidth usage while ensuring the same service content. Therefore, your server can handle more traffic with the same hardware. Perhaps more importantly, customers can get a faster experience on your website, making them more willing to visit your site frequently. Although effective Web Cache is not a silver bullet, setting a proper cache policy will help you reap the best results at minimal cost.

This article permanently updates the link address: