Image Storage Architecture learning: cache, architect's beautiful S3. (1)

I have synchronized it from my independent blog and discussed it together

Copyright Disclaimer: non-commercial reprinting, retaining the integrity of the original content, and signing the author Wang Guofeng and the original text link

Preface

In the previous article, I forcibly broke up the lovers of web servers and image servers. This may be a bit unfriendly, but it will definitely be good for their future development. In the near future, they will thank me.

Now, let's go back to today's topic. This article will focus on the important position of Cache Technology in Image Storage Architecture and explore the implementation of image cache solutions. My personal opinion is: a large system with excellent personality and strong scalability is bound to cache all resources that can be cached, because nothing is faster than operating in the memory (the CPU is not counted, and the available space is too small ).

PS: some of the content in this article will be played by an empty instructor.

Why cache images?

Simply put, the cache can directly access the memory to increase the image reading speed and ease the image I/O bottleneck caused by the high site access volume. In many cases, images are a type of static resource that is not updated frequently. They are typically much larger than writes. Therefore, they fully comply with the cache principle: one write and countless reads. Of course, not all those that satisfy this need will use caching. As mentioned in the previous Youku Architecture Study note, Youku does not use a large number of video caching technologies when storing videos, there are two reasons:

1. Squid write () User process space is consumed. Lighttpd 1.5 AIO (asynchronous I/O) reads files to the user memory, resulting in low efficiency. This is caused by memory locks.
2. If you receive a notification from your eldest brother that you want to remove a video, it will be troublesome to cache it (this is a bright spot, O (∩ _ ∩) O)

But in general, static files such as images and videos are very suitable for caching. Caching is always the beauty of architects, haha.

Research on Image cache Solutions

Next, we will focus on exploring the specific solution for Implementing Image caching. The article tries its best to go deeper based on the normal thinking of people. From Simplicity to complexity, let's take a look at it, we hope that after reading this article, we will slightly improve your design taste. Fashion design is a process of constant scrutiny and running-in. I believe you have a deep understanding of programmers and siege lions. This article describes how to set HTTP headers in Web containers such as IIS to cache images in two articles.

Note: The Web Container here refers to IIS, tomcat, or other web server software. I will use IIS as an example below.

Cache in IIS is divided into server cache and Client Cache. For static resources (such as HTML, CSS, JS, and images), server cache is enabled by default, that is, when we request static resources from the server, the server first retrieves files from its memory, and then finds the files on the hard disk if they cannot be retrieved (so tired !), Sometimes it is annoying to enable server cache, which we will mention later. As the name implies, the client cache caches the requested resources on the client. When a user requests resources, he/she first searches for the requested resources from the local location, and cannot find the requested resources before bothering the server. The following describes the client cache in detail.

1. Client Cache

How to Set it? See

(Image source)

Indicates that the cache module of IIS is enabled and the expiration time is set to one day, that is, within one day, users can read the image from their local cache copy when accessing the image, you don't have to download images from the server. Of course, the cache will expire one day later, and the images will be loaded into the browser cache again after being downloaded. Of course, this method requires you to have the permission to manage IIS. I think you have the permission to be an architect.

How does this work? Don't worry. Please listen to me again. Please be patient.

This is mainly controlled by HTTP headers. HTTP headers is the core of HTTP request and response. It carries information about the client browser, request page, server and other related information. Simply put, it is an information card for the interaction between the browser and the server, the browser informs the Server Client about the relevant information and request information (for example, what type of browser I am using, whether I can receive the gzip content on your server, how long the request is, whether or not it is allowed) cache and so on ); the server informs the Browser Server of relevant information (such as server variables, returned length, and cookie) in response to this request. For details about HTTP headers, refer to this article.

After learning about HTTP headers, I will explain this cache problem. My explanation is as follows: when I send an HTTP headers request a photo image from an empty instructor to the server, the server extracts the HTTP headers and findsCache-control is public, that is, it can be cached. Then the server checks the cache expiration time set by itself and finds that the cache has not expired, so it tells me, kiddie, find the picture of the empty teacher in your local area first. If you cannot find the picture, ask for it again. So I went back and found that there was really a picture of an empty teacher in the local area. He didn't lie to me, so happy.

I don't know. Do you understand what I mean? Whether you understand it or not, I understand it! Of course there is something wrong, please point out, thank you.

2. Server Cache

As mentioned above, the server cache in IIS is enabled by default. By default, IIS caches static resources for fast reading. When static files are changed, the cache can also be automatically updated. However, there is a very annoying problem. If I have a large number of images (millions or tens of millions) and they are all updated in real time (such as stock market charts), this is the problem, when I update so many images, the IIS cache has not been updated yet (the amount of images is too huge), so I will access the images in the old version for a long time, I am very annoyed by this. So since it violates the cache principle, how can we disable it? The method is definitely not as simple as you think. I suggest you read this article to directly modify metabase. XML file. If you are not familiar with the server, you may not be able to touch this file at will, but you are an architect.

Through the above explanation, I think you should use this simplest method to set the cache. If you are really stupid, you can understand it, so please use your left hand to hit the right hand twice. If you still don't understand it, then use your right hand to hit the left two times. I just hit it until I understood it, haha.

Okay, the above is about how to set cache in IIS. Here I have referenced the empty instructor several times, so you are tired. Thank you.

In the next article, I will continue to talk about the image caching technology, but it is about distributed caching, which is a little more advanced than this one. Please stay tuned.

Reproduced Please consciously indicate the original link: http://www.itivy.com/ivy/archive/2012/2/18/image-storage-cache-1.html