Cache and Buffer

Original article link

Another such thing is:Buffer. Here we will talk about buffer and cache.

So what exactly are they used? In fact, they are an intermediate layer between two relatively independent systems to avoid unnecessary interaction and unnecessary or repeated synchronization between the two systems. Synchronization, you know, you know about synchronization between systems of different orders of magnitude. For example, between memory and disk, such as between application and database. Buffer is for writing, and cache is for reading.

This articleArticleLet's take a look at the cache and buffer in the Linux system (some cache in the hardware, such as the CPU instruction cache, will not be discussed here ).

First, file IO

For write operations, we usually encounter two buffers ):
One is the kernel buffer.

When we call write to write a file, the content returned by write is not immediately written to the hard disk, but written to the kernel cache. When do I write data to a disk? The kernel has a cache mechanism. This has obvious advantages. For example, it takes almost the same time to call 1 Write tO Write 1 kb and 1 K write to write 1 1B data each time. The latter takes more time to switch between user and kernel modes, but the number of disk writes is the same. This greatly improves the efficiency.

The other is the user State buffer maintained by glibc.

What is this buffer used? The kernel and the hard disk are two relatively independent systems. The kernel buffer avoids many unnecessary synchronization between the two systems. Similarly, the kernel and userProgramIt is also two relatively independent systems, and each system call is also costly. Therefore, there is still a gap between the two methods in the previous example of 1 kb write and 1 kb write and 1 kb write each time, the difference is that the latter needs to perform 1 K switching between the user State and the kernel state. Therefore, glibc made another buffer in the user State. When we call the printf output provided by glibc, it is not directly mapped to a Write System Call. Instead, it is stored in the buffer of glibc management, when the conditions are met (as described below), the write operation is called again to write the user-state buffer to the kernel state. Therefore, it takes almost the same time to call printf from 1 to 1 kb and print from 1 kb each time.

In turn, we have cache for reading)

Reading files with read is actually the cache of the read kernel. When the kernel cache does not exist, it reads some content from the disk to the cache, and then returns the content from the cache to the user. Note: In some cases, we can perform further optimization. Let's take a look at it. When we read a file from start to end, is the time between 1 kb read-only and 1 kb read every 1 kb? This requires the kernel to read 1 kb of content to the cache before reading (this is called read ahead ). The kernel does not know whether our subsequent read calls are sequential reads from start to end or random reads. If it is random reads, read ahead will be counterproductive. Fortunately, POSIX specifies the posix_fadvise () system call. Let's tell the kernel how we read files. This call tells us that the kernel is read in ascending order. Read performance comes up. Refreshing.

The following lists the default buffer behavior of glibc (which can be modified using setvbuf ):

If the file is stderr, It is not buffered by default. Each printf corresponds to a write.

For example, if a file is a terminal, the row buffer is used by default. When a line feed is encountered, the write operation is performed once.

If the file corresponds to a disk file, the default value is full buffer. When the cache is full or the file is closed, write is performed.

Of course, you can call fflush () at any time to force the user-state cache to be written to the kernel buffer.

Okay. After talking about file buffering, let's talk about socket buffering.

Yes! The socket is also buffered. In the same example, we apply to the socket: 1 send 1kb and 1 K send 1 1B each time. Which one is time-consuming and laborious? Obviously, this is the latter, because if we know that IP address headers and TCP headers require bandwidth. If a large IP packet contains only one byte of data, it is a big waste of bandwidth. Therefore, John Nagle proposedAlgorithm, Buffer these messy packets, gather n to form a large package, and then send it out. This greatly improves the network efficiency.

Of course, this operation also has a negative effect. When our applications require high real-time network performance, this mechanism may increase network latency (after all, it will not be until the collection is complete ). In this case, use the setsockopt + tcp_nodelay parameter to disable this optimization.

What else? Of course, cache and buffer are everywhere. Cache is everywhere in memory management. For user space, ptmalloc2 and Google tcmalloc in glibc all have cache. The complexity of this part is extremely high, so I will have a chance to analyze it later. This is the case in this article.

 

I talked about cache and buffer at the Linux system level. Here we will talk about the cache at the application layer. Compared with the system-level cache concentrated on Io, the application-layer cache is varied. Let's start with the Web.

Web Cache is indispensable for servers and clients.

For Web servers, cache is very important. It can greatly increase concurrency, shorten the corresponding time, and reduce server load. The principle is simple. Because for a URL, the content changes little in a short period of time. It is too waste if the server counts every request. Therefore, the Web cache is a proxy placed in front of the Web server. It receives user requests and sends back-end requests to cache these responses when returning responses, when a request is returned directly without being computed by the server, the response speed is greatly improved, especially when the request volume is large. The Web Cache uses Squid and varnish.

Cache is also indispensable for the client browser. The cache is supported even in the HTTP protocol. The HTTP return code 304 not modified tells the browser that the content you want is not changed. Use the content in your cache. In addition to the HTTP protocol, the browser will also perform a lot of caching on its own. For images, JS, or other files, requests in a short period of time will directly not be retrieved remotely, which will greatly reduce the number of requests, this saves a lot of time and requires speed. Besides, browsers and servers are born in the same way and do not suffer from each other. So sometimes you have to force refresh.

Databases are closely related to Web servers.

Cache is also ubiquitous in database systems. Because of the same principle, for the same SQL query, under some conditions (for example, the table it queries has not been updated since the last query) the result is the same as that of the last query. Therefore, MySQL provides query cache. Some frameworks also provide caching functions, such as hibernate. This is a read cache, which aims to read a lot and improve the read performance when there are few writes. If there are many writes and there are few reads, this type of cache will be useless. Therefore, in some cases, we hope to introduce the write cache for the database. A typical example is primary key query and update. As a result, Kv databases (such as memcached) appear ). Read/write cache based on primary key query is provided. This is extremely important for improving the overall performance of the data system.

What other varied caches do?

For example, DNS caching also exists in the client and DNS server, and works the same way as Web caching. Cache DNS resolution results.

For example, ARP cache caches ARP results.

Even the compilation system introduces cache mechanisms such as ccache, such as PCH in Visual Studio.

Finally, I want to say: cache is king! Cache is everywhere!