CPU Level 1 cache, level 2 cache, and cache hit rate

The cache memory is a temporary memory located between the CPU and memory. It has much smaller capacity than the memory, but the switching speed is much faster than the memory. The emergence of cache mainly aims to solve the conflict between the CPU operation speed and the memory read/write speed, because the CPU operation speed is much faster than the memory read/write speed, this will take the CPU a long time to wait for the data to arrive or write the data into the memory. The data in the cache is a small part of the memory, but this small part is about to be accessed by the CPU in a short time. When the CPU calls a large amount of data, you can avoid calling the memory directly from the cache to speed up reading. It can be seen that adding cache to the CPU is an efficient solution, so that the entire internal memory (Cache + Memory) becomes a high speed of the existing cache, there is a large memory storage system. The cache has a great impact on CPU performance, mainly because of the Data Exchange sequence of the CPU and the bandwidth between the CPU and the cache.

The principle of caching is that when the CPU needs to read a piece of data, it first looks for it from the cache. if it finds it, it immediately reads it and sends it to the CPU for processing. If it does not, it reads data from the memory at a relatively slow speed and sends it to the CPU for processing. At the same time, it transfers the data block of the data to the cache, this allows you to read the entire block of data from the cache without calling the memory.

This reading mechanism makes the CPU read cache hit rate very high (most CPUs can reach about 90%), that is, 90% of the data to be read by the CPU next time is in the cache, only about 10% needs to be read from memory. This greatly saves the time for the CPU to directly read the memory, and does not need to wait for the CPU to read data. In general, the CPU reads data in the first cache and then the memory.

According to the data reading sequence and the closeness with the CPU, the CPU cache can be divided into Level 1 cache and level 2 Cache. Some high-end CPUs also have Level 3 cache, all the data stored in each level of cache is part of the next level of cache. the technical difficulty and manufacturing cost of these three caches are relatively decreasing, so their capacity is also relatively increasing. When the CPU needs to read a piece of data, it first looks for it from the level-1 cache. If it does not find it, it looks for it from the level-3 cache or in the memory. Generally, the hit rate of each level of cache is about 80%, that is, 80% of the total data volume can be found in the level-1 cache, only 20% of the total data volume needs to be read from the second-level cache, third-level cache, or memory. It can be seen that the first-level cache is the most important part of the entire CPU Cache architecture.