How smart women can cook rice cakes (3) -- compress cookies (zram)

This is a reliable method for small memory processing-zram. It looks like a compressed biscuit. Although a small biscuit does not look big (zram's compressed page occupies memory), it feels full when drinking water (releasing the content of a page ).

1. Introduction

2. How to enable

3. Workflow

4. What else can I do?

Introduction:
Zram does not replace the page to external storage when a swap event occurs. The external storage in the mobile phone is emmc, and the external storage in the computer is the hard disk. Their read/write speed is similar to the memory read/write speed, just like the speed of turtles and cars (the memory read/write speed is much higher than the external storage read/write speed ), therefore, smart developers compress the page to be replaced and store it in the memory to simulate a portion of the memory as external storage. When swap events occur, the time spent is the compression and decompression time, which greatly improves the performance.
The platform described below is Qualcomm msm8974.
How to enable zram:
Zram is registered as a block device into the kernel. For Linux kernel, zram is an external storage device and its file path (Qualcomm ):

Linux/Android/kernel/Drivers/staging/zram/zram_drv.c

The relevant code will be explained later.

Zram is not enabled by default in the high-end platform. Some MTK platforms are open. According to the official Google tutorial, You need to enable the switch in the configuration file. The steps are as follows:

1) in the source code: Linux/Android/kernel/ARCH/ARM/configs, find msm8974_defconfig and add the following configuration options:

CONFIG_SWAP=yCONFIG_CGROUP_MEM_RES_CTLR=yCONFIG_CGROUP_MEM_RES_CTLR_SWAP=yCONFIG_ZRAM=yCONFIG_ZSMALLOC=y

The first four options are prompted to be opened in the official Android tutorial. However, in my actual operations, I found that zram0 files cannot be loaded, that is, zram is not compiled into the kernel at all, after verification, it turns out that config_zram relies on config_zsmalloc, block, and sysfs. Therefore, after compilation, the compiled target file is found in the target directory.

2) Update fstab, which is located in Linux/Android/device/qcom/msm8974 and has fstab. qcom in the directory.

This file is strongly dependent on the current CPU vendor. In this tutorial, we use fstab. X stands for different vendors. Add the fstab format statement to this file. The added command format is not described here.

3) modify the init. RC file

The Directory of the modified file is located in: Linux/Android/device/qcom/msm8974, which may not be found by the reader. RC file, good. It does not exist. The real name of this file is inir.tar get. RC. It is located in the same directory as fstab. qcom.

By default, when swap partitions are used, eight memory pages are read at the same time. When zram is used, a memory page is read at the same time, the advantage of this is that the time for reading a memory page is negligible and the memory pressure can be reduced.

To make the number of replaced pages 1 at the same time, add the following command:

Write/proc/sys/Vm/page-cluster 0,

In mount_all/fstab. qcom (or mount_allfstab.qcom. qcom is located in the root directory "/" and is also in the directory of the current script, so both writing methods are acceptable.) Add the statement: swapon_all/fstab. qcom

4) Compile the kernel and wait for the machine to restart.

Verify that the/dev/block/zram0 device character exists. Run the free command or CAT/proc/SWAPs command to verify that the swap partition is mounted and used properly.

Does the above process seem complicated? After I read and practiced it for the first time, I felt too complicated. I dared not to make it simple. MTK wrote a script. during initialization, run this script. I can only say, MTK, good job.

#No path is set up at this point so we have to do it here.PATH=/sbin:/system/sbin:/system/bin:/system/xbinexport PATHmkswap /dev/block/zram0swapon /dev/block/zram0

The above is the MTK enabling script, as long as you execute the first step above (the subsequent tedious processes can be omitted), execute this script at startup, zram can be enabled, isn't it easy?

Zram workflow zram uses the lzo compression and decompression algorithm. When zram compression algorithm is selected, the compression ratio and compression decompression speed must be weighed. The two methods also affect the overall performance.

After learning about the compression and decompression algorithms, how is the page compressed? What is the process? Everything is going back to the zram block device driver.

Path of the device driver file: Linux/Android/kernel/Drivers/staging/zram/zram_drv.c

As we all know, zram is a device that can be stored randomly. It is different from the disk that needs to consider Io scheduling. Therefore, when writing the zram Device Driver, select not the default request_queue, instead, a "request manufacturing function" is customized ":

zram->queue = blk_alloc_queue(GFP_KERNEL);

For the zram device, it first applies to the kernel for a queue. When I/O data is generated, It redirects to its own request. After the custom request queue is successfully applied:

blk_queue_make_request(zram->queue, zram_make_request);

The blk_queue_make_request function associates the applied queue with the zram_make_request function. When I/O data is generated, the zram_make_request function is called.

In the process of block data transmission, in addition to request_queue, struct Bio is also a very important data structure. zram_make_request will traverse struct bio, according to the value of the RW parameter, to determine whether to read data from zram or write data to zram, the compression and decompression functions are called here.

When it is determined that the data is written to zram, it is called:

Ret =Zram_bvec_write(Zram, bvec, index, offset );

When reading data, it calls:
Ret = Zram_bvec_read(Zram, bvec, index, offset, bio );

In Zram_bvec_readFirst, it will determine whether the page to be decompressed is zero page (the page data is all 0). If yes, it will directly write 0 to the decompressed page. Some code:

If (unlikely (! Meta-> table [Index]. Handle) | zram_test_flag (Meta, index, zram_zero) {handle_zero_page (bvec); // process zero page return 0 ;}

If it is not a zero page, the function will perform a temporary kernel ing on the decompressed page to ensure that the decompressed page can be correctly written into the memory.

The next step is to start the decompression process. The decompression function is:Zram_decompress_pageBut this function is not really just a decompression function, it will do some work before decompression, such as some ing, read the page data to be decompressed, and then make some judgments, actually decompressed Functions

Lzo1x_decompress_safe is complete. The call statement of this function is located in Linux/Android/kernel/lib/lzo/lzo1x_decompress.c:

Ret = lzo1x_decompress_safe (cmem, meta-> table [Index].Size,Mem, & Clen );

Finally, the extracted data exists in the memory where the mem pointer is located. Meta-> table [Index].SizeIs the size of the compressed data, while cmem is the data before decompression,

Clen is the size after decompression.

What is the compression process like? ReadZram_bvec_writeSource code, it also calledZram_decompress_pageWhat is the problem with this function? Why does the decompression function be called during the compression process? The comments of the Code are explained as follows: even if the I/O transmission of the incomplete page is required, the whole page needs to be read. View and discoverZram_decompress_pageThere are several statements:

if (meta->table[index].size == PAGE_SIZE)copy_page(mem, cmem);elseret = lzo1x_decompress_safe(cmem, meta->table[index].size, mem, &clen);

When determining that the size of the page to be decompressed is equal to page_size, simply copy the page instead of decompressing it. This is called before compression.Zram_decompress_page.

After a series of processing, such as zero page processing, the following code is called:

Ret = lzo1x_compress (uncmem, page_size,SRC, & Clen, meta-> compress_workmem );

Uncmem is the pointer address before compression, SRC is the data pointer after compression, and clen is the length after compression. SRC is assigned a value at the beginning:

SRC= Meta-> compress_buffer; that is, the compressed buffer is set at the beginning.

After obtaining the compressed data and length, the following task is not written to the compression page, but to determine whether the compressed data is larger than page_size. Generally, the size after compression is smaller than page_size/2, which is referred to as "thin compression". The size greater than page_size/2 is called "Fat compression", and the size greater than page_size is called "bad compression ", if bad compression occurs, call am-> stats. bad_compress ++; this indicates that this page is unnecessary. Therefore, the original page is saved to zram. If "thin compression" is displayed, zram-> stats is called. good_compress ++; indicates that the compression is successful, and the memory always hopes that the more "thin compression", the better. In fact, generally, a "Fat compression" and a "thin compression" share a page. If the compression density is too high, the decompression process will be very time-consuming, which will affect the overall performance of the system.

Under normal compression, compress the page data and save it to the page. Finally, update the data to be saved in the zram data structure, and remove some memory ing to complete the compression process.

What can be improved? Because zram has been mature for many years and has been used for many years, I visited GitHub again and found that someone used lz4 to replace the lzo compression algorithm. This is a real improvement. After successful transplantation, I personally did not feel a significant improvement. In theory, lz4 decompression speed is more than three times that of lzo, but because the memory decompression and compression speed is already very fast, the improvement is not obvious, but we suggest you try to change it. After all, technology is changing with each passing day. Do you need more data in the future? If it takes 3 minutes to decompress a file and 1 minute to decompress the file, the difference will come out. I also tried to compress and decompress the Linux kernel with lz4, the improvements are not obvious, but we are looking forward to the future performance of lz4 and more outstanding algorithms. Lz4 algorithm reference: Linear? Page = 11 the porting process is cumbersome. There are many commit and you need to be patient. After porting, You need to configure config to open related options. This author does not explain
The zram algorithm itself increases energy consumption, which is also described in the previous article, because zram consumes a lot of power without stopping decompression and compression, this requires reasonable configuration of zram size. The official recommendation is to set it to 25% of the memory size. I personally recommend that you set it to 25%-50% based on the actual situation. The larger the zram, the higher the power consumption, because it increases the chance of compression and decompression, but can increase the memory, you need to weigh it yourself. You also need to test the zram enabling, application startup speed, and smoothness, work with the test to get a suitable zram value. Each mobile phone is different from the system. Google said Android runs smoothly in M memory. I will wait and see if it will consume more power?

How smart women can cook rice cakes (3) -- compress cookies (zram)