Linux memory mechanism and manual release of swap and memory, linux mechanism to release swap

Linux memory mechanism and manual release of swap and memory, linux mechanism to release swap

Today, let's talk about the Linux memory mechanism.

First, let's take a look at the concept

1. What is the linux memory mechanism?

We know that reading and writing data directly from the physical memory is much faster than reading and writing data from the hard disk. Therefore, we hope that all data can be read and written in the memory, while the memory is limited, this introduces the concept of physical memory and virtual memory.

Physical memory is the memory size provided by the system hardware and is the real memory. Compared with physical memory, there is also a concept of virtual memory in linux, virtual Memory is a strategy proposed to meet the shortage of physical memory. It is a logical memory virtualized by the disk Space. The disk Space used as the virtual memory is called the Swap Space ).

As an extension of the physical memory, linux uses the virtual memory of swap partitions when the physical memory is insufficient. In more details, the kernel will write the memory block information that is not used for the moment to the swap space, in this way, the physical memory has been released, and this memory can be used for other purposes. When you need to use the original content, this information will be re-read from the swap space into the physical memory.

Linux memory management adopts a paging access mechanism. To ensure that the physical memory can be fully utilized, when appropriate, the kernel automatically exchanges infrequently used data blocks in the physical memory to the virtual memory, and retains frequently used information to the physical memory.

To learn more about the linux memory running mechanism, you need to know the following aspects:

1. The Linux system will switch pages from time to maintain as much free physical memory as possible. Even if there is no need for memory, Linux will swap out memory pages that are temporarily unavailable. This avoids waiting for the switching time.

2. In Linux, page switching is conditional. Not all pages are switched to the virtual memory when they are not used. The Linux kernel uses the "most recently used" algorithm, swap some infrequently used page files to the virtual memory. Sometimes we can see that there is still a lot of linux physical memory, but the swap space is also used. In fact, this is not surprising. For example, a process that occupies a large amount of memory requires a lot of memory resources to run, and some infrequently used page files will be exchanged to the virtual memory, however, when the process that occupies a lot of memory resources ends and releases a lot of memory, the page files that have just been exchanged will not be automatically exchanged into the physical memory unless necessary, now the physical memory of the system will be much idle and the swap space will be used. Don't worry about this. You just need to know what it is.

3. The page of the swap space will be first exchanged to the physical memory when used. If there is not enough physical memory to accommodate these pages, they will be immediately exchanged, there may not be enough space in the virtual memory to store these swap pages, which will eventually lead to issues such as false crashes and service exceptions in linux. Although linux can recover itself within a period of time, however, the recovered system is basically unavailable.

Therefore, reasonable planning and design of Linux memory usage is very important.

In Linux, when an application needs to read data from a file, the operating system first allocates some memory to read data from the disk into the memory, then, the data is distributed to the application. When you need to write data to a file, the operating system first allocates memory to receive user data, and then writes the data from the memory to the disk. However, if a large amount of data needs to be read from the disk to the memory or written into the disk by the memory, the system's read/write performance becomes very low, because whether it is reading data from the disk, writing data to a disk is a very time-consuming and resource-consuming process. In this case, the buffers and cached mechanisms are introduced in Linux.

Both buffers and cached are in-memory operations to save the files that have been opened by the system and the file attribute information. In this way, when the operating system needs to read some files, it will first search in the buffers and cached memory areas, if the data is found, read the data directly and send it to the application. If the data is not found, the data is read from the disk. This is the operating system's cache mechanism, this greatly improves the performance of the operating system. However, the buffer content of buffers and cached is different.

Buffers is used to buffer Block devices. It only records metadata (metadata) and tracking in-flight pages of the file system, while cached is used to buffer files. More broadly speaking, buffers is mainly used to store contents in directories, file attributes, and permissions. Cached is used directly to remember the files and programs we opened.

To verify whether our conclusion is correct, you can open a very large file through vi to see the changes in cached, and then vi the file again, I feel the similarities and differences between the two open speeds, is the second open faster than the first one?
Run the following command:

find /* -name  *.conf

Check whether the buffers value has changed, and then execute the find command again to see if the display speed is different twice.

 

 

Ii. When does linux start to use virtual memory (swap )?

A. [root @ wenwen ~] # Cat/proc/sys/vm/swappiness

60

The above 60 indicates that swap is used only when 60% is used in the physical environment.

When swappiness = 0, it indicates that the physical memory is used to the maximum extent, and then the swap space is used,

Swappiness = 100 indicates that swap partitions are actively used, and the data in the memory is promptly moved to the swap space.

Generally:

We recommend that you set the swap partition to twice the memory (when the memory is smaller than or equal to 4 GB). If the memory is greater than 4 GB, the swap can only be larger than the memory. In addition, try to lower the swappiness so that the system performance will be better.

B. Modify the swappiness parameter.

Temporary modification:

[Root @ wenwen ~] # Sysctl vm. swappiness = 10

Vm. swappiness = 10

[Root @ wenwen ~] # Cat/proc/sys/vm/swappiness

10

 

Permanent modification:

[Root @ wenwen ~] # Vim/etc/sysctl. conf

Add parameters:

Vm. swappiness = 35

Then directly:

[Root @ wenwen ~] # Sysctl-p

Check whether it takes effect:

Cat/proc/sys/vm/swappiness

35

 

 

 

3. How to release the memory?

Generally, the system does not automatically release the memory.

Key configuration file/proc/sys/vm/drop_caches. This file records the cache release parameters. The default value is 0, that is, do not release the cache. The value can be 0 ~ Any number between 3 represents different meanings:

0-do not release
1-release page Cache
2-release dentries and inodes
3-release all caches

Practice:

 

Obviously, a lot of idle memory is available.

 

 

4. How to release swap?

Prerequisites: Make sure that the remaining memory is greater than or equal to the usage of swap. Otherwise, the system will go down! According to the memory mechanism, once a swap partition is released, all files stored in the swap partition will be transferred to the physical memory. Generally, the swap is released by re-attaching the swap partition.

A. Check where the current swap partition is mounted?

B. Shut down the partition.

C. view the status:

D. Check whether the swap partition is shut down. The bottom row shows all 0.

E. Mount swap to/dev/sda5

F. Check whether the mounting is successful.

 

 

5. Some small examples?

When setting mysql

My. cnf File

Innodb_buffer_pool_size = 6 GB (the memory of my operating system is 6 GB. Generally, this value is set to 80% of the memory of the operating system)

This value is not set to the greater the better. If the setting is too large, the system's swap space will be occupied, resulting in a slow operating system, thus reducing the SQL query efficiency.

As you can understand, when I set this buffer_pool_size to a large value and the operating system memory size, mysql will call a large amount of data into the memory within a period of time, because of the linux memory mechanism, some data is filtered into the swap partition based on the latest optimal principle, while the swap partition does not clean up the data in time. When other data is called again, some data is filtered into the swap partition, which may result in the remaining physical memory or even null while the swap partition is full.