Linux Kernel shared memory Analysis

See this articleArticleThere are only two words: depressed. Still in the Linux-2.6.29, it is to look at the Linux kernel virtual machine, then I feel that multiple VMS certainly have a lot of memory is shared, can be combined, I did not expect the development of Linux is so fast, the linux-2.6.32 officially supported the KSM mechanism, to achieve the combination of shared memory, but also specifically for the virtual machine, really let me crazy, it's a waste of time. Next, I will repost an article related to KSM and make a record after learning it. I only need to prepare for my graduation project.

 

Introduction:As a system managementProgram(Hypervisor) Linux has several innovations. One interesting change in 2.6.32 kernel is the kernel shared memory (KSM ). KSM allows this system administrator to increase the number of concurrent virtual machines by merging memory pages. This article explores the concepts behind KSM (such as storage decoupling), implementation of KSM, and how to manage KSM.

Server Virtualization

Virtualization Technology began to emerge in the 1960s s and has become popular through IBM System/360 mainframe. More than 50 years later, virtualization technology has developed by leaps and bounds, making it possible for multiple operating systems and applications to share one server. This special purpose (calledServer Virtualization) Is evolving into a data center because a single physical machine can be used to host 10 (generally) or more virtual machines (VMS), as shown in 1. This virtualization makes the infrastructure more dynamic, more energy-efficient, and more economical.

 

The pages are the same. Assume that the operating system and applicationsCodeAnd constant data is the same between VMS, so this feature is very useful. When the page is unique, they can be merged to release the memory for use by other applications. Figure 2 demonstrates Memory Sharing and shows the benefits of more available idle memory when pages are shared between VMS with the same content.
Figure 2. Cross-VMS Memory Sharing
 

Feature name

The features described in this article are very new; therefore, their names have undergone some changes. You will find that this Linux kernel feature is calledKernel shared memoryOrKernel samepage Merging.

You will soon find that, despite the advantages of Linux memory sharing in Virtual Environments (KSM was originally designed for Kernel-based virtual machines), it is still useful in non-virtual environments. In fact, KSM is even useful in Embedded Linux systems and shows the flexibility of this method. Next, we will explore this Linux memory sharing method and how to use this method to increase the memory density of the server, thus increasing its ability to host other applications or VMS.

Other technical support

One of the storage technologies isDe-duplication)The latest development is a pioneer in Memory Sharing in Linux and other system management programs. Decoupling this technology reduces stored data by deleting redundant data (based on data blocks, or based on larger data fragments, such as files. Public Data fragments are merged (in a copy-on-write [Cow] mode) to release space for other purposes. Using this method, the storage cost is lower, and the storage required is less. In view of the current data growth rate, this function is very important.

KSM operations

KSM acts as the daemon process in the kernel (calledKsmd), Which periodically performs page scans, recognizes and merges copies, and releases these pages for use. The process of KSM performing the preceding operations is transparent to users. For example, the copy page is merged (and then marked as read-only), but if one of the users on this page changes the page for some reason, the user will (in cow Mode) receive your own copy. You canSource code./MM/KSM. C. Find the complete implementation of the KSM kernel module.

KSM relies on advanced applications to provide guidance on determining the pooled candidate memory regions. Although KSM can only scan anonymous pages in the system, it will waste CPU and memory resources (taking into account the space required to manage page merging processes ). Therefore, an application can register a virtual area that may contain a copy page.

KSM application programming interface (API) throughMadviseSystem Call (see Listing 1) and a new suggested parameter (advice parameter)Madv_mergeable(It indicates that the defined regions can be merged. You can useMadv_unmergeableParameters (unmerge any merged pages from a region immediately) delete a region from the merged status. Note thatMadviseTo delete a page area may result inEagainError, because this operation may exhaust the memory during the cancel merge process, which may cause greater trouble (insufficient memory ).

Listing 1. madvise system call

# Include <sys/Mman. h> intMadvise(Void * Start, size_t length, int advice );

Once a region is defined as "merged", KSM adds the region to its working memory list. When KSM is enabled, it searches for the same page, retains one page in write-protected cow mode, and releases another page for use.

KSM uses different methods than memory decoupling. In traditional decoupling, objects are hashed, and then hash values are used for initial similarity check. When the hash values are consistent, the next step is to compare an actual object (in this example, a memory comparison) to formally determine whether these objects are consistent. KSM used this method in its first implementation, but later developed a more intuitive method to simplify it.

In the current KSM, the page is managed through two "red-black" trees, one of which is temporary. The first tree is calledUnstable treeFor storage, it cannot be understood as a stable new page. In other words, pages that are used as candidate objects for merging (remain unchanged for a period of time) are stored in this unstable tree. Pages in the unstable tree are not write-protected. The second tree is calledStability treeTo store pages that have been found to be stable and merged through KSM. To determine whether a page is stable, KSM uses a simple 32-bit checksum ). When a page is scanned, its checksum is calculated and stored together with the page. In a subsequent scan, if the new checksum is not equal to the previously calculated checksum, the page is being changed, so it is not a qualified merge candidate object.

When using the KSM process to process a single page, the first step is to check whether the page can be found in the stability tree. The process of searching for a stable tree is interesting because every page is considered a very large number (page content ). OneMemcmp(Memory comparison) the operation will be executed on the page of this page and related nodes. IfMemcmpIf the return value is 0, the page is the same and a matching value is found. Otherwise, ifMemcmpReturn-1, Indicates that the candidate page is smaller than the page of the current node.1, Indicates that the candidate page is greater than the page of the current node. Although a relatively 4 kb page seems to be a heavyweight comparison, in most cases, once a difference is found,MemcmpIt will end early. See figure 3 to view the visual presentation of this process.

Figure 3. Page search process in the Search Tree

 

If the candidate page is in the stable tree, the page is merged and the candidate page is released. The code is located in KSM. c/Stable_tree_search ()(KSM. c/Cmp_and_merge_page (). If no candidate page is found, go to the unstable tree (see KSM. c/Unstable_tree_search ()).

When searching in an unstable tree, the first step is to re-calculate the checksum on the page. If this value is different from the original checksum, the page will be discarded for subsequent searches for this scan (because it is changed and is not worth tracking ). If the checksum does not change, the unstable tree is searched for the candidate page. The processing of the unstable tree is somewhat different from that of the stable tree. First, if the search code does not find a page in the unstable tree, add a new node to the page in the unstable tree. However, if a page is found in the unstable tree, merge the page and migrate the node to the stable tree.

When the scan is completed (via KSM. c/Ksm_do_scan ()When executed), the stable tree is saved, but the unstable tree is deleted and re-built during the next scan. This process greatly simplifies the work, because the organization of the unstable tree can be changed according to the changes in the page (Do you still remember that the pages in the unstable tree are not write-protected ?). Since all pages in the stability tree are write-protected, a page failure is generated when a page tries to be written, allowing the cow process to cancel page merging for the writer (see KSM. c/Break_cow ()). The isolated page in the stability tree will be deleted later (unless two or more users of the page exist, it indicates that the page is still shared ).

As mentioned above, KSM uses the "red-black" tree to manage pages to support quick query. In fact, Linux contains some "red-black" trees as reusable data structures, which can be widely used. The red-black tree can also be used by completely fair scheduler (CFS) to store tasks in chronological order. You can find the implementation of the "red-black" tree in./lib/rbtree. C.

KSM configuration and monitoring

KSM management and monitoring are performed through sysfs (located at the root/sys/kernel/MM/KSM. In this sysfs subdirectory, you will find some files, some for control, and others for monitoring.

First fileRunUsed to enable and disable KSM page merging. By default, KSM is disabled (0), But you can1Write this file to enable the KSM daemon (for example,Echo 1> sys/kernel/MM/KSM/run). Write0You can disable the daemon from the running status (but keep the current set of the merged page ). In addition, by writing2From the running status (1) Stop KSM and request to cancel merging all merged pages.

When running KSM, you can control it through three parameters (files in sysfs. Sleep_millisecs file definition before executing another page ScanKsmdSleep in milliseconds. Max_kernel_pages file DefinitionKsmdMaximum number of pages that can be used (the default value is 25% of the available memory, but one0). Finally, the pages_to_scan file defines the number of pages that can be scanned during a given scan. Any user can view these files, but the user must have root permissions to modify them.

Five other monitoring files exported through sysfs (both read-only) indicateKsmd. The full_scans file indicates the number of times a full-area scan has been performed. The remaining four files indicate the page-level statistics of KSM:

• Pages_shared: Number of non-interchangeable kernel pages that KSM is using.
• Pages_sharing: memory storage indicator.
• Pages_unshared: number of unique pages that are checked repeatedly for merging.
• Pages_volatile: the number of frequently changed pages.

The author of KSM defines that a higher pages_sharing/pages_shared ratio indicates efficient page sharing (otherwise, it indicates a waste of resources ).

Conclusion

Linux does not use page sharing to improve memory efficiency. However, it is unique in implementing it as an operating system. VMware's ESX Server System Administrator named this feature transparent page sharing (TPS ), xen calls it memory cow . Regardless of the name and implementation, this feature provides a better memory utilization, allowing the Operating System (KVM System Management Program) to overuse the memory, supports more applications or VMS. You can find KSM-and many other interesting features in the latest 2.6.32 Linux kernel.