Linux HugePage features

Linux HugePage features

HugePage refers to the large page memory management mode. Compared with the traditional 4 kb common page management method, HugePage is more efficient for managing large memory (more than 8 GB. This article describes what is HugePage and some features of HugePage.

1. Introduction of Hugepage
The operating system's direct access to data from the physical memory is much faster than reading and writing data from the disk, but the physical memory is limited, which leads to the concept of physical memory and virtual memory. Virtual Memory is a strategy proposed to meet the shortage of physical memory. It is a logical memory virtualized by disk space. This part of disk space is called virtual memory in Windows, in Linux, it is called a Swap Space ).
 
For the management of this large memory (physical memory + virtual memory), most operating systems use segmented or paging management. Segmentation is a coarse-grained management method, while paging is a fine-grained management method, which can avoid the waste of memory space. Correspondingly, the concept of physical and virtual addresses in memory also exists. In the preceding two methods, the CPU must convert the virtual address to the physical memory address to truly access the memory. To improve the conversion efficiency, the CPU caches the ing between the nearest virtual memory address and the physical memory address and stores it in a CPU-maintained ing table. To increase the memory access speed as much as possible, you need to save as many mappings as possible in the ing table.
 
Linux memory management adopts a paging access mechanism. To ensure that the physical memory can be fully utilized, the kernel automatically switches memory pages that are not frequently used in the physical memory to the virtual memory according to the LRU algorithm when appropriate, and retains frequently used information to the physical memory. Generally, Linux uses 4 K entries per page by default, which means that if the physical memory is large, there will be a lot of ing table entries, which will affect the retrieval efficiency of the CPU. Because the memory size is fixed, to reduce the number of ing table entries, you can only increase the page size. So Hugepage came here. That is, to break the traditional Memory Management Mode of small pages, and use large pages such as 2 m, 4 m, and 16 m. In this way, the ing entries are significantly reduced. If the system has a large amount of physical memory (greater than 8 GB), Hugepage should be used for both 32-bit physical operating systems and 64-bit operating systems.
 
2. Hugepage Related Terms
Page Table:
A page table is the data structure of a virtual memory system in an operating system to store the mapping between virtual addresses and physical addresses. this means that on a virtual memory system, the memory is accessed by first accessing a page table and then accessing the actual memory location implicitly.
As mentioned above, page table is an implementation method for memory management, which is used for ing physical addresses to virtual addresses. Therefore, for memory access, first access the Page Table, and then implicitly transfer to the physical address to access the data according to the ing relationship in the Page Table.
 
TLB:
A Translation Lookaside Buffer (TLB) is a buffer (or cache) in a CPU that contains parts of the page table. This is a fixed size buffer being used to do virtual address translation faster.
A fixed cache in the CPU contains some page table ing relationships for fast conversion from virtual addresses to physical addresses.
 
Hugetlb:
This is an entry in the TLB that points to a HugePage (a large/big page larger than regular 4 K and predefined in size ). hugePages are implemented via hugetlb entries, I. e. we can say that a HugePage is handled by a "hugetlb page entry ". the 'hugetlb "term is also (and mostly) used synonymously with a HugePage (See Note 261889.1 ). in this document the term "HugePage" is going to be used but keep in mind that mostly "hugetlb" refers to the same concept.
Hugetlb is an entry pointing to HugePage in TLB (usually larger than 4 K or pre-defined page size ). HugePage is implemented through hugetlb entries. It can also be understood that HugePage is a handle of hugetlb page entry.
 
Hugetlbfs:
This is a new in-memory filesystem like tmpfs and is presented by 2.6 kernel. Pages allocated on hugetlbfs type filesystem are allocated in HugePages.
A new in-memory filesystem similar to tmpfs is proposed in the 2.6 kernel.
 
3. common error concepts
WRONG: HugePages is a method to be able to use large SGA on 32-bit VLM systems
RIGHT: HugePages is a method to have larger pages where it is useful for working with very large memory. It is both useful in 32-and 64-bit deployments

WRONG: HugePages cannot be used without USE_INDIRECT_DATA_BUFFERS
RIGHT: HugePages can be used without indirect buffers. 64-bit systems does not need to use indirect buffers to have a large buffer cache for the RDBMS instance and HugePages can be used there too.

WRONG: hugetlbfs means hugetlb
RIGHT: hugetlbfs is a filesystem type ** BUT ** hugetlb is the mechanic employed in the back where hugetlb can be employed WITHOUT hugetlbfs

WRONG: hugetlbfs means hugepages
RIGHT: hugetlbfs is a filesystem type ** BUT ** HugePages is the mechanic employed in the back (synonymously with hugetlb) where HugePages can be employed WITHOUT hugetlbfs.
 
4. Regular Pages and HugePages
A. Regular Pages
There are two different processes. The two processes access the memory first by accessing the local page table, the local page table references the page of the system-wide table (that is, the TLB described above). In the end, the entry in the system-wide table points to the actual physical address. In the figure, the size of the physical address page is 4 kb. You can also see that process 1 and process 2 point to page2 in system-wide table, that is, the same physical address. The usage of shared memory in Oracle sga will occur in the above situation.


B. Huge Pages
In, the local page table and system page table both contain the huge page attribute. Therefore, any page in the page table may use a regular page,
You may also use the huge page. Process 1 and process 2 Share Hpage2. In the figure, the regular page size of the physical memory is 4 kb, and the huge page size is 4 mb.

-- Author: Robinson
-- Blog: http://blog.csdn.net/robinson_0612
 
5. Size of the huge page
The size of the huge page depends on the kernel version of the operating system used and different hardware platforms.
You can use $ grep Hugepagesize/proc/meminfo to view the size of the huge page.
The following figure shows the size of the commonly used huge page on different platforms.
HW Platform Source Code Tree Kernel 2.4 Kernel 2.6
---------------------------------------------------------------
Linux x86 (IA32) i386 4 MB 4 MB *
Linux x86-64 (AMD64, EM64T) x86_64 2 MB 2 MB
Linux Itanium (IA64) ia64 256 MB 256 MB
IBM Power Based Linux (PPC64) ppc64/powerpc N/A ** 16 MB
IBM zSeries Based Linux s390 N/A 1 MB
Ibm s/390 Based Linux s390 N/
 
6. advantages of using huge page
Hugepage can greatly improve the performance of Oracle databases for large system memory and sga.

A. Not swappable
HugePages are not swappable. Therefore there is no page-in/page-out mechanisms overhead. HugePages are universally regarded as pinned.
No exchange is required. That is to say, there is no page swap out due to insufficient memory.

B. Relief of TLB pressure
Hugepge uses fewer pages to cover the physical address space, so the size of "book keeping" (mapping from the virtual to the physical address) decreases, so it requiring fewer entries in the TLB
TLB entries will cover a larger part of the address space when use HugePages, there will be fewer TLB misses before the entire or most of the SGA is mapped in the SGA
Fewer TLB entries for the SGA also means more for other parts of the address space
Reduces the TLB load, that is, reduces the address ing pressure that can be cached by the cpu cache. Because the huge page is used, the number of virtual addresses managed decreases with the same memory size.
The TLB entry can contain more address space, and the addressing capability of the cpu is enhanced accordingly.

C. Decreased page table overhead
Each page table entry can be as large as 64 bytes and if we are trying to handle 50 GB of RAM, the pagetable will be approximately 800 MB in size which is practically will not fit in 880 MB size lowmem (in 2.4 kernels-the page table is not necessarily in lowmem in 2.6 kernels) considering the other uses of lowmem. when 95% of memory is accessed via 256 MB hugepages, this can work with a page table of approximately 40 MB in total. see also Document 361468.1.
Reduces the load on the page table. For normal pages, each entry needs 64 bytes of memory. For 50 GB of memory, it takes MB to manage these entries.
(50*1024*1024) kb/4 kb * 64 bytes/1024/1024 = 800 mb.

D. Eliminated page table lookup overhead
Since the pages are not subject to replacement, page table lookups are not required. (eliminate page table lookup load)

E. Faster overall memory performance
On virtual memory systems each memory operation is actually two abstract memory operations. since there are fewer pages to work on, the possible bottleneck on page table access is clearly avoided. (improves the overall memory performance)
 
7. Risks of incorrect huge page Configuration
Based on the management of large memory (> 8 GB), if the configuration value or the correct configuration value huge page, the following uncertain hidden problems may exist:
HugePages not used (HugePages_Total = HugePages_Free) at all wasting the amount configured
Poor database performance
System running out of memory or excessive swapping
Some or any database instance cannot be started
Crucial system services failing (e.g.: CRS)
 
8. configuration steps based on the 2.6 kernel
The kernel parameter used for HugePages is vm. nr_hugepages which is based on the number of the pages. SLES9, RHEL4 and Asianux 2.0 are examples of distributions with the 2.6 kernel. for the configuration, follow steps below:
A. Start instance (s)
B. Calculate nr_hugepages using script from Document 401749.1
C. Set kernel parameter:
# Sysctl-w vm. nr_hugepages = <value from above>
And make sure that the parameter is persistent to reboots. e.g. On SLES9:
# Chkconfig boot. sysctl on
D. Check available hugepages:
$ Grep Huge/proc/meminfo
E. Restart instances
F. Check available hugepages:
$ Grep Huge/proc/meminfo
 
9. Notes
A. HugePage uses shared memory, which is dynamically allocated and retained during operating system startup because they are not replaced.
B. Because it is not replaced, hugepage memory cannot be used by other processes. Therefore, set this value properly to avoid Memory waste.
C. For servers that only use Oracle, set Hugepage to SGA (sum of all instance SGA.
D. If HugePage is added, physical memory is added, or new instances and SGA instances are added to the current server, set the required HugePage again.
E. reference: HugePages on Linux: What It Is... and What It Is Not... [ID 361323.1] To Bottom