) Hugepage Introduction

1. Related Concepts

Hugepage/big page:

The system process accesses the memory through a virtual address, but the CPU must convert it to the physical memory address to truly access the memory. To improve the conversion efficiency, the CPU caches the ing between the nearest virtual memory address and the physical memory address and stores it in a CPU-maintained ing table. To increase the memory access speed as much as possible, you need to save as many mappings as possible in the ing table.

In RedHat Linux, memory is divided into pages. By default, each page is 4 kb, which means that if the physical memory is large, there will be a lot of entries in the ing table, which will affect the retrieval efficiency of the CPU. Because the memory size is fixed, to reduce the number of ing table entries, you can only increase the page size. This increase in memory page size is called Big page in Linux 2.1; In as 3/4, it is called hugepage.

If the system has a large amount of physical memory (greater than 8 GB), hugepage should be used for both 32-bit physical operating systems and 64-bit operating systems.

Note: The hugepage memory is shared memory, which keeps the keep in the memory and will not be swapped out. That is to say, the memory using hurgepage cannot be used by other processes, set this value properly to avoid waste. For servers that only use Oracle, set hugepage_pool to the SGA size.

Vlm (very large memory): this parameter is intended for 32-bit operating systems. For 64-bit operating systems, you need to set vlm. With hugepage enabled, 32-bit oracle can extend SGA to 62 GB. Note that vlm is only valid for buffer cache in SGA and is invalid for shared pool, large pool, and Java pool.

The principle of vlm is to virtualize the memory into a file. The system process reads the memory file to use the memory.

If Oracle wants to use vlm, The use_indirect_data_buffers parameter must be set to true. For a 10 Gb database, you also need to convert db_cache_size to the old version of db_block_buffers. Otherwise, an error is returned.

When SGA uses vlm, the shared memory of SGA is divided into two parts:

. Common system shared memory, that is, the part that can be viewed from IPCS-Ma. This part mainly corresponds to the SGA (large pool/Shared Pool/Java pool/streams pool) of non-buffer cache).

. Shared Memory Based on memory files, which can be viewed through LS-Al/dev/SHM. This part mainly corresponds to the data buffer section in the SGA.

Note: When vlm is used, the memory used for the non-buffer cache part will retain m for vlm management. For example, if GB is allocated to a non-buffer cache, the actual available memory is only 2 GB.

When vlm is used, the sum of the shared memory of the above two parts is equal to SGA. (If vlm is not used, the SGA size is basically the same as that displayed by IPCS-ma)

The following is an example of the relationship between the four:

1. SGA Problems

SQL> show SGA

Total system global area 1879048192 bytes

Fixed size 778452 bytes

Variable Size 802430764 bytes

Database buffers 1073741824 bytes

Redo buffers 2097152 bytes

----- Non-buffer cache part

SQL> show parameter shared_pool_size

Name type value

-----------------------------------------------------------------------------

Shared_pool_size big integer 512 m

SQL> show parameter java_pool_size

Name type value

-----------------------------------------------------------------------------

Java_pool_size big integer 32 m

SQL> show parameter large_pool_size

Name type value

-----------------------------------------------------------------------------

Large_pool_size big integer 128 m

SQL> show parameter streams_pool_size

Name type value

-----------------------------------------------------------------------------

Streams_pool_size big integer 80 m

SQL> show parameter log_buffer

Name type value

-----------------------------------------------------------------------------

Log_buffer integer 2097152

----- Buffer cache

SQL> show parameter block_size

Name type value

-----------------------------------------------------------------------------

Db_block_size integer 8192

SQL> show parameter db_block_buffers

Name type value

-----------------------------------------------------------------------------

Db_block_buffers integer 131072

2. Size of IPCS display

$ IPCS-

------ Shared memory segments --------

Key shmid owner perms bytes nattch status

0xf258e130 32769 Oracle 600 807403520 0

0x00000000 65538 Oracle 640 4096 0

------ Semaphore arrays --------

Key Semid owner perms nsems

0x610520f4 98304 Oracle 640 602

0x610520f5 131073 Oracle 640 602

0x610520f6 163842 Oracle 640 602

------ Message queues --------

Key msqid owner perms used-bytes messages

3./dev/SHM size

$ LS-Al/dev/SHM

Total 120

Drwxrwxrwt 1 Root 0 Feb 15 02: 11.

Drwxr-XR-x 22 Root 118784 Feb 15 ..

-RW-r ----- 1 Oracle DBA 1073741824 Feb 15 ora_test_65538

(Shared_pool_size + java_pool_size + large_pool_size + streams_pool_size + log_buffer) = (512 + 32 + 128 + 80) * 1024*1024 + 2097152 = 790626304 and IPCS result 807403520 are basically the same.

131072*8192 = 1073741824 and LS-Al/dev/SHM result 1073741824 are equal.

The results also validate the above conclusions.

Ii. Configure hugepage

Because the shared memory of hugepage cannot be swap or used by other processes, if the shared memory of hugepage is set too large, the system hang will be occupied.

Considering both performance and memory waste, we must set the best value for the shared memory that uses hugepage. The following are some references:

1) if it is a 32-bit operating system and vlm is not used, set hugetlb_pool = sum of all instances running on the server. If there is ASM, each ASM instance will be increased by about MB.

2) if it is a 32-bit operating system and vlm is used, set hugetlb_pool = sum of all instances running on the server except data buffer. If there is ASM, each ASM instance will be increased by about MB.

3) For a 64-bit operating system, set hugetlb_pool = the sum of all instances running on the server, regardless of whether or not vlm is enabled. If there is ASM, each ASM instance will be increased by about MB.

The methods for setting hugepage in RHEL 2.1/3/4 are different because version 2.1 is too low and will not be introduced here.

1. Set hugepage in rhel3

You just need to add the following lines in/etc/sysctl. conf:

# Set 1024 M hugepage momory

VM. hugetlb_pool = 1024

Run sysctl-P to make the modification take effect.

In this case, you can verify whether the settings take effect from/proc/meminfo:

$ More/proc/meminfo | grep-I hugepage

Hugepages_total: 512

Hugepages_free: 512

Hugepagesize: 2048 KB

The 512 MB hugepage_pool is set to 2 MB per page, so there will be pages.

Ideally, when the Oracle instance is started, hugepages_free should be equal to or close to 0.

2. Set hugepage in RHEL4

Similar to rhel3, it is easy to configure hugepage in RHEL4,

1) Add the following lines in/etc/sysctl. conf:

# Set 1024 M hugepage momory

VM. nr_hugepages = 512

In RHEL4, the number of hugepage pages is set directly.

Run sysctl-P to make the modification take effect.

2) Add the following lines in/etc/security/limits. conf:

Oracle soft memlock 1048576

Oracle hard memlock 1048576

You must set this parameter. Otherwise, an error may occur when you start the database:

ORA-27103: Internal error

Linux error: 11: Resource temporarily unavailable

In this case, you can verify whether the settings take effect from/proc/meminfo:

$ More/proc/meminfo | grep-I hugepage

Hugepages_total: 512

Hugepages_free: 512

Hugepagesize: 2048 KB

Ideally, when the Oracle instance is started, hugepages_free should be equal to or close to 0.

If you do not want to set hugepage, set VM. nr_hugepages = 0.

Pay attention to the following points:

1) No matter rhel3 or RHEL4, if hugepage_pool or nr_hugepages is set, the memory of the specified size is pin in the memory. Even if the shared memory required by SGA is smaller than the set hugepage_pool, this part of the memory cannot be used by other processes. Therefore, you must calculate the required size. It is not recommended to set a large hugepage shared memory, avoid waste.

3. Configure vlm

In rhel3 and RHEL4, you can configure vlm using two memory file methods:

. Shmfs/tmpfs: This memory file mode will change pages and conflict with hugepage. It is not applicable to hugepage memory management. Here, shmfs only applies to rhel3, and tempfs applies to rhel3 and RHEL4.

. Ramfs: This method does not change pages. It can be used with hugepage.

To configure vlm, the system kernel must support one of the above three file systems. You can use the following command to determine whether:

Egrep "SHM | tmpfs | ramfs"/proc/filesystems

If the results have corresponding entry output, it indicates that this type of file system is supported, such:

Nodev tmpfs

Nodev ramfs

Indicates that the memory supports tmpfs and ramfs memory file systems.

The following uses a 32-bit database with 8 GB buffer cache as an example to describe how to configure vlm in rhel3/4 (ensure Kernel support and hugepage has been configured first)

1. Mount the memory File System

Umount/dev/SHM

Mount-T ramfs/dev/SHM

Chown ORACLE: DBA/dev/SHM

Note:

1) In order to take effect after restart, it is best to write the above lines in/etc/rc. Local.

2) If the Oracle user's master group is not dBA, if it is oinstall, chown ORACLE: oinstall/dev/SHM

2. modify Oracle Parameters

Use_indirect_data_buffers = true

DB _ block_size = 8192

Db_block_buffers = 1048576

Shared_pool_size = 2831155200

3. modify Oracle resource limits

Set memlock = 3145728 in/etc/security/limits. conf, that is, add the following two lines to the file:

Oracle soft memlock 3145728

Oracle hard memlock 3145728

Verify with ulimit-l

When using vlm:

1) No matter what version, the db_cache_size parameter cannot be used and must be converted to the corresponding db_block_buffers and db_block_size

2) The shared memory of the data buffer part comes from vlm, but other parts of the SGA, such as the shared memory used by the shared_pool, come from the common shared memory of the system.

3) The setting of kernel. shmmax should be greater than or equal to the sum of the SGA parts other than data buffer.

4) the operating system automatically generates a memory file of the same size as the data buffer setting.

5) Even if hugepage is large enough, the shared memory required by data buffer will not be allocated from hugepage_pool. Instead, the shared memory segment will be extended using vlm memory files.

6) Even if hugepage is insufficient, the shared memory required by the SGA, which is not part of the data buffer, will not be allocated from the vlm. Instead, the shared memory segment will be extended using other idle memory of the system.

Finally, let's make a summary:

1. hugepage is used to improve memory performance. vlm is used to enable 32-bit operating systems to use large memory.

2. If the server has a large amount of physical memory:

For a 64-bit operating system, you only need to set hugepage, and you can use the new parameter db_cache_size.

For a 32-bit operating system, you must configure hugepage and vlm, and do not use the new db_cache_size parameter. Hugepage_pool is set to the total memory size of the non-buffer cache. You do not need to set the shared memory part corresponding to the buffer cache. The OS will automatically allocate the appropriate size of memory files as the shared memory.