Configure hugepage and linuxhugepage in Linux

Configure hugepage and linuxhugepage in Linux

For Oracle database servers with large memory, it is necessary to configure HugePage. The following describes how to configure Huge:

My environment:

OS: CentOS release 6.4 (Final)

Oracle: 11.2.0.4 RAC
1. Set memlock

Edit/etc/security/limits. conf

Added:

* Soft memlock is slightly smaller than the RAM Value

* Hard memlock is slightly smaller than the RAM Value

It does not matter if the Memlock value is greater than sga, so we can set this value between the desired SGA size and the physical memory size, which is measured in kb.

[Root @ node1 ~] # Grep-I memtot/proc/meminfo

MemTotal: 132250576 kB

The physical memory size is 126 GB, so I can set memlock to 100 GB.

Set in two nodes:

[Root @ node1 ~] # Tail-2/etc/security/limits. conf

* Soft memlock 104857600

* Hard memlock 104857600

After the settings are complete, log on to the Oracle user for verification:

Node2-> ulimit-l

104857600

2. Disable the AMM feature:

To disable the AMM feature, you need to reset the memory_max_target and memory_max_targetparameters, instead of setting them to 0. The best way is to create a pfile and delete these two parameters in pfile, create a spfile based on this pfile.

The main memory parameters are as follows:

Node1->

Node1-> grep target/tmp/init. ora

*. Pga_aggregate_target = 1572864000

*. Sga_target = 64424509440

SQL> create spfile from pfile = '/tmp/init. ora ';

 

File created.

 

SQL> startup force

ORA-27102: out of memory

Linux-x86_64 Error: 28: No space left on device

Additional information:-134217728

Additional information: 1

SQL>

This error is reported because the shmall setting is too small.

Shmall indicates the total size of the shared memory, in the unit of pagesize.

According to my configuration, SGA with a maximum of 8 GB is supported:

Node1-> getconf PAGE_SIZE

4096

Node1-> cat/proc/sys/kernel/shmall

2097152

Modify shmall and restart the database. The modification method is simple. edit/etc/sysctl. conf:

Kernel. shmall = 16252928

You can.

After modifying kernel. shmall, You can restart the database.

SQL> startup nomount pfile = '/tmp/init. ora ';

ORACLE instance started.

 

Total System Global Area 6.4137E + 10 bytes

Fixed Size 2269072 bytes

Variable Size 6979321968 bytes

Database Buffers 5.7043E + 10 bytes

Redo Buffers 112762880 bytes

SQL>

SQL>

After the DB is started, re-create the spfile:

SQL> create spfile = '+ OCR/sszgdb/spfilesszgdb. ora' from pfile = '/tmp/init. ora ';

 

File created.

 

SQL>

3. Calculate the hugepage Size Based on the Oracle script (make sure that Oracle has been started During computation). You can also calculate the size by yourself:

 

 

Press Enter to proceed...

 

Recommended setting: vm. nr_hugepages = 30739

Node1->

4. edit/etc/sysctl. conf Based on the computing value of the script on two nodes.

Join vm. nr_hugepages = 30739

[Root @ node2 ~] # Grep-I huge/proc/meminfo

AnonHugePages: 602112 kB

HugePages_Total: 30739

HugePages_Free: 30739

HugePages_Rsvd: 0

HugePages_Surp: 0

Hugepagesize: 2048 kB to stop a database instance

5. Restart the server. Generally, the DB is automatically started after the server is restarted. After the server and DB are restarted, run the following command to verify the usage of the large page:

Grep-I Hugepages/proc/meminfo

In this case, you can view the Oracle alert Log and find:

Starting ORACLE instance (normal)

* ********************** Large Pages Information ************* ******

Per process system memlock (soft) limit = UNLIMITED

 

Total Shared Global Region in Large Pages = 58 GB (96%)

 

Large Pages used by this instance: 29761 (58 GB)

Large Pages unused system wide = 24 (48 MB)

Large Pages configured system wide = 30739 (60 GB)

Large Page size = 2048 KB

 

RECOMMENDATION:

Total System Global Area size is 60 GB. For optimal performance,

Prior to the next instance restart:

1. Increase the number of unused large pages

At least 936 (page size 2048 KB, total size 1872 MB) system wide

Get 100% of the System Global Area allocated with large pages

We can find that sga is not all placed on a large page, which may have a negative impact on performance. We recommend that you add another 936 large page in Oracle. It can be seen that the large page script computing in Oracle is not accurate, we need to increase the page size and set the parameter Use_large_pages = only. This parameter ensures that all the Oracle shared memory uses a large page.

Restart the database again. The alert Log is as follows:

* ********************** Large Pages Information ************* ******

Parameter use_large_pages = ONLY

Per process system memlock (soft) limit = 100 GB

 

Total Shared Global Region in Large Pages = 60 GB (100%)

 

Large Pages used by this instance: 30721 (60 GB)

Large Pages unused system wide = 0 (0 KB)

Large Pages configured system wide = 31675 (62 GB)

Large Page size = 2048 KB

We can see that all sga uses big pages.

Make sure that the environment variable DISABLE_HUGETLBFS is unset. This variable will disable Oracle's use of Hugepage.

 

 

Appendix: scripts used to calculate the HugePage size:

#! /Bin/bash

#

# Hugepages_settings.sh

#

# Linux bash script to compute values for

# Recommended HugePages/HugeTLB configuration

#

# Note: This script does calculation for all shared memory

# Segments available when the script is run, no matter it

# Is an Oracle RDBMS shared memory segment or not.

#

# This script is provided by Doc ID 401749.1 from My Oracle Support

# Http://support.oracle.com

 

# Welcome text

Echo"

This script is provided by Doc ID 401749.1 from My Oracle Support

(Http://support.oracle.com) where it is intended to compute values

The recommended HugePages/HugeTLB configuration for the current shared

Memory segments. Before proceeding with the execution please note following:

* For ASM instance, it needs to configure ASMM instead of AMM.

* The 'pga _ aggregate_target 'is outside the SGA and

You shoshould accommodate this while calculating SGA size.

* In case you changes the db sga size,

As the new SGA will not fit in the previous HugePages configuration,

It had better disable the whole HugePages,

Start the DB with new SGA size and run the script again.

And make sure that:

* Oracle Database instance (s) are up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not setup

(See Doc ID 749851.1)

* The shared memory segments can be listed by command:

# Ipcs-m

 

 

Press Enter to proceed ..."

 

Read

 

# Check for the kernel version

KERN = 'uname-r | awk-F. '{printf ("% d. % d \ n", $1, $2 );}''

 

# Find out the HugePage size

HPG_SZ = 'grep Hugepagesize/proc/meminfo | awk '{print $2 }''

If [-z "$ HPG_SZ"]; then

Echo "The hugepages may not be supported in the system where the script is being executed ."

Exit 1

Fi

 

# Initialize the counter

NUM_PG = 0

 

# Cumulative number of pages required to handle the running shared memory segments

For SEG_BYTES in 'ipcs-m | cut-c44-300 | awk '{print $1}' | grep "[0-9] [0-9] *"'

Do

MIN_PG = 'echo "$ SEG_BYTES/($ HPG_SZ * 1024)" | bc-Q'

If [$ MIN_PG-gt 0]; then

NUM_PG = 'echo "$ NUM_PG + $ MIN_PG + 1" | bc-Q'

Fi

Done

 

RES_BYTES = 'echo "$ NUM_PG * $ HPG_SZ * 1024" | bc-Q'

 

# An SGA less than 100 MB does not make sense

# Bail out if that is the case

If [$ RES_BYTES-less than 100000000]; then

Echo "***********"

Echo "** ERROR **"

Echo "***********"

Echo "Sorry! There are not enough total of shared memory segments allocated

HugePages configuration. HugePages can only be used for shared memory segments

That you can list by command:

 

# Ipcs-m

 

Of a size that can match an Oracle Database SGA. Please make sure that:

* Oracle Database instance is up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not configured"

Exit 1

Fi

 

# Finish with results

Case $ KERN in

'2. 2') echo "Kernel version $ KERN is not supported. Exiting .";;

'2. 4') HUGETLB_POOL = 'echo "$ NUM_PG * $ HPG_SZ/1024" | bc-Q ';

Echo "Recommended setting: vm. hugetlb_pool = $ HUGETLB_POOL ";;

'2. 6') echo "Recommended setting: vm. nr_hugepages = $ NUM_PG ";;

'3. 8') echo "Recommended setting: vm. nr_hugepages = $ NUM_PG ";;

Esac

 

# End