Install Oracle 11g R2 + RAC + ASM in Oracle Linux 6.4

Install Oracle 11g R2 + RAC + ASM in Oracle Linux 6.4

I. Version description
Many articles related to the Internet are found, but they are not very specific. Therefore, the detailed version is used to make the article more practical.
This is the RAC environment of Oracle 11g R2 11.2.0.4. The operating system version is Oracle Linux 6.4.

1. [Root @ node1 ~] # Lsb_release-
2. LSB Version: base-4.0-amd64: base-4.0-noarch: core-4.0-amd64: core-4.0-noarch: graphics-4.0-amd64: graphics-4.0-noarch: printing-4.0-amd64: printing-4.0-noarch:
3. Distributor ID: OracleServer
4. Description: Oracle Linux Server release 6.4
5. Release: 6.4
6. Codename: n/

Kernel version:

1. [Root @ node1 ~] # Uname-r
2. 2.6.39-400.17.1.el6uek.x86 _ 64

UEK stands for Unbreakable Enterprise Kernel. It is a Kernel customized by Oracle for Oracle Linux and can provide online support and database support. It is one of the selling points of OracleLinux. (Some configuration procedures in this article will be simplified by using the UEK version)

Ii. Steps for installing Oracle 11g R2 RAC

1. Server preparation (system, storage, and network administration ):

• Install the operating system and upgrade it to a version that meets the Oracle installation requirements.
• Create the group, user, and software home directory required for installation.
• Set GNS domain name if you plan to deploy GNS and configure the network address on the DNS and server.
• Set the required storage.
• Copy all the installation files to one node.

2. Install the infrastructure of the Oracle Grid cluster, including Oracle Clusterware and Oracle ASM (system and storage administration ):

• Install the Oracle Grid infrastructure software for the cluster. During the installation process, the Fixup script makes additional adjustments to parameters such as operating system parameters, SSH, and user environment variables.
• Upgrade Oracle Clusterware and Oracle ASM to the latest patch.

3. Install Oracle RAC (database administration ):

• Installing Oracle RAC
• Patch to the latest version.
• Debug after installation

Iii. Detailed installation process and instructions (refer to the official documentation)

1. Use SecureCRT or TerminalX to establish a command line connection.

2. Add the users, groups, and home directories for installing Oracle Grid on each node, and set permissions.

1. #/Usr/sbin/groupadd-g 1000 oinstall
2. #/Usr/sbin/groupadd-g 1020 asmadmin
3. #/Usr/sbin/groupadd-g 1021 asmdba
4. #/Usr/sbin/groupadd-g 1022 asmoper
5. #/Usr/sbin/groupadd-g 1031 dba
6. #/Usr/sbin/groupadd-g 1032 bytes
7. # Useradd-u 1100-g oinstall-G asmadmin, asmdba, asmoper, role, dba grid
8. # Useradd-u 1101-g oinstall-G dba, asmdba, external oracle
9. # Mkdir-p/u01/app/11.2.0/grid
10. # Mkdir-p/u01/app/grid
11. # Chown-R grid: oinstall/u01
12. # Mkdir/u01/app/oracle
13. # Chown oracle: oinstall/u01/app/oracle
14. # Chmod-R 775/u01/

Refer to the official document to adopt separate GI and DB installation and permission policies, which is advantageous for multi-instance management.

3. Check whether the configuration of each node on the server meets the installation requirements.

I. Requirements for server hard disk space
/Tmp directory size: at least 1 GB
Space required to install Grid Infrastracture: 6.6 GB
Space required for installing Oracle Database: 4 GB
Space required for analyzing, collecting, and tracking files during installation: 10 GB
We recommend a total of at least 30 GB. Don't worry! (The space requirements for ASM or NFS are not included here)

II. Server Memory Requirements

Memory size: at least 2.5 GB
Swap size:
When the memory is GB-16 GB, Swap must be greater than or equal to the system memory.
If the memory is larger than 16 GB, Swap is equal to 16 GB.

III. Check and debug code

• The command to view the memory and Swap size is as follows.

1. # Grep MemTotal/proc/meminfo
2. # Grep SwapTotal/proc/meminfo

• Run the following command to view the/tmp directory and configure an independent lv.

1. # Df-h/tmp
2. # Lvcreate-L 2G-n lv_tmp vg_temp
3. # Mount/dev/vg_temp/lv_tmp/tmp
4. # Df-h/tmp

4. Set Operating System Parameters

This is a tedious task in non-Oracle Linux operating systems. However, the version we installed this time is Oracle Linux 6.4 with Unbreakable Enterprise Kernel, therefore, we have a simpler method, that is, by installing Oracle Preinstallation RPM to adjust the relevant operating system parameters and install the software package.
This installation package mainly completes the following tasks:

• Automatically downloads and installany additional RPM packages needed for installing Oracle Grid Infrastructure and Oracle Database, and resolves any dependencies

• Creates an oracle user, and creates the oraInventory (oinstall) and OSDBA (dba) groups for that user

• As needed, sets sysctl. conf settings, system startup parameters, and driver parameters to values based on recommendations from the Oracle Preinstallation RPM program

• Sets hard and soft resource limits

• Sets other recommended parameters, depending on your kernel version

The installation package is located in the Packages directory of the operating system installation disk.

Click (here) to fold or open

1. # Cd/mnt/install_DVD
2. # Cd Packages
3. # Ll | grep preinstall
4. -Rw-r -- 1 root 15524 Jan 16 2013 oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64.rpm
5. # Rpm-ivh oracle-rdbms-server-11gR2-preinstall-1.0-7.el6.x86_64.rpm

Modify/etc/sysctl. conf in the installation package as follows:

The parameters marked with the oracle-rdbms-server-11gR2-preinstall are the parameters added to the installation package.

1. # Cat/etc/sysctl. conf
2. # Kernel sysctl configuration file for Red Hat Linux
3. #
4. # For binary values, 0 is disabled, 1 is enabled. See sysctl (8) and
5. # Sysctl. conf (5) for more details.
6. # Controls IP packet forwarding
7. Net. ipv4.ip _ forward = 0
8. # Controls source route verification
9. Net. ipv4.conf. default. rp_filter = 1
10. # Do not accept source routing
11. Net. ipv4.conf. default. accept_source_route = 0
12. # Controls the System Request debugging functionality of the kernel
13. Kernel. sysrq = 0
14. # Controls whether core dumps will append the PID to the core filename.
15. # Useful for debugging multi-threaded applications.
16. Kernel. core_uses_pid = 1
17. # Controls the use of TCP syncookies
18. Net. ipv4.tcp _ syncookies = 1
19. # Disable netfilter on bridges.
20. Net. bridge. bridge-nf-call-ip6tables = 0
21. Net. bridge. bridge-nf-call-iptables = 0
22. Net. bridge. bridge-nf-call-arptables = 0
23. # Controls the default maxmimum size of a mesage queue
24. Kernel. msgmnb = 65536
25. # Controls the maximum size of a message, in bytes
26. Kernel. msgmax = 65536
27. # Controls the maximum shared segment size, in bytes
28. # Controls the maximum number of shared memory segments, in pages
29. # Oracle-rdbms-server-11gR2-preinstall setting for fs. file-max is 6815744
30. Fs. file-max = 6815744
31. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. sem is '2017 250 32000 100'
32. Kernel. sem = 250 32000 100 128
33. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. shmmni is 4096
34. Kernel. shmmni = 4096
35. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. shmall is 1073741824 on x86_64
36. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. shmall is 2097152 on i386
37. Kernel. shmall = 1073741824
38. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. shmmax is 4398046511104 on x86_64
39. # Oracle-rdbms-server-11gR2-preinstall setting for kernel. shmmax is 4294967295 on i386
40. Kernel. shmmax = 4398046511104
41. # Oracle-rdbms-server-11gR2-preinstall setting for. net. core. rmem_default is 262144
42. Net. core. rmem_default = 262144
43. # Oracle-rdbms-server-11gR2-preinstall setting for net. core. rmem_max is 4194304
44. Net. core. rmem_max = 4194304
45. # Oracle-rdbms-server-11gR2-preinstall setting for net. core. wmem_default is 262144
46. Net. core. wmem_default = 262144
47. # Oracle-rdbms-server-11gR2-preinstall setting for net. core. wmem_max is 1048576
48. Net. core. wmem_max = 1048576
49. # Oracle-rdbms-server-11gR2-preinstall setting for fs. aio-max-nr is 1048576
50. Fs. aio-max-nr = 1048576
51. # Oracle-rdbms-server-11gR2-preinstall setting for net. ipv4.ip _ local_port_range is 9000 65500
52. Net. ipv4.ip _ local_port_range = 9000 65500

5. Configure the Oracle Network

I. hardware configuration requirements

• Each server node requires at least two NICs, one external network interface and one private network interface (Heartbeat ).
• If you install the Oracle cluster software through OUI, ensure that each node is used for the Internet or private network interface (NIC name) to ensure consistency. For example, if node1 uses eth0 as the external interface, node2 cannot use eth1 as the external interface.

II. IP Configuration Requirements

• DHCP service for the cluster exists

• DHCP can provide sufficient IP addresses, one virtual IP address and three scan ip addresses for each node.

III. manually configure IP instances in non-GNS


6. Check the operating system software package
First, find the installation package requirements list in the official documentation based on the operating system version, as shown in

You can check and install the required package by mounting the disk or directly using the yum Source:

1. # Rpm-qa packages_name
2. # Rpm-ivh packages_name
4. # Yum list packages_name
5. # Yum install packages_name

Here we will also mention that the CVUdisk package can also be installed together at this time for later use of CVU.

1. # Check whether the installation is complete
2. # Rpm-qi cvuqdisk
3. # Uninstall the previous version if it has been installed
4. # Rpm-e cvuqdisk
5. # Install the new version
6. # CVUQDISK_GRP = oinstall; export CVUQDISK_GRP
7. # Rpm-ivh cvuqdisk-1.0.9-1.rpm

7. Disable NTP and modify port range Parameters

1. # Oracle recommends using the Oracle Cluster Time Synchronization Service. Therefore, disable NTP deletion.
2. #/Sbin/service ntpd stop
3. # Chkconfig ntpd off
4. # Rm/etc/ntp. conf (mv/etc/ntp. conf/etc/ntp. conf. old)
5. # Rm/var/run/ntpd. pid
7. Check the TCP/UDP port range
8. # Cat/proc/sys/net/ipv4/ip_local_port_range
9. If 9000 65500 is displayed, skip the following steps.
10. # Echo 9000 65500>/proc/sys/net/ipv4/ip_local_port_range
11. # Vim/etc/sysctl. conf
12. # Add this row:
13. # TCP/UDP port range
14. Net. ipv4.ip _ local_port_range = 9000 65500
15. # Restarting the network
16. #/Etc/rc. d/init. d/network restart

8. Adjust the. bash_profile content and set the display address.

1. Set parameters:
2. $ Su-root
3. # Mkdir/mount_point/tmp
4. # Chmod 775/mount_point/tmp
5. # Exit
7. $ Vi. bash_profile
8. Add row:
9. TEMP =/mount_point/tmp
10. TMPDIR =/mount_point/tmp
11. Export TEMP TMPDIR
12. Add row: umask 022
13. Add row: DISPLAY = local_IP: 0.0; export DISPLAY
14. Local_IP: The desktop address displayed in the installation View
16. Check setting parameters
17. $ Umask
18. $ Env | more
19. $ Echo $ DISPLAY
20. $ Echo $ TEMP
21. $ Echo $ TMPDIR

9. Configure SSH mutual trust
This is a key step. Although the official documents claim that OUI will automatically configure SSH when GI and RAC are installed, it is better to manually configure mutual trust by using CVU to check configurations before installation.

1. The configuration process is as follows:
2. Each node generates Keys:
3. [Root @ rac1 ~] # Su-oracle
4. [Oracle @ rac1 ~] $ Mkdir ~ /. Ssh
5. [Oracle @ rac1 ~] $ Chmod 700 ~ /. Ssh
6. [Oracle @ rac1 ~] $ Ssh-keygen-t rsa
7. [Oracle @ rac1 ~] $ Ssh-keygen-t dsa
8. [Root @ rac2 ~] # Su-oracle
9. [Oracle @ rac2 ~] $ Mkdir ~ /. Ssh
10. [Oracle @ rac2 ~] $ Chmod 700 ~ /. Ssh
11. [Oracle @ rac2 ~] $ Ssh-keygen-t rsa
12. [Oracle @ rac2 ~] $ Ssh-keygen-t dsa
14. Configure mutual trust on node 1:
15. [Oracle @ rac1 ~] $ Touch ~ /. Ssh/authorized_keys
16. [Oracle @ rac1 ~] $ Cd ~ /. Ssh
17. [Oracle @ rac1. ssh] $ ssh rac1 cat ~ /. Ssh/id_rsa.pub> authorized_keys
18. [Oracle @ rac1. ssh] $ ssh rac2 cat ~ /. Ssh/id_rsa.pub> authorized_keys
19. [Oracle @ rac1. ssh] $ ssh rac1 cat ~ /. Ssh/id_dsa.pub> authorized_keys
20. [Oracle @ rac1. ssh] $ ssh rac2 cat ~ /. Ssh/id_dsa.pub> authorized_keys
22. In rac1, the verification file storing the public key information is transferred to rac2.
23. [Oracle @ rac1. ssh] $ pwd
24. /Home/oracle/. ssh
25. [Oracle @ rac1. ssh] $ scp authorized_keys rac2: 'pwd'
26. Oracle @ rac2's password:
27. Authorized_keys 100% 1644 1.6KB/s
29. Set authentication File Permissions
30. Run the following command on each node:
31. $ Chmod 600 ~ /. Ssh/authorized_keys
33. Enable User consistency
34. Run the OUI node as an oracle user (rac1 is selected here ):
35. [Oracle @ rac1. ssh] $ exec/usr/bin/ssh-agent $ SHELL
36. [Oracle @ rac1. ssh] $ ssh-add
37. Identity added:/home/oracle/. ssh/id_rsa (/home/oracle/. ssh/id_rsa)
38. Identity added:/home/oracle/. ssh/id_dsa (/home/oracle/. ssh/id_dsa)
40. Verify that the ssh configuration is correct
41. Run the following commands on all oracle nodes:
42. Ssh rac1 date
43. Ssh rac2 date
44. Ssh rac1-priv date
45. Ssh rac2-priv date
47. If you do not need to enter the password, the time can be output, indicating that the ssh authentication configuration is successful. You must run the preceding commands on both nodes. For each command, you must enter yes during the first execution.
48. If you do not run these commands, an error occurs when you install clusterware even if the ssh authentication has been configured:
49. The specified nodes are not clusterable
50. After ssh is configured, you must enter "yes" during the first access to access other servers.

Note that SSH mutual trust requires that SSH access can be performed between nodes without a password.

10. Configure storage for the basic software of the Grid cluster (Using ASM)
Oracle allows storage of ASM, NFS, and ACFS. Here we only introduce the configuration process of ASM.
(1) install the ASM configuration software.

1. It can still be installed through yum or system DVD
2. # Cd/mnt/install_DVD/Packages
3. # Rpm-qi oracleasm
4. # Rpm-qi oracleasm-support
5. # Rpm-qi oracleasm-support-2.1.8-1.el6.x86_64.rpm
6. // We are an Oracle Linux system. This version has been integrated with oracleasm and oracleasmlib, so you do not need to install it again ~

(2) Plan the ASM disk group
The official documentation specifies the sizes required for OCR, Voting Disk, Database, and Recovery under different redundancy policies.


Find the storage administrator and configure all the disks for you ~

(3) Configure ASM

1. Format each disk
2. [Root @ rac1 ~] # Fdisk/dev/sdb
4. After that, use fdisk-l to view the information.
5. [Root @ rac1 ~] # Fdisk-l
7. [Root @ rac1 ~] #/Usr/sbin/oracleasm configure-I
8. Default user to own the driver interface []: grid
9. Default group to own the driver interface []: asmadmin
10. Start Oracle ASM library driver on boot (y/n) [n]: y
11. Scan for Oracle ASM disks on boot (y/n) [y]: y
13. [Root @ rac1 ~] #/Etc/init. d/oracleasm createdisk VOL1/dev/sdb1
14. [Root @ rac1 ~] #/Etc/init. d/oracleasm createdisk VOL2/dev/sdc1
15. [Root @ rac1 ~] #/Etc/init. d/oracleasm createdisk VOL3/dev/sdd1
17. The following operations are performed on another node and on RAC2:
18. [Root @ rac2 ~] # Oracleasm scandisks
19. Here we use asmlib for disk device management. Here we can also use UDEV for management. In RHEL6, asmlib has been deprecated and will be the world of UDEV.

11. Use CVU to check the configuration of each node
This is the last step before installation ~

1. [Root @ node1 ~] # Su-grid
2. [Grid @ node1 ~] $ Cd grid_sw
3. [Grid @ node1 grid_sw] $./runcluvfy. sh stage-pre crsinst-n node1, node2-fixup-verbose
4. "Grid_sw" is the grid installation file extracted from the installation package.

This step focuses on how to handle errors:
I encountered a PRVF-5636 for this installation, which was caused by no DNS and/etc/resolv. conf settings, because the script uses the nslookup command to detect connectivity between the two nodes.

12. Install GI and RAC
(1) install Oracle Grid Infrastructure

During the installation process, the udev error is reported. The oracleasmlib we use directly ignores ~

(2) install Oracle Database 11g with Oracle Real Application Clusters

It is difficult to install the Grid. As long as the Grid is successfully installed, it is not a problem. During the installation process, use CVU for pre-Installation check. Some file permissions are encountered during the installation, pay more attention to the installation. Especially the permissions of ASM and the home directory permissions of the installation user.

13. Use ASMCA and DBCA to manage databases
There are a lot of online articles on these two management software, so I will not repeat them here.