Oracle 11g RAC with ASM installed on RHEL5 (simplified version)

1. Clear Tasks

Prepare a RHEL5 PC to act as a San device, a SAN (both as a DNS and timing system)

Prepare two RHEL5 PCs to act as RAC nodes, R1 and R2

Install RAC on ASM disk, including OCR, voting, library

2. Modify kernel parameters and shell limitations of R1 and R2

On R1 and R2, respectively.

Vim/etc/sysctl.conf
------------------------------------------
Kernel.shmmax = 4294967296
Kernel.shmmni = 4096
Kernel.shmall = 2097152
Kernel.sem = 250 32000 100 128
Fs.file-max = 6815744
FS.AIO-MAX-NR = 1048576
Net.ipv4.ip_local_port_range = 9000 65500
Net.core.rmem_default = 262144
Net.core.rmem_max = 4194304
Net.core.wmem_default = 262144
Net.core.wmem_max = 1048586

Sysctl-p

Vim/etc/security/limits.conf
------------------------------------------
# < Span style= "font-size:0px;" > ora c l e Configure shell parameters
oracle Soft nofile 65536
oracle hard nofile 65536
oracle Soft nproc 16384
oracle hard Nproc 16384

grid Soft Nofile 65536
grid Hard Nofile 65536
grid Soft Nproc 16384
grid Hard Nproc 16384

3, configure the SAN, R1, R2 Network

Configured separately on SAN, R1, R2

Vim/etc/hosts

127.0.0.1san localhost.localdomain localhost
:: 1localhost6.localdomain6 Localhost6

192.168.33.100san

192.168.33.101 R1
192.168.33.102 R2

192.168.33.201 r1-vip# do not configure on the NIC
192.168.33.202 r2-vip# do not configure on the NIC

10.0.0.1 R1-priv
10.0.0.2 R2-priv

4. Configure DNS on the SAN

Yum Install bind Bind-chroot caching-nameserver

Cd/var/named/chroot/etc

Cp-p named.caching-nameserver.conf named.conf
Vim/var/named/chroot/etc/named.conf

Options {
Listen-on Port: (any;};
Listen-on-v6 Port 53 {:: 1;};
Directory "/var/named";
Dump-file "/var/named/data/cache_dump.db";
Statistics-file "/var/named/data/named_stats.txt";
Memstatistics-file "/var/named/data/named_mem_stats.txt";

allow-query {any;};
Allow-query-cache {any;};
};
Logging {
Channel Default_debug {
File "Data/named.run";
Severity dynamic;
};
};
View Localhost_resolver {
match-clients {any;};
match-destinations {any;};
recursion Yes;
Include "/etc/named.zones";
};

Cp-p Named.rfc1912.zones Named.zones

Vim/var/named/chroot/etc/named.zones

Named.rfc1912.zones:
//
Provided by the Red Hat Caching-nameserver Package
//
ISC BIND named zone configuration for zones recommended by
RFC 1912 Section 4.1:localhost TLDs and address zones
//
See/usr/share/doc/bind*/sample/for example named configuration files.
//
Zone "." in {
Type hint;
File "named.ca";
};

Zone "Localdomain" in {
Type master;
File "Localdomain.zone";
allow-update {none;};
};

Zone "localhost" in {
Type master;
File "Localhost.zone";
allow-update {none;};
};

Zone "0.0.127.in-addr.arpa" in {
Type master;
File "Named.local";
allow-update {none;};
};

Zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" in {
Type master;
File "Named.ip6.local";
allow-update {none;};
};

Zone "255.in-addr.arpa" in {
Type master;
File "Named.broadcast";
allow-update {none;};
};

Zone "0.in-addr.arpa" in {
Type master;
File "Named.zero";
allow-update {none;};
};

Zone "Ora11grac.com" in {
Type master;
File "Ora11grac.com.zone";
allow-update {none;};
};

Zone "33.168.192.in-addr.arpa" in {
Type master;
File "33.168.192.local";
allow-update {none;};
};

Cd/var/named/chroot/var/named
Cp-p Named.zero Ora11grac.com.zone
Cp-p named.local 33.168.192.local

Vim/var/named/chroot/var/named/ora11grac.com.zone

$TTL 86400
@ in SOA dns.ora11grac.com. Root.ora11grac.com. (
42; Serial (d. Adams)
3H; Refresh
15M; Retry
1W; Expiry
1D); Minimum
@ in NS dns.ora11grac.com.
R1 in A 192.168.33.101
R2 in A 192.168.33.102
R1-VIP in A 192.168.33.201
R2-VIP in A 192.168.33.202
Scan in A 192.168.33.195
Scan in A 192.168.33.196
Scan in A 192.168.33.197

Vim/var/named/chroot/var/named/33.168.192.local

$TTL 86400
@ in SOA dns.ora11grac.com. Root.ora11grac.com. (
1997022700; Serial
28800; Refresh
14400; Retry
3600000; Expire
86400); Minimum
@ in NS dns.ora11grac.com.
101 in PTR r1.ora11grac.com.
102 in PTR r2.ora11grac.com.
201 in PTR r1-vip.ora11grac.com.
202 in PTR r2-vip.ora11grac.com.
195 in PTR scan.ora11grac.com.
196 in PTR scan.ora11grac.com.
197 in PTR scan.ora11grac.com.

Service named restart

Chkconfig named on

After completing the test on the client

Vim/etc/resolve.conf

Search ora11grac.com

NameServer 192.168.33.100

Then ping scan.ora11grac.com, or nslookup scan.ora11grac.com

5. Add users and groups in R1, R2

In R1, R2 respectively

GROUPADD-G 501 Oinstall
GROUPADD-G 502 DBA
GROUPADD-G 503 Oper
GROUPADD-G 504 Asmadmin
GROUPADD-G 505 ASMDBA
GROUPADD-G 506 Asmoper

Useradd-u 501-g oinstall-g dba,oper,asmdba Oracle
Useradd-u 502-g oinstall-g dba,asmadmin,asmdba,asmoper grid

Modify the configuration file on R1, R2

Vim/home/grid/.bashrc

Export Oracle_base=/u01/app/Oracle
Export ORACLE_home=/u01/app/11.2.0/grid
Export path=$ORACLE_home/bin: $PATH
Export Oracle_owner=Oracle
Export Oracle _SID=+ASM1 #r2节点为 export Oracle_SID=+ASM2
Export ORACLE_term=vt100
Export Threads_flag=native
Export ld_library_path=$ORACLE_home/lib: $LD _library_path
Export path=$ORACLE_home/bin: $PATH
Export Lang=en_us
Alias sqlplus= ' Rlwrap sqlplus '
Alias lsnrctl= ' Rlwrap lsnrctl '
Alias asmcmd= ' Rlwrap asmcmd '

Vim/home/oracle/.bashrc

ExportORACLE_base=/u01/app/Oracle
ExportORACLE_home=$ORACLE_base/product/11.2.0/db_1
Export path=$ORACLE_home/bin: $PATH
ExportORACLE_owner=Oracle
ExportORACLE_SID=ORCL1 #rac2节点为 ExportORACLE_sid=orcl2
ExportORACLE_term=vt100
Export Threads_flag=native
Export ld_library_path=$ORACLE_home/lib: $LD _library_path
Export path=$ORACLE_home/bin: $PATH
Export Editor=vi
Export sqlpath=/home/Oracle
Export Lang=en_us
Alias sqlplus= ' Rlwrap sqlplus '
Alias lsnrctl= ' Rlwrap lsnrctl '
Alias rman= ' Rlwrap Rman '
Alias Dgmgrl= ' Rlwrap dgmgrl '

6, create the directory in R1, R2

In R1, R2 respectively

Mkdir-p/u01/app/11.2.0/grid
Chown-r grid:oinstall/u01
mkdir/u01/app/Oracle
Chown-r Oracle: oinstall/u01/app/Oracle
Chmod-r 775/u01/

7, configure the grid, Oracle users under the two-machine trust (need a single step)

On the R1.

Su-grid

SSH-KEYGEN-T RSA

Ssh-keygen-t DSA

CD. SSH

Cat *.pub > Authorized_keys

On the R2.

Su-grid

SSH-KEYGEN-T RSA

Ssh-keygen-t DSA

CD. SSH

Cat *.pub > Authorized_keys

On the R1.

SCP Authorized_keys [email protected]:/HOME/GRID/.SSH/K1

On the R2.

CAT/HOME/GRID/.SSH/K1 >> Authorized_keys

SCP Authorized_keys [email protected]:/home/grid/.ssh/

On the R1.

Su-oracle

SSH-KEYGEN-T RSA

Ssh-keygen-t DSA

CD. SSH

Cat *.pub > Authorized_keys

On the R2.

Su-oracle

SSH-KEYGEN-T RSA

Ssh-keygen-t DSA

CD. SSH

Cat *.pub > Authorized_keys

On the R1.

SCP Authorized_keys oracle@r2:/home/Oracle/.SSH/K1

On the R2.

CAT/HOME/ORACLE/.SSH/K1 >> Authorized_keys

SCP Authorized_keys oracle@r1:/home/Oracle/.ssh/

Test the trust of two machines on R1 and R2 respectively

SSH R1 Date

SSH R2 Date

SSH R1-priv Date

SSH R2-priv Date

8, configure the NTP server on the San (not on the San to do on other machines can also do)

On the SAN

Vim/etc/sysconfig/ntpd
# Drop ROOT to ID ' ntp:ntp ' by default.
options= "-x-u ntp:ntp-p/var/run/ntpd.pid"

# Set to ' yes ' to sync HW clock after successful ntpdate
Sync_hwclock=no

# Additional options for Ntpdate

Vim/etc/ntp.conf
Restrict default nomodify
Restrict 127.0.0.1
Server 127.127.1.0
Fudge 127.127.1.0 Stratum 8
Driftfile/var/lib/ntp/drift
Broadcastdelay 0.008

Start the NTPD service

Service NTPD Restart

Chkconfig ntpd on

On the client R1, R2

Ntpdate San

Then use the CRONTAB-E

*/5 * * * */usr/sbin/ntpdate san

It is recommended that you copy the configuration file to R1, R2, and then start ntpd in R1, R2, because the grid will be verified when it is installed.

9. Configuring a centralized storage san

On the SAN

Yum Install Scsi-target-utils-y
Vim/etc/tgt/targets.conf

<target lun1>
Backing-store/dev/sdb1
Backing-store/dev/sdc1
Backing-store/dev/sdd1
Backing-store/dev/sde1
Backing-store/dev/sdf1
Backing-store/dev/sdf2
Backing-store/dev/sdf3
</target>

Service TGTD Start
Tgtadm--lld iscsi-m target--op Show
Chkconfig TGTD on

On the R1 and R2.

Yum Install Iscsi-initiator-utils-y
Service Iscsid Start
Iscsiadm-m discovery-t st-p San
Service iSCSI Start
Chkconfig Iscsid on
Chkconfig iSCSI on

10. Configure Udev to bind bare devices on R1, R2

On the R1 and R2

ll/dev/sd*

Brw-r-----1 root disk 8,0 Jan4 19:31/DEV/SDA
Brw-r-----1 root disk 8,1 Jan4 11:31/dev/sda1
Brw-r-----1 root disk 8,2 Jan4 19:31/dev/sda2
Brw-r-----1 Root disk 8, Jan4 13:31/dev/sdb
Brw-r-----1 Root disk 8, Jan4 16:30/DEV/SDB1
Brw-r-----1 Root disk 8, Jan4 13:31/DEV/SDC
Brw-r-----1 Root disk 8, Jan4 16:30/DEV/SDC1
Brw-r-----1 Root disk 8, Jan4 13:32/DEV/SDD
Brw-r-----1 Root disk 8, Jan4 16:30/DEV/SDD1
Brw-r-----1 Root disk 8, Jan4 13:32/DEV/SDE
Brw-r-----1 Root disk 8, Jan4 16:30/dev/sde1
Brw-r-----1 Root disk 8, Jan4 13:32/DEV/SDF
Brw-r-----1 Root disk 8, Bayi Jan4 16:29/dev/sdf1
Brw-r-----1 Root disk 8, Jan4 13:33/DEV/SDG
Brw-r-----1 Root disk 8, Jan4 16:29/DEV/SDG1
Brw-r-----1 Root Disk 8, 98 Jan4 16:29/DEV/SDG2

Vim/etc/udev/rules.d/60-raw.rules

action== "Add", kernel== "Sdf1", run+= "/bin/raw/dev/raw/raw1%N"
action== "Add", kernel== "SDG1", run+= "/bin/raw/dev/raw/raw2%N"
action== "Add", kernel== "Sdg2", run+= "/bin/raw/dev/raw/raw3%N"
kernel== "Raw1", owner= "grid", group= "Asmadmin", mode= "660"
kernel== "Raw2", owner= "grid", group= "Asmadmin", mode= "660"
kernel== "RAW3", owner= "grid", group= "Asmadmin", mode= "660"

11. Install Dependent Packages

Yum install binutils compat-libstdc++ elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel GL Ibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make Sysstat UnixODBC unixodbc-devel-y

12. Install ASM RPM Package (be sure to note the version and the platform)

RPM-IVH oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm oracleasm-support-2.1.7-1.el5.x86_64.rpm oracleasmlib-2.0.4-1.el5.x86_64.rpm

13. Configuring ASM

On the R1.

/etc/init.d/oracleasm Configure

User designation: Grid

Group designation: Asmadmin

All the rest is Yes

Create a ORACLEASM disk

/etc/init.d/oracleasm Createdisk V1/DEV/SDB1

/etc/init.d/oracleasm Createdisk V2/DEV/SDC1

/etc/init.d/oracleasm Createdisk V3/DEV/SDD1

/etc/init.d/oracleasm Createdisk V4/dev/sde1

Scan on the R2

/etc/init.d/oracleasm Scandisks

14. Install grid check with grid user

./runcluvfy.sh Stage-pre Crsinst-n R1,r2-fixup-verbose

15. Installing the Grid

Under the grid user

./runinstaller

In Instalaation type Select

Install and Configure Grid infrustructure for a Cluster

In Product languages Select

Advanced Installation

Fill in grid Plug and Play information

Cluster Name:r-cluster

SCAN Name:scan.ora11grac.com

SCAN port:1521

Configure GNS Not checked

Fill in cluster Node information

R1.ora11grac.comr1-vip.ora11grac.com

R2.ora11grac.comr2-vip.ora11grac.com

This VIP do not use Ifconfig to add

Fill in specify Network Interface usage

Eth0192.168.33.0public

Eth110.0.0.0private

In Storage option information Select

Automatic Storage Management (ASM)

In Create ASM Disk Group

Select the Raw1, RAW2, raw3 that you just recognized, make disk group name grid

In Specify ASM Password

Fill in User password

In failure isolation support selection

Do not use Intelligent Platform Management (IPMI)

Fill in privileged Operating System groups:

ASM Database Administrator (OSDBA) group:asmdba

ASM Instance Administration Operator (osoper) group:asmoper

ASM Instance Administrator (osasm) group:asmadmin

Fill in the Specify installation location:

Oracle Base:/u01/app/oracle

Software Location:/u01/app/oracle/product/11.2.0/grid

Fill in the Create inventory:

Inventory Directory:/u01/app/orainventory

Orainventory Group Name:oinstall

After installation is complete

Running in R1

/u01/app/orainventory/orainstroot.sh

Running in R2

/u01/app/orainventory/orainstroot.sh

Running in R1

/u01/app/11.2.0/grid/root.sh

Running in R2

/u01/app/11.2.0/grid/root.sh

16. Install the database

First, make sure all the important services are online.

Crs_stat-t

General oc4j will not be on the line, then

Srvctl Enable oc4j

Srvctl Start oc4j

Olsnodes-n

Appears R1,R2

That's OK.

(GSD service not online does not affect the installation of library software and the establishment of libraries)

With a grid user, open Asmca and create an ASM disk group +asm

Switch to the R1 in the

Su-oracle

./runinstaller

In Select Installation Option

Select Install database software only, and only the databases engine software is installed

At Node Selection

Tick R1,R2 of two nodes

In select Database Edition choose

Enterprise Edition

Fill in Specify Installation location

Oracle Base:/u01/app/oracle

Software Location:/u01/app/oracle/product/11.2.0/dbhome_1

Run after execution, run in R1, R2

/u01/app/oracle/product/11.2.0/db_1/root.sh

Complete.

Oracle 11g RAC with ASM installed on RHEL5 (simplified version)