[Oracle] adds a new node to RAC

[Oracle] adds a new node to RAC

Add nodes to RAC:

Environment:

OS: OEL5.6

RAC: 10.2.0.1.0

The original rac1 and rac2 nodes need to be added now:

Procedure:

Modify the/etc/hosts file on the three Nodes

192.168.90.2 rac1

192.168.90.5 rac2

192.168.90.6 rac3

192.168.91.3 rac1-priv

192.168.91.6 rac2-priv

The 192.168.91.7 rac3-priv

Rac1-vip 192.168.90.3

192.168.90.4 rac2-vip

The 192.168.90.7 rac3-vip

Install the required rpm package on the rac3 node. In this tutorial, The OEL system is used, so oracle-validated is used.

[Root @ rac3 ~] # Mount/dev/cdrom/mnt

Mount: block device/dev/cdrom is write-protected, mounting read-only

[Root @ rac3 ~] # Vi/etc/yum. repos. d/public-yum-el5.repo

[Oel5]

Name = Enterprise Linux 5.6 DVD

Baseurl = file: // mnt/Server/

Gpgcheck = 0

Enabled = 1

[Root @ rac3 ~] # Yum install oracle-validated

Loaded plugins: rhnplugin, security

This system is not registered with ULN.

ULN support will be disabled.

...

Install the ASMLib package:

[Root @ rac3 ~] # Cd/mnt/Server

[Root @ rac3 Server] # rpm-ivh oracleasm-support-2.1.4-1.el5.i386.rpm

Warning: oracleasm-support-2.1.4-1.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing... ######################################## ### [100%]

1: oracleasm-support ##################################### ###### [100%]

[Root @ rac3 Server] # rpm-ivh oracleasm-2.6.18-238.el5-2.0.5-1.el5.i686.rpm

Warning: oracleasm-2.6.18-238.el5-2.0.5-1.el5.i686.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159

Preparing... ######################################## ### [100%]

1: oracleasm-2.6.18-238.el ####################################### #### [100%]

Configure mutual trust between nodes 3 and 1:

3 nodes:

[Oracle @ rac3 ~] $ Ssh-keygen-t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/. ssh/id_rsa ):

Enter passphrase (empty for no passphrase ):

Enter same passphrase again:

Your identification has been saved in/home/oracle/. ssh/id_rsa.

Your public key has been saved in/home/oracle/. ssh/id_rsa.pub.

The key fingerprint is:

18: d5: 6a: 94: 4c: 12: 90: 8a: 19: ca: 96: 4d: 97: cc: fe: f0 oracle @ rac3

1 node:

[Oracle @ rac1. ssh] $ ssh 192.168.90.6 cat ~ /. Ssh/*. pub> authorized_keys

Oracle@192.168.90.6's password:

[Oracle @ rac1. ssh] $ scp authorized_keys 192.168.90.6 :~ /. Ssh /.

Oracle@192.168.90.6's password:

Authorized_keys 100% 1780 1.7KB/s

[Oracle @ rac1. ssh] $ scp authorized_keys 192.168.90.5 :~ /. Ssh /.

Authorized_keys 100% 1780 1.7KB/s

Each node executes the following script to test the mutual trust of oracle users:

Ssh rac1 date

Ssh rac2 date

Ssh rac3 date

Ssh rac1-priv date

Ssh rac2-priv date

Ssh rac3-priv date

 

Configure time synchronization:

[Root @ rac3 Server] # ntpdate 192.168.90.2

22 Aug 10:45:31 ntpdate [9511]: adjust time server 192.168.90.2 offset-0.109663 sec

[Root @ rac3 Server] # crontab-e

* ***/Usr/sbin/ntpdate 192.168.90.2>/ntp. log

Because openfiler is used as storage, the openfiler configuration process is added here:

Configure it on the rac3 node:

[Root @ rac3 ~] # Vi/etc/iscsi/initiatorname. iscsi

# InitiatorName = iqn.1988-12.com. oracle: 48354521e5f2

InitiatorName = rac3

[Root @ rac3 ~] # Iscsiadm-m discovery-t st-p 192.168.90.8

192.168.90.8: 3260,1 rac3

[Root @ rac3 ~] # Service iscsi restart

Stopping iSCSI daemon:

Iscsid is stopped [OK]

Starting iSCSI daemon: [OK]

[OK]

Setting up iSCSI targets: Logging in to [iface: default, target: rac3, portal: 192.168.90.8, 3260]

Login to [iface: default, target: rac3, portal: 192.168.90.8, 3260] successful.

[OK]

Check whether the disk is successfully mounted:

[Root @ rac3 ~] # Fdisk-l

Disk/dev/sda: 42.9 GB, 42949672960 bytes

255 heads, 63 sectors/track, 5221 cylinders

Units = cylinders of 16065*512 = 8225280 bytes

Device Boot Start End Blocks Id System

/Dev/sda1*1 13 104391 83 Linux

/Dev/sda2 14 5221 41833260 8e Linux LVM

Disk/dev/dm-0: 39.5 GB, 39594229760 bytes

255 heads, 63 sectors/track, 4813 cylinders

Units = cylinders of 16065*512 = 8225280 bytes

Disk/dev/dm-0 doesn't contain a valid partition table

Disk/dev/dm-1: 3221 MB, 3221225472 bytes

255 heads, 63 sectors/track, 391 cylinders

Units = cylinders of 16065*512 = 8225280 bytes

Disk/dev/dm-1 doesn't contain a valid partition table

Disk/dev/sdb: 1073 MB, 1073741824 bytes

34 heads, 61 sectors/track, 1011 cylinders

Units = cylinders of 2074*512 = 1061888 bytes

Device Boot Start End Blocks Id System

/Dev/sdb1 1 1011 1048376 + 83 Linux

Disk/dev/sdc: 1073 MB, 1073741824 bytes

34 heads, 61 sectors/track, 1011 cylinders

Units = cylinders of 2074*512 = 1061888 bytes

Device Boot Start End Blocks Id System

/Dev/sdc1 1 1011 1048376 + 83 Linux

Disk/dev/sdd: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048*512 = 1048576 bytes

Device Boot Start End Blocks Id System

/Dev/sdd1 1 10240 10485744 83 Linux

Disk/dev/sde: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048*512 = 1048576 bytes

Device Boot Start End Blocks Id System

/Dev/sde1 1 10240 10485744 83 Linux

Disk/dev/sdf: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048*512 = 1048576 bytes

Device Boot Start End Blocks Id System

/Dev/sdf1 1 10240 10485744 83 Linux

After the mounting is successful, configure the device to be fixed:

[Root @ rac3 ~] # Raw/dev/raw/raw1/dev/sdb1

/Dev/raw/raw1: bound to major 8, minor 17

[Root @ rac3 ~] # Raw/dev/raw/raw2/dev/sdc1

/Dev/raw/raw2: bound to major 8, minor 33

[Root @ rac3 ~] # Raw/dev/raw/raw3/dev/sdd1

/Dev/raw/raw3: bound to major 8, minor 49

[Root @ rac3 ~] # Raw/dev/raw/raw4/dev/sde1

/Dev/raw/raw4: bound to major 8, minor 65

[Root @ rac3 ~] # Raw/dev/raw/raw5/dev/sdf1

/Dev/raw/raw5: bound to major 8, minor 81

[Root @ rac3 ~] # Vi/etc/udev/rules. d/60-raw.rules

Add the following content:

ACTION = "add", KERNEL = "/dev/sdb1", RUN + = "/bin/raw/dev/raw/raw1% N"

ACTION = "add", ENV {MAJOR} = "8", ENV {MINOR} = "17 ", RUN + = "/bin/raw/dev/raw/raw1% M % m"

ACTION = "add", KERNEL = "/dev/sdc1", RUN + = "/bin/raw/dev/raw/raw2% N"

ACTION = "add", ENV {MAJOR} = "8", ENV {MINOR} = "33 ", RUN + = "/bin/raw/dev/raw/raw2% M % m"

ACTION = "add", KERNEL = "/dev/sdd1", RUN + = "/bin/raw/dev/raw/raw3% N"

ACTION = "add", ENV {MAJOR} = "8", ENV {MINOR} = "49 ", RUN + = "/bin/raw/dev/raw/raw3% M % m"

ACTION = "add", KERNEL = "/dev/sde1", RUN + = "/bin/raw/dev/raw/raw4% N"

ACTION = "add", ENV {MAJOR} = "8", ENV {MINOR} = "65 ", RUN + = "/bin/raw/dev/raw/raw4% M % m"

ACTION = "add", KERNEL = "/dev/sdf1", RUN + = "/bin/raw/dev/raw/raw5% N"

ACTION = "add", ENV {MAJOR} = "8", ENV {MINOR} = "81 ", RUN + = "/bin/raw/dev/raw/raw5% M % m"

KERNEL = "raw [1-5]", OWNER = "oracle", GROUP = "oinstall", MODE = "640"

ASMLib Configuration:

[Root @ rac3 ~] #/Etc/init. d/oracleasm configure

Processing the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

Driver. The following questions will determine whether the driver is

Loaded on boot and what permissions it will have. The current values

Will be shown in brackets ('[]'). Hitting <ENTER> without typing

Answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle

Default group to own the driver interface []: dba

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

Initializing the Oracle ASMLib driver: [OK]

Scanning the system for Oracle ASMLib disks: [OK]

[Root @ rac3 ~] #/Etc/init. d/oracleasm listdisks

ASM01

ASM02

ASM03

Create the/u01 directory on the rac3 node and grant it to oracle:

[Root @ rac3 ~] # Mkdir/u01

[Root @ rac3 ~] # Chown oracle: oinstall/u01

Node 1:

[Oracle @ rac1 ~] $ Cd $ CRS_HOME/oui/bin/

[Oracle @ rac1 bin] $ ls

AddLangs. sh lsnodes resource runInstaller

AddNode. sh ouica. sh runConfig. sh runInstaller. sh

[Oracle @ rac1 bin] $ sh addNode. sh

[Root @ rac3 ~] #/U01/app/oracle/oraInventory/orainstRoot. sh

Changing permissions of/u01/app/oracle/oraInventory to 770.

Changing groupname of/u01/app/oracle/oraInventory to oinstall.

The execution of the script is complete

[Root @ rac1 ~] #/U01/app/oracle/product/10.2.0/db_1/install/rootaddnode. sh

Clscfg: EXISTING configuration version 3 detected.

Clscfg: version 3 is 10G Release 2.

Attempting to add 1 new nodes to the configuration

Using ports: CSS = 49895 CRS = 49896 EVMC = 49898 and EVMR = 49897.

Node <nodenumber>: <nodename> <private interconnect name>

Node 3: rac3 rac3-priv rac3

Creating OCR keys for user 'root', privgrp 'root '..

Operation successful.

/U01/app/oracle/product/10.2.0/db_1/bin/srvctl add nodeapps-n rac3-A rac3-vip/255.255.255.0/eth0-o/u01/app/oracle/product/10.2.0 /db_1

[Root @ rac3 ~] #/U01/app/oracle/product/10.2.0/db_1/root. sh

WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root

WARNING: directory '/u01/app/oracle/product' is not owned by root

WARNING: directory '/u01/app/oracle' is not owned by root

WARNING: directory '/u01/app' is not owned by root

WARNING: directory '/u01' is not owned by root

Checking to see if Oracle CRS stack is already configured

/Etc/oracle does not exist. Creating it now.

Ocr locations =/dev/raw/raw1

OCR backup directory '/u01/app/oracle/product/10.2.0/db_1/cdata/crs' does not exist. Creating now

Setting the permissions on OCR backup directory

Setting up NS directories

Oracle Cluster Registry configuration upgraded successfully

WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root

WARNING: directory '/u01/app/oracle/product' is not owned by root

WARNING: directory '/u01/app/oracle' is not owned by root

WARNING: directory '/u01/app' is not owned by root

WARNING: directory '/u01' is not owned by root

Clscfg: EXISTING configuration version 3 detected.

Clscfg: version 3 is 10G Release 2.

Assigning default hostname rac1 for node 1.

Assigning default hostname rac2 for node 2.

Successfully accumulated necessary OCR keys.

Using ports: CSS = 49895 CRS = 49896 EVMC = 49898 and EVMR = 49897.

Node <nodenumber>: <nodename> <private interconnect name>

Node 1: rac1 rac1-priv rac1

Node 2: rac2 rac2-priv rac2

Clscfg: Arguments check out successfully.

No keys were written. Supply-force parameter to override.

-Force is destructive and will destroy any previous cluster

Configuration.

Oracle Cluster Registry for cluster has already been initialized

Startup will be queued to init within 90 seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

Rac1

Rac2

Rac3

CSS is active on all nodes.

Waiting for the Oracle CRSD and EVMD to start

Waiting for the Oracle CRSD and EVMD to start

Oracle CRS stack installed and running under init (1 M)

Running vipca (silent) for processing nodeapps

/U01/app/oracle/product/10.2.0/db_1/jdk/jre // bin/java: error while loading shared libraries: libpthread. so.0: cannot open shared object file: No such file or directory

Install database software on rac3 nodes:

Run $ ORACLE_HOME/oui/bin/addNode. sh on the rac1 node.

[Root @ rac3 ~] #/U01/app/oracle/product/10.2.0/db_2/root. sh

Running Oracle10 root. sh script...

The following environment variables are set:

ORACLE_OWNER = oracle

ORACLE_HOME =/u01/app/oracle/product/10.2.0/db_2

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to/usr/local/bin...

Copying oraenv to/usr/local/bin...

Copying coraenv to/usr/local/bin...

Creating/etc/oratab file...

Entries will be added to the/etc/oratab file as needed

Database Configuration Assistant when a database is created

Finished running generic part of root. sh script.

Now product-specific root actions will be saved med.

Then add the instance on the rac1 node using dbca:

 

Now the node has been added:

[Oracle @ rac1 crsd] $ crs_stat-t

Name Type Target State Host

------------------------------------------------------------

Ora... B1.inst application ONLINE rac1

Ora... B2.inst application ONLINE rac2

Ora... B3.inst application ONLINE rac3

Ora. RACDB. db application ONLINE rac2

Ora... SM1.asm application ONLINE rac1

Ora... C1.lsnr application ONLINE rac1

Ora. rac1.gsd application ONLINE rac1

Ora. rac1.ons application ONLINE rac1

Ora. rac1.vip application ONLINE rac1

Ora... SM2.asm application ONLINE rac2

Ora... C2.lsnr application ONLINE rac2

Ora. rac2.gsd application ONLINE rac2

Ora. rac2.ons application ONLINE rac2

Ora. rac2.vip application ONLINE rac2

Ora... SM3.asm application ONLINE rac3

Ora... C3.lsnr application ONLINE rac3

Ora. rac3.gsd application ONLINE rac3

Ora. rac3.ons application ONLINE rac3

Ora. rac3.vip application ONLINE rac3