Author A.Kishore http:/www.appsdba.

info

Build Your Own Oracle RAC 11g Release 2 Cluster on Red Hat 4 and iSCSI
Here is my setup which I have used for Installing Oracle RAC 11gR2

Laptop windows XP To access linux1 and linux2

Linux1 There are 2 operating systems running on linux1, one is the host and one is the guest for openfiler 4 GB 250 GB

Linux2 4 GB 250 GB

openfiler1 using vmware with 1 GB RAM 250 GB

High Level Steps > > > > Install Install Install Install openfiler on linux1 using VMWARE Grid Infrastructure on linux1 and linux2 Oracle 11gR2 software on linux1 and linux2 Cluster database

Author A.Kishore http:/www.appsdba.info Install 11g R2 database Install 11g R2 database software

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

cd /d03/oracle/app/oracle/product/11.2.0/dbhome_1/ [root@linux2 dbhome_1]# sh root.sh Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /d03/oracle/app/oracle/product/11.2.0/dbhome_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying dbhome to /usr/local/bin ... The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ... The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by

Author A.Kishore http:/www.appsdba.info Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions.

Author A.Kishore http:/www.appsdba.info Create the Oracle Cluster Database

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

Author A.Kishore http:/www.appsdba.info

su - oracle -c "crsctl status resource -w \"TYPE co 'ora'\" -t"

Author A.Kishore http:/www.appsdba.info

crsctl check cluster CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online srvctl status database -d racdb Instance racdb1 is running on node linux1 Instance racdb2 is running on node linux2 srvctl status instance -d racdb -i racdb1 Instance racdb1 is running on node linux1

Author A.Kishore http:/www.appsdba.info srvctl status instance -d racdb -i racdb1 Instance racdb1 is running on node linux1 [oracle@linux1 ~]$ srvctl status nodeapps VIP linux1-vip is enabled VIP linux1-vip is running on node: linux1 VIP linux2-vip is enabled VIP linux2-vip is running on node: linux2 Network is enabled Network is running on node: linux1 Network is running on node: linux2 GSD is disabled GSD is not running on node: linux1 GSD is not running on node: linux2 ONS is enabled ONS daemon is running on node: linux1 ONS daemon is running on node: linux2 eONS is enabled eONS daemon is running on node: linux1 eONS daemon is running on node: linux2 srvctl config nodeapps VIP exists.:linux1 VIP exists.: /linux1-vip/192.168.1.200/255.255.255.0/eth0 VIP exists.:linux2 VIP exists.: /linux2-vip/192.168.1.201/255.255.255.0/eth0 GSD exists. ONS daemon exists. Local port 6100, remote port 6200 eONS daemon exists. Multicast port 18123, multicast IP address 234.154.108.237, listening port 2016 srvctl config database racdb srvctl config database -d racdb -a Database unique name: racdb Database name: racdb Oracle home: /d03/oracle/app/oracle/product/11.2.0/dbhome_1 Oracle user: oracle Spfile: +DATA/racdb/spfileracdb.ora Domain: oracle.com Start options: open Stop options: immediate Database role: PRIMARY

Author A.Kishore http:/www.appsdba.info Management policy: AUTOMATIC Server pools: racdb Database instances: racdb1,racdb2 Disk Groups: DATA Services: Database is enabled Database is administrator managed srvctl status asm ASM is running on linux1,linux2 srvctl config asm -a ASM home: /d01/oracle/app/11.2.0/grid ASM listener: LISTENER ASM is enabled.

rvctl status listener Listener LISTENER is enabled Listener LISTENER is running on node(s): linux1,linux2 srvctl config listener -a Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> /d01/oracle/app/11.2.0/grid on node(s) linux2,linux1 End points: TCP:1521

srvctl status scan SCAN VIP scan1 is enabled SCAN VIP scan1 is running on node linux1 srvctl config scan SCAN name: racnode-cluster-scan, Network: 1/192.168.1.0/255.255.255.0/eth0 SCAN VIP name: scan1, IP: /racnode-cluster-scan/192.168.1.187 srvctl status vip -n linux1 VIP linux1-vip is enabled VIP linux1-vip is running on node: linux1 srvctl status vip -n linux2

Author A.Kishore http:/www.appsdba.info VIP linux2-vip is enabled VIP linux2-vip is running on node: linux2 srvctl config nodeapps -a -g -s -l -l option has been deprecated and will be ignored. VIP exists.:linux1 VIP exists.: /linux1-vip/192.168.1.200/255.255.255.0/eth0 VIP exists.:linux2 VIP exists.: /linux2-vip/192.168.1.201/255.255.255.0/eth0 GSD exists. ONS daemon exists. Local port 6100, remote port 6200 Name: LISTENER Network: 1, Owner: oracle Home: <CRS home> /d01/oracle/app/11.2.0/grid on node(s) linux2,linux1 End points: TCP:1521

cluvfy comp clocksync -verbose Verifying Clock Synchronization across the cluster nodes Checking if Clusterware is installed on all nodes... Check of Clusterware install passed Checking if CTSS Resource is running on all nodes... Check: CTSS Resource running on all nodes Node Name Status ------------------------------------ -----------------------linux1 passed Result: CTSS resource check passed

Querying CTSS for time offset on all nodes... Result: Query of CTSS for time offset passed Check CTSS state started... Check: CTSS state Node Name State ------------------------------------ -----------------------linux1 Observer CTSS is in Observer state. Switching over to clock synchronization checks using NTP

Author A.Kishore http:/www.appsdba.info

Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... The NTP configuration file "/etc/ntp.conf" is available on all nodes NTP Configuration file check passed Checking daemon liveness... Check: Liveness for "ntpd" Node Name Running? ------------------------------------ -----------------------linux1 yes Result: Liveness check passed for "ntpd" Checking NTP daemon command line for slewing option "-x" Check: NTP daemon command line Node Name Slewing Option Set? ------------------------------------ -----------------------linux1 yes Result: NTP daemon slewing option check passed Checking NTP daemon's boot time configuration, in file "/etc/sysconfig/ntpd", for slewing option "-x" Check: NTP daemon's boot time configuration Node Name Slewing Option Set? ------------------------------------ -----------------------linux1 yes Result: NTP daemon's boot time configuration check for slewing option passed NTP common Time Server Check started... NTP Time Server "202.71.140.36" is common to all nodes on which the NTP daemon is running NTP Time Server "209.81.9.7" is common to all nodes on which the NTP daemon is running NTP Time Server ".INIT." is common to all nodes on which the NTP daemon is running NTP Time Server "218.45.21.199" is common to all nodes on which the NTP daemon is running Check of common NTP Time Server passed

Author A.Kishore http:/www.appsdba.info Clock time offset check from NTP Time Server started... Checking on nodes "[linux1]"... Check: Clock time offset from NTP Time Server Time Server: 202.71.140.36 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ -----------------------linux1 -7.054 passed Time Server "202.71.140.36" has time offsets that are within permissible limits for nodes "[linux1]". Time Server: 209.81.9.7 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ -----------------------linux1 12.711 passed Time Server "209.81.9.7" has time offsets that are within permissible limits for nodes "[linux1]". Time Server: .INIT. Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ -----------------------linux1 0.0 passed Time Server ".INIT." has time offsets that are within permissible limits for nodes "[linux1]". Time Server: 218.45.21.199 Time Offset Limit: 1000.0 msecs Node Name Time Offset Status ------------ ------------------------ -----------------------linux1 45.507 passed Time Server "218.45.21.199" has time offsets that are within permissible limits for nodes "[linux1]". Clock time offset check passed Result: Clock synchronization check using Network Time Protocol(NTP) passed

Oracle Cluster Time Synchronization Services check passed Verification of Clock Synchronization across the cluster nodes was successful.

Author A.Kishore http:/www.appsdba.info

SQL> SQL> select name from v$datafile union select member from v$logfile union select name from v$controlfile union select name from v$tempfile; 2 3 4 5 6 7 NAME -------------------------------------------------------------------------------+DATA/racdb/controlfile/current.260.725453693 +DATA/racdb/controlfile/current.261.725453681 +DATA/racdb/datafile/example.267.725453895 +DATA/racdb/datafile/sysaux.257.725452927 +DATA/racdb/datafile/system.256.725452923 +DATA/racdb/datafile/undotbs1.258.725452931 +DATA/racdb/datafile/undotbs2.268.725454523 +DATA/racdb/datafile/users.259.725452931 +DATA/racdb/onlinelog/group_1.262.725453717 +DATA/racdb/onlinelog/group_1.263.725453741 +DATA/racdb/onlinelog/group_2.264.725453765 NAME --------------------------------------------------------------------------------

Author A.Kishore http:/www.appsdba.info +DATA/racdb/onlinelog/group_2.265.725453789 +DATA/racdb/onlinelog/group_3.269.725454915 +DATA/racdb/onlinelog/group_3.270.725454941 +DATA/racdb/onlinelog/group_4.271.725454967 +DATA/racdb/onlinelog/group_4.272.725455003 +DATA/racdb/tempfile/temp.266.725453857

SQL> SQL> SELECT path FROM v$asm_disk; 2 PATH -------------------------------------------------------------------------------ORCL:VOL1 ORCL:VOL2 ORCL:VOL3

Author A.Kishore http:/www.appsdba.info

Start/Stop All Instances with SRVCTL Finally, you can start/stop all instances and associated services using the following:
[oracle@linux1 ~]$ srvctl stop database -d racdb [oracle@linux1 ~]$ srvctl start database -d racdb

Stopping the Oracle Clusterware Stack on the Local Server /d01/oracle/app/11.2.0/grid/bin/crsctl stop cluster CRS-2673: Attempting to stop 'ora.crsd' on 'linux1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'linux1' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'linux1' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'linux1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'linux1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'linux1' CRS-2677: Stop of 'ora.scan1.vip' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'linux2' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.linux1.vip' on 'linux1' CRS-2677: Stop of 'ora.linux1.vip' on 'linux1' succeeded CRS-2672: Attempting to start 'ora.linux1.vip' on 'linux2' CRS-2676: Start of 'ora.scan1.vip' on 'linux2' succeeded CRS-2676: Start of 'ora.linux1.vip' on 'linux2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'linux2' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'linux2' succeeded CRS-2677: Stop of 'ora.DATA.dg' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'linux1' CRS-2677: Stop of 'ora.asm' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.eons' on 'linux1' CRS-2673: Attempting to stop 'ora.ons' on 'linux1' CRS-2677: Stop of 'ora.ons' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.net1.network' on 'linux1' CRS-2677: Stop of 'ora.net1.network' on 'linux1' succeeded CRS-2677: Stop of 'ora.eons' on 'linux1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'linux1' has completed CRS-2677: Stop of 'ora.crsd' on 'linux1' succeeded

Author A.Kishore http:/www.appsdba.info CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'linux1' CRS-2673: Attempting to stop 'ora.ctssd' on 'linux1' CRS-2673: Attempting to stop 'ora.evmd' on 'linux1' CRS-2673: Attempting to stop 'ora.asm' on 'linux1' CRS-2677: Stop of 'ora.cssdmonitor' on 'linux1' succeeded CRS-2677: Stop of 'ora.evmd' on 'linux1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'linux1' succeeded CRS-2677: Stop of 'ora.asm' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'linux1' CRS-2677: Stop of 'ora.cssd' on 'linux1' succeeded CRS-2673: Attempting to stop 'ora.diskmon' on 'linux1' CRS-2677: Stop of 'ora.diskmon' on 'linux1' succeeded

/d01/oracle/app/11.2.0/grid/bin/crsctl stop cluster all

Starting the Oracle Clusterware Stack on the Local Server

/d01/oracle/app/11.2.0/grid/bin/crsctl start cluster /d01/oracle/app/11.2.0/grid/bin/crsctl start cluster all

Manual startup on both nodes cd /etc/oracle/scls_scr/linux1/root/ Modify files crsstart and ohasdstr change enable to disable