Create an OCM(Oracle Configuration Manager) Response file to Apply a Patch in RAC

Step1 : -
          Download  and install latest OPatch

Patch 6880880: OPatch patch of version 11.2.0.3.15 
for Oracle software releases 11.2.0.x (AUG 2016)

Step 2:
          create  response  file for  patch


                  cd /u01/app/11.2.0/grid_1/OPatch/ocm/bin/

                    ./emocmrsp -no_banner -output /u01/oracle_software/orcl_grid.rsp


Give permission  777 to orcl_grid.rsp file


How to use :

[root@rac2 oracle_software]# /u01/app/11.2.0/grid_1/OPatch/opatch auto /u01/oracle_software/18139609 -ocmrf /u01/oracle_software/orcl_grid.rsp


How to Create an OCM Response file to Apply a 
Patch in Silent Mode - opatch silent (Doc ID 966023.1)

Enable and Disable oracle feature using chopt command : RAT disable using chopt command

RAT disable in oracle RAC using chopt command 
==========================================================

Check  RAT status in database using below query

SQL>  select * from gv$option where parameter='Real Application Testing';

   INST_ID PARAMETER                                          VALUE
---------- -------------------------------------------------- ----------------------------------------------------------------
         1 Real Application Testing                           TRUE
         2 Real Application Testing                           TRUE

SQL>
SQL>


Check database  and listener status using below command
[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1
Instance orcl2 is running on node rac2
[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): rac2,rac1
[oracle@rac1 ~]$
[oracle@rac1 ~]$

Stop listener and database services on all node in case RAC

[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl stop listener
[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is not running
[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl stop database -d orcl
[oracle@rac1 ~]$
[oracle@rac1 ~]$ srvctl status database -d orcl
Instance orcl1 is not running on node rac1
Instance orcl2 is not running on node rac2
[oracle@rac1 ~]$
[oracle@rac1 ~]$

Check database and listener status on second node : RAC2

[oracle@rac2 ~]$
[oracle@rac2 ~]$
[oracle@rac2 ~]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is not running
[oracle@rac2 ~]$
[oracle@rac2 ~]$ srvctl status database -d orcl
Instance orcl1 is not running on node rac1
Instance orcl2 is not running on node rac2
[oracle@rac2 ~]$
[oracle@rac2 ~]$

Go to $ORACLE_HOME/bin location using cd command  on node rac1


[oracle@rac1 bin]$ pwd
/u01/app/oracle/product/11.2.0/dbhome_1/bin
[oracle@rac1 bin]$
[oracle@rac1 bin]$ ls -lrt *chopt*
-rwxr-xr-x 1 oracle oinstall  234 Sep 20 13:07 chopt
-rw-r--r-- 1 oracle oinstall 6384 Sep 20 13:07 chopt.pl
-rw-r--r-- 1 oracle oinstall 3780 Sep 20 13:07 chopt.ini
[oracle@rac1 bin]$
[oracle@rac1 bin]$

Disable RAT on node rac1 using below command

[oracle@rac1 bin]$ ./chopt disable rat

Writing to /u01/app/oracle/product/11.2.0/dbhome_1/install/disable_rat.log...
/usr/bin/make -f /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk rat_off ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
/usr/bin/make -f /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1

[oracle@rac1 bin]$

Same thing do on second node rac2
[oracle@rac2 bin]$ pwd
/u01/app/oracle/product/11.2.0/dbhome_1/bin
[oracle@rac2 bin]$
[oracle@rac2 bin]$ ls -lrt *chopt*
-rwxr-xr-x 1 oracle oinstall  234 Sep 20 13:18 chopt
-rw-r--r-- 1 oracle oinstall 3780 Sep 20 13:18 chopt.ini
-rw-r--r-- 1 oracle oinstall 6384 Sep 20 13:18 chopt.pl
[oracle@rac2 bin]$
[oracle@rac2 bin]$

[oracle@rac2 bin]$ pwd
/u01/app/oracle/product/11.2.0/dbhome_1/bin
[oracle@rac2 bin]$
[oracle@rac2 bin]$ ls -lrt *chopt*
-rwxr-xr-x 1 oracle oinstall  234 Sep 20 13:18 chopt
-rw-r--r-- 1 oracle oinstall 3780 Sep 20 13:18 chopt.ini
-rw-r--r-- 1 oracle oinstall 6384 Sep 20 13:18 chopt.pl
[oracle@rac2 bin]$
[oracle@rac2 bin]$ ./chopt disable rat

Writing to /u01/app/oracle/product/11.2.0/dbhome_1/install/disable_rat.log...
/usr/bin/make -f /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk rat_off ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
/usr/bin/make -f /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1

[oracle@rac2 bin]$


start database and listener services  on all node  RAC1,RAC2  and verify

[oracle@rac1 bin]$
[oracle@rac1 bin]$ srvctl start database -d orcl
[oracle@rac1 bin]$
[oracle@rac1 bin]$
[oracle@rac1 bin]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1
Instance orcl2 is running on node rac2
[oracle@rac1 bin]$
[oracle@rac1 bin]$
[oracle@rac1 bin]$ srvctl start listener
[oracle@rac1 bin]$
[oracle@rac1 bin]$ srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): rac2,rac1
[oracle@rac1 bin]$
[oracle@rac1 bin]$


Then verify using below command


SQL>
SQL> select * from gv$option where parameter='Real Application Testing';

   INST_ID PARAMETER                                          VALUE
---------- -------------------------------------------------- ----------------------------------------------------------------
         1 Real Application Testing                           FALSE
         2 Real Application Testing                           FALSE

SQL>



How to Check and Enable/Disable Oracle Binary Options (Doc ID 948061.1)

Note: In case of RAC environment, the chopt enable / disable <option> command has to be executed on all RAC nodes.

[oracle@rac1 bin]$ chopt -h

usage:

chopt <enable|disable> <option>

options:
                  dm = Oracle Data Mining RDBMS Files
                  dv = Oracle Database Vault option
                lbac = Oracle Label Security
                olap = Oracle OLAP
        partitioning = Oracle Partitioning
                 rat = Oracle Real Application Testing

e.g. chopt enable rat


[oracle@rac1 bin]$

Start and Stop CRS (Cluster Ready Services) in RAC

 This command stop whole services (CRS+HA) on current node only (rac1)

[root@rac1 bin]# ./crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'rac1'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac1'
CRS-2677: Stop of 'ora.cvu' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'rac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2676: Start of 'ora.cvu' on 'rac2' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan3.vip' on 'rac2'
CRS-2677: Stop of 'ora.orcl.db' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA2.dg' on 'rac1'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2676: Start of 'ora.scan3.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA2.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2679: Attempting to clean 'ora.DATA1.dg' on 'rac1'
CRS-2681: Clean of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac1'
CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 bin]#
[root@rac1 bin]#
[root@rac1 bin]# ps -ef|grep d.bin
root      2206 22673  0 02:58 pts/1    00:00:00 grep d.bin
[root@rac1 bin]#
[root@rac1 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 bin]#

wait for some time all services will come  online 


[root@rac1 bin]#  ps -ef|grep d.bin
root      2582     1  1 03:01 ?        00:00:02 /u01/app/11.2.0/grid_1/bin/ohasd.bin reboot
grid      2733     1  0 03:01 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/mdnsd.bin
grid      2744     1  0 03:01 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/gpnpd.bin
grid      2759     1  0 03:01 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/gipcd.bin
root      2775     1  1 03:01 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/osysmond.bin
grid      2816     1  1 03:01 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/ocssd.bin
root      2922     1  0 03:02 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/octssd.bin reboot
grid      2954     1  0 03:02 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/evmd.bin
root      3030     1  1 03:02 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/crsd.bin reboot
root      3620 22673  0 03:03 pts/1    00:00:00 grep d.bin
[root@rac1 bin]#



[root@rac1 bin]# ./crsctl check cluster -all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@rac1 bin]#

[root@rac1 bin]# ./crsctl stat res -init -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
      1        ONLINE  ONLINE       rac1                     Started
ora.cluster_interconnect.haip
      1        ONLINE  ONLINE       rac1
ora.crf
      1        ONLINE  ONLINE       rac1
ora.crsd
      1        ONLINE  ONLINE       rac1
ora.cssd
      1        ONLINE  ONLINE       rac1
ora.cssdmonitor
      1        ONLINE  ONLINE       rac1
ora.ctssd
      1        ONLINE  ONLINE       rac1                     ACTIVE:0
ora.diskmon
      1        OFFLINE OFFLINE
ora.evmd
      1        ONLINE  ONLINE       rac1
ora.gipcd
      1        ONLINE  ONLINE       rac1
ora.gpnpd
      1        ONLINE  ONLINE       rac1
ora.mdnsd
      1        ONLINE  ONLINE       rac1
[root@rac1 bin]#
[root@rac1 bin]#

[root@rac1 bin]#  ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
               ONLINE  ONLINE       rac1
               ONLINE  ONLINE       rac2
ora.DATA2.dg
               ONLINE  ONLINE       rac1
               ONLINE  ONLINE       rac2
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1
               ONLINE  ONLINE       rac2
ora.asm
               ONLINE  ONLINE       rac1                     Started
               ONLINE  ONLINE       rac2                     Started
ora.gsd
               OFFLINE OFFLINE      rac1
               OFFLINE OFFLINE      rac2
ora.net1.network
               ONLINE  ONLINE       rac1
               ONLINE  ONLINE       rac2
ora.ons
               ONLINE  ONLINE       rac1
               ONLINE  ONLINE       rac2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac2
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac2
ora.cvu
      1        ONLINE  ONLINE       rac2
ora.oc4j
      1        ONLINE  ONLINE       rac2
ora.orcl.db
      1        ONLINE  ONLINE       rac1                     Open
      2        ONLINE  ONLINE       rac2                     Open
ora.rac1.vip
      1        ONLINE  ONLINE       rac1
ora.rac2.vip
      1        ONLINE  ONLINE       rac2
ora.scan1.vip
      1        ONLINE  ONLINE       rac1
ora.scan2.vip
      1        ONLINE  ONLINE       rac2
ora.scan3.vip
      1        ONLINE  ONLINE       rac2
[root@rac1 bin]#


This command stop and start  cluster services local node only .
Need to do on each node to stop whole cluster-ware 


http://www.freelists.org/post/oracle-l/crsctl-stop-has-vs-crsctl-stop-crs,2

CRS-4000: Command Start failed, or completed with errors. during start CRS in RAC cluster

 Unable to start CRS :
CRS-4000: Command Start failed, or completed with errors.



[root@rac1 bin]# ./crsctl stop cluster
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.DATA1.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.orcl.db' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA2.dg' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-2677: Stop of 'ora.DATA2.dg' on 'rac1' succeeded
CRS-2677: Stop of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2679: Attempting to clean 'ora.DATA1.dg' on 'rac1'
CRS-2681: Clean of 'ora.DATA1.dg' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1'
CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'
CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
[root@rac1 bin]#

[root@rac1 bin]#  ps -ef|grep d.bin
root     24016     1  0 01:43 ?        00:00:05 /u01/app/11.2.0/grid_1/bin/ohasd.bin reboot
grid     24180     1  0 01:43 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/mdnsd.bin
grid     24191     1  0 01:43 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/gpnpd.bin
grid     24202     1  0 01:44 ?        00:00:03 /u01/app/11.2.0/grid_1/bin/gipcd.bin
root     24218     1  1 01:44 ?        00:00:07 /u01/app/11.2.0/grid_1/bin/osysmond.bin
root     25860 22673  0 01:54 pts/1    00:00:00 grep d.bin

After try to start crs then got  below error 

[root@rac1 bin]# ./crsctl start crs
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
[root@rac1 bin]#

[root@rac1 bin]# ./crsctl check  crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[root@rac1 bin]#

Try to start with below command

[root@rac1 bin]# ./crsctl start cluster
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
[root@rac1 bin]#


[root@rac1 bin]# ps -ef|grep d.bin
root     24016     1  0 01:43 ?        00:00:08 /u01/app/11.2.0/grid_1/bin/ohasd.bin reboot
grid     24180     1  0 01:43 ?        00:00:00 /u01/app/11.2.0/grid_1/bin/mdnsd.bin
grid     24191     1  0 01:43 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/gpnpd.bin
grid     24202     1  0 01:44 ?        00:00:06 /u01/app/11.2.0/grid_1/bin/gipcd.bin
root     24218     1  1 01:44 ?        00:00:15 /u01/app/11.2.0/grid_1/bin/osysmond.bin
grid     25921     1  0 01:55 ?        00:00:03 /u01/app/11.2.0/grid_1/bin/ocssd.bin
root     25975     1  0 01:55 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/octssd.bin
grid     25991     1  0 01:55 ?        00:00:01 /u01/app/11.2.0/grid_1/bin/evmd.bin
root     26113     1  1 01:55 ?        00:00:04 /u01/app/11.2.0/grid_1/bin/crsd.bin reboot
root     27141 22673  0 02:01 pts/1    00:00:00 grep d.bin
[root@rac1 bin]#

How to change/Modify SCAN Listener port in oracle RAC

 CHANGE  SCAN LISTENER PORT IN RAC DATABASE
================================================

Step 1:- verify current status and port of scan listener 

[grid@rac1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node rac2
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node rac1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node rac1
[grid@rac1 ~]$



Step 2:-  Change port number of the SCAN LISTENER

[grid@rac1 ~]$ srvctl modify scan_listener -p 1585
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1585
[grid@rac1 ~]$

Step 3:- Restart SCAN LISTENER using below command 

[grid@rac1 ~]$ srvctl stop scan_listener
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl start scan_listener
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node rac2
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node rac1
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node rac1
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1585
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1585
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1585
[grid@rac1 ~]$
[grid@rac1 ~]$

Step 4:- Modify remote_listener parameter in each rac instance  also change port number in  tns entries

Before database stop change 

alter system set remote_listener='rac-scan:1585' scope=both sid='*';

Bounce database
[grid@rac1 ~]$ srvctl stop database -d orcl
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl start database -d orcl
[grid@rac1 ~]$
[grid@rac1 ~]$
[grid@rac1 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1
Instance orcl2 is running on node rac2
[grid@rac1 ~]$
[grid@rac1 ~]$ lsnrctl status LISTENER_SCAN2

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 28-JAN-2017 23:41:14

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN2)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN2
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                28-JAN-2017 23:32:15
Uptime                    0 days 0 hr. 8 min. 59 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/11.2.0/grid_1/network/admin/listener.ora
Listener Log File         /u01/app/11.2.0/grid_1/log/diag/tnslsnr/rac1/listener_scan2/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN2)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.110)(PORT=1585)))
Services Summary...
Service "orcl.localdomain.com" has 2 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
  Instance "orcl2", status READY, has 1 handler(s) for this service...
Service "orclXDB.localdomain.com" has 2 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
  Instance "orcl2", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@rac1 ~]$
[grid@rac1 ~]$ lsnrctl status

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 28-JAN-2017 23:41:23

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                28-JAN-2017 17:45:42
Uptime                    0 days 5 hr. 55 min. 41 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/11.2.0/grid_1/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/rac1/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.201)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.40)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "orcl.localdomain.com" has 1 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
Service "orclXDB.localdomain.com" has 1 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@rac1 ~]$

[grid@rac2 bin]$ lsnrctl status LISTENER_SCAN1

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 28-JAN-2017 23:45:01

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 11.2.0.4.0 - Production
Start Date                28-JAN-2017 23:32:13
Uptime                    0 days 0 hr. 12 min. 48 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/11.2.0/grid_1/network/admin/listener.ora
Listener Log File         /u01/app/11.2.0/grid_1/log/diag/tnslsnr/rac2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.112)(PORT=1585)))
Services Summary...
Service "orcl.localdomain.com" has 2 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
  Instance "orcl2", status READY, has 1 handler(s) for this service...
Service "orclXDB.localdomain.com" has 2 instance(s).
  Instance "orcl1", status READY, has 1 handler(s) for this service...
  Instance "orcl2", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@rac2 bin]$

Relocate Voting disk to +DATA1 to different disks

Step1: Verify voting disk location using below command 

[root@rac1 bin]#
[root@rac1 bin]#
[root@rac1 bin]# ./crsctl   query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   10ee09a0a8a44fe5bf6184e1f3abdc06 (ORCL:DISK1) [DATA1]
Located 1 voting disk(s).
[root@rac1 bin]#
[root@rac1 bin]#


Step2:-  Now move voting disk  to +DATA2 using below command 

[root@rac1 bin]# ./crsctl  replace votedisk +DATA2
Successful addition of voting disk 2edb42aeb93d4fa7bfd06327ae391776.
Successful deletion of voting disk 10ee09a0a8a44fe5bf6184e1f3abdc06.
Successfully replaced voting disk group with +DATA2.
CRS-4266: Voting file(s) successfully replaced
[root@rac1 bin]#
[root@rac1 bin]#

Step3:- verify voting disk location

[root@rac1 bin]# ./crsctl   query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   2edb42aeb93d4fa7bfd06327ae391776 (ORCL:DISK2) [DATA2]
Located 1 voting disk(s).
[root@rac1 bin]#


Note : make sure asm_diskstring configure properly  before relocation

Relocate OCR from +DATA1 to different disks

Task:-
 Relocate OCR from +DATA1 to +DATA2

Requirement:-

Need +DATA2 disk group 


Step 1: Check current OCR location 


[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3064
         Available space (kbytes) :     259056
         ID                       :  918475055
         Device/File Name         :     +DATA1
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded


         Logical corruption check succeeded

[root@rac1 bin]#
[root@rac1 bin]# pwd
/u01/app/11.2.0/grid_1/bin
[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3064
         Available space (kbytes) :     259056
         ID                       :  918475055
         Device/File Name         :     +DATA1
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

Step 2: Verify asm diskgroup for relocate 

[root@rac1 bin]# oracleasm listdisks
DISK1
DISK2
DISK3

Step 3:-  Now add +DATA2 as new OCR location and verify using below command  
[root@rac1 bin]#
[root@rac1 bin]# ./ocrconfig -add +DATA2
[root@rac1 bin]#
[root@rac1 bin]#
[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3064
         Available space (kbytes) :     259056
         ID                       :  918475055
         Device/File Name         :     +DATA1
                                    Device/File integrity check succeeded
         Device/File Name         :     +DATA2
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

Step4:- Now delete old location +DATA1 using below command  and verify

[root@rac1 bin]# ./ocrconfig -delete +DATA1
[root@rac1 bin]#
[root@rac1 bin]#
[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       3064
         Available space (kbytes) :     259056
         ID                       :  918475055
         Device/File Name         :     +DATA2
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

[root@rac1 bin]#