Practice 12 Upgrading Oracle RAC Database P a g e |1

Practice 12

Upgrading Oracle RAC Database

Practice Overview

In this practice, you will upgrade the Oracle RAC database from release 12.1.0.2 to release 12.2.0.1.

Note: In the previous practice you have taken a backup of all the Virtualbox appliance files. If you
have not done that, it is recommended to do it now. Shutdown the appliances and exit Oracle
Virtualbox then take backup of all the virtual appliance files including the shred disk files.

Note: This practice is a long practice. It may take three hours to implement it all.

Note: Do not consider this document as a reference to upgrade a production database. The following
should be your references:
o Oracle Grid Infrastructure Installation and Upgrade Guide 12c Release 2 (12.2) for Linux

o Oracle Database Upgrade Guide 12c Release 2 (12.2)

o Doc ID 2189854.1: Complete Checklist for Upgrading to Oracle Database 12c Release 2
(12.2) using DBUA

Practice Assumptions
• A database backup has been taken on rac database. You have taken this backup in the previous
practice.

• You downloaded the following Oracle software products:

o Oracle Database 12c R2 Grid Infrastructure (12.2.0.1.0) for Linux x86-64 ( link )

o Oracle Database 12c R2 (12.2.0.1.0) for Linux x86-64 ( link )

Oracle Release 12.2 Home Locations

• Following are the locations of the upgraded Oracle software products that you will install in this
practice:

Oracle Software Product Home Location

Oracle Grid Infrastructure /u01/app/12.2.0/grid

Oracle Database /u01/app/oracle/product/12.2.0/db_1

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |2

Expanding CRS Disk Group in ASM

Oracle Grid Infrastructure 12.2 has added new features that require plenty of disk space in the CRS
disk group. Therefore, you must expand the size of the CRS disk group before you can upgrade Oracle
Grid to the new release.
In this section of the practice, you will take the steps to expand the CRS disk group in ASM.

1. Shut down srv1 and srv2.

2. Add a new fixed-size sharable disk to srv1. Give it the name DISK4 and set its size to 40 GB.
In Oracle VirtualBox, click on srv1 | click on Settings | click on Storage in the right pane | click
on Controller: SATA | click on Add Hardisk | click on Add a New Disk | select VDI option |
make it fixed-size | enter the full-path filename of the disk | set its size to 40 GB

3. Once the disk is created, make it sharable.

File menu item | Virtual Media Manager | select the created disk DISK4 | click on the Modify
button | choose the option to make this file shareable | press Ok | press Close

4. Link the new disk to srv2.

click on srv2 | click on Settings | click on Storage in the right pane | click on Controller: SATA
| click on Add Hardisk | click on an Existing Disk button | navigate to the new disk file and
press OK

5. Start srv1 and wait for its OS to load.

6. Start Putty and login to srv1 as root user. Format the added disk.

# display all the available disks:

ls -l /dev/sd*

# format the disk:

# answer "n", "p","1", default, default, "w" when prompted
fdisk /dev/sde

7. Add the partitioned disk to the ASM recognized disk list.

oracleasm listdisks
oracleasm createdisk DISK4 /dev/sde1
oracleasm listdisks

8. Start srv2 and wait for its OS to load.

9. Start Putty and login to srv2 as root user.

10. Scan the ASM disks and make sure DISK4 is seen by srv2.

oracleasm scandisks
oracleasm listdisks

11. Login to the VirtualBox window of srv1 as grid.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |3

12. Start asmca utility and add DISK4 to the CRS disk group.

Right-click on the CRS disk group | select Add Disks | select DISK4 then click on OK button

You should see the CRS disk group size increased to nearly 50GB, as shown in the following
screenshot:

13. Exit from the asmca utility.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |4

Preparing Upgrade Installation files

In this section of the practice, you will prepare the Oracle RAC database stack installation files.

14. Copy the Oracle 12.2 Grid Infrastructure installation file locally in the shared folder in your
hosting machine.
We mean by the shared folder, the folder that is shared by the Oracle Virtualbox machines and
which you used to install Oracle 12.1 and apply PSU on it.

15. Extract the Oracle 12.2 database installation zip file locally into the shared folder.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |5

Upgrade Oracle Grid Infrastructure

In this section of the practice, you will upgrade Oracle Grid Infrastructure. Oracle Grid Infrastructure
supports rolling upgrade. When nodes are being upgraded in a batch, the nodes in the other batches
remain in operation.

Note: In real life scenario, it is recommended to perform RAC check Upgrade Readiness Assessment.
This involves downloading an automated upgrade-specific health check for upgrades to Oracle Grid
Infrastructure from Oracle Support (Doc ID 1457357.1). Because upgrading Grid Infrastructure on the
practice environment has already be tested and for the sake of minimizing the practice time, I have
skipped this step from this practice document.

Note: Oracle Grid Infrastructure upgrade required patch number 21255373. This patch was already
included in the PSU that you applied in the previous lecture.

16. Make sure you have Putty sessions connected to srv1 and srv2 as root.
17. In srv1, create the directory of the new Oracle Grid home. Make grid the owner of the home
directory.
mkdir -p /u01/app/12.2.0/grid
chown grid:oinstall /u01/app/12.2.0/grid

18. Create the same directory in srv2.

ssh srv2
mkdir -p /u01/app/12.2.0/grid
chown grid:oinstall /u01/app/12.2.0/grid
exit

19. In the VirtualBox window of srv1, login as grid user.

20. Open a terminal window and change the current directory to the directory where the Oracle Grid
Infrastructure installation file was copied.
cd /media/sf_staging/12.2/grid/

[grid@srv1 grid]$ ls
linuxx64_12201_grid_home.zip

21. Unzip the linuxx64_12201_grid_home.zip file to the new Grid home directory.

unzip linuxx64_12201_grid_home.zip -d /u01/app/12.2.0/grid

22. Install the new Oracle Grid Infrastructure software. The procedure that you follow will upgrade
the running Oracle Grid instance.

a) In the terminal window, change the current directory to the new Oracle Grid directory.
cd /u01/app/12.2.0/grid

b) Set the ORACLE_HOME variable to the new Oracle Grid home directory then run gridSetup.sh

export ORACLE_HOME=/u01/app/12.2.0/grid
./gridSetup.sh

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |6

c) Follow the installer windows to install and upgrade the Oracle Grid software. Make your
responses to the installer windows are as follows:

Installer Window Response

Configuration Option Select the radio button "Upgrade Oracle Grid

Infrastructure…"

Node Selection Click on SSH Connectivity button

Enter the OS grid user password.
Click on Test button. If it reports that the SSH
connectivity is not configured, click on Setup button.

Note: If the installer reports the following error even

after SSH Connectivity is successfully set up:
INS-06006 Passwordless SSH connectivity not set up…
Try the following steps:
- exit from the installer
- In the command line, issue the following command:
ssh srv2 date
- If the command above returns the following error:
Agent Admitted Failure To Sign Using The Key
- ,issue the following command:
ssh-add
- Then try the “ssh srv2 date” command again. It
should work with no error.
- Issue the installer again.
If the procedure above does not work in your case, try
referring to Doc ID 1323714.1

Management Options click on Next button

Operating System Groups Make sure asmadmin and asmdba are selected.

Installation Location Make sure /u01/app/grid is selected.

Root Script execution Mark the check box "Automatically run configuration
scripts"
Enter the root password

Batch Selection Set the upgrade on srv2 to Batch 2.

If you keep it in Batch 1, the system will not be available
while the upgrade is going on.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |7

Prerequisites Checks If you receive error: "cvuqdisk-1.0.10-1" being

unavailable on the system, click on "Fix and Check
Again" button.
The following warning can be ignored (for a production
system they must be addressed):
- Memory is less than 8 GB
- resolv.conf Integrity
- (Linux)resolv.conf Integrity

Select Ignore All check box then click on Next button

Note: You could have run the Cluster Verification Utility

(CVU) before running the installer. Running it from within
the installer gives the same results.

Summary click on Install button

When prompted to execute configuration scripts, click on
Yes button.

Batch execution When prompted to execute the next batch, click on

confirmation "Execute Now" button.

Oracle Cluster Verification Just click on OK, Next and Close buttons.
Utility failed

23. In the Putty window, verify the upgrade has been successfully concluded:
/u01/app/12.2.0/grid/bin/crsctl check cluster -all
/u01/app/12.2.0/grid/bin/crsctl query crs activeversion

24. Sign out from srv1 VirtualBox window.

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |8

Installing Oracle Database 12.2 Software

In this section of the practice, you will install Oracle Database 12.2 software. You will not upgrade the
RAC database yet at this stage.

25. In the VirtualBox window of srv1, login as oracle user.

26. Open a terminal window and change the current directory to the directory where the Oracle
Database installation files were extracted.
cd /media/sf_staging/12.2/database/

27. Set the ORACLE_HOME variable to the new Oracle Database home directory then invoke the OUI
utility.
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/db_1
./runInstaller

28. Follow the installer windows to install the Oracle Database software only. Make your responses to
the installer windows as follows:

Installer Window Response

Configuration Security Unmark the checkbox " I wish to receive security… "
Updates

Installation Option Install Database Software only

Database Installation Options select the option "Oracle Real Application Clusters …"

Node Selection Make sure all nodes selected

Click on SSH Connectivity button
Enter the OS oracle user password.
Click on Test button.

Database Edition Make sure "Enterprise Edition" selected

Installation Location Make sure Oracle home is set to

/u01/app/oracle/product/12.2.0/db_1

Operating System Groups Make sure dba is selected as the group for all the
mandatory options

Prerequisites Checks The following warnings can be ignored:

Mark Ignore All checkbox and click on Next button

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e |9

Summary click on Install button

Execute Configuration Scripts Run the configuration scripts as root on srv1 then on
srv2.

Finish click on Close button

29. In the Putty session, switch the current user to oracle then run the Pre-Upgrade Information
Tool (preupgrade.jar) by issuing the following command. This is a single-line command. Make
sure you copy it all before you paste it in the Putty window.
/u01/app/oracle/product/12.1.0/db_1/jdk/bin/java -jar
/u01/app/oracle/product/12.2.0/db_1/rdbms/admin/preupgrade.jar FILE TEXT DIR
/home/oracle/scripts

The command should give the following output:

Preupgrade generated files:
/home/oracle/scripts/preupgrade.log
/home/oracle/scripts/preupgrade_fixups.sql
/home/oracle/scripts/postupgrade_fixups.sql

30. Examine the contents of the generated log file. It included recommendations on the steps to
perform before you start upgrading the database.

31. As sysdba, run the preupgrade_fixups.sql script.

sqlplus / as sysdba
@/home/oracle/scripts/preupgrade_fixups.sql

# to obtain list of invalid objects:

# compile invalid views
SELECT OWNER, OBJECT_NAME FROM DBA_OBJECTS WHERE STATUS='INVALID';

# to obtain list of the materialized views:

SELECT OWNER, OBJECT_NAME FROM DBA_OBJECTS WHERE OBJECT_TYPE LIKE '%MATERIA%'

32. Gather the optimizer statistics on the dictionary objects. Run the following command as sys user.
This is recommended action to reduce the upgrade time.
EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;

33. Purge the Recycle Bin

PURGE DBA_RECYCLEBIN;

34. Make sure that the parameter SEC_CASE_SENSITIVE_LOGON is set to TRUE.

SHOW PARAMETER SEC_CASE_SENSITIVE_LOGON

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e | 10

Upgrading Oracle RAC Database

In this section of the practice, you will use the DBUA to upgrade rac database to release 12.2.

35. In the VirtualBox window of srv1, make sure you are logged in as oracle user and that you have
a terminal window opened.

36. Start the dbua utility by issuing the following commands:

export ORACLE_HOME=$ORACLE_BASE/product/12.2.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH
cd $ORACLE_HOME/bin
./dbua

37. Respond to the utility windows as follows:

DBUA Window Response

Select Database Enter the sys username and password

Prerequisites Checks click on Next button

Select Upgrade Options click on Next button

Select Recovery Options select "I have my own backup and restore strategy"

Configure Management Make sure "Configure Enterprise.." is selected

Summary Click on Finish button

When the upgrade finishes (it took one hour in my case), it should display a message like the
following screenshot:

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e | 11

Post-upgrade Tasks

In this section of the practice, you will perform tasks that should be performed after the upgrade is
finished.

38. In the Putty window, switch current user to oracle and fix the ORACLE_HOME variable setting in
.bash_profile.

vi .bash_profile

...
ORACLE_HOME=$ORACLE_BASE/product/12.2.0/db_1; export ORACLE_HOME
...

39. Source the bash_profile file.

source .bash_profile

40. Test connecting to rac database using the SQL*Plus in the upgraded Oracle home
sqlplus / as sysdba

41. Copy the tnsnames.ora file from the old Oracle home directory to the upgraded Oracle home
directory.
cp /u01/app/oracle/product/12.1.0/db_1/network/admin/*.ora
/u01/app/oracle/product/12.2.0/db_1/network/admin

42. Perform the same steps in srv2.

ssh srv2
# fix the ORACLE_HOME variable setting
vi .bash_profile

# copy the network configuration files:

cp /u01/app/oracle/product/12.1.0/db_1/network/admin/*.ora
/u01/app/oracle/product/12.2.0/db_1/network/admin

exit

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e | 12

43. In srv1 and srv2, switch to grid user and fix the ORACLE_HOME variable settings and copy the
network configuration files.
su - grid

# fix the value set to the variable ORACLE_HOME

vi .bash_profile

# source the bash profile file

source .bash_profile

# copy the network files:

cp /u01/app/12.1.0/grid/network/admin/*.ora $ORACLE_HOME/network/admin

ssh srv2
# perform the same steps.

44. Run rman and delete the archivelog files (just to save the disk space). Plenty of archive logs must
have been generated by the upgrade process.
rman target /
delete archivelog all;

45. Switch the current user to oracle then start SQL*Plus and execute the postupgrade_fixups.sql
script:
su - oracle
sqlplus / as sysdba
@/home/oracle/scripts/postupgrade_fixups.sql

46. While srv1 is still in operation, restart srv2. You will restart the instances just to make sure
everything goes well after upgrade even after restarting the nodes.

47. Wait until srv2 is fully restarted and allow a few minutes for its instance to startup.

48. Make sure the database is running in both servers.

srvctl status database -d rac

49. Restart srv1.

50. Wait until srv1 is fully restarted and allow a few minutes for its instance to startup.

51. Make sure the database is running in both servers.

srvctl status database -d rac

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e | 13

52. Delete the existing database rac backupset files and take a new database backup.

# delete existing backup sets (just to avoid running out of free disk space)
rman target /
DELETE BACKUPSET;

# fix ORACLE_HOME value in the backup script:

vi /home/oracle/scripts/rman_script.sh

# perform backup:
/home/oracle/scripts/rman_script.sh

# Verify the backup was successful

tail -n 80 ~/scripts/rman.log

# verify that backupset files have been put in their place:

rman target /
list backup of database;

Oracle RAC Administration, a course by Ahmed Baraka

 Practice 12 Upgrading Oracle RAC Database P a g e | 14

Summary
• Upgrading an Oracle RAC environment includes upgrading first Oracle Grid Infrastructure and
then upgrading Oracle database.

Oracle RAC Administration, a course by Ahmed Baraka