Oracle 12c 2 Node Rac Instaliation

Documentation
==============================================================

Networking - Physical Network Interface Card:

Cluster Hosts- Type Address Subnet Mask Network Resolved-by
Node Entry setup
Node 1 lnxsrv- DNS 192.168.56.60 255.255.255.0 NIC-Host Host file - This would
dns only be a DNS SRVER

Node 1 raclnx1 Public 192.168.56.120 255.255.255.0 NIC-Host DNS/Host file

only
Node 1 raclnx1- Private 10.2.2.1 255.0.0.0 NIC– Host file/ with separate
priv Internal subnet.
Ntwk
Node 1 raclnx1- Virtual 192.168.56.130 255.255.255.0 Auto Assign DNS/host file, should
vip not ping before install.

Node 2 raclnx2 Public 192.168.56.121 255.255.255.0 NIC-Host DNS/Host file

only
Node 2 raclnx2- Private 10.2.2.2 255.0.0.0 NIC– Host file/ with separate
priv Internal subnet.
Ntwk
Node 2 raclnx2- Virtual 192.168.56.131 255.255.255.0 Auto Assign DNS/Host file, should
vip not ping before install.

Scan VIP: lnxrac- Virtual 192.168.56.140 255.255.255.0 Auto Assign lnxsrv-dns - DNS Server
scan
192.168.56.141
192.168.56.142

SCAN requires network configuration in place prior to install

• Requires either DNS or GNS (Grid Naming Service), in our case its DNS.
• GNS requires 3 IPs acquired from DHCP
• DNS recommended for “manual” configuration, which is in our case.
• 3 additional IPs on public network for SCAN-VIPs
• Single SCAN hostname resolves to 3 IPs, SCAN is the virtual hostname to provide for all clients
connecting to the cluster.
• During installation, DNS resolution provides 3 IPs which are used to create 3 SCAN-
VIP/Listener pairs scattered across the cluster.
• Use round robin for up to 3 IPs
• Set Time To Live (TTL) very short
– Especially for pre 11gR2 clients
• Beware of
– Routers w/caching DNS
– Windows DNS client
• Test with repeated nslookup / dig commands
1. The hostname command should return the fully qualified hostname as shown
below:
[root@raclnx1 ~]# hostname
raclnx1
[root@raclnx1 ~]# cat /etc/hostname
raclnx1
[root@raclnx1 ~]# vi /etc/hostname
raclnx1.mgdom.com

[root@raclnx1 ~]# hostname

raclnx1.mgdom.com
2. Check Network Requirements
[root@raclnx1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.56.120 raclnx1 raclnx1.mgdom.com
192.168.56.121 raclnx2 raclnx2.mgdom.com
10.2.2.1 raclnx1-priv
10.2.2.2 raclnx2-priv
192.168.56.130 raclnx1-vip raclnx1-vip.mgdom.com
192.168.56.131 raclnx2-vip raclnx2-vip.mgdom.com
192.168.0.106 oranet oranet.mgdom.com (To Access Internet)
192.168.56.60 lnxsrv-dns lnxsrv-dns.mgdom.com (Is a DNS Server)

[root@raclnx1 ~]# ifconfig -a

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.56.120 netmask 255.255.255.0 broadcast 192.168.56.255
inet6 fe80::a00:27ff:fe0f:f3b9 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:0f:f3:b9 txqueuelen 1000 (Ethernet)
RX packets 1646 bytes 185275 (180.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 28 bytes 4100 (4.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.108 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::a00:27ff:fed4:7f3f prefixlen 64 scopeid 0x20<link>
ether 08:00:27:d4:7f:3f txqueuelen 1000 (Ethernet)
RX packets 10125 bytes 1192920 (1.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 600 bytes 389688 (380.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 10.2.2.1 netmask 255.0.0.0 broadcast 10.255.255.255
inet6 fe80::a00:27ff:fe99:50ba prefixlen 64 scopeid 0x20<link>
ether 08:00:27:99:50:ba txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
 RX errors 0 dropped 0 overruns 0 frame 0
TX packets 24 bytes 3849 (3.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.56.60 netmask 255.255.255.0 broadcast 192.168.56.255
inet6 fe80::a00:27ff:fe0e:7b23 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:0e:7b:23 txqueuelen 1000 (Ethernet)
RX packets 1828 bytes 203733 (198.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 172 bytes 32630 (31.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

3. Config DNS and add SCAN NAME’s in DNS

[root@raclnx1 ~]# nslookup lnxrac-scan
Server: 192.168.56.60
Address: 192.168.56.60#53

Name: lnxrac-scan.mgdom.com
Address: 192.168.56.140
Name: lnxrac-scan.mgdom.com
Address: 192.168.56.141
Name: lnxrac-scan.mgdom.com
Address: 192.168.56.142
[root@raclnx1 ~]# dig lnxrac-scan.mgdom.com +noall +answer

; <<>> DiG 9.9.4-RedHat-9.9.4-29.el7 <<>> lnxrac-scan.mgdom.com +noall +answer

;; global options: +cmd
lnxrac-scan.mgdom.com. 86400 IN A 192.168.56.140
lnxrac-scan.mgdom.com. 86400 IN A 192.168.56.141
lnxrac-scan.mgdom.com. 86400 IN A 192.168.56.142

4 . Packages Installation
====================

https://docs.oracle.com/database/121/CWLIN/prelinux.htm#CWLIN168

Check Section 1 : for more details

binutils-2.23.52.0.1-12.el7.x86_64
compat-libcap1-1.10-3.el7.x86_64
compat-libstdc++-33-3.2.3-71.el7.i686
compat-libstdc++-33-3.2.3-71.el7.x86_64
gcc-4.8.2-3.el7.x86_64
gcc-c++-4.8.2-3.el7.x86_64
glibc-2.17-36.el7.i686
glibc-2.17-36.el7.x86_64
glibc-devel-2.17-36.el7.i686
glibc-devel-2.17-36.el7.x86_64
libaio-0.3.109-9.el7.i686
libaio-0.3.109-9.el7.x86_64
libaio-devel-0.3.109-9.el7.i686
libaio-devel-0.3.109-9.el7.x86_64
ksh
make-3.82-19.el7.x86_64
libXi-1.7.2-1.el7.i686
libXi-1.7.2-1.el7.x86_64
libXtst-1.2.2-1.el7.i686
libXtst-1.2.2-1.el7.x86_64
libgcc-4.8.2-3.el7.i686
libgcc-4.8.2-3.el7.x86_64
libstdc++-4.8.2-3.el7.i686
libstdc++-4.8.2-3.el7.x86_64
libstdc++-devel-4.8.2-3.el7.i686
libstdc++-devel-4.8.2-3.el7.x86_64
sysstat-10.1.5-1.el7.x86_64

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils \

compat-libcap1 \
compat-libstdc++-33 \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
libaio \
libaio-devel \
ksh \
make \
libXi \
libXtst \
libgcc \
libstdc++ \
libstdc++-devel \
sysstat \
elfutils-libelf \
elfutils-libelf-devel \
glibc-headers \
unixODBC \
unixODBC-devel |sort

package compat-libstdc++-33 is not installed

package elfutils-libelf-devel is not installed
package ksh is not installed
package libaio-devel is not installed
package pdksh is not installed
 package unixODBC-devel is not installed
package unixODBC is not installed

4. Kernel Parameter
Check Section 2 below, for more details

CVU error:
==========
Starting check for zeroconf check ...
ERROR:
PRVE-10077 : NOZEROCONF parameter was not specified or was not set to 'yes' in file
"/etc/sysconfig/network" on node "raclnx2.mgdom.com"
PRVE-10077 : NOZEROCONF parameter was not specified or was not set to 'yes' in file
"/etc/sysconfig/network" on node "raclnx1.mgdom.com"
Check for zeroconf check failed

=====================================================================================
=====================================================
cat /etc/sysconfig/network|grep -i NOZEROCONF

[root@raclnx2 ~]# ls -l /etc/sysconfig/network

-rw-r--r--. 1 root root 22 Jan 31 22:59 /etc/sysconfig/network

[root@raclnx2 ~]# cat /etc/sysconfig/network

# Created by anaconda

[root@raclnx2 ~]# vi /etc/sysconfig/network

NOZEROCONF=yes

Restart network service

====================================
service network restart
or
systemctl restart network

5. User & Group creation

Add Groups
==========
/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 dba
/usr/sbin/groupadd -g 503 oper
/usr/sbin/groupadd -g 504 asmadmin
/usr/sbin/groupadd -g 506 asmdba
/usr/sbin/groupadd -g 507 asmoper
 Add User
========
cat /etc/passwd
/usr/sbin/useradd -u 601 -c "Oracle Grid-RDBMS Owner" -g oinstall -G
dba,oper,asmadmin,asmdba,asmoper -d /home/oracle -s /bin/bash oracle

Set the password of the oracle user:

====================================
passwd oracle

6. u01 - mount point creation / disk addition / adding swap space and Creating the Directories in /u01

Check Section 3 below, for more details

# mkdir -p /u01/app/12.1.0.2/grid
# chown -R oracle:oinstall /u01
# mkdir -p /u01/app/oracle
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/

https://jameshuangsj.wordpress.com/2016/09/14/the-size-of-in-memory-file-system-mounted-at-
devshm-is-259072-megabytes-which-does-not-match-the-size-in-etcfstab-as-0-

megabytes/ -------------->>
https://docs.oracle.com/database/121/CWLIN/prelinux.htm#CWLIN219
---------------------------------------------------------------------
Cvu Error:
------------
Starting check for /dev/shm mounted as temporary file system ...
ERROR:

PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

Check for /dev/shm mounted as temporary file system failed

-------------------------------------------------------------------------
Follow below Steps
===================
Checking Shared Memory File System Mount on Linux
Ensure that the /dev/shm mount area is of type tmpfs and is mounted with the following
options:
- With rw and exec permissions set on it
- Without noexec or nosuid set on it

Use the following procedure to check the shared memory file system:
-------------------------------------------------------------------
 [oracle@raclnx1 ~]$ df -h /dev/shm
Filesystem Size Used Avail Use% Mounted on
tmpfs 2.1G 145M 2.0G 7% /dev/shm

1.) Check current mount settings. For example:

$ more /etc/fstab |grep "tmpfs"

tmpfs /dev/shm /tmpfs defaults 00

2.) If necessary, change mount settings. For example, log in as root, open the /etc/fstab file with
a text editor, and modify the tmpfs line:

tmpfs /dev/shm /tmpfs rw,exec 00

7. Verify SELinux

Below I found in link: https://docs.oracle.com/cd/E52668_01/E53499/E53499.pdf

2.10 Oracle ASM Fails to Initialize with SELinux in Enforcing Mode

Requirements for Installing Oracle 11.2.0.4 RDBMS on OL7 or RHEL7 64-bit (x86-64) (Doc ID
1962100.1)

NOTE:
-------------
UPDATE: Internal testing suggests that there is no problem running "sqlplus" with SELinux in
"enforcing" mode on RHEL7/OL7. The problem only affects RHEL5/OL5.

SELinux
=========
#verify if semodule status is enable or not
-------------------------------------------
[root@ngtsvr01 ~]# semodule -l | grep oracleasm
oracleasm 1.0.0

[root@ngtsvr01 ~]# getenforce

Enforcing

[root@ngtsvr01 ~]# cat /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing ====> check this value
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
 # minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

IF any Issues with SeLinux then u can disable SELINUX : I have not Disabled in my system
==============================================================================
Disable
-------
semodule -d oracleasm

Enable :
===========
semodule -e oracleasm

Check status :
=================
semodule -l | grep oracleasm

7.a) Setting the Disk I/O Scheduler on Linux

https://docs.oracle.com/database/121/CWLIN/prelinux.htm#CWLIN236

8. Verify and Disable NTP

Network Time Protocol Setting

========================

# /sbin/service ntpd stop

# chkconfig ntpd off
# mv /etc/ntp.conf /etc/ntp.conf.orig

Also remove the following file:

ls -l /var/run/ntpd.pid
mv /var/run/ntpd.pid /var/run/ntpd.pid.orig

9. Disable avahi-daemon it should not be running

http://www.hhutzler.de/blog/cluvfy_error/

Reference:
Cluster After Private Network Recovered if avahi Daemon is up and Running (Doc ID 1501093.1)

CVU error:
----------
Check: Daemon "avahi-daemon" not running
Node Name Running? Status
 ------------ ------------------------ ------------------------
raclnx2 yes failed
raclnx1 yes failed
Daemon not running check failed for process "avahi-daemon"

[root@raclnx2 ~]# /usr/sbin/avahi-daemon (Do not run this, it will try to start avahi)
Daemon already running on PID 715

[root@raclnx2 ~]# /usr/sbin/avahi-daemon -c

[root@raclnx2 ~]# /usr/sbin/avahi-daemon -k (To Kill)

[root@raclnx2 ~]# /usr/sbin/avahi-daemon =============> starts avahi , do not execute this.

Found user 'avahi' (UID 70) and group 'avahi' (GID 70).
Successfully dropped root privileges.
avahi-daemon 0.6.31 starting up.
WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Successfully called chroot().

[root@raclnx2 ~]# /usr/sbin/avahi-daemon -c (checking )

Process 26417 died: No such process; trying to remove PID file. (/var/run/avahi-daemon//pid)

[root@raclnx2 ~]# /usr/sbin/avahi-daemon -k

Failed to kill daemon: No such file or directory

[root@raclnx2 ~]# /usr/sbin/avahi-daemon -c

[root@raclnx2 ~]#

[root@raclnx2 ~]# /sbin/chkconfig avahi-daemon

Note: Forwarding request to 'systemctl is-enabled avahi-daemon.service'.
enabled

[root@raclnx2 ~]# /sbin/chkconfig avahi-daemon off

Note: Forwarding request to 'systemctl disable avahi-daemon.service'.
Removed symlink /etc/systemd/system/multi-user.target.wants/avahi-daemon.service.
Removed symlink /etc/systemd/system/sockets.target.wants/avahi-daemon.socket.
Removed symlink /etc/systemd/system/dbus-org.freedesktop.Avahi.service.
[root@raclnx2 ~]#

[root@raclnx2 ~]# /sbin/chkconfig avahi-daemon

Note: Forwarding request to 'systemctl is-enabled avahi-daemon.service'.
disabled

-c | --check
Return 0 as return code when avahi-daemon is already running.

-k | --kill
 Kill an already running avahi-daemon. (equivalent to sending a SIGTERM)

-h | --help
Show help

10. Shared Disk allocation to VM

Check Below => Section 4 : Shared Disk

Note :
---------
After disks are partitioned on node 1, you can refresh them on second or remaining nodes
# partprobe

11. Ping all IPs

12. SSH Setup

copy the GRID software file on to node 1 through WINSCP.
Oracle Grid Infrastructure (includes Oracle ASM, Oracle Clusterware, and Oracle Restart)
p21419221_121020_platform_5of10.zip
Note: you must download both zip files to install Oracle Grid Infrastructure.
p21419221_121020_platform_6of10.zip

Install the ssh setup file on node 1

cd /u01/software/grid/sshsetup
./sshUserSetup.sh -hosts "raclnx1 raclnx2" -user oracle -advanced -noPromptPassphrase

Verify the ssh connectivity from both nodes.

ssh all nodes self and remote with names and IP's

To disable the banner while ssh run the following command.

cd $HOME
touch .hushlogin
{then check whether banner is coming still}

In rare cases, Oracle Clusterware installation may fail during the "AttachHome" operation when
the remote node closes the SSH connection.
To avoid this problem, set the following parameter in the
SSH daemon configuration file /etc/ssh/sshd_config on all cluster nodes to set the timeout wait
to unlimited:
[root@raclnx1 ~]# cat /etc/ssh/sshd_config |grep LoginGraceTime
#LoginGraceTime 2m

[root@raclnx1 ~]# vi /etc/ssh/sshd_config (Modify above 2m to 0)

LoginGraceTime 0

[root@raclnx1 ~]# cat /etc/ssh/sshd_config |grep LoginGraceTime

#LoginGraceTime 2m
LoginGraceTime 0
13 As root user Install CVU Rpm (cvuqdisk-1.0.9-1.rpm) on both the nodes cvuqdisk Available on Grid
Infrastructure Media (under rpm folder)

[root@raclnx1 rpm]$ ls -ltrh /u01/software/grid/rpm/cvu*

-rw-r--r--. 1 oracle oinstall 8.1K Aug 26 2013 /u01/software/grid/rpm/cvuqdisk-1.0.9-1.rpm

As root on both nodes

---------------------------
cd /u01/software/grid/rpm/
export CVUQDISK_GRP=oinstall
echo $CVUQDISK_GRP
cd /u01/software/grid/rpm/
rpm -ivh cvuqdisk-1.0.9-1.rpm

[root@raclnx1 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm

Preparing... ################################# [100%]
Using default group oinstall to install package
Updating / installing...
1:cvuqdisk-1.0.9-1 ################################# [100%]

rpm -qa cvuqdisk

[root@raclnx1 rpm]# rpm -qa cvuqdisk

cvuqdisk-1.0.9-1.x86_64

14. ASM Lib Configuration all Nodes

See Section 5 Below

15. Create ASM Disk using asmlib Only One Node

See Section 6 Below

16. RUN Cluverify (Cluster requirement verification) as oracle user

id
cd /u01/software/logs/
pwd
/u01/software/grid/runcluvfy.sh comp nodereach -n CCTSVR03,CCTSVR04 -verbose |tee
nodereach_"$(date +\%d"."\%b"."\%Y"."\%H"."\%M"."\%S)".txt
/u01/software/grid/runcluvfy.sh stage -post hwos -n CCTSVR03,CCTSVR04 -verbose| tee
posthwos_"$(date +\%d"."\%b"."\%Y"."\%H"."\%M"."\%S)".txt
/u01/software/grid/runcluvfy.sh stage -pre crsinst -n CCTSVR03,CCTSVR04 -verbose| tee
precrsinst_"$(date +\%d"."\%b"."\%Y"."\%H"."\%M"."\%S)".txt
17. Run Install GRID
export ORACLE_BASE=/u01/app/oracle
cd /u01/software/grid
./runInstaller &

20. cluverify post GRID

id
cd /u01/software/logs/
pwd
/u01/software/grid/runcluvfy.sh stage -post crsinst -n CCTSVR03,CCTSVR04 -verbose| tee
postcrsinst_"$(date +\%d"."\%b"."\%Y"."\%H"."\%M"."\%S)".txt

20. cluverify for RDBMS

id
cd /u01/software/logs/
pwd
/u01/software/grid/runcluvfy.sh stage -pre dbinst -n CCTSVR03,CCTSVR04 -verbose | tee
predbinst_"$(date +\%d"."\%b"."\%Y"."\%H"."\%M"."\%S)".txt

22. Install RDBMS Software using OUI

Oracle Database (includes Oracle Database, Oracle RAC, and Deinstall)

p21419221_121020_platform_1of10.zip
Note: you must download both zip files to install Oracle Database Enterprise Edition.
p21419221_121020_platform_2of10.zip

export ORACLE_BASE=/u01/app/oracle
cd /u01/software/database/
./runInstaller &

23. create diskgroup using asmca

24.DBCA to create database

######################End of Install Steps ###################################

Section 1 : Install the Packages

**********************************************************************************

Create Repository for Yum

===========================

Mount CD rom
--------------------
mount /dev/sr0 /mnt

cd /etc/yum.repos.d

1) vi local.repo
-----------------
[local]
name=local
baseurl=file:///mnt/
gpgcheck=0
enabled=1

2) vi global.repo
-----------------
[globalrepo]
name=globalrepo
baseurl=http://mirror.centos.org/centos/7/os/x86_64/
enabled=1
gpgcheck=0

[root@raclnx1 yum.repos.d]# pwd

/etc/yum.repos.d

[root@raclnx1 yum.repos.d]# ls -ltrh

total 20K
-rw-r--r--. 1 root root 5.0K Dec 2 04:19 public-yum-ol7.repo
-rw-r--r--. 1 root root 358 Feb 4 19:10 redhat.repo
-rw-r--r--. 1 root root 61 Feb 4 19:59 local.repo
-rw-r--r--. 1 root root 104 Feb 6 16:45 global.repo

[root@raclnx1 yum.repos.d]# yum repolist

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
repo id repo name

status
globalrepo globalrepo

9,363
local local

4,620
ol7_UEKR4/x86_64 Latest Unbreakable Enterprise Kernel Release 4 for Oracle
Linux 7Server (x86_64)

277
ol7_latest/x86_64 Oracle Linux 7Server Latest (x86_64)

18,362
repolist: 32,622

1) yum install compat-libstdc++-33

[root@raclnx1 yum.repos.d]# yum install compat-libstdc++-33

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
globalrepo |

3.6 kB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package compat-libstdc++-33.x86_64 0:3.2.3-72.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
compat-libstdc++-33 x86_64 3.2.3-72.el7
globalrepo

191 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total size: 191 k

Installed size: 811 k
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check

Running transaction test

Transaction test succeeded
Running transaction
Installing : compat-libstdc++-33-3.2.3-72.el7.x86_64

1/1
local/productid |

1.6 kB 00:00:00
Verifying : compat-libstdc++-33-3.2.3-72.el7.x86_64

1/1

Installed:
compat-libstdc++-33.x86_64 0:3.2.3-72.el7

Complete!

********************************************************end of install ************

2) yum install elfutils-libelf-devel

[root@raclnx1 yum.repos.d]# yum install elfutils-libelf-devel

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package elfutils-libelf-devel.x86_64 0:0.166-2.el7 will be installed
--> Processing Dependency: elfutils-libelf(x86-64) = 0.166-2.el7 for package: elfutils-libelf-devel-0.166-
2.el7.x86_64
--> Processing Dependency: pkgconfig(zlib) for package: elfutils-libelf-devel-0.166-2.el7.x86_64
--> Running transaction check
---> Package elfutils-libelf.x86_64 0:0.163-3.el7 will be updated
--> Processing Dependency: elfutils-libelf(x86-64) = 0.163-3.el7 for package: elfutils-0.163-3.el7.x86_64
--> Processing Dependency: elfutils-libelf(x86-64) = 0.163-3.el7 for package: elfutils-libs-0.163-
3.el7.x86_64
---> Package elfutils-libelf.x86_64 0:0.166-2.el7 will be an update
---> Package zlib-devel.x86_64 0:1.2.7-17.el7 will be installed
--> Processing Dependency: zlib = 1.2.7-17.el7 for package: zlib-devel-1.2.7-17.el7.x86_64
--> Running transaction check
---> Package elfutils.x86_64 0:0.163-3.el7 will be updated
---> Package elfutils.x86_64 0:0.166-2.el7 will be an update
---> Package elfutils-libs.x86_64 0:0.163-3.el7 will be updated
---> Package elfutils-libs.x86_64 0:0.166-2.el7 will be an update
---> Package zlib.x86_64 0:1.2.7-15.el7 will be updated
---> Package zlib.x86_64 0:1.2.7-17.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
elfutils-libelf-devel x86_64 0.166-2.el7 globalrepo

37 k
Installing for dependencies:
zlib-devel x86_64 1.2.7-17.el7 globalrepo

50 k
Updating for dependencies:
elfutils x86_64 0.166-2.el7 globalrepo

280 k
elfutils-libelf x86_64 0.166-2.el7 globalrepo

208 k
elfutils-libs x86_64 0.166-2.el7 globalrepo

262 k
zlib x86_64 1.2.7-17.el7 globalrepo

90 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package (+1 Dependent package)
Upgrade ( 4 Dependent packages)

Total download size: 926 k

Is this ok [y/d/N]: y
Downloading packages:
No Presto metadata available for globalrepo
(1/6): elfutils-libelf-0.166-2.el7.x86_64.rpm
|

208 kB 00:00:03
(2/6): elfutils-0.166-2.el7.x86_64.rpm
|

280 kB 00:00:03
(3/6): elfutils-libelf-devel-0.166-2.el7.x86_64.rpm
|

37 kB 00:00:00
(4/6): zlib-1.2.7-17.el7.x86_64.rpm
|

90 kB 00:00:00
(5/6): zlib-devel-1.2.7-17.el7.x86_64.rpm
|

50 kB 00:00:00
(6/6): elfutils-libs-0.166-2.el7.x86_64.rpm
|

262 kB 00:00:01
-------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------

---------------------
Total 195 kB/s |

926 kB 00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : zlib-1.2.7-17.el7.x86_64

1/10
Updating : elfutils-libelf-0.166-2.el7.x86_64

2/10
Updating : elfutils-libs-0.166-2.el7.x86_64

3/10
Installing : zlib-devel-1.2.7-17.el7.x86_64
 4/10
Installing : elfutils-libelf-devel-0.166-2.el7.x86_64

5/10
Updating : elfutils-0.166-2.el7.x86_64

6/10
Cleanup : elfutils-0.163-3.el7.x86_64

7/10
Cleanup : elfutils-libs-0.163-3.el7.x86_64

8/10
Cleanup : elfutils-libelf-0.163-3.el7.x86_64

9/10
Cleanup : zlib-1.2.7-15.el7.x86_64

10/10
Verifying : zlib-devel-1.2.7-17.el7.x86_64

1/10
Verifying : elfutils-libs-0.166-2.el7.x86_64

2/10
Verifying : elfutils-libelf-devel-0.166-2.el7.x86_64

3/10
Verifying : elfutils-0.166-2.el7.x86_64

4/10
Verifying : zlib-1.2.7-17.el7.x86_64

5/10
Verifying : elfutils-libelf-0.166-2.el7.x86_64

6/10
Verifying : elfutils-libs-0.163-3.el7.x86_64

7/10
Verifying : elfutils-0.163-3.el7.x86_64

8/10
Verifying : zlib-1.2.7-15.el7.x86_64

9/10
Verifying : elfutils-libelf-0.163-3.el7.x86_64
 10/10

Installed:
elfutils-libelf-devel.x86_64 0:0.166-2.el7

Dependency Installed:
zlib-devel.x86_64 0:1.2.7-17.el7

Dependency Updated:
elfutils.x86_64 0:0.166-2.el7 elfutils-libelf.x86_64 0:0.166-2.el7 elfutils-libs.x86_64
0:0.166-2.el7 zlib.x86_64 0:1.2.7-

17.el7

Complete!

********************************************************end of install ************

3) yum install ksh

[root@raclnx1 yum.repos.d]# /bin/ksh

-bash: /bin/ksh: No such file or directory

[root@raclnx1 yum.repos.d]# yum install ksh

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package ksh.x86_64 0:20120801-26.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
ksh x86_64 20120801-26.el7 globalrepo
 883 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total download size: 883 k

Installed size: 3.1 M
Is this ok [y/d/N]: y
Downloading packages:
ksh-20120801-26.el7.x86_64.rpm
|

883 kB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : ksh-20120801-26.el7.x86_64

1/1
Verifying : ksh-20120801-26.el7.x86_64

1/1

Installed:
ksh.x86_64 0:20120801-26.el7

Complete!

[root@raclnx1 yum.repos.d]# /bin/ksh

# exit

********************************************************end of install ************

4) yum install libaio-devel

[root@raclnx1 Packages]# yum install libaio-devel

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package libaio-devel.x86_64 0:0.3.109-13.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
libaio-devel x86_64 0.3.109-13.el7 globalrepo

13 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total download size: 13 k

Installed size: 7.8 k
Is this ok [y/d/N]: y
Downloading packages:
libaio-devel-0.3.109-13.el7.x86_64.rpm
|

13 kB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libaio-devel-0.3.109-13.el7.x86_64

1/1
Verifying : libaio-devel-0.3.109-13.el7.x86_64

1/1

Installed:
libaio-devel.x86_64 0:0.3.109-13.el7
Complete!

********************************************************end of install ************

5) yum install pdksh

[root@raclnx1 yum.repos.d]# cat whiteboxrepo.repo

[whiteboxrepo]
name=whiteboxrepo
baseurl=http://ftp.pbone.net/mirror/www.whiteboxlinux.org/whitebox/4/en/os/i386/
enabled=1
gpgcheck=0

[root@raclnx1 yum.repos.d]# yum repolist

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
repo id repo name

status
globalrepo globalrepo

9,363
local local

4,620
ol7_UEKR4/x86_64 Latest Unbreakable Enterprise Kernel Release 4 for Oracle
Linux 7Server (x86_64)

277
ol7_latest/x86_64 Oracle Linux 7Server Latest (x86_64)

18,362
whiteboxrepo whiteboxrepo

2,238
repolist: 34,860

[root@raclnx1 yum.repos.d]# yum install pdksh

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package pdksh.i386 0:5.2.14-30 will be installed
--> Processing Dependency: libc.so.6(GLIBC_2.0) for package: pdksh-5.2.14-30.i386
--> Processing Dependency: libc.so.6(GLIBC_2.1) for package: pdksh-5.2.14-30.i386
--> Processing Dependency: libc.so.6(GLIBC_2.2) for package: pdksh-5.2.14-30.i386
--> Processing Dependency: libc.so.6(GLIBC_2.3) for package: pdksh-5.2.14-30.i386
--> Processing Dependency: libc.so.6(GLIBC_2.3.3) for package: pdksh-5.2.14-30.i386
--> Processing Dependency: libc.so.6 for package: pdksh-5.2.14-30.i386
--> Running transaction check
---> Package glibc.x86_64 0:2.17-105.el7 will be updated
--> Processing Dependency: glibc = 2.17-105.el7 for package: glibc-devel-2.17-105.el7.x86_64
--> Processing Dependency: glibc = 2.17-105.el7 for package: glibc-headers-2.17-105.el7.x86_64
--> Processing Dependency: glibc = 2.17-105.el7 for package: glibc-common-2.17-105.el7.x86_64
---> Package glibc.i686 0:2.17-157.el7_3.1 will be installed
--> Processing Dependency: libfreebl3.so for package: glibc-2.17-157.el7_3.1.i686
--> Processing Dependency: libfreebl3.so(NSSRAWHASH_3.12.3) for package: glibc-2.17-157.el7_3.1.i686
---> Package glibc.x86_64 0:2.17-157.el7_3.1 will be an update
--> Running transaction check
---> Package glibc-common.x86_64 0:2.17-105.el7 will be updated
---> Package glibc-common.x86_64 0:2.17-157.el7_3.1 will be an update
---> Package glibc-devel.x86_64 0:2.17-105.el7 will be updated
---> Package glibc-devel.x86_64 0:2.17-157.el7_3.1 will be an update
---> Package glibc-headers.x86_64 0:2.17-105.el7 will be updated
---> Package glibc-headers.x86_64 0:2.17-157.el7_3.1 will be an update
---> Package nss-softokn-freebl.x86_64 0:3.16.2.3-13.el7_1 will be updated
---> Package nss-softokn-freebl.i686 0:3.16.2.3-14.4.el7 will be installed
---> Package nss-softokn-freebl.x86_64 0:3.16.2.3-14.4.el7 will be an update
--> Processing Conflict: ksh-20120801-26.el7.x86_64 conflicts pdksh
--> Finished Dependency Resolution
Error: ksh conflicts with pdksh-5.2.14-30.i386
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

********************************************************end of install ************

6) yum install unixODBC

[root@raclnx1 yum.repos.d]# yum install unixODBC

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package unixODBC.x86_64 0:2.3.1-11.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================
=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
unixODBC x86_64 2.3.1-11.el7 globalrepo

413 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total download size: 413 k

Installed size: 1.2 M
Is this ok [y/d/N]: y
Downloading packages:
unixODBC-2.3.1-11.el7.x86_64.rpm
|

413 kB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : unixODBC-2.3.1-11.el7.x86_64

1/1
Verifying : unixODBC-2.3.1-11.el7.x86_64

1/1

Installed:
unixODBC.x86_64 0:2.3.1-11.el7

Complete!

********************************************************end of install ************

7) yum install unixODBC-devel

[root@raclnx1 yum.repos.d]# yum install unixODBC-devel
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package unixODBC-devel.x86_64 0:2.3.1-11.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
unixODBC-devel x86_64 2.3.1-11.el7
globalrepo

55 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total download size: 55 k

Installed size: 186 k
Is this ok [y/d/N]: y
Downloading packages:
unixODBC-devel-2.3.1-11.el7.x86_64.rpm
|

55 kB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
 Installing : unixODBC-devel-2.3.1-11.el7.x86_64

1/1
Verifying : unixODBC-devel-2.3.1-11.el7.x86_64

1/1

Installed:
unixODBC-devel.x86_64 0:2.3.1-11.el7

Complete!

********************************************************end of install ************

Download ASM Package

-----------------------------
mkdir /root/asmpkg

[root@raclnx1 asmpkg]# pwd

/root/asmpkg

Download ASMLib
----------------
http://www.oracle.com/technetwork/server-storage/linux/asmlib/rhel7-2773795.html

[root@raclnx1 asmpkg]# wget http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.12-

1.el7.x86_64.rpm

--2017-02-06 22:12:02-- http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.12-

1.el7.x86_64.rpm
Resolving download.oracle.com (download.oracle.com)... 104.79.197.30, 104.79.197.11
Connecting to download.oracle.com (download.oracle.com)|104.79.197.30|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 19360 (19K) [application/x-redhat-package-manager]
Saving to: âoracleasmlib-2.0.12-1.el7.x86_64.rpmâ

100%
[====================================================================================
===============================================================>] 19,360

--.-K/s in 0.03s

2017-02-06 22:12:03 (685 KB/s) - âoracleasmlib-2.0.12-1.el7.x86_64.rpmâ saved [19360/19360]

Download ASMsupport
------------------------
[root@raclnx1 asmpkg]# wget
http://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/getPackage/oracleasm-support-2.1.8-
3.el7.x86_64.rpm
SSSS
--2017-02-06 22:12:19--
http://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/getPackage/oracleasm-support-2.1.8-
3.el7.x86_64.rpm
Resolving yum.oracle.com (yum.oracle.com)... 104.79.197.22, 104.79.197.30
Connecting to yum.oracle.com (yum.oracle.com)|104.79.197.22|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 80968 (79K) [application/x-rpm]
Saving to: âoracleasm-support-2.1.8-3.el7.x86_64.rpmâ

100%
[====================================================================================
===============================================================>] 80,968

--.-K/s in 0.03s

#####################################################################################
################
[root@raclnx1 yum.repos.d]# pwd
/etc/yum.repos.d

[root@raclnx1 yum.repos.d]# cat public-yum-ol7.repo

[ol7_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/latest/$basearch/
gpgcheck=0
enabled=1

Download ASMkernel
-----------------------------
[root@raclnx1 yum.repos.d]# yum install kmod-oracleasm
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
ol7_UEKR4 |

1.2 kB 00:00:00
ol7_latest |

1.4 kB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package kmod-oracleasm.x86_64 0:2.0.8-17.0.1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
kmod-oracleasm x86_64 2.0.8-17.0.1.el7
ol7_latest

277 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total size: 277 k

Installed size: 1.4 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : kmod-oracleasm-2.0.8-17.0.1.el7.x86_64

1/1
Verifying : kmod-oracleasm-2.0.8-17.0.1.el7.x86_64

1/1

Installed:
kmod-oracleasm.x86_64 0:2.0.8-17.0.1.el7

Complete!
***************************end of ASMkernel install ************

[root@raclnx1 ~]# rpm -q kmod-oracleasm

kmod-oracleasm-2.0.8-17.0.1.el7.x86_64

[root@raclnx1 ~]# yum install oracleasmlib

Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
No package oracleasmlib available.
Error: Nothing to do
[root@raclnx1 ~]# yum install oracleasm-support
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-manager
This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use
subscription-manager to assign subscriptions.
Resolving Dependencies
--> Running transaction check
---> Package oracleasm-support.x86_64 0:2.1.8-3.1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================
===================================================================================

=====================
Package Arch Version Repository

Size
=====================================================================================
===================================================================================

=====================
Installing:
oracleasm-support x86_64 2.1.8-3.1.el7
ol7_latest

79 k

Transaction Summary
=====================================================================================
===================================================================================

=====================
Install 1 Package

Total download size: 79 k

Installed size: 243 k
Is this ok [y/d/N]: y
Downloading packages:
oracleasm-support-2.1.8-3.1.el7.x86_64.rpm
|

79 kB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : oracleasm-support-2.1.8-3.1.el7.x86_64

1/1
Note: Forwarding request to 'systemctl enable oracleasm.service'.
Created symlink from /etc/systemd/system/multi-user.target.wants/oracleasm.service to
/usr/lib/systemd/system/oracleasm.service.
Verifying : oracleasm-support-2.1.8-3.1.el7.x86_64

1/1

Installed:
oracleasm-support.x86_64 0:2.1.8-3.1.el7

Complete!

************************end of install ***********************

Install ASM RPMS

[root@raclnx1 asmpkg]# ls -ltrh

total 100K
-rw-r--r--. 1 root root 80K Jul 10 2014 oracleasm-support-2.1.8-3.el7.x86_64.rpm
-rw-r--r--. 1 root root 19K Oct 15 2015 oracleasmlib-2.0.12-1.el7.x86_64.rpm

rpm -ivh oracleasmlib-2.0.12-1.el7.x86_64.rpm

rpm -ivh oracleasm-support-2.1.8-3.el7.x86_64.rpm

[root@raclnx1 asmpkg]# rpm -ivh oracleasmlib-2.0.12-1.el7.x86_64.rpm

warning: oracleasmlib-2.0.12-1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03:
NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:oracleasmlib-2.0.12-1.el7 ################################# [100%]

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' kmod-oracleasm \

oracleasm-support \
oracleasmlib

[root@raclnx1 asmpkg]# rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' kmod-oracleasm \

> oracleasm-support \
> oracleasmlib
kmod-oracleasm-2.0.8-17.0.1.el7 (x86_64)
oracleasm-support-2.1.8-3.1.el7 (x86_64)
oracleasmlib-2.0.12-1.el7 (x86_64)

Prashant repository location

------------------------------
[root@ngtsvr01 yum.repos.d]# cat rhel-local.repo
[rhel-local.repo]
name=rhel-local
baseurl=file:///var/ftp/pub/rhel
enabled=1
gpgcheck=0
*************************End of Section 1 : Install the Packages *************************

Section 2 : Kernel Parameters setting

******************************************************************************

vi /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
# +---------------------------------------------------------+
# | SHARED MEMORY |
# +---------------------------------------------------------+
# 4294967296 is 4GB RAM
kernel.shmmax = 4294967296
kernel.shmmni = 4096
kernel.shmall = 4294967296

# +---------------------------------------------------------+
# | SEMAPHORES |
# | ---------- |
#| |
# | SEMMSL_value SEMMNS_value SEMOPM_value SEMMNI_value |
#| |
# +---------------------------------------------------------+
kernel.sem = 500 32767 100 256

# +-------------------------------------------------------------------------+
# | FILE HANDLES
# |fs.file-max = 512 x processes (for example 7680000 for 15000 processes) |
# --------------------------------------------------------------------------+
fs.file-max = 7680000
fs.aio-max-nr = 1048576

# +---------------------------------------------------------+
# | LOCAL IP RANGE |
# ----------------------------------------------------------+
net.ipv4.ip_local_port_range = 9000 65500

# +---------------------------------------------------------+
# | NETWORKING |
# ----------------------------------------------------------+
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576

# +---------------------------------------------------------+
# | Huge Pages |
# ----------------------------------------------------------+
vm.nr_hugepages=300
# End of file

#*****************************************************************************
Configure Limits

#*****************************************************************************
vi /etc/security/limits.conf

oracle soft nproc 131072

oracle hard nproc 131072
oracle soft nofile 300000
oracle hard nofile 300000
oracle soft stack 10240
oracle soft core unlimited
oracle hard core unlimited

# Oracle-Validated setting for memlock soft-hard limit is 300mb .

#this matches huge page settings. specified in kb.
# 600*1024 = 614400 KB

oracle soft memlock 614400

oracle hard memlock 614400
# End of file
#************************************************************************************
 Add below code to /etc/profile
****************************************************
vi /etc/profile

if [ $USER = "oracle" ]; then

if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 300000
else
ulimit -u 131072 -n 300000
fi
umask 022
fi

==========================================================================
[root@CCTSVR03 ~]# /sbin/sysctl -a |egrep 'shmmax|shmmni|shmall|hugepages|rmem|wmem|
ip_local_port_range|file-max|aio-max-nr|sem'
fs.aio-max-nr = 1048576
fs.file-max = 3225638
kernel.sem = 250 32000 32 128
kernel.sem_next_id = -1
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
net.core.rmem_default = 212992
net.core.rmem_max = 212992
net.core.wmem_default = 212992
net.core.wmem_max = 212992
net.ipv4.ip_local_port_range = 32768 61000
net.ipv4.tcp_rmem = 4096 87380 6291456
net.ipv4.tcp_wmem = 4096 16384 4194304
net.ipv4.udp_rmem_min = 4096
net.ipv4.udp_wmem_min = 4096
vm.hugepages_treat_as_movable = 0
vm.lowmem_reserve_ratio = 256 256 32
vm.nr_hugepages = 0
vm.nr_hugepages_mempolicy = 0
vm.nr_overcommit_hugepages = 0

To avoid restart => /sbin/sysctl -p

sysctl -w vm.nr_hugepages=23040

Verify Ulimits or above setting by below commands

------------------------------------------------------

Syntax
ulimit [-acdfHlmnpsStuv] [limit]
Options

-S Change and report the soft limit associated with a resource.

-H Change and report the hard limit associated with a resource.

-a All current limits are reported.

-c The maximum size of core files created.
-d The maximum size of a process's data segment.
-f The maximum size of files created by the shell(default option)
-l The maximum size that can be locked into memory.
-m The maximum resident set size.
-n The maximum number of open file descriptors.
-p The pipe buffer size.
-s The maximum stack size.
-t The maximum amount of cpu time in seconds.
-u The maximum number of processes available to a single user.
-v The maximum amount of virtual memory available to the process.
ulimit provides control over the resources available to the shell and to processes started by it, on s

The maximum number of processes available to a single user.

--------------------------------------------------------------
ulimit -Su
ulimit -Hu

echo "host: nproc(Soft) = $(ulimit -Su)"

echo "host: nproc(Hard) = $(ulimit -Hu)"

The maximum number of open file descriptors.

--------------------------------------------------------------
ulimit -Sn
ulimit -Hn

echo "host: nofile(Soft) = $(ulimit -Sn)"

echo "host: nofile(Hard) = $(ulimit -Hn)"

The maximum stack size.

--------------------------------------------------------------
ulimit -Ss
ulimit -Hs

echo "host: stack(Soft) = $(ulimit -Ss)"

echo "host: stack(Hard) = $(ulimit -Hs)"

The maximum size of core files created.

--------------------------------------------------------------
ulimit -Sc
ulimit -Hc

echo "host: core(Soft) = $(ulimit -Sc)"

echo "host: core(Hard) = $(ulimit -Hc)"

The maximum size that can be locked into memory.(Huge Pages)

--------------------------------------------------------------
ulimit -Sl
ulimit -Hl

echo "host: memlock (Soft) = $(ulimit -Sl)"

echo "host: memlock (Hard) = $(ulimit -Hl)"

Clu Verify Error

=====================
ERROR:
PRVE-0068 : Maximum locked memory setting is less than the recommended value [Expected =
"3947339", Actual="614400"] when huge pages are enabled on node

"raclnx2.mgdom.com".
PRVE-0068 : Maximum locked memory setting is less than the recommended value [Expected =
"3947339", Actual="614400"] when huge pages are enabled on node

"raclnx1.mgdom.com".
Check for Maximum locked memory check failed

******************End of Section 2 : Kernel Parameters setting *************************

Section 3 : Add Hard Disk /u01

*********************************************************************
Existing Disk where OS is Installed
==================================
[root@raclnx1 ~]# fdisk -l

Disk /dev/sda: 16.1 GB, 16106127360 bytes, 31457280 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00032e91

Device Boot Start End Blocks Id System

/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 31457279 15215616 8e Linux LVM
 Disk /dev/mapper/rhel-root: 13.9 GB, 13925089280 bytes, 27197440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-swap: 1610 MB, 1610612736 bytes, 3145728 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Now add new disk in Virtual Box

================================

[root@raclnx1 ~]# fdisk -l

Disk /dev/sda: 16.1 GB, 16106127360 bytes, 31457280 sectors ==> existing disk
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00032e91

Device Boot Start End Blocks Id System

/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 31457279 15215616 8e Linux LVM

Disk /dev/sdb: 30.1 GB, 30064771072 bytes, 58720256 sectors ==> new added disk 28 GB but
showing 30 Gb
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-root: 13.9 GB, 13925089280 bytes, 27197440 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-swap: 1610 MB, 1610612736 bytes, 3145728 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@raclnx1 ~]# cd /dev

 [root@raclnx1 dev]# ls -ltr sd*
brw-rw----. 1 root disk 8, 16 Feb 19 14:00 sdb ==>>new disk
brw-rw----. 1 root disk 8, 0 Feb 19 14:00 sda==> existing disk (root / ) with below two partitions
brw-rw----. 1 root disk 8, 1 Feb 19 14:00 sda1
brw-rw----. 1 root disk 8, 2 Feb 19 14:00 sda2

Very the existing /etc/fstab file

===================================
[root@raclnx1 dev]# cat /etc/fstab

# /etc/fstab
# Created by anaconda on Tue Jan 31 22:41:03 2017
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/rhel-root / xfs defaults 00
UUID=a46a3693-4de6-441b-a60c-36245e8e0ea4 /boot xfs defaults 00
/dev/mapper/rhel-swap swap swap defaults 00

http://www.idevelopment.info/data/Oracle/DBA_tips/Linux/LINUX_20.shtml#Disk Configuration
------------------------------------------------------------------------------------------
The Linux operating system was installed on the first disk /dev/sda.
The second hard disk (/dev/sdb) will be used to store the Oracle Database 11g Software and
mounted to the directory /u01.

[ CREATE SINGLE PARTITION ON SECOND INTERNAL HARD DISK (/dev/sdb)]

================================================================
[root@raclnx1 dev]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table

Building a new DOS disklabel with disk identifier 0xc25daa4e.

Command (m for help): n ==> type n for new partition

Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p ==> type p for primary partition
Partition number (1-4, default 1): 1 ==> partition number type 1
First sector (2048-41943039, default 2048): ==> Press blank Enter to use default first sector
which is 2048
Using default value 2048
 Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): ==> Press blank Enter
to use entire disk
Using default value 41943039
Partition 1 of type Linux and of size 20 GiB is set

Command (m for help): p ==> Type p to print details of partition

Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xc25daa4e

Device Boot Start End Blocks Id System

/dev/sdb1 2048 41943039 20970496 83 Linux

Command (m for help): w ===> type w to save the partition

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.
[root@raclnx1 dev]#

Verify the partition

======================
[root@raclnx1 dev]# fdisk /dev/sdb -l

Disk /dev/sdb: 30.1 GB, 30064771072 bytes, 58720256 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1f06bf0b

Device Boot Start End Blocks Id System

/dev/sdb1 2048 58720255 29359104 83 Linux => check partition name as /dev/sdb1

verify new parttion disk

============================
cd /dev
[root@raclnx1 dev]# ls -ltr sd*
[root@raclnx1 dev]# ls -ltr sd*
brw-rw----. 1 root disk 8, 0 Feb 19 14:00 sda
brw-rw----. 1 root disk 8, 1 Feb 19 14:00 sda1
brw-rw----. 1 root disk 8, 2 Feb 19 14:00 sda2
brw-rw----. 1 root disk 8, 16 Feb 19 14:10 sdb
brw-rw----. 1 root disk 8, 17 Feb 19 14:10 sdb1 ==> this new partition got created
now create filesystem
==========================
mkfs.ext3 /dev/sdb1
OR
mkfs.ext3 -b 4096 /dev/sdb1

[root@raclnx1 dev]# mkfs.ext3 /dev/sdb1

mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1835008 inodes, 7339776 blocks
366988 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
224 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000

Allocating group tables: done

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now create directory u01

============================
mkdir /u01

Edit fstab file for automount

=================================
vi /etc/fstab
/dev/sdb1 /u01 ext3 defaults 1 1

[root@raclnx1 dev]# cat /etc/fstab |grep u01

/dev/sdb1 /u01 ext3 defaults 1 1

# [ MOUNT THE NEW FILE SYSTEMS ]

df -h |grep u01
# mount /u01

[root@raclnx1 dev]# mount /u01

[root@raclnx1 dev]# df -h
 Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel-root 13G 5.3G 7.8G 41% /
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 84K 1.9G 1% /dev/shm
tmpfs 1.9G 8.8M 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 497M 213M 285M 43% /boot
tmpfs 380M 16K 380M 1% /run/user/42
tmpfs 380M 0 380M 0% /run/user/0
/dev/sdb1 28G 45M 26G 1% /u01

[root@raclnx1 dev]# cd /u01

[root@raclnx1 u01]# ls -ltrh

total 16K
drwx------. 2 root root 16K Jun 18 16:41 lost+found

[root@raclnx1 u01]# touch abc.txt

[root@raclnx1 u01]# ls -ltrh

total 16K
drwx------. 2 root root 16K Jun 18 16:41 lost+found
-rw-r--r--. 1 root root 0 Jun 18 16:47 abc.txt

To Increase Swap space

========================
https://mohamedazar.com/linux-commands/
df -m
mkdir -p /u01/tmp
dd if=/dev/zero of=/u01/tmp/extraswap bs=1M count=2048 (2GB)
mkswap /u01/tmp/extraswap
swapon /u01/tmp/extraswap
cp /etc/fstab /etc/fstab.backup
vi /etc/fstab
****Add lines
/u01/tmp/extraswap swap swap defaults 00

**********End of Section 3 => /u01 mountpoint creation ******************************

Section 4 : Shared Disk Allocation

*************************************************************************************
[root@raclnx1 ~]# lsscsi
[0:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda
[0:0:1:0] disk ATA VBOX HARDDISK 1.0 /dev/sdb
[1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0
[2:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdc
[3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdd
[4:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sde
[5:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdf
[6:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdg
[7:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdh
[8:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdi
[9:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdj

[root@raclnx1 ~]# fdisk -l |grep Disk

Disk /dev/sda: 16.1 GB, 16106127360 bytes, 31457280 sectors
Disk label type: dos
Disk identifier: 0x00032e91
Disk /dev/sdb: 30.1 GB, 30064771072 bytes, 58720256 sectors
Disk label type: dos
Disk identifier: 0x1f06bf0b
Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors ----> New disks
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sde: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdf: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdg: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdh: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdj: 4294 MB, 4294967296 bytes, 8388608 sectors
Disk /dev/sdi: 4294 MB, 4294967296 bytes, 8388608 sectors ----> New disks ends
Disk /dev/mapper/rhel-root: 13.9 GB, 13925089280 bytes, 27197440 sectors
Disk /dev/mapper/rhel-swap: 1610 MB, 1610612736 bytes, 3145728 sectors

cd /dev
[root@raclnx1 dev]# ls -ltr sd*
brw-rw----. 1 root disk 8, 16 Feb 19 22:41 sdb
brw-rw----. 1 root disk 8, 0 Feb 19 22:41 sda
brw-rw----. 1 root disk 8, 2 Feb 19 22:41 sda2
brw-rw----. 1 root disk 8, 1 Feb 19 22:41 sda1
brw-rw----. 1 root disk 8, 96 Feb 19 22:41 sdg
brw-rw----. 1 root disk 8, 32 Feb 19 22:41 sdc
brw-rw----. 1 root disk 8, 80 Feb 19 22:41 sdf
brw-rw----. 1 root disk 8, 112 Feb 19 22:41 sdh
brw-rw----. 1 root disk 8, 64 Feb 19 22:41 sde
brw-rw----. 1 root disk 8, 128 Feb 19 22:41 sdi
brw-rw----. 1 root disk 8, 144 Feb 19 22:41 sdj
brw-rw----. 1 root disk 8, 48 Feb 19 22:41 sdd
brw-rw----. 1 root disk 8, 17 Feb 19 22:41 sdb1

Format shared storage as below steps for each whole disk:

=============================================================
Command (m for help): n ==> type n for new partition
Select (default p): p ==> type p for primary partition
Partition number (1-4, default 1): 1 ==> partition number type 1
First sector (2048-41943039, default 2048): ==> Press blank Enter to use default first sector which is
2048

Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): ==> Press blank Enter to use
entire disk

Command (m for help): p ==> Type p to print details of partition

Command (m for help): w ===> type w to save the partition

=============================================================

fdisk /dev/sdc (To Format)

fdisk /dev/sdc -l (To verify formatted Partition)

fdisk /dev/sdd
fdisk /dev/sdd -l

fdisk /dev/sde
fdisk /dev/sde -l

fdisk /dev/sdf
fdisk /dev/sdf -l

fdisk /dev/sdg
fdisk /dev/sdg -l

fdisk /dev/sdh
fdisk /dev/sdh -l

fdisk /dev/sdi
fdisk /dev/sdi -l

fdisk /dev/sdj
fdisk /dev/sdj -l

[root@raclnx1 disks]# lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 15G 0 disk
ââsda1 8:1 0 500M 0 part /boot
ââsda2 8:2 0 14.5G 0 part
âârhel-root 253:0 0 13G 0 lvm /
âârhel-swap 253:1 0 1.5G 0 lvm [SWAP]
sdb 8:16 0 28G 0 disk
ââsdb1 8:17 0 28G 0 part /u01
sdc 8:32 0 12G 0 disk
ââsdc1 8:33 0 12G 0 part
sdd 8:48 0 5G 0 disk
ââsdd1 8:49 0 5G 0 part
sde 8:64 0 5G 0 disk
ââsde1 8:65 0 5G 0 part
sdf 8:80 0 5G 0 disk
ââsdf1 8:81 0 5G 0 part
sdg 8:96 0 4G 0 disk
ââsdg1 8:97 0 4G 0 part
sdh 8:112 0 4G 0 disk
ââsdh1 8:113 0 4G 0 part
sr0 11:0 1 1024M 0 rom

Verify new partitions or luns

=========================================
[root@raclnx1 dev]# cd /dev
[root@raclnx1 dev]# ls -ltr sd*
brw-rw----. 1 root disk 8, 16 Feb 19 22:41 sdb
brw-rw----. 1 root disk 8, 0 Feb 19 22:41 sda
brw-rw----. 1 root disk 8, 2 Feb 19 22:41 sda2
brw-rw----. 1 root disk 8, 1 Feb 19 22:41 sda1
brw-rw----. 1 root disk 8, 17 Feb 19 22:41 sdb1
brw-rw----. 1 root disk 8, 32 Feb 19 22:59 sdc
brw-rw----. 1 root disk 8, 33 Feb 19 22:59 sdc1
brw-rw----. 1 root disk 8, 48 Feb 19 23:03 sdd
brw-rw----. 1 root disk 8, 49 Feb 19 23:03 sdd1
brw-rw----. 1 root disk 8, 64 Feb 19 23:04 sde
brw-rw----. 1 root disk 8, 65 Feb 19 23:04 sde1
brw-rw----. 1 root disk 8, 80 Feb 19 23:04 sdf
brw-rw----. 1 root disk 8, 81 Feb 19 23:04 sdf1
brw-rw----. 1 root disk 8, 96 Feb 19 23:06 sdg
brw-rw----. 1 root disk 8, 97 Feb 19 23:06 sdg1
brw-rw----. 1 root disk 8, 112 Feb 19 23:06 sdh
brw-rw----. 1 root disk 8, 113 Feb 19 23:06 sdh1
brw-rw----. 1 root disk 8, 128 Feb 19 23:07 sdi
brw-rw----. 1 root disk 8, 129 Feb 19 23:07 sdi1
brw-rw----. 1 root disk 8, 144 Feb 19 23:08 sdj
brw-rw----. 1 root disk 8, 145 Feb 19 23:08 sdj1

Check or verify the size of each Disk/Luns

=============================================

[root@raclnx1 ~]# ls -ltr /dev/sd?[1]*

brw-rw----. 1 root disk 8, 1 Feb 21 15:47 /dev/sda1
brw-rw----. 1 root disk 8, 17 Feb 21 15:47 /dev/sdb1
brw-rw----. 1 root disk 8, 81 Feb 21 15:47 /dev/sdf1
brw-rw----. 1 root disk 8, 33 Feb 21 15:47 /dev/sdc1
brw-rw----. 1 root disk 8, 113 Feb 21 15:47 /dev/sdh1
brw-rw----. 1 root disk 8, 145 Feb 21 15:47 /dev/sdj1
brw-rw----. 1 root disk 8, 129 Feb 21 15:47 /dev/sdi1
brw-rw----. 1 root disk 8, 65 Feb 21 15:47 /dev/sde1
brw-rw----. 1 root disk 8, 97 Feb 21 15:47 /dev/sdg1
brw-rw----. 1 root disk 8, 49 Feb 21 15:47 /dev/sdd1

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdc1

1072693248 =========> 1GB

Note:
-----------
Refreshed the kernel on the remaining nodes with the latest partition table using partprobe.

[root@node2 ~]# partprobe

************End of Section 4 => Shared Disk Allocation ******************************

Section 5 : ASM Lib Configuration all Nodes

*************************************************************************************
As Root
----------
/usr/sbin/oracleasm configure -i

[root@raclnx1 ~]# /usr/sbin/oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle

Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

Enter the following command to load the oracleasm kernel module: (All the RAC nodes):
--------------------------------------------------------------------------------------------
/usr/sbin/oracleasm init

[root@raclnx1 ~]# /usr/sbin/oracleasm init

Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Verify asm is loaded

--------------------
[root@raclnx1 ~]# lsmod | grep -i oracleasm
oracleasm 63318 1

[root@raclnx1 ~]# /usr/sbin/oracleasm status

Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes

[root@raclnx1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel-root 13G 5.3G 7.8G 41% /
devtmpfs 1.2G 0 1.2G 0% /dev
tmpfs 1.2G 84K 1.2G 1% /dev/shm
tmpfs 1.2G 8.8M 1.2G 1% /run
tmpfs 1.2G 0 1.2G 0% /sys/fs/cgroup
/dev/sdb1 28G 4.4G 22G 17% /u01
/dev/sda1 497M 213M 285M 43% /boot
tmpfs 229M 16K 229M 1% /run/user/42
tmpfs 229M 0 229M 0% /run/user/601
[root@raclnx1 ~]#

Enable ASM services

-------------------
systemctl enable oracleasm.service

To restart ASMLib
----------------------------------
/etc/init.d/oracleasm restart
/usr/sbin/oracleasm status

OR
[root@raclnx2 ~]# /etc/init.d/oracleasm stop
Stopping oracleasm (via systemctl): [ OK ]

[root@raclnx2 ~]# /usr/sbin/oracleasm status

Checking if ASM is loaded: no
Checking if /dev/oracleasm is mounted: no

[root@raclnx2 ~]# /etc/init.d/oracleasm start

Starting oracleasm (via systemctl): [ OK ]

[root@raclnx2 ~]# /usr/sbin/oracleasm status

Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
Troubleshooting command
------------------------
modinfo oracleasm
[root@raclnx2 ~]# modinfo oracleasm
filename: /lib/modules/3.10.0-514.el7.x86_64/extra/oracleasm/oracleasm.ko
description: Kernel driver backing the Generic Linux ASM Library.
author: Joel Becker <[email protected]>
version: 2.0.8
license: GPL
rhelversion: 7.3
srcversion: 3AA2BF05E1F8A2D0EEAFD4E
depends:
vermagic: 3.10.0-514.el7.x86_64 SMP mod_unload modversions
signer: Oracle Linux RHCK Module Signing Key
sig_key: DD:99:5B:15:5C:19:B3:A7:C3:EF:77:07:B9:69:E2:5F:96:39:66:6E
sig_hashalgo: sha256
parm: use_logical_block_size:Prefer logical block size over physical (Y=logical, N=physical [default])
(bool)

************************End of Section 5 => ASM Lib Configuration all Nodes

Section 6 : Create ASM Disk using asmlib Only One Node

******************************************************************************

cd /dev/oracleasm/disks

[root@raclnx1 ~]# lsscsi

[0:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda
[0:0:1:0] disk ATA VBOX HARDDISK 1.0 /dev/sdb
[1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0
[2:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdc
[3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdd
[4:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sde
[5:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdf
[6:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdg
[7:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdh
[8:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdi
[9:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdj

cd /dev/oracleasm/disks
[root@raclnx1 disks]# ls -ltr /dev/sd?[1]*
brw-rw----. 1 root disk 8, 1 Feb 21 15:47 /dev/sda1
brw-rw----. 1 root disk 8, 17 Feb 21 15:47 /dev/sdb1
brw-rw----. 1 root disk 8, 81 Feb 21 15:47 /dev/sdf1
brw-rw----. 1 root disk 8, 113 Feb 21 15:47 /dev/sdh1
brw-rw----. 1 root disk 8, 145 Feb 21 15:47 /dev/sdj1
brw-rw----. 1 root disk 8, 129 Feb 21 15:47 /dev/sdi1
brw-rw----. 1 root disk 8, 65 Feb 21 15:47 /dev/sde1
brw-rw----. 1 root disk 8, 97 Feb 21 15:47 /dev/sdg1
brw-rw----. 1 root disk 8, 49 Feb 21 15:47 /dev/sdd1
brw-rw----. 1 root disk 8, 33 Feb 21 20:57 /dev/sdc1

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdc1

1072693248 ==> 1GB

ls -ltrh
/usr/sbin/oracleasm createdisk OCRVOTEDISK1 /dev/sdc1
output
-------
[root@raclnx1 disks]# /usr/sbin/oracleasm createdisk OCRVOTEDISK1 /dev/sdc1
Writing disk header: done
Instantiating disk: done

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdd1

1072693248 ==> 1GB

/usr/sbin/oracleasm createdisk OCRVOTEDISK2 /dev/sdd1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sde1

1072693248 ==> 1GB

/usr/sbin/oracleasm createdisk OCRVOTEDISK3 /dev/sde1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdf1

5367660544 ==> 5GB

/usr/sbin/oracleasm createdisk DATADISK1 /dev/sdf1

/usr/sbin/oracleasm listdisks
ls -ltrh
[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdg1
5367660544 ==> 5GB

/usr/sbin/oracleasm createdisk DATADISK2 /dev/sdg1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdh1

5367660544 ==> 5GB

/usr/sbin/oracleasm createdisk DATADISK3 /dev/sdh1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdi1

4293918720 ==> 4GB

/usr/sbin/oracleasm createdisk FRADISK1 /dev/sdi1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 ~]# /sbin/blockdev --getsize64 /dev/sdj1

4293918720 ==> 4GB

/usr/sbin/oracleasm createdisk FRADISK2 /dev/sdj1

/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 disks]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@raclnx1 disks]# /usr/sbin/oracleasm listdisks

DATADISK1
DATADISK2
DATADISK3
FRADISK1
FRADISK2
OCRVOTEDISK1
OCRVOTEDISK2
OCRVOTEDISK3

[root@raclnx1 disks]# ls -ltrh

total 0
brw-rw----. 1 oracle asmadmin 8, 33 Feb 21 20:57 OCRVOTEDISK1
brw-rw----. 1 oracle asmadmin 8, 49 Feb 21 21:03 OCRVOTEDISK2
brw-rw----. 1 oracle asmadmin 8, 65 Feb 21 21:04 OCRVOTEDISK3
brw-rw----. 1 oracle asmadmin 8, 81 Feb 21 21:07 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Feb 21 21:08 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Feb 21 21:09 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Feb 21 21:10 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Feb 21 21:11 FRADISK2

Delete Disk to create higher size votedisk

------------------------------------------------
cd /dev/oracleasm/disks

[root@raclnx1 disks]# ls -ltrh

total 0
brw-rw----. 1 oracle asmadmin 8, 33 Mar 8 12:41 OCRVOTEDISK1
brw-rw----. 1 oracle asmadmin 8, 49 Mar 8 12:41 OCRVOTEDISK2
brw-rw----. 1 oracle asmadmin 8, 65 Mar 8 12:41 OCRVOTEDISK3
brw-rw----. 1 oracle asmadmin 8, 81 Mar 8 12:41 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Mar 8 12:41 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Mar 8 12:41 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Mar 8 12:41 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Mar 8 12:41 FRADISK2

/usr/sbin/oracleasm deletedisk OCRVOTEDISK1

/usr/sbin/oracleasm deletedisk OCRVOTEDISK2
/usr/sbin/oracleasm deletedisk OCRVOTEDISK3

[root@raclnx1 disks]# /usr/sbin/oracleasm deletedisk OCRVOTEDISK1

Clearing disk header: done
Dropping disk: done

[root@raclnx1 disks]# /usr/sbin/oracleasm deletedisk OCRVOTEDISK2

Clearing disk header: done
Dropping disk: done
[root@raclnx1 disks]# /usr/sbin/oracleasm deletedisk OCRVOTEDISK3
Clearing disk header: done
Dropping disk: done

[root@raclnx1 disks]# /usr/sbin/oracleasm listdisks

DATADISK1
DATADISK2
DATADISK3
FRADISK1
FRADISK2
[root@raclnx1 disks]# ls -ltrh
total 0
brw-rw----. 1 oracle asmadmin 8, 81 Mar 8 12:41 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Mar 8 12:41 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Mar 8 12:41 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Mar 8 12:41 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Mar 8 12:41 FRADISK2

Node 2: SCAN them

---------------------
[root@raclnx2 ~]# cd /dev/oracleasm/disks
[root@raclnx2 disks]# ls -ltrh
total 0
brw-rw----. 1 oracle asmadmin 8, 33 Mar 8 12:41 OCRVOTEDISK1
brw-rw----. 1 oracle asmadmin 8, 49 Mar 8 12:41 OCRVOTEDISK2
brw-rw----. 1 oracle asmadmin 8, 65 Mar 8 12:41 OCRVOTEDISK3
brw-rw----. 1 oracle asmadmin 8, 81 Mar 8 12:41 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Mar 8 12:41 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Mar 8 12:41 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Mar 8 12:41 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Mar 8 12:41 FRADISK2

[root@raclnx2 disks]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done
Cleaning any stale ASM disks...
Cleaning disk "OCRVOTEDISK1"
Cleaning disk "OCRVOTEDISK2"
Cleaning disk "OCRVOTEDISK3"
Scanning system for ASM disks...

[root@raclnx2 disks]# /usr/sbin/oracleasm listdisks

DATADISK1
DATADISK2
DATADISK3
FRADISK1
FRADISK2

[root@raclnx2 disks]# ls -ltrh

total 0
brw-rw----. 1 oracle asmadmin 8, 81 Mar 8 12:41 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Mar 8 12:41 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Mar 8 12:41 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Mar 8 12:41 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Mar 8 12:41 FRADISK2
[root@raclnx2 disks]#

[root@raclnx1 ~]# fdisk -l /dev/sdc

Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x67b05f75

Device Boot Start End Blocks Id System

/dev/sdc1 2048 2097151 1047552 83 Linux

--------------------------------------------------------------------------------------
Delete the partition Now
--------------------------------------------------------------------------------------
fdisk /dev/sdc

[root@raclnx1 ~]# fdisk /dev/sdc

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p

Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x67b05f75

Device Boot Start End Blocks Id System

/dev/sdc1 2048 2097151 1047552 83 Linux

Command (m for help): d

Selected partition 1
Partition 1 is deleted

Command (m for help): p

Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x67b05f75

Device Boot Start End Blocks Id System

Command (m for help):

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@raclnx1 ~]#

fdisk -l /dev/sdd
fdisk /dev/sdd

fdisk -l /dev/sde
fdisk /dev/sde

[root@raclnx1 ~]# lsscsi

[0:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sda
[0:0:1:0] disk ATA VBOX HARDDISK 1.0 /dev/sdb
[1:0:0:0] cd/dvd VBOX CD-ROM 1.0 /dev/sr0
[2:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdc
[3:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdd
[4:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sde
[5:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdf
[6:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdg
[7:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdh
[8:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdi
[9:0:0:0] disk ATA VBOX HARDDISK 1.0 /dev/sdj
[root@raclnx1 ~]#

[root@raclnx1 ~]# fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes, 25165824 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@raclnx1 disks]# /sbin/blkid

/dev/sda1: UUID="a46a3693-4de6-441b-a60c-36245e8e0ea4" TYPE="xfs"
/dev/sda2: UUID="0MmCwZ-LMID-R6hc-LZ14-eHlj-HmjJ-1PJbaz" TYPE="LVM2_member"
/dev/sdb1: UUID="9b2a07f0-0c43-464e-aff4-6fa909033862" TYPE="ext3"
/dev/sdd1: LABEL="DATADISK1" TYPE="oracleasm"
/dev/sdf1: LABEL="DATADISK3" TYPE="oracleasm"
/dev/sde1: LABEL="DATADISK2" TYPE="oracleasm"
/dev/sdg1: LABEL="FRADISK1" TYPE="oracleasm"
/dev/sdh1: LABEL="FRADISK2" TYPE="oracleasm"
/dev/mapper/rhel-root: UUID="089a8e19-a1a0-4add-a02c-b7f0e994471f" TYPE="xfs"
/dev/mapper/rhel-swap: UUID="92c02718-0e62-40a4-8f42-5500760986d7" TYPE="swap"

create the partition for new OCRVOTE

--------------------------------------
fdisk /dev/sdc
[root@raclnx1 disks]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table

Building a new DOS disklabel with disk identifier 0x30504e94.

Command (m for help): n

Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-25165823, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-25165823, default 25165823):
Using default value 25165823
Partition 1 of type Linux and of size 12 GiB is set

Command (m for help): p

Disk /dev/sdc: 12.9 GB, 12884901888 bytes, 25165824 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x30504e94

Device Boot Start End Blocks Id System

/dev/sdc1 2048 25165823 12581888 83 Linux

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@raclnx1 disks]# fdisk -l /dev/sdc

Disk /dev/sdc: 12.9 GB, 12884901888 bytes, 25165824 sectors

Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x30504e94

Device Boot Start End Blocks Id System

/dev/sdc1 2048 25165823 12581888 83 Linux
[root@raclnx1 disks]#

On second Node
----------------
[root@raclnx2 disks]# ls -ltr /dev/sd?[1]*
brw-rw----. 1 root disk 8, 1 Mar 8 13:15 /dev/sda1
brw-rw----. 1 root disk 8, 17 Mar 8 13:15 /dev/sdb1
brw-rw----. 1 root disk 8, 49 Mar 8 13:16 /dev/sdd1
brw-rw----. 1 root disk 8, 65 Mar 8 13:16 /dev/sde1
brw-rw----. 1 root disk 8, 81 Mar 8 13:16 /dev/sdf1
brw-rw----. 1 root disk 8, 113 Mar 8 13:16 /dev/sdh1
brw-rw----. 1 root disk 8, 97 Mar 8 13:16 /dev/sdg1

[root@raclnx2 disks]# partprobe

[root@raclnx2 disks]#
[root@raclnx2 disks]# ls -ltr /dev/sd?[1]*
brw-rw----. 1 root disk 8, 1 Mar 8 13:23 /dev/sda1
brw-rw----. 1 root disk 8, 17 Mar 8 13:23 /dev/sdb1
brw-rw----. 1 root disk 8, 33 Mar 8 13:23 /dev/sdc1
brw-rw----. 1 root disk 8, 49 Mar 8 13:23 /dev/sdd1
brw-rw----. 1 root disk 8, 65 Mar 8 13:23 /dev/sde1
brw-rw----. 1 root disk 8, 81 Mar 8 13:23 /dev/sdf1
brw-rw----. 1 root disk 8, 97 Mar 8 13:23 /dev/sdg1
brw-rw----. 1 root disk 8, 113 Mar 8 13:23 /dev/sdh1

NOW Create ASM Disk on Node 1

====================================
/usr/sbin/oracleasm createdisk OCRVOTEDISK01 /dev/sdc1
/usr/sbin/oracleasm listdisks
ls -ltrh

[root@raclnx1 disks]# /sbin/blockdev --getsize64 /dev/sdc1

12883853312 ===> 11.99 ~ 12GB

Size of the Disk

---------------------
[root@raclnx1 disks]# /usr/sbin/oracleasm-discover 'ORCL:OCRVOTEDISK01'
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]
Discovered disk: ORCL:OCRVOTEDISK01 [25163776 blocks (12883853312 bytes), maxio 512, integrity
none]

[root@raclnx1 disks]# /usr/sbin/oracleasm querydisk -d -p OCRVOTEDISK01

Disk "OCRVOTEDISK01" is a valid ASM disk on device [8,33]
/dev/sdc1: LABEL="OCRVOTEDISK01" TYPE="oracleasm"
[root@raclnx1 disks]# /usr/sbin/oracleasm querydisk -d /dev/sdc1
Device "/dev/sdc1" is marked an ASM disk with the label "OCRVOTEDISK01"

[root@raclnx1 disks]# /sbin/blkid /dev/sdc1

/dev/sdc1: LABEL="OCRVOTEDISK01" TYPE="oracleasm"

on second Node
================
[root@raclnx2 disks]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCRVOTEDISK01"

************************************************************************************
On the Remaining RAC nodes(NODE2), simply scanned the ASM disks to instantiate these newly created
disks
************************************************************************************
cd /dev/oracleasm/disks
ls -ltrh
/usr/sbin/oracleasm scandisks
/usr/sbin/oracleasm listdisks
ls -ltrh

Output
---------
[root@raclnx2 ~]# cd /dev/oracleasm/disks
[root@raclnx2 disks]# ls -ltr
total 0

[root@raclnx2 disks]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "OCRVOTEDISK1"
Instantiating disk "OCRVOTEDISK2"
Instantiating disk "OCRVOTEDISK3"
Instantiating disk "DATADISK1"
Instantiating disk "DATADISK2"
Instantiating disk "DATADISK3"
Instantiating disk "FRADISK1"
Instantiating disk "FRADISK2"

[root@raclnx2 disks]# /usr/sbin/oracleasm listdisks

DATADISK1
DATADISK2
DATADISK3
FRADISK1
FRADISK2
OCRVOTEDISK1
OCRVOTEDISK2
OCRVOTEDISK3

[root@raclnx2 disks]# ls -ltr

total 0
brw-rw----. 1 oracle asmadmin 8, 33 Feb 20 16:18 OCRVOTEDISK1
brw-rw----. 1 oracle asmadmin 8, 49 Feb 20 16:18 OCRVOTEDISK2
brw-rw----. 1 oracle asmadmin 8, 65 Feb 20 16:18 OCRVOTEDISK3
brw-rw----. 1 oracle asmadmin 8, 81 Feb 20 16:18 DATADISK1
brw-rw----. 1 oracle asmadmin 8, 97 Feb 20 16:18 DATADISK2
brw-rw----. 1 oracle asmadmin 8, 113 Feb 20 16:18 DATADISK3
brw-rw----. 1 oracle asmadmin 8, 129 Feb 20 16:18 FRADISK1
brw-rw----. 1 oracle asmadmin 8, 145 Feb 20 16:18 FRADISK2

Verified that these ASM disk can be discovered by ASM Libraries (oracleasmlib) on all the RAC Nodes
during Installation of Grid Infrastructure.
------------------------------------------------------------------------------------------------------------------------------------------
/usr/sbin/oracleasm-discover 'ORCL:*'

Output
---------
[root@raclnx1 disks]# /usr/sbin/oracleasm-discover 'ORCL:*'
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]
Discovered disk: ORCL:DATADISK1 [10483712 blocks (5367660544 bytes), maxio 512, integrity none]
Discovered disk: ORCL:DATADISK2 [10483712 blocks (5367660544 bytes), maxio 512, integrity none]
Discovered disk: ORCL:DATADISK3 [10483712 blocks (5367660544 bytes), maxio 512, integrity none]
Discovered disk: ORCL:FRADISK1 [8386560 blocks (4293918720 bytes), maxio 512, integrity none]
Discovered disk: ORCL:FRADISK2 [8386560 blocks (4293918720 bytes), maxio 512, integrity none]
Discovered disk: ORCL:OCRVOTEDISK1 [2095104 blocks (1072693248 bytes), maxio 512, integrity none]
Discovered disk: ORCL:OCRVOTEDISK2 [2095104 blocks (1072693248 bytes), maxio 512, integrity none]
Discovered disk: ORCL:OCRVOTEDISK3 [2095104 blocks (1072693248 bytes), maxio 512, integrity none]

ls -ltrh network/oracleasm

how to identify the configured ASM disk to physical Disk name as tricky step use the blkid command.
---------------------------------------------------------------------------------------------------
/sbin/blkid

OutPut
--------
[root@raclnx1 disks]# /sbin/blkid
/dev/sda1: UUID="a46a3693-4de6-441b-a60c-36245e8e0ea4" TYPE="xfs"
/dev/sda2: UUID="0MmCwZ-LMID-R6hc-LZ14-eHlj-HmjJ-1PJbaz" TYPE="LVM2_member"
/dev/sdb1: UUID="9b2a07f0-0c43-464e-aff4-6fa909033862" TYPE="ext3"
/dev/sdc1: LABEL="OCRVOTEDISK1" TYPE="oracleasm"
/dev/sdd1: LABEL="OCRVOTEDISK2" TYPE="oracleasm"
/dev/sde1: LABEL="OCRVOTEDISK3" TYPE="oracleasm"
/dev/sdf1: LABEL="DATADISK1" TYPE="oracleasm"
/dev/sdg1: LABEL="DATADISK2" TYPE="oracleasm"
/dev/sdh1: LABEL="DATADISK3" TYPE="oracleasm"
/dev/sdi1: LABEL="FRADISK1" TYPE="oracleasm"
/dev/sdj1: LABEL="FRADISK2" TYPE="oracleasm"
/dev/mapper/rhel-root: UUID="089a8e19-a1a0-4add-a02c-b7f0e994471f" TYPE="xfs"
/dev/mapper/rhel-swap: UUID="92c02718-0e62-40a4-8f42-5500760986d7" TYPE="swap"

/usr/sbin/oracleasm querydisk -d -p OCRVOTEDISK1

/usr/sbin/oracleasm querydisk -d /dev/sdc1

Output
---------
[root@raclnx1 disks]# /usr/sbin/oracleasm querydisk -d -p OCRVOTEDISK1
Disk "OCRVOTEDISK1" is a valid ASM disk on device [8,33]
/dev/sdc1: LABEL="OCRVOTEDISK1" TYPE="oracleasm"

[root@raclnx1 disks]# /usr/sbin/oracleasm querydisk -d /dev/sdc1

Device "/dev/sdc1" is marked an ASM disk with the label "OCRVOTEDISK1"

*********************** End of Section 6 **********************************************

# mkdir -p /u01/app/11.2.0.4/grid
# chown -R oracle:oinstall /u01
# mkdir -p /u01/app/oracle
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/