Exadata Database Machine

Initial Configuration

Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Objectives

After completing this lesson, you should be able to describe:

• Installation and configuration process for Database
Machine
• Default configuration for Database Machine
• Supported and unsupported customizations for Database
Machine

5-2 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Database Machine Implementation: Overview

Four phases:
1. Pre-installation
– Various planning and scheduling activities including:
— Site planning: space, power, cooling, logistics
— Configuration planning: host names, IP addresses, databases
— Network preparation: DNS, NTP, cabling
– Oracle and customer engineers can work together
2. Installation and configuration
– Hardware and software installation and configuration
– Result is a working system configured by using the desired
configuration settings
– Recommended to be performed by Oracle engineers

5-3 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Database Machine Implementation: Overview

3. Additional configuration
– Additional activities for production readiness including:
— Reconfigure storage using non-default settings.
— Create additional databases.
— Configure Enterprise Manager.
— Configure backup and recovery.
— Connect Oracle Exalogic Elastic Cloud.
– Conducted by customer or Oracle services engagement
4. Post-installation
– Ongoing monitoring and maintenance
– Conducted by customer or Oracle services engagement

5-4 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Key Documentation

Oracle Exadata Database Machine Owner’s Guide

• Important reference document covering:
– Site planning
– Network planning
– Initial configuration
– Core maintenance

5-5 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Database Machine Site Preparation

• Use the following sections in the Owner’s Guide to direct

site preparation activities:
– Chapter 2: Site Requirements for Oracle Exadata Database
Machine
— General Environmental Requirements
— Space Requirements
— Flooring Requirements
— Electrical Power Requirements
— Temperature and Humidity Requirements
— Ventilation and Cooling Requirements
— Network Connection and IP Address Requirements
– Appendix B: Site Checklists

5-6 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Oracle Exadata Deployment Assistant: Overview

Oracle Exadata Deployment Assistant :

• Captures site-specific configuration settings including:
– Host and domain names
– IP addresses
– Region and time zone information
– Name servers and NTP time servers
– Exadata cell notification settings
• Generates files that drive the OneCommand configuration
programs
• Is a Java-based program that is part of the OneCommand
patch bundle
– My Oracle Support note 888828.1 lists the latest version

5-7 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Oracle Exadata Deployment Assistant:
Customer Details

5-8 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Oracle Exadata Deployment Assistant:
Hardware Selection

5-9 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
 Oracle Exadata Deployment Assistant:
Networking

5 - 10 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Administration Network

5 - 11 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Administration Network
IP Address Allocation: Example
Interface Type Hostname IP Address
Database server management interface eidmdbadm01 10.7.7.101
Database server management interface eidmdbadm02 10.7.7.102
Exadata cell management interface eidmceladm01 10.7.7.103
Exadata cell management interface eidmceladm02 10.7.7.104
Exadata cell management interface eidmceladm03 10.7.7.105
Database server ILOM interface eidmdbadm01-ilom 10.7.7.106
Database server ILOM interface eidmdbadm02-ilom 10.7.7.107
Exadata cell ILOM interface eidmceladm01-ilom 10.7.7.108
Exadata cell ILOM interface eidmceladm02-ilom 10.7.7.109
Exadata cell ILOM interface eidmceladm03-ilom 10.7.7.110
Ethernet switch management interface eidmsw-adm0 10.7.7.111
InfiniBand switch management interface eidmsw-iba0 10.7.7.112
InfiniBand switch management interface eidmsw-ibb0 10.7.7.113
PDU management interface eidm-pdua0 10.7.7.114
PDU management interface eidm-pdub0 10.7.7.115

5 - 12 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Client Ethernet Network

5 - 13 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Client Ethernet Network
IP Address Allocation: Example

Address Type Hostname IP Address

Database server physical client access interface eidmclient01 172.16.1.101
Database server physical client access interface eidmclient02 172.16.1.102
Database server VIP eidmclient01-vip 172.16.1.103
Database server VIP eidmclient02-vip 172.16.1.104
SCAN address eidm-scan 172.16.1.105
SCAN address eidm-scan 172.16.1.106
SCAN address eidm-scan 172.16.1.107

5 - 14 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
InfiniBand Network

5 - 15 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 InfiniBand Network IP
Address Allocation: Example

Address Type Hostname IP Address

Database server InfiniBand network interface eidmdb01-priv 192.168.10.1
Database server InfiniBand network interface eidmdb02-priv 192.168.10.2
Storage server InfiniBand network interface eidmcel01-priv 192.168.10.3
Storage server InfiniBand network interface eidmcel02-priv 192.168.10.4
Storage server InfiniBand network interface eidmcel03-priv 192.168.10.5

5 - 16 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Backup / Data Guard Network

5 - 17 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
OS Configuration

5 - 18 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Home and Database

5 - 19 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Choosing the Right
Disk Group Redundancy Setting
• HIGH redundancy:
– Triple mirroring across three separate storage servers
– Provides maximum protection
— Double mirroring still maintained if one cell is offline
– Requires more storage capacity
– Writes require greater I/O bandwidth.
• NORMAL redundancy:
– Double mirroring across two separate storage servers
– Provides one layer of redundancy
— No additional protection if one cell is offline
— Requires extra time and effort to maintain redundancy through planned
maintenance
• Free space management:
– Free space is required to preserve redundancy.
– Consider the impact of losing a disk and the impact of losing a cell.
– Consider also the desired protection level and the number of available cells.

5 - 21 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Cell Alerting

5 - 23 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Oracle Configuration Manager

5 - 24 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Auto Service Request

5 - 25 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Grid Control Agent

5 - 26 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Review and Edit Details

5 - 27 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Oracle Exadata Deployment Assistant:
Create Files

5 - 28 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Database Machine
Hardware Installation: Overview
Refer to the Owner’s Guide for details on the following
hardware installation tasks:
• Reviewing Safety Guidelines
• Unpacking Oracle Exadata Database Machine
• Placing Oracle Exadata Database Machine in Its Allocated
Space
– Moving Oracle Exadata Database Machine
– Stabilizing Oracle Exadata Database Machine
– Attaching a Ground Cable
• Powering On the System the First Time
– Inspecting the Machine After It Is in Place
– Connecting Power Cords
– Powering On Oracle Exadata Database Machine

5 - 29 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Configuring Oracle Exadata Database
Machine: Overview
• Connect to the Network.
• Configure the KVM Switch (if installed).
• Configure Sun Datacenter InfiniBand Switch 36 Switch.
• Configure the Cisco Ethernet Switch.
• Configure the Power Distribution Units.
• Check Exadata Storage Servers.
• Check Oracle Database Servers.
• Perform Additional Checks and Configuration.
• Verify the InfiniBand Network.
• Select the Operating System.
• Reclaim Disk Space After Selecting the Operating System.
• Perform Initial Network Configuration.
• Load the Configuration Information and Install the Software.
• Install Oracle Enterprise Manager.

5 - 30 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Selecting the Database Server Operating System

• Customers now have a choice for the database server

operating system:
– Oracle Linux X86_64 (default)
– Oracle Solaris for x86
• Servers are shipped from the factory with both operating
systems preloaded
• Space consumed by the unused operating system is
reclaimed and reused

5 - 31 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Deploying Solaris on the Database Servers

As the root user on each database server:

# cd /opt/oracle.SupportTools
# ./defaultOSchoose.pl

Default OS is : LINUX_BOOT_0
Please choose new default OS:
[0] LINUX_BOOT_0
[1] SOLARIS_BOOT_1
[2] SOLARIS_BOOT_2
Please type the number you would like to make a new
default OS: 1

5 - 32 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Reclaiming Unused Operating
System Disk Space Using Linux
As the root user on each database server:
# cd /opt/SupportTools
# ./reclaimdisks.sh -check

<check for expected output>

# ./reclaimdisks.sh –free –reclaim

<monitor /var/log/cellos/reclaimdisks.bg.log>

# ./reclaimdisks.sh -check

<check for expected output>

5 - 33 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Reclaiming Unused Operating
System Disk Space Using Solaris
As the root user on each database server:
# cd /opt/SupportTools
# ./reclaimdisks.pl

This script changes the Linux virtual and physical disks

to Solaris disks.

<Output truncated>

The virtual disks will be deleted. Do you want to

proceed? (yes/no) yes
This is an irreversible operation. Confirm that you want
to proceed. (yes/no) yes

Deleting the virtual disks ...

<Output continues>

5 - 35 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Performing Initial Network Configuration

IP addresses and host names are applied to the Database

Machine servers:
# cd /opt/oracle.SupportTools/firstconf
# ./applyconfig.sh -preconf \
/opt/oracle.SupportTools/onecommand/preconf.csv \
-dbm-node-ips <size>

IP addresses,
host names

preconf.csv
applyconfig.sh

5 - 36 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Loading the Configuration Information and
Installing the Software
Steps performed by OneCommand:
• Step 0: Validate the environment. • Step 15: Patch the grid home software.
• Step 1: Create work directory. • Step 16: Relink Reliable Data Socket
• Step 2: Unzip files. (RDS) for Grid Infrastructure.
• Step 3: Setup SSH for the root user. • Step 17: Run the grid root scripts.
• Step 4: Update the /etc/hosts file. • Step 18: Install the Oracle Database
• Step 5: Create the cellip.ora and software.
cellinit.ora files • Step 19: Patch the database software.
• Step 6: Validate the InfiniBand network. • Step 20: Create the Oracle ASM disk
• Step 7: Update the cell software. groups.
• Step 8: Validate the cells. • Step 21: Run Oracle Database
Configuration Assistant.
• Step 9: Check RDS using the ping
command. • Step 22: Unlock Oracle Grid Infrastructure.
• Step 10: Run CALLIBRATE on the cells. • Step 23: Relink RDS for Oracle Database.
• Step 11: Create the user accounts for • Step 24: Lock Oracle Grid Infrastructure.
celladmin and cellmonitor. • Step 25: Apply any security fixes.
• Step 12: Set up SSH for the user accounts. • Step 26: Configure ASR.
• Step 13: Create the grid disks. • Step 27: Set up storage server alerts.
• Step 14: Install the grid software. • Step 28: Secure Oracle Exadata
Database Machine.

5 - 38 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Running OneCommand on Database Machine

1. Log in as the root user on the first database server. The default
password is welcome1.
– Configuration files should already be located at
/opt/oracle.SupportTools/onecommand
2. Download all necessary Oracle Exadata Storage Server Software and
Oracle Database patches.
– See My Oracle Support note 888828.1 for details.
3. Apply updates for the OneCommand utility.
– See associated README file for instructions.
4. Change to the OneCommand directory:
# cd /opt/oracle.SupportTools/onecommand

5. Run the OneCommand utility using the following command:

# ./deploy112.sh –i [ -s <n> | -r <n>-<N> | -l ]

5 - 39 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Exadata Storage Configuration

System Area (Disks 0-1)

or DBFS_DG (Disks 2-11) Exadata Cell 1
(29G)
Exadata Cell 2
Exadata Cell n
DATA_EIDM
(40%)

RECO_EIDM
(60%)

5 - 40 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Result After Installation and Configuration
eidmclient01 172.16.1.101 eidmclient02 172.16.1.102

eidmclient01-vip 172.16.1.103 eidmclient02-vip 172.16.1.104 eidm-scan 172.16.1.105

eidmdbadm01 eidmdbadm01-ilom eidmdbadm02 eidmdbadm02-ilom 172.16.1.106
10.7.7.101 10.7.7.106 10.7.7.102 10.7.7.107 172.16.1.107

DB Server 1 DB Server 2

DB Instance: dbm1 DB Instance: dbm2

ASM Instance: +ASM1 ASM Instance: +ASM2
Cluster: eidm Cluster: eidm

eidmdb01-priv eidmdb02-priv
192.168.10.1 192.168.10.2
InfiniBand Storage Network
eidmcel01-priv eidmcel02-priv eidmcel03-priv
192.168.10.3 192.168.10.4 192.168.10.5

DBFS_DG DBFS_DG DBFS_DG

DATA_EIDM DATA_EIDM DATA_EIDM

RECO_EIDM RECO_EIDM RECO_EIDM

Exadata Server 1 Exadata Server 2 Exadata Server 3

eidmceladm01 eidmceladm01-ilom eidmceladm02 eidmceladm02-ilom eidmceladm03 eidmceladm03-ilom
10.7.7.103 10.7.7.108 10.7.7.104 10.7.7.109 10.7.7.105 10.7.7.110

5 - 42 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Supported Additional Configuration Activities

• Earthquake protection using a third-party Seismic Isolation

Platform

• Replace the Ethernet switch.

• Connect a tape library for backup.
• Customize the storage configuration.
• Create and configure databases.
• Configure database features.
– Oracle Data Guard
– Database File System (DBFS)
• Configure Enterprise Manager.

5 - 43 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Hardware Re-Racking

Hardware re-racking is supported subject to the following:

• Only Half Racks and smaller can be re-racked
– Re-racking of Full Racks is not supported
• The Exadata Hardware Re-rack Service must be purchased in
addition to the standard installation service.
– Re-racking occurs prior to installation
• The customer must supply a suitable target rack.
– The customer may also need to supply alternative PDUs
– See your Oracle representative for detailed specifications
• The original component layout must not be changed.
• No additional equipment may be installed.
• A re-racked Database Machine may be upgraded.
– But only up to a Half Rack

5 - 44 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Unsupported Configuration Activities

• Adding components to servers

• Swapping Linux distributions
• Configuring ACFS

5 - 45 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Quiz

Using the Oracle Exadata Deployment Assistant, you can set

specific IP addresses for each database server and Exadata
cell in Database Machine:
a. True
b. False

5 - 46 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Quiz

Which of the following options for connecting to existing SAN

storage is supported in conjunction with Database Machine?
a. Install a fiber channel HBA into each database server.
b. Use a server connected to the existing SAN as a storage
gateway and connect it to Database Machine using NFS
over Ethernet.
c. Use a server connected to the existing SAN as a storage
gateway and connect it to Database Machine using NFS
over InfiniBand.

5 - 47 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Summary

In this lesson, you should have learned how to describe:

• Installation and configuration process for Database
Machine
• Default configuration for Database Machine
• Supported and unsupported customizations for Database
Machine

5 - 48 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

 Practice 5 Overview:
Using the Oracle Exadata Deployment Assistant
In this practice, you are introduced to the Oracle Exadata
Deployment Assistant. You will use the assistant to generate a
set of configuration files for an example Database Machine
implementation scenario.

5 - 49 Copyright © 2015, Oracle and/or its affiliates. All rights reserved.