Oracle Exadata Cloud Service (ExaCS)

Technical Architecture
Copyright © 2021, Oracle and/or its affiliates. All rights reserved.

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and
are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not
use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any
form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for
interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors,
please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S.
Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on
the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the
applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure,
modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the
hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other
rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or
intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use
this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup,
redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages
caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and
are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are
trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third
parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to
third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle
Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-
party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Copyright © 2021, Oracle and/or its affiliates 2

Exadata Cloud Service (ExaCS) Architecture Overview

Oracle Exadata Cloud Service provides Oracle's Exadata Database Machine as a service in an Oracle
Cloud Infrastructure (OCI) data center. Alternatively, you can choose Exadata Cloud@Customer,
which provides Exadata Cloud Service that is hosted in your data center.

The Oracle Exadata Cloud Service instance is a virtual machine (VM) cluster that resides on Exadata
racks in an OCI region.

Your on-premises network or data center connects to your OCI tenancy by using Site-to-Site VPN or
OCI FastConnect. Your tenancy connects to the VM cluster through client and backup networks that
you create. You access your database through standard Oracle database connection methods, such
as Oracle Net. You access the VM cluster through standard Oracle Linux methods, such as token-
based Secure Shell (SSH).

Your administrator can use the web-based OCI Console, OCI command-line interface (CLI), and
REST APIs to connect to your OCI tenancy over an HTTPS connection. Your tenancy connects to the
Oracle service tenancy through the service management virtual cloud network (VCN). The Oracle
service tenancy connects to the Exadata Cloud Service infrastructure through the Oracle-managed
admin VCN.

Oracle Cloud Operations can use a console or REST APIs over an HTTPS or SSH connection to
manage the infrastructure through the Oracle service tenancy and the admin VCN.

Copyright © 2021, Oracle and/or its affiliates 3

Virtual Cloud Network (VCN)

The Oracle Exadata Cloud Service instance is a virtual machine (VM) cluster that resides on Exadata
database and storage server racks in an Oracle Cloud Infrastructure (OCI) region and availability
domain. A region is a localized geographic area, and an availability domain is one or more data
centers located within a region.

When setting up your Exadata Cloud Service instance, you create a virtual cloud network (VCN) to
connect your on-premises network or data center to the VM cluster that runs your database.

You connect your on-premises equipment to your VCN by using Site-to-Site VPN or OCI
FastConnect through a dynamic routing gateway (DRG). You can also optionally set up a network
address translation (NAT) gateway between the client subnet and the internet to reach public
endpoints that are not supported by the service gateway.

Within your VCN you have, at a minimum, two private subnets:

 Client subnet: Use this subnet to provide access from your on-premises network or data
center applications to your database. You can also connect your database to other cloud
services within the Oracle services network by using the client network through the service
gateway.

Copyright © 2021, Oracle and/or its affiliates 4

  Backup subnet: Use this subnet to back up your database to OCI Object Storage through the
service gateway. The backup network keeps high-load activities separate from application
connections. If you customize backups using the backup API, you might also need to set up a
static route to Object Storage.

To learn more about the Exadata Cloud Service network setup, see Network Setup for Exadata
Cloud Service Instances.

Copyright © 2021, Oracle and/or its affiliates 5

Rack Overview

Each instance of Oracle Exadata Cloud Service contains multiple database servers and Exadata
storage servers connected by high-speed, low-latency network fabric. The Exadata database and
storage server racks reside in an Oracle Cloud Infrastructure (OCI) region.

With the Exadata X8M elastic expansion option, the starting configuration is similar to a quarter
rack (2 database and 3 storage servers), and you can expand up to 32 database servers and 64
storage servers to support workloads of different sizes. For details about the hardware
configurations, see Exadata Shape Configurations.

Copyright © 2021, Oracle and/or its affiliates 6

The client and backup networks provide access to your virtual machine (VM) cluster on the
database servers. Oracle manages the infrastructure through the management network, which
connects the database and storage server hardware.

Copyright © 2021, Oracle and/or its affiliates 7

VM Cluster and Database Servers

Each Oracle Exadata Cloud Service database server contains a virtual machine (VM) guest running
on a hypervisor. This configuration ensures a distinct separation between the Oracle-managed and
customer-managed components.

Oracle manages the hypervisors through the management network. Each hypervisor uses minimal
resources: only 2 CPU cores (OCPUs) and 16 GB of RAM.

The client and backup networks connect to the VM guest through bonded network interfaces to
maximize performance and availability.

Copyright © 2021, Oracle and/or its affiliates 8

Each VM guest has a complete Oracle Database installation that includes all the features of Oracle
Database Enterprise Edition plus all the database enterprise management packs and all the
Enterprise Edition options, such as Oracle Database In-Memory and Oracle Real Application
Clusters (RAC), as well as Oracle Grid Infrastructure.

VM guests require an SSH public/private key pair for operating system security. You register a
public key in each guest, and you retain the private key that enables access to the VM operating
system. VM guests include standard user accounts, such as oracle, opc, grid, and root.

As a result of this configuration, you manage the VM guests and all the software they contain,
including the Oracle-provided management tools, including dbaascli for database lifecycle
management and ExaCLI for monitoring and managing your Exadata storage servers.

Each X8M database server can have up to 50 OCPUs and 1390 GB of DRAM. For details about other
hardware configurations, see Exadata Shape Configurations.

To learn more about the Exadata database configuration, see Scaling Options.

Each instance of Exadata Cloud Service contains multiple database servers and Exadata storage
servers connected through network fabric ports with active bonding. The Exadata database and
storage server racks reside in an Oracle Cloud Infrastructure (OCI) region. With the Exadata X8M
elastic expansion option, the starting configuration is similar to a quarter rack (two database and
three storage servers).

Oracle manages the infrastructure through the management network, which connects the database
and storage server hardware.

Copyright © 2021, Oracle and/or its affiliates 9

Storage Servers

When you create your Oracle Exadata Cloud Service instance, Oracle Automatic Storage
Management (ASM) provisions the storage space inside the Exadata storage servers. By default,
ASM creates the following disk groups:

 The DATA disk group stores Oracle Database data files.

 The RECO disk group stores the Fast Recovery Area (FRA), which is an area of storage where
Oracle Database can create and manage various files that are related to backup and recovery.
 (Not shown in the diagram) The /acfs file systems contain system files that support various
operations.

You can optionally create the SPARSE disk group to support Exadata snapshots. If you create the
SPARSE disk group, ASM allocates less space to the DATA and RECO disk groups.

Copyright © 2021, Oracle and/or its affiliates 10

You also need to decide whether to allocate storage to perform local backups to your Exadata
storage. If you choose this option, ASM allocates more space to the RECO disk group and less to the
DATA disk group.

To learn more about how ASM allocates space to disk groups, see Storage Configuration.

In X8M systems at the time of provisioning, each Exadata storage server has 25.6 TB of raw flash
storage and 1.5 TB of persistent memory (PMEM). The total for three storage servers is 76.8 TB of
raw flash storage and 4.5 TB of PMEM.

Three storage servers have a total of 504 TB of raw storage capacity. With this configuration the
ASM disk groups have the following usable space allowances:

 DATA: 52.2 - 119.2 TB

 RECO: 29.8 - 89.4 TB
 (Optional) SPARSE: 0 - 29.8 TB

The usable storage capacity is the storage available for Oracle Database files after taking into
account high-redundancy ASM mirroring (triple mirroring), which provides highly resilient database
storage on all Exadata Cloud Service configurations. The usable storage capacity does not factor in
the use of Exadata compression capabilities, which can increase the effective storage capacity.

For details about other hardware configurations, see Exadata Shape Configurations.

Each instance of Exadata Cloud Service contains multiple database servers and Exadata storage
servers connected through network fabric ports with active bonding. The Exadata database and
storage server racks reside in an Oracle Cloud Infrastructure (OCI) region. With the Exadata X8M
elastic expansion option, the starting configuration is similar to a quarter rack (two database and
three storage servers).

The client and backup networks provide access to your virtual machine (VM) cluster on the
database servers. Oracle manages the infrastructure through the management network, which
connects the database and storage server hardware.

Copyright © 2021, Oracle and/or its affiliates 11

Network Interfaces

Copyright © 2021, Oracle and/or its affiliates 12

The Oracle Exadata Cloud Service instance is a virtual machine (VM) cluster that resides on Exadata
database and storage server racks in an Oracle Cloud Infrastructure (OCI) region.

Note: For clarity, the diagram displays only one database server and one storage server.

The client and backup networks provide access to your virtual machine (VM) cluster on dedicated
(single-tenant) database servers. These networks use dedicated virtual cloud network (VCN)
hardware and connect to your VMs through bonded virtual network interface cards (VNICs).

Your VMs access your dedicated Exadata storage servers through a private, nonrouted interconnect
network with SR-IOV mapped interfaces. Each physical Exadata database server and storage server
has a highly available (HA) (active/standby) connection to a pair of redundant storage networking
switches.

The storage networking switches are shared (multitenant).

The database and storage servers are interconnected through a layer 2 management network.
There is no direct access from the management network to your client and backup networks. A
subset of Oracle cloud automation functionality accesses your VMs through the management
network by using a vNIC interface and a network address translation (NAT) address.

The Oracle-managed admin VCN connects to the management network through shared VCN
hardware and management network switches.

The following interfaces connect the various networks to the VM:

1. bondeth0 for the client network

2. bondeth1 for the backup network
3. eth0 for the admin network
4. re0 for a storage/Real Application Clusters (RAC) interconnect (isolated through VLAN)
5. re1 for a storage/RAC interconnect (isolated through VLAN)

To learn more about the network interfaces, see Exadata Cloud Service Security Controls (PDF).

Copyright © 2021, Oracle and/or its affiliates 13