Introduction to Oracle

GoldenGate
 What is Oracle GoldenGate?
Oracle GoldenGate is a software product that allows you to replicate, filter, and transform data from
one database to another database. Using Oracle GoldenGate, you can move committed transactions
across multiple heterogeneous systems
 Why Do You Need Oracle GoldenGate?
Enterprise data is typically distributed across the enterprise in heterogeneous databases. To get data
between different data sources, you can use Oracle GoldenGate to load, distribute, and filter
transactions within your enterprise in real-time and enable migrations between different databases in
near zero-downtime.
 When Do You Use Oracle GoldenGate?
Oracle GoldenGate meets almost any data movement requirements you might have.
 How Do You Use Oracle GoldenGate?
After installation, Oracle GoldenGate can be configured to meet your organization's business needs.
Oracle GoldenGate key
features:
 Data movement is in real-time, reducing latency.
 Only committed transactions are moved, enabling consistency and improving
performance.
 Different versions and releases of Oracle Database are supported along with a wide
range of heterogeneous databases running on a variety of operating systems. You can
replicate data from an Oracle Database to a different heterogeneous database.
 Simple architecture and easy configuration.
 High performance with minimal overhead on the underlying databases and infrastructure.
 Oracle GoldenGate supports two architectures, the Classic Architecture and
the Microservices Architecture (MA).
Oracle GoldenGate Classic Architecture
Overview of the Oracle
GoldenGate Architecture
Oracle GoldenGate can be configured for the following purposes:
 A static extraction of data records from one database and the loading of those records to another database.
 Continuous extraction and replication of transactional Data Manipulation Language (DML) operations and
data definition language (DDL) changes (for supported databases) to keep source and target data
consistent.
 Extraction from a database and replication to a file outside the database.

Oracle GoldenGate is composed of the following components:

 Extract
 Data pump
 Replicat
 Trails or extract files
 Checkpoints
 Manager
 Collector
 1.Extract Process
 Oracle GoldenGate extract process resides on the source system and captures the committed transactions from the
source database. The DB logs may contain committed as well as uncommitted data but, remember, extract process
captures only committed transactions and write them to local trail files. It is important to note that Extract
captures only the committed transaction from its data source.
The extract can be configured for any of the following purposes:

 Initial Load: For the Initial Load method of replication, extract captures a static set of data directly from the source
table or objects.
 Change Synchronization: In this method of replication, extract process continuously captures data (DML and DDL)
from the source database to keep the source and target database in a consistent state of replication and it is the sole
method to implement continuous replication between the source and target database.

The data source of the extract process could be one of the following

 Source table (if the extract is configured for initial load)

 The database transaction logs or recovery logs such as (Oracle Redo Logs, Oracle Archive Logs, or SQL audit trails
or Sybase transaction logs) depending on the type of source database.
 Third-party capture module can also be used to extract transactional data from the source database. In this method, the
data and metadata from an external API are passed to the extract API.
 1.Extract Process Cont..

 Extract captures changes from the source database based on the extract configuration (contains the
objects to be replicated from the source database).
 Multiple extract processes can be configured on a source database to operate on same/different
source
objects.
 The extract performs either of the following tasks after extracting the data/records from the source
database objects.
• Delivers the data extracted from the source to the target server Trail Files through the collector
process
• Writes the data extracted from the source on to the Local Trail Files on the source system
 Optionally, Extract can also be configured to perform data filtering, transformation and mapping while
capturing data and or before transferring the data to the target system.
2.
DataPump

 This is an optional GoldenGate process (server process) on the source system and comes into
picture when the extracted data from the source is not directly transferred to the target Trail Files.
In the DataPump setup, the extract process gets the records/data from a source and keeps it in the
local file system by means of local Trail Files. The DataPump acts as a secondary extract process
where it reads the records from Local Trail Files and delivers to the Target system Trail files through
the collector.
 Data Pump is also known as secondary extract process. It is always recommended to include data
Pump in Goldengate configuration.
 3.
Collector
 The collector is a server process that runs in the background on the target system in a GoldenGate
replication setup where the extract is configured for continuous Change Synchronization.
 Collector has the following roles to perform in the GoldenGate replication.
• When a connection request is sent from the source extract, the collector process on the target system
scan and map the requesting connection to the available port and send the port details back to the
manager for assignment to the requesting extract process.
• Collector receives the data sent by source extract process and writes them to Trail Files on the target
system.
 There is one collector process on the target system per one extract process on the source system, i.e. it is
a one to one mapping between extract and collector process.
 4.
Replicat
 The Replicat process runs on the target system and is primarily responsible for replicating the extracted
data delivered to the target trail files by the source extract process.
 The replicat process scans the Trail Files on the target system, generates the DDL and DML from the Trail
Files and finally applies them on to the target system.
 Replicat has the following two types of configuration which relate to the type of extract being configured on
the source system.
• Initial loads: In initial data loads configuration, Replicat can apply a static data copy which is extracted by the
Initial load extract to target objects or route it to a high-speed bulk-load utility.
• Change synchronization: In change synchronization configuration, Replicat applies the continuous stream of
data extracted from the source objects to the target objects using a native database interface or ODBC
drivers, depending on the type of the target database.
 Optionally, Replicat can also be configured to perform data filtering, transformation, and mapping
before applying the transaction on to the target database
 5. Trail or Extract
Files
 Trails or Extract Files are the Operating system files which GoldenGate use to keep
records extracted from the source objects by the extract process. Trail files can be
created on the source system and target system depending on the GoldenGate
replication setup. Trail Files on the source system are called Extract Trails or Local
Trails and on the target system called as Remote Trails.

 Trail files are the reason why Goldengate is platform-independent.

 By using trail GoldenGate minimizes the load on the source database as once the
transaction logs/online logs/redo logs/ archive logs are extracted and loaded by the
extract process to trail files, all the operations like filtering, conversions, mapping
happens out of the source database. Use of trail file also makes the extraction and
replication process independent of each other.
6.
Manager
 Manager can be considered as the parent process in a GoldenGate replication setup on both the
source and target system. Manager, controls, manages and maintains the functioning of other
GoldenGate processes and files. A manager process is responsible for the following tasks.

1. Starting up Oracle GoldenGate processes

2. Maintaining port number for processes
3. Starting up dynamic processes
4. Performing GoldenGate Trail Management
5. Creating events, errors and threshold report.
7.
Checkpoints
 Checkpoints store the current read and write positions of a process to disk for recovery purposes. Checkpoints
ensure that data changes that are marked for synchronization actually are captured by Extract and applied to
the target by Replicat, and they prevent redundant processing. They provide fault tolerance by preventing the
loss of data should the system, the network, or an Oracle GoldenGate process need to be restarted. For complex
synchronization configurations, checkpoints enable multiple Extract or Replicat processes to read from the same
set of trails.

 Checkpoints work with inter-process acknowledgments to prevent messages from being lost in the network.
Oracle GoldenGate has a proprietary guaranteed-message delivery technology.

 Extract creates checkpoints for its positions in the data source and in the trail. Because Extract only captures
committed transactions, it keeps track of the operations in all open transactions, in the event that any of them
are committed. This requires Extract to record a checkpoint where it is currently reading in a transaction log, plus
the position of the start of the oldest open transaction, which can be in the current or any preceding log
Commit Sequence Number
(CSN)
 When working with Oracle GoldenGate, you might need to refer to a
Commit Sequence Number, or CSN. A CSN is an identifier that
Oracle GoldenGate constructs to identify a transaction for the
purpose of maintaining transactional consistency and data integrity.
It uniquely identifies a point in time in which a transaction commits
to the database.

 The CSN can be required to position Extract in the transaction log, to

reposition Replicat in the trail, or for other purposes. It is returned by
some conversion functions and is included in reports and certain
GGSCI output.
 Process
Types
 Depending on the requirement, Oracle GoldenGate can be configured with the following processing types.

 An online Extract or Replicat process runs until stopped by a user. Online processes maintain recovery checkpoints in the trail so
that processing can resume after interruptions. You use online processes to continuously extract and replicate DML and DDL
operations (where supported) to keep source and target objects synchronized. The EXTRACT and REPLICAT parameters apply to
this process type.

 A source-is-table (also known as in initial-load Extract) Extract process extracts a current set of static data directly
from the source objects in preparation for an initial load to another database. This process type does not use checkpoints. The
SOURCEISTABLE parameter applies to this process type.

 A special-run Replicat process applies data within known begin and end points. You use a special Replicat run for initial data
loads, and it also can be used with an online Extract to apply data changes from the trail in batches, such as once a day rather
than continuously. This process type does not maintain checkpoints, because the run can be started over with the same begin
and end points. The SPECIALRUN parameter applies to this process type.

 A remote task is a special type of initial-load process in which Extract communicates directly with Replicat over TCP/IP. Neither a
Collector process nor temporary disk storage in a trail or file is used. The task is defined in the Extract parameter file with the
RMTTASK parameter.