<Insert Picture Here>

Oracle Partitioning in Oracle Database 11g

Oracle Partitioning
Ten Years of Development
Core functionality Performance Static partition pruning Partition-wise joins Dynamic pruning Manageability Basic maintenance operations: add, drop, exchange Merge operation Global index maintenance Fast partition split Local Index maintenance Multi-dimensional pruning Fast drop table

Oracle8

Range partitioning Global range indexes Hash and composite range-hash partitioning List partitioning Composite range-list partitioning Global hash indexes 1M partitions per table

Oracle8i Oracle9i Oracle9i R2 Oracle10g Oracle10g R2

Oracle Partitioning
Ten Years of Development
Core functionality Performance Static partition pruning Partition-wise joins Dynamic pruning Manageability Basic maintenance operations: add, drop, exchange Merge operation Global index maintenance Fast partition split Local Index maintenance Multi-dimensional pruning Fast drop table Interval Partitioning Partition Advisor

Oracle8

Range partitioning Global range indexes Hash and composite range-hash partitioning List partitioning Composite range-list partitioning Global hash indexes 1M partitions per table More composite choices REF Partitioning Virtual Column Partitioning

Oracle8i Oracle9i Oracle9i R2 Oracle10g Oracle10g R2 Oracle Database 11g

Oracle Partitioning Fundamental Concepts for Success

While performance seems to be the most visible one, don't forget about the rest
partitioning must address all business-relevant areas of Performance, Manageability, and Availability

Partition autonomy is crucial

fundamental requirement for any partition maintenance operations acknowledge partitions as metadata in the data dictionary

Oracle Partitioning Enhancements

Complete the basic partitioning strategies

defines HOW data is going to be partitioned

new composite partitioning methods

Introduce partitioning extensions

defines WHAT controls the data placement

enhance the manageability and automation virtual column based partitioning REF partitioning interval partitioning partition advisor

<Insert Picture Here>

Composite Partitioning in Oracle Database 11g

Extended Composite Partitioning Strategies

Concept of composite partitioning Data is partitioned along two dimensions (A,B) A distinct value pair for the two dimensions uniquely determines the target partitioning Composite partitioning is complementary to multi-

column range partitioning Extensions in Oracle Database 11g ..

New 11g Strategy List Range Range - Range List - Hash List - List Use Case Geography -Time ShipDate - OrderDate Geography - OrderID Geography - Product

Composite Partitioning - Concept

Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date Jan 2006 Feb 2006

...
...

...
...

...
May 2006

... ... ...

...
Mar 2006 Jan 2007

...
Jan 2006 Feb 2006 order_date

Composite Partitioning - Concept

Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date

All records with

Jan 2006 Feb 2006

...
...

...
...

order_date in March 2006

...
May 2006

... ... ...

...
Mar Mar2006 2006 Jan 2007

...
Jan 2006 Feb 2006 order_date

Composite Partitioning - Concept

Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date

All records with

Jan 2006 Feb 2006

...
...

...
...

ship_date in May 2006

...
May May 2006

... ... ...

...
Mar 2006 Jan 2007

...
Jan 2006 Feb 2006 order_date

Composite Partitioning - Concept

Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date

All records with

Jan 2006 Feb 2006

...
...

...
...

order_date in March 2006 AND ship_date in May 2006

...
May May 2006

... ... ...

...
Mar Mar2006 2006 Jan 2007

...
Jan 2006 Feb 2006 order_date

<Insert Picture Here>

Virtual Column based Partitioning

Virtual Columns
Business Problem
Extended Schema attributes are fully derived and dependent on

existing common data Redundant storage or extended view definitions are solving this problem today requires additional maintenance and creates overhead

Solution
Oracle Database 11g introduces virtual columns purely virtual, meta-data only Treated as real columns except no DML can have statistics eligible as partitioning key Enhanced performance and manageability

Virtual Columns - Example

Base table with all attributes ...

CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ...

12500 12507 12666 12875

Adams Blake King Smith

Virtual Columns - Example

Base table with all attributes ... ... is extended with the virtual (derived) column
CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))

12500 12507 12666 12875

Adams Blake King Smith

12 12 12 12

Virtual Columns - Example

Base table with all attributes ... ... is extended with the virtual (derived) column ... and the virtual column is used as partitioning key
CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2))) partition by list (acc_branch) ...

12500 12507 12666 12875

Adams Blake King Smith

12 12 12 12

...

32320 32407 32758 32980

Jones Clark Hurd Phillips

32 32 32 32

<Insert Picture Here>

Interval Partitioning

Interval Partitioning
Partitioning is key-enabling functionality for managing

large volumes of data

Application

one logical object for application transparency multiple physical segments for administration

SQL
CDRs

but Physical segmentation requires additional data management overhead new partitions must be created on-time for new data

Jan Feb

Mar

Automate the partition management

Interval Partitioning
Interval Partitioning
extension to range partitioning full automation for equi-sized range partitions

Partitions are created as metadata information only

start partition is made persistent

Segments are allocated as soon as new data arrives

no need to create new partitions local indexes are created and maintained as well

No need for any partition management

Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');

Table SALES

...
Jan 2006 Feb 2006 Mar 2006 Jan 2007

...
Oct 2009 Nov 2009

...

First segment is created

Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');

Table SALES

...
Jan 2006 Feb 2006 Mar 2006 Jan 2007

...
Oct 2009 Nov 2009

...

Other partitions only exist in metadata

Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');

Table SALES

...
Jan 2006 Feb 2006 Mar 2006 Jan 2007

...
Oct 2009 Nov 2009

...

New segment is automatically allocated

INSERT INTO sales (order_date DATE, ...) VALUES ('04-MAR-2006',...);

Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');

Table SALES

...
Jan 2006 Feb 2006 Mar 2006 Jan 2007

...
Oct 2009 Nov 2009

...

... whenever data for a new partition arrives

INSERT INTO sales (order_date DATE, ...) VALUES ('17-OCT-2009',...);

Interval Partitioning
How it works
Interval partitioned table can have classical range and

automated interval section

Automated new partition management plus full partition maintenance capabilities: Best of both worlds Table SALES

...
Jan 2006 Feb 2006 Mar 2006 Jan 2007

...
Oct 2009 Nov 2009

...

Interval Partitioning
How it works
Interval partitioned table can have classical range and

automated interval section

Automated new partition management plus full partition maintenance capabilities: Best of both worlds
Table SALES

...
2006 Range partition section MERGE and move old partitions for ILM Jan 2007

...
Oct 2009 Nov 2009

...

Interval Partitioning
How it works
Interval partitioned table can have classical range and

automated interval section

Automated new partition management plus full partition maintenance capabilities: Best of both worlds Table SALES

...
2006 Range partition section MERGE and move old partitions for ILM Insert new data - Automatic segment creation

...
Jan 2007 Oct 2009 Interval partition section Nov 2009

...

INSERT INTO sales (order_date DATE, ...) VALUES ('13-NOV-2009',...);

<Insert Picture Here>

REF Partitioning

REF Partitioning
Business Problem
Related tables benefit from same partitioning strategy

e.g. order lineitem

Redundant storage of the same information solves this problem

data overhead maintenance overhead

Solution
Oracle Database 11g introduces REF Partitioning
child table inherits the partitioning strategy of parent table through PK-FK relationship intuitive modelling

Enhanced Performance and Manageability

Before REF Partitioning

Table ORDERS
RANGE(order_date)

...
Jan 2006 Feb 2006

...

Primary key order_id

Redundant storage of order_date

Redundant maintenance

Table LINEITEMS

...
Jan 2006 Feb 2006

...

RANGE(order_date) Foreign key order_id

REF Partitioning
Table ORDERS
RANGE(order_date)

...
Jan 2006 Feb 2006

...

Primary key order_id

PARTITION BY REFERENCE Partitioning key inherited through PK-FK relationship Table LINEITEMS RANGE(order_date) Foreign key order_id

...
Jan 2006 Feb 2006

...

Partitioning Advisor
Packaged Apps Custom Apps

SQL Workload SQL Advisor SQL SQL Plan Access Structure Tuning Analysis Analysis

New!

Considers entire query

Partition Analysis

workload to improve query performance Advises on partitioning methods

Range (equal-interval), range key and interval Hash, hash key

SQL Profile

SQL Advice

Indexes & MVs

Partition Advice

Integrated, non-conflicting

advice with Indexes, MVs

Well-tuned SQL & Schema

Oracle Partitioning Summary

Proven functionality in 7th generation experience comes with age and customer usage Fundamental concepts for success the most comprehensive and complete offering 8th generation introduces ground-breaking new functionality extended COMPOSITE strategies further flexibility and performance INTERVAL for simplicity, ease of management VIRTUAL COLUMNS extend applicability with low maintenance REF enhanced performance for related tables

<Insert Picture Here>

Q&A

<Insert Picture Here>

Advanced Compression Option

Challenges
Explosion in data volume managed by Enterprises Government regulations (Sarbanes-Oxley, HIPPA, etc) User generated content (Web 2.0) IT managers must support larger volumes of data with

limited technology budgets

Need to optimize storage consumption Also maintain acceptable application performance

Intelligent and efficient compression technology can

help address these challenges

Introducing Advanced Compression Option

Oracle Database 11g introduces

a comprehensive set of compression capabilities

Structured/Relational data compression Unstructured data compression Compression for backup data Network transport compression

Reduces resource requirements

and costs

Storage System Network Bandwidth Memory Usage

Redo logs

Standby

Backups

Table Compression
Introduced in Oracle9i Release 2

Supports compression during bulk load operations (Direct Load, CTAS) Data modified using conventional DML not compressed
Optimized compression algorithm for relational data Improved performance for queries accessing large amounts of data

Fewer IOs Buffer Cache efficiency

Data is compressed at the database block level

Each block contains own compression metadata improves IO efficiency Local symbol table dynamically adapts to data changes
Compression can be specified at either the table or partition levels Completely transparent to applications Noticeable impact on write performance

OLTP Table Compression

Oracle Database 11g extends compression for OLTP

data
Support for conventional DML Operations (INSERT, UPDATE, DELETE)

New algorithm significantly reduces write overhead Batched compression ensures no impact for most OLTP transactions No impact on reads Reads may actually see improved performance due to fewer IOs and enhanced memory efficiency

OLTP Table Compression

Inserts are again uncompressed

Overhead Free Space Uncompressed

Block usage reaches PCTFREE triggers Compression Inserts are uncompressed

Compressed

Block usage reaches PCTFREE triggers Compression

Adaptable, continuous compression

Compression automatically triggered when block usage

reaches PCTFREE Compression eliminates holes created due to deletions and maximizes contiguous free space in block

Using OLTP Table Compression

Requires database compatibility level at 11.1 or greater New Syntax extends the COMPRESS keyword COMPRESS [FOR {ALL | DIRECT_LOAD} OPERATIONS] DIRECT_LOAD (DEFAULT) Refers to Bulk load operations from 10g and prior releases ALL OLTP + Direct loads Enable compression for a new table
CREATE TABLE t1 COMPRESS FOR ALL OPERATIONS

Enable only direct load compression on existing table

ALTER TABLE t2 COMPRESS

only new rows are compressed, existing rows are uncompressed

SecureFiles - Deduplication

Secure hash

Enables storage of a single physical image for duplicate data

Significantly reduces space consumption

Dramatically improves write and copy operations No adverse impact on read operations

may actually improve read performance for cache data

Duplicate detection happens within a table, partition or sub-partition
Specially useful for content management, email applications and

data archival applications

SecureFiles - Compression
Huge storage savings
Industry standard compression algorithms 2-3x compression for typical files (doc, pdf, xml) Minimal CPU overhead during compression

Automatically detects if SecureFile data is compressible

Skips compression for already compressed data Auto-turn off compression when space savings are minimal or zero

Two levels of compression provide different compression ratios

Compression Levels: MEDIUM (default), HIGH Higher the degree of compression, higher the latency and CPU overhead incurred

SecureFiles Compression is independent of table or index

compression Server-side compression

Allows for random reads and writes to SecureFile data Can be specified at a partition level

Data Pump Compression

Metadata compression available since Oracle

Database 10g Oracle Database 11g extends compression to table data during exports
no need to decompress before import

Single step compression of both data and metadata compressed data directly hits disk resulting in reduced disk space requirements 75% reduction in dump file size on export of sample OE and SH schemas Compression factor comparable to GNU gzip utility Application transparent complete Data Pump functionality available on compressed files

Backup data and Network transport Compression

Fast RMAN Compression compresses the backup set contents before writing them to disk or tape no extra decompression steps are required during recovery when you use RMAN compression. high performance, industry standard compression algorithm 40% faster backup compression versus Oracle Database 10g suitable for fast, incremental daily backups reduces network usage Data Guard Network Compression compression of redo traffic over the network improves redo transport performance gap resolution is up to 2x faster

Summary
Advanced Compression Option contains

comprehensive data compression capabilities for all types of data

Structured, Unstructured, Backup, Network Transport

Reduces storage consumption by 2 to 3 times Improves read performance

Enhances memory, buffer cache utilization

Complete application transparency Benefits diverse application workloads

VLDB
Partitioning Compression