<Insert Picture Here>
Oracle Partitioning in Oracle Database 11g
Oracle Partitioning
Ten Years of Development
Core functionality Performance Static partition pruning Partition-wise joins Dynamic pruning Manageability Basic maintenance operations: add, drop, exchange Merge operation Global index maintenance Fast partition split Local Index maintenance Multi-dimensional pruning Fast drop table
Oracle8
Range partitioning Global range indexes Hash and composite range-hash partitioning List partitioning Composite range-list partitioning Global hash indexes 1M partitions per table
Oracle8i Oracle9i Oracle9i R2 Oracle10g Oracle10g R2
Oracle Partitioning
Ten Years of Development
Core functionality Performance Static partition pruning Partition-wise joins Dynamic pruning Manageability Basic maintenance operations: add, drop, exchange Merge operation Global index maintenance Fast partition split Local Index maintenance Multi-dimensional pruning Fast drop table Interval Partitioning Partition Advisor
Oracle8
Range partitioning Global range indexes Hash and composite range-hash partitioning List partitioning Composite range-list partitioning Global hash indexes 1M partitions per table More composite choices REF Partitioning Virtual Column Partitioning
Oracle8i Oracle9i Oracle9i R2 Oracle10g Oracle10g R2 Oracle Database 11g
Oracle Partitioning Fundamental Concepts for Success
While performance seems to be the most visible one, don't forget about the rest
partitioning must address all business-relevant areas of Performance, Manageability, and Availability
Partition autonomy is crucial
fundamental requirement for any partition maintenance operations acknowledge partitions as metadata in the data dictionary
Oracle Partitioning Enhancements
Complete the basic partitioning strategies
defines HOW data is going to be partitioned
new composite partitioning methods
Introduce partitioning extensions
defines WHAT controls the data placement
enhance the manageability and automation virtual column based partitioning REF partitioning interval partitioning partition advisor
<Insert Picture Here>
Composite Partitioning in Oracle Database 11g
Extended Composite Partitioning Strategies
Concept of composite partitioning Data is partitioned along two dimensions (A,B) A distinct value pair for the two dimensions uniquely determines the target partitioning Composite partitioning is complementary to multi-
column range partitioning Extensions in Oracle Database 11g ..
New 11g Strategy List Range Range - Range List - Hash List - List Use Case Geography -Time ShipDate - OrderDate Geography - OrderID Geography - Product
Composite Partitioning - Concept
Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date Jan 2006 Feb 2006
...
...
...
...
...
May 2006
... ... ...
...
Mar 2006 Jan 2007
...
Jan 2006 Feb 2006 order_date
Composite Partitioning - Concept
Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date
All records with
Jan 2006 Feb 2006
...
...
...
...
order_date in March 2006
...
May 2006
... ... ...
...
Mar Mar2006 2006 Jan 2007
...
Jan 2006 Feb 2006 order_date
Composite Partitioning - Concept
Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date
All records with
Jan 2006 Feb 2006
...
...
...
...
ship_date in May 2006
...
May May 2006
... ... ...
...
Mar 2006 Jan 2007
...
Jan 2006 Feb 2006 order_date
Composite Partitioning - Concept
Table SALES RANGE(order_date)-RANGE(ship_date)
ship_date
All records with
Jan 2006 Feb 2006
...
...
...
...
order_date in March 2006 AND ship_date in May 2006
...
May May 2006
... ... ...
...
Mar Mar2006 2006 Jan 2007
...
Jan 2006 Feb 2006 order_date
<Insert Picture Here>
Virtual Column based Partitioning
Virtual Columns
Business Problem
Extended Schema attributes are fully derived and dependent on
existing common data Redundant storage or extended view definitions are solving this problem today requires additional maintenance and creates overhead
Solution
Oracle Database 11g introduces virtual columns purely virtual, meta-data only Treated as real columns except no DML can have statistics eligible as partitioning key Enhanced performance and manageability
Virtual Columns - Example
Base table with all attributes ...
CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ...
12500 12507 12666 12875
Adams Blake King Smith
Virtual Columns - Example
Base table with all attributes ... ... is extended with the virtual (derived) column
CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2)))
12500 12507 12666 12875
Adams Blake King Smith
12 12 12 12
Virtual Columns - Example
Base table with all attributes ... ... is extended with the virtual (derived) column ... and the virtual column is used as partitioning key
CREATE TABLE accounts (acc_no number(10) not null, acc_name varchar2(50) not null, ... acc_branch number(2) generated always as (to_number(substr(to_char(acc_no),1,2))) partition by list (acc_branch) ...
12500 12507 12666 12875
Adams Blake King Smith
12 12 12 12
...
32320 32407 32758 32980
Jones Clark Hurd Phillips
32 32 32 32
<Insert Picture Here>
Interval Partitioning
Interval Partitioning
Partitioning is key-enabling functionality for managing
large volumes of data
Application
one logical object for application transparency multiple physical segments for administration
SQL
CDRs
but Physical segmentation requires additional data management overhead new partitions must be created on-time for new data
Jan Feb
Mar
Automate the partition management
Interval Partitioning
Interval Partitioning
extension to range partitioning full automation for equi-sized range partitions
Partitions are created as metadata information only
start partition is made persistent
Segments are allocated as soon as new data arrives
no need to create new partitions local indexes are created and maintained as well
No need for any partition management
Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');
Table SALES
...
Jan 2006 Feb 2006 Mar 2006 Jan 2007
...
Oct 2009 Nov 2009
...
First segment is created
Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');
Table SALES
...
Jan 2006 Feb 2006 Mar 2006 Jan 2007
...
Oct 2009 Nov 2009
...
Other partitions only exist in metadata
Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');
Table SALES
...
Jan 2006 Feb 2006 Mar 2006 Jan 2007
...
Oct 2009 Nov 2009
...
New segment is automatically allocated
INSERT INTO sales (order_date DATE, ...) VALUES ('04-MAR-2006',...);
Interval Partitioning
How it works
CREATE TABLE sales (order_date DATE, ...) PARTITON BY RANGE (order_date) INTERVAL(NUMTOYMINTERVAL(1,'month') (PARTITION p_first VALUES LESS THAN ('01-FEB-2006');
Table SALES
...
Jan 2006 Feb 2006 Mar 2006 Jan 2007
...
Oct 2009 Nov 2009
...
... whenever data for a new partition arrives
INSERT INTO sales (order_date DATE, ...) VALUES ('17-OCT-2009',...);
Interval Partitioning
How it works
Interval partitioned table can have classical range and
automated interval section
Automated new partition management plus full partition maintenance capabilities: Best of both worlds Table SALES
...
Jan 2006 Feb 2006 Mar 2006 Jan 2007
...
Oct 2009 Nov 2009
...
Interval Partitioning
How it works
Interval partitioned table can have classical range and
automated interval section
Automated new partition management plus full partition maintenance capabilities: Best of both worlds
Table SALES
...
2006 Range partition section MERGE and move old partitions for ILM Jan 2007
...
Oct 2009 Nov 2009
...
Interval Partitioning
How it works
Interval partitioned table can have classical range and
automated interval section
Automated new partition management plus full partition maintenance capabilities: Best of both worlds Table SALES
...
2006 Range partition section MERGE and move old partitions for ILM Insert new data - Automatic segment creation
...
Jan 2007 Oct 2009 Interval partition section Nov 2009
...
INSERT INTO sales (order_date DATE, ...) VALUES ('13-NOV-2009',...);
<Insert Picture Here>
REF Partitioning
REF Partitioning
Business Problem
Related tables benefit from same partitioning strategy
e.g. order lineitem
Redundant storage of the same information solves this problem
data overhead maintenance overhead
Solution
Oracle Database 11g introduces REF Partitioning
child table inherits the partitioning strategy of parent table through PK-FK relationship intuitive modelling
Enhanced Performance and Manageability
Before REF Partitioning
Table ORDERS
RANGE(order_date)
...
Jan 2006 Feb 2006
...
Primary key order_id
Redundant storage of order_date
Redundant maintenance
Table LINEITEMS
...
Jan 2006 Feb 2006
...
RANGE(order_date) Foreign key order_id
REF Partitioning
Table ORDERS
RANGE(order_date)
...
Jan 2006 Feb 2006
...
Primary key order_id
PARTITION BY REFERENCE Partitioning key inherited through PK-FK relationship Table LINEITEMS RANGE(order_date) Foreign key order_id
...
Jan 2006 Feb 2006
...
Partitioning Advisor
Packaged Apps Custom Apps
SQL Workload SQL Advisor SQL SQL Plan Access Structure Tuning Analysis Analysis
New!
Considers entire query
Partition Analysis
workload to improve query performance Advises on partitioning methods
Range (equal-interval), range key and interval Hash, hash key
SQL Profile
SQL Advice
Indexes & MVs
Partition Advice
Integrated, non-conflicting
advice with Indexes, MVs
Well-tuned SQL & Schema
Oracle Partitioning Summary
Proven functionality in 7th generation experience comes with age and customer usage Fundamental concepts for success the most comprehensive and complete offering 8th generation introduces ground-breaking new functionality extended COMPOSITE strategies further flexibility and performance INTERVAL for simplicity, ease of management VIRTUAL COLUMNS extend applicability with low maintenance REF enhanced performance for related tables
<Insert Picture Here>
Q&A
<Insert Picture Here>
Advanced Compression Option
Challenges
Explosion in data volume managed by Enterprises Government regulations (Sarbanes-Oxley, HIPPA, etc) User generated content (Web 2.0) IT managers must support larger volumes of data with
limited technology budgets
Need to optimize storage consumption Also maintain acceptable application performance
Intelligent and efficient compression technology can
help address these challenges
Introducing Advanced Compression Option
Oracle Database 11g introduces
a comprehensive set of compression capabilities
Structured/Relational data compression Unstructured data compression Compression for backup data Network transport compression
Reduces resource requirements
and costs
Storage System Network Bandwidth Memory Usage
Redo logs
Standby
Backups
Table Compression
Introduced in Oracle9i Release 2
Supports compression during bulk load operations (Direct Load, CTAS) Data modified using conventional DML not compressed
Optimized compression algorithm for relational data Improved performance for queries accessing large amounts of data
Fewer IOs Buffer Cache efficiency
Data is compressed at the database block level
Each block contains own compression metadata improves IO efficiency Local symbol table dynamically adapts to data changes
Compression can be specified at either the table or partition levels Completely transparent to applications Noticeable impact on write performance
OLTP Table Compression
Oracle Database 11g extends compression for OLTP
data
Support for conventional DML Operations (INSERT, UPDATE, DELETE)
New algorithm significantly reduces write overhead Batched compression ensures no impact for most OLTP transactions No impact on reads Reads may actually see improved performance due to fewer IOs and enhanced memory efficiency
OLTP Table Compression
Inserts are again uncompressed
Overhead Free Space Uncompressed
Block usage reaches PCTFREE triggers Compression Inserts are uncompressed
Compressed
Block usage reaches PCTFREE triggers Compression
Adaptable, continuous compression
Compression automatically triggered when block usage
reaches PCTFREE Compression eliminates holes created due to deletions and maximizes contiguous free space in block
Using OLTP Table Compression
Requires database compatibility level at 11.1 or greater New Syntax extends the COMPRESS keyword COMPRESS [FOR {ALL | DIRECT_LOAD} OPERATIONS] DIRECT_LOAD (DEFAULT) Refers to Bulk load operations from 10g and prior releases ALL OLTP + Direct loads Enable compression for a new table
CREATE TABLE t1 COMPRESS FOR ALL OPERATIONS
Enable only direct load compression on existing table
ALTER TABLE t2 COMPRESS
only new rows are compressed, existing rows are uncompressed
SecureFiles - Deduplication
Secure hash
Enables storage of a single physical image for duplicate data
Significantly reduces space consumption
Dramatically improves write and copy operations No adverse impact on read operations
may actually improve read performance for cache data
Duplicate detection happens within a table, partition or sub-partition
Specially useful for content management, email applications and
data archival applications
SecureFiles - Compression
Huge storage savings
Industry standard compression algorithms 2-3x compression for typical files (doc, pdf, xml) Minimal CPU overhead during compression
Automatically detects if SecureFile data is compressible
Skips compression for already compressed data Auto-turn off compression when space savings are minimal or zero
Two levels of compression provide different compression ratios
Compression Levels: MEDIUM (default), HIGH Higher the degree of compression, higher the latency and CPU overhead incurred
SecureFiles Compression is independent of table or index
compression Server-side compression
Allows for random reads and writes to SecureFile data Can be specified at a partition level
Data Pump Compression
Metadata compression available since Oracle
Database 10g Oracle Database 11g extends compression to table data during exports
no need to decompress before import
Single step compression of both data and metadata compressed data directly hits disk resulting in reduced disk space requirements 75% reduction in dump file size on export of sample OE and SH schemas Compression factor comparable to GNU gzip utility Application transparent complete Data Pump functionality available on compressed files
Backup data and Network transport Compression
Fast RMAN Compression compresses the backup set contents before writing them to disk or tape no extra decompression steps are required during recovery when you use RMAN compression. high performance, industry standard compression algorithm 40% faster backup compression versus Oracle Database 10g suitable for fast, incremental daily backups reduces network usage Data Guard Network Compression compression of redo traffic over the network improves redo transport performance gap resolution is up to 2x faster
Summary
Advanced Compression Option contains
comprehensive data compression capabilities for all types of data
Structured, Unstructured, Backup, Network Transport
Reduces storage consumption by 2 to 3 times Improves read performance
Enhances memory, buffer cache utilization
Complete application transparency Benefits diverse application workloads
VLDB
Partitioning Compression
