Scribd
Upload
70 views

MySQL Databases at RightMedia

This document discusses MySQL databases used at RightMedia, an advertising exchange. It describes three main MySQL databases: 1) Rpt_prod is a 6.5TB reporting data warehouse stored across 51 database servers with a flat schema that is being converted to a star schema. 2) Trf_prod is the operational store for key systems like the YieldManager UI and processes over 500 million transactions per day across 7 database servers. 3) Predict databases store data for adserver prediction engines to optimize costs and revenue. The document also provides an overview of MySQL architecture, features, storage engines and the query cache.

Uploaded by

Saeed Meethal
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
70 views

MySQL Databases at RightMedia

This document discusses MySQL databases used at RightMedia, an advertising exchange. It describes three main MySQL databases: 1) Rpt_prod is a 6.5TB reporting data warehouse stored across 51 database servers with a flat schema that is being converted to a star schema. 2) Trf_prod is the operational store for key systems like the YieldManager UI and processes over 500 million transactions per day across 7 database servers. 3) Predict databases store data for adserver prediction engines to optimize costs and revenue. The document also provides an overview of MySQL architecture, features, storage engines and the query cache.

Uploaded by

Saeed Meethal
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 17

MySQL Databases at RightMedia

Intro To RightMedia
RightMedia is an advertising exchange
Publishers, Advertisers, Networks compete and co-operate Rules and links between networks, publishers and advertisers lead to auctions for every ad call for fairness.

Over 7 Billion impressions per day!


Almost 8,000 ad servers in SP1,SK1,AC4,CH1,JP1
At peak, servers capable of processing 100K impressions/second With pixels, cookie requests, etc, 250K calls/second

-2-

Yahoo! Confidential

Data Cycle
Adserver data is aggregated and flushed every 10 minutes

Multiple aggregation layers per colo


Multiple data pipelines (similar to Data Highway) Reporting/Impression Log data stored in MySQL (rpt_prod and logdb) Customers use hourly reporting data to make decisions on campaigns and spending (trf_prod) Cache generation cycle every 2 hours
Traffic cache Predict cache Budget cache

-3-

Yahoo! Confidential

Data Cycle (Cont.)


Caches are pushed to each colo and distributed to ad servers. Cycle starts again

-4-

Yahoo! Confidential

MySQL Databases @ RightMedia


Rpt_prod Reporting Data Warehouse
51 database servers : 16 masters, 32 slaves (3 for archive) Data sharded over 16 partitions Middle ware server ReportWare converts XML requests to SQL and returns data in XML.

Reportware aggregates the 16 result set queries into one


DB size is 6.5TB All Dell PE 2950s 32GB memory, 2xQuad CPU, 6x146Gb SAS drives

Current schema very flat all tables are similar to materialized views
New version (AC4) Star/Snowflake schema

-5-

Yahoo! Confidential

MySQL Databases @ RightMedia (Cont.)


Trf_prod Heart of RightMedia
7 database servers: 1 master 6 slaves per colo Provides operational store for YieldManager UI, API, Creative Tester, Analytics, Predict, Budget processes. Very high transaction rate : nearly Half a billion per day

Relatively small but important


Downtime leads to larger failures
Ideal candidate for MySQL Cluster (Fault-tolerant system)

-6-

Yahoo! Confidential

MySQL Databases @ RightMedia (Cont.)


Predict Databases
Data store for adserver learning Prediction engine uses this data to lower CPM and maximize revenue Key system for LZ1.1, LZ2, LZ3

-7-

Yahoo! Confidential

MySQL Databases @ RightMedia (Cont.)


Budget Databases

-8-

Yahoo! Confidential

MySQL Architecture
Features and Components

The Big Picture

- 10 -

Yahoo! Confidential

Features MySQL vs Oracle


Features/Functionality
Strenths

MySQL
Price/Performance Great performce

Oracle
Capable of running large OLTPs and VLDBs

Database Server Instance

Instance stores global memory in mysqld background process


Made up of database schemas. Each storage engine stores information differently.

Instance has numerous background processes.


System SGA shared by procs. Tablespaces used for system metadata. $$$ Lots of options $$$ More complex, but lots of options. RMAN Advanced features

Physical Storage

Partitioning Replication

Free, in Version 5.1 Limited functionality Free, relatively simple setup. Great horizontal scalability

Backup/Recovery Stored Procedures

No online backup Home-grown/3rd party Very basic


- 11 -

Yahoo! Confidential

Query Cache
Global memory area used to cache queries and result sets.
SQL query (SELECT) and result stored in hash table Query is checked before parsing and optimization and result set returned immediately WICKED FAST PERFORMANCE!!

Excellent for static tables or tables with low change rate


Any INSERT,UPDATE,DELETE or any other DML operation on a table will invalidate all cache entries for that table.

- 12 -

Yahoo! Confidential

Storage Engine Architecture


Most flexible (and coolest) part of MySQL server.

Allows the freedom of choice to mix and match storage engines within the same database and server instance.
Storage engines are pluggable and can be dynamically loaded into the server instance.

Storage engine API allows developers to enhance and create own storage engines.
The MySQL community offers different pluggable engines for different business solutions.

- 13 -

Yahoo! Confidential

MySQL Storage Engines


MyISAM
Default storage engine for MySQL Disk Based storage - BTREE Table level locking No transactional support

Very low memory footprint


Full Text searching capability Index cache only OS buffers for data cache Extremely high performance Prone to corruption on server crash

- 14 -

Yahoo! Confidential

MySQL Storage Engines (Cont.)


InnoDB
ACID Compliant Transaction/XA support Row level locking Clustered index

Both data and index cached


Recovery can be time consuming can very slow Not scalable (Can be fixed with patches)

- 15 -

Yahoo! Confidential

MySQL Storage Engines (Cont.)


Memory (HEAP)
Similar to MyISAM, but all data/index resides in RAM Extremely fast for quick reference lookups Has finite storage limits (Can be configurable with max_heap_table_size)

Supports both BTREE and Hash indexes


No TEXT or BLOB columns allowed

- 16 -

Yahoo! Confidential

MySQL Storage Engines (Cont.)


Archive
Highly compressed data files Useful for storage of historical data Only allowed operations are SELECT and INSERT No indexes
MySQL 5.1 now allows index on AUTO_INCREMENT columns

- 17 -

Yahoo! Confidential

You might also like