Data Warehouse Architecture
Data warehousing Architecture
OLTP ETL Data Warehouse Reporting
Data warehousing Architecture
Cleansing, Transformation & Loading
Extract-
Push/Pull
Source 1
Canned Reports
Source 2
Ad-hoc analysis
Source 3
Summaries
/
Aggregatio
ns
Source n
Sources
Staging Data Reporting
Layer Warehouse Layer
Internal/External ETL DWH Reporting
Sources
Data warehousing Architecture (ODS)
Cleansing, Transformation & Loading
Extract-
Push/Pull
Source 1
Canned Reports
Source 2
Ad-hoc analysis
Source 3 Summaries
Detail Data /
Aggregatio
ns
Source n Transformatio
n
Summarization
Aggregation
Sources Staging ODS Data Reporting
Layer Warehouse Layer
Internal/External ETL ODS DWH Reporting
Sources
Data warehousing Architecture (ODS & Data Marts)
Cleansing, Transformation & Loading
Extract-
Push/Pull
Source 1
Canned Reports
Source 2
Ad-hoc analysis
Source 3 Summaries
Detail Data / Cubes-
Aggregatio Conformed
ns Dimension
s
Source n Transformatio
n
Summarization
Aggregation
Sources Staging ODS Data Data Reporting
Layer Warehouse Marts Layer
Internal/External ETL ODS DWH Data Marts Reporting
Sources
Data Modeling
STEPS in DATA MODELING
Requirement Gathering
Analysis
Logical Database Design
Physical Database design
DATA MODELING Types
Conceptual Data modeling
Describe data requirements from a business point of
view without technical details
Logical Data modeling
Refine conceptual models
Data structure oriented, platform independent
Physical Data modeling
Detailed specification of what is physically
implemented using specific technology
Conceptual Data Model
A conceptual model shows data through business eyes.
All entities which have business meaning.
Important relationships
Few significant attributes in the entities.
Logical Data Model
This is actual implementation and extension of conceptual data
model.
Logical data model includes all required entities, attributes, key
groups and relationships that represent business information and
define business rules.
Physical Data Model
A Physical data model may include
Referential Integrity
Indexes
Views
Alternate keys and other constraints
Table spaces and physical storage objects.
Enterprise Data Model
Enterprise data model sometimes called as Global
business model and the entire information would
be captured in the form of entities.
Enterprise Data Model Example
Entity-Relationship Modeling
Traditional modeling technique
Technique of choice for OLTP
Suited for corporate data warehouse
Limitations of E-R Modeling
Poor Performance
Tend to be very complex and difficult to
navigate.
Dimensional Modeling
Dimensional data modeling comprises of one or more dimension tables and fact tables.
Eg . Dimension table - Location, Product, Time , Organization etc.,
A Dimensional table stores Columns or dimensions that describe the objects in a fact
table. Dimension table contain the textual descriptors of the business. Each dimension is
defined by its single primary key.
End users can easily understand and navigate the data structure.
Dimensional Modeling
Dimensional modeling uses two basic concepts :
facts (measures), dimensions.
Is powerful in representing the requirements of the
business user in the context of database tables.
Focuses on numeric data, such as values counts,
weights, balances and occurrences.
Dimensional modeling
Must identify
Business process to be supported
Grain (level of detail)
Dimensions
Facts
What is Fact?
A fact is a collection of related data items,
consisting of measures and context data.
Each fact typically represents a business item, a
business transaction, or an event that can be used
in analyzing the business or business process.
Facts are measured, “continuously valued”,
rapidly changing information. Can be calculated
and/or derived.
Types of Facts
Additive
Additive facts are facts that can be summed up through all of the
dimensions in the fact table.
Able to add the facts along all the dimensions
Eg. Retail sales in $ (or) A sales fact
Semi-Additive
Semi-Additive facts are facts that can be summed up for some of the dimensions
in the fact table, but not the others.
Eg. Daily balances fact can be summed up through the customers dimension
but not through the time dimension.
Non-Additive
Non-Additive facts cannot be summed up for any of the dimensions present in
the fact table.
Eg. %(Percentages) , Ratios etc.,
Classification of Facts
Based on the classification , there are 2 types of Fact tables.
Cumulative Facts
Snapshot Facts
Cumulative Facts - This type of fact table describes what has happened over period
of time.
Eg. Additive Facts , Total sales by product by store by day or week or month or year .
Snapshot Facts – This type of fact table describes the state of things in a particular
instance of time.
Eg. Semi-Additive & Non-Additive facts .
Factless Fact Table
Some event tables have no obvious numeric facts
(measures) are called Factless fact tables.
Events often are modeled as a fact table containing a series
of keys, each representing a participating dimension in the
event.
Example :- Promotion table
PROMO ID Promotion Start Dt End Dt Description
2213 Credit card 230413 270413 10% cash back
2214 Credit Card 280413 010513 15% cash back
In the above example PROMO ID ‘s Surrogate Keys
and those are not measures.
Dimensions Types
Conformed Dimension
Junk Dimension
Slowly Changing Dimension
Degenerated Dimension
Dimensions Types
Conformed Dimension
A conformed dimension is a dimension, which is standard
across all data marts.
For example :- Enterprise Data Warehouse's data can
segmented into Sales Data Mart, Inventory and Shipping
Data Mart, Finance Data Mart, Geographical Data Mart,
HR and Management Data Mart and so on.
Dimensions Types
Dimensions Types
Junk Dimension
Junk Dimension is used to records a collection of low-
cardinality Flags and Indicators data.
Flag data may be non-generic question's answers like
Yes/No or True/False or Activate/Deactivate.
Indicator data may be tiny text data like Height,
Width,Weight, Color, Status.
Dimensions Types
Figure 1 :
Dimensions Types
Figure 2 :
Dimensions Types
Degenerated Dimension
The term degenerate dimension, refers to a field that will be
used as a criterion of analysis and that is stored in the fact
table.
For example :- If any fields from dimensions can not
perform grouping or summarized by the field in the fact
table.
Item number, Ticket numbers, Transaction number etc., are
examples of degenerated dimensions.
Data marts (DM)
- Data Mart is a subset of Data Warehouse.
It is really similar to a data warehouse but limited in scope and purpose
and is usually aligned with one department, function, application or
business unit.
Several names for DMs:
• Departmental DSS DBs
• OLAP Data bases
• multi-dimensional DBs (MDDB) or Cubes
• lightly summarized tables
Data marts Types
• Dependent data marts are marts that are fed directly by the DW,
sometimes supplemented with other feeds, such as external data.
• Independent data marts are marts that are fed directly by external
sources and do not use the DW.
• Embedded data marts are marts that are stored within the central DW.
They can be stored relationally as files or cubes.
Operational Data Store (ODS)
An ODS
• pulls together, validates, cleanses and integrates data
• foundation for providing integrated view of enterprise data.
• tactical decision support, day-to-day operations and management
reporting.
Characteristics
Integrated
Subject-oriented
Volatile (including update)
Current valued
Types of Schemas
- Star schema
- Snowflake schema
- constellation (or) Integrated (or) Galaxy (or)
Hybrid schema
Star Schema Design
Single fact table surrounded by denormalized dimension
tables
Fact table contains transaction type information.
Many star schemas in a data mart.
Easily understood by end users, more disk storage
required.
Example of Star Schema
Snowflake Schema
Single fact table surrounded by normalized dimension tables.
When dimensions become very large.
Less intuitive, slower performance due to joins.
Example of Snowflake Schema
