Scribd
Upload
392 views

Best Practices in Data Warehouse Testing GOOD

The document discusses best practices for testing data warehouses. It describes the various phases of data warehouse testing including business understanding, test case creation, and deployment. It outlines common testing goals like ensuring no data losses and validating data transformations. The types of testing covered include business flow testing, one-shot/prospective testing, data quality testing, incremental load testing, version testing, and production checkout testing. Challenges with data warehouse testing are the huge data volumes and complexity, broad testing scope, and different business rules for predictions versus actual data. Automating testing can help address issues like repeatedly performing regressions and better managing test data and results.

Uploaded by

srinivassutha
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
392 views

Best Practices in Data Warehouse Testing GOOD

The document discusses best practices for testing data warehouses. It describes the various phases of data warehouse testing including business understanding, test case creation, and deployment. It outlines common testing goals like ensuring no data losses and validating data transformations. The types of testing covered include business flow testing, one-shot/prospective testing, data quality testing, incremental load testing, version testing, and production checkout testing. Challenges with data warehouse testing are the huge data volumes and complexity, broad testing scope, and different business rules for predictions versus actual data. Automating testing can help address issues like repeatedly performing regressions and better managing test data and results.

Uploaded by

srinivassutha
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 18

Best Practices in Data Warehouse Testing

By :- Anindya Mookerjea Prasanth Malisetty

Agenda
Introduction to Data Warehouse Why require Best Practices in testing DWH Phases in data warehouse testing Data warehouse testing goals Types of Data Warehouse Testing Generic Challenges
www.Test2008.in 2

Introduction to Data Warehouse


A data warehouse is a: subject oriented Integrated Non-Volatile Time variant collection of data in support of managements decision.

www.Test2008.in

Why require Best Practices in testing DWH


How much confident a company can be to implement its data warehouse in the market without actually testing it thoroughly ? How much testing is enough to implement their data warehouse in the market?

www.Test2008.in

Why require Best Practices in testing DWH


Would it be possible for a company to remain at the top of the business if the bug is detected at the later stage of testing cycles? Undoubtedly we need to have in place the best practices in testing a data warehouse application

www.Test2008.in

Phases in data warehouse testing


Business understanding Test plan creation Test case creation Test data file for predictions Test predictions creation Test case execution Deployment in production environment

www.Test2008.in

Data warehouse testing goals


No Data losses Correct transformation rules Data validation Regression Testing Oneshot/ retrospective testing Prospective testing View testing Sampling Post implementation
www.Test2008.in 7

Types of Data Warehouse Testing


Business Flow One-Shot / Prospective Data Quality Incremental Load Performance Version Production Checkout

www.Test2008.in

Business Flow in Data Warehouse Testing


Understand Business Specification Create Test Plan/ Test Approach Develop Test Cases And Predictions Business Test Execution Cycle 1 (Initial Load)

Data Mocking

Business Test Execution Cycle 2 (Incremental Load)

Business Testing Sign off

www.Test2008.in

One-Shot/Prospective in Data Warehouse Testing


Business Requirements Specifications Retrospective Testing Data Sources MVS Prospective Testing

Test Data

Production

Test Data

Sampling of Data

Source File

Sampling of Data

Defect Test Cases and Predictions Test Cases and Predictions

Compare and Regression Testing

Compare and Regression Testing

Correct YES

NO

Analysis

NO

Correct

YES

Back out Testing

Correct YES

NO

Defect

BA Sign Off

Production Checkout

Production Checkout Sign Off

www.Test2008.in

10

Data Quality Data Warehouse Testing


Business Requirement Specification Test Data Transformation Rules Testing

DQ Testing for Key Fields

IS DQ % > Emergency %

IS DQ % > Threshold %

DQ E-mail And Abort ETL

DQ E-mail

www.Test2008.in

11

Incremental Load in Data Warehouse testing


Source Data Data loaded in the table (Initial Load) R: SE Complete Initial Load BA Testing (Cycle 1,Cycle 2) R: TESTER

Mocked data for Incremental Load Testing in Source file R: TESTER

Condition 1 Mocked Data for existing row update scenario (Natural Key match condition) Change any Non Natural Key attribute in Existing data Condition 3

Condition 2

Mocked Data for new row Insert scenario (Natural Key not match condition) Change any Natural Key attribute in Existing data

Mocked Data for do not delete scenarios (Natural Key and all other Non natural key attribute match condition) Create a new record same as exiting record.

OUTPUT: New record should be inserted in the table with changed natural key combination Load dt, Update Date = Job run date

OUTPUT: Existing record should get updated with new changes. Update Date = Job run date

OUTPUT: Keep record as it is. No change in Load and Update date.

Defects Found Raise MQC ticket in MQC and Communicate the issue on open line conference to Version DBA

Defects Found

Defects Found

Compare Before and After results

BA Testing Sign-off

NON Timeline Tables

www.Test2008.in

12

Version Data Warehouse Testing


Requirement in the form of Excel sheet ( Version change Sheet )
Changes/ Corrections Required Validate the requirements with Business Specification documents No Changes Develop Test Plan Approach Document

(This involves adds to physical tables and drops and recreates for view objects)

Addition of columns to objects

Deletion / Expansion of columns or Datatype changes to columns

(This involves renames to physical objects, drops and recreates for view objects)

Physical Tables
Running Describes on for physical tables for adding columns to the tables and Renames

Views
Select 1-5 rows (pre & post version predictions) to verify that columns were added with appropriate default value and appropriate changes

Compare Before and After results

Defects Found

Raise MQC ticket in MQC and Communicate the issue on open line conference to Version DBA

No Defects

BA Testing Sign-off

www.Test2008.in

13

Production Checkout Data Warehouse Testing


Testing only valid values get populated into table (as per BRDs)
Testing default values cross threshold % (for given attributes) Validates data is not corrupted (environmental factors) Verification vs. predictions

www.Test2008.in

14

Generic Challenges
Huge data volume and complexity. Data volume and complexity of data impacts performance & productivity. Scope of testing is broad as it also involves regression of data. There are different Change Requests which have different testing flow and associated rules. The data used for predictions is different and have to be mapped with actual data, based on different business rules. Chances of one scenario, in testing being repeated, is not much frequent. Currently all types of testing including regression is done by hand.
www.Test2008.in 15

Comparison of Flow - Manual Vs Automated


All Modules involve regression Automation Tool Manual Testing

Regression data captured in automation tool

Regression data captured in excel format

Code Implementation

Execute Automation Script

Create the query to Concatenate key fields & extracting data.

Captured data saved in excel format

Data Comparison

Data Sorting/ Formatting

Test Results are published saved in the required format.

www.Test2008.in

16

Any Questions?

www.Test2008.in

17

www.Test2008.in

18

You might also like