Proceedings of the MIT 2007 Information Quality Industry Symposium

Data Quality Management

and Financial Services

financial services practice

Loretta O’Connor
Data Quality Sales Manager
Data Quality Division
May 2007

financial services practice

1 PG 961
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Content

• Introduction
• Defining the Data Quality Problem
• Solutions for Data Quality Issues
• Data Quality Reporting – Dashboards
• Data Quality Methodology – Successfully
Implementing a Data Quality Strategy
• Customer Examples
• Demo
• Q&A

2
PG 962
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Data Quality:
financial services practice
Problem Definition

3 PG 963
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Problem statement:
Poor Data Quality causes numerous business
problems

TDWI 2006

4
PG 964
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Initiatives Driving Data Quality

Industry / Business Driver

• CDI, Master Data Management (All)
Q u a l i t y

• Radio Frequency Identification (Manufacturing, CPG)

• Risk Management (Financial)
• Electronic availability of all services (Government)

Regulatory Compliance
• Basel II
• Sarbanes Oxley (SOX)
• Anti-Money Laundering (AML)
D a t a

Internal Drivers
• Data Warehouse / BI
•Data Migrations - Mergers and Acquisitions
• Application Consolidation

5
PG 965
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

The Impact

Problems Impact
• Applications crash • Applications unavailable
• Angry business people call the • Time consuming to trace and fix
operations team • Unhappy business people
• Ops track down the problems • Incorrect results
• Problems with the accuracy of the • Risk concerns
information being reported • Regulatory concerns
• Fixes being made without audit

Root Causes Contributory Factors

• Data didn’t arrive
• Data entry errors • Unclear / fragmented process
• Loose rules on source systems • Problem / data ownership
• Data consistency errors - Risk operations
• File column changes - Data providers
• Corrupted data • Multitudes of Log files

Source: Kevin Allen: Information Quality for Risk Management 6

PG 966
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

The Vicious Money Circle

Source Systems “Load takes too long Target Applications
and this is increasing
Agent our exposure”
Branches
Network $ AML
Engine
General
Ledger
$$
Analyze +$ $
Subsidary
Mainframe
$
Rework $$
$ $ Risk
CRM
$ Load Data
$ trust the $
Engine
“Can’t
so we must $
$
data,
$$ it”$$
Finance
Load
manually
3rd
check
Party $$
$ $ $$$ Reports SAP
Business

This vicious circle

“Wecreates high costs which
are non-compliant and cannot
we
be anticipated know the regulator will see this”

7
PG 967
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Data Quality:
financial services practice
The Solution

8 PG 968
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Existing fixes
Financial Institutions develop entire ecosystems to
compensate for poor data quality
• IT Operations
• All Ad Hoc
• Unix Scripts
• Application monitoring • All Manual
• Log file analysis • Expensive to Manage
• Manual updates to files to ‘make it work’
• Unreliable
• Business And management
• MS Access checks – run by business wonder why the annual
• Manual updates to files to ‘make it work’ IT budget keeps
• Same changes, every week! getting bigger?

Source: Kevin Allen: Information Quality for Risk Management 9

PG 969
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

DQM Approach & Methodology

ANALYZE ALIGN CLEANSE SUSTAIN

Data Quality is not a one off exercise!

Organizations must not only align and cleanse data,
but MUST also keep data clean over time
1. Content Profiling
1. Identify & Measure
Data Quality
2. Scorecarding
2. Define Data Quality
Rules & Targets 3. Align: e.g.
Standardization,
Analyze
Analyze 5. Monitor Data Quality
Versus Targets removing noise, align
Enhance
Enhance product attributes,
measures, classification.
3. Design Quality
4. Implement Quality
Improvement Processes
Improvement Processes
4. Cleanse / Address
duplicates

5. Re-Scorecard/Monitor

10
PG 970
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Data Quality Dimensions

Column
ColumnProfiling
Profiling What
Whatisisthe
thedata’s
data’sphysical
physicalcharacteristics
characteristics??Across
Acrossmultiple
multipletables?
tables?
Data
Relationship What
Exploration Relationship Whatrelationships
relationshipsexist
existininthe
thedata
dataset?
set?Across
Acrossmultiple
multipletables?
tables?

Redundancy What
Redundancy Whatdata
dataisisredundant?
redundant?Orphan
OrphanAnalysis
Analysis

Completeness What
Completeness Whatdata
dataisismissing
missingor
orunusable?
unusable?

Conformity What
Conformity Whatdata
dataisisstored
storedininaanon-standard
non-standardformat?
format?

Consistency What
Consistency Whatdata
datagives
givesconflicting
conflictinginformation?
information?
Data
Quality Accuracy What
Accuracy Whatdata
dataisisincorrect
incorrector
orout
outofofdate?
date?

Duplication What
Duplication Whatdata
datarecords
recordsare
areduplicated?
duplicated?

Integrity What
Integrity Whatdata
dataisismissing
missingimportant
importantrelationship
relationshiplinkages?
linkages?

Range What
Range Whatscores,
scores,values,
values,calculations
calculationsare
areoutside
outsideofofrange?
range?

11
PG 971
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Sample DQ Issues

Consistency:
Consistency:
Data is in correct format and
Incorrect
complete, Format
but breaks a business rule
Duplication:
Fuzzy matching
Completeness:
Conformity:
Missing Key Values
Incorrect Format Range:
Identify outliers

Integrity:
Accuracy:
UsingRelationship Identification
reference data to validate

COMPLETENESS CONFORMITY CONSISTENCY DUPLICATION INTEGRITY ACCURACY RANGE

12
PG 972
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Data Quality Maturity Model

Managed
Proactive
Reactive
Aware

Attitude • DQ seen as a cost • DQ for IT • DQ driven by business • DI and DQ seen

as key enabler
• Fully integrated
Tech. • Hand coded • Silos of DQ • Linked projects
DQ initiative

Benefit • Few • Few tactical • Key tactical gains • Strategic

Drivers depend on where you are and where you want to go

13
PG 973
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Sample Financial Services

financial services practice Business Intelligence
Dashboards

14PG 974
 Proceedings of the MIT 2007 Information Quality Industry Symposium

IDQ: Data Accuracy Scorecard

15
PG 975
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

3rd Party Reporting using IDQ

16
PG 976
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Methodology

financial services practice

17PG 977
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Scorecarding Back to Source™

1 2 3
Design &
Plan and Scope Data Acquisition Configure
Key
DQ Approach Plan Define Data Business Rules
& Data
Approach List
DQ Plan Priortise Quality Criteria

Business
Rules

6 5 4
Data Improvement Results Assessment
Analyse & Baseline
Data Updates & Cleansing
Data Cleansing Guidelines
Profile Data
Communicate Baseline
New processes Root Cause
Assessment Build Scorecard
Data Validation Set Improvement
Targets

18
PG 978
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Customers

financial services practice

19PG 979
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Master Data Management

Improve
• Enable business user to build data quality monitoring rules
• Provide standard platform that could be extended for further data quality initiatives

Challenge How We Helped Business Value

• Problems managing • Ability to monitor and • Data quality
trade promotions cleanse all types of data improvement leads to
because of poor data product, customer and more streamlined supply
quality business chain
• Flexibility to manage and • Faster more successful
control different data data migrations and
• Data migrations put at
quality problems on one systems consolidation
risk because of data
quality issues platform

20
PG 980
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium ref123

Third Largest Bank in the US

Informatica In Action
KEY BUSINESS IMPERATIVE

Regulatory Compliance
• Compliance with anti-money laundering regulations
IT/BUSINESS INITIATIVE: • Provide robust DQ reporting and metrics system for
Regulatory Reporting AML Unit

DATA QUALITY INITIATIVE:

DQ Reporting & Monitoring
THE CHALLENGE INFORMATICA ADVANTAGE RESULTS/BENEFITS

• Enable AML team to • Data quality workbench • Avoided regulatory

build, manage and for business users penalties of up $20m
customize AML business • Implemented AML DQ
rules Monitoring ahead of
deadline using existing
• Track and monitor data • Scorecard aggregating AML team resources
quality across key data quality metrics from
systems multiple systems • Saved estimated $3m+
cost of bespoke of AML
solution

21
PG 981
financial services practice
 Proceedings of the MIT 2007 Information Quality Industry Symposium

Reuters: Global CRM management

Key Business Requirements:
• “Fix data quality within existing Siebel systems”
Approach:
• Provide data quality metrics to drive improvement
processes
•Implement one off and ongoing data quality processes

Challenge Solution Expected Results

• Lack of ROI on Siebel due to • Increase in sales force and
• Informatica Data Quality
low quality data marketing efficiency
• To implement an
• Poor client management automated Data Quality • Recognised Data Quality
Scorecard per country metrics process in place
• Inaccurate mailing
processes • To implement one off
and then ongoing
• Inefficient marketing cleansing and
standardization
processes
• Informatica Data Explorer
• The manual generation of
monthly data quality reports • To profile new data
sources
very inefficient.

22
PG 982
financial services practice