© Copyright 2010 Hitachi Consulting
www.hitachiconsulting.com
Master Data Management (MDM) Data Governance Leadership and Best Practices
Dinesh Chandrasekar Practice Director CRM & MDM Hitachi Consulting , GDC
1
© Copyright 2010 Hitachi Consulting
Agenda
2 2
Impact of Poor Data & Need for DQ Why MDM & Customer Hub Customer Data Problems & Solutions Significance of Data Governance Data Governance Leadership Strategies Data Stewardship Best Practices Open Forum
© Copyright 2010 Hitachi Consulting
Acronyms
Commercial in Confidence 3
EIM – Enterprise Information Management EDM – Enterprise Data Management MDM – Master Data Management DM – Data Management DG – Data Governance DQ – Data Quality SOR – System of Record KPI – Key Performance Indicators UCM – Universal Customer Master CDH – Customer Data Hub PDH – Product Data Hub SH – Supplier Hub & Site Hub CH – Customer Hub
© Copyright 2010 Hitachi Consulting
How clean is your Wind Shield ?
Commercial in Confidence 4
“ Ultimately, poor data is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some
point, you either have to stop and clear the windshield or Risk everything.” - Ken Orr Institute
© Copyright 2010 Hitachi Consulting
Impact of Poor Data Quality
Commercial in Confidence 5
“… Fortune 1000 enterprises will lose more money in operational inefficiency due to data quality issues than they will spend on data warehouse and CRM
initiatives.”
“Data integration and data quality are fundamental prerequisites for the successful implementation of enterprise applications,
such as CRM, SCM, and ERP.”
Ineffective Cross-sell/Up-sell
Lower call center productivity
Increased marketing mailing costs
Reduced CRM adoption rate
Customer Service
Risk, Compliance Management
Heightened credit risk costs
Potential non-compliance risk
Increased report generation costs
Increased data management costs
Increased sales order error
Delayed sales cycle time (B2B)
Mediocre campaign response rate
Operational Efficiency
Reduced IT Agility
Increased integration costs
Increased the time to bring new projects and services to market
Proliferation of data problems from silos to more applications
© Copyright 2010 Hitachi Consulting Commercial in Confidence 6
Ever proliferating islands of information
…in disparate applications covering multiple channels, divisions & functions
…duplicated, incomplete, inaccurate data
• Key enterprise processes based on unclean / incomplete data Marketing, sales, service & customer retention processes, regulatory compliance, new product introduction,…
• Unclean data makes Analytics invalid
• Error prone integration
• Slows enterprise agility and innovation
Web site
Call Center SFA Partner Fusion
App
Fusion App
SCM ERP2 Legacy ERP 1
Fragmented data is the source of the problem
© Copyright 2010 Hitachi Consulting Commercial in Confidence 7
Consolidate/Federate shared information into one place
Cleanse data centrally
Share data as a single point of truth as a service
Consistency siloed environments (Integrated Best of Breed) Lower data management costs Better reporting Enterprise foundation for agility & innovation
ETL
ETL
Middleware
Application Integration Architecture BI
Analytics
Web site
Call Center
SFA Partner Fusion App
Fusion App
SCM ERP2 Legacy ERP 1
MDM
MDM : The source of clean data for the enterprise Nurture one of your most valuable asset
© Copyright 2010 Hitachi Consulting
The New Age Digital Customer
© Copyright 2010 Hitachi Consulting
Why Customer Hub ?
Commercial in Confidence
Unify your Customer View with Customer Hub
9
Maximize Customer Retention Provides complete knowledge of customers value and history to improve customer loyalty Ensures effective marketing and selling while avoiding missteps Enables sharing of customer information with applications, business processes and point of
contact personnel
Increase Selling Efficiencies Facilitates accurate up-selling and cross-selling of products and services Provides accurate product data which reduces order entry errors and decreases days sales
outstanding Delivers full quality customer and product information at the point of contact
Reduces Cost and Risk Provides clean data to all applications and business processes increasing ROI from existing
investments Enables data governance to insure compliance and reduce risk Accelerates time-to-market of new products and services
© Copyright 2010 Hitachi Consulting
Why Organizations engage in Customer Hub Projects?
Commercial in Confidence
Benefits
10
GROWTH Improve CRM
performance to increase revenue and
market share
EFFICIENCY
Operational efficiency across
multi-functions of an enterprise
IT AGILITY
Increase IT resiliency in a changing
business landscape
COMPLIANCE
Reduce operational risk and improve
regulatory compliance
CUSTOMERS ON AVERAGE
GENERATED 2%-5% INCREASED
REVENUE FROM SALES WITH
MDM
EFFICIENCY OF OPERATIONS
INCREASE WITH IMPROVED
PROCESSES AND DATA
GOVERNANCE
EFFICIENCY OF IT
OPERATIONS RESULTING IN
GREATER AGILITY OF
BUSINESS MODELS
EFFICIENCY OF IT OPERATIONS
RESULTING IN GREATER
AGILITY OF BUSINESS MODELS
© Copyright 2010 Hitachi Consulting
Customer Hub Styles
Commercial in Confidence 11
Registry Style •Various Source System publish their data and a Subscribing Hub stores only the Foreign Keys , Source System Ids and Key data values needed for matching •The Hub runs the cleansing and matching algorithms and assigns unique global identifier to the matching records , but does not send any data back to the Source Systems •The Registry Style Hub is to build the “ Virtual Golden View of the master entity from the Source Systems”
Consolidation Style • The Consolidation Style MDM Hub has a physically instantiated, "golden" record stored in the central Hub • The authoring of the data remains distributed across the spoke systems and the master data can be updated based on events, but is not guaranteed to be up to date. •The master data in this case is usually not used for transactions, but rather supports reporting; however, it can also be used for reference operationally.
Transaction Style • In this architecture, the Hub stores, enhances and maintains all the relevant (master) data attributes. • It becomes the authoritative source of truth and publishes this valuable information back to the respective source systems. • The Hub publishes and writes back the various data elements to the source systems after the linking, cleansing, matching and enriching algorithms have done their work. Upstream, transactional applications can read master data from the MDM Hub, and, potentially, all spoke systems subscribe to updates published from the central system in a form of harmonization. •The Hub needs to support merging of master records. Security and visibility policies at the data attribute level need to be supported by the Transaction Style hub, as well.
Simple & Faster Medium Complex Complex
Short term Gain Mid term Gain Long term Gain
© Copyright 2010 Hitachi Consulting
Oracle Enterprise Master Data Management
© Copyright 2010 Hitachi Consulting Commercial in Confidence 13
Gartner Magic Quadrant for Customer Hub Solutions
“UCM has the strength of the Oracle name behind it, leading to an impressive number of commitments from blue chip names in the Siebel customer base across a range of industries”
John Radcliffe, Gartner, May 2008
© Copyright 2010 Hitachi Consulting Commercial in Confidence 14
Unclean to clean data(Initial & Delta load) Operational exchanges
Hub / Apps
Application Integration
Architecture
Siebel
EBS
SAP
JDE
Custom
MDM Aware Apps
Hyperion DRM for Customer Hub
Data Governance Manager
MDM Analytics
Customer
Hub 8.2
Oracle Customer Hub (Siebel UCM) 8.2 Best in Class MDM Solution
Oracle Data Quality
Source
Systems
Siebel
EBS
SAP
JDE
Custom
© Copyright 2010 Hitachi Consulting
Key Components of Oracle Customer Hub
© Copyright 2010 Hitachi Consulting
Commercial in Confidence 16
Example of Customer Data Quality Issue A Simple Customer Table Sample
Name Address City State Zip Phone Email
Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 [email protected]
Robert Williams 36 Jones Av. MA 02106 617555000
Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532(9550) [email protected]
Jason Bourne,
Bourne & Cie. 76 East 51st Newton MA 617-536-5480 6175541329
… … … … … … …
Mis-fielded data
Matching Records
Typos Mixed business and
contact names
Multiple Names
Non Standard formats
Missing Data
© Copyright 2010 Hitachi Consulting Commercial in Confidence 17
COMPLETENESS
CONFORMITY
CONSISTENCY
DUPLICATION
INTEGRITY
ACCURACY
Customer Data Problems today
© Copyright 2010 Hitachi Consulting Commercial in Confidence 18
Functionality Feature
Profiling/Pattern Detection
Parsing and Standardization
Address Validation / Cleansing
Matching and Linking
Enrichment
* OEDQ is formerly known as Datanomics Data Quality Application
Understand data status & deduce meaning from unstructured patterns
Create structured records from unstructured data Spot and correct errors; transform to std format
Valid address identification and correction
Spot / eliminate duplicates & identify related entities
Attach additional attributes and categorizations
Examples
Name: LN+FN (CHS, KOR, JPN); FN+MN+ PN+LN (Latin); Tel# is null 30%
Address field -> Address Line 1, City, State,… Nationality: US, USA, American-> USA
809 Newel rd, PALO ALTO 94301 -> 809 Newel Road, Palo Alto, CA 94303-3453
Oracle Offering
Universal DQ Connector + D&B connector + AIA 2.5 PIP for Acxiom
OEDQ Matching Server
OEDQ Cleansing Server
OEDQ Parsing & Standardization Server
OEDQ Profiling Server
Haidong Song = 宋海东 =
Haidong Song: “single, 1 child, Summit Estate, DoNot Mail”
Oracle Enterprise Data Quality Functionality in a Glance
© Copyright 2010 Hitachi Consulting
Data Governance Leadership
Commercial in Confidence 19
© Copyright 2010 Hitachi Consulting
DG is all about establishing the strategies, objectives and policies to effectively manage corporate data by specifying accountability on data and its related processes including decision rights.
For example, DG defines
• Who owns the data;
• Who creates records;
• Who can update them; and also,
• Who arbitrates decisions when data management disagreements arise.
People, processes and technologies are the building blocks for Data Governance
Data Governance ( DG )
© Copyright 2010 Hitachi Consulting
Data Governance Technology Requirements
Define, Communicate & Enforce
Easily Operate hub
Monitor hub operations Fix data issues
Define enterprise master data
Define and view data policies
Data accountability
Escalation process
Administer hub
• Execute day-to-day hub operations (Consolidate, Cleanse, Share & Master)
• Perform data steward tasks, such as merge/unmerge
• Analyze hub DQ metrics
• Track sources of bad data
• Monitor hub transaction load
• Fix import errors and resubmit corrected data
• Proactively watch & repair data
• Tune data quality rules
© Copyright 2010 Hitachi Consulting
Potential Data Governance Leadership Council
22
Data Governance Committee
Client DG Leadership Council
Lead / Business Data Managers
Roles and Responsibilities
Subject Area Business OwnersCustomer/Contact, Booking, Services etc.
IT Domain OwnersClient IT Systems
Data Stewards
· Source Steward
· End User Steward
· Data Hygiene
Steward
Process Stewards
· Sales Process
· Service Process
· Orders/Bookings
· Cancellation
Consumer Base
Executive Layer· Approve Strategy Roadmap
· Align Business and IT Goals
· Align to Client Strategy
· Approve Project Prioritization
· Advocate Compliance
Management Layer· Recommend Strategy and Goals
· Prioritize and Execute Projects
· Define Standards and Policies
· Advocate Compliance
· Act as Subject Matter Experts (SMEs)
Operations/Execution Layer· Stewardship of Data, Data SME
· IT/System/Database Administration (DBAs)
· Interface Daily with Customer Groups
· Ensure Compliance
Business IT Enterprise Wide
IT Architect
· DBA
· ETL Specialist
· Data Modeler
Development
& Maintenance
Manager
· Application Leads
· Technology Leads
· Project Delivery
Technical
Manager
IT Data
Personnel
IT Application
Personnel
· MDM Specialist
· DQM Specialist
· DQ Tools
Specialist
IT Integration
Personnel
Leadership Layer· Sponsorship, Oversight & Approval
Commercial in Confidence
© Copyright 2010 Hitachi Consulting
DG Council Task Force
Commercial in Confidence
23
Leadership Council
• Champions of the DG Council provides the Leadership, Sponsorship and Overall Vision & Direction Serves as the Final Authority on all decisions
• The council would typically consists of a Chief Sponsor ( MDM )and top leadership from Business & IT (for e.g. CIO, VP Operations etc.)
Governance Committee
• Defines business strategies and champions the importance of data governance & data quality domain-specific data, processes, and business rules throughout Client Organization
• Sets priorities for domain-specific data quality improvement projects
• Arbitrates competing interests and makes final decisions regarding issues the Management Layer is unable to resolve
Business Data Managers & IT Administrators
• Responsible for managing specific domain-data sets and is responsible for the data stewardship and quality of that data
• Recommend specific data projects to support better Data Governance and Data Quality efforts
• Responsible for assigning IT resources to support various data projects and initiatives
• Responsible for the upkeep of IT systems and tools to support better Data Management
Data Stewards
• Stewardship of the data for a particular domain (e.g. Customer)
• Perform data cleansing, and other data quality activities for that data domain
• Ensure data standards and compliance
• Perform audits and security checks
• Serve as a liaison between IT & business with regards to data
Process Stewards
• Responsible for entering data for each business process (e.g. Sales , Marketing, Order Entry, Service Request etc.)
• Aid better data quality by supporting data corrections and communication
• Provide inputs to data collection process improvements for the specific process domain
• Serve as SME for specific data sets within the process domain
© Copyright 2010 Hitachi Consulting
Data Governance Program Activities
Commercial in Confidence 24
High-level Activities
1. Establish Data
Governance Leadership
Organization
Define Data Governance
Organization Framework
Detailed tasks
Data Governance Activities
2. Establish Data
Governance Charter &
Vision
Establish Data
Governance Committee
Establish
Leadership Council
Identify DG Council
Champions
Formalize & Kick off Data Governance
Leadership Organization internally
Establish Governance
Charter & Vision
Define Data Governance
Goals & Objectives
Refine Data Governance Charter after
socializing with the LeadershipReview & Refine Data
Governance Goals & Objectives
Define Data Governance
Foundations & Framework
Define & Refine Leadership
Roles & Responsibilities
Nominate Data
Governance Lead
Subject Area Owners & IT Domain Owners
Communicate Charter & Vision to their teams
3. Establish the Data
Governance Framework
Processes
Identify Business Data
Managers for Customer Master
Define Data Governance
Framework Process
Identify IT Management
Resources
Review & Refine Data Governance
Framework Processes
Define Standards,
Policies & Procedures
Establish Data Governance
Compliance & Monitoring Framework
5. Establish the
Stewardship Processes
& Organization
Identify and Align
Process Stewards
Identify IT, Technical
& Project Resources
Identify/Recruit
Data Stewards
Define & Refine Stewardship
Processes including DQ Processes
Formalize the operational Data
Governance Organization
6. Formalize & Kick Off
Customer Master Data
Governance Initiative
Formalize & Kickoff Customer
Data Governance Initiative
Define Stewardship
Roles & Responsibilities
Publish, Communicate and Kick Off Data
Governance Organization across the Enterprise
4. Operationalize
Standards & Policies
Align standards with vision &
strategy; Refine standards;Establish processes to manage
and monitor standards & policiesDefine/Refine additional policies
around audit & security
© Copyright 2010 Hitachi Consulting
Process Definitions and Improvement Activities
Commercial in Confidence 25
High-level Activities
1. Establish Data
Governance Processes
Refer & Align with Data
Governance Roadmap
Detailed tasks
Process Definitions & Improvement Activities
2. Refine Program/
Project Management
Processes
Identify Current Program
management Framework
Identify Current Change
Management Framework
Refine/Redefine Program
Management Framework
Refine/Redefine Change
Management Framework
Establish Change
Control Processes
Identify project Management
processes in place and refine/
adopt to MDM/DG projects
3. Refine Business
Processes to support
MDM/DG Processes
Inventory current Business Processes
with touch point to customer data
Identify process improvements
for each process
Refine/Redefine business process to
align better with future state MDM
Implement Identified
Changes
© Copyright 2010 Hitachi Consulting
Metrics Definition & Monitoring Activities
Commercial in Confidence 26
High-level Activities
1. Establish Governance
Metrics
Detailed tasks
Metrics Definitions & Monitoring Activities
3. Refine System SLAs
and System Metrics
Identify & Define Governance
& Stewardship Metrics
Monitor & Report Governance
& Stewardship Metrics
Operationalize
Governance Metrics
2. Establish Data Quality
Metrics
Identify & Define Data Quality
Metrics for Customer Domain
Monitor & Report Governance
& Stewardship Metrics
Operationalize DQ Metrics for each system
(Oracle CRM on Demand , BRM etc..)
Refine/Define System SLAs
and Metrics
Monitor & Report System
SLAs and Metrics
Operationalize System
SLAs Metrics
© Copyright 2010 Hitachi Consulting
Data Governance – Key Takeaways
Establish Data Governance Leadership Council
Establish Data Governance procedures To ensure data standards and compliance around
Data Consolidation
Data Cleansing
Data Governance
Data Sharing
Data Protection
Data Analysis
Data Decay
Commercial in Confidence
© Copyright 2010 Hitachi Consulting Commercial in Confidence 28
Addition of any global languages needs DGC approval
Rules to curtail data decay need to be formalized .e.g.. All golden records that are not updated for the last 6 months needs revisit from customer calls.
Hierarchy Management of customers needs to be visited occasionally, as new branches can be added to accounts.
Exception management process (DQ Assistant)related functionality needs revision and monitoring from DGC.
Any updates for Transports and Connectors w.r.t. change, upgrade etc needs DGC approval
Any changes to Authorization and Registry services needs approval of DGC
Some Examples of DG Council Action Items
© Copyright 2010 Hitachi Consulting
Customer Hub
Data Stewardship Best Practices
Commercial in Confidence 29
© Copyright 2010 Hitachi Consulting
Data Stewardship with OCH 8.2 v …
© Copyright 2010 Hitachi Consulting
Data Stewardship with OCH 8.2 v
Data Steward performs the following operations on a day to day basis using the Data Stewardship application screens provided with OCH 8.2 o Suspect Match o Merge Request o Incoming Duplicate Overview o Guided Merge & Unmerge o Incomplete Records o Survivorship Rules o Data Decay Management
The idea is to present the features available and supported by Oracle Customer Hub 8.2 v
This is only sample set of functionalities and you may choose to
explore other options and enhancements available with the product
Commercial in Confidence 31
© Copyright 2010 Hitachi Consulting
Merge
There are 3 possible outcomes:
Threshold Type Threshold Score Description
Auto Threshold
(Auto-merge)
>= 90 UCM will automatically merge the two records (except for Sales Records)
Manual Threshold <90 and =>70
UCM will flag the records to have a Data Steward review and determine whether or not to merge
Auto Threshold
(Create New Record)
<70 UCM will create a new record and publish the record to the boundary systems
UC Matching Threshold Scores M Merging Process
Record is sent back to boundary system
UCM process the record based on
the Matching Threshold
UCM calculates Matching
Threshold score based on the
defined attributes
Record is updated based on
Survivorship Rules
Record is sent back to boundary
system
© Copyright 2010 Hitachi Consulting
Merge Criteria used within UCM
Threshold Score: 90% or above - the incoming record will merge with the existing record using the
survivorship rules* Less than 90% greater than 70% - the incoming record will be potentially merged depending
on the Data Steward’s decision
Matching Threshold
Accounts Attributes Survivorship Rules
• Recent – Incoming value will always
survive
• History – Existing value will always
survive
• Source – The value from the
source will survive., External
Systems or Siebel.
>=90%
<90%
>=70%
<70%
UCM Merging Process
• Account Name
• Main Phone
• Address
• City
• State
• Postal Code
If the Matching Threshold score falls within this range, the Survivorship Rules will apply
* Sales Records will never be auto merged
© Copyright 2010 Hitachi Consulting
Data Stewards needs to review the record within the “Incoming Duplicates” screen when a Matching Threshold score is within the range of >= 70 and < 90
Data Stewards will determine if the record needs to be merged with another record
or should be treated as a new record
Matching Threshold
Accounts Attributes
• Account Name
• Main Phone
• Address
• City
• State
• Postal Code
>=90%
<90%
>=70%
<70%
Data Steward
Survivorship Rules
Create New Record
Link and Update
Create New
Create and Merge Accounts
© Copyright 2010 Hitachi Consulting
Manual Link and Update Process
Yes
No
Record
Matches?
End
Data Steward
logs onto
Incoming
Duplicates
Screen in UCM
Data Steward
reviews
incoming record
Data Steward
selects “Link and
Update”
UCM updates
record using
Survivorship
Rules
Data Steward
selects “Create”
UCM updates
record as a new
record
Data Steward
queries for their
record
Create and Merge Accounts
All Data Stewards will see the same records within the “Incoming Duplicates” Screen
Incoming Duplicate Process
© Copyright 2010 Hitachi Consulting
Link and Update a Record After reviewing the record information, the Data Steward can return to
the “Incoming Duplicates” Screen to “Link and Update” or “Create New” When a Data Steward selects “Link & Update”, UCM will update the
record based on the predefined survivorship rules
Link and Update
© Copyright 2010 Hitachi Consulting
Create a New Record After reviewing the record information, the Data Steward can return to the
“Incoming Duplicates” Screen to “Link and Update” or “Create New” If the Data Steward selects “ Create New”, UCM will update the record as a new
record and no survivorship rules are applied
Create New
© Copyright 2010 Hitachi Consulting
UCM Existing Duplicates The “Existing Duplicates” screen is only used when records are loaded into UCM
using a batch process Only potential duplicates will be displayed in the “Existing Duplicates” screen Potential duplicates can be view “Duplicate Contacts” under Administration-
Data Quality and “Existing Duplicates” under Administration – Universal Customer screen.
Create and Merge Accounts
Potential Duplicate Records
Merge Button
Guided Merge and Un Merge Process
© Copyright 2010 Hitachi Consulting
Unmerging Records
The Unmerge Profile Screen is where the account and contact records can be unmerged:
Unmerging Records
Records that were merged within the “existing Duplicate”
screen
Un Merge Button
© Copyright 2010 Hitachi Consulting
Reject Button
Guided Merge Button Merge Button
Merge, Un Merge and Reject Records
© Copyright 2010 Hitachi Consulting
Guided Merge allows end-user to review duplicate records and propose merge by presenting three versions of the duplicate records and allows end user to decide how the record in the UCM should look like after the merge task is approved and committed.
• Victim: the record that will be deleted (from master BC)
• Survivor: the record that will be (from master BC)
• Suggested: output from Surviving Engine (transient to the task)
Guided Merge
© Copyright 2010 Hitachi Consulting
Incomplete Records processing
Data Steward will analyze and re-process the Incomplete data through UCM Batch process.
© Copyright 2010 Hitachi Consulting
43
Survivorship Rules
Survivorship Rules are used to automate the quality of the master customer data.
Once a record is determined to be merged, UCM will compare each attribute within a record and update the record accordingly
Data Steward will change the Survivorship rule weight age depends on source system’s and surviving field in Master record level.
There are three comparison methods used by Survivorship rules: • Recent – Incoming value will always survive
• History – Existing value will always survive
• Source – The value from the source will survive a.k.a., External Systems or Siebel.
UCM Merging Process UCM process the record based on
the Matching Threshold
UCM calculates Matching
Threshold score based on the
defined attributes
Record is updated based on Survivorship
Rules
Record is sent back to
boundary system
Remember that whether a record is auto merged by UCM or manually selected to be merged, the survivorship rules will apply.
UCM Survivorship Rules
© Copyright 2010 Hitachi Consulting 44
Survivorship Rule Example - Source
Name Verizon
Phone Number 4085467880
Fax Number 4086548980
Street Address 5649 Tasman Drive
City San Jose
State CA
Postal Code 93425
Country USA
Name Verizon
Phone Number 4085467880
Fax Number 4086548980
Street Address 5649 Tasman Drive
City San Jose
State CA
Postal Code 93425
Country USA
Name Verizon
Phone Number 5105467880
Fax Number 4086548980
Street Address 5649 Tasman Drive
City San Jose
State CA
Postal Code 93425
Country USA
Best version UCM record
New incoming record from Siebel (primary source) Existing Record within UCM ( from Siebel )
UCM Survivorship Rules
© Copyright 2010 Hitachi Consulting
UCM Survivorship Rules
UCM Survivorship Rule set View
© Copyright 2010 Hitachi Consulting
Enhanced Data Stewardship Capabilities
© Copyright 2010 Hitachi Consulting
48 © Copyright 2009 Hitachi Consulting Commercial in Confidence
For any Questions & Clarifications Twitter : din2win Email : [email protected] [email protected]
Top Related