Master Data ManagementFrom Assessment, Design up to operation
Ali BELCAID – Managing Consultant
Master Data Management : An OverviewInformation is a Priority
Quality and actionable information is fundamental to deliver many business strategies.
MDM
Enterprise OperationsManagement & Capabilities
Enterprise InformationManagement & Capabilities
Solutions• ERP• CRM• Supply Chain
Solutions• BI/DW• BPM• Portals
MDM is the glue that blends operational and informationmanagement solutions
Analytical MDMOperational MDM
Master Data Management : An OverviewMDM Requires Both IT and Business
MDM is a component that promotes process efficiency, simplicity, and data quality, improving the value IT brings to business.
Global Master Data Management
Business IT
EnterpriseWideConsistency
CostEffectiveness
ReliableAnalytics andReporting
Centralized, Efficient Data Storage
Improved System Integration
Minimal Data Conversions
• Avoid data redundancies• Assure data consistency
• Centralize data distribution (one source)
• Provide unique identifier• Create global hierarchies
and attributes
Impact of MDM Initiative on Business and IT
Master Data Management : An OverviewMDM Implementation Styles
Implementation Style Description
External Databases (Service Provider)
Third party suppliers and managers of domain specific master data
Examples: database marketing, government service bureaus
Persistent(Database)
Master information file/database, system of record (SOR) Operational data store, active data warehouse Relational DBMS + extract-transform-load (ETL) + data
quality (DQ)
Registry(Virtual)
Metadata layer + distributed query (e.g., EII) Enterprise application integration (e.g., EAI), distributed
system Portal
Composite(Hybrid)
Ability to fine-tune performance and availability by altering amount of master data persisted
XML, web services, service-oriented architecture (SOA)
Master Data Management An Overview “Persistent” Master Data Repository
(Illustrative Scenario)
This Customer Data Integration (CDI) solution architecture illustrates how process and technology work together through a centralized “persistent” master data repository
Customer Master Data RepositoryBusiness AnalystsData Stewards
Workflow Business Rules Mapping Rules
Automated Entry Updates
SAP
Siebel
IMSCustomer
PBMS
Extr
act T
rans
form
Loa
d(E
TL)
Ente
rpris
e Ap
plic
ation
Inte
grati
on(E
AI)
OperationalData Store(ODS)
DIM
DIM
DIM
DIM
DIM
DIM
FACT
Data Mart
Data Mart
Data Mart
CareReporting
CustomerReporting
FinancialReporting
Customer Care
CRM
CampaignManagement
ContractNegotiations
FinancialConsolidation
Monthly EndClose
Enterprise Warehouse Data Marts
Aggregate
DATA INFORMATION
Map
ping
Up
date
System Owners
Initiate Entry/Update
Initiate Entry/Update
Operational Systems Master Data Management Business Process
EvaluateRequest
ApproveRequest
Catalogue/Index
Approach to MDM ImplementationBusiness Assessment & Technology Selection
Quick scan
Current State Details
Requirements
Gap Analysis
Organization & Governance
Process & Methodology
Technology Selection
Implementation Roadmap
Current State Future State Develop Roadmap
Deliverables
Project Initiation Current State : Data Mgt. Current Sate : Organization Current State : Architecture
Gap Analysis Future State : Data Mgt. Future Sate : Organization Future State : Architecture
Prioritization Roadmap
4 to 6 weeks varies with scope
Activities
Approach to MDM ImplementationBusiness Assessment & Technology Selection
Topic What to Do ? What to Deliver ?
Quick Scan
• Maturity Assessment• Business Direction, objectives, …• Engagement Management (project
management, change management, quality management and risk management )
• Scope of the Project : Which MDM should be implemented ?
Business Requirement Analysis • workshops with key client stakeholders to
identify business issues with master data (Data quality, Duplicity, Incoherence, …)
• MDM Finding & AssessmentTechnical Requirement Analysis
• workshops with key client stakeholders to identify technology issues that could delay the delivery of accurate and reliable master data to consumers (multi-systems, duplicity, non-synchronization, …)
Gap Analysis • based on business and technical findings and requirements
Future State Recommendations • that consists of business, technology and data architecture
Roadmap Definition • for attaining the future state • Roadmap Definition & Planning
Approach to MDM ImplementationBusiness Assessment & Technology Selection
Maturity Assessment
Approach to MDM ImplementationMDM Implementation Framework
Business Requirement
Technology Assessment & Software Selection
Design
Build
Integrate
Operate
Roadmap & Foundation Activities
Iteration 1
Iteratio
n 2
Iteratio
n n
Continues Implementation Phases
Begin next Iteration
This part is done once(Part of the Assessment Phase)
Define & Validate the data governance & operating model Implement the data governance & Operating model
Kick
Off
of th
e M
DM
Initi
ative
Approach to MDM ImplementationRoadmap & Foundation Activities
The Roadmap provides the detailed requirements and solution definition that applies to the continuous implementation. It has the following objectives:
Refine strategic business requirements to a detailed level for iterative design
Establish standards and develop solutions to common problems Define the development and delivery environments Detailed planning for this cycle of the implementation
The Roadmap can be summarized as providing the Plan, the Solution Requirements and the Solution Definition for the continuous implementation part.
Foundation Activities focus on aspects of each of the streams of development. These activities are :
Meta Data Management Data Modelling Data Migration Data Integration Data Reengineering Data Profiling Data Solution Architecture
Master Data Modelling
Master Data Migration
Master Data Integration
Master Data Re-engineering
Master Data Profiling
Meta Data Management
Master Data Architecture
Major deliverables and points to be addressed when setting up the roadmap & foundation activities :
Detailed Project Roadmap Testing and Deployment plans Detailed Information Modeling Detailed Migration Plan (historical Data) Recommended process and system changes for improved
Data Governance Identification of root causes leading to Data Governance
issues Data Governance Metrics Quantitative Data Investigation Improved Data Quality Create/Revise Solution Architecture Ensure the availability of Software Development
Environment
Approach to MDM ImplementationMDM Work streams
MDM Program Management
Change/Issue Management
Operations Management
Training and Support
Meta Data Management
Master Data Modelling
Master Data Migration
Master Data Integration
Master Data Re-engineering
Master Data Profiling
Master Data Architecture
OperateIntegrateBuildDesign
Iteration
Approach to MDM ImplementationMeta Data Management
Significant metadata artifacts are produced related to data definition, business rules, transformation logic and data quality. This information should be stored in a metadata repository; getting this repository in place from the early stages of the MDM project.
Model Management is the capability to manage structures and processes used to describe the metadata in a system.
Metadata Integration capability provides a basic ability to build metadata flows into and out of a managed metadata environment.
Identity Matching as a foundation capability ensures consistent and accurate reuse of metadata. a system must have the ability to identify metadata uniquely so that the metadata may be reused, validated and versioned within the managed metadata environment.
Validation capabilities ensure the quality and consistency of metadata flowing through the managed metadata environment
Versioning of metadata provides the ability for looking back into history to gain a more comprehensive understanding of the current state
Configuration Management is a fundamental process for developing metadata. It is the role that process and governance plays in the development and operations of a managed metadata environment.
Model Query provides the fundamental ability for publication of metadata. Its capabilities form the foundation of providing Metadata Reporting Packages
Metadata Access Control is a capability for providing a control layer over metadata models. Metadata can often be sensitive information that should have restrictive controls to prevent unauthorized access
Approach to MDM ImplementationMaster Data Modelling
The data modeling process is used as an intermediary data store to bring data together from multiple systems in a hub fashion. This data store provides a common, integrated model where data may undergo significant re-engineering.
Implement Physical Master Data Model
Design Logical Master Data Model
Input: Conceptual Data Model Data Specification Standards Data Modeling Standards Data Security Standards Detailed Business Requirements for
each Iteration
Output: Logical Data Model
Input: Logical Data Model Solution Architecture Data Specification Standards Data Modeling Standards Data Security Standards Detailed Business Requirements for each iteration
Output: Physical Data Model Database Definition Language (DDL) Scripts Sizing Estimates
Approach to MDM ImplementationMaster Data Integration
Input: Business requirements Designed Process Flow Source & Target interfaces load dependencies and integration with
metadata processes Source & Target Data models
Output: ETL Logical Design
Input: ETL Logical Design Solution Architecture Data Specification Standards Data Modeling Standards Data Security Standards
Output: ETL flows and Jobs
ETL flows &jobs Testing
ETL Physical Design
ETL Logical Design
Input: Test scenarios Data Sampling
Output: Tested flows and jobs
Dependencies: Metadata Management Data Profiling Data Re-engineering Data Modeling Data Migration
Data Integration is one of the Foundation Capabilities of MDM Development. It provides a mechanism for bringing together information from a number of distributed systems by interfacing into sources, providing a capability to transform data between the systems, enforcing business rules and being able to load data into a different types of target areas.
Approach to MDM ImplementationMaster Data Re-engineering
Data Re-Engineering is a term used to describe a number of related functions for standardizing data to a common format, correcting data quality issues, removing duplicate information/building linkages between records that did not exist previously, or enriching data with supplementary information.
Data standardization brings data into a common format for migrating into target environment. It addresses problems related to:
Redundant domain values Formatting problems Non-atomic data from complex fields Embedded meaning in data
Data Correction typically addresses problems related to:
Missing data Value issues due to range problems Value issues related non-unique fields Temporal or state issues Name and address data that can be referenced against
existing reference sets
In the Data Matching and Consolidation task, data is associated with other records to identify matching sets. Matching records can then either be consolidated to remove duplications or linked to another to form new associations.
Data Enrichment provide an organisation’s internal data with data from external sources like :
Personal data such as date-of-birth and gender codes Geographical data Postal Data, such as Delivery Point Identifiers (DPID) Demographic information Economic data World event information
Approach to MDM ImplementationMaster Data Profiling
Data Profiling focuses on conducting an assessment of actual data and data structures. It helps provide the following:
Identifies data quality issues - measurements are taken against a number of dimensions, to help identify issues at the individual attribute level, at the table-level and between tables.
Captures metadata. Identifies business rules – The next step is to perform the data mapping. Data profiling will assist in gaining an understanding of
the data held in the system and in identifying business rules for handling the data. This will feed into the future data mapping exercise.
Assesses the source system data to satisfy the business requirements. The focus is on gaining a very detailed understanding of the source data that will feed the MDM target system, to ensure that the quality level is sufficient to meet the requirements.
Major Deliverables
Data Quality Assessment Report (per Source System) Data Quality Metrics updated to Metadata Repository Mapping Rules and Business Rules updated to Metadata Repository
Finalize Data Quality Report(Signoff of Data Quality Report)
Perform Multi Tables Profiling
(Analyze redundancy and referential integrity issues)
Perform Table Profiling
(Analyze Data across rows in
single table)
Perform Column Profiling
(Analysis of single or complex field)
Approach to MDM ImplementationMaster Data Profiling
1.Column Profiling
2.Table Profiling
3.Multi-Table Profiling
4. Quality Report
Input: Completion of Table Profiling Information Requirements for multi-table level data
analysis Relevant data extracts Output: Completion of Multi-Table Profiling Redundancy Analysis will identify:
Potential relationships with fields in other tables Redundant data between tables Potential referential integrity issues eg.
Identification of orphans records Completion of the relevant sections of the Data Quality
Assessment Report Updates to metadata repository
Input:
Information Requirements for column-level data analysis
Relevant data extracts
Output: Completion of Column Profiling Understanding all the fields and document their
descriptions in the profiling tool Completion of the relevant sections of the Data Quality
Assessment Report Updates to metadata repository
Input:
Completion of Column Profiling Information Requirements for table-level data analysis Relevant data extracts
Output:
Completion of Table Profiling Understand all the fields and document their
descriptions in the profiling tool Primary keys for each table Completion of the relevant sections of the Data Quality
Assessment Report Updates to metadata repository
Input: Completion of Column Profiling Completion of Table Profiling Completion of Multi-Table Profiling
Output: Completion of the Data Quality Assessment Report
Data Producers (ERP, CRM, Legacy, …)
Approach to MDM ImplementationMaster Data Migration
An MDM program will typically involve a migration of historical data across systems, into or through a centralized hub. This is where many of the data quality issues are resolved in a progressive fashion before operationalizing some of these rule-sets for the ongoing implementation.
Migration Staging
• Attribute Scan• Tables Scan• Assessment• Reporting
Transformations
Dat
a In
tegr
ation
Prod Target
Test Target
Data Profiling Data Integration
Data Re-engineering
Metadata Management
Integrated Data Store
• Common Data Model• Detailed Data• Apply Re-engineering rules
Master Data Modelling
1
2
3
4
5
6
7
Approach to MDM ImplementationMaster Data Migration
The key activities in the MDM migration process include:
1. Extraction of data from producers (ERP, CRM, Legacy systems, …) into a staging area.
2. The data in the staging area will be profiled to measure down columns, across rows and between tables. This information will be used to determine which business rules and transformations need to be invoked early in the process.
3. Metadata such as data mapping rules will begin to be established at this time. Data Standards will be agreed to and invoked at this stage in preparation for data movement. All source attributes will be mapped into the target attributes within the metadata management environment.
4. All agreed to transformations and standardizations required to move the data into the staging area for testing and production are implemented. The data is moved into the Integrated Data Store.
5. Data Profiling is done again and measured against the agreed upon move success criteria for all steps up to this point. Additional data standardizations are performed in to assist in the data matching and generally measure data quality against agreed upon criteria. After the standardizations the rules for which records can not or should not be moved are applied. It expected that this step will require considerable analysis.
6. This step involves the actual move of the data into either the testing environment
7. Data is loaded into the production system where some further data quality cleanup may be required. Production Verification Testing is conducted, which should also include functional testing of features that are environment specific. After testing is complete, the system is activated as a live production system.
Approach to MDM ImplementationMaster Data Architecture
The Master Data Architecture defines in detail the Solution Architecture for the MDM environment. The Solution Architecture provides the overall technology solution for a specific increment and ties together the overall approach.
Define Data Quality Processes- Data model-Profiling- Re-engineering
Deliverables:• Data Quality Design architecture• Data Quality Implementation
Software Documents• Data Quality Technical architecture
document
Define ETL conceptual Design- List of sources- List of targets- Major Transformations- Estimate volumes- Timing
Deliverables:• ETL Design architecture• ETL Implementation Software
Documents• ETL Technical architecture document
Define Metadata Management conceptual Design- Business definition of the data - Physical data models - Data Re-Engineering metadata -Data Quality metadata formulas used to derive data
Deliverables:• Metadata Design architecture• Metadata Implementation Software
Documents• Metadata Technical architecture
document
Define Security conceptual Design- Security Standards-Security Requirements
Deliverables:• Security Implementation Document
Define Infrastructure Management conceptual Design-Backup & Recovery-Archiving-Controlling & Monitoring- Environments (dev, test, prod) setup
Deliverables:• Configuration Management Document
Define SDLC conceptual Design- Testing Strategy - SDLC Procedures - Testing Plans for Applications & Infrastructure-Deployment Plan
Deliverables:• SDLC procedures document• Testing Plan• Deployment Plan
Master Data Solution Architecture
MDM Software Implementation -Software Implementation Planning- Parameterization/Configuration- Software Testing and deployment
Deliverables:• Software Installation and Configuration
Document
Approach to MDM ImplementationPrototyping the Architecture
Prototyping the architecture helps to :
• test some of the major technology risk areas for the proposed MDM Solution Architecture• gain a better understanding of how the solution will work before moving into a more formalized
design process. • Prototyping the proposed solution should provide an end-to-end approach that includes each of
the major components of the architecture.
Key Lessons
Joint business and IT team
Make the case for change
Data as a common good
Think big but start small
Measure and communicate success
Processes first, technology last
Business ownership of data
Roles and responsibilities
Data cleanliness and migration
Communicate, communicate, communicate !
Approach to MDM ImplementationMDM - Key Lessons Learned
In an MDM implementation, there are some key lessons learned that should be considered when initiating an MDM program.
Knowledge, is quite simply question of sharing.
http://intelligenteenterprise.blogspot.com/http://www.linkedin.com/in/albel
Top Related