Collaborative Metadata Management with David Loshin
-
Upload
embarcadero-technologies -
Category
Technology
-
view
124 -
download
1
description
Transcript of Collaborative Metadata Management with David Loshin
![Page 1: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/1.jpg)
Breaking Down the Fence: Using Business-IT Collaboration to Foster
Consistent Enterprise Metadata
David Loshin Knowledge Integrity, Inc.
www.knowledge-integrity.com
1 © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
![Page 2: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/2.jpg)
Islands of Data?
p Different business functions have their own views of necessary data p Requirements are based on each function’s particular needs p Data subsystems represent each function’s particular views
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
2
Finance
Financial Data
Sales
Sales Data
Marketing
Marketing Data
Service
Service Data
H/R
H/R Data
Transactions
Transaction Data
Compliance
Compliance Data
![Page 3: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/3.jpg)
Cross-Function Data Sharing
p Cross functional applications requires data sharing and exchange p Data concepts may exist in different forms in different applications
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
3
Finance
Financial Data
Sales
Sales Data
Marketing
Marketing Data
Service
Service Data
H/R
H/R Data
Transactions
Transaction Data
Compliance
Compliance Data
![Page 4: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/4.jpg)
The Most Dangerous Question in Data Management
p Hundreds have posed it… p Thousands have considered it… p Tens of thousands of hours have been invested in answering… p The most dangerous question in data management:
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
4
![Page 5: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/5.jpg)
Ambiguity in Contextual Definitions
p Example: Uses of the business term “customer”
p These are potentially conflicting definitions p Correspondingly, the representations and underlying meanings may
differ p Identify business terms and provide a means for
n Identifying the authoritative definitions n Resolving similar concepts n Differentiating distinct concepts
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
5
Sales: Someone who pays for our products or services
Support: Someone who has a license for use of our product
![Page 6: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/6.jpg)
The Challenge of Meaning in Different Contexts
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
6
p “Employee hire date”
p Any selected definition changes the meaning of the other uses! p Any use may be dependent on contextual meaning
The date the employer offered the position
The date the prospect accepted the offer
The first date the employee came to work
The date the employee first provided services for pay
The date the rehired employee resumed providing services for pay
![Page 7: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/7.jpg)
Understanding the Data Lifecycle
p Data element use is considered only at design and implementation
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
7
Conception
Definition
Design
Implementation
Deployment
Utilization
Maintenance
Repurposing
Retention
Retirement
p A more mature approach looks at data repurposing
p Ultimately, all lifecycle events have implications to structure and semantics
![Page 8: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/8.jpg)
Evolution of the Business Metadata Glossary
p Many sources of entity concepts and business terms may conflict with each other
p The data steward must facilitate the collection and documentation of business terms
p The data steward must also prepare for harmonization of terms
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
8
Policies
System Docs
Processes
Models
Standards
Applications
Business Rules
Profiling
Etc.
Entity Concepts
Business Terms
Definition Contextual Meaning
… …
Definition Contextual Meaning
Definition Contextual Meaning
Definition Contextual Meaning
![Page 9: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/9.jpg)
Documenting Metadata: Business Term Definitions
p Goal: n Identify key business terms n Find authoritative definitions n Document definition(s) and
use(s)
p Establish a template to capture: n Business term n Type of term (noun, verb,
etc.) n Definition n Authority n Usage
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
9
Business Term ID
Term Type Definition Authority Usage
Biii-HQLA1
High-quality liquid asset
Noun, Entity
“Assets that can be immediately converted into case at little or no loss of value.”
“Basel III: International framework for liquidity risk measurement, standards and monitoring ” http://www.bis.org/publ/bcbs188.pdf
Standard for Liquidity Coverage Ratio (LCR) in “Basel III: International framework for liquidity risk measurement, standards and monitoring ” http://www.bis.org/publ/bcbs188.pdf
![Page 10: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/10.jpg)
Evolving a Community of Interest
p Different stakeholders have a vested interest in variance in n Naming n Definition n Use
of common business terms and entity concepts p Each stakeholder may register
his/her interest in the status of particular metadata objects
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
10 10 10
Definition Contextual Meaning
… …
Definition Contextual Meaning
Definition Contextual Meaning
Definition Contextual Meaning
Business Term
![Page 11: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/11.jpg)
Defining Business Glossary Terms
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
11
Business Term
Use Definition Data Element Concept
Data Element
Data Element
Data Element Concept
Data Element
Data Element
Data Element
Use Definition
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
![Page 12: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/12.jpg)
Identifying Lineage of Common Business Terms
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
12
Business Term
Use Definition Data Element Concept
Data Element
Data Element
Data Element Concept
Data Element
Data Element
Data Element
Use Definition
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Conceptual Domain
Value Domain
Data Element
Type
First VARCHAR(25)
Middle VARCHAR(25)
Last VARCHAR(30)
SocialSec CHAR(9)
Data Element
Type
FirstName VARCHAR(35)
LastName VARCHAR(40)
SSN CHAR(11)
Telephone VARCHAR(20)
![Page 13: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/13.jpg)
Centrally Managing a Business Term Glossary
p Common business terms are likely to have many variant definitions
p All definitions can be captured in a shared, central repository
p Variances can be resolved in two ways: n Consolidation of
meaning n Differentiation into
distinct concepts
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
13
Data Element
Type
First VARCHAR(25)
Middle VARCHAR(25)
Last VARCHAR(30)
SocialSec CHAR(9)
Data Element
Type
FirstName VARCHAR(35)
LastName VARCHAR(40)
SSN CHAR(11)
Telephone VARCHAR(20)
Social Security Number Identifier
Unique number assigned by Social
Security Administration
![Page 14: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/14.jpg)
Centralizing Metadata Across the Organization
p Logging business terms and their definitions in a centralized repository provides a single source of information about the chain of definition
p As data is shared across functions and among business processes, interested parties can review the variety of definitions
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
14
![Page 15: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/15.jpg)
Collaborative Metadata Management
p The repository becomes the “one-stop shop” for information about n Business terms n Definitions n Conceptual domains n Value domains n Data element concepts n Data elements n Entities
p A collaborative platform creates an environment for logging a “vested interest” in the metadata
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
15
Metadata
![Page 16: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/16.jpg)
Harmonization through Collaboration
p The collaborative platform exposes the variances that may impact semantic consistency downstream
p Harmonization (via resolution or differentiation) can be facilitated by the virtual workgroup
p Other potentially interested parties can be invited to join the conversation
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
16
Social Security Number
Identifier Unique number assigned by Social Security Administration
Authentication Last four digits of number
assigned by the Social Security Administration
![Page 17: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/17.jpg)
Governance through Collaboration
p A collaborative platform allows interested individuals to track those business terms, data element concepts, and reference metadata relevant to their business processes
p A comprehensive directory provides visibility to all related concepts
p Data policy changes can be governed through the collaborative platform
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
17
Social Security Number
Identifier Unique number assigned by Social Security Administration
Authentication Last four digits of number
assigned by the Social Security Administration
Identifier Unique number assigned by the company Customer ID
![Page 18: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/18.jpg)
Encouraging a Culture of Semantic Harmony
p Small variance in definitions in isolated functions become magnified when data is shared across functions
p Establish a level playing ground by n Instituting a common business term glossary n Harmonizing business term definitions n Unifying shared reference data into conceptual domains and
corresponding value domains n Socializing use of shared metadata n Establishing standards for future development n Integrate methods for monitoring compliance with standards
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
18
![Page 19: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/19.jpg)
Collaborative Metadata: Considerations
p Process n Are there defined practices, disciplines, and training to ensure socialization
and integration? p Governance
n Does the metadata platform incorporate compliance with defined data policies for oversight of shared metadata?
p Comprehensiveness n Can the shared platform capture all aspects of lineage, relationships,
business glossary, definitions, etc.? p Ease of Use
n The system must be easy to use by consumers who will range from being technically proficient IT staff to informed business users
p Maintenance n Is the organization willing to support the ongoing investment in
maintaining the collaborative resource?
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
19
![Page 20: Collaborative Metadata Management with David Loshin](https://reader037.fdocuments.in/reader037/viewer/2022110302/549a459db4795996658b471c/html5/thumbnails/20.jpg)
Check Out These Resources!
p www.knowledge-integrity.com p www.dataqualitybook.com p If you have questions,
comments, or suggestions, please contact me David Loshin 301-754-6350 [email protected]
© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com
(301) 754-6350
20