Reimagine Data Governance: Collaboration, Integration ... · Reimagine Data Governance:...

`

Moscow, 30 May 2018

Reimagine Data Governance: Collaboration, Integration, Automation & MoreGiuseppe Mura

Principal Architect – Data Governance

BIG DATA

MANAGEMENT

New data

consumers

CLOUD DATA

MANAGEMENT

Increase in

data-centric

regulations

DATA

INTEGRATION

Data is a

strategic asset

DATA

QUALITY

Growing volume

and type of data

Trends of DataManagement

3 © Informatica. Proprietary and Confidential.3 © Informatica. Proprietary and Confidential.3 © Informatica. Proprietary and Confidential.

Data Governance Trends

GDPRDigital

ConsumerIFRS9

Digital

TransformationCyber Risk

Changing

Regulatory

Regimes

Common Requirements

Visibility

Cross-functional views

Collaboration between LOBs and IT

Documentation and understanding of

all data in the enterprise

Control

Define where to implement controls

over the data processing pipeline

Have visibility of Control

implementation and their outcomes

Management of Change

Understand the business and

technical impact of change

Minimise analysis cost

Simplify Information Architectures

4 © Informatica. Proprietary and Confidential.4 © Informatica. Proprietary and Confidential.

Real Example Of AConversation With A CEO


Real Example Of AConversation With A CEO

2 simple questions asked:

• “How old is the data?”

• “Where did the data come from?”

2 simple follow up questions:

• Do you trust the numbers?

• What would make you trust them?

Full time500

Part time80

Temporary

120

LoA50


Why do we struggle?Against all investment efforts, we still see silos. What is holding us back?

Organization

• Governance as a

specialist discipline, rather

than a collaborative

efforts, further drives silo

culture!

• Difficulty in showing

benefits of investment –

complex documentation,

difficult to interpret by

business users

Processes

• Focus on governance

process, as opposed to

“enablement” prevents

user engagement

• Roles and responsibilities

– but are people in a

position of actually making

a difference?

Technology

• Not connecting all the

“facets” or dimensions of

governance: metadata,

quality, reporting, change

management

• Not scalable enough:

limited in scope and depth


Factors needed to shift to next-gen Data Governance

CULTURE AND PEOPLE

PROCESS AND POLICIES

GOVERNANCE PLATFORM

TECHNOLOGY COMPONENTS

Break down the organizational silos


Disconnected competency islands

Legal & Compliance

Fiunance

Senior Executives

Information Technology

Chief Data Office

Sales & Marketing

People

ReportsProject

Data

Policy Product

System

Glossary

Process

Regulation

Change Request

Capability


Embrace the Data Governance Community

Data

Steward

How can I

manage metadata

for key enterprise

data assets?

How do I assess

and manage data

quality through the

lifecycle?

Data

Governance Office

How can we

implement data

governance

standards,

facilitate change

programs, monitor

compliance?

How can I ensure

data managed

within application

and supporting

processes deliver

value to the

business?

Data

Owner

How can I

discover,

understand and

trust data required

for my analysis?

Data

Consumer

How can IT

enable business

discover data

assets with

verified data

quality and

traceability?

Data

Architect

TechnicalBusiness


Integrated Enterprise Data Knowledge Base


People contribute their definitions


And collaborate across functions, roles, knowledge domains


Dynamic Viewpoints: Quality, Lineage, Glossary, etc.

Add a methodology and process


High Level Governance Process

Organisation

Business

Definitions

Physical

Controls

Organisation: Operating Model, Roles, Responsibilities, Reporting

Policies and Controls Definitions

Data Discovery, Data Quality Assessment, Controls ImplementationMaster

Party & Ref Data

Data Semantics (Glossaries), Business Context, Lineage, Processes

Data Reality: On Premise, Cloud, Legacy and Next-Gen Architectures


Governance Process Steps

Organisation

Business

Definitions

Physical

Controls

Establish

Operating

Model

Define Roles &

Responsibilities

Glossaries KDEs

Systems and

Business

Linage

Define Policies

(Quality, etc.)

Implement

Controls

Identify

Master & Ref

Data

Create Data

Catalog

Document

Lineage

Manage

Master

& Ref Data

Document

PoliciesMonitor

Outcomes

Document

Business

Processes


Use existing assets

Definitions

Processes

Organization

Systems

Policies

Legacy Documents: Static Visio Diagrams

Manual

KDE

System

Type

Which systems use ‘Days Past Due (DPD)’ ?

RBS – Data Lineage Initial Loading

• Benefit

- Re-use Business Analysis work carried out over the last 3 years

- Provide Business with a level of confidence that Axon can represent the reality they know / understand

- Use API to automate loads

So

urc

e S

yste

mS

ou

rce

Sys

tem

So

urc

e S

yste

m

Da

ta W

are

ho

use

Str

ate

gy

Ma

na

ge

r

So

urc

e S

yste

m

Co

lle

ctio

ns

& R

eco

veri

es

De

bt

Ma

na

ge

me

nt

Co

lle

ctio

ns

& R

eco

veri

es

Str

ate

gy

Ma

na

ge

rD

ata

Wa

reh

ou

seC

red

it S

cori

ng

So

urc

e S

yste

m (

ext

ern

al)

Athena - Toolkits

Ba

ck O

ffic

e

So

urc

e S

yste

m (

ext

ern

al)

Cre

dit

Bu

rea

u (

ext

ern

al)

Outcome 415 | Retail Credit Risk | Mortgages | Full Compliance L1 | Architectural Data Flow Diagram | Page 1

Mortgage Manager (MMR)

Group Mortgage System (GMS)

GMS MIDW

Probe SM

Risk Analysis & ApprovalClient Interaction

Customer Mortgage System (CMS)

IMM connectivity blocked reason

Technical block

Manual block

No current block

Reference data source

Data warehouse

Data manipulation

Administration system

Reporting system/report

Manual process

Legend

System type

Form (paper or electronic)

Unknown/not applicable

KDEs

n KDE created

n KDE only persisted, not created

Click here for full legend.

Mortgage Toolkit

Probe For Collections

Debt Manager

APage 2

BPage 2

Probe EID (Event Identification)

Customer Decision Engine

UKBMIS

Credit Scoring Platform

Transact (Experian)

Re-keyed In

CAUSTIC

To Athena

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

Agent Toolkit

MOPS Toolkit

PLU Toolkit

Flat Files

Customer DB

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

Re-keyed in

APage 2

Charlbury

to Athena

Re-keyed in

41 42 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 4444 4444

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 37 38 39 40

41 41 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 37 38 39 40 41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 37 38 39 40

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 37 38 39 40

4141 414641 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44

41 42 43

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 45 42 43 44

41 42 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 41 42 24 44

41 24 24 44 41 24 24 44

41 24 24 44

41 42 43

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 41 42 47 44

41 24 43

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 45 24 43 44

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 41 24 24 44

41 24 24 44

41 24 24 44

Equifax

41 42

24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34

35

36 37 38 4039

4435 45 3535

41 24 24

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 41 24 24 44

Flat Files

Flat Files

Version 1.4 |29 Mar 2017 |Kasia Malczynska

Write-Off Amount

Individual Country of Residence

General Ledger Cost Centre Code

474645

Loan To Value (LTV) - OriginalLoan To Value (LTV) - IndexedLoan Loss Rate

Days Past Due (DPD)

Exposure at Default (EAD)Drawn Balance

Risk Adjusted Return on Equity (RAROE)

Current ExposureCredit LimitArrears Percentage

Probability of Default (PD)Potential Exposure Loss Given Default (LGD)

10987654321

131211

KDE number to name mapping

Valuation DateValuation AmountRisk Weighted Assets (RWAs)

Divested Property

Repossessed Property

1817161514

Product Type

Credit Responsibility Indicator

Initial Drawdown Date

Recoveries Segment

Channel

Loan Repayment TypeLoan To Income Ratio

Forbearance

Individual Gross Income

Buy To Let Indicator

Agreement Application Date

Credit Grade

2625242322212019

292827

Property Indexed Valuation Amount

Recoveries Classified Flag

3130

Accrued Interest

Expected Loss (EL)

Gross Rental Income

Debtflow

Market Share of Gross New Mortgage Lending

Internal Legal Entity

Region

3635343332

393837

New Build Indicator4140

424344

Loan Impairment Provision – Collectively AssessedLoan Impairment Provision – Latent Loss

Write-Off

Market Share of Mortgage Stock

41 24 43

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 45 24 43 4441 42 43

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

17 18 19 20 21 22 23 24

25 26 27 28 29 30 31 32

33 34 35 36 37 38 39 40

44 42 43 4442

Existing lineage artefacts (Visio)

Visio to

XMLXML to

Axon

RBS-provided XML docs Axon Templates


Next: Self Service

Add the power of AI to scale the effort


Why do we need to add intelligence?

• Volume and complexity of data

• Lack of awareness around metadata

• Lack of data understanding

• Lack of data process understanding

• Manual data processing

• Finding a ‘needle in a haystack’


CLAIRE Powers the Intelligent Data Platform

Driven by:

Metadata AI /

Machine Learning

Intelligent Data

Similarity

Intelligent Data

Tagging

Intelligent Domain

& Entity Discovery

Intelligent Data

Recommendations

Intelligent Anomaly

Detection


At-Scale Profiling, Discovery and ClassificationAI and ML used to auto-assign physical columns to entities


Example: Find Customer data in Files and TablesFind duplicates, understand system dependencies, drive IT simplification


Example: Use of Catalog during Analytics workflowAI Suggestions drive Re-Use and Higher Quality Analytics

Think Platform


Intelligent Data Governance capabilities

Data Processing FactoryData Supply Data Consumers

Technical Controls Design & Implementation

Business Direction, Controls and Visibility

Desig

n a

nd O

rchestra

tion C

ontr

ols

Outc

om

es


Intelligent Data Governance capabilities

Data Processing FactoryData Supply Data Consumers

Technical Controls Design & Implementation

Data Quality

ManagementData Catalog

Security

Discovery

Security

Controls

Testing,

Archiving,

Disposal

Pub/SubBatchStreamingMessage

ProcessingAPIs

Master And

Reference Data

Business Direction, Controls and Visibility

Business

SemanticsQuality

Dashboard

Security

DashboardSystems Lineage

Process & Data

Dependencies

Enterprise Unified Metadata Intelligence

Roles &

Responsibilities

Desig

n a

nd O

rchestra

tion C

ontr

ols

Outc

om

es

Data Platform

ACLOUDREAL TIME/

STREAMINGBIG DATA TRADITIONAL

DATA

INTEGRATION

BIG DATA

MANAGEMENT

MASTER DATA

MANAGEMENT

DATA QUALITY

& GOVERNANCE

DATA

SECURITY

CLOUD DATA

MANAGEMENT

Products

Solutions

MONITOR AND MANAGE

CONNECTIVITY

COMPUTE

CUSTOMER

360

DATA

GOVERNANCE

REFERENCE

360

ENTERPRISE

DATA LAKESECURE@SOURCEPRODUCT

360ENTERPRISE

DATA CATALOG

SUPPLIER

360

ENTERPRISE UNIFIED METADATA

INTELLIGENCE

DATA

CATALOG


In summary


Informatica Intelligent Data Governance

1. Enable collaboration across stakeholders to collect the

knowledge and context already in your business.

2. Realize governance by connecting policy efforts with the technical and

operational efforts.

3. Enable the data governance stakeholders to act with authority and

confidence

to achieve measurable results with searchable and actionable enterprise

views.

4. Empower the data governance stakeholder to always be in control by

monitoring and reporting attainment over time in the right business

context.

5. Democratize data access for all with the right user experiences spanning

non-technical business users to IT.

Moscow, 30 May 2018

Thank you foryour time!

Giuseppe Mura

Reimagine Data Governance: Collaboration, Integration ... · Reimagine Data Governance:...

Documents

Transcript of Reimagine Data Governance: Collaboration, Integration ... · Reimagine Data Governance:...