Information and Integration Management (IIM)Colin Bell <cpbell@uwaterloo.ca> - December 2016

Operational Data Stores

Relational databases created by extracting, transforming, and loading data from operational systems into restructured logical data models.

Dimensional Data Stores

Relational databases created by extracting, transforming, and loading data conformed to a star schema from operational data stores.

Non-Relational Data Stores

Data storage platforms that allow arbitrary data to be stored. Data can usually be paired with metadata.

Graph Data Stores

Data storage platforms to capture sets of nodes and their relationships.

Distributed Data Processing

Data processing platforms allow data processing workloads to be spread between a number of hosts. Allows large scale search and inquiry.

Distributed Transaction Stores

Data storage for transaction records maintained in distributed chains of cryptographically signed blocks.

Data Access, Lineage, and Provenance

Interfaces that allow access to core data elements, meaning of those data elements, and associated lifecycle / origin metadata

Cognitive Processing (AI)

Semantic modelling, natural language, machine learning, and cognitive systems to support data processing.

Dat

a M

anag

emen

t Pla

tform

On-Premise

QUEST

QUEST DB

PeopleSoft CS

Waterloo Works

Works DB

Orbis Comms.

Request Tracker (RT)

RT DB

Best Practical RT

Library

Library DB

Product Names

System Name

Database Name

Product Name

Operational Systems

The systems running on-premise or in the cloud that support University operations.

Cloud

Finance

Aggresso DB

Business World ERP

Learn

Learn DB

Brightspace

Student Email

Office Cloud

Office 365

Research

Pure DB

Pure

System Name

Database Name

Product Name

Info

rmat

ion

Life

cycl

e M

anag

emen

t

Key

Cam

pus

Rel

atio

nshi

ps

Secretariat (SEC)Information and Integration Management (IIM) will support SEC’s University Records Manager and Privacy Officer through metadata management around records, data requests, and data uses on campus.

Institutional Analysis and Planning (IAP)

The Director, Institutional Analysis and Planning, "designs measures and collects data to assess University-wide goals and objectives and coordinates the design, implementation and management of a University-wide decision support data warehouse.”

Information and Integration Management (IIM) will support IAP in this effort as a data broker and technical support partner. IIM will be focused on providing high quality data feeds from operational systems to support IAP’s designed measures on University-wide goals and objectives.

IAP will implement and manage data warehousing for designed measures and official reporting/analysis purposes. Internal and external governance reporting and strategic planning will be serviced from IAP’s warehousing. IIM will support IAP by providing access to knowledge, expertise, tooling, information infrastructure, and captured data elements.

IST-IIM

Data Capture, Cleaning, and Enrichment

Capability to capture, clean, and enrich disparate large volume static or streaming data sources. Make unstructured, semi-structured, and structured data a valuable asset.

De-identification of Personal Information

Capability to de-identify, redact, pseudonymize, and anonymize data. Transform Personally Identifiable Information (PII) so that it can be used in analysis without presenting a privacy risk.

Information Cataloging, Documentation, and Metadata

ManagementCapability to catalog data elements, document them through information modelling, and describe them with metadata.

Institutional Analysis and Planning

(IAP)

Secretariat(SEC)

University Community

MissionThis group exists at the University of Waterloo to provide the knowledge, expertise, and centralized

capability required to make information a valuable asset for the University—to create a trusted information asset base to support the University.

VisionIf useful information exists, make it a competitive advantage for Waterloo; capture, organize, and make

information accessible for future decision making.

Policy 46Principles and Practices

Accountability and Accessibility

Privacy and ConfidentialityCompliance

Information Quality Information Security

Information Lifecycle Management

Capture Organize Store Use Disposition

Digital Imaging &Auto-Uploads

Coding &Indexing

Backups &Versioning

Business ProcessWorkflows

Retention Schedules &Record Destruction

IndexCatalog

SearchDiscover

Access Uses

Privacy Risk

Document Management

Physical Document

Digital Document

Digital Imaging

Captured Document

Document

Metadata Coding and Indexing

Document

Document

Metadata

Business Processing Workflows

Retention / Disposition Schedule

Record of Disposition

Case Delivery

Plan

Case

Case Event Case Activity Case Communication

Case Participant

Case Output

Case Outcome

Case Document

includes

Case Facility

Case Party Relationship

Case Location

Case-Party-Document Model

Content Management

Sharepoint

WCMS

Network Drives

Email Archives

Confluence

Info

rmat

ion

and

Inte

grat

ion

Man

agem

ent

Meaning and Metadata

Physical

Logical

Conceptual

Data Models

Information Models

Semantic Models Ontologies

Glossaries and Taxonomies

Data ElementRules and Definitions

Metadata Management

A robust metadata management capability and platform is crucial to the University’s ability to leverage its information assets.

Capture Organize

Storage

Use Disposition

Integration Platform

Current State Integration Future State Integration

Most systems follow a point-to-point integration pattern.

Data is exchanged through batch copies.

Some exchange is through direct database links increasing complexity of change efforts.

Pattern of operations is effective but inefficient.

Duplication Complexity

Inaccuracy Risk

Dept Sys Dept Sys Dept Sys Dept Sys

Faculty Sys Faculty Sys Faculty Sys Faculty Sys

HR

Student Finance

Research

IDM

ReliabilityScalability

EvolvabilityConsistency

Systems flow requests for information through a common integration platform.

The platform:

- enforces security, - creates audit logs for lineage/ provenance needs, - clarifies interpretation/meaning, and; - captures/creates metadata.

Reduces risks and improves efficiency.

Student

HR

Finance

IAM

Research

Dept Sys

Faculty

Open Data

Enterprise Integration

Plaform

Inno

vatio

n Pl

atfo

rm

User-Centric Design (UCD)

Information Architecture User Experience Design (UX) Platform and Agile

Current State

Future State

X Y

Y

Y

X

Y YY X X

Intrapreneurship

As a user…

http://bit.ly/2h3OFeGInstitute for Government

System Error: Fixing the Flaws in Government IT.

User Developer

Co-op Co-op

CustomWeb Apps

JIRAConfluence

Com

pelli

ng D

ata

Envi

ronm

ent

System Name

Database Name

Product Name

System Name

Database Name

Product Name

System Name

Database Name

Product Name

Operational Data Capture

Real-Time

Robust

Reliable

Operational Storage

Dimensional Storage

Non-Relational Storage

Graph Data Storage

Distributed Processing

Transaction Storage

Access, Lineage, and Provenance

Cognitive (AI) Processing

Identify

De-Identify

Clean

EnrichIndex

StoreAcceptable Uses

ReportUsers

DataScientists

DataAccess

"DIKW Pyramid" by Longlivetheux - Own work. Licensed under CC BY-SA 4.0 via Commonshttps://commons.wikimedia.org/wiki/File:DIKW_Pyramid.svg#/media/File:DIKW_Pyramid.svg

Unstructured Information

“Documents”

Semi-Structured Information“Content”

Structured Information

“Data”

Tableau

Jupyter Notebooks(Python, R, F#)

Sharepoint(MSBI)

Visualization Users

Power BI

Reporting Services

SQL Server

Dapper.NET APIs

Ad-HocInquiryUsers

Excel

Meaning and Metadata

Information and Integration Management

Data Curation

QlikView

Info

rmat

ion

Asse

t Bas

e

Local UnitOperational, Tactical, and

Strategic Documentation

Information System

Architecture Documentation

Internal and External

Governance Reporting

Project / Initative Documentation

Web Content

Service Documentation

Information System Logs

Data Documentation

…

…

http://motivationmodel.com/

Information AssetBase

InformationPortfolio

ProcessPortfolio

ServicePortfolio

HRPortfolio

FinancialPortfolio Strategy

Portfolio

TechnologyPortfolio

Strategic Information

Stream

ApplicationPortfolio

ProjectPortfolio