Conspect G

roup

Wisdom never fails

06 October 2011

Data Vault Automation

Conspect G

roup

Wisdom never fails

So Conspect?

� Dutch ICT & Consulting organization

� Headquarters @ Almere

� 120 Highly Skilled Professionals

� Conspect is a company that excels in Attitude and Behavior

� Within the ICT domain we provide professional services in Application Integration, Business Intelligence, Custom Development and CRM on Demand

� Conspect = Open Culture - Social Engaged – Sustainability & Corporate Social Responsibility

Conspect G

roup

Wisdom never fails

Why Conspect….

‘Think Big, Start small, Scale fast’

Conspect G

roup

Wisdom never fails

Conspect and BI, our history

2001

Beginning of

Conspect

First Data Vault

2007

2009

Developed the idea of automating

our Data Vaults

Brought BI Team

together with

Conspect

2011

2011

made our ideas real

Conspect G

roup

Wisdom never fails

Who is this guy?

Name: Justin Eelzak

Date of birth: January 13th 1981

Employer: Conspect

Occupation: Business Intelligence Consultant

Active in BI since: 2008

BI Specialty: Reporting and as of today speaking to large groups about Data Vault

Conspect G

roup

Wisdom never fails

Next Generation of Data Warehouses

Conspect G

roup

Wisdom never fails

Conspect G

roup

Wisdom never fails

Conspect G

roup

Wisdom never fails

Conspect G

roup

Wisdom never fails

Conspect G

roup

Wisdom never fails

Conspect G

roup

Wisdom never fails

An Actual Slide Slide

� “I was so hoping it wouldn’t be a guy reading the bullet points from his slides.

� He lasted ten slides before failing…”

Value in automation• Complex• Dynamic• Great deal of initial work

Difficulty of automation• Non standard• Complex

Conspect G

roup

Wisdom never fails

Value in automation

Complex DynamicGreat deal of initial

work

Difficulty of automation

Non standard

Very Complex

DWH to Staging Out

Conspect G

roup

Wisdom never fails

Value in automation

Reasonably simple

Static once in place

Not a lot of initial work

Difficulty of automation

Non standard

Metadata rules are complex

Source to Staging

Conspect G

roup

Wisdom never fails

Staging to Data Vault

Value in automation

Moderate maintenance

costs

Moderate initial

development time

Difficulty of automation

Highly standardized

Simple Metadata

rules

Conspect G

roup

Wisdom never fails

What did we end up doing?

� CDC (hardly a confusing acronym)

� Fully automated the Staging to Data Vault E(T)L

� Fully automated the generation of the Data Vault Database

� Left a human being in charge of the Data Vault design.

Conspect G

roup

Wisdom never fails

100% Reusable ETL

Conspect G

roup

Wisdom never fails

Data Vault

datawarehouse

Generated Data Vault

Conspect G

roup

Wisdom never fails

Custom Component

Conspect G

roup

Wisdom never fails

Results

Current record: 5 hours of ETL in 5 minutes. Build ETL get your

data vault for free

Conspect G

roup

Wisdom never fails

What will we add later?