Data Quality Services+

What’s new in SSISin

SQL Server 2012James Beresford

www.bimonkey.com@BI_Monkey

But first….

Data Quality Services SQL 2012 RC0 Edition

The Data Quality problem

!=

Hands up if you’ve been 2 years old

Your first data quality problem

Data Quality = Shape Sorting

There is Good Data

There is Bad Data

There is Repairable Data

Data Quality Services TermsA Shape = Domain

Data Quality Services TermsCollection of Shapes = Knowledge Base

Data Quality Services TermsShape Sorting = Domain Rule

The Data Quality Client

Allows you to:

• Create or Maintain• Knowledge Bases• Data Quality Projects

• Manage and Configure DQS

The Data Quality ClientCreate a Knowledge Base

The Data Quality ClientCreate a Knowledge Base

What did we see?

• Create a Knowledge Base from Data using Knowledge Discovery

• Set values as Correct, Error or Invalid• Teach the KB some simple auto correction using

Domain Rules

The Data Quality ClientPop Quiz: Who owns a Knowledge Base?

Maintaining KB’s is done by:

a) Qualified Data Professionalsb) A BI Developerc) Usersd) Tony Abbott

The Data Quality ClientPop Quiz: Who owns a Knowledge Base?

Maintaining KB’s is done by:

a) Qualified Data Professionalsb) A BI Developerc) Usersd) Tony Abbott

The Data Quality ClientCreate a Composite Domain

+ n

The Data Quality ClientCreate a Composite Domain

What did we see?

• Use values from one domain to affect another

The Data Quality ClientOther Domain Functions

Things that I won’t demonstrate

• Term Based Relations – autocorrect substrings (e.g. Inc. > Incorporated)

• Reference Data – Validation against external sources on Azure DataMarket, e.g. Melissa Data

The Data Quality ClientCreate a Data Quality Project

The Data Quality ClientCreate a Data Quality Project

What did we see?

• Interactive Cleansing of data• Export of cleansed data set

The Data Quality ClientMatching Policies

Fuzzy & Domain matching

• Matching Polices• Exact – Must match to count in score• Similar – Approximate match counts• Prerequisite – Must match or match fails

• De-Duplication

Automation with SSIS

Automation with SSISThe DQS Cleansing Task

What’s new in SSISDenali CTP3 Edition

What’s new in SSISProjects & Parameters

What’s new in SSISThe SSIS Catalog

What’s new in SSISDeveloper Experience• Visual Studio 2010 & .Net 4 support• Undo / Redo• Auto-save project recovery

What’s new in SSISDeveloper ExperienceNew Toolbox

What’s new in SSISDeveloper ExperienceExpressions

• New Expressions• LEFT• TOKEN• TOKENCOUNT

• Removal of 4000 character limit

What’s new in SSISDeveloper ExperienceNew Data Flow Column Mapper

What’s new in SSISDeveloper ExperienceExpression Markers on icons

What’s new in SSISDeveloper ExperienceVariable Window improvements

1. Package Scope by Default2. Change Scope button

What’s new in SSISDeveloper ExperienceData Quality Services Integration – DQS Cleansing Task

What’s new in SSISAdministration Experience• Simplified deployment & configuration• Automatic logging• Management dashboard• Built-in reports• Improved SSMS management experience• Advanced troubleshooting features• Package performance analysis

What’s new in SSISSQL 2012 RC0 Edition

What’s new in SSISDeveloper Experience

Change Data Capture

What’s new in SSISDeveloper ExperienceODBC Support• Source and Destination Adapters

What’s new in SSISDeveloper ExperienceConnection Manager Changes

What’s new in SSISDeveloper ExperienceFlat File Source Improvements

What’s new in SSISDeveloper Experience• Package Format Changes• Visual Studio Configurations• Scripting Improvements• Troubleshooting & Logging

What’s new in SSISAdministration Experience• Data Taps• Server Management with PowerShell

What’s new in SSISDeveloper ExperienceOther Changes

• Updated look for the Control Flow and Data Flow• Pivot UI• Row Count UI• New Expression:

◦REPLACENULL• BIDS is now SQL Server Data Tools• Many small fixes and improvements based on CTP feedback