Data Quality Services+
What’s new in SSISin
SQL Server 2012James Beresford
www.bimonkey.com@BI_Monkey
But first….
Data Quality Services SQL 2012 RC0 Edition
The Data Quality problem
!=
Hands up if you’ve been 2 years old
Your first data quality problem
Data Quality = Shape Sorting
There is Good Data
There is Bad Data
There is Repairable Data
Data Quality Services TermsA Shape = Domain
Data Quality Services TermsCollection of Shapes = Knowledge Base
Data Quality Services TermsShape Sorting = Domain Rule
The Data Quality Client
Allows you to:
• Create or Maintain• Knowledge Bases• Data Quality Projects
• Manage and Configure DQS
The Data Quality ClientCreate a Knowledge Base
The Data Quality ClientCreate a Knowledge Base
What did we see?
• Create a Knowledge Base from Data using Knowledge Discovery
• Set values as Correct, Error or Invalid• Teach the KB some simple auto correction using
Domain Rules
The Data Quality ClientPop Quiz: Who owns a Knowledge Base?
Maintaining KB’s is done by:
a) Qualified Data Professionalsb) A BI Developerc) Usersd) Tony Abbott
The Data Quality ClientPop Quiz: Who owns a Knowledge Base?
Maintaining KB’s is done by:
a) Qualified Data Professionalsb) A BI Developerc) Usersd) Tony Abbott
The Data Quality ClientCreate a Composite Domain
+ n
The Data Quality ClientCreate a Composite Domain
What did we see?
• Use values from one domain to affect another
The Data Quality ClientOther Domain Functions
Things that I won’t demonstrate
• Term Based Relations – autocorrect substrings (e.g. Inc. > Incorporated)
• Reference Data – Validation against external sources on Azure DataMarket, e.g. Melissa Data
The Data Quality ClientCreate a Data Quality Project
The Data Quality ClientCreate a Data Quality Project
What did we see?
• Interactive Cleansing of data• Export of cleansed data set
The Data Quality ClientMatching Policies
Fuzzy & Domain matching
• Matching Polices• Exact – Must match to count in score• Similar – Approximate match counts• Prerequisite – Must match or match fails
• De-Duplication
Automation with SSIS
Automation with SSISThe DQS Cleansing Task
What’s new in SSISDenali CTP3 Edition
What’s new in SSISProjects & Parameters
What’s new in SSISThe SSIS Catalog
What’s new in SSISDeveloper Experience• Visual Studio 2010 & .Net 4 support• Undo / Redo• Auto-save project recovery
What’s new in SSISDeveloper ExperienceNew Toolbox
What’s new in SSISDeveloper ExperienceExpressions
• New Expressions• LEFT• TOKEN• TOKENCOUNT
• Removal of 4000 character limit
What’s new in SSISDeveloper ExperienceNew Data Flow Column Mapper
What’s new in SSISDeveloper ExperienceExpression Markers on icons
What’s new in SSISDeveloper ExperienceVariable Window improvements
1. Package Scope by Default2. Change Scope button
What’s new in SSISDeveloper ExperienceData Quality Services Integration – DQS Cleansing Task
What’s new in SSISAdministration Experience• Simplified deployment & configuration• Automatic logging• Management dashboard• Built-in reports• Improved SSMS management experience• Advanced troubleshooting features• Package performance analysis
What’s new in SSISSQL 2012 RC0 Edition
What’s new in SSISDeveloper Experience
Change Data Capture
What’s new in SSISDeveloper ExperienceODBC Support• Source and Destination Adapters
What’s new in SSISDeveloper ExperienceConnection Manager Changes
What’s new in SSISDeveloper ExperienceFlat File Source Improvements
What’s new in SSISDeveloper Experience• Package Format Changes• Visual Studio Configurations• Scripting Improvements• Troubleshooting & Logging
What’s new in SSISAdministration Experience• Data Taps• Server Management with PowerShell
What’s new in SSISDeveloper ExperienceOther Changes
• Updated look for the Control Flow and Data Flow• Pivot UI• Row Count UI• New Expression:
◦REPLACENULL• BIDS is now SQL Server Data Tools• Many small fixes and improvements based on CTP feedback
Top Related