The ABC of ETL with SSIS
description
Transcript of The ABC of ETL with SSIS
The ABC of ETL with SSIS
James Beresfordwww.bimonkey.com
To cover the basics of ETL using Microsoft BI tools
AIM
Agenda Tools Database Structure SSIS Package Level Components
•Database Engine•Integration Server•SQL AgentSQL Server
•dtexec•dtutil
Command Line
•SSMS: SQL Server Management Studio•BIDS: Business Intelligence Development Studio•SQL ProfilerGUI Tools
SQL Server Tools for ETL
SQL Server•Database Engine
stores data – optional
•Integration Server
executes packages
•SQL Agentschedules packages - optional
GUI Tools – SSMS
GUI Tools – BIDS
Where you will spend 95% of your time
GUI Tools – SQL Profiler
Command Line
dtutil - moves
dtexec - does
Database Structure
Audit and Control
Staging
Persistent Consumed
Control Structures
Audit and Control
Control Mechanisms
Logging
BIDS doesn’t work properly!
Use dtutil batch scripts
Deployment
File Systemvs
Package Storevs
SQL Server
Deployment
Create an Execution Account
Do not use SA account
Execution
SSIS SSIS functional units are Packages…
…then within Packages:
Control Flow
Data Flow
SSIS Packages
• Connection managers • Checkpoints• Event Handlers • Configurations• Variables & Expressions• Logging
Connection Managers
Components which refer to a data source or destination
CheckpointsKey Properties at Package Level:
• CheckpointFileName• CheckpointUsage• SaveCheckpoints
FailPackageOnFailure = True for all restart points
Event Handlers
As packages execute, a series of events occur, e.g: • Validation Completing• Errors• Variable values changing
Configurations
Configurations are External Sources
of
Package Execution Settings
Variables & Expressions
Variables provide changeable values to be used by packages
SSIS has its own expression language
Logging
Logging Providers record execution details
SSIS Packages
Control Flow
&
Data Flow
Control Flow
Mission Control!
Control Flow
Containers
Precedence Constraints
Non Data Tasks
Containers
Provide looping
Support Transactions
Precedence Constraints
Control flow of package
Success / Failure & Expressions
Non Data Tasks
• Execute SQL• Execute Package• FTP• Send Mail• Process SSAS Objects• etc…
Data Flow• Sources and Destinations• SCD• Lookup• Data Conversion• Derived Columns• OLEDB Command• Custom components
Learning Resources
learnintegrationservices.com
http://blogs.conchango.com/jamiethomson/