Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such...
Transcript of Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such...
![Page 1: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/1.jpg)
Azure Data Factory
vs SSISREZA RAD
![Page 2: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/2.jpg)
About Reza Rad
DW/BI Consultant, Mentor, Trainer
SQL Server MVP
Author of SQL Server and BI books
Author of MSBI webcast series on RADACAD
Microsoft Certified Trainer
Microsoft Certified Professional
Co-Leader of NZ BI User Group
![Page 3: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/3.jpg)
Agenda
Azure Data Factory Overview
SSIS Overview
Comparison
![Page 4: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/4.jpg)
Why This Session?
![Page 5: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/5.jpg)
I was working with SSIS
![Page 6: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/6.jpg)
I’ve heard about Azure Data
Factory
![Page 7: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/7.jpg)
Panic!
![Page 8: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/8.jpg)
Give it a go
![Page 9: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/9.jpg)
Found it as a Great tool
![Page 10: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/10.jpg)
Compare
![Page 11: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/11.jpg)
![Page 12: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/12.jpg)
Azure Data Factory
Fully Managed Service for Composing Data
Storages, Processing, and Movement Services into
Streamlined, Scalable, and Reliable Data
Production Pipelines.
![Page 13: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/13.jpg)
What You can do with Azure Data
Factory
Access to data sources
such as SQL Server On premises, SQL Azure, and Azure Blob storage
Data transformation
through Hive, Pig, Stored Procedure, and C#.
Monitoring
the pipeline of data, validation and execution of scheduled jobs
Load it into desired Destinations
such as SQL Server On premises, SQL Azure, and Azure Blob storage
And last but not least; This is Cloud based service.
![Page 14: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/14.jpg)
Azure Data Factory Architecture
![Page 15: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/15.jpg)
Administration Panel
![Page 16: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/16.jpg)
ADF Editor
Write JSON
Edit/Create/Delete Objects
Deploy
![Page 17: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/17.jpg)
Main Benefits
Pay as Usage
Cloud Based
HDInsight compatibility
Built for Massive Data Movement
![Page 18: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/18.jpg)
Demo: Azure Data Factory
![Page 19: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/19.jpg)
Evolving Approaches to Analytics
ETL Tool(SSIS, etc)
EDW(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed Data
Transform
BI Tools
Data Marts
Data Lake(s)
Dashboards
Apps
![Page 20: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/20.jpg)
ETL Tool(SSIS, etc)
EDW(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed Data
Transform
BI Tools
Ingest (EL)
Original Data
Data Marts
Data Lake(s)
Dashboards
Apps
Evolving Approaches to Analytics
![Page 21: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/21.jpg)
ETL Tool(SSIS, etc)
EDW(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed Data
Transform
BI Tools
Ingest (EL)Original Data
Scale-out Storage & Compute
(HDFS, Blob Storage, etc)
Transform & Load
Data Marts
Data Lake(s)
Dashboards
Apps
Streaming data
Evolving Approaches to Analytics
![Page 22: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/22.jpg)
![Page 23: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/23.jpg)
SSIS
Data Transformation and Consolidation Tool
Part of SQL Server Installation
Next generation of legacy DTS technology
Introduced in 2005, Mature, but still new features coming
![Page 24: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/24.jpg)
What you can do with SSIS
Defining Flow of Activities
Get data from many types of data sources
Transform data
Load data into many types of destinations
![Page 25: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/25.jpg)
Main Benefits
Rich Development Tool: SSDT
Many Built-in Transformations
![Page 26: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/26.jpg)
![Page 27: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/27.jpg)
Development Tool Administration Tool User TypeData
Sources/DestinationsData Transformations
Extension & Automation
Pricing Error Handling Deployment Monitoring
Security TechnologyEnvironment
RequirementsHDInsight
CompatibilityAzure Analytics
Integration
Comparison Factors
![Page 28: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/28.jpg)
Development Tools: SSIS
SSDT / BIDS
Rich and Powerful
Available Free
![Page 29: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/29.jpg)
Development Tools: ADF
Azure Portal
New ADF Editor
Azure Power Shell
JSON scripts
![Page 30: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/30.jpg)
Visual Studio Templates for ADF
Awesome!
But way far from Rich
Editor
No GUI
No Intellisense
No Debugging
….
![Page 31: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/31.jpg)
Copy Data - Preview
Wizard for Copy Data
Still Preview
Not for Edit, only for Create
Good Start
![Page 32: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/32.jpg)
Comparison: Development Tool
SSIS ADF
Standalone Tool Yes Yes
Powerful GUI Yes No
Availability Free Free
Prerequisite SQL Server License Azure Subscription
![Page 33: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/33.jpg)
Administration Tool: SSIS
SSMS
Available Free (New Versions)
![Page 34: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/34.jpg)
Administration Tool: ADF
Azure Portal
Available under Azure subscription
Azure Power Shell; Scheduling Pipelines…
Azure Data Factory Cmdlets
![Page 35: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/35.jpg)
The New Monitoring App
Data Factory Monitoring App released 10 days ago!
![Page 36: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/36.jpg)
Comparison: Administration Tool
SSIS ADF
Standalone Tool Yes Yes
Powerful GUI Yes Yes, few with Power Shell
Availability Free Under subscription
Prerequisite SQL Server License Azure Subscription
![Page 37: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/37.jpg)
User Type
Both SSIS and ADF are Developer Tools and Services, Not for Power User
![Page 38: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/38.jpg)
Data Src/Dest: SSIS
Wide RangeSQL Server
Oracle
SAP
Salesforce
Access
Sybase
Informix
Teradata
MySQL
PostgreSQL
FoxPro
CSVs
Web Services
Many other Data Sources and Destinations types
![Page 39: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/39.jpg)
Data Src/Dest: ADF
Data Store Types Supported so far:
+ Azure SQL Data Warehouse
+ Azure Data Lake Store
+Web
+HDFS
+ODBC
+OData
![Page 40: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/40.jpg)
Comparison: Data Src/Dest
SSIS ADF
SQL Server On Premises Yes Yes
SQL Azure Yes Yes
Azure Storage Yes Yes
Oracle Yes Yes
SAP Yes
MySQL Yes Yes
DB2 Yes Yes
CSVs Yes Yes
Web Services Yes
Many other Data src/dest Yes Fewer
Azure Data Lake Store Yes
![Page 41: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/41.jpg)
Data Transformations: SSIS
Wide range of Built-in Transformation
Ability to extend more with Script Component
Merge Join
Aggregate
Union
Lookup
Fuzzy Lookup/Grouping
Conditional Split
Derived Column
OLE DB Command
Many other Data Transformations
![Page 42: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/42.jpg)
Data Transformations: ADF
Copy Activity
Stored Procedure (SPROC) Activity
Hive and Pig Activity
Support wide range of transformations through HDInsight
Custom C# Activity
Write your own transformation with C#
Azure ML Batch Scoring Activity
Azure Data Lake Analytics U-SQL Activity
![Page 43: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/43.jpg)
Comparison: Data Transformation
More Built-in Transformation will come for ADF
SSIS ADF
Copy Yes Yes
C# Custom Transformations Yes Yes
Pig and Hive With Scripting Yes
Azure ML Batch Scoring With Scripting Yes
Stored Procedure Yes Yes
Built-in Data Transformations Yes
![Page 44: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/44.jpg)
Extension & Automation: SSIS
Programing SDK
BIML; Automation
Third-party tasks, and components: Pragmatic Works, CozyRoc….
![Page 45: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/45.jpg)
Extension & Automation: ADF
Power Shell
No Programing SDK
No third-party components YET!
![Page 46: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/46.jpg)
Comparison: Extension &
AutomationSSIS ADF
Programing SDK Yes No
Automation Yes, BIML Yes, Power Shell
Third-Party components Yes No
![Page 47: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/47.jpg)
Pricing: SSIS
Basic SSIS for Free (Import and Export Wizard)
Express Edition
Standard SSIS features
Standard and BI Edition
Advanced SSIS Features
Such as CDC and advanced adapters
Enterprise Edition
![Page 48: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/48.jpg)
Pricing: ADF
Pay for Usage
![Page 49: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/49.jpg)
Comparison: Pricing
SSIS ADF
Licensing Yes
Pay for Features Yes
Pay for Usage Yes
![Page 50: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/50.jpg)
Environment: SSIS
You need usually a physical good spec Server
Example SSIS practice for 1TB data movement in 30 Minutes!
Hardware Requirements
Software Requirements
Administration Efforts
![Page 51: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/51.jpg)
Environment: ADF
Azure will take care of Environment
Cloud based benefits
HDInsight for ADF is also supported
No Hardware, Software Administration efforts
![Page 52: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/52.jpg)
Comparison: Environment
SSIS ADF
Hardware Setup Yes
Software Setup Yes
Administration Costs Yes
Azure Environment Usage Yes
![Page 53: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/53.jpg)
Error Handling: SSIS
Error Handling through Event Handlers (OnError)
Error Output in Data Flow
Failure Precedence Constraint
![Page 54: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/54.jpg)
Error Handling: ADF
Error Message Logging
Alert Rules
No Error Handling Events
![Page 55: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/55.jpg)
Comparison: Error Handling
SSIS ADF
Alerts Yes Yes
Error Logging Yes Yes
Error Handling Yes
![Page 56: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/56.jpg)
Deployment: SSIS
Deployment Wizard
DTSPAC files to export
![Page 57: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/57.jpg)
Deployment: ADF
Power Shell scripts
All steps can be automated through a Power Shell Script
![Page 58: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/58.jpg)
Deployment through SSDT: ADF
![Page 59: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/59.jpg)
Comparison: Deployment
SSIS ADF
Deployment GUI Yes Yes
Deployment Scripts Yes Yes
Corporate Deployment Scenario with automated scripting Yes Yes
![Page 60: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/60.jpg)
Monitoring: SSIS
SSIS Logging
SSIS Catalog Reports
![Page 61: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/61.jpg)
Monitoring: ADF
Diagram View
Drill through monitoring features
Powerful GUI
Data Slice execution
Data Lineage
Very Important Feature
![Page 62: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/62.jpg)
Comparison: Monitoring
SSIS ADF
Monitoring GUI Yes Yes
Drill through Monitoring Yes Yes
Data Slice Yes
Data Lineage Yes
![Page 63: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/63.jpg)
Security: SSIS
Role Based
Roles for Deploy, Execute, Monitor…
![Page 64: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/64.jpg)
Security: ADF
Role Based
Roles:
Owner
Contributor
Reader
Data Factory Contributor
User Access Administrator
![Page 65: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/65.jpg)
Comparison: Security
SSIS ADF
Role Based Security Yes Yes
![Page 66: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/66.jpg)
HDInsight Compatibility
Azure Data Factory can work with existing HDInsight Clusters
ADF Can create HDInsight cluster on demand
ADF HDInsight Activity run Pig and Hive scripts
![Page 67: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/67.jpg)
Azure Analytics Integration
Azure ML Batch Scoring Activity
Data Lake Analytics U-SQL Activity
![Page 68: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/68.jpg)
Comparison SummarySSIS ADF
Development Tool Yes Yes (More Scripting Required)
Administration Tool Yes Yes (More Scripting Required)
User Type Developer Developer
Data Source and Destinations Wide Range Azure Storage, SQL Azure, SQL Server On Premises
Data Transformations Many Custom Code required
Environment Setup Required Azure Supported
Pricing Pay for Features Pay for Usage
Environmental and Administration Costs Yes No Costs
Error Handling Event Handers, Alerts Alerts
Logging Yes Yes
Deployment Yes Yes
Monitoring Yes Yes
Data Lineage Yes
Security Role Based Role Based
HDInsight Compatibility Normal High
Azure Analytics Integration High
![Page 69: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/69.jpg)
![Page 70: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/70.jpg)
Purpose
These tools/services didn’t built for the same purpose
They are complement of each other
![Page 71: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/71.jpg)
ETL Tool(SSIS, etc)
EDW(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed Data
Transform
BI Tools
Ingest (EL)
Original Data
Scale-out Storage & Compute
(HDFS, Blob Storage, etc)
Transform & Load
Data Marts
Data Lake(s)
Dashboards
Apps
Streaming data
Evolving Approaches to Analytics
![Page 72: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/72.jpg)
Get Benefits of Both
Hybrid SSIS and ADF
Cloud based data movement, Computing and monitoring
Rich On Premises Data Transformations
![Page 73: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/73.jpg)
Summary
If you are dealing with Big Data
If source or destination of data is on cloud
If you are dealing with Azure Storage
If you want to cut down environmental costs
If you want to cut down administration costs
If you want to cut down Pricing
If Azure is one side of the data
Then ADF Can be good response
But always Consider Hybrid Scenarios
![Page 74: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/74.jpg)
Summary
ADF comes as a Complementary service
Not to Compete with SSIS
![Page 75: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/75.jpg)
Reference To Study
Azure Data Factory Documentation
http://azure.microsoft.com/en-us/documentation/services/data-factory/
Azure Data Factory posts on my Blog:
http://www.radacad.com/tag/azure-data-factory
SSIS Book
https://www.packtpub.com/networking-and-servers/microsoft-sql-server-2012-integration-services-expert-cookbook
![Page 76: Azure Data Factory vs SSIS · What You can do with Azure Data Factory Access to data sources such as SQL Server On premises, SQL Azure, and Azure Blob storage Data transformation](https://reader030.fdocuments.in/reader030/viewer/2022040403/5e8b2e2df4f9931e7e23c20b/html5/thumbnails/76.jpg)
Questions?
@Rad_Reza
www.radacad.com
www.linkedin.com/in/rezarad