Fascinate with SQL SSIS Parallel processing
25
SSIS Parallel Processing
-
Upload
vishal-pawar -
Category
Technology
-
view
721 -
download
1
Transcript of Fascinate with SQL SSIS Parallel processing
SSIS Parallel Processing
About Me - Vishal Pawar
• 8 + years of experience in Microsoft BI Stack• Microsoft BI Solution Architect in The Most Group• BI Consulting partner of Right Click Info • Professional Microsoft BI Trainer at Mentalist Network • NMUG Community Leader , Speaker • Microsoft Community Representative for Navi Mumbai region.• SQL PASS Chapter Lead for Mumbai & Navi Mumbai • MCTS: Microsoft® Certified Technology Specialist -SQL Server
2008,Business Intelligence Development and Maintenance• MCSA: Microsoft® Certified Solutions Associate• CSM: Certified Scrum Master from Scrum Alliance, US• SQL Server Expert and 3 Complex SQL tool innovator on Codplex• BE-Computer Science & Engineering , RAIT , Mumbai Univercity• MBA-Information System, Arianth College , Pune• Blogging @ http://bimentalist.com , http://sqlmentalist.com• BI Articles :: 350+ , SQL Article – 400+, SQL Script – 350+
What is Parallel Processing ?
3
Where Parallel Processing can be use ?
SSIS and BI
SSIS vs SQL Stored Procedure
3
4
Presentation Agenda
3
2
1
SSIS and Parallel Processing
How to do SSIS Parallel Processing ?
DEMO
Off Topic – How to use Auto Test Script for Review and Testing
5
6
7
8
Comparison of all with Time9
New Innovation – Parallel Processing with Continuous Serial Threading 10
33
Question, Files and Links11
What is Parallel Processing ?
Serial processing
P1
TASK 1
P2P3P4
TASK 2
TASK 3
TASK 4
At t1
At t2
At t3
At t4
Serial Processing
Parallel Processing
P1P2P3P4
TASK 2TASK 3TASK 4
At t1
At t2
TASK 2
Where Parallel Processing can be use ?
Can we use Parallel Processing to read text file
C++ Code - http://www.cplusplus.com/reference/cstdio/fread/
SSIS and Parallel Processing
Kick of Parallel Process to save time
Architecture diagram of SQL server
SQL Server 2014 and Parallelism
Will see this when we have SQL 2014
SSIS Architecture & Parallels
SSIS and BI
MS BIImplementation
SSIS 60%
SSRS25%
SSAS15%
SSIS vs SQL Stored Procedure
Performance Features/capabilities Current skill set
Complexity ExtensibilityArchitecture of
sources
Data Quality Checking
ETL Framework Development Time
How Banner Fact Calculation doable in SSIS Parallel Processing ?
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
P1P2P3P4
Develop SQL Statement to
Run Only One Survey ID
DEMO
How Scale Mean Score doable in SSIS Parallel Processing ?
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
Develop SQL Statement to
Run Only One Survey ID
P1P2P3P4
Develop SQL Statement to
Run Only One Survey ID
All Scale mean Calculation
But not required as we can calculate this using
Response Matrix Calculation
Off Topic – How to use Auto Test Script for Review and Testing
Testing Possibilities and Expertise !
Example of SQL Test Script & Review of Parallel Stats
Create Automated Test Script Create Review Test Script
1 2
DEMO
Valid N Count Test
Banner Fact Count Test
New Innovation – Parallel Processing with Continuous Serial Threading
P1P2P3P4
TASK 2TASK 3TASK 4
At t1
At t2
TASK 2
Batch Of Parallel Processor
Parallel Processing with Continuous Serial Threading
Show me Real Stats !
Parallel Processing with Continuous Serial Threading Stats
Comparison of all with Time
SP with SQL JOB
SP with Paralleled SQL JOB
SSIS in SQL JOB
SSIS in SQL JOB
SSIS with SP in SQL JOB
SSIS Parallel Processing With SQL JOB
SSIS Parallel Processing With SP With SQL JOB
SSIS Parallel Processing With Serial Threading SP With SQL JOB
SSIS Parallel Processing With CLR With Serial Threading SP With SQL JOB
SQL 2014-SSIS Parallel With CLR With Serial Threading SP With SQL JOB
Question, PPT and Visio Files and Links
• SSIS – An Inside View Part 4
• SSIS Nugget: Engine Threads
• Improving the Performance of the Data
Flow
• Designing Your SSIS Packages for
Parallelism (SQL Server Video)
• Too Many Sources in a Data Flow
• SSIS Operational and Tuning Guide• What Is Parallel Execution In SSIS, How Many
Tasks A SSIS Package Can Execute In Parallel?
• SSIS How to Process Data as Fastest, Parallel , Multithreaded or in Very Efficient Way !!!
