SSIS Dataflow Performance Tuning Jamie Thomson
-
Upload
elijah-contreras -
Category
Documents
-
view
38 -
download
0
description
Transcript of SSIS Dataflow Performance Tuning Jamie Thomson
![Page 1: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/1.jpg)
SSIS Dataflow Performance Tuning
Jamie Thomson
![Page 2: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/2.jpg)
• Buffer Architecture
• Design for perf!
• General tuning tips
SSIS Dataflow Performance Tuning
![Page 3: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/3.jpg)
Buffer Architecture
![Page 4: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/4.jpg)
• An area of memory
• Created by asynchronous components
• Does not move or change shape
• Data in a buffer can be changed by components
• Is what you see in a data viewer
What is a buffer?
![Page 5: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/5.jpg)
Demo 1What is a buffer
![Page 6: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/6.jpg)
Synchronous• Aka Row transformations
Asynchronous• Partially blocking• Fully blocking
Buffer Architecture
![Page 7: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/7.jpg)
Synchronous components
• Same buffer used for input and output• Number of rows in = Number of rows out• Generally very quick• Examples:• Derived Column• Conditional Split• Multicast
![Page 8: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/8.jpg)
Asynchronous components
• Creates new buffers for output• Different “shaped” input and output
buffers• Number of rows in <> Number of rows
out• Generally slower• Examples:• Aggregate• Sort
![Page 9: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/9.jpg)
…is a section of data flow starting from an asynchronous output and terminating
at inputs on transforms that have no synchronous outputs
-Kirk Haselden, P546 of “Microsoft SQL Server Integration Services”
Execution trees
![Page 10: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/10.jpg)
Demo 2Synchronous and
Asynchronous components(Count the expression
trees)
![Page 11: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/11.jpg)
Execution trees
![Page 12: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/12.jpg)
Inside an execution tree – What we think happens
Buffersdon’tmove
![Page 13: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/13.jpg)
Inside an execution tree – What actually happens
![Page 14: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/14.jpg)
Design for perf!
![Page 15: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/15.jpg)
“The Data Flow Task is performant by design, without any tuning or
optimization the default settings generally deliver great performance”
Kirk Haselden, Microsoft SQL Server 2005 Integration Services, Chapter 23 – Data Flow Task Internals and
Tuning
![Page 16: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/16.jpg)
• Remove unrequired columns (heed the warnings)
• Fixed-width files – only parse what you need
• ALWAYS use a SQL statement
• Only parse when needed (or leave as strings)
Only do what you have to
![Page 17: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/17.jpg)
Demo 3• Parse at source or in flow• Lookups vs Merge Joins
![Page 18: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/18.jpg)
General tuning tips!
![Page 19: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/19.jpg)
• Use FastParse if possible
• Turn off OnPipelineRowsSent
• Let the database do what its good at
• Raw files
• 64bit
![Page 20: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/20.jpg)
• Keep columns narrow• Point BufferTempStoragePath/BLOBTempStoragePath
at fast drives
• Increase DefaultBufferMaxSize & DefaultBufferMaxRows
• Optimise the destination (fast load, table lock, simple/bulk logged recovery/disable indexes)
• Identify bottlenecks
![Page 21: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/21.jpg)
Demo 4• Chaining expression
components• Redirect or Passthrough in a
Lookup• Fast Parse
![Page 22: SSIS Dataflow Performance Tuning Jamie Thomson](https://reader036.fdocuments.in/reader036/viewer/2022062321/5681365c550346895d9de84e/html5/thumbnails/22.jpg)
SSIS Performance Tuning Whitepaper – by Elizabeth Vitt et al
Microsoft SQL Server 2005 Integration Services (Chapter 23) by Kirk Haselden
Further reading