Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices
-
Upload
dmitry-vostokov -
Category
Documents
-
view
264 -
download
0
Transcript of Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices
![Page 1: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/1.jpg)
Software Trace and Memory Dump Analysis
Presenter: Dmitry Vostokov Memory Dump Analysis Services
![Page 2: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/2.jpg)
Prerequisites
Experience in software troubleshooting and reading software logs
Advantage: Citrix CDF and
Microsoft ETW trace analysis including Process Monitor logs
© 2011 Memory Dump Analysis Services
![Page 3: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/3.jpg)
Agenda Memory Dump Analysis Services Root Cause Analysis Methodology Software Traces and Memory Dumps Examples
© 2011 Memory Dump Analysis Services
![Page 4: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/4.jpg)
MDA Services Memory Dump Analysis Audit Software Trace Analysis Audit (New) Software Error Reporting Audit Remote Training Debugging Bureau Tool Objects and EasyDbg
Powered by DA+TA DumpAnalysis.org + TraceAnaysis.org
© 2011 Memory Dump Analysis Services
![Page 5: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/5.jpg)
A.C.P. Root Cause Analysis
© 2011 Memory Dump Analysis Services
Artifacts
Checklists
Patterns
Checklists and patterns as best practices
Iterative and Incremental
![Page 6: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/6.jpg)
DA+TA DA: Dump Artifact / Dump Analysis Memory snapshots: process, kernel, physical memory dumps
TA: Trace Artifact / Trace Analysis Software traces: Event Tracing for Windows, logs
© 2011 Memory Dump Analysis Services
![Page 7: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/7.jpg)
Spatiality vs. Narrativity
© 2011 Memory Dump Analysis Services
Narrativity
Spartiality
SoftwareTrace
Memory Dump
Software trace as software narrative, the story of a computation
![Page 8: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/8.jpg)
Tools for Artifact Analysis Memory dumps:
WinDbg from Debugging Tools for Windows Notepad (textual debugger logs)
Software traces:
CDFAnalyzer* / CDFControl from Citrix Process Monitor* from Microsoft * supports adjoint threads
© 2011 Memory Dump Analysis Services
![Page 9: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/9.jpg)
Checklists for Analysis Memory dumps:
http://www.dumpanalysis.org/blog/index.php/2007/06/20/crash-dump-analysis-checklist/
Software traces:
http://www.dumpanalysis.org/blog/index.php/2011/03/10/software-trace-analysis-checklist/
© 2011 Memory Dump Analysis Services
![Page 10: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/10.jpg)
Software Behavior Patterns Memory dump and software trace Examples: Spiking Thread, Discontinuity +200 patterns (DA+TA) DumpAnalysis.org
© 2011 Memory Dump Analysis Services
![Page 11: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/11.jpg)
DA: Software Behavior
Memory dump: a memory snapshot Definition, partial classification and
historical list Pattern identification case studies
© 2011 Memory Dump Analysis Services
![Page 12: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/12.jpg)
TA: Software Behavior
“Imagine you got a software trace from hundreds of modules you haven’t written or haven’t seen source code of...”
Software trace: a sequence of memory fragments ordered in time
Definition, and historical list Pattern identification case studies
© 2011 Memory Dump Analysis Services
![Page 13: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/13.jpg)
CDFAnalyzer Filters
© 2011 Memory Dump Analysis Services
![Page 14: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/14.jpg)
Threads Time
# PID TID Time Message
Time
# PID TID Time Message
© 2011 Memory Dump Analysis Services
![Page 15: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/15.jpg)
Adjoint Threads
© 2011 Memory Dump Analysis Services
Time
# PID TID Time Message
Time
# PID TID Time Message (ATID)
![Page 16: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/16.jpg)
Significant Event csrss.exe winlogon.exe LogonUI.exe userinit.exe …
Custom events: CDFMarker
© 2011 Memory Dump Analysis Services
Time
# PID TID Time Message
![Page 17: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/17.jpg)
Discontinuity
© 2011 Memory Dump Analysis Services
… 14:23:02.146 14:23:02.345 14:31:10.254 14:31:10.341 …
Time
# PID TID Time Message
![Page 18: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/18.jpg)
No Activity Expecting messages from Module X Absence of such messages may
suggest that a process or a thread was hang / blocked
© 2011 Memory Dump Analysis Services
![Page 19: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/19.jpg)
Guest Component Sudden appearance of an unexpected
module, for example, werfault.exe or faultrep.dll
© 2011 Memory Dump Analysis Services
![Page 20: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/20.jpg)
Statement Current The flood of messages
Normal case: 15 msg/s Abnormal case: 3500 msg/s
May point to a CPU spike
© 2011 Memory Dump Analysis Services
![Page 21: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/21.jpg)
Resources DumpAnalysis.org Pattern-Driven Memory Dump Analysis Memory Dump and Trace Analysis: A Unified Pattern Approach Introduction to Pattern-Driven Software Problem Solving Advanced Software Debugging Reference:
OpenTask publishes this talk with extra case studies (ISBN: 978-1908043238)
© 2011 Memory Dump Analysis Services
![Page 22: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/22.jpg)
More Resources August remote training season: Accelerated Windows Memory Dump Analysis Complete Physical Memory Dump Analysis
Visit Memory Dump Analysis Services for registration details:
www.DumpAnalysis.com
© 2011 Memory Dump Analysis Services
![Page 23: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/23.jpg)
Free Summer Webinars The Old New Crash: Cloud Memory Dump
Analysis (June 6th) Cyber Warfare Memory Dump Analysis
(forthcoming in July-August)
Visit Memory Dump Analysis Services for registration details:
www.DumpAnalysis.com
© 2011 Memory Dump Analysis Services
![Page 24: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/24.jpg)
Q&A
Please send your feedback using the contact form on DumpAnalysis.com
© 2011 Memory Dump Analysis Services
![Page 25: Software Trace and Memory Dump Analysis: Patterns, Tools, Processes and Best Practices](https://reader031.fdocuments.in/reader031/viewer/2022020314/58a151d41a28abbe3c8b5415/html5/thumbnails/25.jpg)
Thank you!
© 2011 Memory Dump Analysis Services
Join DA+TA Facebook Group