Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether:...
Transcript of Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether:...
![Page 1: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/1.jpg)
Visualizing Compiled ExecutablesVisualizing Compiled Executables for Malware Analysis
Daniel QuistLorie Liebrock
New Mexico Tech Los Alamos National Laboratory
![Page 2: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/2.jpg)
OverviewOverview
Explanation of ProblempOverview of Reverse Engineering ProcessRelated WorkRelated WorkVisualization for Reverse EngineeringVERA ArchitectureVERA ArchitectureCase Study: Mebroot
dUser StudyContributions
![Page 3: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/3.jpg)
Explanation of ProblemExplanation of Problem
• Reverse engineering is a difficult and esotericReverse engineering is a difficult and esoteric skill to learn
• Most new reversers struggle with understanding overall structureunderstanding overall structure
K i h i h diffi l• Knowing where to start is the most difficult task
![Page 4: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/4.jpg)
Reverse Engineering ProcessReverse Engineering Process
Setup an Isolated Environment• VMWare, Xen, Virtual PC• Dedicated Hardwarep
Initial Analysis and Execution
Co
• Dedicated Hardware
• Sysinternals, CWSandbox• Look for OS State Changesy
Deobfuscation / Software Dearmoring
omplexity In
Look for OS State Changes• Files, registry, network
• UnpackingDeobfuscation / Software Dearmoring
Disassembly / Code‐level analysis
ncreases
• Debuggers, Saffron, Ether
• IDA ProOll DbDisassembly / Code level analysis
Identify Relevant and Interesting Features
• OllyDbg
• Experience basedbi h bl i h hi• Newbies have trouble with this
![Page 5: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/5.jpg)
Addressing the SituationAddressing the Situation
Setup an Isolated EnvironmentSetup an Isolated Environment
Initial Analysis and Execution
C
Initial Analysis and Execution
Deobfuscation / Software Dearmoring
omplexity
Deobfuscation / Software Dearmoring
Disassembly / Code level analysis
Increases
Disassembly / Code‐level analysis
Identify Relevant and Interesting FeaturesIdentify Relevant and Interesting Features
![Page 6: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/6.jpg)
Packing and EncryptionPacking and Encryption
• Self‐modifying codeSelf modifying code– Small decoder stub– Decompress the main executablep– Restore imports
• Play “tricks” with the executablePlay tricks with the executable– OS Loader is inherently lazy (efficient)– Hide the importsp– Obscure relocations– Use bogus values for various unimportant fieldsg p
![Page 7: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/7.jpg)
Normal PE FileNormal PE File
![Page 8: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/8.jpg)
Packed PE FilePacked PE File
![Page 9: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/9.jpg)
Related WorkRelated Work
![Page 10: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/10.jpg)
IDA Pro ‐ Graphing CrossreferencesIDA Pro Graphing Crossreferences
• Illustrates Relationship of Function Calls• Magenta represents imported API calls• Black represents module subroutines
![Page 11: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/11.jpg)
IDA Pro – Visualization ProblemsIDA Pro Visualization Problems
Firefox Initialization
• Some graphs are uselessS h l• Some graphs are too complex
• No indication of heavily executed portions• Obfuscated code is gibberish
idag.exe (IDA Pro) overview
![Page 12: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/12.jpg)
Alex Dragulescu – MyDoom Visualization
http://www.sq.ro/malwarez.php
![Page 13: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/13.jpg)
Visualization for Reverse EngineeringVisualization for Reverse Engineering
• Identify major program functional areasde t y ajo p og a u ct o a a eas– Initialization– Main loops– Communications / organizational structure
D bf ti / d i• Deobfuscation / dearmoring– Identify packing loopsFind self modifying code– Find self‐modifying code
• Take “intuition” out of the reversing processTake intuition out of the reversing process
![Page 14: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/14.jpg)
Enabling Technology: EtherEnabling Technology: Ether
• Patches to the Xen Hypervisoryp• Instruments a Windows system• Base modules available
– Instruction tracing– API tracingUnpacking– Unpacking
• “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif, Lee
ACM CCS 2008ACM CCS 2008
![Page 15: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/15.jpg)
Ether System ArchitectureEther System Architecture
Linux Dom0 Management OS Instrumented Windows XP SP2
VM Disk ImageEther
Management Tools
Xen Hypervisor with Ether PatchesRing ‐1
![Page 16: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/16.jpg)
Visualizing Executables for Reversing d land Analysis
• OpenGL rendering of dynamic programOpenGL rendering of dynamic program execution
• Vertices represent addresses• Vertices represent addresses• Edges represent execution from one address
hto another• Thicker edges represent multiple executions• Colors to help identify type of code
![Page 17: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/17.jpg)
Graph PreviewGraph Preview
![Page 18: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/18.jpg)
VERA ArchitectureVERA Architecture
Ether AnalysisSystem Gengraph
OGDF
VERA
OpenGL
Open Graph Display Framework‐ Handles all layout and arrangement of the graphs‐ Similar to Graphviz‐Works with large datasetsWorks with large datasets
![Page 19: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/19.jpg)
Vertices (Addresses)Vertices (Addresses)• Basic blocks
– Fundamental small grouping of code
– Reduces data sizeReduces data size– Useful for large commercial programs
I t ti• Instructions– Useful for small programs– Greater aesthetic valueGreater aesthetic value– Larger datasets can produce useless graphs
![Page 20: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/20.jpg)
Edges (Transition)Edges (Transition)• Transitions between addresses
• Thicker lines represent more executionsexecutions– Easy identification of loops– Find heavy concentration ofFind heavy concentration of execution
l i l d f d• Multiple edges from a node represent decision point
![Page 21: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/21.jpg)
ColorsColors
• Yellow – Normal uncompressed low‐entropyYellow Normal uncompressed low entropy section data
• Dark Green Section not present in the• Dark Green – Section not present in the packed versionLi h P l Si OfR D 0• Light Purple – SizeOfRawData = 0
• Dark Red – High Entropy • Light Red – Instructions not in the packed exe• Lime Green – Operands don’t matchLime Green Operands don t match
![Page 22: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/22.jpg)
Netbull Virus (Not Packed)Netbull Virus (Not Packed)
![Page 23: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/23.jpg)
Netbull Zoomed ViewNetbull Zoomed View
![Page 24: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/24.jpg)
UPXUPX
![Page 25: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/25.jpg)
UPX ‐ OEPUPX OEP
![Page 26: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/26.jpg)
ASPackASPack
![Page 27: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/27.jpg)
FSGFSG
![Page 28: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/28.jpg)
MEWMEW
![Page 29: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/29.jpg)
Case Study: MebrootCase Study: Mebroot
• Took latest Mebroot sample from Offensive pComputing collection
• Analyzed inside of VERA• Analyzed inside of VERA
• Seemed to be idling for long periods of timeSeemed to be idling for long periods of time
• Actually executed based on network traffic
• Hybrid user mode / kernel malware
![Page 30: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/30.jpg)
Mebroot – Initial Busy LoopMebroot Initial Busy Loop
• Initial analysis shows decoder for driver• Sits for 30 minutes waiting for us to get bored• Moves on to the rest of the program
![Page 31: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/31.jpg)
Mebroot – After Busy LoopMebroot After Busy Loop
![Page 32: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/32.jpg)
Mebroot – Entire ViewMebroot Entire View
Main Unpacking Loop
30 Minute Busy Loop
Initialization
Kernel CodeInsertion
![Page 33: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/33.jpg)
User StudyUser Study
• Students had just completed week longStudents had just completed week long reverse engineering course
• Analyzed two packed samples of the Netbull• Analyzed two packed samples of the NetbullVirus with UPX and MEWA k d f i f k b d• Asked to perform a series of tasks based on the typical reverse engineering process
• Asked about efficacy of visualization tool
![Page 34: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/34.jpg)
User Study: Tasks PerformedUser Study: Tasks Performed
• Find the original entry point (OEP) of theFind the original entry point (OEP) of the packed samples
• Execute the program to look for any• Execute the program to look for any identifying outputId if i f h bl• Identify portions of the executable:– Packer code– Initialization– Main loops
![Page 35: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/35.jpg)
![Page 36: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/36.jpg)
![Page 37: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/37.jpg)
![Page 38: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/38.jpg)
![Page 39: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/39.jpg)
Results of User StudyResults of User Study
![Page 40: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/40.jpg)
Selected CommentsSelected Comments
• “Wonderful way to visualize analysis and toWonderful way to visualize analysis and to better focus on areas of interest”
• “Fantastic tool. This has the potential to significantly reduce analysis time ”significantly reduce analysis time.
“I k R l ASAP”• “It rocks. Release ASAP.”
![Page 41: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/41.jpg)
Recommendations for improvementRecommendations for improvement
• Need better way to identify beginning and endNeed better way to identify beginning and end of loops
• Many loops overlap and become convoluted
• Be able to enter memory address and see b i bl k h hbasic blocks that match
![Page 42: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/42.jpg)
Future WorkFuture Work
• General GUI / bug fixesGeneral GUI / bug fixes• Highlight temporal nature of execution
i li i• Memory access visualization• System call integration• Function boundaries• Interactivity with unpacking processInteractivity with unpacking process
![Page 43: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/43.jpg)
ConclusionConclusion
• Overall process for analyzing and reverseOverall process for analyzing and reverse engineering malware is shortened
• Program phases readily identified
• Integration with existing tools
• Preliminary user study shows tool holds promise for speeding up reverse engineering
![Page 44: Visualizing Compiled Executables - VizSec · Visualizing Compiled Executables ... • “Ether: Malware Analysis via Hardware Virtualization Extensions” Dinaburg, Royal, Sharif,](https://reader030.fdocuments.in/reader030/viewer/2022040412/5f061ca77e708231d416597a/html5/thumbnails/44.jpg)
Questions?Questions?
• Source tools and latest slides can be foundSource, tools, and latest slides can be found at:http://www offensivecomputing nethttp://www.offensivecomputing.net
• If you use the tool please give feedback• If you use the tool, please give feedback
• Contact info: [email protected]