Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika...
Transcript of Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika...
![Page 1: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/1.jpg)
1
Malware Detection
Prof. Ravi SandhuExecutive Director and Endowed Chair
Lecture 14
© Ravi Sandhu World-Leading Research with Real-World Impact!
CS 5323
![Page 2: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/2.jpg)
Virus detection is undecidable Cohen dissertation (1985), paper (1987)
Anti-virus (more generally anti-malware) is a great business model Need regular updates Infinite supply of new malware
Malware can be stealthy Malware can be really stealthy
© Ravi Sandhu 2World-Leading Research with Real-World Impact!
Highlights
![Page 3: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/3.jpg)
© Ravi Sandhu 3World-Leading Research with Real-World Impact!
Malware Detection Techniques
Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University, Feb 2007.
![Page 4: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/4.jpg)
© Ravi Sandhu 4World-Leading Research with Real-World Impact!
Malware Detection Techniques
Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University, Feb 2007.
MisuseDetection
Behavior-BasedDetection
![Page 5: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/5.jpg)
© Ravi Sandhu 5World-Leading Research with Real-World Impact!
Signature Limitations
Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University, Feb 2007.
S needsregularupdates
![Page 6: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/6.jpg)
© Ravi Sandhu 6World-Leading Research with Real-World Impact!
Anomaly Based
TrainingPhase
DetectionPhase
Inferpatterns
Inferspecifications
![Page 7: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/7.jpg)
© Ravi Sandhu 7World-Leading Research with Real-World Impact!
Anomaly Based Limitations
Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University, Feb 2007.
Blue area is false positivesIf white area extends outside blue area we have false negatives
![Page 8: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/8.jpg)
Defeat signature-based detection Encrypted malware Polymorphic malware Metamorphic malware
Rootkit can misrepresent the existence or content of executable files
© Ravi Sandhu 8World-Leading Research with Real-World Impact!
Stealthy Malware
You, I., and Yim, K. Malware obfuscation techniques: A brief survey. IEEE International Conference on Broadband, Wireless Computing, Communication and Applications, Nov 2010, pp. 297-300.
![Page 9: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/9.jpg)
© Ravi Sandhu 9World-Leading Research with Real-World Impact!
Encrypted Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Decryptor
execute malware
Key
propagatemalware
![Page 10: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/10.jpg)
© Ravi Sandhu 10World-Leading Research with Real-World Impact!
Encrypted Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Decryptor
execute malware
Key
propagatemalware
revealssignature
![Page 11: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/11.jpg)
© Ravi Sandhu 11World-Leading Research with Real-World Impact!
Polymorphic Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Obfuscated Decryptor
execute malware
Key
propagatemalware
![Page 12: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/12.jpg)
© Ravi Sandhu 12World-Leading Research with Real-World Impact!
Polymorphic Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Obfuscated Decryptor
execute malware
Key
propagatemalware
nosignature
![Page 13: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/13.jpg)
© Ravi Sandhu 13World-Leading Research with Real-World Impact!
Polymorphic Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Obfuscated Decryptor
execute malware
Key
propagatemalware
nosignature
Execute in a sandbox and detect the signature after
decryption
![Page 14: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/14.jpg)
© Ravi Sandhu 14World-Leading Research with Real-World Impact!
Polymorphic Malware
Encrypted Main Body Decryptor Cleartext Main Body
Encrypted Main Body Key’ Obfuscated Decryptor
execute malware
Key
propagatemalware
nosignature
Execute in a sandbox and detect the signature after
decryptionMutation Engines automate this construction
![Page 15: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/15.jpg)
© Ravi Sandhu 15World-Leading Research with Real-World Impact!
Metamorphic Malware
Original Main Body Original Main Body
execute malware
propagatemalware
nosignature
Obfuscated Main Body
execute malware
Obfuscated Main Body
Obfuscated Main Body
execute malware
Obfuscated Main Body
propagatemalware
![Page 16: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/16.jpg)
Dead-Code Insertion Register Reassignment Subroutine Reordering Instruction substitution Code transposition Code Integration
© Ravi Sandhu 16World-Leading Research with Real-World Impact!
Obfuscation Techniques
![Page 17: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/17.jpg)
Not visible in source code Reappears in binary code due to malware infected
compiler In theory could reappear in binary code due to other
components in binary execution workflow Loader Linker OS BIOS
© Ravi Sandhu 17World-Leading Research with Real-World Impact!
Really Stealthy Malware
Ken Thompson. Reflections on trusting trust. Commun. ACM 27, 8 (August 1984), 761-763.
![Page 18: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/18.jpg)
© Ravi Sandhu 18World-Leading Research with Real-World Impact!
Malicious Compiler Inserts a Backdoor
Malicious CompilerBinary
OS Login module
Infected Login Binary
![Page 19: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/19.jpg)
© Ravi Sandhu 19World-Leading Research with Real-World Impact!
Malicious Compiler Inserts a Backdoor
Malicious CompilerBinary
OS Login module
Infected Login Binary
Assumption: Malicious behavior cannot be detected
in binary, but may be detectable in
compiler source
![Page 20: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/20.jpg)
© Ravi Sandhu 20World-Leading Research with Real-World Impact!
Self-Compiler
Compiler binary for language L
Compiler source for language L
Compiler binary for language L
![Page 21: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/21.jpg)
© Ravi Sandhu 21World-Leading Research with Real-World Impact!
Malicious Self-Compiler in Binary and Source
Compiler binary for language L
Malicious Compiler source for language L
Malicious Compiler binary for language L
![Page 22: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/22.jpg)
© Ravi Sandhu 22World-Leading Research with Real-World Impact!
Malicious Self-Compiler in Binary and Source
Compiler binary for language L
Malicious Compiler source for language L
Malicious Compiler binary for language L
Source code analysis will reveal malicious behavior
![Page 23: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/23.jpg)
© Ravi Sandhu 23World-Leading Research with Real-World Impact!
Doubly Malicious Self-Compiler in Binary and Source
Compiler binary for language L
Doubly Malicious Compiler source for language L
Doubly Malicious Compiler binary for language L
Source code analysis will reveal doubly malicious
behavior
![Page 24: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/24.jpg)
© Ravi Sandhu 24World-Leading Research with Real-World Impact!
Doubly Malicious Complier Binary Behavior
Doubly Malicious Compiler binary for language L
Compiler source for language L
Doubly Malicious Compiler binary for language L
OS Login module
Infected Login Binary
Doubly Malicious Compiler binary for language L
![Page 25: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/25.jpg)
© Ravi Sandhu 25World-Leading Research with Real-World Impact!
Doubly Malicious Complier Binary Behavior
Doubly Malicious Compiler binary for language L
Compiler source for language L
Doubly Malicious Compiler binary for language L
OS Login module
Infected Login Binary
Doubly Malicious Compiler binary for language L
No trace of malicious behavior
in source code
![Page 26: Prof. Ravi Sandhu Executive Director and Endowed Chair · Anomaly Based Limitations. Nwokedi Idika and Aditya Mathur, A Survey of Malware Detection Techniques, Purdue University,](https://reader033.fdocuments.in/reader033/viewer/2022050611/5fb27aeff75538788a62ee9a/html5/thumbnails/26.jpg)
© Ravi Sandhu 26World-Leading Research with Real-World Impact!
Malicious Self-Compiler in Binary but not in Source
Malicious Compiler binary for language L
Compiler source for language L
Malicious Compiler binary for language L
Wheeler, D.A., Countering trusting trust through diverse double-compiling, 21st Annual Computer Security Applications Conference, pp.13-48, 5-9 Dec. 2005.
partial countermeasure