Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

93
Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013

Transcript of Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Page 1: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Networking & Malware

CS 598: Network Security

Michael Rogers & Leena Winterrowd

March 26, 2013

Page 2: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Types of Malware

Image courtesy of prensa.pandasecurity.com

Page 3: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Types of Malware

Viruses 16,82% Trojan horses

69.99%

No standardize

d definitions!

Page 4: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Viruses

• Programs capable of self-replication

• Spread to other systems

• Cannot execute on their own

• Must attach themselves to other programs

• Effectively need user-interaction to spread

Page 5: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Worms

• Standalone programs

• Self-replicating

• Rely on exploits to self-execute

• Self-propagating

• No user interaction!

Page 6: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Ye Olde Computyre Virus

Thou hast presently received ye olde virus!

Since it doth not useth 'electricitee' or 'computyres', thou art on ye olde 'Honore

Systeme'.

Please deleteth all of thy files from thy hard drive and forward ye olde virus to thy

friends.

Page 7: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Trojans

• Masquerade as legitimate files

• Often 'gifts' or free downloads

• Gives (unauthorized) access to a system

• Most often propagated with worms

• Most often contains spyware

Page 8: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Backdoors

• Bypass security to directly access data/service

• Often default/hard-coded password

• Maintain undetectability

• Example (2003):

• 2-line Linux kernel change: http://kerneltrap.org/node/1584

• Frequently used by worms

Page 9: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Rootkits

DAEMON Tools is actually a beneficial rootkit!

(Intercepts Windows API calls)

• Hide existence of a payload

• Payload is often a trojan• Generally subvert/disable security

programs• Usually enable root access (elevated

privilege)

• Modern rootkits do not do this!• Most often perform injection:

• Enable a backdoor• Replace a library• Hide on devices or in BIOS• CompuTrace & LoJack

Page 10: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Spyware

• Collects information without user knowledge/permission

• Often trojans• May be intentional• Keyloggers

Page 11: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Adware

• Automatically renders ads

• Generates money for developer(s)

• Often intentional• Ideally non-

intrusive

Page 12: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Typhoid Adware

• An infected machine poses as the legitimate access point

• Intercepts and hijacks other users connections via ARP spoofing

• The infected machine inserts ad-content into video streams

• Infected machine shows no symptoms

• Only a NAT-box proxyPaper available at:http://pages.cpsc.ucalgary.ca/~aycock/papers/eicar10.pdf

Page 13: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Infection Mechanisms• Droppers

o Inject malware (single-stage)o Download malware to the machine (two-

stage)o Pretend to be legitimate programs

(Trojans)o Injector: dropper which installs to

memory only

• Drive-By Downloadso Placed on systems by compromised

websiteso Serves as point of entry for other

malwareo Recent Example: FBI virus (Java exploit)

Image courtesy of http://www.technobuffalo.com

Page 14: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Infection Mechanisms• DECEPTION!

• Exploitation

• OS design defectso Zero-dayo Unpatched

• Software bugs• Privilege elevation• Preexisting (related or unrelated) backdoors• 'Auto-run' on removable devices (USB, CD, etc.)• Purposely install malicious code• Physical access

Image courtesy of http://www.technobuffalo.com

Page 15: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Well-Known Malware Examples

Page 16: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Stuxnet

• In June 2010, VirusBlokAda discovered an unprecedented type of Malware – Stuxnet.

• But what made Stuxnet different?

(usu < 1KB)

Page 17: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Stuxnet's Infection Mechanisms

Image courtesy of http://www.symantec.com/connect/blogs/exploring-stuxnet-s-plc-infection-process

• Infected Windows systems via USB (auto-run)

o 3 infections/drive; self-replicates to removable drives

• Worm attempts to spread to any Windows system for 21 days

• Systems were 'air-gapped' (not connected to internet)

• Uses four zero-day Windows exploits

o Copies itself through LAN via a print-spooler exploito Spreads through SMBo Exploits a Windows Server Service RPC vulnerability (same as

Conficker worm; patched in 2008)o 2 escalation of privilege vulnerabilities

Page 18: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Stuxnet's Propagation Mechanisms

Image courtesy of http://www.symantec.com/connect/blogs/exploring-stuxnet-s-plc-infection-process

• Spreads via network shares

• Looks for and injects itself into specific control software project

o Software has a hard-coded passwordo Copies to server via SQL injection

• Can self-update or report data via 'command & control' servers

o Self-updating via LAN or p2p

• Contained a Windows rootkit to further avoid detection

• Digitally signed with stolen certificates from Realtek & Jmicron

Page 19: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

What did Stuxnet do?

• Targeted Siemen's 315 and 417 PLCs

o Fingerprinted by model number, configuration, and actual PLC code

• Exploited a driver DLL to copy itself to the PLCs

• Changed frequency controller drives' speeds

o Alternated between slowing down and speeding up the normal frequency

o Could cause a PLC-controlled centrifuge to fly apart over time

Image courtesy of http://www.symantec.com/connect/blogs/exploring-stuxnet-s-plc-infection-process

Centrifuge

SpeedSettings

Page 20: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Flame

• "Arguably the most sophisticated malware ever found"o ~20 MB

• Spreads via LAN or USB

• Compromised Microsoft code-signing certificateo MD5 chosen-prefix collision attack

• Modular design

Page 21: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

What did Flame do?

• Steals information

• Records Skype calls

• Activates Bluetootho Steals information from other Bluetooth

devices

• Communicates information back to command & control server and awaits further instructions

Page 22: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

DNSChanger• Drive-by download claiming to be a

required video codec

• Modified DNS config to go through a rogue name server

• Injected/substituted advertising on web pages & redirected some links

• Could spread within a LANo Mimicked a DHCP servero Pointed others towards the rogue DNS

servers• Perpetrators apprehended, but rogue

DNS servers left running for fear of knocking infected machines off the internet

Page 23: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Nimda

• Virus/worm hybrid

• Infected via multiple avenueso Emailo Network shareso Compromised websiteso Microsoft IIS vulnerability exploitso Backdoors left by other worms (Code Red

II and sadmind/IIS)

• Became the internet's most widespread worm within 22 minutes

Page 24: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Why Malware is Written

• 'For teh lulz' (entertainment value)o Causing distraction or destruction just because it's

amusing

• To show offo Exploit remote systems as a show of skill

• Anonymityo Attacks may act as the victim

• Sociopoliticalo Anonymous, Lulzsec, hacktivistso Stuxnet & Flameo May cause physical damage! (Stuxnet)

• For profit

Page 25: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Malware for Profit• Spyware

o Gain personal information for various purposeso Targeted marketing or identity thefto Corporate espionage/sabotage

• Botnetso Cloud-based attacks (DDOS, click fraud, spam)

• Adware/scareware/ransomwareo Directly bilk money from victims

• Recursiveo Sell dropper/backdoor kitso Promote further infection

Page 26: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Malware Propagation

Page 27: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Target Selection

• Completely targeted• Semi-targeted• Brute-force/random• Pseudorandom• Diffusion

Page 28: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Completely Targeted

• Predetermined list of targets• Common to spam/phishing• Tend to employ social engineering

techniques

Page 29: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Semi-Targeted

• Takes a good guess at the next target

• Often target machines on the local network (worms)

• Uses the concept of homogeneityo Exploit one in network → may be

able to exploit all• E-mail contact lists (trojans)

Page 30: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Brute-Force

• Port-scanning and IP scanning the entire address space

• Often start from a randomized offset and skip around

Page 31: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Pseudorandom

• Brute-force with restrictions (for better performance)

• Example: Blacklist known darknet/honeypot addresses

• Example: Prioritize IPs belonging to a specific country

Page 32: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Diffusion

• Design malware to use alternate channels of infection (USB drives or smartphones)

• Hope someone plugs the wrong thing in the wrong place

• Can be random or targeted• Targeted often requires research

on habits/behaviors of individuals in the target environment

Page 33: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Actual Propagation

• Self-propagation• Social engineering• Secondary infections• Malicious code sources:

o From central sourceo From infectoro Inject as part of exploitation

Page 34: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Self-Propagation

• Uses exploits on the remote machine to self-install

• Examples:o Unpatched network daemons (several in

older versions of Samba)o Insecure driver code (thumb drives and

other out-channel exploits)o Insecure system settings (autoplay, no

UAC)

Page 35: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Social Engineering

• Sends a copy of the malware disguised as something innocuouso "Funny cat video!.mpg.exe"

• Spread by malicious user, unwitting infected user, or the malware itself

Page 36: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Secondary Infections

• Create an artificial vulnerability or exploit

• Serves as the vehicle for other malware

• Primary approach of droppers & backdoors

Page 37: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Honeypots

• Detection mechanism that exploits random/pseudorandom propagation

o Pose as a vulnerable system

o Capture malware samples

• Often run by known organizations

o Known IP spaces = easy to avoid

• Low interaction honeypots

o Emulate aspects of a vulnerable system

o Safer but only emulate specific aspects

• High interaction honeypots

o Actual full systems/VMs

o Specialized firewall

o Infection (hopefully) cannot spread

Page 38: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Communication and Control

Page 39: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Four different classifications

• Uncontrolled and silent

• Controlled and silent

• Uncontrolled and noisy

• Controlled and noisy

Page 40: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Silent

• No interaction with programmer in either direction

• No transmitting of information back to source

• Behavior must be pre-programmed, e.g. Stuxnet

• Often used simply to cause destruction

Page 41: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Silent

• Pros

• Cannot be disrupted by compromising command method

• Less likely to be detected by network monitoring (under correct conditions)

Page 42: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Silent

• Cons

• No dynamic control

• Cannot be used for data theft, reconnaissance

Page 43: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Silent

• Can receive commands

• Numerous channels available, such as IRC, DHT, Google link bombing, establishing direct network contact, P2P networks, file drops

• Does not transmit information

• Often used for targeted attacks, occasionally used for botnets, planting backdoors

Page 44: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Silent

• Pros

• Behavior can change dynamically after launch in direct response to controller

• Less likely to be detected by network monitoring (under correct conditions, initially)

Page 45: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Silent

• Cons

• Cannot be used for data theft, reconnaissance

• Can be disrupted or even destroyed by subversion of command mechanism

Page 46: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Noisy

• Can communicate information about infected systems

• Methods include file drops on a central server or to online hosting services (e.g. Mega), IRC channels, P2P services

• More useful for reconnaissance, smash-and-grab

Page 47: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Noisy

• Pros

• Easiest for ‘blitz’ style attacks

• Good for blind mapping

Page 48: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Uncontrolled and Noisy

• Cons

• No dynamic control

• More likely to be detected

Page 49: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Noisy

• Allows for both control and communication

• Allows for targeting and exploiting specific systems

• Frequently used for more sophisticated malware

• High-end botnets, spyware, backdoors

Page 50: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Noisy

• Pros

• Can dynamically alter behavior

• Can gain information about infected systems

• Allows for most sophisticated behavior

Page 51: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Controlled and Noisy

• Cons

• Most likely to be detected

• Can be disrupted or destroyed by subversion of communication mechanism

• Provides most chances for perpetrator to be caught

Page 52: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Malware

Warning signs at the network level

Page 53: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting the Act of Infection

• Look for network packets which indicate an attack or exploit

• Known bad packets

• Malformed packets

• Often requires deep packet inspection (NIDS such as Snort and Bro)

Page 54: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Types

• Probes on multiple ports from the same source (single-origin port scanning)

• Can be frustrated or defeated by a distributed scan (likely via botnet), use of proxies or anonymization services such as Tor, cooldown periods

Page 55: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Types

• Encrypted traffic on unusual ports

• Can be frustrated or defeated by tunneling through normally encrypted ports such as 443 for HTTPS

Page 56: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Types

• Requests for multiple IP addresses on the same LAN from a single source

• Can be frustrated or defeated by a distributed scan (likely via botnet) and/or use of proxies or anonymization services such as Tor if done remotely, cooldown periods

Page 57: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Types

• Requests with unusual strings and/or misspellings

• Browser type "MoZilla", "InertNet Esplorer"

• User-Agent: %^&NQvt

• Requests with unusual IP headers and/or flags

• <!--- malicious message --->

Page 58: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Volume

• Observe the (networking) behavior of a suspect machine

• Look for large traffic spikes

• Look for strange traffic behavior

Page 59: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Volume

• Large traffic spikes may indicate an attempt at a ‘fire hose’ or ‘spray and pray’ method of infection

• Large traffic spikes may also indicate cooption of system resources such as Bitcoin mining, click fraud, or distributed cryptographic attacks

Page 60: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic Volume

• Strange behavior is more subtle

• Look for port scanning behavior

• Look for network communications while the system is otherwise idle

• Look for network communications to a large number of IP addresses in a relatively short time

• ESPECIALLY if the IP addresses are sequential

• Look for network communications using unusual protocols

• IRC traffic when no IRC client is installed

Page 61: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Suspicious Traffic End Points• Blacklist approach

• Look for communication attempts with known bad IP addresses

• Look for suspicious network requests

• A DNS lookup for “pwnz0rd-j00.l33t.net” is unlikely to be a good thing

• A VPN connection being established FROM a workplace (depending on the workplace)

• Unexpected P2P or Tor traffic

Page 62: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Reverse Engineering Networking

• Given a malware binary, look for networking codeo Check for common API calls

o Identify how the malware puts networking requests togethero Create an outline of the protocol and possible values placed

in the traffico Identify how/if this differs from normal traffico Write signatures based on the differences

Page 63: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.
Page 64: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Anti Techniques

•Anti-Disassembly•Anti-Debugger•Anti-Virtual Machine

•Goal: Make it too difficult for beginners or even average malware analysts to handle

Page 65: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Anti-Disassembly

•Goal: Trick Disassemblers into showing incorrect code•Raises the bar for malware analysts•Debugging assembly is difficult enough already•Can make it too difficult for novice malware analysts….

Page 66: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Types of Disassembly

•Linear Disassembly• Disassemble one instruction at a time• Do not look at type of instruction

•Flow-oriented Disassembly• Look at instruction and disassemble based on program

flow• Used by IDA Pro and other commercial products

Page 67: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

33 C0 xor eax,eax

74 01 jz short near ptr loc+1

E9 junk

58 Pop eax

C3 retn

Jump Tricks

•Fool Linear Disassembly with jump instructions• Same target (jz and jnz)• Constant condition (xor to zero and jz)

•Example

33 C0 XOR eax, eax

74 01 jz short near ptr loc+1

E9 58 C3 68 94 jmp near ptr 94A8D521h

Page 68: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Impossible Disassembly

•Recall Assembly instructions are multiple byte lengths depending on instruction•Jump back a byte into an instruction already disassembled and use it as part of another instruction•Screws over disassemblers

Page 69: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Impossible Disassembly cont.

EB FF JMP -1

C0 48 ???

FF CO INC EAX

48 DEC EAX

66 B8 EB 05 mov ax, 5EBh

31 C0 xor eax, eax

74 F9 jz (-7)

E8 58 C3 90 90 call near ptr 98A8D525h

EB 05 jmp 5

58 Pop eax

C3 ret

Page 70: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Hiding Cross Referenced Code

•Graph view in IDA is nice but….•It is easy to hide function calls made through pointers•C++ uses this extensively•Be aware that function calls through pointers can get lost!!!

Page 71: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Fun with Ret

•When a program returns, it pops the return address from the stack and jumps execution there

004011C0 var_4 = byte ptr -4

004011C0 call $+5

004011C5 add [esp+4+var_4], 5

004011C9 ret

004011C9 sp-analysis failed

004011CA Confused IDA Pro…..

Page 72: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Handling Exception Handlers

•Exception handling is common in C•When an exception is thrown, handlers are searched by looking through a linked list data structure•New exception handlers are added to the list by appending to the end•During an exception, each exception handler is called in sequence pushing itself onto the stack

Page 73: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Stack Example

00401050 mov eax, (offset loc_40106B+1)

00401055 add eax, 14h

00401058 push eax

00401059 push large dword ptr fs:0

00401060 mov large fs:0, esp

00401067 xor ecx, ecx

00401069 div ecx

0040106B call whatever

00401070 retn

00401071 IDA Panic Time ??????

Page 74: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Screwing up Automated Function Analysis

•Automated analysis of function parameters is determined by looking at what variables are accessed

00401543 sub esp, 8

00401546 sub esp, 4

00401549 cmp esp, 1000h

0040154F jl short loc_401556

00401551 add esp, 4

00401554 jmp short loc_40155C

00401556 add esp, 104h

0040155C IDA confusion ensues…..

Page 75: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Anti-Disassembly

•Disassemblers are nice but not infallible•We get to keep our jobs ☺•Good malware analysts can recognize impossible assembly and run through the code to figure out what is going on•IDA supports manually re-classifying code as well as code replacement to “fix” problem areas

Page 76: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Anti-Debugging

•Goal is to cause malware to exit or skip important behavior during debugging•As a malware analyst, we need to find this code to get to the fun parts•Thus, we need to jump over and circumvent anti-debugging techniques

Page 77: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Windows API Calls

•IsDebuggerPresent•CheckRemoteDebuggerPresent•NTQueryInformationProcess•OutputDebugString

• Watch for fun calls to OutputDebugString• OutputDebugString(“%s%s%s%s%s”)

•If these calls exist, the program is looking for an attached debugger

Page 78: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Manual Checks for Debuggers

•Check the PEB• Structure of this header is available on MSDN• BeingDebuggedFlag: Set if process is being debugged• ProcessHeap flag: Pointer to the first entry of the heap

for a program• Contains a header with a flag telling the kernel if a debugger is

present• Offset 0x10 in XP and 0x40 in Windows 7

Page 79: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Manual Checks cont.

•NTGlobalFlag• Debuggers set different heap flags when running

programs• Typically:• Enable Tail Check• Enable Free Check• Validate parameters• All allow the debugger to watch for heap errors

Page 80: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Manual Checks cont.

•Debuggers leave residue on the system:•Check to see if the default debugger (DrWatson) has been replaced in the registry•Look for key: KHLM\SOFTWARE\Microsoft\Windows\CurrentVersion\AeDebug•Malware may also look for known Debug windows with FindWindow

Page 81: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Looking for Debugger Behavior

•Malware can scan its own code in memory looking for software breakpoints inserted by debuggers

• Int 3 (0xCC): Most common software breakpoint

•Or just calculate an MD5 checksum of the loaded memory

• If it does not match, quit

Page 82: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Debugger Behavior cont.

•Timing Checks• Debugged programs run slower especially if the analyst

is stepping through code• Strategy: Perform system time check, run code, perform

another time check• If time 2 is too much later than time 1, then quit

assuming a debugger

Page 83: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Debugger Behavior cont.

•Common ways to check system time:• Rdtsc: Number of ticks since last reboot• QueryPerformanceCounter: • GetTickCount: Windows API time since last boot in

milliseconds•Look for two calls to these functions along with a compare•Then jump over the compare to continue to the good stuff

Page 84: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Messing with Debuggers

•Thread Local Storage• Another cool place to hide code• Supposed to be used to initialize thread specific storage

variables• Of course can be used for anything and often skipped

by debuggers (debuggers break on the main code)• Easy to find requires a separate .tls section in the PE

header• If found force your debugger to load the code

Page 85: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Playing with Exceptions

•By default, debuggers break on exceptions to let the programmer view what caused the error•Timing detection works really well here since debugger (almost) always stop execution•When debugging, step back into the code’s exception handlers to make sure they are clean

Page 86: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Inserting Breakpoints

•Debuggers use 0xCC to trigger a breakpoint•Malware can insert these too!!!!•Often this is used as 0xDC (valid instruction when not debugging)•When debugged, the debugger will pause at 0xCC then set the SP to the next byte•All subsequent code is then out of alignment

Page 87: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Invalid PE headers

•Debuggers are often more strict when reading PE headers than the Windows loader•Certain Size variables have a well known maximum value that the Windows loader enforces•Debuggers can take these at face value

• NumberOfRVAandSizes• SizeOfRawData

Page 88: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Detecting Virtual Machines

•This is becoming less popular as virtual machines become more popular•In the beginning, most virtual machines were bad targets because they were power users and malware analysts•Virtualization is now popular for everyday use•Just because a system is running in a VM does not mean it is not useful now!!!!

Page 89: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

VMWare

•VMWare does not try to hide…..•Drivers are named with well known names•VMTools is commonly installed

• VMTools in program files• VMTray• System services

•Look for any string with VMWare to find these checks

Page 90: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Virtualization Artifacts

•VMWare and others run the VM in an isolated environment•The software traps most CPU calls which request hardware information through interrupts•Unfortunately, some of these instructions do not generate interrupts•To virtualize these, every instruction would need to be tested before they were run

Page 91: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Virtualization Artifacts cont

•This would be a huge performance hit!!!!•Strategy is then:

• Query one of these functions• Check through another method what the value is (Often

implemented in a virtualized fashion)• If the values differ, then we are running in a VM

Page 92: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Vulnerable Instructions

•Sidt – Store interrupt descriptor table •Sgdt – Store global descriptor table•Sldt – Store local descriptor table•Smsw – Store machine status word•Str – Store task Register•In (with second operand VX)•cpuid

Page 93: Networking & Malware CS 598: Network Security Michael Rogers & Leena Winterrowd March 26, 2013.

Circumvention

•These instructions will not normally be used in programs

•Search for them in IDA and analyze…..