AIT 681 Secure Software Engineering...Security Testing •Look for unexpected but intentional...
Transcript of AIT 681 Secure Software Engineering...Security Testing •Look for unexpected but intentional...
AIT 681 Secure Software Engineering
Topic #5. Seven Software Security Touchpoints (II)
Instructor: Dr. Kun Sun
Reading
• This lecture– McGraw: Ch.6
2
Seven Touchpoints
1. Code review2. Architectural risk analysis3. Penetration testing4. Risk-based security testing5. Abuse cases6. Security requirements7. Security operations
Plus external analysis
3
3. Penetration Testing
4
Penetration Testing
• Artifact: system running in its environment• Understanding fielded software in its environment• Information supplied by architectural risk analysis• Who does it?• White/Black Hat
Requirement and Use cases
Architecture and Design Test Plans Code Tests and
Test ResultsFeedback from
the Field
5. Abuse cases
6. Security Requirements
2. Risk Analysis
External Review
4. Risk-Based Security Tests
1. Code Review(Tools)
2. Risk Analysis
3. Penetration Testing
7. Security Operations
5
Software Testing
• Application fulfills functional requirements• Dynamic, functional tests late in the SDLC• Contextual information
6
Security Testing
• Look for unexpected but intentional misuses of the system
• Must test for all potential misuse types using– Architectural risk analysis results– Abuse cases
• Verify that – All intended security features work (white hat)– Intentional attacks cannot compromise the system
(black hat)
7
Penetration Testing
• Testing for negative – what must not exist in the system
• Difficult – how to prove “non-existence”?• If penetration testing does not find errors then – Can conclude that under the given circumstances
no security faults occurred – Little assurance that application is immune to
attacks
• Feel-good exercise
8
What is Pen Testing?
• Penetration testing assesses security by actively trying to find exploitable vulnerabilities– Black hat activity (for a good purpose). Practitioners
variously called red teams, tiger teams, etc.
• Can be applied at different levels of granularity– program (single process)– complete application (communicating processes)– network of many applications– generally not libraries or incomplete pieces of code
9
Pen Testing is Both Art and Science
• Humans probe and interact with a system, looking for different weaknesses or attack vectors– Employing cleverness, adaptation, ingenuity
• Once patterns of exploration and exploitation emerge, you write computer programs (tools) to do the work– Ingenuity automated
Science is what we understand well enough to explain to a computer. Art is everything else we do. —Donald Knuth
10
Who and How?
• Pen testers employ ingenuity and automated tools– To rapidly explore a system’s attack surface, looking
for weaknesses to exploit • Typically carried out by a separate group within
or outside an organization, separated from developers– Avoids tunnel vision: Same reason doctors tend to not
treat themselves or their own families• Given varied access to system internals (threat
model)– From no access, like outside attacker, to full access,
like a knowledgeable insider
11
History of Pen Testing
• 1967 Ware Report– Task force of experts headed by Willis Ware of RAND
Corp. formally assessed the security problem for time-sharing computer systems. Used term “penetration”• http://www.rand.org/pubs/reports/R609-1/index2.html
• 1970s: DOD penetration testing teams emerge to assess “real” security of government computer systems
• Today: Penetration testing is expanding– Popular with students, e.g., “CTF” competitions– Many companies can be contracted to do it– IACRB Certified Penetration Tester (CPT)
• http://www.iacertification.org/cpt_certified_penetration_tester.html
12
Success of Penetration Testing
• Depends on skill, knowledge, and experience of the tester
• Important! Result interpretation• Disadvantages of penetration testing: – Often used as an excuse to declare victory and go
home– Everyone looks good after negative testing results
13
Testing Process
• External Testing: across the internet.– Simulate attacker’s environment– Gathering information related to remote access, IP
addresses, open ports, allowed services, etc.– Tools to support
• Internal Testing: onsite. – View of the system behind the external perimeters– Software penetration testing tools– Attempt to exploit vulnerabilities
14
Testing Activities
• Scoping: assessing target system• Discovery: building information about the
system– Offline and online activities
• Vulnerability scanning: testing system components
• Target penetration: within testing parameters• Analysis: of results of previous stages• Reporting: detailed findings and
recommendations
15
Pen Tester’s Bag of Tricks
• A pen tester approaches a target knowing– the workings of the target domain (e.g., the web)– how systems are built in that domain• Protocols (e.g., HTTP, TCP, …)• Languages (e.g., PHP, Java, Ruby, …)• Frameworks (e.g., Rails, Dream Weaver, Drupal)
– common weaknesses in the software/system• Bugs (e.g., SQL injections, XSS, CSRF, …)• Misconfigurations, bad design (e.g., default passwords,
“hidden” files, …)
16
Web Hacking: A Professional’s View
70% messing with parametersIf the URL is http://tgt.com/buy?item=1&price=5.00
Then change it to: • /buy?item=1&price=0.01• /buy?item=10&price=5.00• /buy?item=1&price=5.00<script>alert(“test”);<
/script>
• /buy?item=1&price=5.00’
Client parameters (unwisely) trusted?
Susceptible to XSS?
Susceptible to other injection?
17
Web Hacking: A Professional’s View
10% default passwords– Always research the default password and try it • Works way more often than you’d think
10% hidden files and directories– Look through the manuals for clues– Directory brute forcing
10% other– Authentication problems (bypass, replay, …)– Insecure web services– Configuration page gives away your root password
18
Benefits
• Penetrations are certain and reproducible,demonstrated by tests– Not hypothetical– Applied to a whole component• not (code) fragments
– No false alarms
• “Feel good” factor– Produces evidence of real vulnerabilities that
would otherwise have gone unfixed– Thus results in a clear improvement to security
Beware of bugs in the above code; I have only proved it correct, not tried it —Donald Knuth, 1977
19
Drawbacks
• Absence of penetrations is not evidence of security– After fixing any issues there may be others still lurking
• Changes to the system necessitate a retest– Security is not compositional: a change to one
component may render another component insecure• So must retest the entire system
– But changes are common!• Can be expensive to retest too frequently
• Nevertheless, penetration testing worth doing
20
Pen Testing Tools
• Pen testers use tools to– Probe a target– Gather information and test hypotheses about it– Exploit a vulnerability (or attempt to)
• Which tool depends on the goal, and the target– If an enterprise network, want to find, probe, and
exploit machines, routers, topology, etc.– If a single machine, want to consider installed
software, running programs, interesting files– if a single program, want to explore and exploit
possible inputs and interactions
21
Sample Tools
• Science is captured in tools. • We’ll briefly consider
• Nmap for network scanning• Zap web proxy for probing, exploitation• Metasploit for general-purpose exploitation• … and provide pointers to more tools
22
A comprehensive list is at http://sectools.org/
Nmap for Network Probing
• Nmap stands for “network mapper”. Figures out– what hosts are available on the network, – what services (application name and version) those
hosts are offering, – what operating systems (and OS versions) they are
running, – what type of packet filters/firewalls are in use– … and more
• Works by sending raw IP packets into the network and observing the effects
• Free, open source (commercial tools too)http://nmap.org/
23
Finding Hosts, Services
• Nmap will ping a specified range of IP addresses– ICMP Echo Request and/or Timestamp request
• Standard “ping” protocol– TCP SYN to port 443, TCP SYN/ACK to port 80
• Looking for running HTTPS or HTTP servers– Other things, as determined by the operator
• Protocol-specific UDP packets to particular ports• Probes to other TCP ports• Probes that elicit different responses on different OSes
(“fingerprinting”)
• Be stealthy– A flurry of scanning activity may be detected– Control the rate of scanning to “work under the radar”
24
Web Proxies
• Web applications are common pen testing targets
• Web proxies sit between the browser and server– Displaying exchanged packets– Modifying them as directed by the tester
Proxy
Server
Some proxies have additional features for vulnerability scanning/exploitation, site probing, etc.
25
Zap
• OWASP Zed Attack Proxy (Zap)– GUI-based inspection/modification of captured packets – Can set “breakpoints” to allow packets through until a
certain condition is met• Additional features– Active scanning: attempts XSS, SQL injection, etc.– Fuzzing: context-specific payloads– Spider: explores a site to construct a model of its structure
• Free, open-source https://github.com/zaproxy/zaproxy
• See also the Burp suitehttp://portswigger.net/burp/
26
Metasploit
• Metasploit advanced open-source platform for developing, testing, and using exploit code. – Boasts an extensible model through which payloads,
encoders, no-op generators, and exploits can be integrated
• Scripting attacks– Probe remote site looking for vulnerable services– Construct payload based on versions, other features– Encode payload to avoid detection– Inject payload– Wait for shellcode to connect back; command
prompt!
27
Metasploit
1. Probe vulnerable target
28
Metasploit
2. Generate attack payload
3. Send payload
4. Trigger vulnerability and compromise target
29
Metasploit UI
• msfconsole — interactive console for executing metasploit commands– Also web-based frontend and command-line interface– Supports probing and communications commands,
payload construction (and encoding)– Supports active (go get `em) and passive (wait til they
come to us) attacks• Meterpreter - command processor injected into
the target, e.g., in the memory of a compromised process– Permits the pen tester to probe more stealthily
• msfpayload, msfencode — generate (stealthy) shellcode
30
100’s of Modules, Scripts
• Exploits against particular vulnerabilities– Along with stagers and other modifiers to generalize
these exploits to different platforms
• Password sniffing – Reading unprotected passwords off of the network
• Privilege escalation– After penetrating, try to get SYSTEM privileges
• Keylogging and backdoors– For persistent presence
• … and much morehttp://www.offensive-security.com/metasploit-unleashed/Main_Page
31
Kali
• Kali is a Linux distribution with many open-source pen testing tools installed and configured
• The ones we have already mentioned– Nmap, Zap, Metasploit, Burp Suite
• and dozens more– John the Ripper for password cracking– Valgrind for dynamic binary analysis– Reaver for Wifi password cracking– peepdf for scanning PDF files for attack vectors– … and more
http://www.kali.org/32
Ethical Hacking
• Penetration testing tools are meant to reveal security vulnerabilities– So they can be fixed– Not so they can be exploited in the wild
• But people use tools for nefarious purposes– Don’t be one of them!
Beware the dark side!
33
Fuzzing
34
What is Fuzzing?
• A kind of random testing– Find improperly handled inputs, where failure implies
a good chance for exploitation• Goal: make sure certain bad things don’t happen,
no matter what– Crashes, thrown exceptions, non-termination– All of these things can be the foundation of security
vulnerabilities• Complements functional testing– Test features (and lack of misfeatures) directly– Normal tests can be starting points for fuzz tests
35
Kinds of Fuzzing
• Black box– The tool knows nothing about the program or its input– Easy to use and get started, but will explore only
shallow states unless it gets lucky• Grammar based – The tool generates input informed by a grammar– More work to use, to produce the grammar, but can
go deeper in the state space• White box– The tool generates new inputs at least partially
informed by the code of the program being fuzzed– Often easy to use, but computationally expensive
36
Fuzzing Inputs
• Mutation– Take a legal input and mutate it, using that as input– Legal input might be human-produced, or automated,
e.g., from a grammar or SMT solver query• Mutation might also be forced to adhere to grammar
• Generational– Generate input from scratch, e.g., from a grammar
• Combinations– Generate initial input, mutate, generate new inputs, …– Generate mutations according to grammar
37
File-based Fuzzing
• Mutate or generate inputs• Run the target program with them• See what happens
38
Examples: Radamsa and Blab
• Radamsa is a mutation-based, black box fuzzer– Mutates inputs that are given, passing them along
• Blab generates inputs according to a grammar (grammar-based), specified as regexps and CFGs
% echo "1 + (2 + (3 + 4))" | radamsa --seed 12 -n 45!++�� (3 + -5))1 + (3 + 41907596644)1 + (-4 + (3 + 4))1 + (2 +�� (3 + 4% echo … | radamsa --seed 12 -n 4 | bc -l
https://code.google.com/p/ouspg/wiki/Radamsa https://code.google.com/p/ouspg/wiki/Blab
% blab -e '(([wrstp][aeiouy]{1,2}){1,4} 32){5} 10’soty wypisi tisyro to patu
39
Example: American Fuzzy Lop
• It is a mutation-based, white-box fuzzer. Process: – Instrument target to gather run-time information
• Tuple of <ID of current code location, ID last code location>
– Run a test. Mutate test input to create a new one if unseen tuple generated; otherwise discard the test• Mutations include bit flips, arithmetic, other standard stuff
– Periodically cull gathered tests, to avoid local minima% afl-gcc -c … -o target% afl-fuzz -i inputs -o outputs targetafl-fuzz 0.23b (Sep 28 2014 19:39:32) by <[email protected]>[*] Verifying test case 'inputs/sample.txt'...[+] Done: 0 bits set, 32768 remaining in the bitmap. …———————Queue cycle: 1n time : 0 days, 0 hrs, 0 min, 0.53 sec …
http://lcamtuf.coredump.cx/afl/
40
Network-based fuzzing
• Act as 1/2 of a communicating pair– Inputs could be produced by replaying previously
recorded interaction, and altering it, or producing it from scratch (e.g., from a protocol grammar)
41
Network-based fuzzing
• Act as a “man in the middle”– mutating inputs exchanged between parties
(perhaps informed by a grammar)
42
Example: SPIKE
• SPIKE is fuzzer creation kit, providing a C-language API for writing fuzzers for network based protocols
s_size_string(“post”,5);s_block_start(“post”);s_string_variable(“user=bob”);s_block_end(“post”);spike_tcp_connect(host,port);spike_send();spike_close_tcp();
backpatch for length field… for this block
prefix of string to fuzzend of block; include length
connect to serversend this buffer
close connection
http://www.immunitysec.com/downloads/SPIKE2.9.tgzhttp://resources.infosecinstitute.com/intro-to-fuzzing/
43
Example: Burp Intruder
• Burp automates customized attacks against web applications
• Similar to SPIKE in allowing the user to craft the template of a request, but leave “holes” (called payloads) for fuzzing– Nice GUI front end
• Integrates with the rest of the Burp Suite, which includes a proxy, scanner, spider, and more
http://portswigger.net/burp/intruder.html44
Dealing with Crashes
• You fuzz. A crash occurs. Questions:• What is the root cause (so it can be fixed)?– Is there a way to make the input smaller, so it is
more understandable?– Are two or more crashes signaling the same bug?• Yes, if they “minimize” to the same input
• Does the crash signal an exploitable vulnerability?– Dereferencing NULL is rarely exploitable– Buffer overruns often are
45
Finding Memory Errors
• Compile the program with Address Sanitizer (ASAN)– Instruments accesses to arrays to check for overflows,
and use-after-free errors– https://code.google.com/p/address-sanitizer/
• Fuzz it• Did the program crash with an ASAN-signaled
error? Then worry about exploitability• Similarly, you can compile with other sorts of
error checkers for the purposes of testing– E.g., valgrind memcheck http://valgrind.org/
46
A few Other Fuzzers
• CERT Basic Fuzzing Framework (BFF)– Based in part on Zzuf (http://caca.zoy.org/wiki/zzuf)– Found bugs in Adobe Reader and Flash Player, Apple
Preview and QuickTime, and others– https://resources.sei.cmu.edu/library/asset-
view.cfm?assetID=507974
• Sulley ( https://github.com/OpenRCE/sulley) provides lots of extras to manage the testing process:– watches the network and methodically maintains records – instruments and monitors the health of the target, capable
of reverting to a known good state– detects, tracks and categorizes detected faults– fuzzes in parallel, if desired
There are many more …47
Summary
• Penetration testers simulate real attackers– Try to find exploitable vulnerabilities in complete
systems
• Penetrations signal real problems– Lack of penetrations is not proof of impossibility
• Pen testers employ a variety of tools– Scanners, proxies, exploit injectors, fuzzers
• And require ingenuity and guile
48