© 2011 Codenomicon. all rights reserved. Robustness Testing: Discover unknown vulnerabilities with...

© 2011 Codenomicon. all rights reserved.

Robustness Robustness Testing:Testing:

Discover unknown Discover unknown vulnerabilities vulnerabilities

withwithTesting & QATesting & QA

Ari TakanenCodenomicon Ltd.


Be Proactive with Security

• Modern security testing is about finding unknown zero-day vulnerabilities in devices and software before and after release

• Provides a quick technique for security assurance for any device or software

www.codenomicon.com/unknown/


Security Vulnerability = Just A Bug


Same Applies to (Legacy) Mobile Phones


The Challenge


Internet of Things = Future market for security and testing

1875 1900 1925 1950 1975 2000 2025

50 B

5.0 B

~0.5 BPLACES

PEOPLE

THINGS

Inflectionpoints

Global Connectivity

Personal Mobile

Digital Society Sustainable World

Source: Ericsson


Codenomicon Labs Test Results

http://www.codenomicon.com/labs/results


Smart phone – attack surface

WIRELESS: Bluetooth:L2CAP, RFCOMM, SDP, OPP, A2DP, AVRCP, PBAP, DUN,...

WIRELESS: 802.11:802.11a/b/g/n, WPA, WPA2,..

WIRELESS: GPRS, EDGE/3G GSM, SMS, MMS, SMIL, OTA

updates,...

PHYSICAL CONNECTIVITY:USB, SERIAL, MEMORY CARD,

SIM,..

IP CONNECTIVITY:IPv4 (ARP, ICMP, IGMP, IP,

UDP, TCP), IPv6 (IP, ICMP, ND, RD, SEND, MLD, TCP, UDP), HTTP, TLS/SSL, OCSP, RTSP,

SIP/IMS, RTP/RTCP, SigComp, DNS, MDNS, DHCP, NTP , SOAP, REST/JSON, SMTP,

POP3, IMAP4, WAP/WMLC,..

[WEB] APPLICATIONS:XML, DRM, HTML5 (CSS, HTML, Javascript) , AT commands,

inter process APIs/RPCs,

MEDIA:AUDIO (AAC, MP3, MP4, 3GP, WAV, ...), IMAGES (JPG, GIF,

PNG, TIFF, ...), VIDEO (MPG1, MPG2, MP4/H.264,

WEBM,... ), ARCHIVES (ZIP, JAR, CAB, ...), DOCUMENTS

(PDF, DOC, PPT,..), X509, EMAIL (MIME, calendar,

vcards,...), DRM, Flash, Java classes , Application

installers,...


Approaches to testing, how does fuzzing fit in?

Feature/conformance testing Performance/load testing Robustness testing

– Fuzzing – Static Code Analysis


Microsoft SDL & fuzzing & static code analysis


Microsoft SDL: Fuzz Here?

Many organizations choose to deploy fuzzingin other parts of the SDL as well.

Many organizations choose to deploy fuzzingin other parts of the SDL as well.


Definition of fuzzing

Fuzzing is a technique for – intelligently and – automatically

generating and passing into a target system – valid and – invalid

message sequences to see if the system breaks, and if it does, what it is that makes it break.


Product Security Terminology

Vulnerability – a weakness in software, a bug.

Threat/Attack – exploit against a specific vulnerability

Protocol Modeling – functional behavior, interface message sequences and message structures

Anomaly – abnormal or unexpected input

Failure – crash, busy-loop, memory corruption, or other indication of a bug in software


Types of fuzzing

Random fuzzing– Apple 1980’s– Barton P. Miller 1980’s, 1990’s

Template based fuzzing– Capture traffic OR use sample files OR...

create mutated test cases

Specification based fuzzing– Model the specification, inject

anomalies, transmit to target system


Example Fuzzing Session


What kinds of bugs does it find?


Why We Must Fuzz?

UpdateFrequency

Designing systems for very long operational and legacy device support, security?

Try to secure devices that get infrequent updates or those needing very high severity updates out of band

“Always-on” applications or devices will have to deal with live updates, no down-time and still function in rugged/robust environments

Mission critical devices will bring their own unique set of requirements – guaranteed up-time, high security and immunity from updates being an attack source


Fuzzing vs. Common Criteria

Calculation of attack potential for Fuzzing tools:Factor Open Source

FuzzersScore Commercial

FuzzersScore

Elapsed Time to Exploitation

less than a week

1 less than a day 0

Expertise Expert 6 Layman 0

Knowledge of TOE

Public 0 Public 0

Window of Opportunity

Easy 1 Easy 1

Equipment Standard 0 Specialized 4


Attack Potential for Fuzzing Tools

Attack potential for fuzzing tools is 5-8…What does that mean:

0-9 = Basic = AVA_VAN.1-5 should not fail10+ = Enhanced Basic required at EAL4All Common criteria evaluated products should survive basic attacks such as fuzz-testing?


“Models” and “Rules”


Scaling Fuzz Tests

Robustness is also about performance, and therefore model-based tools have to be fast in generating test cases

Robustness is also about performance, and therefore model-based tools have to be fast in generating test cases


Testing In The Cloud

Test the interior and exterior of the cloud, including services, devices, applications, and hypervisor stability

Test the interior and exterior of the cloud, including services, devices, applications, and hypervisor stability


Model-based Fuzz-Testing Examples

• ”[FUZZING] tools are *amazing*. Using them is like being attacked by the most relentless adversary who uses every possible method to find flaws in your code

We fixed subtle crash bugs in Samba that had been in the code for over ten years. We would *never* have found those bugs without the [FUZZING] tools.

If you're serious about implementing protocols correctly, you need [FUZZING] tools.“

-- Jeremy Allison, Co Creator of Samba.


Model-based Fuzz-Testing Examples


Conclusions

Why is fuzzing always an excellent choice for a testing solution... – ... and sometimes the only feasible one?

• Easy to automate, systematic, top coverage,

top efficiency• Increasingly widely adopted, some

contractors/customers require it• Real life examples indicate: you will find

security critical bugs by fuzzing

© 2011 Codenomicon, Ltd. 32

DEFEND. THEN DEPLOY.

PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS

THANK YOU – QUESTIONS?

“Thrill to the excitement of the chase! Stalk bugs with care, methodology, and

reason. Build traps for them. ....

Testers!Break that software (as you must) and

drive it to the ultimate- but don’t enjoy the programmer’s pain.”

[from Boris Beizer]

© 2011 Codenomicon. all rights reserved. Robustness Testing: Discover unknown vulnerabilities with...

Documents

Transcript of © 2011 Codenomicon. all rights reserved. Robustness Testing: Discover unknown vulnerabilities with...