Penetration Testing with Improved Input Vector Identification

William G.J. Halfond, Shauvik Roy Choudhary, and Alessandro Orso

College of ComputingGeorgia Institute of Technology

2

Web Application Overview

DB

Other Systems

Web Server

End Users

HTTP Requests

HTML Pages

Web Application

HTML

Servlets

3

Penetration Testing Overview

DB

Other Systems

White HatTester

!@#$

Secret Data!

Web Application

HTML

Servlets

Penetration Testing Phases

White HatTester

Web Applicatio

nHTML

Servlets

InformationGathering

AttackGeneration

ResponseAnalysisReport

TargetSelection

AnalysisFeedback

Information Attacks

Responses

public void service(HttpServletRequest req) 1. String action = req.getParameter(“userAction”) 2. if (action.equals(“createLogin”)) 3. String password = req.getParameter(“password”) 4. String loginName = req.getParameter(“login”) 5. if (isInteger(password)) 6. db.execute(“insert into UserTable ” + “(login, password) values (” + loginName + “, ” + password + “)”) 7. displayAddressForm() 8. else 9. displayErrorPage(“Bad password.”)10. else if (action.equals(“provideAddress”)) 11. String loginName = req.getParameter(“login”)12. String address = req.getParameter(“address”)13. db.execute(“update UserTable set” + “ address =’” + address + “’” + “where loginName=” + loginName)14. else15. displayCreateLoginForm()

Example Web Application Code

!

Our Approach

Improvements to penetration testing:1. Information gathering Static interface analysis2. Attack Generation Generate realistic test-inputs3. Response Analysis Produce observable side

effect of attack

Goal:Improve penetration testing by improving information gathering and response analysis.

7

InterfacesInterfaceAnalysis

[FSE 2007]

1) Information Gathering: Interface Analysis

Phase 1: Identify Input Parameters (IP) namesPhase 2: Compute IP domain informationPhase 3: Group IP into distinct interfaces

Web Application

HTML

Servlets

Compute IP Domains

Group IPs

Identify IP Names

1) Interface Analysis: Identify IP Names public void service(HttpServletRequest req) 1. String action = req.getParameter(“userAction”) 2. if (action.equals(“createLogin”)) { 3. String password = req.getParameter(“password”) 4. String loginName = req.getParameter(“login”) 5. if (isInteger(password)) 6. db.execute(“insert into UserTable ” + “(login, password) values (” + loginName + “, ” + password + “)”) 7. displayAddressForm() 8. else 9. displayErrorPage(“Bad password.”)10. else if (action.equals(“provideAddress”)) 11. String loginName = req.getParameter(“login”)12. String address = req.getParameter(“address”)13. db.execute(“update UserTable set” + “ address =’” + address + “’” + “where loginName=” + loginName)14. else15. displayCreateLoginForm()

userAction

login

address

login

password

1) Interface Analysis: Compute IP Domains

userAction

login

login

address

userAction:String{“createLogin”, “provideAddress”}

passwordpassword:Stringpassword:Integer

login:String

login:String

address:String

public void service(HttpServletRequest req) 1. String action = req.getParameter(“userAction”) 2. if (action.equals(“createLogin”)) 3. String password = req.getParameter(“password”) 4. String loginName = req.getParameter(“login”) 5. if (isInteger(password)) 6. db.execute(“insert into UserTable ” + “(login, password) values (” + loginName + “, ” + password + “)”) 7. displayAddressForm() 8. else 9. displayErrorPage(“Bad password.”)10. else if (action.equals(“provideAddress”)) 11. String loginName = req.getParameter(“login”)12. String address = req.getParameter(“address”)13. db.execute(“update UserTable set” + “ address =’” + address + “’” + “where loginName=” + loginName)14. else15. displayCreateLoginForm()

1) Interface Analysis: Group IPs public void service(HttpServletRequest req) 1. String action = req.getParameter(“userAction”) 2. if (action.equals(“createLogin”)) { 3. String password = req.getParameter(“password”) 4. String loginName = req.getParameter(“login”) 5. if (isInteger(password)) 6. db.execute(“insert into UserTable ” + “(login, password) values (” + loginName + “, ” + password + “)”) 7. displayAddressForm() 8. else 9. displayErrorPage(“Bad password.”)10. else if (action.equals(“provideAddress”)) 11. String loginName = req.getParameter(“login”)12. String address = req.getParameter(“address”)13. db.execute(“update UserTable set” + “ address =’” + address + “’” + “where loginName=” + loginName)14. else15. displayCreateLoginForm()

userAction

login

login

address

userAction:String{“createLogin”, “provideAddress”}

passwordpassword:Stringpassword:Integer

login:String

login:String

address:String

1

14

10

2

15

11

12

13

4

3

5

7

6

9

8

1) Information Gathering: Summary

Interface Parameter Domain Relevant Values

1

userAction String “createLogin”, “provideAddress”

login String

password Integer

2

userAction String “createLogin”, “provideAddress”

login String

address String

3 userAction String “createLogin”, “provideAddress”

2) Attack Generation

White HatTester

Interface userAction login password

userAction = ?login = <attack string> password = ?

IP Domain Information

userAction = createLoginlogin = <attack string> password = 1234

3) Response Analysis with WASP

WASP:1. Positive tainting: Identify and mark

developer-trusted strings. Propagate taint markings at runtime

2. Syntax-Aware Evaluation: Check that all keywords and operators in a query were formed using marked strings

Response Analysis:1. Send attack to web application2. If WASP detects attack

1. Block attack2. Send out-of-band signal

3. Check for signal on client side

public void service(HttpServletRequest req) 1. String action = req.getParameter(“userAction”) 2. if (action.equals(“createLogin”)) { 3. String password = req.getParameter(“password”) 4. String loginName = req.getParameter(“login”) 5. if (isInteger(password)) 6. db.execute(“insert into UserTable ” + “(login, password) values (‘” + loginName + “’, ” + password + “)”) 7. displayAddressForm() 8. else 9. displayErrorPage(“Bad password.”)10. else if (action.equals(“provideAddress”)) 11. String loginName = req.getParameter(“login”)12. String address = req.getParameter(“address”)13. db.execute(“update UserTable set” + “ address =’” + address + “’” + “where loginName=” + loginName)14. else15. displayCreateLoginForm()

3) WASP: Identify Trusted Data

update userTable set address = ‘Home’ where

login = ‘GJ’ ; drop table userTable -- ’

update userTable set address = ‘Home’ where login = ‘GJ’

3) WASP: Syntax Aware Evaluation

Legitimate Query:

Attempted SQL Injection:

Input: login = “GJ”, address = “Home”

Input: login = “GJ’ ; drop table userTable -- ”, address = “Home”

16

Empirical Evaluation

Goal: Evaluate the usefulness of our approach as compared to a traditional penetration testing approach.

Research Questions (RQ):1. Runtime of analysis2. Thoroughness of the penetration testing3. Number of vulnerabilities discovered

Implementation: Baseline Approach

• Information Gathering OWASP WebScarab• Widely used code-base• Actively maintained

• Attack Generation SQLMap• Widely used penetration testing tool• Commonly used attack generation heuristics

• Response analysis WASP[FSE 2006]

SQLMap++ SQLMap integrated with OWASP WebScarab Spider

Implementation: Our Approach

• Analyzes bytecode of Java Enterprise Edition (JEE) based web applications

• Interface analysis WAM[FSE 2007]

• Attack generation leverages SQLMap• Response analysis WASP[FSE 2006]

SDAPT Static and Dynamic Analysis-based Penetration Testing

Subject Applications

Subject LOC Classes ServletsBookstore 19,402 28 27

Checkers 5,415 59 32

Classifieds 10,702 18 18

Daffodil 18,706 119 70

Employee Directory 5,529 11 9

Events 7,164 13 12

Filelister 8,671 41 10

Office Talk 4,670 63 39

Portal 16,089 28 27

RQ1: Runtime

Bookstore Checkers Classifieds Daffodil Empl. Dir Events Filelister Officetalk Portal1

10

100

1000

10000Analysis Time (s)

SQLMAP++SDAPT

• SDAPT ranged from 8 to 40 mins• Positive note: Testing was more thorough

RQ2: Thoroughness

Bookstore Checkers Classifieds Daffodil Empl. Dir Events Filelister Officetalk Portal0

50

100

150

200

250Number of Input Vectors SQLMAP++

SDAPT

Bookstore Checkers Classifieds Daffodil Empl. Dir Events Filelister Officetalk Portal0

10

20

30

40

50Number of Components SQLMAP++

SDAPT

RQ3: Number of Vulnerabilities

Bookstore Checkers Classifieds Daffodil Empl. Dir. Events Filelister Officetalk Portal0

2

4

6

8

10

12

14

16

18Number of Discovered Vulnerabilities

SQLMAP++

SDAPT

Average increase: 246%

Summary of Results

• Improvements to penetration testing• Information gathering with static analysis• Response analysis with dynamic detection

• Relatively longer analysis time• More thorough and more vulnerabilities

discovered during penetration testing