Eliminating Navigation Errors in Web Applications via Model Checking and Runtime Enforcement of Navigation State Machines

Sylvain Halle, Taylor Ettema, Chris Bunch, and Tevfik Bultan

University of California, Santa Barbara

Web software

• Web software is becoming increasingly dominant

• Web applications are used extensively in many areas:– Commerce: online banking, online shopping, …– Entertainment: online music, videos, …– Interaction: social networks

• We will rely on web applications more in the future:– Health records (Microsoft HealthVault, Google Health)– Controlling and monitoring of national infrastructures (Google

Powermeter)

• Web software is also rapidly replacing desktop applications– cloud computing + software-as-service

– Google Docs, Google …

One Major Road Block

• Web applications are not dependable!

• Web applications are error prone– Many web applications have navigation errors: They mishandle

unexpected user requests

• Web applications are notorious for security vulnerabilities – Their global accessibility makes them a target for many malicious

users

• As web applications are becoming increasingly dominant and as their use in safety critical areas is increasing, their dependability is becoming a critical issue

Web applications are error prone

• Most web applications have navigation errors where an unexpected user request can cause a web application to

– display cryptic error messages

– display sensitive information that might be exploited by malicious users

– execute an unintended action

A Web Application: Bamboo Invoice

A Web Application: Bamboo Invoice

• At the top of the Bamboo Invoice project homepage, it states:

“BambooInvoice is free Open Source invoicing software intended for small businesses and independent contractors. Our number one priorities are ease of use, user-interface, and beautiful code.”

Navigation errors: Bamboo Invoice

Another Web Application: Digitalus

Another Web Application: Digitalus

• At the top of the Digitalus project homepage, it states:

“Digitalus CMS is a new kind of CMS. The focus of this open source project is usable software as opposed to endless lists of features.”

Navigation errors: Digitalus

Navigation errors: Digitalus

How Did We Generate These Screens?

• Not very difficult, just try to do something unexpected• For example

• delete yourself• try to access a page that you should not have access to

• See the step by step scenarios in the paper

• The point is:• A normal user can accidentally do these operations• A malicious user can intentionally do these operations

Why are web applications error prone?

• Script-oriented programming:– A web application consists of a collections of scripts– These scripts call each other indirectly through interaction by the

user and the browser• The form that one script generates has the address of the next

script that will consume the user input – There are no systematic checks that guarantee that the caller and

the callee agree on an interface• For example in a procedure call, the caller and the callee must

agree on the number of arguments and their types – There is no explicit control flow identifying the execution order

• The control flow is buried in the links of the generated html pages

Why are web applications error prone?

• Extensive string manipulation:– Web applications use extensive string manipulation

• To construct html pages, to construct database queries in SQL, etc.

– The user input comes in string form and must be validated and sanitized before it can be used

• This requires the use of complex string manipulation functions such as string-replace

– String manipulation is error prone

Why are web applications error prone?

• Interactivity– User interaction is not under the control of the developer

• The user can use the back button of the browser

• The user can open multiple windows

• The user can cut and paste the URL

– Imagine you develop a desktop application where all dialog boxes and all menu items could be displayed by the user at any moment...

...regardless of whether this makes sense in the current state ofthe application

Why are web applications error prone?

• Interactivity– Stateful interaction over stateless protocols (HTTP)– Interactions between different software components

• browser, server, back-end database• the need to maintain session state across these components

– One web application can be composed of many applications• Mash-ups, web services

Automated Verification to the Rescue

• What can automated verification do for you?– Exhaustive state-space exploration

• Using state space reduction techniques to enable exhaustive exploration of the state space of a program

– Symbolic analysis• Using compact symbolic representations (such as BDDS) to

explore large sate spaces– Runtime verification

• Check or enforce properties at runtime– Combining static and dynamic checks

• Check as much as possible statically, for the rest use runtime enforcement

• What can you do for automated verification?• Specify the intended behavior!

Request processing in a Web application

• Request processing in Web applications that use MVC (Model View Controller) frameworks

Navigation modeling and analysis

• We developed a simple language to specify navigation state machines– It is a state machine that shows the allowable sequences of

controller action executions in each session of a web application• MVC frameworks typically use a hierarchical structure where actions

are combined in the controllers and controllers are grouped into modules– We exploit this hierarchy to specify the navigation state machines

as hierarchical state machines

Navigation state machines

• The states of a navigation state machine are defined by – the values of the session variables, – the last action executed by the application– and the request parameters of the last action

• We assume that this information is enough to figure out what are the next actions that can be executed by the application

• NSM specification and verification is session modular

What can we do with NSMs?

• If we can check that the web application conforms to the NSM, – then we can verify navigation properties on the NSM and conclude

that the navigation properties hold for the application– We can also use automated verification techniques to check

properties of NSMs– This way we can eliminate the navigation errors

• Problem: How do we ensure that the application conforms to the NSM?– Two approaches

• Automatically extract the NSM from the application• Manually specify the NSM and use runtime enforcement to

make sure that the application follows the NSM• Or use a combination of these two

Runtime Enforcement with NSMs

• Statically verifying that a web application conforms to a navigation state machine is a very difficult problem (in general undecidable)

• So, instead, we use runtime enforcement– We have a plugin that can be easily added to an MVC web

application that takes a NSM as input and makes sure that every incoming request conforms to the NSM

– If the incoming request does not obey the NSM, then the plugin either ignores the request and refreshes the previous page or generates an appropriate error message

– This way non-compliant user requests can be handled uniformly without generating strange error messages

Model Checking NSMs

• Runtime enforcement ensures that the violations of NSM behavior will be handled uniformly at runtime

• However, we may also want to check properties of NSMs• Is the logout page always followed by the homepage?

• Our approach:• Ask developer to write properties of NSMs as temporal logic

formulas• We translate NSMs to SMV specifications• We check if the properties hold on the NSMS using the NuSMV

model checker

Overview of Our Approach

NSM to SMV Translator

NSM Plugin(NSM Interpreter)

Navigation StateMachine (NSM)Specification

SMV

Verified

CounterExampe

ACTLProperties

Static Verification

Runtime Enforcement

Some examples

• We studied three real-world, freely available web applications:

– BambooInvoice: invoice management application159,000 lines of code, 60 actions

– Capstone: student project management system41,000 lines of code, 33 actions

– Digitalus: content management system401,000 lines of code, 26 actions

Extracting the NSM

• We extracted the NSMs from the applications by hand, by exploring potential error sequences

– Amount of effort: Half a day per application (including taking screenshots, drawing the graph, etc.)

Application States Transitions Variables

Digitalus 32 48 7

BambooInvoice 63 80 8

Capstone 8 16 1

Extracting the NSM

• Most NSMs are a collection of groups of logically interrelated pages, with few entry and exit points between groups

• Extraction by hand is easier than numbers show• The NSM only needs to be a (reasonably) conservative

approximation of all the paths that the application tolerates

• Future work:

1) semi-automated extraction of NSM from source code

2) promote NSM as part of code documentation

Fragment of BambooInvoice's NSM

• Yellow transitions have guards which, if violated cause PHP warnings• Target states for red transitions cause PHP error messages• Black transitions either have guards that are handled gracefully by the

application and do not cause PHP messages or have no guard

Model Checking NSMs

• We used the NuSMV model checker to statically check navigation properties of NSMs expressed in ACTL

• Some examples:– Once you login, the only way to go back to the login page is by

traversing the logout page– Each controller has the 'index' action as its only entry point from

other controllers• The original BambooInvoice and Digitalus assume, but do not enforce

either of these properties– This is the cause for many cryptic error messages we found

• We could statically verify with NuSMV that the NSM for both applications did fulfill these properties– 4.4 MB of memory– 0.4 sec running time

Runtime enforcement of NSMs

• PHP plugin for enforcement of NSMs at runtime– Intercepts page requests in an MVC application and validates them

against the NSM (supplied in an external XML file)– One line of code to insert in MVC frameworks (Zend, CodeIgniter)– Simple: 1,100 lines of PHP code (3% of the smallest application)

• Average processing time when an action conforms to the NSM:

Application Time without plugin (ms)

Time with plugin (ms)

Digitalus 11 12

BambooInvoice 183 199

Capstone 90 122

Runtime enforcement of NSMs

• Processing time when an action provokes a PHP warning (which the NSM-enabled application blocks):

• ...and when an action provokes a PHP error:

Application Action Time without plugin (ms)

Time with plugin (ms)

Digitalus Create folder 28 26

Digitalus Upload media 18 32

BambooInvoice New invoice 938 574

Application Action Time without plugin (ms)

Time with plugin (ms)

Digitalus Edit page 416 32

Digitalus Delete folder 424 36

BambooInvoice View invoice 564 594

Runtime enforcement of NSMs

• Take-home point: runtime enforcement pays

– Reasonable overhead when everything is OK– Can actually save CPU time by sparing the application from

processing an error

• Other advantage: prevents an error from occurring, instead of recovering from it after the fact– E.g.: rolling back database operations– Catching an exception at the earliest moment vs. propagating it

deeper in the stack trace

• Reminder: also guarantees that the results obtained by static verification hold

How many errors do we prevent?

• We can provide an estimate based on the ''colored'' transitions we found while extracting the NSM

• Count the number of valid traces of length k-1 (i.e., that follow the NSM from its start state)

• Then count the ways these traces can be extended to an invalid trace of length k by executing an unexpected action

• Calculate the percentage of these unexpeted traces that are not caught by the application

• This ratio represents the proportion of traces of length k for which a navigation constraint is assumed, but not checked, by the application.

• BambooInvoice: 64% of all unexpected navigation traces longer than 4 can generate a cryptic error message

• Digitalus: 52% of all unexpected navigation traces longer than 4 can generate a cryptic error message

Related Work

• Navigation problems in Web applications have been identified a while ago [Licata and Krishnamurthi, ASE 2004]

• There are programming language based solutions for this problem that use continuations [Krishnamurthi et al. 2006]

• Modeling web applications as state machines has been proposed and investigated before [Miao, Zeng ICECCS 2008], [Han, Hofmeister MODELS 2007]

• Runtime enforcement of navigation state machines is related to earlier work on runtime monitoring and verification [see Runtime Verification Conference]

Conclusions

• Web applications suffer from weak enforcement mechanisms for valid navigation sequences; this is the source of cryptic and confusing errors

• Navigation State Machines (NSM) are finite state machines that can formally represent valid navigation paths, along with constraints on request parameters

• By combining enforcement of NSMs at runtime with static verification of NSMs, we can...– Prevent navigation errors from occuring– Verify navigation properties by model checking the NSMs rather

than the applications themselves