Java PathFinder and Model Checking of Programs
-
Upload
byron-maynard -
Category
Documents
-
view
40 -
download
0
description
Transcript of Java PathFinder and Model Checking of Programs
Carnegie Mellon University
Java PathFinder and
Model Checking of Programs
Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh, Corina Pasareanu, Charles Pecheur, John Penix, Willem Visser
NASA Ames Research Center
Automated Software Engineering Group
Alex Groce, Flavio Lerda
Carnegie Mellon University
School of Computer Science
Matt Dwyer, John Hatcliff
Kansas State University
Department of Computing and Information Sciences
Carnegie Mellon University
Outline
• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking
Carnegie Mellon University
Motivation
• Software errors are expensive– Mars Polar Lander– Ariane 501
Software bugs in space do not fly
Carnegie Mellon University
Model Checking
• Verification and Validation are crucial– Model checking has been shown effective
OK
Error traceorFinite-state model
Temporal logic formula
Model Checker
Line 5: …Line 12: …Line 15:…Line 21:…Line 25:…Line 27:… …Line 41:…Line 47:…
Carnegie Mellon University
The dream
• Model Check Programs
OK
Error traceorProgram
Temporal logic formula
Model Checker
Line 5: …Line 12: …Line 15:…Line 21:…Line 25:…Line 27:… …Line 41:…Line 47:…
void add(Object o) { buffer[head] = o; head = (head+1)%size;}
Object take() { … tail=(tail+1)%size; return buffer[tail];}
Carnegie Mellon University
Some of the Issues
• Semantics Gap– Programming Languages
vs.
Modeling Languages
• Complexity
• Not Automated
void add(Object o) { buffer[head] = o; head = (head+1)%size;}
Object take() { … tail=(tail+1)%size; return buffer[tail];}
Gap
Carnegie Mellon University
Outline
• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking
Carnegie Mellon University
Model Checking and Testing
• Software complexity is too high
• Some of the presented methods are not sound
• This is not model checking anymore
• It is “automated” testing
Carnegie Mellon University
The assumption
• Programs have bugs– Knowing that there are doesn’t mean knowing
where they are
• Testing is not always effective– Requires a lot of knowledge of the system
• Model checking can be used to find bugs systematically– If no bug is found we have a non-result
Carnegie Mellon University
Coverage Metrics
• Testing has coverage metrics– They tell you how good your testing is
– They can be used to measure confidence
• Testing is not very effective for concurrent systems– You don’t just have to guess the inputs but also the
timing of the inputs and the scheduling
• Model checking can address these issues– We are still missing metrics for concurrent programs
Carnegie Mellon University
Bug hunting
• Bug hunting instead of trying to prove something correct– We can accept unsound methods– We may be able to handle real world examples– If we allow for modeling we are still not
checking the correctness of the system itself
Carnegie Mellon University
Outline
• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking
Carnegie Mellon University
Model Checking for Java
• Explicit State Model Checker
• Java Bytecode as Input Language
• Assertions, Deadlock Freedom,
LTL Properties
• Source Level Error Trace
• Special JVM– Allows guided execution
SpecialJVM
SpecialJVM
ModelChecker
ModelChecker
StateSpace
StateSpace
ClassesBytecode
ClassesBytecode
Carnegie Mellon University
Architecture
Generic Verification EnvironmentGeneric Verification Environment
GenericGeneric C++C++
CC
JavaJavaSearch Algorithms(model checking,
testing)
Search Algorithms(model checking,
testing)
StorageSubsystem
(hash table, bitstate hashing)
StorageSubsystem
(hash table, bitstate hashing)
SpecialJVM
SpecialJVM
ClassLoader
ClassLoader
Expression EvaluatorExpression Evaluator
Carnegie Mellon University
Outline
• Motivation
• Model Checking and Testing
• Java PathFinder
• Program Model Checking
Carnegie Mellon University
Programs are complex
• Enabling Technologies– Slicing– Abstractions– State Compression– Partial Order Reduction– Heuristic Search
Carnegie Mellon University
Property-directed Slicing
• Slicing criterion automatically generated
• Backwards slicing automatically finds dependencies
Resultingslice
Slice
Source programmentionedin property
indirectlyrelevant
Carnegie Mellon University
Abstractions
• Remove behaviors but preserve errors– manual or partially automated
• Over-approximation– Preserve correctness– Type-based abstractions– Predicate abstraction– Semi-automated
Carnegie Mellon University
JPF Predicate Abstraction
• Annotation used to indicate abstractions
• Source-to-source translation
• Java PathFinder can find abstract error traces
…Abstract.remove(x);Abstract.remove(y);Abstract.addBoolean(“EQ”, x==y);…
…Abstract.remove(x);Abstract.remove(y);Abstract.addBoolean(“EQ”, x==y);…
Carnegie Mellon University
Choice-bounded Search
• An abstract trace that does not contain any non-deterministic choice correspond to at least one concrete trace
• Bias the model checker to look only choice-free traces
Carnegie Mellon University
Storing the States
• States are complex objects– Classes, Instances, Threads, Stack Frames
ClassesClasses
ObjectsObjects
ThreadsThreadsThread
Stack Frame (Locals, Stack)
Stack Frame (Locals, Stack)
Stack Frame (Locals, Stack)
Thread
Stack Frame (Locals, Stack)
Stack Frame (Locals, Stack)
ClassFields/Methods
ObjectFields/Methods
ObjectFields/Methods
ClassFields/Methods
Carnegie Mellon University
State Compression• Instructions modify only part of a state
• Different states share common subparts
X0 X1X = X + 1
X11Y27Z75T45W11
X11Y27Z75T45W11
Carnegie Mellon University
State CompressionClassFields
ObjectFields
ClassMonitors
ObjectMonitors
ThreadData
StackFrames
State
Pools
Array
Compression is very effective: up to 94%!
Carnegie Mellon University
Partial Order Reduction
• Do not explore “equivalent” traces
• Requires analysis before model checking
X=11Y=28
X=12Y=27
X=11Y=27
X=12Y=28
X++
Y++X++
Y++
Access to local variable is perfect candidate for partial order reduction.
Java does not provide enough information.
Assume that every access to a shared object is made in mutual exclusion.
Massive use of partial order reduction.
Use lockset algorithm to check that mutual exclusion is actually present.
Carnegie Mellon University
Heuristic Search• Depth first search leads to very long counter examples• Reactive system often exhibit periodic behavior• It is possible to discover errors at a shorter depth
• Heuristic Search– Breadth first like state generation– Priority queue for the states based on some heuristic
• The challenge– Find good heuristics:
• Based on the property being checked• Based on the program structure• JPF offers an API for user-defined heuristics
Carnegie Mellon University
An example
• DEOS– Real time OS from Honeywell– 1500 lines of code– Subtle concurrency error
• Testing did not reveal it• We (re)discovered the bug!
– Dependency analysis– Type abstraction– Choice-free heuristic
Carnegie Mellon University
Conclusion
• Model check programs poses some specific issues– Some we can deal with
– Some we looked for a way around
• Model checking can be used for systematic testing– Can be automated
– Can handle concurrent systems
• This is still work in progress!
Carnegie Mellon University
Future directions
• Apply the same techniques to C/C++– Next summer internship proposal
• Combine property and heuristic specification– Allow the model checker to direct the search
• Combine coverage, model checking and runtime analysis– Develop metrics– Check the system under certain assumptions