Automated Formal Verification of Software

38
Automated Automated Formal Verification Formal Verification of Software of Software http://www.cs.cmu.edu/~bryant Randal E. Bryant Carnegie Mellon University

description

Automated Formal Verification of Software. Randal E. Bryant Carnegie Mellon University. http://www.cs.cmu.edu/~bryant. The Dream … and the Reality. Programs Should be Formally Specified and Proved 1960s: Logical foundations of programs Floyd, Hoare, Dijkstra, Scott … - PowerPoint PPT Presentation

Transcript of Automated Formal Verification of Software

Page 1: Automated Formal Verification of Software

AutomatedAutomatedFormal VerificationFormal Verification

of Softwareof Software

AutomatedAutomatedFormal VerificationFormal Verification

of Softwareof Software

http://www.cs.cmu.edu/~bryant

Randal E. BryantCarnegie Mellon University

Page 2: Automated Formal Verification of Software

– 2 –

The Dream … and the RealityThe Dream … and the Reality

Programs Should be Formally Specified and ProvedPrograms Should be Formally Specified and Proved

1960s: Logical foundations of programs1960s: Logical foundations of programs Floyd, Hoare, Dijkstra, Scott …

1970s: Attempts to automate proof process1970s: Attempts to automate proof process Based on automated theorem provers Intensive human effort required, even for small programs

1980s, 90s: General disillusionment1980s, 90s: General disillusionment Hard to fully describe what program should do Software is far to complex to verify completely

Page 3: Automated Formal Verification of Software

– 3 –

Changing Software EnvironmentChanging Software Environment

Early IT SystemsEarly IT Systems Expert users Understood and expected failures

Modern EnvironmentModern Environment More demanding users

Nontechnical people less tolerant of errors

Safety critical systemsComputers control everything from pacemakers to stock exchanges

Hostile operating conditionsSystems under constant attack by malicious forcesSoftware errors make systems vulnerable

Page 4: Automated Formal Verification of Software

– 4 –

Use of Formal Verification for HardwareUse of Formal Verification for HardwareEnvironmentEnvironment

Higher need for first-time success Simulation speeds limit amount of testing achievable

Equivalence CheckersEquivalence Checkers Prove that low-level implementation matches high-level

descriptionRoutinely performed on all designs

Component VerificationComponent Verification Floating-point units, memory arrays, … Common practice for large manufacturers

Model CheckingModel Checking Automatically check specific properties of system

Page 5: Automated Formal Verification of Software

– 5 –

Temporal Logic Model CheckingTemporal Logic Model Checking

Verify Reactive SystemsVerify Reactive Systems Construct state machine representation of concurrent

systemNondeterminism expresses range of possible behaviors

Express desired behavior as formula in temporal logic Determine whether or not property holds

Traffic LightController

Design

Traffic LightController

Design

“It is never possible to have a green light for both N-S and E-W.”

ModelChecker

True

False+ Counterexample

Page 6: Automated Formal Verification of Software

– 6 –

Hardware Modeling ExampleHardware Modeling Example

MultiprocessorMemorySystem

Coherent CachingCoherent Caching Can have multiple

read-only copies of word

Writing requires getting exclusive copy

Interface

Cluster #2 Cluster #3

Interface

Mem. Cache Cache

Global Bus

Cluster #1 Bus

Proc. Proc.

Arbitrary reads & writes

Page 7: Automated Formal Verification of Software

– 7 –

Symbolic Model Checking ExampleSymbolic Model Checking Example K. McMillan, E. Clarke (CMU) J. Schwalbe (Encore Computer)

Encore Gigamax Cache SystemEncore Gigamax Cache System Distributed memory multiprocessor Cache system to improve access time Complex hardware and synchronization protocol.

VerificationVerification Create “simplified” finite state model of system (109 states!) Verify properties about set of reachable states

Bug DetectedBug Detected Sequence of 13 bus events leading to deadlock With random simulations, would require 2 years to generate

failing case. In real system, would yield MTBF < 1 day.

Page 8: Automated Formal Verification of Software

– 8 –

Boolean Inference EnginesBoolean Inference Engines

Core Verification TechnologyCore Verification Technology Satisfiability

Find one solution, or prove none exist

More advanced operationsDetermine all solutionsSolve quantified formula

BooleanFormula

BooleanFormula

SATChecker

Satisying solution

Unsatisfiable(+ proof)

BooleanFormula

BooleanFormula

BDDConstruction

All possiblesolutions

Page 9: Automated Formal Verification of Software

– 9 –

Recent Progress in SAT SolvingRecent Progress in SAT Solving

766

147 118 81 46

3600

0

1,000

2,000

3,000

Gra

sp (2

000)

zChaf

f (200

1)

BerkM

in (2

002)

zChaf

f (200

3-04

)

Siege

(200

4)

SatElit

eGTI (

2005

)

Ru

n-t

ime

(sec

.)

Page 10: Automated Formal Verification of Software

– 10 –

BDDs: Conceptual BasisBDDs: Conceptual Basis

Truth Table Decision Tree

Vertex represents decision Follow green (dashed) line for value 0 Follow red (solid) line for value 1 Function value determined by leaf value.

0 0

x3

0 1

x3

x2

0 1

x3

0 1

x3

x2

x100001111

00110011

01010101

00010101

x1 x2 x3 f

Page 11: Automated Formal Verification of Software

– 11 –

Example BDDExample BDD

Initial Graph Reduced Graph

Canonical representation of Boolean functionCanonical representation of Boolean function For given variable ordering Two functions equivalent if and only if graphs isomorphic

Can be tested in linear time

Desirable property: simplest form is canonical.

x2

0 1

x3

x1

0 0

x3

0 1

x3

x2

0 1

x3

0 1

x3

x2

x1(x1 x2) x3

Page 12: Automated Formal Verification of Software

– 12 –

Representing Complex FunctionsRepresenting Complex Functions

b3 b3

a3

Cout

b3

b2 b2

a2

b2 b2

a2

b3

a3

S3

b2

b1 b1

a1

b1 b1

a1

b2

a2

S2

b1

a0 a0

b1

a1

S1

b0

10

b0

a0

S0

FunctionsFunctions Add 4-bit words a and b Get 4-bit sum S Carry output bit Cout

Shared RepresentationShared Representation Graph with multiple roots 31 nodes for 4-bit adder 571 nodes for 64-bit adder Linear growth!

Page 13: Automated Formal Verification of Software

– 13 –

Model Checking State of ArtModel Checking State of Art

Symbolic Model CheckersSymbolic Model Checkers Use combinations of SAT checking and BDDs to represent

and manipulate sets of possible system states Can verify systems with over 1020 states

Far too large to represent states explicitly

ApplicationsApplications Standard tool for cache & bus protocol designers Requires sophisticated users

Large companies have own tool and support groupSome commercial versions available

Page 14: Automated Formal Verification of Software

– 14 –

Automatic Verification of SoftwareAutomatic Verification of Software

ChallengeChallenge Apply techniques devised for verifying hardware to software

StrategyStrategy Minimize human effort

Do not expect detailed specificationsAutomated inference techniques

Partial verificationOnly verify limited properties of systemExample: Program invulnerable to buffer overflow attacks

Page 15: Automated Formal Verification of Software

– 15 –

Example: Program Equivalence CheckingExample: Program Equivalence Checking

Do these functions produce identical results?Do these functions produce identical results?

StrategyStrategy Represent and reason about bit-level program behavior

int abs(int x) { int mask = x>>31; return (x ^ mask) + ~mask + 1;}

int test_abs(int x) { return (x < 0) ? -x : x; }

Page 16: Automated Formal Verification of Software

– 16 –

Bit-Level Program VerificationBit-Level Program Verification

View computer word as 32 separate bit valuesView computer word as 32 separate bit values

Each output becomes Boolean function of Each output becomes Boolean function of inputsinputs

abs

x0

x1

x2

x31

y0

y1

y2

y31

x0

x1

x2

x31

yi

absi

Page 17: Automated Formal Verification of Software

– 17 –

Program VerificationProgram Verification

Do these functions produce Do these functions produce identical results?identical results?

int bitOr(int x, int y){ return ~(~x & ~y);}

int test_bitOr(int x, int y){ return x | y;}

y

x

v1 = ~x

v2 = ~y

v3 = v1 & v2

v4 = ~v3

v5 = x | y

t = v4 == v5

Straight-Line Evaluation

Page 18: Automated Formal Verification of Software

– 18 –

Symbolic ExecutionSymbolic Execution (3-bit word size)

x x2

10

x1

10

x0

10

y y2

10

y1

10

y0

10

v1 = ~x x2

01

x1

01

x0

01

v2 = ~y y2

01

y1

01

y0

01

Page 19: Automated Formal Verification of Software

– 19 –

Symbolic Execution (cont.)Symbolic Execution (cont.)

v3 = v1 & v2 x2

y2

01

x1

y1

01

x0

y0

01

v4 = ~v3 x2

y2

10

x1

y1

10

x0

y0

10

v5 = x | y x2

y2

10

x1

y1

10

x0

y0

10

t = v4 == v5 1

Page 20: Automated Formal Verification of Software

– 20 –

Counterexample GenerationCounterexample Generation

Find values of Find values of xx & & yy for which these for which these programs produce different resultsprograms produce different results

int bitOr(int x, int y){ return ~(~x & ~y);}

int bitXor(int x, int y){ return x ^ y;}

y

x

v1 = ~x

v2 = ~y

v3 = v1 & v2

v4 = ~v3

v5 = x ^ y

t = v4 == v5

Straight-Line Evaluation

Page 21: Automated Formal Verification of Software

– 21 –

Symbolic ExecutionSymbolic Execution

v4 = ~v3 x2

y2

10

x1

y1

10

x0

y0

10

v5 = x ^ y x2

y2

10

y2

x1

y1

10

y1

x0

y0

10

y0

t = v4 == v5

x2

y2

x1

y1

x0

y0

01

x = 111y = 001

Page 22: Automated Formal Verification of Software

– 22 –

Performance: GoodPerformance: Goodint addXY(int x, int y){ return x+y;}

int addYX(int x, int y){ return y+x;}

0

100

200

300

400

500

600

700

800

900

1000

0 8 16 24 32

Word Size

Sec

onds

Enumerate

BDD

Page 23: Automated Formal Verification of Software

– 23 –

Performance: BadPerformance: Badint mulXY(int x, int y){ return x*y;}

int mulYX(int x, int y){ return y*x;}

0

100

200

300

400

500

600

700

800

900

1000

0 8 16 24 32

Word Size

Se

con

ds

Enumerate

BDD

Page 24: Automated Formal Verification of Software

– 24 –

Why Is Multiplication Slow?Why Is Multiplication Slow?Multiplication function

intractable for BDDsExponential growth,

regardless of variable ordering

BitsBits AddAdd MultMult

44 21 155

88 41 14560

Multiplication-4Add-4

Node Counts

Page 25: Automated Formal Verification of Software

– 25 –

What if Multiplication were Easy?What if Multiplication were Easy?

Determine case where factorK(x,y) != zero(x,y) Would enable us to factor numbers

int factorK(int x, int y){ int K = XXXX...X; int rangeOK = 1 < x && x <= y; int factorOK = x*y == K; return rangeOK && factorOK;}

int zero(int x, int y){ return 0;}

Page 26: Automated Formal Verification of Software

– 26 –

Evaluation of Bit-Level CheckingEvaluation of Bit-Level Checking

ApplicationApplication BDD version used on student programming examples Similar capabilities provided by CBMC

Clarke, Kroening, Lerda, TACAS ’04Uses SAT solver

StrengthsStrengths Provides 100% guarantee of correctness Performance very good for simple arithmetic functions

WeaknessesWeaknesses Important integer functions have exponential blowup Not practical for programs that build and operate on large

data structures

Page 27: Automated Formal Verification of Software

– 27 –

Model Checking SoftwareModel Checking Software

Program is Hard to Model as Finite-State MachineProgram is Hard to Model as Finite-State Machine Large number of large data words means lots of bits

Although “finite”, bound is very large

Recursion requires stackConceptually unbounded

Creating AbstractionCreating Abstraction Focus on control structure of program Minimal modeling of data representations & operations

Page 28: Automated Formal Verification of Software

– 28 –

Microsoft SLAM ProjectMicrosoft SLAM Project

http://research.microsoft.com/slam/

ApplicationApplication Windows device drivers Check adherence to Windows driver

guidelines

Page 29: Automated Formal Verification of Software

– 29 –

Device Driver EnvironmentDevice Driver Environment

Device DriversDevice Drivers Software to support external devices Part of trusted software base

Has kernel-level privilegesCan easily cause system to crash

Provided by device manufacturerAs object code

Trusted Software

Driver #2

Driver #1OperatingSystemKernel

Device #1

Device #2

Page 30: Automated Formal Verification of Software

– 30 –

SLAM OperationSLAM Operation

Abstraction / RefinementAbstraction / Refinement Extract Boolean program from C code

Like C program, except only Boolean data types

Model check to find typical driver bugs Refine model if necessary

Page 31: Automated Formal Verification of Software

– 31 –

Code Verification ExampleCode Verification Example

Adapted by Tom Ball from PCI device driver code

do { lock(v); old = new; if (test()) { unlock(v); new++; } } while (new != old);unlock(v);

Properties to CheckProperties to Check Cannot unlock v unless

locked Cannot lock v unless

unlocked Must exit code with v

unlocked

Page 32: Automated Formal Verification of Software

– 32 –

Model as Boolean ProgramModel as Boolean ProgramProgramming ModelProgramming Model

Retain original control flow Abstract data to be just Boolean values

Initial abstractionInitial abstraction No data. Branches occur arbitrarily

do { lock(); if (*) { unlock(); } } while (*);unlock();

Apparent bug: May call lock twice

Apparent bug: May call unlock twice

Page 33: Automated Formal Verification of Software

– 33 –

Boolean Program DetailsBoolean Program Details

States encode program point + Variable values

do {

lock();

NOP

if (*) {

unlock();

NOP

}

} while (*);

unlock();

!L

L

!L

L

ERR

L

!L

!L

do {

lock(v);

old = new;

if (test()) {

unlock(v);

new++;

}

} while (new != old);

unlock(v);

!L

L

L: Locked

ERR

Original Program

Page 34: Automated Formal Verification of Software

– 34 –

Refining AbstractionRefining Abstraction

Keep generating refinements until find bug or prove doesn’t exist

do {

lock();

E = true;

if (*) {

unlock();

E = E?false:*;

}

} while (!E);

unlock();

do {

lock(v);

old = new;

if (test()) {

unlock(v);

new++;

}

} while (new != old);

unlock(v);

L: LockedE: old == new

E !L

E L

E !L

E L

E L

!E !L

E !L

!E !L

E L

ERR

ERR

Original Program

!E !L

!E L

Page 35: Automated Formal Verification of Software

– 35 –

Windows Success StoryWindows Success Story

“Things like even software verification, this has been the Holy Grail of computer science for many decades but now in some very key areas, for example, driver verification we're building tools that can do actual proof about the software and how it works in order to guarantee the reliability”

-- Bill Gates, WinHec Conference, 2002

Available in Windows Driver Development KitAvailable in Windows Driver Development Kit Project moved from Microsoft Research into Windows

Development Group

Page 36: Automated Formal Verification of Software

– 36 –

Why the Recent Successes in Software Verification?Why the Recent Successes in Software Verification?Need for Improved Software QualityNeed for Improved Software Quality

Worldwide network of hackers creates hostile testing environment

Improved TechnologyImproved Technology Program analysis + hardware verification tools

Loss of AmbitionLoss of Ambition Bug finding rather than complete verification Vs. earlier attitude: must do complete verification of entire

program

Page 37: Automated Formal Verification of Software

– 37 –

Some Research ChallengesSome Research Challenges

Understanding Memory ReferencesUnderstanding Memory References In C, hard to determine whether two pointers refer to same

objectPointer aliasing

Real programs construct and manipulate complex data structures

Current verification techniques do best with control-intensive applications

Working with Large Code BaseWorking with Large Code Base Runtime libraries

Even simple programs build upon extensive library code

» Especially Java

Page 38: Automated Formal Verification of Software

– 38 –

Application ChallengeApplication Challenge

Cannot Apply Directly to Full Scale DesignCannot Apply Directly to Full Scale Design Verify smaller subsystems Verify abstracted versions of full system

Must understand system & tool to do effectively

SystemSize

Degree of Concurrency

ChallengingSystems to Design

Model checkingCapacity