From Program Verification to Program Synthesis
Saurabh Srivastava *Sumit Gulwani ♯
Jeffrey S. Foster *
* University of Maryland, College Park♯ Microsoft Research, Redmond
Program Synthesis
• Goal: Automatically generate program
• Important benefits of program synthesis– Reduced programmer burden– Potential to generate novel algorithms
• Objective: Automatic program synthesis, given– Pre/post condition (functional specification)– Some hints about the form of the program
• Ideally: program is automatically proved correct– Synthesized programs can be complicated
scaffold
Proof-theoretic synthesis
Our approach: Synthesize program and proof simultaneously
Benefits:• Pruning invalid proofs prunes invalid programs
– E.g., A program that manipulates a ordered list• Ordered list invariant has to hold, preventing any program
that would violate it
• View synthesis as generalized verification– Reuse existing verification tools and technology
• Tool synthesizes: linear arithmetic programs, sorting, dynamic programming programs
Discrete Line Drawing
(0,0)
(X,Y)
|y-(Y/X)x| ≤ 1/2
Output values
Input
(0,0)
(1,1) (2,1)
(3,2) (4,2)
(5,3)
(6,4) (7,4)
(8,5) (9,5)
(10,6)(11,6)
(12,7)0<Y≤X
Using only linear operations:Bresenham’s line drawing algorithm
Scaffold(1): Functional Specification
• Precondition Fpre
– Formal specification of possible valid inputs– E.g., 0<Y≤X
• Output Postcondition Fpost
– Formal specification that outputs need to meet– E.g., |y-(Y/X)x| ≤ 1/2
•
•
–
–
– E.g., R loop = *;(*)• Acyclic fragment followed by a loop with an acyclic fragment inside
•
–
– E.g., R stack = 1 • Only one extra local variable is available
• Hints about the form of the desired program
• Looping structure R loop
– Acyclic fragments (*), loops (), sequencing (;)– Grammar: L ::= * | (L) | L;L–
•
• Stack R stack
– Maximum number of local variables permitted–
•
Scaffold(2): Resource Constraints
*
*
Scaffold(3): Domains
• Domains for program elements– Expressions Dexp
– Guards Dgrd
– E.g.• linear or quadratic expression over program variables• predicates with array lookups
• Domain for invariants Dprf
– Choice of solver– E.g., linear arithmetic
• Invariant form: ⋀i aix+biy+ci ≥ 0
• Linear arithmetic fixed-point computation tool, i.e., verifier
Outline
Input: Scaffold
Approach: Synthesis conditions
Experiments
Modeling Basic Blocks as Transitions
v := v + 2Y; x++;
Basic Block as input->output transition
inputs: v,x,Youtputs: v’,x’
Sbody : v’ = v + 2Y ⋀ x’ = x + 1;
transition : ⋀i (output vari = fn(input vars))
Task: • Model statements in a
manner that a solver can reason about
• State update and ordering problematic
Sinit
while ( Gloop ) { output (x,y) []Gcond1 → Sbody1
[]Gcond2 → Sbody2 }
expand
0<Y≤X Sinit
while ( Gloop ) { |y-(Y/X)x| ≤ 1/2 []Gcond1 → Sbody1
[]Gcond2 → Sbody2 }
Synthesis Task
Generate values for unknown Sinit/body1/body2, Gloop/cond
Generate proof (invariants)
*;(*)
0<Y≤X Sinit
while ( Gloop ) { |y-(Y/X)x| ≤ 1/2 []Gcond1 → Sbody1
[]Gcond2 → Sbody2 }
Verification condition
(VC)
VC - Safety Constraints
entryoutput
induct
induct:
entry:
output:
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’I
linear
ari
thm
eti
c
verifier
Verification condition
(VC)
Fixed point solutions, i.e., invariants
induct:
entry:
output:
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
ik, i.e., invariant I
SAT s
olv
er
boolean clauses
Dom
ain
speci
fic
reduce
r
linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]
∃ik: integer SAT instance
Linear arithmetic tricks
i1x+i2y+i3≥0 ⋀ ⋀ ⇒ i1x’+i2y’+i3≥0
Assume a form for I, e.g., i1x+i2y+i3≥0 for I (given the
domains)Gcond2 Sbody2
linear
ari
thm
eti
c
verifier
Verification condition
(VC)
Fixed point solutions, i.e., invariants
induct:
entry:
output:
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
ik, i.e., invariant I
SAT s
olv
er
boolean clauses
Dom
ain
speci
fic
reduce
r
linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]
∃ik: integer SAT instance
Linear arithmetic tricks
i1x+i2y+i3≥0 ⋀ ⋀ ⇒ i1x’+i2y’+i3≥0
Assume a form for I, e.g., i1x+i2y+i3≥0 for I (given the
domains)Gcond2 Sbody2
linear
ari
thm
eti
c
verifier
Verification condition
(VC)
Unknown statements and guards
induct:
entry:
output:
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
ik, i.e., invariant I
SAT s
olv
er
boolean clauses
Dom
ain
speci
fic
reduce
r
linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]
∃ik,gk,sk: integer SAT instance
Linear arithmetic tricks
i1x+i2y+i3≥0 ⋀ g1x+g2y+g3≥0 ⋀ x’=s1x+s2y+s3 ⋀ y’=… ⇒ i1x’+i2y’+i3≥0
Assume a form for I, Gcond2,Sbody2, e.g., i1x+i2y+i3≥0 for I (given the
domains)gk and sk
Synthesize statements and
guards!
guards/statements
Statements unknown
Infer I and Sinit,Sbody1,Sbody2
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
false false
false
false
falsefalse false
falsefalse
Trivial soln: Statements
unknown:
Ask solver
Issue: Well-formedness not ensured•Statements•Conditional Guards
But transitions, i.e., Sinit,Sbody1,Sbody2,
cannot imply false
Remember, • statement unknowns:
transition• ⋀i (output vari = fn(input
vars))• E.g., v’ = v + 2Y ⋀ x’ = x + 1
• Can never imply false
Infer I and Sinit,Sbody1,Sbody2,Gloop,Gcond
Additional Constraints
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
Well-formedness constraints:•Statement unknowns are not false•Conditional guards form tautologies
Outputs reachable/termination constraints:Exists ranking function r•r bounded from below•r decreases in each iteration
Safety
Termination
Well-formedness
Issue: Termination constraints
Issue: Well-formedness not ensured•Statements•Conditional Guards
Infer I and Sinit,Sbody1,Sbody2,Gloop,Gcond
Additional Constraints
0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2
I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’
I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’
Solving synthesis conditions does yield valid programs
SynthesisConditions
Safety
Termination
Well-formedness
User input: Program
VCGen
User input: Scaffold
SCGen
Verification Conditions
Verified Program +
Proof
Fixed-pt Solver
Verification(Invariant inference)
Synthesized Program+ Proof
Fixed-pt Solver
Synthesis(Invariant + Program
inference)
Synthesis Conditions
Synthesis Conditions
User Input
linear arithmetic
: one extra variable : *;(*)
: 2|y-(Y/X)x| ≤ 1
: 0<Y≤X
:
:constraint-basedlinear arithmetic+ verifier
Program and Proof
Solution strategy(verifier)
Example: Discrete Line Drawing
FpreFpost
R loop
R stack
Dexp
Program:v := 2Y-X; y := 0; x := 0;while ( x≤X ) { print (x,y) if ( v<0 ) { v := v + 2Y; x++; } else { v := v + 2(Y-X); y++; x++; }}
Invariant:0<Y≤Xv = 2(x+1)Y-(2y+1)X2(Y-X) ≤ v ≤ 2Y
Ranking function:X-x
Dgrd
Dprf
Scaffold
Constraint-based verifier
Outline
Input: Scaffold
Approach: Synthesis conditions
Experiments
Experiments: Linear Arithmetic
• LIA verifier from previous work [PLDI’08]
• Benchmarks– Strassen’s 2x2 matrix multiplication
• R loop = * and R stack = 7 --- for holding the seven intermediate results
– Swapping two integers without a temporary• R loop = * and R stack = 0
– Computing the integral square root• Linear search
– R loop = *;(*) and R stack = 1
• Binary search– R loop = *;(*) and R stack = 2
– Discrete line drawing• R loop = *;(*) and R stack = 1
Experiments: Predicate Abstraction
• Predicate abstraction verifier from previous work [PLDI’09]
• Benchmarks: Sorting– Requires quantified specification and invariants– E.g., All major sorting programs
• Specification: sortedness• Vary resource constraints
– Nested loop, 0 extra variables: bubble sort, insertion sort– Nested loop, 1 extra variable: selection sort– Recursive, 0 extra variables: merge sort– Recursive, 1 extra variable: quick sort
• Benchmarks: Dynamic Programming– Iterative program from its recursive specification– E.g., Fibonacci
• Specification f(0)=1, f(1)=1, ∀k>2: f(k)=f(k-1)+f(k-2)• R loop = *;(*) and R stack = 2
• Generated program maintains sliding window (of size 2) and computes f(n)
1 around 10,000 seconds
8 under 1000 seconds
8 under 10 seconds
Experiments: Synthesis times
Swap
Two
Strass
en's
Sqrt
(linea
r)
Sqrt
(binary)
Bresen
ham's
Bubble So
rt
Insertion So
rt
Selec
tion Sort
Merge S
ort
Quick So
rt
Fibonacc
i
Checkboard LC
S
Matrix
Chain
SS Sh
ortest
Path
All Shorte
st Path
All Shorte
st Path
100
1,000
10,000
100,000
1,000,000
10,000,000
Verification
Synthesis
Discret
e lin
e
0.1
0.01
0.001
seco
nds
(log
scal
e)
Predicate abstraction verifierLIA verifier
Sorti
ng Prg
1
Sorti
ng Prg
2
Sorti
ng Prg
3
Sorti
ng Prg
4
Sorti
ng Prg
5
All pairs
SP (lo
op)
All pairs
SP (b
ody)
Summary: Proof-theoretic synthesis
• Proof-theoretic synthesis– Synthesize programs and proof simultaneously– Treat synthesis as generalized verification
• Principled approach to solving for programs using verification
• User input: scaffold– Functional specification– Space of desired program
• Domains of guards and statements• Resource constraints
• Approach: One-shot synthesis, i.e., no iteration– Constraints, synthesis conditions, encode the desired program
• Safety, termination, and well-formedness
– Solve using existing constraint-based verifiers– Possible to synthesize wide variety of programs
Top Related