SNU OOPSLA Lab. Chap6. Expressions and statements © copyright 2001 SNU OOPSLA Lab.
Optimizing the Automated Programming Stackbornholt/papers/thesis-phd19...Symbolic profiling...
Transcript of Optimizing the Automated Programming Stackbornholt/papers/thesis-phd19...Symbolic profiling...
Optimizing theAutomated Programming StackJames Bornholt University of Washington
Software is everywhere
Bugs are everywhere
Bugs are everywhere
Automated programming tools
VerifierProgram
Specification✅
❌ + test caseLanguage
Automated programming tools
VerifierProgram
Specification✅
❌ + test case
SynthesizerSpecification✅
❌
+ program
Language
Language
Automated programming successes
Verified operating systems
[Nelson et al, SOSP’17]
Synthesized network configs
[McClurg et al, PLDI’15]
Verified SQL optimizers
[Chu et al, VLDB’18]
Synthesized crypto primitives
[Erbsen et al, Oakland’19]
Synthesized biology experiments
[Köksal et al, POPL’13]
Synthesized memory models
[Bornholt et al, PLDI’17]
Synthesized educational models
[Butler et al, VMCAI’18]
Challenges in automated programming
Intractability SpecificationMost problems in automated programming are intractable (many undecidable).
Automated programming requiresa specification, which is often difficult to construct and audit.
Challenges in automated programming
Intractability SpecificationMost problems in automated programming are intractable (many undecidable).
Automated programming requiresa specification, which is often difficult to construct and audit.
Domain specializationSpecialization reduces the size of the search space, eliminating irrelevant programs/behaviors.
Specialization allows for concise and expressive specificationsthat capture programmer intent.
Domain specialization
Automated programming stack
Domain-specific tools
Domain specialization
Automated programming stack
SAT/SMT solving improvements in scale and expressiveness
Domain-specific tools
Domain specialization
Automated programming stack
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Domain-specific tools
Domain specialization
Automated programming stack
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Domain-specific tools
Automated programming stack
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Domain-specific tools
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
MemSynth [PLDI’17]: an automated tool for synthesizing memory consistency models
Ferrite [ASPLOS’16]: a tool for synthesizing crash-safe file system code
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
MemSynth [PLDI’17]: an automated tool for synthesizing memory consistency models
Metasketches [POPL’16]: a strategy abstrac`on for synthesis problems
SymPro [OOPSLA’18]: a technique for systema`cally building scalable tools
Ferrite [ASPLOS’16]: a tool for synthesizing crash-safe file system code
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
MemSynth [PLDI’17]: an automated tool for synthesizing memory consistency models
Metasketches [POPL’16]: a strategy abstrac`on for synthesis problems
SymPro [OOPSLA’18]: a technique for systema`cally building scalable tools
Ferrite [ASPLOS’16]: a tool for synthesizing crash-safe file system code
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello? 3 41 2
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?
3 41 2
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?
3 41 2
1 23 4
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?nothing?
3 41 2
1 23 4
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?nothing?
3 41 2
1 23 4
3 41 2
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?nothing?both?
3 41 2
1 23 4
3 41 2
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?nothing?both?
3 41 2
1 23 4
3 41 2
No! (sequential consistency)
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
Can this print… hello?goodbye?nothing?both?
3 41 2
1 23 4
3 41 2
No! (sequential consistency)
Yeah! We wanna go fast!
All variables initialized to 0
Memory models define the memory ordering behavior of multiprocessors
…correctness of my compiler…
Compiler writers#
…rules to verify against…
Verifica`on tools✅
…possible low-level behaviors…
Kernel/library developers$
Memory models define the memory ordering behavior of multiprocessors
…correctness of my compiler…
Compiler writers#
…rules to verify against…
Verifica`on tools✅
…possible low-level behaviors…
Kernel/library developers$
Litmus tests and prose
Memory models define the memory ordering behavior of multiprocessors
…correctness of my compiler…
Compiler writers#
…rules to verify against…
Verifica`on tools✅
…possible low-level behaviors…
Kernel/library developers$
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Memory models define the memory ordering behavior of multiprocessors
…correctness of my compiler…
Compiler writers#
…rules to verify against…
Verifica`on tools✅
…possible low-level behaviors…
Kernel/library developers$
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
x86 [Sewell et al, CACM’10]
PowerPC [Alglave et al, CAV’10, etc]
ARM [Flur et al, POPL’16]
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Synthesize specifica`ons from litmus tests
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Synthesize specifica`ons from litmus tests
Detect ambigui`es in synthesized models
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Synthesize specifica`ons from litmus tests
Detect ambigui`es in synthesized models
x86: 2 seconds PowerPC: 12 seconds
x86: 4 ambigui`es PowerPC: 9 ambigui`es
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Litmus testsas rela`ons
Memory models as constraints
Synthesis via sketches
1 2 3
Litmus tests as relations
X = 1 Y = 1if Y == 0: print “hello”
if X == 0: print “goodbye”
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Litmus tests as relations
X = 1 Y = 1r0 = Y r1 = X
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Litmus tests as relations
X = 1 Y = 1r0 = Y r1 = X
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Encode programs and behaviors as relations in relational logic(like Alloy)
Litmus tests as relations
X = 1 Y = 1r0 = Y r1 = X
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Encode programs and behaviors as relations in relational logic(like Alloy)
Program relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order: (a,b) ∈ po if b is aoer a on the same thread
Litmus tests as relations
X = 1 Y = 1r0 = Y r1 = X
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Encode programs and behaviors as relations in relational logic(like Alloy)
Program relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order: (a,b) ∈ po if b is aoer a on the same thread
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from: (r,w) ∈ rf if r reads the value wripen by w
Litmus tests as relations
X = 1 Y = 1r0 = Y r1 = X
Thread 1 Thread 2
1
2
3
4
All variables initialized to 0
Encode programs and behaviors as relations in relational logic(like Alloy)
Program relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order: (a,b) ∈ po if b is aoer a on the same thread
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from: (r,w) ∈ rf if r reads the value wripen by w
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
M(T, E) ≜
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
M(T, E) ≜
(&& (&& (&& (in rf (& (-> Writes Reads) (join loc (~ loc)) (join data (~ data)))) (no (- (join rf (~ rf)) iden)) (all ((r4 (- Reads (join Writes rf)))) (= (join r4 data) Zero))) (&& (in ws (& (-> Writes Writes) (join loc (~ loc)))) (no (& iden ws)) (in (join ws ws) ws) (all ((r5 Writes)) (all ((r6 (- (& Writes (join loc (join r5 loc))) r5))) (or (in (-> r5 r6) ws) (in (-> r6 r5) ws)))) (in ws (join loc (~ loc))))) (no (& (^ (+ (+ rf ws (+ (join (~ rf) ws) (& (-> (- Reads (join Writes rf)) Writes) (join loc (~ loc))))) (& po (join loc (~ loc))))) iden)) (all ((r7 Writes)) (=> (&& (in r7 (- (join univ ws) (join ws univ))) (some (join (join r7 loc) finalValue))) (= (join r7 data) (join (join r7 loc) finalValue)))) (no (& (^ (+ (& po dp) ws (+ (join (~ rf) ws) (& (-> (- Reads (join Writes rf)) Writes) (join loc (~ loc)))) (-> none none) (+ (^ (+ (+ (join (:> po Syncs) po) (join (join (:> po Syncs) po) rf)) (join rf (join (:> po Syncs) po)))) (^ (+ (+ (& (join (:> po Lwsyncs) po) (+ (-> Writes Writes) (-> Reads MemoryEvent))) (:> (join rf (& (join (:> po Lwsyncs) po) (+ (-> Writes Writes) (-> Reads MemoryEvent)))) Writes)) (<: Reads (join (& (join (:> po Lwsyncs) po) (+ (-> Writes Writes) (-> Reads MemoryEvent))) rf))))))) iden)))
(no (& (^ (+ po rf)) iden))
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
M(T, E) ≜ (no (& (^ (+ po rf)) iden))
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
M(T, E) ≜ (no (& (^ (+ po rf)) iden))Constraining the possible values of rf…
…by forbidding cycles involving rf ∪ po
Memory models as relational constraintsProgram relations extracted from program text:
po = {( , ), ( , )}1 2 3 4
Program order
Execution relations describe dynamic behavior:
rf = {( , ), ( , )}2 3 4 1
Reads-from
A memory model constrains the allowed executions of a program
Written as a predicate in relational logic
M(T, E) ≜ (no (& (^ (+ po rf)) iden))Constraining the possible values of rf…
…by forbidding cycles involving rf ∪ po
A memory model allows a test T if there exists an execu`on E that sa`sfies the predicate
(no (& (^ (+ po rf)) iden))
Synthesis from a memory model sketch
M(T, E) ≜
(no (& (^ (+ po rf)) iden))
Synthesis from a memory model sketch
M(T, E) ≜ (no (& (^ (+ ?? ??)) iden))Expression holes for a synthesizer to complete
(no (& (^ (+ po rf)) iden))
Synthesis from a memory model sketch
M(T, E) ≜ (no (& (^ (+ ?? ??)) iden))po rf po + rf po & rf po - rf …
Expression holes for a synthesizer to complete
(no (& (^ (+ po rf)) iden))
Synthesis from a memory model sketch
M(T, E) ≜ (no (& (^ (+ ?? ??)) iden))po rf po + rf po & rf po - rf …
Expression holes for a synthesizer to complete
A sketch specifies things we know (e.g., want a happens-before ordering)…
(no (& (^ (+ po rf)) iden))
Synthesis from a memory model sketch
M(T, E) ≜ (no (& (^ (+ ?? ??)) iden))po rf po + rf po & rf po - rf …
Expression holes for a synthesizer to complete
A sketch specifies things we know (e.g., want a happens-before ordering)…
…and defines the shape of the parts we don’t know
Memory model frameworks
(no (& (^ (+ ws rf ppo grf)) iden))
[Alglave et al, CAV’10]
M(T, E) ≜
Preserved program order: same-thread reorderings
Memory model frameworks
(no (& (^ (+ ws rf ppo grf)) iden))
[Alglave et al, CAV’10]
Global reads-from: inter-thread reorderings
M(T, E) ≜
Preserved program order: same-thread reorderings
Memory model frameworks
(no (& (^ (+ ws rf ppo grf)) iden))
[Alglave et al, CAV’10]
Global reads-from: inter-thread reorderings
Sequential consistency
Total storeorder (x86)
po rf
po - (Wr→Rd) rf & SameThd
M(T, E) ≜
Preserved program order: same-thread reorderings
Memory model frameworks
(no (& (^ (+ ws rf ppo grf)) iden))
[Alglave et al, CAV’10]
Global reads-from: inter-thread reorderings
Sequential consistency
Total storeorder (x86)
po rf
po - (Wr→Rd) rf & SameThd
M(T, E) ≜ ?? ??
Ocelot DSL for relational logic with holes
M(T, E) ≜
Ocelot embeds rela`onal logic in the RoseZe solver-aided language [Torlak & Bodik 2014]
Also in use for SQL query synthesis and protocol reasoning
http://ocelot.tools
(no (& (^ (+ ws rf ppo grf)) iden))?? ??
Expression holes for a synthesizer to complete
The synthesis query
Allowed litmus tests
Forbidden litmus testsSynth Completed memory model M
Memory model sketch M̂
The synthesis query
Synth
x86
53
2 allowed tests
1 2 4 6
7 8 9 10
8 forbidden tests
Total store order
Memory model sketch M̂
The synthesis query
Allowed litmus tests
Forbidden litmus testsSynth Completed memory model M
Memory model sketch M̂
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. M allows T
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. M allows T
∃ M. ∀ T∈T-. M forbids T
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. M allows T
∃ M. ∀ T∈T-. M forbids T
Standard exists-forall quan`fier papern for synthesis
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. M allows T
∃ M. ∀ T∈T-. M forbids T
M allows T: ∃ E. M(T, E)Standard exists-forall quan`fier
papern for synthesis
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. ∃ E. M(T,E)
∃ M. ∀ T∈T-. ∀ E. ¬M(T,E)
M allows T: ∃ E. M(T, E)Standard exists-forall quan`fier
papern for synthesis
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. ∃ E. M(T,E)
∃ M. ∀ T∈T-. ∀ E. ¬M(T,E)
M allows T: ∃ E. M(T, E)
Higher-order quan`fica`on over rela`ons! 😱
Standard exists-forall quan`fier papern for synthesis
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. ∃ E. M(T,E)
∃ M. ∀ T∈T-. ∀ E. ¬M(T,E)
M allows T: ∃ E. M(T, E)
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. ∃ E. M(T,E)
∃ M. ∀ T∈T-. ∀ E. ¬M(T,E)
M allows T: ∃ E. M(T, E)
Handled by a quan`fied boolean formula (QBF) solver
The synthesis query
Allowed litmus tests
Forbidden litmus testsMemory model M
Memory model sketch M̂
∃ M. ∀ T∈T+. ∃ E. M(T,E)
∃ M. ∀ T∈T-. ∀ E. ¬M(T,E)
M allows T: ∃ E. M(T, E)
Handled by a quan`fied boolean formula (QBF) solver
Handled by incremental synthesis engine
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T T
T T T T T T T T
T1
T2
T T T T T
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T T
T T T T T T T T
T1 T2
T T T T T
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T T
T T T T T T T T
T1 T2
Handled by a quan`fied boolean formula (QBF) solver
T T T T T
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T T
T T T T T T T T
T1 T2
M’
Handled by a quan`fied boolean formula (QBF) solver
T T T T T
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T T
T T T T T T T T
T1 T2
M’
T
Handled by a quan`fied boolean formula (QBF) solver
Incremental synthesis
Allowed litmus tests
Forbidden litmus tests
Synth T T T T T T T
T T T T T T T T
T1 T2
M’T
Handled by a quan`fied boolean formula (QBF) solver
Incremental synthesis
Allowed litmus tests
Forbidden litmus testsCompleted memory model M
Synth T T T T T T T
T T T T T T T T
T1 T2
M’T
Handled by a quan`fied boolean formula (QBF) solver
Incremental synthesis
Allowed litmus tests
Forbidden litmus testsCompleted memory model M
Synth T T T T T T T
T T T T T T T T
T1 T2
M’T
Empirically, need very few itera`ons to converge
Handled by a quan`fied boolean formula (QBF) solver
Disambiguating synthesized models
Ambig
Key idea: aoer synthesis, is there a different memory model that also explains the input tests?
Disambiguating synthesized models
Allowed litmus tests
Forbidden litmus tests Ambig
Key idea: aoer synthesis, is there a different memory model that also explains the input tests?
Memory model sketch M̂
Disambiguating synthesized models
Allowed litmus tests
Forbidden litmus tests AmbigCompleted memory model M
Key idea: aoer synthesis, is there a different memory model that also explains the input tests?
Memory model sketch M̂
Disambiguating synthesized models
Allowed litmus tests
Forbidden litmus tests Ambig Litmus test T
Completed memory model M
Completed memory model M2
Key idea: aoer synthesis, is there a different memory model that also explains the input tests?
Memory model sketch M̂
Disambiguating synthesized models
Allowed litmus tests
Forbidden litmus tests Ambig Litmus test T
Completed memory model M
Completed memory model M2
Key idea: aoer synthesis, is there a different memory model that also explains the input tests?
Difference between M and M2 is not just syntac`c: they disagree about test T
Memory model sketch M̂
Synthesizing existing memory models
x86
PowerPC
Synthesizing existing memory models
x86
PowerPC 768 tests[Alglave et al, CAV’10]
10 tests
Synthesis
Synthesizing existing memory models
x86
PowerPC 768 tests[Alglave et al, CAV’10]
10 tests ✓ 2 seconds
✓ 12 seconds
Synthesis
Synthesizing existing memory models
x86
PowerPC 768 tests[Alglave et al, CAV’10]
10 tests ✓ 2 seconds
✓ 12 seconds
Not equivalent to TSO!
Synthesis
Synthesizing existing memory models
x86
PowerPC 768 tests[Alglave et al, CAV’10]
10 tests ✓ 2 seconds
✓ 12 secondsNot equivalent to published model!
Not equivalent to TSO!
Synthesis
Synthesizing existing memory models
x86
PowerPC 768 tests[Alglave et al, CAV’10]
10 tests ✓ 2 seconds
✓ 12 secondsNot equivalent to published model!
4 new tests
9 new tests
Ambiguity
Not equivalent to TSO!
mfence, xchg
sync, lwsync
MemSynth: automated programming for memory consistency models
Litmus tests and prose
∀∃ ∈
∧
∨
∩∪⊂
⋈⇒
Formalspecifica`ons
Synthesize specifica`ons from litmus tests
Detect ambigui`es in synthesized models
x86: 2 seconds PowerPC: 12 seconds
x86: 4 ambigui`es PowerPC: 9 ambigui`es
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Scaling a synthesis tool is hard work
12 seconds
Scaling a synthesis tool is hard workTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
Scaling a synthesis tool is hard workTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
3 hours
Scaling a synthesis tool is hard workTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
3 hours Finding these op`miza`on opportuni`es is the key to good performance and new func`onality
Symbolic profilingTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
3 hours A symbolic profiler identifies optimization opportunities in an automated tool.
Symbolic profilingTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
3 hours A symbolic profiler identifies optimization opportunities in an automated tool.
What makes scaling an automated programming tool hard?
How does symbolic profiling work?
How effective is symbolic profiling?
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
‘()
¬(even? 3)
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
‘()
‘(6)
¬(even? 3)
(even? 6)
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
‘()
‘(6)
‘(6 8)
¬(even? 3)
(even? 6)
(even? 8)
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
‘()
‘(6)
‘(6 8)
¬(even? 3)
(even? 6)
‘(6 8 2)
(even? 8)
(even? 2)
Symbolic evaluation executes all paths through a program
(filter even? ‘(3 6 8 2 ))
‘()
‘()
‘(6)
‘(6 8)
¬(even? 3)
(even? 6)
‘(6 8 2)
(even? 8)
(even? 2)
Does this expression always return only even numbers?
Values of list elements are unknown (e.g., verifying filter for all inputs)
Symbolic evaluation executes all paths through a program
(filter even? ‘(x0 x1 x2 x3))
‘()
‘()
‘(6)
‘(6 8)
¬(even? 3)
(even? 6)
‘(6 8 2)
(even? 8)
(even? 2)
Does this expression always return only even numbers?
Values of list elements are unknown (e.g., verifying filter for all inputs)
Symbolic evaluation executes all paths through a program
(filter even? ‘(x0 x1 x2 x3))
‘()
‘()
‘(x1)
‘(x1 x2)
¬(even? x0)
(even? x1)
‘(x0)
‘() ‘(x0) ‘(x0 x1)
‘(x2) ‘(x1) ‘(x0) ‘(x0 x2) ‘(x0 x1)
‘(x1 x2 x3)‘(x1 x3)‘(x1 x2)‘(x1)‘(x2 x3) ‘(x0)‘(x0 x3)‘(x0 x2)‘(x0 x2 x3)‘(x0 x3)
(even? x2)
(even? x3)
Does this expression always return only even numbers?
Values of list elements are unknown (e.g., verifying filter for all inputs)
Symbolic evaluation executes all paths through a program
(filter even? ‘(x0 x1 x2 x3))
‘()
‘()
‘(x1)
‘(x1 x2)
¬(even? x0)
(even? x1)
‘(x0)
‘() ‘(x0) ‘(x0 x1)
‘(x2) ‘(x1) ‘(x0) ‘(x0 x2) ‘(x0 x1)
‘(x1 x2 x3)‘(x1 x3)‘(x1 x2)‘(x1)‘(x2 x3) ‘(x0)‘(x0 x3)‘(x0 x2)‘(x0 x2 x3)‘(x0 x3)
(even? x2)
(even? x3)
Does this expression always return only even numbers?
(length )
Values of list elements are unknown (e.g., verifying filter for all inputs)
Symbolic evaluation executes all paths through a program
(filter even? ‘(x0 x1 x2 x3))
‘()
‘()
‘(x1)
‘(x1 x2)
¬(even? x0)
(even? x1)
‘(x0)
‘() ‘(x0) ‘(x0 x1)
‘(x2) ‘(x1) ‘(x0) ‘(x0 x2) ‘(x0 x1)
‘(x1 x2 x3)‘(x1 x3)‘(x1 x2)‘(x1)‘(x2 x3) ‘(x0)‘(x0 x3)‘(x0 x2)‘(x0 x2 x3)‘(x0 x3)
(even? x2)
(even? x3)
Does this expression always return only even numbers?
(length )
2 2 3 1 2 2 3 212
Symbolic evaluation techniques
Symbolic execution
Bounded model checking
Always fork into independent paths (more paths, but more concrete)
Merge aoer every fork (fewer paths, but less concrete)
Symbolic evaluation techniques
Symbolic execution
Bounded model checking
Always fork into independent paths (more paths, but more concrete)
Merge aoer every fork (fewer paths, but less concrete)
Rosepe [Torlak & Bodik 2014]Jalangi [Sen et al 2014]
Crucible [Galois, Inc.]
Symbolic evaluation techniques
Symbolic execution
Bounded model checking
Always fork into independent paths (more paths, but more concrete)
Merge aoer every fork (fewer paths, but less concrete)
Rosepe [Torlak & Bodik 2014]Jalangi [Sen et al 2014]
Crucible [Galois, Inc.]
Controlling the trade-off between these strategies is key to good scalability
Two data structures to summarize symbolic evaluation
‘()
‘() ‘(x0)
‘() ‘(x1) ‘(x0) ‘(x0 x1)
¬(even? x0) (even? x0)
(even? x1)¬(even? x1) (even? x1)¬(even? x1)
(even? x0)
∧
¬
∧∧ ∧
¬
(even? x1)
Symbolic evalua\on graph Reflects the evaluator’s strategyfor all-paths execu`on of the program
Symbolic heap Shape of all symbolic valuescreated by the program
Two data structures to summarize symbolic evaluation
‘()
‘() ‘(x0)
‘() ‘(x1) ‘(x0) ‘(x0 x1)
¬(even? x0) (even? x0)
(even? x1)¬(even? x1) (even? x1)¬(even? x1)
(even? x0)
∧
¬
∧∧ ∧
¬
(even? x1)
Symbolic evalua\on graph Reflects the evaluator’s strategyfor all-paths execu`on of the program
Symbolic heap Shape of all symbolic valuescreated by the program
Any symbolic evalua\on technique can be summarized by these two data structures
Analyzing symbolic data structures
For each procedure, measure metrics that summarize the evolu`on of the symbolic evalua`on graph and symbolic heap
Summarize metrics as a score to rank procedures in the program
Analyzing symbolic data structures
For each procedure, measure metrics that summarize the evolu`on of the symbolic evalua`on graph and symbolic heap
Summarize metrics as a score to rank procedures in the program
Analyzing symbolic data structures
The most likely bopleneck is not the slowest procedure
The most likely bopleneck is not the slowest procedure
Three symbolic profilersWe developed two implementa`ons:
• The RoseZe solver-aided language (Racket)
• The Jalangi dynamic analysis framework (JavaScript)
Since publica`on, based on our work:
• The Crucible symbolic simula`on library (C, Java, …) by Galois
Three symbolic profilersWe developed two implementa`ons:
• The RoseZe solver-aided language (Racket)
• The Jalangi dynamic analysis framework (JavaScript)
Since publica`on, based on our work:
• The Crucible symbolic simula`on library (C, Java, …) by Galois
Today
Symbolic profiling in practice
Refinement type checker for Ruby [VMCAI’18] 6× speedupCryptographic protocol verifier [FM’18] 29× speedupSQL query verifier [CIDR’17] 75× speedupSafety-critical radiotherapy system verifier [CAV’16] 290× speedup
Case studies: fixed 8 performance issues in 15 Rosette tools
Symbolic profiling in practice
Refinement type checker for Ruby [VMCAI’18] 6× speedupCryptographic protocol verifier [FM’18] 29× speedupSQL query verifier [CIDR’17] 75× speedupSafety-critical radiotherapy system verifier [CAV’16] 290× speedup
Used in produc`on at the UW Medical Center
Case studies: fixed 8 performance issues in 15 Rosette tools
Symbolic profiling in practice
Refinement type checker for Ruby [VMCAI’18] 6× speedupCryptographic protocol verifier [FM’18] 29× speedupSQL query verifier [CIDR’17] 75× speedupSafety-critical radiotherapy system verifier [CAV’16] 290× speedup
Case studies: fixed 8 performance issues in 15 Rosette tools
User study: 8 Rosette users tasked with finding known performance issues in 4 programs
Users solved every task more quickly when they had access to symbolic profiling
6 failures without symbolic profiling vs. none with it
Symbolic profilingTi
me
(sec
s)
0
3000
6000
9000
12000
January February March April May June
12 seconds
3 hours A symbolic profiler identifies optimization opportunities in an automated tool.
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Automated tools are worth building The case of memory models [PLDI’17]
Building them can be made systematic Symbolic profiling [OOPSLA’18]
The future is more automation Automating the automated programming stack
Automated programming abstractions
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Metasketches [POPL’16]
Symbolic profiling [OOPSLA’18]
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Diagnosing SMT solver behavior
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Metasketches [POPL’16]
Symbolic profiling [OOPSLA’18]
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness ?
Diagnosing SMT solver behavior
SAT/SMT solving improvements in scale and expressiveness
Diagnosing SMT solver behavior
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Metasketches [POPL’16]
Symbolic profiling [OOPSLA’18]
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Solver profiling
Self-optimizing automated tools
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Metasketches [POPL’16]
Symbolic profiling [OOPSLA’18]
Solver-aided languages front-end abstractions for verification/synthesis
Symbolic evaluation algorithms to translate programs to SAT/SMT
SAT/SMT solving improvements in scale and expressiveness
Solver profiling
Exploit this profiling data for profile-guided op`miza`on
Application opportunities
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Hardware accelerator design/programming
High-performancelow-precision kernels
2
4 . . .
3
5 ⇥. . .
⇤
<latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit>
File systems [ASPLOS’16, OSDI’16]
Operating systems [SOSP’17, OSDI’18]
Memory models [PLDI’17]
Metasketches [POPL’16]
Symbolic profiling [OOPSLA’18]
Solver profiling
Hardware accelerator design/programming
High-performancelow-precision kernels
2
4 . . .
3
5 ⇥. . .
⇤
<latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit><latexit sha1_base64="82XuOOPTGgCFx506OuCN1kMU+88=">AAAHl3icdVXfj9NGEPZBuUBa4ICnqi9WA4inKM4BOXQPICpV9A2qHiBlo9N6M05Wtz/M7hgSWf47K/4YpI5/0ATbseRkPN83387MjtdxqqTHyeTr0bXrP904Hty8Nfz5l9t37p7cu//e28wJuBBWWfcx5h6UNHCBEhV8TB1wHSv4EF/9UeIfPoPz0pp/cJvCQvOVkYkUHMl1efKJKUhwPmQxrKTJuXN8W+SiGLLzkLEhW1r0lXE+ZGCWDWHInFytcRH2R4syvozsi7k8GU3GZ7Ppi6fPQzLOTl88m9bGbDoLo/GkukZBc729vHd8h8REpsGgUNz7eTRJcUGyKIUCEs48pFxc8RXMyTRcg1/kVW+K8BF5lmFiHd0Gw8q7H5Fz7f1Wx8TUHNe+jZXOPmyeYXK2yKVJMwQj6oWSTIVow7LR4VI6EKi2ZHDhJOUaijV3XCBtx5AZ+CKs1pz6w/4u5tEip175zEG5Us6q9ZJ8FBUF23has6xzP0YoW7K7kQgbRKz/uRdSCumoeQVJ9StlRn4uKIm8MlqgsQYqsDJaoFyCqcDKaIOGCqVZxKKbnxf5X//DB9KSxiM3KDnCIYUdYafx6AcRVae/H21TcBytK8ckr/ADCehMoUzrBjtQfMNkorVdQp2B0+XusPOCgfIwilgi+3WMLb6LURfbbfJWww6vnlqMuoSG0LMLbll0Z0divjxUmAfdHRulYprNK8BwFIXMueapvRiX5Y6XITG98JVBjSBnh2gs7tfdVaLfHqnS26HSi9xlkrObnaZDE3xfig3SDUmSPjp5uxkrtSupfGgTYEMntt9xmucWDVqVP24TEnRZe2ybvCqkw084jWB/QA21IzBLe0aAm5WCev8r89D8pCCK+WlngFLu6Iheg5eeRM7D0ZTu03KY9pADmpgWOY0ylx7G0Qw2bB3bTf6QeTo6U/S4VcC81A/LSugL8v0zER423k/H0WQcvXs6evW6+ZbcDH4Lfg+eBFEwC14Fb4K3wUUggn+Db0fHR4PBr4OXgz8Hb2rqtaMm5kHwwzV49x8Tq9ah</latexit>
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
New abstractions and tools can empower programmers to build specialized automated
programming tools that improve software reliability.
Thanks!
[email protected] https://unsat.org