Abstraction-Based Interaction Model for Synthesishpeleg/formalmodel.pdfSynthesis user โขThe user is...
Transcript of Abstraction-Based Interaction Model for Synthesishpeleg/formalmodel.pdfSynthesis user โขThe user is...
-
Abstraction-Based Interaction Model for
SynthesisHila Peleg, Technion
Shachar Itzhaky, TechnionSharon Shoham, Tel Aviv University
The research leading to these results has received funding from the European Union's - Seventh Framework Programme (FP7) under grant agreement nยฐ 615688 โ ERC- COG-PRIME.
-
Programming by Example
Task: find the most frequent bigram in a string.
2Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming by Example
Task: find the most frequent bigram in a string."abdfibfcfdebdfdebdihgfkjfdebd"
โ"bd"
2Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming by Example
Task: find the most frequent bigram in a string."abdfibfcfdebdfdebdihgfkjfdebd"
โ"bd"
input.takeRight(2)
2Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming by Example
Task: find the most frequent bigram in a string."abdfibfcfdebdfdebdihgfkjfdebd"
โ"bd"
input.takeRight(2)
2Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming by Example
Task: find the most frequent bigram in a string."abdfibfcfdebdfdebdihgfkjfdebd"
โ"bd"
input.takeRight(2)
"abbba"
โ"bb"
2Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming by Example
Task: find the most frequent bigram in a string."abdfibfcfdebdfdebdihgfkjfdebd"
โ"bd"
input.takeRight(2)
"abbba"
โ"bb"
2
input.substring(1,3)
Lieberman, H. (2000). Programming by example. Communications of the ACM, 43(3), 72-72.
-
Programming Not Only by Example
Inputโ
Output
3Peleg, H., Shoham, S., & Yahav, E. (2017). Programming Not Only by Example. ICSE 2018 (Upcoming).
-
Programming Not Only by Example
Inputโ
Output๐ โ ๐ ๐ (๐๐๐๐ข๐ก) = ๐๐ข๐ก๐๐ข๐ก
3Peleg, H., Shoham, S., & Yahav, E. (2017). Programming Not Only by Example. ICSE 2018 (Upcoming).
-
Programming Not Only by Example
Inputโ
Output๐ โ ๐ ๐ (๐๐๐๐ข๐ก) = ๐๐ข๐ก๐๐ข๐ก
input.takeRight(2)
3Peleg, H., Shoham, S., & Yahav, E. (2017). Programming Not Only by Example. ICSE 2018 (Upcoming).
-
Programming Not Only by Example
Inputโ
Output๐ โ ๐ ๐ (๐๐๐๐ข๐ก) = ๐๐ข๐ก๐๐ข๐ก
input.takeRight(2)
Exclude programs with takeRight
3Peleg, H., Shoham, S., & Yahav, E. (2017). Programming Not Only by Example. ICSE 2018 (Upcoming).
-
Programming Not Only by Example
Inputโ
Output๐ โ ๐ ๐ (๐๐๐๐ข๐ก) = ๐๐ข๐ก๐๐ข๐ก
input.takeRight(2)
Exclude programs with takeRight
๐ โ ๐ ๐(๐)
3Peleg, H., Shoham, S., & Yahav, E. (2017). Programming Not Only by Example. ICSE 2018 (Upcoming).
-
Our Goal
โข To model user-driven synthesisโข Works in practice but we do not understand its
limitations
โข Propertiesโข Of the synthesizer
โข Of the user
โข Guaranteesโข Termination (in paper)
โข Are โbad sessionsโ recoverable
4
-
Iterative, interactive synthesis
๐ผโ
๐บ๐
๐ด๐ โ 2๐ซ
๐๐ โ ๐
Partial specificationAvailable predicatesIdeal target set
Candidate programSearch space
State
5
-
Select
๐บ๐
๐๐ โ ๐
?
6
โข Candidate program is selected via someselection criterion: ๐๐๐๐๐๐ก
โข ๐๐๐๐๐๐ก usually designed to return a program from ๐โ ASAP (in 1-2 iterations)
โข There is little theoretical work about the long run
-
Predicates
Programs
2๐
2๐ซ
๐ผ ๐ถ = ๐ โ ๐ซ โ๐ โ ๐ถ.๐ โจ ๐
๐พ ๐ด = ๐ โ ๐ โ๐ โ ๐ด.๐ โจ ๐
An abstract domain
7
-
Predicates
Programs
2๐
2๐ซ
๐ผ ๐ถ = ๐ โ ๐ซ โ๐ โ ๐ถ.๐ โจ ๐
๐พ ๐ด = ๐ โ ๐ โ๐ โ ๐ด.๐ โจ ๐
An abstract domain
7
๐๐๐๐๐๐ก
-
Predicates
Programs
2๐
2๐ซ
๐ผ ๐ถ = ๐ โ ๐ซ โ๐ โ ๐ถ.๐ โจ ๐
๐พ ๐ด = ๐ โ ๐ โ๐ โ ๐ด.๐ โจ ๐
An abstract domain
7
๐๐๐๐๐๐ก
-
Synthesis Session
โข A synthesis session:
๐ฎ = ๐ด0, ๐1 ๐ด1, ๐2 โฆ
โข Synthesizer state: ๐๐ = ๐๐โ1 โ ๐ด๐โข ๐๐ = ๐๐๐๐๐๐ก ๐๐โ1 , or ๐๐ โ ๐พ(๐๐โ1) โช {โฅ}
โข If qi โ ๐โ โช {โฅ}, the session terminates
Initial specifications
Candidate program
User answer
8
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i x+z) + 1
-
Synthesis user
โข The user is aiming for some ideal set of programs ๐โ โ ๐ (where ๐ is the universe of all programs)
โข The realizable target set is ๐โ = ๐ โฉ ๐โ
โข Correctness: A user step is correct when ๐ด๐ โ ๐ โ ๐ซ โฃ โ๐ โ ๐
โ. ๐ โจ ๐
9
(1 until n).fold((x,z)=>x+z)
sum(range(1,n+1))
var sum=0
for(int i = 1; i x+z) + 1
-
User guarantees
โข In a synthesis session:1. The user is correct for as long as possible.
When not possible, ๐ด๐ =โฅ
2. The user will always accept a program from ๐โ
10
-
Progress
โข Adding new predicate doesnโt guarantee that the session is progressing
โข ๐๐โ1 โ ๐๐ could still mean that ๐พ ๐๐ = ๐พ(๐๐โ1)
โข Synthesizers usually donโt check
11
๐ดโ
๐ธ(๐บ๐)๐ธ(๐บ๐)๐1
๐0
๐2
-
Easiest way to make progress
12
๐
๐ด๐ s.t. โ๐ โ ๐ด๐ . ๐ โญ ๐
-
Easiest way to make progress
12
๐
๐ด๐ s.t. โ๐ โ ๐ด๐ . ๐ โญ ๐
โข In other words: something thatโs wrong with the current program
โข Rule out at least the current program
-
Easiest way to make progress
12
๐
๐ด๐ s.t. โ๐ โ ๐ด๐ . ๐ โญ ๐
โข In other words: something thatโs wrong with the current program
โข Rule out at least the current programโข Important when ๐๐ โ ๐พ(๐๐+1) โ ๐๐๐๐๐๐ก ๐๐+1 = ๐๐โข Easy to check, but too strong
-
A Different Model of Progress
โข ๐ด๐ makes weak progress if ๐พ ๐๐โ1 โ ๐ด๐ = ๐พ ๐๐ โ ๐พ ๐๐โ1
โข We can provide positive reinforcement
13
-
A Different Model of Progress
โข ๐ด๐ makes weak progress if ๐พ ๐๐โ1 โ ๐ด๐ = ๐พ ๐๐ โ ๐พ ๐๐โ1
โข We can provide positive reinforcement
13
(1 to n).fold((x,z)=>x+z) + 1
-
A Different Model of Progress
โข ๐ด๐ makes weak progress if ๐พ ๐๐โ1 โ ๐ด๐ = ๐พ ๐๐ โ ๐พ ๐๐โ1
โข We can provide positive reinforcement
13
(1 to n).fold((x,z)=>x+z) + 1
-
A Different Model of Progress
โข ๐ด๐ makes weak progress if ๐พ ๐๐โ1 โ ๐ด๐ = ๐พ ๐๐ โ ๐พ ๐๐โ1
โข We can provide positive reinforcement
โข Harder to check: ๐๐ โ ๐ด๐โข Once we know there is progress, we know
some things about termination (see paper)
13
(1 to n).fold((x,z)=>x+z) + 1
-
Convergence
โข A session converges if ๐พ(๐๐) โ ๐โ
โข User correctness means the session ends at state ๐
โข Converges successfully: โ โ ๐พ ๐๐ โ ๐โ
14
๐ดโ
๐ธ(๐บ๐)
-
Core set
โข Core set: the set of finiteunderapproximations of ๐โ
โฌ = ๐ต โ ๐ซ0 โ ๐พ ๐ต โ ๐โ
โง ๐ต โ โ
๐ผโ
๐ดโ
15
-
Core set
โข Core set: the set of finiteunderapproximations of ๐โ
โฌ = ๐ต โ ๐ซ0 โ ๐พ ๐ต โ ๐โ
โง ๐ต โ โ
๐ผโ
๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
15
-
Core set
๐ผโ
๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
15
โข Core set: the set of finiteunderapproximations of ๐โ
โฌ = ๐ต โ ๐ซ0 โ ๐พ ๐ต โ ๐โ
โง ๐ต โ โ
โข User intention is realizable if ๐โ โ โ
โข ๐-realizability: can converge under ๐ซ
-
Core set
๐ผโ
๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐๐โ is ๐ซ-realizable if โฌ โ โ
15
โข Core set: the set of finiteunderapproximations of ๐โ
โฌ = ๐ต โ ๐ซ0 โ ๐พ ๐ต โ ๐โ
โง ๐ต โ โ
โข User intention is realizable if ๐โ โ โ
โข ๐-realizability: can converge under ๐ซ
-
Infeasible point
โข A state where ๐พ ๐๐ โฉ๐
โ = โ
โข ๐๐๐๐๐๐ก canโt succeed, even in best case
โข The First Infeasible Pointis the first point of failure
โข Can we backtrack before an infeasible point?
16
๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
๐บ๐
-
Point of inevitable failureโข State ๐๐ is an POIF if โ๐ต โ โฌ. ๐พ ๐๐ โฉ ๐พ ๐ต = โ
โข Specifically, ๐๐ is an POIF if ๐พ ๐๐ โฉ๐
โ = โ ๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
๐บ๐๐บ๐
๐บ๐
17
-
Point of inevitable failureโข State ๐๐ is an POIF if โ๐ต โ โฌ. ๐พ ๐๐ โฉ ๐พ ๐ต = โ
โข Specifically, ๐๐ is an POIF if ๐พ ๐๐ โฉ๐
โ = โ
โข But not necessarily ๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
๐บ๐๐บ๐
๐บ๐
17
๐บ๐
-
Point of inevitable failure
๐ดโ
๐ฉ๐๐ฉ๐
๐ฉ๐
๐บ๐๐บ๐
๐บ๐
๐บ๐
17
โข State ๐๐ is an POIF if โ๐ต โ โฌ. ๐พ ๐๐ โฉ ๐พ ๐ต = โ
โข Specifically, ๐๐ is an POIF if ๐พ ๐๐ โฉ๐
โ = โ
โข But not necessarily
โข As long as ๐๐ is not an POIF we can still converge
โข Can we backtrack before an inevitable point of failure?
-
Backtracking from failure
Theorem: for any ๐ โ โ there exists a session ๐ฎ where state ๐๐ is an point of inevitable failure and only state ๐๐+๐ is the first infeasible point.
18
-
Backtracking from failure
Essentially: there is no bound on the number of steps to backtrack once failure is apparent.
Proof: by construction
Theorem: for any ๐ โ โ there exists a session ๐ฎ where state ๐๐ is an point of inevitable failure and only state ๐๐+๐ is the first infeasible point.
18
-
An unbounded session
19
Point of inevitable failure
1st infeasible point
Of length any ๐ โ โ
๐พ ๐๐ โฉ๐โ = โ
โ๐ต โ โฌ. ๐พ ๐ต โฉ ๐พ(๐๐) = โ
-
Construction
โข Candidate program space ๐ is spanned by:โข if-elseโข ==โข lists of ints ([],[1,2], etc.)โข recursive call fโข the input variable iโข consโข maxโข removeโข sortโข reverse
20
-
Construction
โข Candidate program space ๐ is spanned by:โข if-elseโข ==โข lists of ints ([],[1,2], etc.)โข recursive call fโข the input variable iโข consโข maxโข removeโข sortโข reverse
Example candidate:if (i==[]) []
else cons(max(i),
f(remove(i,max(i))
20
-
Construction
โข Candidate program space ๐ is spanned by:โข if-elseโข ==โข lists of ints ([],[1,2], etc.)โข recursive call fโข the input variable iโข consโข maxโข removeโข sortโข reverse
Example candidate:if (i==[]) []
else cons(max(i),
f(remove(i,max(i))
20
Example candidate:reverse(sort(i))
-
Construction
โข Candidate program space ๐ is spanned by:โข if-elseโข ==โข lists of ints ([],[1,2], etc.)โข recursive call fโข the input variable iโข consโข maxโข removeโข sortโข reverse
Example candidate:if (i==[]) []
else cons(max(i),
f(remove(i,max(i))
20
Example candidate:reverse(sort(i))
โข Available predicates in ๐ซ:โข Input-output examples
โข ๐๐ฅ๐๐๐ข๐๐(๐), for any program element e
-
Construction
Task: sort a list in descending order.
21
-
Construction
Input: [], Output: []Input: [1,2], Output: [2,1]
Task: sort a list in descending order.
21
-
Construction
Input: [], Output: []Input: [1,2], Output: [2,1]
reverse(i)
Task: sort a list in descending order.
21
-
Construction
Input: [], Output: []Input: [1,2], Output: [2,1]
reverse(i)
Task: sort a list in descending order.
exclude(reverse)
21
-
Construction
Input: [], Output: []Input: [1,2], Output: [2,1]
reverse(i)
Task: sort a list in descending order.
exclude(reverse) Point of Inevitable Failure
21
-
Construction
Input: [], Output: []Input: [1,2], Output: [2,1]
reverse(i)
Task: sort a list in descending order.
exclude(reverse)
if (i==[1,2])[2,1]
else i
Point of Inevitable Failure
21
-
Construction
if (i==[1,2])[2,1]
else i
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2])[2,1]
else i
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
if (i==[1,2])[2,1]
else i
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
if (i==[1,2])[2,1]
else i
โฎ
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
if (i==[1,2])[2,1]
else i
โฎ
22
Example candidate:if (i==[]) []
else cons(max(i),
f(remove(i,max(i))
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
if (i==[1,2])[2,1]
else i
โฎ
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
exclude(==)
if (i==[1,2])[2,1]
else i
โฎ
22
-
Construction
Input: [1,3], Output: [3,1]
if (i==[1,2]) [2,1]
else if (i==[1,3]) [3,1]
else i
exclude(==)
โฅ
if (i==[1,2])[2,1]
else i
โฎ
22
-
Conclusion
โข An iterative, interactive model of synthesis
โข An abstract domain of predicates
โข Progress
โข Convergence
โข The unboundedness of backtracking
โข We hope these results help future synthesizer designers
23