Self-Adjusting Computation Umut Acar Carnegie Mellon University

Self-Adjusting Computation

Umut Acar

Carnegie Mellon University

Joint work with Guy Blelloch, Robert Harper, Srinath Sridhar, Jorge Vittes, Maverick Woo

14 January 2004 Workshop on Dynamic Algorithms and Applications

2

Dynamic Algorithms

Maintain their input-output relationship as the input changes

Example: A dynamic MST algorithm maintains the MST of a graph as user to insert/delete edges

Useful in many applications involvinginteractive systems, motion, ...


3

Developing Dynamic Algorithms: Approach I

Dynamic by designMany papersAgarwal, Atallah, Bash, Bentley, Chan, Cohen, Demaine, Eppstein, Even, Frederickson, Galil, Guibas, Henzinger, Hershberger, King, Italiano, Mehlhorn, Overmars, Powell, Ramalingam, Roditty, Reif, Reps, Sleator, Tamassia, Tarjan, Thorup, Vitter, ...

Efficient algorithms but can be complex


4

Approach II: Re-execute the algorithm when the input changes

Very simpleGeneralPoor performance


5

Smart re-execution

Suppose we can identify the pieces of execution affected by the input changeRe-execute by re-building only the affected pieces

Execution (A,I)

Execution (A,I+)


6

Smart Re-execution

Time re-execute = O(distance between executions)

Execution (A,I)

Execution (A,I+)


7

Incremental Computation or Dynamization

General techniques for transforming algorithms dynamicMany papers: Alpern, Demers, Field, Hoover, Horwitz, Hudak, Liu, de Moor, Paige, Pugh, Reps, Ryder, Strom, Teitelbaum, Weiser, Yellin...Most effective techniques are

Static Dependence Graphs [Demers, Reps, Teitelbaum ‘81]Memoization [Pugh, Teitelbaum ‘89]

These techniques work well for certain problems


8

Bridging the two worlds

Dynamization simplifies development of dynamic algorithms

but generally yields inefficient algorithms

Algorithmic techniques yield good performance

Can we have the best of the both worlds?


9

Our Work

Dynamization techniques:Dynamic dependence graphs [Acar,Blelloch,Harper ‘02]Adaptive memoization [Acar, Blelloch,Harper ‘04]

Stability: Technique for analyzing performance [ABHVW ‘04]

Provides a reduction from dynamic to static problemsReduces solving a dynamic problem to finding a stable solution to the corresponding static problem

Example: Dynamizing parallel tree contraction algorithm [Miller, Reif 85] yields an efficient solution to the dynamic trees problem [Sleator, Tarjan ‘83], [ABHVW SODA 04]


10

Outline

Dynamic Dependence GraphsAdaptive Memoization

Applications toSortingKinetic Data Structures with experimental resultsRetroactive Data Structures


11

Control dependences arise from function calls

Dynamic Dependence Graphs


12

Control dependences arise from function callsData dependences arise from reading/writing the memory

Dynamic Dependence Graphs

aabb cc


13

Change

Propagation

Change propagation

aabb cc

aabb

cc


14

Change

Propagation

Change propagation

aabb cc

aabb

cc


15

Change

Propagation

Change propagation with Memoization

aabb cc

aabb

cc


16

Change

Propagation

aabb cc

aabb

cc



17

Change

Propagation


aabb cc

aabb

cc


18

Change

Propagation

Change Propagation with Adaptive Memoization

aabb cc

aabb

cc


19

The Internals

1. Order Maintenance Data Structure [Dietz, Sleator ‘87]Time stamp vertices of the DDG in sequential execution order

2. Priority queue for change propagationpriority = time stampRe-execute functions in sequential execution order

Ensures that a value is updated before being read

3. Hash tables for memoizationRemember results from the previous execution only

4. Constant-time equality tests


20

Standard Quicksortfun qsort (l) = let fun qs (l,rest) = case l of NIL => rest | CONS(h,t) => let (smaller, bigger) = split(h,t) sbigger = qs (bigger,rest) in qs (smaller, CONS(h,sbigger)) end in qs(l,NIL) end


21

Dynamic Quicksortfun qsort (l) = let fun qs (l,rest,d) = read(l, fn l' => case l' of NIL => write (d, rest) | CONS(h,t) => let (less,bigger) = split (h,t) sbigger = mod (fn d => qs(bigger,rest,d)) in qs(less,CONS(h,sbigger,d)) endin modmod (fn d => qs (l,NIL,d)) end


22

Performance of QuicksortDynamized Quicksort updates its output in expected

O(logn) time for insertions/deletions at the end of the inputO(n) time for insertions/deletions at the beginning of the inputO(logn) time for insertions/deletions at a random location

Other Results for insertions/deletions anywhere in the inputDynamized Mergesort: expected O(logn) Dynamized Insertion Sort: expected O(n) Dynamized minimum/maximum/sum/...: expected O(logn)


23

Function Call Tree for Quicksort


24



25



26

Insertion at the end of the input


27

Insertion in the middle


28

Insertion in the middle


29

Insertion at the start, in linear time

15

1 30

5

3

26 35

4616 279

Input: 15,30,26,1,5,16,27,9,3,35,46


30

Insertion at the start, in linear time

15

1

30

5

3

26 35

4627

9

20

16

15

1 30

5

3

26 35

4616 279

Input: 20,15,30,26,1,5,16,27,9,3,35,46


31

Kinetic Data Structures [Basch,Guibas,Herschberger ‘99]

Goal: Maintain properties of continuously moving objects

Example: A kinetic convex-hull data structure maintains the convex hull of a set of continuously moving objects


32

Kinetic Data StructuresRun a static algorithm to obtain a proof of the propertyCertificate = Comparison + Failure timeInsert the certificates into a priority queue

Priority = Failure time

A framework for handling motion [Guibas, Karavelas, Russel, ALENEX 04]

while queue empty do { certificate = remove (queue) flip (certificate) update the certificate set (proof) }


33

Kinetic Data Structures via Self-Adjusting Computation

Update the proof automatically with change propagation

A library for kinetic data structures [Acar, Blelloch, Vittes]Quicksort: expected O(1), Mergesort: expected O(1)Quick Hull, Chan’s algorithm, Merge Hull: expected O(logn)

while queue empty do { certificate = remove (queue) flip (certificate) propagate ()}


34

Quick Hull: Find Min and Max

A

C

B

D

F

G

H

I

E

J

M

K

L

N

O

P

[A B C D E F G H I J K L M N O P]


35

Quick Hull: Furthest Point

A

C

B

D

F

G

H

I

E

J

M

K

L

N

O

P

[A B D F G H J K M O P]


36

Quick Hull: Filter

A

C

B

D

F

G

H

I

E

J

M

K

L

N

O

P

[ [A B F J ] [J O P] ]


37

Quick Hull: Find left hull

A

C

B

D

F

G

H

I

E

J

M

K

L

N

O

P

[ [A B] [B J] [J O] [O P] ]


38

Quick Hull: Done

A

C

B

D

F

G

H

I

E

J

M

K

L

N

O

P

[ [A B] [B J] [J O] [O P] ]


39

Static Quick Hull fun findHull(line as (p1,p2),l,hull) = let pts = filter l (fn p => Geo.lineside(p,line)) in case pts of EMPTY => CONS(p1, hull) | _ => let pm = max (Geo.dist line) l left = findHull((pm,p2),l,hull,dest) full = findHull((p1,pm),l,left) in full end end fun quickHull l = let (mx,xx) = minmax (Geo.minX, Geo.maxX) l in findHull((mx,xx),points,CONS(xx,NIL) end


40

Kinetic Quick Hullfun findHull(line as (p1,p2),l,hull,dest) = let pts = filter l(fn p => Kin.lineside(p,line))in modr (fn dest => read l (fn l => case l of NIL => write(dest,CONS(p1, hull)) | _ => read (max (Kin.dist line) l) (fn pm => let gr = modr (fn d => findHull((pm,p2),l,hull,d)) in findHull((p1,pm),l,gr,dest)))) end end

fun quickHull l = let (mx,xx) = minmax (Kin.minX, Kin.maxX) lin modr(fn d => read (mx,xx)(fn (mx,xx) => split ((mx,xx),l, CONS(xx,NIL),d)))) end


41

Kinetic Quick Hull

Input size

Cert

ificate

s /

Even

t


42

Dynamic and Kinetic Changes

Often interested in dynamic as well as kinetic changesInsert and delete objectsChange the motion plan, e.g., direction, velocity

Easily programmed via self-adjusting computationExample: Kinetic Quick Hull code is both dynamic and kinetic

Batch changesReal time changes: Can maintain partially correct data structures (stop propagation when time expires)


43

Retroactive Data Structures[Demaine, Iacono, Langerman ‘04]

Can change the sequence of operations performed on the data structure

Example: A retroactive queue would allow the user to go back in time and insert/remove an item


44

Retroactive Data Structures via Self-Adjusting Computation

Dynamize the static algorithm that takes as input the list of operations performed

Example: retroactive queuesInput: list of insert/remove operationsOutput: list of items removed Retroactive change: change the input list and propagate


45

Rake and Compress Trees [Acar,Blelloch,Vittes]

Obtained by dynamizing tree contraction [ABHVW ‘04]

Experimental analysisImplemented and applied to a broad set of applications

Path queries, subtree queries, non-local queries etc.

For path queries, compared to Link-Cut Trees [Werneck]

Structural changes are relatively slow Data changes are faster


46

Conclusions

Automatic dynamization techniques can yield efficient dynamic and kinetic algorithms/data structuresGeneral-purpose techniques for

transforming static algorithms to dynamic and kineticanalyzing their performance

Applications to kinetic and retroactive data structuresReduce dynamic problems to static problems

Future work: Lots of interesting problemsDynamic/kinetic/retroactive data structures


47

Thank you!

Self-Adjusting Computation Umut Acar Carnegie Mellon University

Documents

Transcript of Self-Adjusting Computation Umut Acar Carnegie Mellon University