ECE 665 – Computer Algorithms 1 ECE 665 Spring 2004 ECE 665 Spring 2004 Computer Algorithms Binary...

1ECE 665 – Computer Algorithms

ECE 665ECE 665Spring 2004Spring 2004

Computer Algorithms

Binary Decision DiagramsBinary Decision Diagrams

Implementation IssuesImplementation Issues

Slides adopted (with permission) from A. Kuehlmann, UC Berkeley 2003Slides adopted (with permission) from A. Kuehlmann, UC Berkeley 2003

ECE 665 – Computer Algorithms 2

OutlineOutline

• ROBDD’sROBDD’s• TED’sTED’s

– Canonical representationsCanonical representations

• Implementation of ROBDD’sImplementation of ROBDD’s– ITE OperatorITE Operator

– Unique TableUnique Table

– Computed TableComputed Table

– Garbage collectionGarbage collection

– Variable orderingVariable ordering

• Other Decision DiagramsOther Decision Diagrams


ROBDD’sROBDD’s

• Directed acyclic graph (DAG)Directed acyclic graph (DAG)

• one root node, two terminals 0, 1one root node, two terminals 0, 1

• each node, two children, and a variableeach node, two children, and a variable

• Shannon co-factoring tree, except Shannon co-factoring tree, except reducedreduced and and ordered ordered ((ROROBDD)BDD)

– Reduced:Reduced:

• any node with two identical children is removedany node with two identical children is removed

• two nodes with isomorphic BDD’s are mergedtwo nodes with isomorphic BDD’s are merged

– Ordered:Ordered:

• Co-factoring variables (splitting variables) always follow the Co-factoring variables (splitting variables) always follow the same order along all pathssame order along all paths

xxii11 < x < xii22

< x < xii33 < … < x < … < xiinn


ExampleExample

Two different orderings, same function.Two different orderings, same function.

a

b b

c c

d

0 1

c+bd b

root node

c+dc

d

f = ab+a’c+bc’d a

c

d

b

0 1

c+bd

d+b

b

1

0


ROBDDROBDDOrdered BDDOrdered BDD ( (OBDDOBDD) Input variables are ordered - each path from root to ) Input variables are ordered - each path from root to

sink visits nodes with labels (sink visits nodes with labels (variablesvariables) in ascending order.) in ascending order.

a

c c

b

0 1

orderedorder = a,c,b

Reduced Ordered BDD (Reduced Ordered BDD (ROBDDROBDD) - reduction rules:) - reduction rules:1.1. if the two children of a node are the if the two children of a node are the samesame, the node is eliminated: f , the node is eliminated: f

= vf + vf= vf + vf2.2. if two nodes have if two nodes have isomorphic isomorphic graphs, they are replaced by one of graphs, they are replaced by one of

themthemThese two rules make it so that each node represents a distinct logic These two rules make it so that each node represents a distinct logic function.function.

a

b c

c

0 1

notordered

b


Arithmetic functions Arithmetic functions ((F: Int F: Int Int Int ) )

TreatTreat F F as a function of integers as a function of integers

Taylor Expansion Taylor Expansion (around (around x=0x=0))::

F(x) = F(0) + x F’(0) + F(x) = F(0) + x F’(0) + ½ x½ x22 F’’(0) + … F’’(0) + …

NotationNotation

F(x=0) 0-child - - - - - - F(x=0) 0-child - - - - - -

F’(x=0) 1-child ----------F’(x=0) 1-child ----------½ ½ F’’(x=0) 2-child ======F’’(x=0) 2-child ======etc.etc.

F(x) = 0-childF(x) = 0-child + x (1-child) + x+ x (1-child) + x22 (2-child)/2 (2-child)/2 + …+ …

Taylor Expansion Diagram (TED)Taylor Expansion Diagram (TED)

x

F(0) F’(0) F’’(0)/2 …

F(x)

7

TED – a few ExamplesTED – a few Examples

(A+B)(A+2C)

10

B

C

A

B

1

2

1

x0

x1

x2

x3

2

4

1

0

1x0

x1

x2

1

1

1

44

816

16

64

11

X2 = (8x3 + 4x2 + 2x1 + x0)2


TED: Composition & ManipulationTED: Composition & Manipulation Analogous to BDD & *BMD, TED:Analogous to BDD & *BMD, TED:

Requires an Requires an orderingordering of variables of variables Has to be Has to be reducedreduced Has to be Has to be normalizednormalized, such that, such that Reduced Ordered Normalized TED: Reduced Ordered Normalized TED: Canonical!Canonical!

Composition of TED:Composition of TED: f = g + h; APPLY(+, g, h)f = g + h; APPLY(+, g, h) f = g * h; APPLY(*, g, h)f = g * h; APPLY(*, g, h) f = g – h; APPLY(+, g, APPLY(*, -1, h))f = g – h; APPLY(+, g, APPLY(*, -1, h))

9

Properties of TEDProperties of TED

CanonicalCanonical CompactCompact Linear for polynomials of arbitrary Linear for polynomials of arbitrary

degreedegree TED for XTED for Xkk, , k k = const, with = const, with nn bits, has bits, has

k(n-1)+1k(n-1)+1 nodes nodes *BMD is *BMD is polynomial in polynomial in nn

Can contain symbolic, word-level, and Can contain symbolic, word-level, and Boolean variablesBoolean variables

It is It is notnot a Decision Diagram a Decision Diagram

n = 4, k = 2

1

x0

x1

x2

x3

2

4

1

0

1x0

x1

x2

1

1

1

44

816

16

64

11

X2 = (8x3 + 4x2 + 2x1 + x0)2


ROBDD’s (The Canonical Side)ROBDD’s (The Canonical Side)• General idea:General idea:

– instead of: exploring sub-cases by enumerating them in timeinstead of: exploring sub-cases by enumerating them in timestore sub-cases in memorystore sub-cases in memory

– KEY:KEY: two hashing mechanisms: two hashing mechanisms:– unique table: find identical sub-cases and avoid replicationunique table: find identical sub-cases and avoid replication– computed table: reduce redundant computation of sub-casescomputed table: reduce redundant computation of sub-cases

• Representation of a logic function as graph (DAG):Representation of a logic function as graph (DAG):– many logic functions can be represented compactly - usually better than many logic functions can be represented compactly - usually better than

SOP’sSOP’s• Can be made Can be made canonicalcanonical !! !!

– Shift the effort in a Boolean reasoning engine from SAT algorithm to Shift the effort in a Boolean reasoning engine from SAT algorithm to representationrepresentation

• Many logic operations can be performed efficiently on BDD’s:Many logic operations can be performed efficiently on BDD’s:– usually linear in size of result - tautology and complement are constant timeusually linear in size of result - tautology and complement are constant time

• Size of BDD critically dependent on variable orderingSize of BDD critically dependent on variable ordering


Efficient Implementation of BDD’sEfficient Implementation of BDD’sUnique Table:Unique Table:• avoids duplication of existing nodesavoids duplication of existing nodes

– Hash-Table: hash-function(key) = valueHash-Table: hash-function(key) = value– identical to the use of a hash-table in AND/INVERTER circuitsidentical to the use of a hash-table in AND/INVERTER circuits

Computed Table:Computed Table:• avoids re-computation of existing resultsavoids re-computation of existing results

hash valueof key

collisionchain

hash valueof key

No collision chain


Efficient Implementation of BDD’sEfficient Implementation of BDD’s

• BDDs is a compressed Shannon co-factoring tree:BDDs is a compressed Shannon co-factoring tree:

• f = v ff = v fvv + v f + v fvv

• leafs are constants “0” and “1”leafs are constants “0” and “1”• Three components make ROBDDs canonical (Proof: Bryant 1986):Three components make ROBDDs canonical (Proof: Bryant 1986):

– unique nodes for constant “0” and “1”unique nodes for constant “0” and “1”– identical order of case splitting variables along each pathsidentical order of case splitting variables along each paths– hash table that ensures:hash table that ensures:

• (node(f(node(fvv) = node(g) = node(gvv)) )) (node(f(node(fvv) = node(g) = node(gvv)) )) node(f) = node(g)node(f) = node(g)

– provides recursive argument that node(f) is unique when using the unique hash-provides recursive argument that node(f) is unique when using the unique hash-tabletable

v0 1

f

fv fv


ImplementationImplementationVariables are Variables are totally orderedtotally ordered: If : If vv < < ww then then vv occurs “higher” up in the ROBDD occurs “higher” up in the ROBDDTop variableTop variable of a function of a function ff is a variable associated with its is a variable associated with its root noderoot node..Example:Example: f = ab + a’bc + a’bc’f = ab + a’bc + a’bc’. Order is (. Order is (a < b < ca < b < c).).

ffaa = b, f = b, faa = b = b

a

b

0 1

f b is top variable of f

b

0 1

f

reducedf does not depend on a, f does not depend on a,

since since ffa a = f = faa . .

Each node is written as a Each node is written as a triple:triple: f = (v,g,h)f = (v,g,h) where where g = fg = fvv and and h = fh = fvv . .

We read this triple as:We read this triple as:f f = if = if vv then then gg else else h = h = iteite (v,g,h) = vg+v ’ h (v,g,h) = vg+v ’ h

vf

0 1

h g

1 0f

v

g h

mux

v is top variable of f


ITE OperatorITE Operator

ITE operator can implement any two variable logic function. There are 16 such functions corresponding to all subsets of vertices of B 2:

Table Subset Expression Equivalent Form

0000 0 0 0

0001 AND(f, g) fg ite(f, g, 0)

0010 f > g fg ite(f,g, 0)

0011 f f f

0100 f < g fg ite(f, 0, g)

0101 g g g

0110 XOR(f, g) f g ite(f,g, g)

0111 OR(f, g) f + g ite(f, 1, g)

1000 NOR(f, g) f + g ite(f, 0,g)

1001 XNOR(f, g) f g ite(f, g,g)

1010 NOT(g) g ite(g, 0, 1)

1011 f g f + g ite(f, 1, g)

1100 NOT(f) f ite(f, 0, 1)

1101 f g f + g ite(f, g, 1)

1110 NAND(f, g) fg ite(f, g, 1)

1111 1 1 1

ite( , , )f g h fg f h


Unique Table - Hash TableUnique Table - Hash Table

• Before a node (Before a node (v, g, h v, g, h ) is added to BDD data base, it is looked up in ) is added to BDD data base, it is looked up in the the “unique-table”.“unique-table”. If it is there, then existing pointer to node is used to If it is there, then existing pointer to node is used to represent the logic function. Otherwise, a new node is added to the represent the logic function. Otherwise, a new node is added to the unique-table and the new pointer returned.unique-table and the new pointer returned.

• Thus a Thus a strong canonical formstrong canonical form is maintained. The node for is maintained. The node for ff = ( = (v, g, h v, g, h ) ) exists exists iffiff((v, g, h v, g, h ) is in the unique-table. There is only one pointer for () is in the unique-table. There is only one pointer for (v, v, g, h g, h ) and that is the address to the unique-table entry.) and that is the address to the unique-table entry.

• Unique-table allows single multi-rooted DAG to represent all users’ Unique-table allows single multi-rooted DAG to represent all users’ functions:functions:

hash indexof key

collisionchain


Recursive Formulation of ITERecursive Formulation of ITE

vv = top-most variable among the three BDD’s = top-most variable among the three BDD’s f, g, hf, g, h

( , , )

( ) ( )

( ) ( )

( , ( , , ), ( , , ))

( , ( , , ), ( , , ))

( , , )

v v

v v v v v v v v

v v v v v v

v v v v v v

ite f g h fg f h

v fg f h v fg f h

v f g f h v f g f h

ite v ite f g h ite f g h

v ite f g h ite f g h

v f g R


Computed TableComputed TableKeep a record of (Keep a record of (FF, , GG, , H H ) triplets already computed by the ) triplets already computed by the ITEITE operator operator

– software cache software cache ( ( “cache” table“cache” table) ) – simply hash-table without collision chain (lossy cache)simply hash-table without collision chain (lossy cache)


Extension - Complement EdgesExtension - Complement EdgesCombine inverted functions by using complemented edgeCombine inverted functions by using complemented edge

– similar to circuit casesimilar to circuit case– reduces memory requirementsreduces memory requirements– BUT MORE IMPORTANT: BUT MORE IMPORTANT:

• makes some operations more efficient (NOT, ITE)makes some operations more efficient (NOT, ITE)

0 1

G Gonly one DAGusing complementpointer

0 1

G

0 1

G

two differentDAGs


Extension - Complement EdgesExtension - Complement Edges

To maintain strong canonical form, need to resolve 4 equivalences:To maintain strong canonical form, need to resolve 4 equivalences:

VV VV VV VV

VV VV VV VV

Solution: Always choose one on Solution: Always choose one on leftleft, i.e. the “then” leg must have , i.e. the “then” leg must have nono complement edge.complement edge.


Ambiguities in Computed TableAmbiguities in Computed TableStandard Triples:Standard Triples: iteite((FF, , FF, , G G ) ) iteite((FF, 1, , 1, G G ))

iteite((FF, , GG, , F F ) ) iteite((FF, , GG, 0 ), 0 )iteite((FF, , GG,,F F ) ) iteite((FF, , GG, 1 ), 1 )iteite((FF,,FF, , G G ) ) iteite((FF, 0, , 0, GG ) )

To resolve equivalences:To resolve equivalences: iteite((FF, 1, , 1, G G ) ) iteite((GG, 1, , 1, F F ))iteite((FF, 0, , 0, G G ) ) iteite((GG, 1,, 1,F F ))iteite((FF, , GG,, 0 ) 0 ) iteite((GG, , FF, 0 ), 0 )iteite((FF, , GG,, 1 ) 1 ) iteite((GG,,FF, 1 ), 1 )iteite((FF, , GG,,GG ) ) iteite((GG, , FF,,FF ) )

To maximize matches on computed table:To maximize matches on computed table:1. First argument is chosen with smallest top variable.1. First argument is chosen with smallest top variable.2. Break ties with smallest address pointer. 2. Break ties with smallest address pointer. (breaks PORTABILITY!!!!!!!!!!) (breaks PORTABILITY!!!!!!!!!!)

Triples:Triples:iteite((FF, , GG, , H H ) ) ite ite ((FF, , HH, , G G ) ) ite ite ((FF, , GG,,HH) ) ite ite ((FF, , HH, G) , G) Choose the one such that the first and second argument of Choose the one such that the first and second argument of iteite should not be should not be complement edges(i.e. the first one above).complement edges(i.e. the first one above).


Use of Computed TableUse of Computed Table

• Often BDD packaged use optimized implementations for special Often BDD packaged use optimized implementations for special operationsoperations– e.g. ITE_Constant (check whether the result would be a constant)e.g. ITE_Constant (check whether the result would be a constant)– AND_Exist (AND operation with existential quantification)AND_Exist (AND operation with existential quantification)

• All operations need a cache for decent performanceAll operations need a cache for decent performance– local cachelocal cache

• for one operation only - cache will be thrown away after for one operation only - cache will be thrown away after operation is finished (e.g. AND_Existoperation is finished (e.g. AND_Exist

• keep inter-operational (ITE, …)keep inter-operational (ITE, …)– special cache for each operationspecial cache for each operation

• does not need to store operation typedoes not need to store operation type– shared cache for all operationsshared cache for all operations

• better memory handlingbetter memory handling• needs to store operation typeneeds to store operation type


Variable OrderingVariable Ordering

• Static variable orderingStatic variable ordering

– variable ordering is computed up-front based on the problem variable ordering is computed up-front based on the problem structurestructure

– works very well for many combinational functions that come from works very well for many combinational functions that come from circuits we actually buildcircuits we actually build

• general scheme: control variables firstgeneral scheme: control variables first

• DFS order is pretty good for most casesDFS order is pretty good for most cases

– work bad for unstructured problemswork bad for unstructured problems

• e.g. using BDDs to represent arbitrary setse.g. using BDDs to represent arbitrary sets

– lots of research in ordering algorithmslots of research in ordering algorithms

• simulated annealing, genetic algorithmssimulated annealing, genetic algorithms

• give better results but extremely costlygive better results but extremely costly


Dynamic Variable OrderingDynamic Variable Ordering

• Changes the order in the middle of BDD applicationsChanges the order in the middle of BDD applications

– must keep same global ordermust keep same global order

• Problem: External pointers reference internal nodes!!!Problem: External pointers reference internal nodes!!!

BDD Implementation:

...

...

...

...

External reference pointers attached to application data structures



Theorem (Friedman):Theorem (Friedman):

Permuting any top part of the variable order has no effect on the nodes Permuting any top part of the variable order has no effect on the nodes labeled by variables in the bottom part.labeled by variables in the bottom part.

Permuting any bottom part of the variable order has no effect on the Permuting any bottom part of the variable order has no effect on the nodes labeled by variables in the top part.nodes labeled by variables in the top part.

• Trick: Two adjacent variable layers can be exchanged by keeping the Trick: Two adjacent variable layers can be exchanged by keeping the original memory locations for the nodesoriginal memory locations for the nodes

a

b b

c c c c

ff0 f1

f00 f01 f10 f11

mem1

mem2

mem3

bb b

c c c c

ff0 f1

f00 f01 f10 f11

aa

mem1

mem2

mem3



• BDD sifting:BDD sifting:– shift each BDD variable to the top and then to the bottom and see shift each BDD variable to the top and then to the bottom and see

which position had minimal number of BDD nodeswhich position had minimal number of BDD nodes– efficient if separate hash-table for each variableefficient if separate hash-table for each variable– can stop if lower bound on size is worse then the best found so farcan stop if lower bound on size is worse then the best found so far– shortcut:shortcut:

• two layers can be swapped very cheaply if there is no two layers can be swapped very cheaply if there is no interaction between theminteraction between them

– expensive operation, sophisticated trigger condition to invoke itexpensive operation, sophisticated trigger condition to invoke it

• grouping of BDD variables:grouping of BDD variables:– for many applications, pairing or grouping variables gives better for many applications, pairing or grouping variables gives better

orderingordering• e.g. current state and next state variables in state traversale.g. current state and next state variables in state traversal

– grouping them for sifting explores ordering that are otherwise grouping them for sifting explores ordering that are otherwise skippedskipped


Garbage CollectionGarbage Collection• Very important to free and ruse memory of unused BDD nodesVery important to free and ruse memory of unused BDD nodes

– explicitly free’d by an external explicitly free’d by an external bdd_freebdd_free operation operation– BDD nodes that were temporary created during BDD operationsBDD nodes that were temporary created during BDD operations

• Two mechanism to check whether a BDD is not referenced:Two mechanism to check whether a BDD is not referenced:– Reference counterReference counter at each node at each node

• increment whenever node gets one more reference (incl. External)increment whenever node gets one more reference (incl. External)• decrement when node gets de-references (bdd_free from external, de-decrement when node gets de-references (bdd_free from external, de-

reference from internal)reference from internal)• counter-overflow -> freeze nodecounter-overflow -> freeze node

– Mark and SweepMark and Sweep algorithm algorithm• does not need counterdoes not need counter• first pass, mark all BDDs that are referencedfirst pass, mark all BDDs that are referenced• second pass, free the BDDs that are not markedsecond pass, free the BDDs that are not marked• need additional handle layer for external referencesneed additional handle layer for external references


Garbage CollectionGarbage Collection

• Timing is very crucial because garbage collection is expensiveTiming is very crucial because garbage collection is expensive

– immediately when node gets free’edimmediately when node gets free’ed

• bad because dead nodes get often reincarnated in next bad because dead nodes get often reincarnated in next operationoperation

– regular garbage collections based on statistics collected during regular garbage collections based on statistics collected during BDD operationsBDD operations

– ““death row” for nodes to keep them around for a bit longerdeath row” for nodes to keep them around for a bit longer

• Computed table must be cleared since not used in reference Computed table must be cleared since not used in reference mechanismmechanism

• Improving memory locality and therefore cache behavior:Improving memory locality and therefore cache behavior:

– sort free’ed BDD nodes to sort free’ed BDD nodes to


BDD DerivativesBDD Derivatives

• MDD: Multi-valued BDDsMDD: Multi-valued BDDs– natural extension, have more then two branchesnatural extension, have more then two branches– can be implemented using a regular BDD package with binary encodingcan be implemented using a regular BDD package with binary encoding

• advantage that binary BDD variables for one MV variable do not have advantage that binary BDD variables for one MV variable do not have to stay together -> potentially better orderingto stay together -> potentially better ordering

• ADDs: (Analog BDDs) MTBDDsADDs: (Analog BDDs) MTBDDs– multi-terminal BDDsmulti-terminal BDDs– decision tree is binarydecision tree is binary– multiple leafs, including real numbers, sets or arbitrary objectsmultiple leafs, including real numbers, sets or arbitrary objects– efficient for matrix computations and other non-integer applicationsefficient for matrix computations and other non-integer applications

• FDDs: Free BDDsFDDs: Free BDDs– variable ordering differsvariable ordering differs– not canonical anymorenot canonical anymore

• and many more …..and many more …..

ECE 665 – Computer Algorithms 1 ECE 665 Spring 2004 ECE 665 Spring 2004 Computer Algorithms Binary...

Documents

Transcript of ECE 665 – Computer Algorithms 1 ECE 665 Spring 2004 ECE 665 Spring 2004 Computer Algorithms Binary...