Reordering Ranganathan: Shifting behaviors, shifting priorities.
Join-graph based cost-shifting
1
Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination Our task: Finding approximate solutions to combinatorial optimization problems defined over graphical models (e.g. MAP). Our contribution: Combine two well-known approaches: • Mini-Bucket Elimination [Dechter & Rish, 2003] • Linear Programming [Wainwright et al., 2005; Globerson & Jaakkola, 2007; Sontag et al., 2010 etc.] yielding new hybrid schemes: • Mini-Bucket Elimination with Moment-Matching • Join Graph Linear Programming Linear Programming: • iterative scheme • problem relaxed by splitting into independent components • typically operates on original functions Mini-Bucket Elimination: • single-pass algorithm • problem relaxed by duplicated some variables • typically operates on large clusters + Join Graph Linear Programming MBE with Moment-Matching Decomposition bounds Original problem f 12 f 23 f 13 X 1 X 3 X 2 Upper bound max(f 12 +f 13 +f 23 ) max f 12 +max f 13 +max f 23 f 12 f 23 f 13 X ′ 1 X ″ 1 X ′ 2 X ′ 3 X ″ 3 X ″ 2 ximize each factor independently, subject to =X ″ 1 ; X ′ 2 =X ″ 2 ; X ′ 3 =X ″ 3 Introduce functions λ ij (X i ), λ ji (X j ) for each edge (ij) - “re-parametrization” or cost- shifting j i ij X i 0 ) ( , ound the optimal configuration value: j i i ij F ij j i ij X F ij j i ij X X X X f X X f C , ) ( ) ( * ) ( ) , ( max ) , ( max )) ( ) ( ) , ( ( max min ) ( j ji i ij F ij j i ij X X X X X f •Dual decomposition, soft arc consistency, max-product linear programming, max-sum diffusion, etc •Optimum equals a linear programming relaxation. •Can use various updates to tighten the bound •Our coordinate descent update: consider minimizing over a pair λ ij (X i ), λ ik (X i ): • compute “max-marginals”: • update the λ messages reparametrization )) ( ) ( ( 2 1 i ij i ik ij x x ) , ( max ) ( j i ij X i ik X X f X j Given input parameter z and variable ordering o: • based on their scopes, functions are partitioned into “buckets”, associated with variables •buckets are processed according to o and those that have more than z variables are split into “mini- buckets” B1: f 12 (X 1 ,X 2 ) f 13 (X 1 ,X 3 ) B2: f 23 (X 2 ,X 3 ) g′ 1 (X 2 ) B3: g 2 (X 3 ) g ″ 1 (X 3 ) g 3 () q′ 1 q″ 1 Can also be interpreted as exact Bucket Elimination on a relaxed problem with duplicated variables: f 12 f 23 f 13 X ′ 1 X 3 X 2 X″ 1 Can be interpreted using a junction tree view: q′ 1 q″ 1 B2 B3 Experiments 4 benchmarks: • pedigrees • type4 • LargeFam • n-by-n grid networks genetic linkage analysis networks LP-tightening algorithms as bounding schemes 4 algorithms: MBE, MBE-MM, FGLP, JGLP LP-tightening algorithms as search guiding heuristics Anytime AND/OR Branch and Bound produces lower bounds on the optimal solution, until the exact solution is found. 4 heuristics used: MBE, MBE-MM, FGLP+MBE and JGLP. Fixed-point updates • Can use any decomposition updates (message passing, subgradient, augmented, etc.) We study two example iterative forms: • Updating the original factors (FGLP) Tighten all factors involving some X i simultaneously : • Updating the clique functions on the join graph (JGLP) 1. We use MBE to generate the junction tree, defining a function F i for each clique (mini-bucket) q i 2. Reparametrization updates of pairs of cliques along the edges MBE with Moment-Matching MBE-MM is closely related to MBE with bucket propagation (Rollon, Larrosa 2006). However: • their update is heuristical (shift all the cost in a single mini-bucket) and can only be applied once • MBE-MM updates are derived from coordinate descent and, when applied iteratively, are guaranteed to improve the bound • MBE-MM always improves upon MBE, using comparable time and memory •FGLP quickly converges and is less memory consuming than the other schemes •Given sufficient time and memory JGLP produces the tightest bound Summary of experiments q′ 1 q″ 1 B2 B3 • single-pass algorithm, processing mini-buckets top down along ordering • LP-tightening only within each bucket • can be viewed as ½ iteration of JGLP Pascal MPE tasks First-place solver in all three MPE time limits • Factor graph LP reparameterization (to tolerance or time) • Local search procedure • Build join-graph with bound z/2 • Join graph LP reparameterization (to tolerance or time) • Build join-graph with bound z (= memory limit) • Mini-bucket with max-marginal matching • AND/OR Branch & Bound Search with Caching
description
Join-graph based cost-shifting Alexander Ihler , Natalia Flerova , Rina Dechter and Lars Otten University of California Irvine . Introduction. Mini-Bucket Elimination. Pascal MPE tasks. Our task: - PowerPoint PPT Presentation
Transcript of Join-graph based cost-shifting
Join-graph based cost-shiftingAlexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten
University of California Irvine
Introduction Mini-Bucket EliminationOur task: Finding approximate solutions to combinatorial optimization problems defined over graphical models (e.g. MAP).
Our contribution: Combine two well-known approaches:• Mini-Bucket Elimination [Dechter & Rish, 2003]
• Linear Programming [Wainwright et al., 2005; Globerson & Jaakkola, 2007; Sontag et al., 2010 etc.]
yielding new hybrid schemes:• Mini-Bucket Elimination with Moment-Matching• Join Graph Linear Programming
Linear Programming:• iterative scheme• problem relaxed by splitting into independent components• typically operates on original functions
Mini-Bucket Elimination:• single-pass algorithm• problem relaxed by duplicated some variables• typically operates on large clusters
+
Join Graph Linear Programming
MBE with Moment-Matching
Decomposition boundsOriginal problem
f12
f23
f13
X1
X3X2
Upper bound
max(f12+f13+f23) max f12+max f13+max f23
f12
f23
f13X′
1 X″1
X′2
X′3
X″3
X″2
Maximize each factor independently, subject to X′
1=X″1; X′
2=X″2; X′
3=X″3
Introduce functions λij(Xi), λji(Xj) for each edge (ij) - “re-parametrization” or cost-shifting
j
iij Xi 0)(,
Bound the optimal configuration value:
jiiij
FijjiijXFij
jiijXXXXfXXfC
,)()(
* )(),(max),(max
))()(),((maxmin)(
jjiiijFij
jiijXXXXXf
•Dual decomposition, soft arc consistency, max-product linear programming, max-sum diffusion, etc•Optimum equals a linear programming relaxation.•Can use various updates to tighten the bound•Our coordinate descent update: consider minimizing over a pair λij(Xi), λik(Xi):
• compute “max-marginals”:
• update the λ messages
reparametrization
))()((21
iijiikij xx
),(max)( jiijXiik XXfXj
Given input parameter z and variable ordering o:• based on their scopes, functions are partitioned into “buckets”, associated with variables •buckets are processed according to o and those that have more than z variables are split into “mini-buckets”
B1: f12(X1,X2) f13(X1,X3)
B2: f23(X2,X3) g′1(X2)
B3: g2(X3) g″1(X3)
g3()
q′1 q″1
Can also be interpreted as exact Bucket Elimination on a relaxed problem with duplicated variables:
f12
f23
f13X′1
X3X2
X″1
Can be interpreted using a junction tree view:
q′1 q″1
B2
B3
Experiments4 benchmarks:• pedigrees• type4• LargeFam• n-by-n grid networks
genetic linkage analysis networks
LP-tightening algorithms as bounding schemes4 algorithms: MBE, MBE-MM, FGLP, JGLP
LP-tightening algorithms as search guiding heuristics
Anytime AND/OR Branch and Bound produces lower bounds on the optimal solution, until the exact solution is found.4 heuristics used: MBE, MBE-MM, FGLP+MBE and JGLP.
Fixed-point updates• Can use any decomposition updates (message passing, subgradient, augmented, etc.) We study two example iterative forms:• Updating the original factors (FGLP)Tighten all factors involving some Xi simultaneously :
• Updating the clique functions on the join graph (JGLP)1. We use MBE to generate the junction tree, defining a
function Fi for each clique (mini-bucket) qi
2. Reparametrization updates of pairs of cliques along the edges
MBE with Moment-Matching
MBE-MM is closely related to MBE with bucket propagation (Rollon, Larrosa 2006). However:• their update is heuristical (shift all the cost in a single mini-bucket) and can only be applied once • MBE-MM updates are derived from coordinate descent and, when applied iteratively, are guaranteed to improve the bound
• MBE-MM always improves upon MBE, using comparable time and memory•FGLP quickly converges and is less memory consuming than the other schemes•Given sufficient time and memory JGLP produces the tightest bound
Summary of experiments
q′1 q″1
B2
B3
• single-pass algorithm, processing mini-buckets top down along ordering• LP-tightening only within each bucket• can be viewed as ½ iteration of JGLP
Pascal MPE tasksFirst-place solver in all three MPE time limits
• Factor graph LP reparameterization (to tolerance or time)• Local search procedure• Build join-graph with bound z/2• Join graph LP reparameterization (to tolerance or time)• Build join-graph with bound z (= memory limit)• Mini-bucket with max-marginal matching• AND/OR Branch & Bound Search with Caching
