University of Toronto Mechanical & Industrial Engineering An Introduction to Constraint Programming...

University of TorontoMechanical & Industrial Engineering

An Introduction to Constraint Programming

J. Christopher BeckDept. of Mechanical & Industrial EngineeringUniversity of TorontoCanada


Crystal Maze

• Place the numbers 1 through 8 in the nodes such that:– Each number appears exactly once

?

?

?

?

?

?

??

– No connected nodes have consecutive numbers

You have 5 minutes!


Constraint Satisfaction Problem (CSP)• Given:

– V, a set of variables {v0, v1, …, vn}

– D, a set of domains {D0, D1, …, Dn}

– C, a set of constraints {c0, c1, …, cm}

• Each constraint, ci, has a scope ci(v0, v2, v4, v117, …), the variables that it constrains


Constraint Satisfaction Problem (CSP)• A constraint, ci, is a mapping from the

elements of the Cartesian product of the domains of the variables in its scope to {T,F}– ci(v0, v2, v4, v117, …) maps:

(D0 X D2 X D4 X D117 X … ) {T,F}

• A constraint is satisfied if the assignment of the variables in its scope are mapped to T


Constraint Satisfaction Problem (CSP)• In a solution to a CSP:

– each variable is assigned a value from its domain: vi = di, di є Di

– each constraint is satisfied


Crystal Maze

• Place the numbers 1 through 8 in the nodes such that:– Each number appears exactly once

?

?

?

?

?

?

??

– No connected nodes have consecutive numbers


Modeling Crystal Maze

• Each node a variable

• {1, …, 8} values in the domain of each variable

• No consecutive numbers a constraint– (vi, vj) |vi – vj| > 1

• All values used all-different constraint


Heuristic Search

?

?

?

?

?

?

?? 1 8

{1, 2, 3, 4, 5, 6, 7, 8}


Inference/Propagation

?

?

?

?

?

?

?? 1 8

{1, 2, 3, 4, 5, 6, 7, 8}

{1, 2, 3, 4, 5, 6, 7, 8}



?

?

?

?

?

?

?? 1 8

{3, 4, 5, 6}

7 2

{3, 4, 5, 6}

{3, 4, 5, 6} {3, 4, 5, 6}

64

3 5

{3, 4, 5, 6, 7} {2, 3, 4, 5, 6}


Generic CP Algorithm

AssertConstraint

Propagators

Start

SuccessSolution?

MakeHeuristicDecision

BacktrackTechnique

Failure

Nothing toretract?

Dead-end?


The Core of CP

• Modeling– How to represent the problem

• Heuristic search– How to branch– How much effort to find a good branch

• Backtracking

• Inference/propagation


Modeling

• How do you representthe problem with yourtools?

• Like MIP, it’s an art– Not as “advanced” on the theory side as MIP

modeling

• Different models of the same problem can have radically different performance


Modeling

• Often the first model is easy but poor– rich constraint language, “natural”– slow, scales poorly

• Need to think about:– symmetry, “what would inference do?”, search

heuristics, adding redundancy, multiple viewpoints, …

• May have to build custom constraints


Heuristic Search

• Traditional view:– variable-ordering &

value ordering

• More general view:– add constraints that partition the space

• Domain independent vs. domain dependent– min domain/degree, adaptive weights, …– e.g., heuristics for scheduling


The Importance of Heuristics

N = 30, |d| = 8, 45 constraints, 26 tuples/constraint(64-26)/64 nogoods => constraint tightness ~ 0.6

Forward checking search

means for 10 problems lexical min-domain

Searchtree 353,599 nodes 44 nodesTime 7.36s 0.01s


Backtracking

• Need to follow alternative branches– dead-end or a possible sub-optimal solution

• Chronological backtracking (aka DFS)– simplest, most popular, dumbest

• Constraint-directed backjumping– keep track of info to provably jump over some

branches

• Randomized restart– nice mathematical model



• Probably the primary reason for the problem solving power of CP– main mechanism to reduce search

• Most mathematically/theoretically mature sub-area of CP

• 300+ “global constraints” in current catalog – domain-independent to specialized– stamp collecting?


Heuristic Search or Propagation?

?

?

?

?

?

?

?? 1 8

{1, 2, 3, 4, 5, 6, 7, 8}

There must be a 1 or 8here. Why?


Arc Consistency

• Fundamental notion in CP!

• Given: c1(v1,v2)

– a binary constraint

– e.g., v1 < v2

• Given: D1 = D2 = {0, 1, …, 5}


Arc Consistency

• c1 is arc consistent iff

– for all values d1 є D1 there exists a value d2 є D2 such that c1(v1=d1,v2=d2) T

– And similarly for all values d2 є D2

V1 V2<

{0,1,2,3,4,5} {0,1,2,3,4,5}


What Now?

V1 V2<

{0,1,2,3,4,5} {0,1,2,3,4,5}

V3

<<

{0,1,2,3,4,5}


Generalized Arc Consistency (GAC)• Given: c1(v1,..., vm)

• C1 is GAC iff

– for all variables di, for all values di є Di there exists a tuple of values [dj є Dj], j≠i such that C1(vi=di,[vj=dj]) T

• E.g., c1(v1,v2,v3,v4)

– for every value in d1 є D1 there must be a triple [d2 є D2, d3 є D3, d4 є D4] s.t. c1(v1=d1, v2=d2, v3=d3, v4=d4) T


All-different Constraint

• all-diff(v1, v2, …, vn)

– all vi’s must take on a different value

– logically equivalent to

• Useful in many, many combinatorial problems– travelling salesman, timetabling, …

jivv ji ,


All-Diff vs. Clique of ≠

• Logically, all-diff(v1, v2, …, vn) =def vi ≠ vj for 1 ≤ i < j ≤ n

• D1=D2=D3={1,3}

• Establish AC (or GAC) for– v1 ≠ v2, v1 ≠ v3, v2 ≠ v3

– all-diff(v1,v2,v3)

V1 V2≠

{1,3} {1,3}

V3≠≠

{1,3}

V1 V2

{1,3} {1,3}

V3{1,3}

all-diff


All-Diff vs. Clique of ≠

• So GAC on all-diff is stronger than AC on a clique of ≠

• But what about the computational complexity?

• All-diff is a “global constraint”– GAC on some global constraints is NP-

complete– What about all-diff?


GAC Propagation on All-Diff

• Value graph

• Matchings & graph theory

• Main Theorem

• Algorithm Sketch


Value Graph

• Example– D1 = {B,C,D,E},

D2 = {B,C}, D3 = {A,B,C,D}, D4 = {B,C}

– all-diff(v1,…,v4)

1

2

3

4

A

B

C

D

E


Definitions

• Given a graph G(V,E)

• A matching is a set of disjoint edges– disjoint = no common end-points

• M covers a set if all vertices in S are end-points of an edge in M

• A vertex is M-free if it isn’t covered by M

EM

VS


Value Graph

• In terms of matchings, provide a characterization of an assignment that satisfies the all-diff constraint

1

2

3

4

A

B

C

D

E


Theorem [Regin94]

• G is a value graph

• All-diff(v1,…,vn) is GAC iff every edge in G belongs to a matching in G covering {v1,…,vn}


1

2

3

4

A

B

C

D

E


Finding Initial Max-Matching

• Find a maximum matching– O(n√m), n = |V|, m = |Di|

– well-known graph theoryalgorithm

1

2

3

4

A

B

C

D

E


Finding More Max-Matchings 1

• Orient edges:– In M – from variables to values– Not in M – from values to variables

• Find strongly connected components– depth-first search – if you re-enter a node it is

in a strongly connected component

• M-alternating cycle


2

4

B

C

1

2

3

4

A

B

C

D

E


Finding More Max-Matchings 2

• Find an M-alternating path starting from an uncovered vertex

• Breadth first search


A

1

E

3

D

1

2

3

4

A

B

C

D

E


GAC Propagation

• All edges – in M– in an M-alternating circuit, or– in an M-alternating path

• are consistent

• Remove the rest


1

2

3

4

A

B

C

D

E

1

2

3

4

A

B

C

D

E


Value Graph

• D1 = {B,C,D,E}, D2 = {B,C}, D3 = {A,B,C,D}, D4 = {B,C}

• All-different:– D1 = {B,C,D,E}, D2 = {B,C},

D3 = {A,B,C,D}, D4 = {B,C}


GAC All-Diff Propagation Summary• Find a max-matching

– O(n√m), n = |V|, m = |Di|

• Find all M-alternating circuits– DFS to find strongly connected components:

O(n+m)

• Find all M-alternating paths starting from an M-free vertex– Breadth-first search: O(m)


Global Constraints

• Any number of variables

• Accompanying efficient propagation algorithm– to enforce some level of consistency (not

necessarily GAC)

• A custom global constraint becomes both an algorithmic and modeling object– extending the language– inference algorithm can be arbitrary– compositionality


Differences with MIP

• Richer, extensible language

• Traditionally no explicit use of lower bounds– no optimization function

• constraint satisfaction problems

– less focus on relaxations– less true now: LBs “inside” global constraints


CP Engineering Process

• Model, remodel, remodel, …– if you are lucky, you’re done– abstraction?

• Domain-specific heuristics

• Custom global constraints

• Hybrids– CP/LP, CP/MIP, CP/Metaheuristics


When to Choose CP

• <Handwaving mode on>

• Finding a feasible solution is hard– intricate set of complicated constraints

• No tight relaxations

• Existing global constraints

• Strong back-propagation

University of Toronto Mechanical & Industrial Engineering An Introduction to Constraint Programming...

Documents

Transcript of University of Toronto Mechanical & Industrial Engineering An Introduction to Constraint Programming...