Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 1
CSCE 896-004, Fall 2009
www.cse.unl.edu/~choueiry/F09-896-004/Questions: [email protected]
Broadcast: [email protected]
Berthe Y. Choueiry (Shu-we-ri)AVH 360
[email protected] Tel: +1(402)472-5444
Constraint Satisfaction 101
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 2
Outline
Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques
(Implementing backtrack search)
Advanced solving techniques
Issues & research directions
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 3
Motivating example• Context: You are a senior in college
• Problem: You need to register in 4 courses for the Spring semester
• Possibilities: Many courses offered in Math, CSE, EE, CBA, etc.
• Constraints: restrict the choices you can make– Unary: Courses have prerequisites you have/don't have
Courses/instructors you like/dislike
– Binary: Courses are scheduled at the same time– n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered
• You have choices, but are restricted by constraints– Make the right decisions!!
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 4
Motivating example (cont’d)• Given
– A set of variables: 4 courses at UNL– For each variable, a set of choices (values)– A set of constraints that restrict the combinations
of values the variables can take at the same time
• Questions– Does a solution exist? (classical decision problem)– How two or more solutions differ? How to change
specific choices without perturbing the solution?– If there is no solution, what are the sources of
conflicts? Which constraints should be retracted?– etc.
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 5
Practical applications
• Radio resource management (RRM)• Databases (computing joins, view updates)• Temporal and spatial reasoning • Planning, scheduling, resource allocation • Design and configuration • Graphics, visualization, interfaces • Hardware verification and software engineering• HC Interaction and decision support • Molecular biology • Robotics, machine vision and computational linguistics • Transportation • Qualitative and diagnostic reasoning
Adapted from E.C. Freuder
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 6
Constraint Processing
• is about ...– Solving a decision problem…– … While allowing the user to state arbitrary
constraints in an expressive way and– Providing concise and high-level feedback about
alternatives and conflicts
• Related areas: – AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc.
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 7
• Flexibility & expressiveness of representations
• Interactivity
users can constraints
Power of Constraints Processing
relax
reinforce
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 8
Outline
Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal
characterizationBasic solving techniques
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 9
Defining a problem
• General template of any computational problem– Given:
• Example: a set of objects, their relations, etc.
– Query/Question: • Example: Find x such that the condition y is
satisfied
• How about the Constraint Satisfaction Problem?
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 10
Definition of a CSP• Given P = (V, D, C )
– V is a set of variables,
– D is a set of variable domains (domain values)
– C is a set of constraints,
• Query: can we find a value for each variable such that all constraints are satisfied?
nVVV ,,, 21 V
nVVV DDD ,,, 21 D
iVbVaVVVV DDDzyxiba
),,,(,...,,C with
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 11
Other queries…
• Find a solution
• Find all solutions
• Find the number of solutions
• Find a set of constraints that can be removed so that a solution exists
• Etc.
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 12
Representation I• Given P = (V, D, C ), where
• Find a consistent assignment for variables
nVVV ,,, 21 V nVVV DDD ,,, 21 D
Constraint Graph
• Variable node (vertex)
• Domain node label
• Constraint arc (edge) between nodes
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 13
Representation II• Given P = (V, D, C ), where
nVVV ,,, 21 V nVVV DDD ,,, 21 D
Example I:
• Define C ?
{1, 2, 3, 4}
{ 3, 5, 7 }{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4
V4
V2
v1+v3 < 9
V3
V1
v2 < v3
v1 < v2
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 14
Outline
Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal
characterizationBasic solving techniques
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 15
Example II: Temporal reasoning
• Give one solution: …….• Satisfaction, yes/no: decision problem
[ 5.... 18]
[ 4.... 15]
[ 1.... 10 ] B < C
A < B
B
A
2 < C - A < 5C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 16
Example III: Map coloring
Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color
• Variables?
• Domains?
• Constraints?
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 17
Example III: Map coloring (cont’d)
Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color
{ red, green, blue }
{ red, green, blue }
{ red, green, blue }{ red, green, blue }{ red, green, blue }
WY
NE
KS
OKNM
TX
LA
CO
UT
AZ
AR
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 18
Example IV: Resource Allocation
What is the CSP formulation?
{ a, b, c }
{ a, b } { a, c, d } { b, c, d }
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 19
Example IV: RA (cont’d)
{ a, b, c }
{ a, b } { a, c, d } { b, c, d }
{ R1, R3 }T1
{ R1, R3 }
{ R1, R3 } { R1, R3, R4 }
{ R1, R2, R3 }
{ R2, R4 }
{ R2, R4 }
T2
T3 T4
T5
T6
T7
Interval Order
T2
T1Constraint Graph
T4
{ R1, R3 }T3
{ R2, R4 }
{ R2, R4 }
T6
T7T5
{ R1, R2, R3 }
{ R1, R3 }
{ R1, R3 }
{ R1, R2, R3 }
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 20
Example V: PuzzleGiven: • Four musicians: Fred, Ike, Mike, and Sal, play bass, drums,
guitar and keyboard, not necessarily in that order.
• They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’
• Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist.
• etc ...
Query: Who plays which instrument and who composed which song?
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 21
Example V: Puzzle (cont’d)
Formulation 1: • Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song Gentle Rhythm and Nice Melody.• Domains: Fred, Ike, Mike, Sal • Constraints: …
Formulation 2:• Variables: Fred's-instrument, Ike's-instrument, …,
Fred's-song, Ikes's-song, Mike’s-song, …, etc.• Domains:
{ bass, drums, guitar, keyboard } { Blue Sky, Happy Song, Gentle Rhythm, Nice Melody}
• Constraints: …
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 22
Example VI: Product Configuration
Train, elevator, car, etc.
Given: • Components and their attributes (variables)• Domain covered by each characteristic (values)• Relations among the components (constraints) • A set of required functionalities (more constraints)
Find: a product configuration i.e., an acceptable combination of components that realizes the required functionalities
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 23
Example VII: Cryptarithmetic puzzles
• DX1 = DX2 = DX3 = {0,1}• DF=DT=DU=DW=DR=DO=[0,9]
• O+O = R+10X1• X1+W+W = U+10X2• X2+ T+T = O + 10X3• X3=F• Alldiff({F,D,U,V,R,O})
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 24
Constraint types: examples• Example I: algebraic constraints
• Example II:
(algebraic) constraints
of bounded difference
• Example III & IV: coloring, mutual exclusion, difference constraints
• Example V & VI: elements of C must be made explicit
{1, 2, 3, 4}
{ 3, 5, 7 }{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4
V4
V2
v1+v3 < 9
V3
V1
v2 < v3
v1 < v2
[ 5.... 18]
[ 4.... 15]
[ 1.... 10 ] B < C
A < B
B
A
2 < C - A < 5C
{ red, green, blue }
{ red, green, blue }
{ red, green, blue }{ red, green, blue }{ red, green, blue }
WY
NE
KS
OKNM
TX
LA
CO
UT
AZ
AR
T1
Constraint Graph
T4
{ R1, R3 }T3
{ R2, R4 }
{ R2, R4 }
T6
T7T5
{ R1, R2, R3 }
{ R1, R3 }
{ R1, R3 }
{ R1, R2, R3 }
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 25
More examples
• Example VII: Databases– Join operation in relational DB is a CSP– View materialization is a CSP
• Example VIII: Interactive systems– Data-flow constraints– Spreadsheets– Graphical layout systems and animation– Graphical user interfaces
• Example IX: Molecular biology (bioinformatics)– Threading, etc
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 26
Outline
Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal
characterizationBasic solving techniques
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 27
Representation (again)Macrostructure G(P):- constraint graph for
binary constraints - constraint network for non-binary constraints
Micro-structure (P):
Co-microstructure co-(P):
a, b
a, c b, c
V1
V2 V3
(V1, a ) (V1, b)
(V2, a ) (V2, c) (V3, b ) (V3, c)
(V1, a ) (V1, b)
(V2, a ) (V2, c) (V3, b ) (V3, c)
no goods
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 28
Constraint arity I• Given P = (V, D, C ) , where
nVVV ,,, 21 V nVVV DDD ,,, 21 D
• How to represent the constraint V1 + V2 + V4 < 10 ?
{1, 2, 3, 4}
{ 3, 5, 7 }{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4
V4
V2
v1+v3 < 9
V3
V1
v2 < v3
v1 < v2
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 29
Constraint arity II• Given P = (V, D, C ) , where
nVVV ,,, 21 V
nVVV DDD ,,, 21 D
Constraints: universal, unary, binary, ternary, … , global.
A Constraint Network: { 3, 6, 7 }{ 1, 2, 3, 4 }
{ 3, 5, 7 }{ 3, 4, 9 }
V3
V1 V2
v2 > v4
V4
v1 < v2
v2 < v3 v1+v3 < 9
v1+v2+V4 < 10
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 30
Domain types• P = (V, D, C ) where
• Domains:– Restricted to {0,1}: Boolean CSPs– Finite (discrete), enumeration works– Continuous, sophisticated algebraic techniques are
needed• Consistency techniques on domain bounds
nVVV ,,, 21 V nVVV DDD ,,, 21 D
mVlVkVVVV DDDzyxmlk
),,,(,...,,C with
lCCC ,,, 21 C
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 31
Constraint terminology• Scope: The set of variables to which the constraint applies• Arity: unary, binary, ternary, …, global• Definition:
– Extension: all tuples are enumerated– Intension: when it is not practical (or even possible) to list all
tuples• Define types/templates of common constraints to be used
repeatedly: linear constraints, All-Diff (mutex), Atmost, TSP-constraint, cycle-constraint, etc.)
• Implementation: – Predicate function– Set of tuples (list or table) – Binary matrix (bit-matrix)– Constrained Decision Diagrams ([Cheng & Yap, AAAI 05])– etc.
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 32
Complexity of CSP
Characterization:– Decision problem
– In general, NP-complete
by reduction from 3SAT
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 33
Proving NP-completeness
1. Show that 1 is in NP
2. Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1
– Select a known NP-complete problem 2 (e.g., SAT)
– Construct a transformation f from 2 to 1
– Prove that f is a polynomial transformation
(Check Chapter 3 of Garey & Johnson)
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 34
What is SAT?Given a sentence:
– Sentence: conjunction of clauses
– Clause: disjunction of literals
– Literal: a term or its negation
– Term: Boolean variable
Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied.
4326541 ccccccc
32 cc
61, cc
01 11 cc
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 35
CSP is NP-Complete• Verifying that an assignment for all
variables is a solution – Provided constraints can be checked in
polynomial time
• Reduction from 3SAT to CSP– Many such reductions exist in the literature
(perhaps 7 of them)
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 36
Problem reductionExample: CSP into SAT (proves nothing, just an exercise)
Notation: variable-value pair = vvp
• vvp term– V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d),
– V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c).
• The vvp’s of a variable disjunction of terms– V1 = {a, b, c, d} yields
• (Optional) At most one VVP per variable
4321 xxxx
43214321
43214321
xxxxxxxx
xxxxxxxx
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 37
CSP into SAT (cont.)Constraint:
1. Way 1: Each inconsistent tuple one disjunctive clause– For example: how many?
2. Way 2:a) Consistent tuple conjunction of termsb) Each constraint disjunction of these conjunctions
transform into conjunctive normal form (CNF)
Question: find a truth assignment of the Boolean variables such that the sentence is satisfied
)},(),,(),,(),,(),,{(21
adbccbbaaaC VV
71 xx
51 xx
5463
726151
xxxx
xxxxxx
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 38
Outline
Motivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques
• Modeling and consistency checking• Constructive, systematic search• Iterative improvement, local search
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 39
How to solve a CSP?
Search
1. Constructive, systematic
2. Iterative repair, local search
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 40
Before starting search!
Consider:• Importance of modeling/formulation:
– To control the size of the search space
• Preprocessing – A.k.a. constraint filtering/propagation, consistency
checking– reduces size of search space
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 41
4 Rows
V1
V2
V3
V4
Importance of modeling
• N-queen: formulation 1
– Variables? – Domains?– Size of CSP?
• N-queens: formulation 2
– Variables?– Domains?– Size of CSP?
4 Column positions
1 2 3 4
84 244444
{0,1}162
16 Cells
V11 V12 V13 V14
V21 V22 V23 V24
V31 V32 V33 V34
V41 V42 V43 V44
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 42
Constraint checking Arc-consistency
[ 5.... 18]
[ 4.... 15]
[ 1.... 10 ]B < C
A < B
B
A
2 < C - A < 5
C
2- A: [ 2 .. 10 ]
C: [ 6 .. 14 ]
3- B: [ 5 .. 13 ]
C: [ 6 .. 15 ]
1- B: [ 5 .. 14 ]1413
6
2
14
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 43
Constraint checking Arc-consistency: every combination of two adjacent variables 3-consistency, k-consistency (k n)
Constraint filtering, constraint checking, etc.. Eliminate non-acceptable tuples prior to search Warning: arc-consistency does not solve the problem
{ 1, 2, 3 } { 2, 3, 4 }
BA BA
{ 2, 3 } { 2, 3 }
)3()2( BA still is not a solution!
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 44
Systematic search Starting from a root node Consider all values for a variable V1
For every value for V1, consider all values for V2
etc..
For n variables, each of domain size d Maximum depth? fixed! Maximum number of paths? size of search space, size of CSP
S
v1 v4Var 1
Var 2
v3v2
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 45
Systematic search (I) Back-checking
• Systematic search generates dn possibilities• Are all possibilities acceptable?
Expand a partial solution only when it is consistent This yields early pruning of inconsistent paths
S
v1 v4Var 1
Var 2
v3v2
S
v1 v4Var 1
Var 2
v3v2
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 46
Systematic search (II) Chronological backtracking
What if only one solution is needed?
• Depth-first search & Chronological backtracking
• DFS: Soundness? Completeness?
Var 1 v1 v2
S
S
v1 v4Var 1
Var 2
v3v2
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 47
Systematic search (III) Intelligent backtracking
What if the reason for failure was higher up in the tree?
Backtrack to source of conflict !!
Backjumping, conflict-directed backjumping, etc.
Var 1 v1 v2
S S
v1 v2Var 1
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 48
Systematic search (IV) Ordering heuristics
• Which variable to expand first?
• Heuristics:– most constrained variable first (reduce branching factor)
– most promising value first (find quickly first solution)
},{},,,,{,,:2121 baDdcbaDVV VV Exp
)}(),{()}(),{(: 2121 bVcVaVcV andSol
dynamic
staticbecould
ordering value
ordering variableforStrategies
s
c
aV2 b a b a b
ba dV1
V1
s
V2 ba
ba c d
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 49
Systematic search (V) Back-checking
• Search tree with only backtrack search?Root node
1 2 4
1 3 41 2 3 4
1 2 3 4
1Q
3Q Q
2Q
1 2 3
1 2
2Q
4Q
1Q
3Q
Solution!
26 nodes visited.
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 50
Systematic search (VI) Forward checking
Root node
1Q
3Q Q 4
Domain Wipe Out V3
2Q
Domain Wipe Out V4
2Q
4Q
1Q
3Q
Solution!
8 nodes visited.
Search Tree with domains filter by Forward Check
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 51
Improving BT search
• General purpose methods for1. Variable, value ordering
2. Improving backtracking: intelligent backtracking avoids repeating failure
3. Look-ahead techniques: propagate constraints as variables are instantiated
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 52
Search-tree branching
• K-way branching– One branch for every value in the domain
• Binary branching– One branch for the first value– One branch for all the remaining values in the domain– Used in commercial constraint solvers: ILOG, Eclipse
• Have different behaviors (e.g., WRT value ordering
heuristics [Smith, IJCAI 95])
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 53
Summary of backtrack search• Constructive, systematic, exhaustive
– In general sound and complete
• Back-checking: expands nodes consistent with past• Backtracking: Chronological vs. intelligent• Ordering heuristics:
– Static– Dynamic variable– Dynamic variable-value pairs
• Looking ahead:– Partial look-ahead
• Forward checking (FC)• Directional arc-consistency (DAC)
– Full (a.k.a. Maintaining Arc-consistency or MAC)
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 54
CSP: a decision problem (NP-complete)
1. Modeling: abstraction and reformulation2. Preprocessing techniques:
• eliminate non-acceptable tuples prior to search
3. Systematic search:• potentially dn paths of fixed lengths• chronological backtracking• intelligent backtracking• variable/value ordering heuristics
4. Search ‘hybrids:’• Mixing consistency checking with search: look-ahead
strategies
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 55
Non-systematic search• Iterative repair, local search: modifies a
global but inconsistent solution to decrease the number of violated constraints
• Examples: Hill climbing, taboo search, simulated annealing, GSAT, WalkSAT, Genetic Algorithms, Swarm intelligence, etc.
• Features: Incomplete & not sound
– Advantage: anytime algorithm
– Shortcoming: cannot tell that no solution exists
Basic Concepts & Algorithms in CP, Fall 2009
Aug 31, 2009 Overview 1 56
Outline of CSP 101
• We have seenMotivating example, application areasCSP: Definition, representationSome simple modeling examplesMore on definition and formal characterizationBasic solving techniques
• We will move to (Implementing backtrack search)
Advanced solving techniquesIssues & research directions
Top Related