Post on 05-Jan-2016
1
Chapter 5
Constraint Satisfaction Problems
2
OutlinesConstraint Satisfaction ProblemsBacktracking Search for CSPsLocal Search for CSPThe structure of problems
3
Constraint Satisfaction Problems (CSP)
What is a CSP?– Finite set of variables X1, X2, …, Xn
– Domain of possible values for each variable Dx1, Dx2, … Dxn
– {C1, C2, …, Cm} Set of constraint relations that limit the values the variables can take on, e.g., X1 ≠ X2
Solution: Assignment of value to each variable such that the constraints are all satisfied
Consistent, objective function.Applications: Scheduling the time of observations on
the Hubble Space Telescope, Floor planning, Map coloring, Cryptography
4
Search and CSPs Search:
– Each state is a black box with no internal structure– States support goal test, evaluation, successor
function– Path to goal is important
CSP:– State is defined by variables with values from
domain– Goal test is a set of constraints specifying
allowable combinations of values for subsets of variables
– Solution is more important than path to goal
5
ConstraintsGiven a set of variables, each with a set of
possible values (depends on domain), assign a value to each variable that:– satisfies some set of explicit constraints
“hard constraints”– minimizes some cost function, where each
variable assignment has a cost“soft constraints”
6
CSP Example: Map Coloring
[From Russell and Norvig’s AIMA slides]
7
Solution to Map Coloring
8
Map Coloring represented as a Constraint Graph
9
Incremental Formulation of CSPInitial state: {}Successor function: a value can be
assigned to any unassigned variable, subject to constraints
Goal test: current assignment is complete
Path cost: constant cost for every step
10
Complete-state formulationState is a complete assignmentMight or might not satisfy the constraintsLocal search method.
11
DomainDiscrete
– Finite domain: • Map coloring,
• 8-Queens problem: Q1, Q2, …, Q8
– Di={1, 2, …,8}, d=|Di|
• All possible assignment O(dm)• Exponential in the number of variables.• Boolean CSP: 3SAT problem (NP-complete
problem).
12
DomainDiscrete
– Infinite domain: D=N, D=R
– Constraint language• StratJob1 + 5 < StartJob3• Linear Constraints• Nonlinear constraint
13
DomainContinuous
– Operation research (OR)
– AP: Hubble Space Telescope
– Linear programming:
– Integer Linear Programming
– Quadratic programming
14
Types of ConstraintsHard Constraints:
– Unary: single variable– Binary: pairs of variables– Higher-Order: 3 or more variables
Soft constraints: preferences represented by cost for each variable assignment
15
Example of Higher-Order CSP: Cryptarithmetic
16
Auxiliary variablesX1, X2, X3 {0, 1}Constraint hypergraph.Every higher-order, finite-domain constraints can
be reduced to a set of binary constraints if enough auxiliary variables are introduced.
Types of constraints– Absolute constraints: can not be violated.– Preference constraints: timetable scheduling, may be
violated.
17
Turtle Puzzle
•What are your variables?•What are the domains of possible values?•What are the constraints?
18
Search Method to CSPApply BFS to CSPBranch factors
– Incremental state formulation.– First level: nd( d=|D|, n=# of variables)– Second level: (n-1)d( d=|D|, n=# of variables)– …– Size of Trees (n! dn)
19
Commutativity Property of CSPsCommutativity: Order of application of actions
has no effect on the outcome.All CSP search algorithms consider a single
variable assignment at a timeExample: We choose colors for Australian
territories one at a time. The number of leaves is dn.
20
Backtracking SearchDFS that chooses values for one variable at a time
and backtracks when a variables has no legal values left to assign.
21
Backtracking Example: Map Coloring
22
Backtracking Example: Map Coloring
23
Backtracking Example: Map Coloring
24
Backtracking SearchPlain backtracking is uninformed search.
25
Improving Backtracking EfficiencyWhich variable (or value) should be
assigned next?In what order should its values be tried?Can we detect inevitable failure early (and
avoid the same failure in subsequent paths)?
26
Most constrained variableMost constrained variable:
choose the variable with the fewest legal values
a.k.a. minimum remaining values (MRV) heuristic If there is a variable X with zero legal values remaining,
the MRV will select X and failure will be detected immediately (pruning the search tree)
–
27
Most constraining variableTie-breaker among most constrained variables
– Degree heuristic
Most constraining variable:– choose the variable with the most constraints on
remaining variables
–
5
28
Least constraining valueGiven a variable, choose the least constraining
value:– the one that rules out the fewest values in the
remaining variables
Combining these heuristics makes 1000 queens feasible
–
29
Backtracking Search
Plain backtracking is uninformed search.
30
Forward CheckingWhenever a variable X is designed, the forward
checking process looks at each unassigned variable Y that is connected to X by a constraint and deletes from Y’ domain any value that is inconsistent with the value chosen for X.
31
Forward checking Idea:
– Keep track of remaining legal values for unassigned variables
– Terminate search when any variable has no legal values
–
32
Forward checking Idea:
– Keep track of remaining legal values for unassigned variables
– Terminate search when any variable has no legal values
–
33
Forward checking Idea:
– Keep track of remaining legal values for unassigned variables
– Terminate search when any variable has no legal values
–
34
Forward checking Idea:
– Keep track of remaining legal values for unassigned variables
– Terminate search when any variable has no legal values
–
inconsistent
35
Constraint propagation Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for all failures:
NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally
Undetected
36
Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y
–
37
Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y
–
38
Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y
If X loses a value, neighbors of X need to be rechecked
–
39
Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y
If X loses a value, neighbors of X need to be rechecked Arc consistency detects failure earlier than forward checking Can be run as a preprocessor or after each assignment
–
40
Arc Consistency As a preprocessing Step. As a propagation Step (forward checking) after every assignment
during search (MAC algorithm, maintaining arc consistency). Queue for storing arcs.
41
Arc consistency algorithm AC-3
Time complexity: O(n2d3)
42
Arc consistencyK-consistent1-consistent=node consistency2-consistent=arc consistency3- consistency =path consistencyA graph is strongly consistency it is k-
consistency and is also (k-1)-consistency, …
43
Local search for CSPsHill-climbing, simulated annealing typically work
with "complete" states, i.e., all variables assigned– The initial state assigns a value to every one variables– Successor function works by changing the value of
one variable at a time.Example: 8-Queens problem
– Successor function = swap rowsTo apply to CSPs:
– allow states with unsatisfied constraints– operators reassign variable values–
–
44
Min-conflictVariable selection: randomly select any
conflicted variableValue selection by min-conflicts heuristic:
– choose value that violates the fewest constraints
– i.e., hill-climb with h(n) = total number of violated constraints
–
–
45
Min-Conflicts algorithm
46
Example
47
Example: 4-Queens States: 4 queens in 4 columns (44 = 256 states) Actions: move queen in column Goal test: no attacks Evaluation: h(n) = number of attacks
Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)
48
Local search for CSPMin-Conflicts is surprisingly effective for many CSPs.Particular when given a reasonable initial state.For n-Queens problem, (don’t count the initial state) the
runtime of min-conflicts is roughly independent of problem size.
Million-queens problem in 50 steps.Application:
– Schedule
– Plan schedule
– Repair the schedule with a minimum number of changes.
49
Performance measure
50
The structure of Problem The only that we can possibly hope to deal with the real world
problem is decompose it into many small problem. Decompose into independent subproblem.
51
Decompose
Each component CSPi.
If Si be the solution of SCPi, Ui Si is a solution of Ui CSPi.
52
Tree-structures CSP
53
54
55
Cycle Cutset
56
Worst caseC= (n-2)Finding the smallest cycle cutset is NP-hard.Approximation algorithm are known.Cutset conditioningApplication: reasoning about probabilities.
57
Tree-Decomposition
58
Tree-decompose
59
Tree-decomposedSolve each subproblem independently. If any one has no solution, then entire problem has no
solution. If all subproblems have solutions, then construct a global
solution as follows:– View each subproblem as a mega-variable whose domain is the
set of all solutions for the subproblem.• WA=red, SA=blue, NT=green
– Solve the constraints connecting the subproblems using tree-algorithm. (for share variable)
• WA=red, SA=blue, NT=green consist with SA=blue. NT=green, Q=red.
60
Tree-widthThe aim of tree-decomposed is to make the subproblem
as small as possible.Tree-width of a tree decomposition of a graph is one less
than the size of the largest subproblems.The tree-width of the graph is defined to be the minimum
tree width among all its tree decompositions. If a graph has tree-width with w, the problem can be
solved in O(ndw+1) time. (polynominal time solvable)Finding the decomposition with minimal tree width is
NP-hard. (heuristic method).
61
Summary CSPs are a special kind of problem:
– states defined by values of a fixed set of variables– goal test defined by constraints on variable values
Backtracking = depth-first search with one variable assigned per node
Variable ordering and value selection heuristics help significantly Forward checking prevents assignments that guarantee later failure Constraint propagation (e.g., arc consistency) does additional work
to constrain values and detect inconsistencies Iterative min-conflicts is usually effective in practice
–
–
62
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{1,2,3,4}
X3{1,2,3,4}
X4{1,2,3,4}
X2{1,2,3,4}
[4-Queens slides are adapted from Stanford’s CS 121, with major corrections]
63
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{1,2,3,4}
X3{1,2,3,4}
X4{1,2,3,4}
X2{1,2,3,4}
64
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{1,2,3,4}
X3{ ,2, ,4}
X4{ ,2,3, }
X2{ , ,3,4}
65
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{1,2,3,4}
X3{ ,2, ,4}
X4{ ,2,3, }
X2{ , ,3,4}
66
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{1,2,3,4}
X3{ , , , }
X4{ ,2,3, }
X2{ , ,3,4}
67
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1,2,3,4}
X4{1,2,3,4}
X2{1,2,3,4}
68
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, ,3, }
X4{1, ,3,4}
X2{ , , ,4}
69
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, ,3, }
X4{1, ,3,4}
X2{ , , ,4}
70
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, , , }
X4{1, ,3, }
X2{ , , ,4}
71
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, , , }
X4{1, ,3, }
X2{ , , ,4}
72
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, , , }
X4{ , ,3, }
X2{ , , ,4}
73
Another Forward Checking Example: 4-Queens Problem
1
3
2
4
32 41
X1{ ,2,3,4}
X3{1, , , }
X4{ , ,3, }
X2{ , , ,4}