Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1...

89
Sep 22, 2005 Consistency: Properties & Algorithms 1 Foundations of Constraint Processing, Fall 2005 Foundations of Constraint Processing CSCE421/821, Fall 2005: www.cse.unl.edu/~choueiry/F05-421-821/ Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 123B [email protected] Tel: +1(402)472-5444 Consistency Algorithms
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    218
  • download

    0

Transcript of Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1...

Page 1: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

1

Foundations of Constraint Processing Fall 2005

Foundations of Constraint ProcessingCSCE421821 Fall 2005

wwwcseunledu~choueiryF05-421-821

Berthe Y Choueiry (Shu-we-ri)Avery Hall Room 123Bchoueirycseunledu

Tel +1(402)472-5444

Consistency Algorithms

Sep 22 2005 ConsistencyProperties amp Algorithms

2

Foundations of Constraint Processing Fall 2005

Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and

Freuder AIJ852 Sections 31 32 33 Chapter 3 Constraint Processing Dechter

Recommended1 Sections 34mdash310 Chapter 3 Constraint Processing Dechter2 Networks of Constraints Fundamental Properties and Application

to Picture Processing Montanari Information Sciences 743 Consistency in Networks of Relations Mackworth AIJ77 4 Constraint Propagation with Interval Labels Davis AIJ875 Path Consistency on Triangulated Constraint Graphs Bliek amp Sam-

Haroud IJCAI99

Sep 22 2005 ConsistencyProperties amp Algorithms

3

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results box consistency constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

4

Foundations of Constraint Processing Fall 2005

Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by

- problem size- amount of backtracking

bull Backtracking may yield

ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again

Malady of backtrack search

Sep 22 2005 ConsistencyProperties amp Algorithms

5

Foundations of Constraint Processing Fall 2005

(Some) causes of thrashingSearch order V1 V2 V3 V4 V5

What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc

inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and

V4mdashV5 (path inconsistency)

1 2V4

1 2 3

0 1 2

1 2V5

1 2 3V1 V2

V3ltV2

V3gt0

V3ltV4

V5ltV4

V3ltV5

V3ltV1

V3

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 2: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

2

Foundations of Constraint Processing Fall 2005

Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and

Freuder AIJ852 Sections 31 32 33 Chapter 3 Constraint Processing Dechter

Recommended1 Sections 34mdash310 Chapter 3 Constraint Processing Dechter2 Networks of Constraints Fundamental Properties and Application

to Picture Processing Montanari Information Sciences 743 Consistency in Networks of Relations Mackworth AIJ77 4 Constraint Propagation with Interval Labels Davis AIJ875 Path Consistency on Triangulated Constraint Graphs Bliek amp Sam-

Haroud IJCAI99

Sep 22 2005 ConsistencyProperties amp Algorithms

3

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results box consistency constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

4

Foundations of Constraint Processing Fall 2005

Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by

- problem size- amount of backtracking

bull Backtracking may yield

ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again

Malady of backtrack search

Sep 22 2005 ConsistencyProperties amp Algorithms

5

Foundations of Constraint Processing Fall 2005

(Some) causes of thrashingSearch order V1 V2 V3 V4 V5

What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc

inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and

V4mdashV5 (path inconsistency)

1 2V4

1 2 3

0 1 2

1 2V5

1 2 3V1 V2

V3ltV2

V3gt0

V3ltV4

V5ltV4

V3ltV5

V3ltV1

V3

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 3: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

3

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results box consistency constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

4

Foundations of Constraint Processing Fall 2005

Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by

- problem size- amount of backtracking

bull Backtracking may yield

ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again

Malady of backtrack search

Sep 22 2005 ConsistencyProperties amp Algorithms

5

Foundations of Constraint Processing Fall 2005

(Some) causes of thrashingSearch order V1 V2 V3 V4 V5

What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc

inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and

V4mdashV5 (path inconsistency)

1 2V4

1 2 3

0 1 2

1 2V5

1 2 3V1 V2

V3ltV2

V3gt0

V3ltV4

V5ltV4

V3ltV5

V3ltV1

V3

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 4: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

4

Foundations of Constraint Processing Fall 2005

Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by

- problem size- amount of backtracking

bull Backtracking may yield

ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again

Malady of backtrack search

Sep 22 2005 ConsistencyProperties amp Algorithms

5

Foundations of Constraint Processing Fall 2005

(Some) causes of thrashingSearch order V1 V2 V3 V4 V5

What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc

inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and

V4mdashV5 (path inconsistency)

1 2V4

1 2 3

0 1 2

1 2V5

1 2 3V1 V2

V3ltV2

V3gt0

V3ltV4

V5ltV4

V3ltV5

V3ltV1

V3

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 5: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

5

Foundations of Constraint Processing Fall 2005

(Some) causes of thrashingSearch order V1 V2 V3 V4 V5

What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc

inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and

V4mdashV5 (path inconsistency)

1 2V4

1 2 3

0 1 2

1 2V5

1 2 3V1 V2

V3ltV2

V3gt0

V3ltV4

V5ltV4

V3ltV5

V3ltV1

V3

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 6: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

6

Foundations of Constraint Processing Fall 2005

Consistency checkingGoal eliminate inconsistent combinations

Algorithms (for binary constraints)

bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity

ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 7: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

7

Foundations of Constraint Processing Fall 2005

Overview of Recommended Reading

bull Davis AIJ77

ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given

bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)

bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2

bull Mackworth and Freuder AIJ85

ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 8: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

8

Foundations of Constraint Processing Fall 2005

Constraint Propagation with Interval Labels Ernest Davis

bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains

ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg

quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning

ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)

Advice read Section 3 more if you wish

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 9: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

9

Foundations of Constraint Processing Fall 2005

Waltz algorithm for label inference

bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C

bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)

bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised

bull Waltz Algorithm ancestor of arc-consistency

C

Vk Vm Vn

Vi

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 10: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

10

Foundations of Constraint Processing Fall 2005

WARNINGrarr Completeness Running the Waltz Algorithm does not solve

the problem

A=2 and B=3 is still not a solution

rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable

x [0 100] x = y

y [0 100] x = 2y

rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm

f(constraint types domain types) see Table 3

1 2 3 2 3 4

BA BA

2 3 2 3

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 11: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

11

Foundations of Constraint Processing Fall 2005

Importance of this paper Establishes that constraints of bounded differences

(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)

Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)

This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 12: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

12

Foundations of Constraint Processing Fall 2005

Basic consistency algorithms

Examining finite binary CSPs and their complexity

Notation

Given variables i j k with values x y z the predicate

Pijk(x y z) is true iff the 3-tuple x y z Cijk

Node consistency checking Pi(x)

Arc consistency checking Pij(xy)

Path consistency bit-matrix manipulation

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 13: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

13

Foundations of Constraint Processing Fall 2005

Worst-case complexity as a function of the input parameters

Upper bound f(n) is O(g(n)) means that f(n) cg(n)

f grows as g or slower

Lower bound f(n) is (h(n)) means that f(n) ch(n)

f grows as g or faster

Input parameters for a CSP

n = number of variables

a = (max) size of a domain

dk = degree of Vk ( n-1)

e = number of edges (or constraints) [(n-1) n(n-1)2]

Worst-case asymptotic complexitytime

space

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 14: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

14

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node amp arc consistency and their

complexity NC AC-1 AC-3 AC-4 3 Criteria for performance comparison and CSP

parameters 4 Path consistency and its complexity5 Consistency properties of a CSP 6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 15: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

15

Foundations of Constraint Processing Fall 2005

Node consistency (NC)Procedure NC(i)

Di Di x | Pi(x)

Begin

for i 1 until n do NC(i)

end

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 16: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

16

Foundations of Constraint Processing Fall 2005

Complexity of NC Procedure of NC(i)

Di Di x | Pi(x) Begin for i 1 until n do NC(i)end

bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 17: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

17

Foundations of Constraint Processing Fall 2005

Arc-consistency Adapted from Dechter

Definition Given a constraint graph G

bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj

bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi

bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 18: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

18

Foundations of Constraint Processing Fall 2005

Procedure ReviseRevises the domains of a variable i

Procedure Revise(ij) Begin

DELETE false for each x Di do

if there is no y Dj such that Pij(x y) then begin

delete x from Di

DELETE true end return DELETE end

bull Revise is directionalbull What is the complexity of Revise

a2

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 19: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

19

Foundations of Constraint Processing Fall 2005

Revise example R Dechter

Apply the Revise procedure to the following example

5 6 15 1 2 10

YX+Y = 10X

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 20: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

20

Foundations of Constraint Processing Fall 2005

Solution DVi DVi Vi (CViVj DVi)

This is actually equivalent to DVi Vi (CViVj DVi)

Effect of Revise Adapted from Dechter

Question Given

bull two variables Vi and Vj

bull their domains DVi and DVj and

bull the constraint CViVj

write the effect of the Revise procedure as a sequence of operations in relational algebra

Hintbull Think about the domain DVi as a unary constraint CVi and

bull consider the composition of this unary constraint and the binary one

1

22

3

1

2

3

1

2

3

Vi VjVjVi

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 21: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

21

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end

bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 22: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

22

Foundations of Constraint Processing Fall 2005

Arc consistency

1 AC may discover the solution Example borrowed from Dechter

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

V2 V3

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 23: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

23

Foundations of Constraint Processing Fall 2005

Arc consistency

2 AC may discover inconsistency Example borrowed from Dechter

1 2 3

1 2 3 1 2 3 YltZ

ZltX

ZY

X

XltY

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 24: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

24

Foundations of Constraint Processing Fall 2005

NC amp AC Example courtesy of M Fromherz

In the temporal problem below

1 AC propagates bound

2 Unary constraint xgt3 is imposed

3 AC propagates bounds again

[0 10]

[0 10]

x y-3

x

y

[0 7]

[3 10]

x y-3

x

y

[4 7]

[7 10]

x y-3

x

y

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 25: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

25

Foundations of Constraint Processing Fall 2005

Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end

Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij

AC-1 is O(a3 n e)

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 26: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

26

Foundations of Constraint Processing Fall 2005

AC versions

bull AC-1 does not update Q the queue of arcs

bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered

bull AC-3 same as AC-2 nodes are not ordered

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 27: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

27

Foundations of Constraint Processing Fall 2005

Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one

node whose domain has been modified

Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 28: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

28

Foundations of Constraint Processing Fall 2005

Complexity of AC-3Procedure AC-3

1 begin

2 for i 1 until n do NC(i)

3 Q (i j) | (i j) arcs(G) i j

4 While Q is not empty do

5 begin

6 select and delete any arc (k m) from Q

7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m

8 end

9 end

bull First |Q| = 2e then it grows and shrinks 48

bull Worst case ndash 1 element is deleted from DVk per iteration

ndash none of the arcs added is in Q

bull Line 7 a(dk - 1)

bull Lines 4 - 8

bull Revise a2 checks of Pij

AC-3 is O(a2(2e + a(2e-n)))

Connected graph (e n ndash 1) AC-3 is O(a3e)

If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)

)2(2121

neaedaen

k k

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 29: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

29

Foundations of Constraint Processing Fall 2005

Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin

DV1 = 1 2 3 4 5

DV2 = 1 2 3 4 5

DV3 = 1 2 3 4 5

DV4 = 1 2 3 4 5

CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)

CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)

CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

V1

V3

V2

V4

V1

V3

V2

V4 1 3 5 1 2 3 5

1 2 3 4 1 3 5

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 30: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

30

Foundations of Constraint Processing Fall 2005

Applying AC-3 Thanks to Xu Lin

bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2

bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4

bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5

bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2

bull Revise(V1 V3) DV1 1 2 3 4 5

bull Queue = CV2V3 CV3 V1 CV3 V2

bull Revise(V2 V3) DV2 1 2 3 4

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 31: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

31

Foundations of Constraint Processing Fall 2005

Applying AC-3 (cont) Thanks to Xu Lin

bull Queue = CV3 V1 CV3 V2

bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5

bull Queue = CV2 V3 CV3 V2

bull Revise(V2 V3) DV2 DV2

bull Queue = CV3 V2

bull Revise(V3 V2) DV3 1 3 5

bull Queue = CV1 V3

bull Revise(V1 V3) DV1 1 3 5 END

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 32: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

32

Foundations of Constraint Processing Fall 2005

Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy

bookkeeping on which and how many values support a given value for a given variable

bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a

given variable

bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 33: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

33

Foundations of Constraint Processing Fall 2005

Step 1 Generate 3 data structures

bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints

TV

TV

TV

TV

TV

m

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

NILV

NILV

NILV

NILV

NILV

s

)4(

)2(

)3(

)2(

)5(

4

2

3

3

4

04)(

01)(

01)(

02)(

03)(

31

32

24

24

24

VV

VV

VV

VV

VV

counter

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 34: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

34

Foundations of Constraint Processing Fall 2005

Step 2 Prepare data structuresData structures s and counter

bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update

ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1

Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))

Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 35: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

35

Foundations of Constraint Processing Fall 2005

Constraints CV2V3 and CV3V2 S

(V4 5) Nil

(V3 2) ((V2 1) (V2 2))

(V3 3) ((V2 5) (V2 2))

(V2 2) ((V31) (V33) (V3 5) (V32))

(V3 1) ((V2 1) (V2 2) (V2 5))

(V2 3) ((V3 5))

(V1 2) Nil

(V2 1) ((V3 1) (V3 2))

(V1 3) Nil

(V3 4) Nil

(V4 2) Nil

(V3 5) ((V2 3) (V2 2) (V2 4))

(v4 3) Nil

(V1 1) Nil

(V2 4) ((V3 5))

(V2 5) ((V3 3) (V3 1))

(V4 1) Nil

(V1 4) Nil

(V1 5) Nil

(V4 4) Nil

Counter(V4 V2) 3 0

(V4 V2) 2 0

(V4 V2) 1 0

(V2 V3) 1 2

(V2 V3) 3 4

(V4 V2) 5 1

(V2 V4) 1 0

(V2 V3) 4 0

(V4 V2) 4 1

(V2 V4) 2 0

(V2 V3) 5 0

(V2 V4) 3 2

(V2 V4) 4 0

(V2 V4) 5 0

(V3 V1) 1 0

(V3 V1) 2 0

(V3 V1) 3 0

hellip hellip

(V3 V2) 4 0

Etchellip Etc

Updating m

Note that (V3 V2)4 0 thus

we remove 4 from the domain of V3

and update (V3 4) nil in m

Updating counter

Since 4 is removed from DV3 then

for every (Vk l) | (V3 4) s[(Vk l)]

we decrement counter[(Vk V3) l] by 1

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 36: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

36

Foundations of Constraint Processing Fall 2005

Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]

bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4

Better average behavior AC-3 [Wallace IJCAI 93]

bull AC-5 special constraints [Van Hentenryck Deville Teng 92]

functional anti-functional and monotonic constraints

bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping[Bessiegravere amp Reacutegin]

bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33 etc

bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988] ndash all-different (dedicated) [Reacutegin 94]

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 37: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

37

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP

parameters bull Which AC algorithm to choose Project results

may tellhellip 4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 38: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

38

Foundations of Constraint Processing Fall 2005

CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness

bull d is constraint density

where e is the constraints emin=(n-1) and emax = n(n-1)2

Lately we use constraint ratio p = eemax

rarr Constraints in random problems often generated uniform

rarr Use only connected graphs (throw the unconnected ones away)

tuplesall

tuplesforbiddent

)(

)(

minmax

min

ee

eed

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 39: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

39

Foundations of Constraint Processing Fall 2005

Criteria for performance comparison

1 Bounding time and space complexity (theoretical)ndash worst-casendash average-case ndash best- case

2 Counting the number of constraint checks (theoretical empirical)

3 Measuring CPU time (empirical)

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 40: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

40

Foundations of Constraint Processing Fall 2005

Performance comparison Courtesy of Lin XU

AC-7

AC-4

AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 41: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

41

Foundations of Constraint Processing Fall 2005

Wisdom () Free adaptation from Bessiegravere

bull When a single constraint check is very expensive to make use AC-7

bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks

are cheap use AC-3x

Advice Instructors personal opinion

Use AC-3

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 42: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

42

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters

4 Path consistency and its complexity

5 Consistency properties of CSPs

6 Other related results box consistency constraint synthesis look-ahead non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 43: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

43

Foundations of Constraint Processing Fall 2005

AC is not enough Example borrowed from Dechter

Arc-consistentSatisfiable

seek higher levels of consistency

V1

a b

a b a bV2 V3

=

V1

b a

a b b aV2 V3

=

=

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 44: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

44

Foundations of Constraint Processing Fall 2005

Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))

a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied

(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 45: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

45

Foundations of Constraint Processing Fall 2005

Note The same variable can appear more than once in the path

Every time it may have a different value

Constraints considered PV0Vm and those along the path

All other constraints are neglected

for all x DV0 for all y DVm

V0 Vm

Vm-1

V2

V1

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 46: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

46

Foundations of Constraint Processing Fall 2005

Example consistency of a path Check path length = 2 3 4 5 6

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 47: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

47

Foundations of Constraint Processing Fall 2005

Path consistency definition

A path of length m is path consistent A CSP is path consistent Property of a

CSP

Definition A CSP is path consistent (PC) iff every path is

consistent (ie any length of path)

Question should we enumerate every path of any length

Answer No only length 2 thanks to [Mackworth AIJ77]

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 48: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

48

Foundations of Constraint Processing Fall 2005

Making a CSP Path Consistent (PC)

Special case Complete graphTheorem In a complete graph if every path of length 2 is

consistent the network is path consistent [Mackworth AIJ77]

PC-1 two operations composition and intersection

Proof by induction

Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is

consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]

PPC (partially path consistent) PC

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 49: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

49

Foundations of Constraint Processing Fall 2005

Tools for PC-1

Two operators

1 Constraint composition ( bull )

R13 = R12 bull R23

2 Constraint intersection ( )

R13 R13 old R13 induced

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 50: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

50

Foundations of Constraint Processing Fall 2005

Path consistency (PC-1)Achieved by composition and intersection (of binary relations

expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 51: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

51

Foundations of Constraint Processing Fall 2005

Properties of PC-1Discrete CSPs [Montanari74]

1 PC-1 terminates2 PC-1 results in a path consistent CSP

bull PC-1 terminates It is complete sound (for finding PC network)

bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)

Complexity of PC-1

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 52: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

52

Foundations of Constraint Processing Fall 2005

Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn

6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do

9 Ylij Yl-1

ij Yl-1ik bull Yl-1

kk bull Yl-1kj

10 end11 until Yn = Y0

12 Y Yn

10 end

Line 9 a3

Lines 6ndash10 n3 a3

Line 3 at most n2 relations x a2 elements

PC-1 is O(a5n5)

PC-2 is O(a5n3) and (a3n3)

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 53: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

53

Foundations of Constraint Processing Fall 2005

Some improvementsbull Mohr amp Henderson (AIJ 86)

ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal

bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time

bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)

Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 54: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

54

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Path consistency corresponds to inferring a new constraint

(alternatively tightening an existing constraint) between every two

variables given the constraints that link them to a third variable

Considers all subgraphs of 3 variables

3-consistency

B

C

A

B lt C

A lt B

B

C

AA lt B

A lt C

B lt C

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 55: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

55

Foundations of Constraint Processing Fall 2005

Path consistency as inference of binary constraints

Another example

V4

V3

a b

V4

V2

V1

a b

V3

a b

a b

V2

a b

V1

a b

a b

=

=a b a b

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 56: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

56

Foundations of Constraint Processing Fall 2005

Question Adapted from Dechter

Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra

B

C

A

B lt C

A lt B

A + 3 gt C

B

C

A

A lt B

A + 3 gt C

A lt C

B lt C

B

C

A

A lt B

-3 lt A ndashC lt 0

B lt C

Solution CViVj CViVj ij(CViVk CVkVj)

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 57: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

57

Foundations of Constraint Processing Fall 2005

Constraint propagation courtesy of Dechter

After Arc-consistency

After Path-consistency

bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent

bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution

1

2

3

1

2

3

1

2 2

3

( 0 1 )

( 0 1 )( 0 1 )

( 0 1 )

( 0 1 )( 0 1 )

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 58: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

58

Foundations of Constraint Processing Fall 2005

Arc-consistent

Path-consistent

PC can detect unsatisfiability

V3

a b

V4

a bV2

V1 a b

a b

a b

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 59: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

59

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP

parameters bull Which AC algorithm to choose Project results may

tell4 Path consistency and its complexity5 Consistency properties of a CSP6 Other related results box consistency

constraint synthesis look-ahead non-binary constraints etc

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 60: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

60

Foundations of Constraint Processing Fall 2005

Consistency properties of a CSP

bull (Arc Consistency AC)bull (Path Consistency PC)bull Minimalitybull Decomposability (ref lossless join decomposition)bull When PC approximates minimality and

decomposability ndash Example temporal reasoning

bull (strong) k-consistencybull (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 61: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

61

Foundations of Constraint Processing Fall 2005

MinimalityPC tightens the binary constraints

The tightest possible binary constraints yield the minimal network

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 62: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

62

Foundations of Constraint Processing Fall 2005

Minimal CSP

Minimal network aka central problem

Given two values for two variables if they are consistent then they appear in at least one solution

Informallybull In a minimal CSP the remainder of the CSP does not add any

further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth

AIJ77]

bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]

bull The binary constraints are explicit as possible [Montanari74]

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 63: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

63

Foundations of Constraint Processing Fall 2005

Decomposabilitybull Any combination of values for k variables that satisfy the constraints

between them can be extended to a solution

bull Decomposability generalizes minimality

Minimality any consistent combination of values for

any 2 variables is extendable to a solution

Decomposability any consistent combination of values for

any k variables is extendable to a solution

Decomposable Minimal Path Consistent

Strong n-consistent n-consistent Solvable

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 64: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

64

Foundations of Constraint Processing Fall 2005

Relations to (theory of) DB

CSP DatabaseMinimal Pair-wise consistent

Decomposable Complete join

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 65: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

65

Foundations of Constraint Processing Fall 2005

PC approximates In generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)

When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]

PC-1 on the completed graph guarantees minimality and decomposability

bull When constraints are convex [Bliek amp Sam-Haroud 99]

PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 66: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

66

Foundations of Constraint Processing Fall 2005

PC algorithms and propertyCSP0 Complete

the graph PC-1

CSP0 Triangulate the graph PPC

Result

bull CSP0 is not path consistent or

bull PC-1(CSP0) is path consistent

1 It is tight but could be tighter

2 The graph is complete

Result

bull CSP0 is not path consistent or

bull PPC(CSP0) is path consistent

1 It is generally less tight than PC-1(CSP0)

2 The graph is triangulatedWatch for PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 67: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

67

Foundations of Constraint Processing Fall 2005

PC algorithms special casesCSPa Complete

the graph PC-124Result [Montanari 74]

bull CSPa is not path consistent or

bull PC-1(CSPa) is

1 Path consistent minimal amp decomposable

2 Graph is complete

With a(bc)=(ab)(bc)

CSPb Triangulatethe graph PPC

Constraint are convex

Result [BampS-H 99]

bull CSPb is not path consistent or

bull PPC(CSPb) is

1 Path consistent minimal decomposable

2 Existing edges are as tight as PC-1(CSPb)

3 Graph is triangulated

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 68: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

68

Foundations of Constraint Processing Fall 2005

Special conditions examples

bull Distributivity propertyndash 1 loop in PC-1 (PC-3) is sufficient

bull Exploiting special conditions in temporal reasoning ndash Temporal constraints in the Simple Temporal Problem

(STP) composition amp intersectionndash Composition distributes over intersection

bull PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)

ndash Convex constraintsbull PPC

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 69: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

69

Foundations of Constraint Processing Fall 2005

Distributivity propertyIn PC-1 two operations

RAB bull (RBC RBC) = (RAB bull RBC) (RAB bull RrsquoBC)

When ( bull ) distributes over ( ) then [Montanari74]

1 PC-1 guarantees that CSP is minimal and decomposable

2 1 loop in PC-1 is sufficient

Intersection

Composition bull

RAB RrsquoBCRBC

A

B

C

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 70: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

70

Foundations of Constraint Processing Fall 2005

Temporal reasoning constraints of bounded difference

Variables X Y Z etc

Constraints a Y-X b ie Y-X = [a b] = I

Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality

Intersection I1 I2 = [max(a1 a2) min(b1 b2)]

Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)

Proof left as an exercise

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 71: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

71

Foundations of Constraint Processing Fall 2005

Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]

R01 + R13 = [25] + [3 5] = [5 10]

R01 + R13 = [25] + [4 12] = [6 17]

Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]

R01 + (R13 R13) = (R01 + R13) (R01 + R13)

R01 + (R13 R13) = [2 5] + [4 5] = [6 10]

(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]

Path consistency guarantees minimality and decomposability

Example constraints of bounded difference

Rrsquo13

R13 =[35]

R23=[18]R12=[34]

V1 V3

V2

V0 R01=[25] [

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 72: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

72

Foundations of Constraint Processing Fall 2005

PC-1 on the STP

bull PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where ndash composition is lsquoscalar additionrsquo andndash intersection is lsquoscalar minimalrsquo

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 73: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

73

Foundations of Constraint Processing Fall 2005

Convex constraints temporal reasoning (again)

Thanks to Xu Lin (2002)

bull Constraints of bounded difference are convex

bull We triangulate the graph (good heuristics exist)

bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)

bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 74: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

74

Foundations of Constraint Processing Fall 2005

PC is not enough

Arc-consistent

Path-consistent

Satisfiable

we should seek (even) higher levels of consistency

k-consistency k = 1 2 3 hellip

a b c

V2

a b c

V3

a b c

V1

a b c

V4

a b cV7

a b c V5

a b c

V6

All mutex constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 75: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

75

Foundations of Constraint Processing Fall 2005

k-consistencyA CSP is k-consistent iff

ndash given any consistent instantiation of any (k-1) distinct variables

ndash there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables

Example

(courtesy of Dechter) Q

Q

Q

Q

Q

3-consistent 4-consistent

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 76: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

76

Foundations of Constraint Processing Fall 2005

bull Questionndash Is this CSP 3-consistent ndash is it 2-consistent

bull Lesson ndash 3-consistency does not guarantee 2-consistency

Warning Does 3-consistency guarantee 2-consistency

red blue

A

B

red

red blue

red

C

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 77: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

77

Foundations of Constraint Processing Fall 2005

(Strong) k-consistency definition

Definitions

bull k-consistency vs strong k-consistency

bull k-consistency ndash any consistent assignment for (k-1)-variable

can be extended to any kth variable

bull strong k-consistency ndash has to be j-consistent for all j k

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 78: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

78

Foundations of Constraint Processing Fall 2005

(ij)-consistency definition

bull (ij) consistency ndash every consistent instantiation to i variablesndash can be extended to every j variables

bull Consequently ndash 2-consistency i=1 j=1ndash 3-consistency i=2 j=1ndash k-consistency i=k-1 j=1ndash Minimality i=2 j=n-2ndash Decomposability i=k j=n-k

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 79: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

79

Foundations of Constraint Processing Fall 2005

Consistency properties amp algorithms summary

bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC

bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition

intersection in relational algebrabull Path consistency approximates minimality which

approximates decomposabilitybull Situations when path consistency guarantees minimality

and decomposabilitybull Strong k-consistency and (ij)-consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 80: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

80

Foundations of Constraint Processing Fall 2005

Outline1 Motivation and material

2 Node and arc consistency and their complexity

3 Criteria for performance comparison and CSP parameters bull Which AC algorithm to choose Project results may tell

4 Path consistency and its complexity

5 Consistency properties of a CSP

6 Other related results bull constraint synthesisbull box consistency bull look-ahead bull non-binary CSPs etc

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 81: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

81

Foundations of Constraint Processing Fall 2005

Constraint synthesisbull k-consistency an alternative view

ndash Constraint of arity (k-1) are used to synthesize a constraint of arity k which is then used to filter constraints of arity (k-1)

bull Examplesndash 2-consistency DVi DVi Vi (CViVj DVj)

Projection of binary constraint is used to tighten unary constraint

ndash 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Binary constraints are used to synthesize a ternary constraintProjection of the ternary constraint is used to tighten binary constraint

ndash k-consistency

[Freuder 78 82]

Constraints of arity (k-1) are used to synthesize constraints of arity kProjection of k-ary constraint is used to tighten (k-1)-ary constraints

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 82: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

82

Foundations of Constraint Processing Fall 2005

Solving CSPs by Constraint synthesis [Freuder 78]

bull From k=2 to k=n ndash achieve k-consistency by using (k-1)-arity

constraints to synthesize k-arity constraints ndash then use the k-ary constraints to filter

constraints of arity k-1 k-2 etc

bull Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 83: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

83

Foundations of Constraint Processing Fall 2005

Interval constraints

bull Domains are (continuous) intervals ndash Historically also called continuous CSPs continuous

domains

bull Domains are infinite ndash We cannot enumerate consistent valuestuplesndash [Davis AIJ 87] (see recommended reading) showed

that even AC may be incomplete or not terminate

bull We apply consistency (usually arc-consistency) on the boundaries of the interval

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 84: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

84

Foundations of Constraint Processing Fall 2005

Look-ahead (already discussed)

bull Integration of consistency algorithm with searchbull AC

ndash Forward Checking (FC) [Haralick amp Elliott]

ndash Directional Arc-Consistency (DAC) [Dechter]

ndash Maintaining arc consistency (MAC) [Sabin amp Freuder]

bull PCndash Directed path consistency (DPC) [Dechter]

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 85: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

85

Foundations of Constraint Processing Fall 2005

Non-binary CSPs

bull (Almost) all algorithms and properties discussed so far were restricted to binary CSPs

bull Consistency properties for non-binary CSPs are the topic of current research Mainly properties and algorithms forndash GAC generalized arc-consistency [Mohr amp Masini]

ndash Relational m-consistency

[Dechter Chap 8]

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 86: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

86

Foundations of Constraint Processing Fall 2005

Generalized Arc-Consistencybull Conceptually project the constraint on each of

the variables in its scope to tighten the domain of the variable

bull When constraint is not defined in extension GAC may be problematic (eg NP-hard in TCSP)

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 87: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

87

Foundations of Constraint Processing Fall 2005

Relational consistency

bull Dechter (see Section 811) generalizes consistency properties to non-binary constraints as relational m-consistencyndash Relational 1-consistency = relational arc-

consistency = GAC ndash Relational 2-consistency = relational path-

consistency

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 88: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

88

Foundations of Constraint Processing Fall 2005

Look-ahead in non-binary CSPsbull In binary CSPs when a variable is instantiated

Revise (Vf Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc

bull In non-binary CSPs a constraint that applies to a current variable may involve one or more past variables and one or more future variables The question is when to trigger Revise

bull There are 5 strategies nFC0 nFC1 hellip nFC5 in increasing lsquoaggressivityrsquo [Bessiere et al AIJ 02 ]

bull For example nFC0 triggers Revise only when all but one variable has been instantiated

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc

Page 89: Foundations of Constraint Processing, Fall 2005 Sep 22, 2005Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall.

Sep 22 2005 ConsistencyProperties amp Algorithms

89

Foundations of Constraint Processing Fall 2005

Related results summarybull Interval constraints box consistencybull Solving CSPs by Constraint synthesis [Freuder]

ndash From k=2 to k=n-1 achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints then filter constraints of arity k-1 k-2 etc

bull Integration with searchndash AC

bull Forward Checking (FC) [Haralick amp Elliott] bull Directional Arc-Consistency (DAC) [Dechter]

bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]

bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]

ndash Relational consistency [Dechter]

bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5

bull etc