Constrainedness

Including slides from Toby Walsh

Constraint satisfaction

• Constraint satisfaction problem (CSP) is a triple <V,D,C> where:– V is set of variables– Each X in V has set of values, D_X

• Usually assume finite domain• {true,false}, {red,blue,green}, [0,10], …

– C is set of constraints

Goal: find assignment of values to variables to satisfy all the constraints

Constraint solver

• Tree search– Assign value to variable– Deduce values that must be removed from future/unassigned

variables• Constraint propagation

– If any future variable has no values, backtrack else repeat

• Number of choices– Variable to assign next, value to assign

Some important refinements like nogood learning, non-chronological backtracking, …

Constraint propagation

• Arc-consistency (AC)– A binary constraint r(X1,X2) is AC iff

for every value for X1, there is a consistent value (often called support) for X2 and vice versa

E.g. With 0/1 domains and the constraint X1 =/= X2Value 0 for X1 is supported by value 1 for X2Value 1 for X1 is supported by value 0 for X2…

– A problem is AC iff every constraint is AC

Tree search

• Backtracking (BT)

• Forward checking (FC)

• Backjumping (BJ, CBJ, DB)

• Maintaining arc-consistency (MAC)

• Limited discrepancy search (LDS)

• Non-chronological backtracking & learning

• Probing

• …

Modelling

• Choose a basic model

• Consider auxiliary variables– To reduce number of

constraints, improve propagation

• Consider combined models– Channel between views

• Break symmetries

• Add implied constraints– To improve propagation

Propositional Satisfiability

• SAT– does a truth assignment exist that

satisfies a propositional formula?– special type of constraint

satisfaction problem• Variables are Boolean• Constraints are formulae

– NP-complete

• 3-SAT– formulae in clausal form with 3

literals per clause– remains NP-complete

(x1 v x2) & (-x2 v x3 v -x4)

x1/ True, x2/ False, ...

Random 3-SAT

• Random 3-SAT– sample uniformly from space

of all possible 3-clauses– n variables, l clauses

• Which are the hard instances?– around l/n = 4.3

What happens with larger problems?

Why are some dots red and others blue?

Random 3-SAT

• Varying problem size, n

• Complexity peak appears to be largely invariant of algorithm– backtracking algorithms

like Davis-Putnam– local search procedures like

GSAT

What’s so special about 4.3?

Random 3-SAT

• Complexity peak coincides with solubility transition

– l/n < 4.3 problems under-constrained and SAT

– l/n > 4.3 problems over-constrained and UNSAT

– l/n=4.3, problems on “knife-edge” between SAT and UNSAT

• Shape– “Sharp” (in a technical sense) [Friedgut 99]

• Location– 2-SAT occurs at l/n=1 [Chavatal & Reed 92, Goerdt

92]

– 3-SAT occurs at 3.26 < l/n < 4.598

Theoretical results

“But it doesn’t occur in X?”

• X = some NP-complete problem

• X = real problems

• X = some other complexity class

“But it doesn’t occur in X?”

• X = some NP-complete problem

• Phase transition behaviour seen in:– TSP problem (decision not optimization)– Hamiltonian circuits (but NOT a complexity peak)– number partitioning– graph colouring– independent set– ...

“But it doesn’t occur in X?”

• X = real problems

• Phase transition behaviour seen in:– job shop scheduling problems– TSP instances from TSPLib– exam timetables @ Edinburgh– Boolean circuit synthesis – Latin squares (alias sports scheduling)– ...

“But it doesn’t occur in X?”

• X = some other complexity class

• Phase transition behaviour seen in:– polynomial problems like arc-consistency– PSPACE problems like QSAT and modal K– ...

Algorithms at the phase boundary

What do we understand about problem hardness at the phase

boundary?

How can this help build better algorithms?

N

Sol )(log1 2

Kappa

dnessconstraine of measure a kappa,

|)(|log

solutions ofnumber expected the

space statein that solutions ofset the

space state that of size the||

problem a of space state

2

SN

Sol

Sol

S

S

Defined for an ensemble of problems

N

Sol )(log1 2 Kappa (motivation)

|)(|log

)(log1

2

2

S

Sol

Kappa

When every state is a solution 0|)(|log

)(log1

2

2

S

Sol

When every state is not a solution |)(|log

)0(log1

2

2

S

When there is a unique solution 1|)(|log

)1(log1

2

2 S

|)(|log

)(log1

2

2

S

Sol

Kappa

When every state is a solution 0|)(|log

)(log1

2

2

S

Sol

When every state is not a solution |)(|log

)0(log1

2

2

S

When there is a unique solution 1|)(|log

)1(log1

2

2 S

Easy Soluble

Easy Insoluble

On the knife edge

Hard

An Example: random CSP’s

Each of n variables has a uniform domain size mThere is a probability p1 of a constraint between a pair of variablesThere is a probability p2 that a pair of values conflict (when a constraint exists)

21,,, ppmn

tightnessconstraint

graph constraint ofdensity

sizedomain uniform

variablesofnumber

2

1

p

p

m

n

An Example: random CSP’s 21,,, ppmn

21

2

2/)1(22

)2/)1((2

1

1log

2

1

)(log

)1((log1

)1(

||

1

1

pp

n

m

pm

pmSol

mS

m

n

nnpn

nnpn

n

|)(|log

)(log1

2

2

S

Sol

An Example: random CSP’s 21,,, ppmn

21 1

1log

2

1

pp

nm

|)(|log

)(log1

2

2

S

Sol

Can rearrange this formula so that we find the value of p2 such that we are at the phase transition, i.e. when kappa = 1

Then perform experiments to see if it is accurate predictori.e. we put theory to the test

Kappa as a heuristic

Vvv

Ccc

m

p

)(log

)1(log

2

2

Minimise that

An Example: not random CSP’s

Vvv

Ccc

Ccc

Vvv

Vvv

m

p

pmSol

mS

)(log

)1(log

)1(

||

2

2

|)(|log

)(log1

2

2

S

Sol

variablea of sizedomain

constraint a of tightness

sconstraint ofset in the constraint a

variablesofset in the variablea

v

c

m

p

Cc

Vv

An Example: not random CSP’s

Vvv

Ccc

m

p

)(log

)1(log

2

2

|)(|log

)(log1

2

2

S

Sol

variablea of sizedomain

constraint a of tightness

sconstraint ofset in the constraint a

variablesofset in the variablea

v

c

m

p

Cc

Vv

Kappa is general (3 examples)

n

la

)2

11(log2 SAT with n variables, l clauses, a literals/clause

)1

(log

m

m

n

em Graph colouring, n vertices, e edges, m colours

n

lm m )(log

)1(

Number partitioning, n numbers, range (0,l], m bags with same sum

Kappa as a heuristic

Vvv

Ccc

m

p

)(log

)1(log

2

2

Minimise that

Vvv

Ccc

m

p

)(log

)1(log

2

2

Minimise that

• assume we only measure domain size (denominator)• assume we only measure top line (numerator)• assume we measure top and bottom line (numerator and denominator

But that’s costly to do.Is there a low cost surrogate?

Looking inside search

• Three key insights– constrainedness “knife-

edge”– backbone structure– 2+p-SAT

• Suggests branching heuristics– also insight into

branching mistakes

Constrainedness knife-edge

kappa against depth/n

Constrainedness knife-edge

• Seen in other problem domains– number partitioning, …

• Seen on “real” problems– exam timetabling (alias graph colouring)

• Suggests branching heuristic– “get off the knife-edge as quickly as possible”– minimize or maximize-kappa heuristicsmust take into account branching rate, max-kappa often

therefore not a good move!

Minimize constrainedness

• Many existing heuristics minimize-kappa– or proxies for it

• For instance– Karmarkar-Karp heuristic for number partitioning– Brelaz heuristic for graph colouring– Fail-first heuristic for constraint satisfaction– …

• Can be used to design new heuristics– removing some of the “black art”

Backbone

• Variables which take fixed values in all solutions– alias unit prime implicates

• Let fk be fraction of variables in backbone– l/n < 4.3, fk vanishing

(otherwise adding clause could make problem unsat)

– l/n > 4.3, fk > 0discontinuity at phase

boundary!

Backbone

• Search cost correlated with backbone size– if fk non-zero, then can easily assign variable “wrong”

value– such mistakes costly if at top of search tree

• Backbones seen in other problems– graph colouring– TSP– …

Can we make algorithms that identify and exploit the backbone structure of a problem?

2+p-SAT

• 2-SAT is polynomial (linear) but 3-SAT is NP-complete– 2-SAT, unlike 3-SAT, has no

backbone discontinuity

• Morph between 2-SAT and 3-SAT– fraction p of 3-clauses

– fraction (1-p) of 2-clauses

• 2+p-SAT maps from P to NP– p>0, 2+p-SAT is NP-complete