SatisfiabilitySatisfiability

Introduction toIntroduction toArtificial IntelligenceArtificial Intelligence

COS302COS302

Michael L. LittmanMichael L. Littman

Fall 2001Fall 2001

AdministrationAdministration

Questions?Questions?

Types of LogicsTypes of Logics

Logic historically a hot topic in AI.Logic historically a hot topic in AI.• Propositional logic: Boolean Propositional logic: Boolean

variables (simple)variables (simple)• First-order logic: more advanced First-order logic: more advanced

types, objects (expressive)types, objects (expressive)

Book covers first-order logic. Focus Book covers first-order logic. Focus here on propositional.here on propositional.

Propositional SyntaxPropositional Syntax

Formula:Formula:• Constants: T, FConstants: T, F

• Variables: xVariables: x11,…,x,…,xnn..

• Negation: ~f (f formula)Negation: ~f (f formula)• Literal: variable or its negationLiteral: variable or its negation• Grouping: (f) (f formula)Grouping: (f) (f formula)• Binary expressions nextBinary expressions next

Binary ExpressionsBinary Expressions

Given formulae f and g:Given formulae f and g:• Conjunction (“and”): fgConjunction (“and”): fg• Disjunction (“or”): f+gDisjunction (“or”): f+g• Implication: fImplication: fgg• Equivalence: fEquivalence: fgg

Truth TablesTruth Tables

xx yy xyxy x+y xx+y xy xy xy y

TT TT

TT FF

FF TT

FF FF

TTFF

FF

FF

TTTT

TT

FF

TTFF

FF

TT

TTFF

TT

TT

Some EquivalencesSome Equivalences

Write xy in terms of + and ~:Write xy in terms of + and ~:• ~(~x+~y)~(~x+~y)

Write xWrite xy in terms of the othersy in terms of the others• xy+(~x)(~y)xy+(~x)(~y)

Write xWrite xy in terms of the othersy in terms of the others• ~x+y ~x+y • ~(x(~y))~(x(~y))

CNFCNF

Propositional logic syntax is pretty Propositional logic syntax is pretty simple, but can be even simpler.simple, but can be even simpler.

Conjunctive normal form (CNF) is a Conjunctive normal form (CNF) is a conjunction of disjunction of literals conjunction of disjunction of literals (clauses).(clauses).

(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(x+~v+u)(x+~v+u)

Truth Table to CNFTruth Table to CNF

1.1. Put negation of formula in DNFPut negation of formula in DNF• For each “F” row in table, make a For each “F” row in table, make a

term equivalent to the corresponding term equivalent to the corresponding assignmentassignment

2.2. Negate the negationNegate the negation• By DeMorgan’s Law, ands and ors By DeMorgan’s Law, ands and ors

swap and literals negateswap and literals negate

CNF ExampleCNF Example

Express xExpress xy in CNFy in CNF

1.1. Two cases for “F”: x=T, y=F and Two cases for “F”: x=T, y=F and x=F, y=Tx=F, y=T

2.2. Negation in DNF: x(~y)+(~x)yNegation in DNF: x(~y)+(~x)y

3.3. Negate it: (~x+y)(x+~y)Negate it: (~x+y)(x+~y)

It works!It works!

Assignments & ModelsAssignments & Models

Assignment: Mapping of n variables Assignment: Mapping of n variables to truth valuesto truth values

u=F, v=T, w=F, x=T, y=F, z=Tu=F, v=T, w=F, x=T, y=F, z=T

Satisfying assignment (model): Satisfying assignment (model): Makes the formula evaluate to TMakes the formula evaluate to T

(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(x+~v+u)(x+~v+u)

64 assignments, 31 models.64 assignments, 31 models.

Categories of FormulaeCategories of Formulae

A Boolean formula can be:A Boolean formula can be:• Valid (tautology): all assignments Valid (tautology): all assignments

satisfying.satisfying.• Satisfiable: at least one assignment Satisfiable: at least one assignment

true.true.• Unsatisfiable: none true.Unsatisfiable: none true.

Computational ProblemsComputational Problems

Given a formula, determine if it is Given a formula, determine if it is valid: reasoning, proof generation.valid: reasoning, proof generation.

Given a formula, determine if it is Given a formula, determine if it is satisfiable (SAT): search.satisfiable (SAT): search.

~valid(f) = satisfiable(~f)~valid(f) = satisfiable(~f)

Both hard!Both hard!

SAT as CSPSAT as CSP

SAT is determining satisfiability of SAT is determining satisfiability of formula in CNF. Can be solved as a formula in CNF. Can be solved as a CSP!CSP!

(x+y)(~x+~y)(x+y)(~x+~y)

Variables are variablesVariables are variables

Domain is T, FDomain is T, F

Clauses are constraintsClauses are constraints

Generic CSP AlgorithmGeneric CSP Algorithm

• If all values assigned and no If all values assigned and no constraints violated, doneconstraints violated, done

• Apply consistency checkingApply consistency checking• If deadend, backtrackIf deadend, backtrack• Select variable to be assignedSelect variable to be assigned• Select value for the variableSelect value for the variable• Assign variable and recurseAssign variable and recurse

Generic SAT AlgorithmGeneric SAT Algorithm

• If all values assigned and no If all values assigned and no constraintsconstraints violated, done violated, done

• Apply Apply consistency checkingconsistency checking• If If deadenddeadend, backtrack, backtrack• Select variable to be assignedSelect variable to be assigned• Select value for the variableSelect value for the variable• Assign variable and recurseAssign variable and recurse

Generic SAT AlgorithmGeneric SAT Algorithm

• If all values assigned and no If all values assigned and no clausesclauses violated, done violated, done

• Apply Apply consistency checkingconsistency checking• If If deadenddeadend, backtrack, backtrack• Select variable to be assignedSelect variable to be assigned• Select value for the variableSelect value for the variable• Assign variable and recurseAssign variable and recurse

Generic SAT AlgorithmGeneric SAT Algorithm

• If all values assigned and no If all values assigned and no clausesclauses violated, done violated, done

• Apply Apply unit propagationunit propagation• If If deadenddeadend, backtrack, backtrack• Select variable to be assignedSelect variable to be assigned• Select value for the variableSelect value for the variable• Assign variable and recurseAssign variable and recurse

Generic SAT AlgorithmGeneric SAT Algorithm

• If all values assigned and no If all values assigned and no clausesclauses violated, done violated, done

• Apply Apply unit propagationunit propagation• If If unsatisfied clauseunsatisfied clause, backtrack, backtrack• Select variable to be assignedSelect variable to be assigned• Select value for the variableSelect value for the variable• Assign variable and recurseAssign variable and recurse

Pure VariablesPure Variables

(x+y+z)(x+~y+~w)(w+~z+y)(x+y+z)(x+~y+~w)(w+~z+y)

If x is a pure literal (never appears If x is a pure literal (never appears negated), then if there is a negated), then if there is a satisfying assignment with x=F, satisfying assignment with x=F, there must also be one with x=T.there must also be one with x=T.

So, we need only check one case (no So, we need only check one case (no branching).branching).

Purification at WorkPurification at Work

(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(x+~v+u)(~x+w+v)(x+z+~y)(~w+~y+~v)(v+u+y)(x+~v+u)

z=Tz=T

(~x+w+v)(~w+~y+~v)(v+u+y)(x+~v+u)(~x+w+v)(~w+~y+~v)(v+u+y)(x+~v+u)

u=Tu=T

(~x+w+v)(~w+~y+~v)(~x+w+v)(~w+~y+~v)

x=Fx=F

(~w+~y+~v)(~w+~y+~v)

y=Fy=F

Formula satisfiedFormula satisfied

DPLLDPLL

Davis-Putnam-Logemann-Loveland Davis-Putnam-Logemann-Loveland (1962) basis of practical SAT (1962) basis of practical SAT algorithmsalgorithms

• Recursive: stop if SAT or UNSATRecursive: stop if SAT or UNSAT• Unit propagation, recurseUnit propagation, recurse• Purification, recursePurification, recurse• Else, split and recurse on bothElse, split and recurse on both

Splitting HeuristicsSplitting Heuristics

How choose a variable to split?How choose a variable to split?• Most occurrencesMost occurrences• In short clausesIn short clauses• Lots more of one kind of literal than Lots more of one kind of literal than

anotheranother

www.ee.princeton.edu/~chaffwww.ee.princeton.edu/~chaff

DPLL AnalysisDPLL Analysis

n variables. Worst case?n variables. Worst case?

Split on a variable in a shortest Split on a variable in a shortest clause.clause.

What if only k literals per clause (k-What if only k literals per clause (k-CNF)? Say, k=2?CNF)? Say, k=2?

Analysis of 2-CNFAnalysis of 2-CNF

Can be made to run in polynomial Can be made to run in polynomial time.time.

Analysis of 3-CNFAnalysis of 3-CNF

(x+y+z)(~x+u+v)…(x+y+z)(~x+u+v)…

x=T: (u+v)…x=T: (u+v)…

u=T: … (2 vars eliminated)u=T: … (2 vars eliminated)

u=F, v=T: … (3 vars eliminated)u=F, v=T: … (3 vars eliminated)

x=F: (y+z)… (same idea)x=F: (y+z)… (same idea)

R(n) R(n) 2 R(n-2) + 2 R(n-3) 2 R(n-2) + 2 R(n-3)

1.7691.769nn

Analysis ImprovementsAnalysis Improvements

SAT (3-CNF)

11.21.41.61.8

2

1960 1980 2000 2020 2040

Publication Year

Ex

po

ne

nt

ba

se

Analysis ImprovementsAnalysis Improvements

SAT (3-CNF)

11.21.41.61.8

2

1960 1980 2000 2020 2040

Publication Year

Ex

po

ne

nt

ba

se

PHP: Propositional ProofPHP: Propositional Proof

Pigeonhole Principle:Pigeonhole Principle:• If you have n+1 pigeons and n If you have n+1 pigeons and n

holes and each pigeon is assigned holes and each pigeon is assigned a hole, then some hole contains at a hole, then some hole contains at least 2 pigeons.least 2 pigeons.

DPLL takes exponential time to prove DPLL takes exponential time to prove validity.validity.

3-CNF Conversion Ex.3-CNF Conversion Ex.

~(~(~x+y) z)~(~(~x+y) z)

Efficient procedure for creating an Efficient procedure for creating an equivalent 3-CNF expression from equivalent 3-CNF expression from an arbitrary propositional an arbitrary propositional expression.expression.

3-CNF Conversion3-CNF Conversion

1.1. Add a variable for each binary Add a variable for each binary operator in the expression.operator in the expression.

2.2. Create a set of 3-CNF clauses for Create a set of 3-CNF clauses for each of the derived variables.each of the derived variables.

3.3. Add a clause for the root node.Add a clause for the root node.

What to LearnWhat to Learn

Definition of SAT.Definition of SAT.

How to make a CNF expression from How to make a CNF expression from a truth table.a truth table.

The DPLL algorithm.The DPLL algorithm.

How to make a 3-CNF expression How to make a 3-CNF expression from an arbitrary expression.from an arbitrary expression.

Homework 3Homework 3

1.1. Let f=~(x+ ~y(~x+z)). (a) Write out the Let f=~(x+ ~y(~x+z)). (a) Write out the truth table for f. (b) Convert the truth table truth table for f. (b) Convert the truth table to CNF. (c) Show the series of steps DPLL to CNF. (c) Show the series of steps DPLL makes while solving the resulting formula. makes while solving the resulting formula. Assume variables chosen for splitting in the Assume variables chosen for splitting in the order x, y, z.order x, y, z.

2.2. Using the same f from the first part, follow Using the same f from the first part, follow the 3-CNF conversion algorithm to create an the 3-CNF conversion algorithm to create an equivalent 3-CNF formula. equivalent 3-CNF formula.