1

Exploiting Random Walk Strategies in

Reasoning

Wei Wei

2

Universal inference engine One of the oldest AI dreams

General Problem Solver (Newell & Simon, 1961)Use logic reasoning (“Program with Common Sense”, McCart

hy 1968) Largely unsuccessful due to the tradeoff between co

mputational complexity and representation power. => many successful systems are domain-specific (e.g. Dendral, 1968-85, organic chemistry)

Recent improvement in computer’s abilities to perform large-scale search => Revisit the original idea

3

Computational Power - SAT Hardware power Average-case

complexity New algorithmic tools:

randomization, learning

Increasing demand: verification

Research input: annual conference, competition

year #variable

1962 50

1992 300

1996 1,000

2002 1,000,000

4

Boolean Satisfiability Problem Boolean Satisfiability Problem (SAT) asks if a Boolean

expression can be made true by assigning Boolean values to its variables.

The problem is well-studied in AI community with direct application in reasoning, planning, CSP, VLSI microprocessor verification etc.

Does statement s hold in world A (represented by a set of clauses)?

A s (¬s) ^ A unsatisfiable

5

SAT SAT (even 3SAT) is NP-complete. Best theoreti

cal bound so far is (2-2/k)N randomized (Schoening 1999) or

(2-2/(k+1))N deterministic (Dantsin et al 2002) for k-SAT

In practice, there are two different kinds of solvers DPLL (Davis, Logemann and Loveland 1962) Local Search (Selman et al 1992)

6

DPLL (x1 x2 x3) (x1 x2 x3) (x1 x2) DPLL was first proposed as a basic depth-

first tree search.

x1

x2

FT

T

null

F

solution

x2

Potential problem: early commitment

7

DPLL Recently (since late 90’s), many

improvements: Randomization restarts out-of-order backtracking clause learning

8

Local Search(x1 x2 x3) (x1 x2 x3) (x1 x2 ) The idea: Start with a random assignment.

And make local changes to the assignment until a solution is reached (010 011 001)

Pro: often efficient in practice. Sometimes the only feasible way for some problems

Con: Cannot prove nonexistence of solutions. Difficult to analyze theoretically.

Example GSAT (Selman et al. 1992)

9

Local Search Schemes local search schemes used:

Simulated annealing Tabu search Genetic algorithms Random Walk and its variants the most successf

ul so far

10

Unbiased (Pure) Random Walk for SATProcedure Random-Walk (RW)Start with a random truth assignmentRepeat

c:= an unsatisfied clause chosen at randomx:= a variable in c chosen at randomflip the truth value of x

Until a satisfying assignment is found

11

Unbiased RW on any satisfiable 2SAT Formula

Given a satisfiable 2SAT formula with n variables, a satisfying assignment will be reached by Unbiased RW in O(n2) steps with high probability. (Papadimitriou, 1991)

12

We have an unbiased random walk with a reflecting barrier at distance N from T (max Hamming distance) and an absorbing barrier (satisfying assignment) at distance 0.

We start at a Hamming distance of approx. ½ N.

Property of unbiased random walks: after N^2 flips, with high probability, we will hit the origin (the satisfying assignment). (Drunkards walk)

So, O(N^2) randomized algorithm (worst-case!) for 2-SAT.

T A0T A0

reflecting

13

Unbiased RW on 3SAT Formulas

Random walk takes exponential number of steps to reach 0.

T A0

reflecting

14

Comments on RW

1) Random Walk is highly “myopic” does not take into account any gradient of the objective function (= number of unsatisfied clauses)!

Purely “local” fixes.

2) Can we make RW practical for SAT? Yes --- inject greedy bias into walk biased Random Walk.

15

Biased Random Walk

Procedure WalkSatRepeat

c:= an unsatisfied clause chosen at random

if there exist a variable x in c with break value = 0 // greedy biasflip the value of x (“freebie move”)

else with probability p // pure walk

x:= a variable in c chosen at random flip the value of x

with probability (1-p)x:= a variable in c with smallest break value // more greedy biasflip the value of x

Until a satisfying assignment is found

Note: tune parameter p.

16

Random Walk algorithms Random walk algorithm (e.g. Walksat) offer sig

nificant improvement on performance over hill-climbing algorithms.

ID Vars clauses GSAT+w WalksatSsa7552-038 1501 3575 129 2.3

Ssa7552-158 1363 3034 90 2

Ssa7552-159 1363 3032 14 0.8

Ssa7552-160 1391 3126 18 1.5

17

First, bringing out the worst in random walks…

(2-SAT)

X1 X2

X2 X3

X3 X4

Xn X4

Xn X1

Note: Only 2 satisfying assignments, all False and all True.

18

Binary Chains

Walk is exactly balanced.

19

Results: Speeding up Random Walks on Binary Chains

*: empirical results**: proof available

Pure binary chain

Chain withredundancies

RW (n2)** (n2)**

RWF (n2)** (n1.2)*

WalkSat (n2)** (n1.1)*

Becomesalmost likeunit prop.

20

Ternary Chains

In general, even a small bias in the wrong direction leads to exponential time to reach 0.

reflecting

21

What about 3-SAT? Again, consider “chain” formulas.

X1 & X2 X3

X2 & X3 X4

X_(n-2) & X_(n-1) X_n

X1 & X2

X1 & X2

X_floor(n/2) & X_(n-1) X_n

22

Theoretical Results on 3-SAT Chains

Function low(i) Expected run time of pure RW

i-2 (highly local) ~ Fib(n) (i.e., exp.)

i/2 (interm. reach) ~ nlog n (i.e., quasi-poly)

log i (interm. reach) ~ n2 . (log n)2 (i.e., poly)

1 (full back reach) ~ n2

low(i) captures how far back the clauses reach.

23

Recurrence RelationsOur formula structure gives us:

E(f(zi)) = (E(f(zlow(i)) + E(f(zi) + 1) * 1/3

+ (E(f(zi-1) + E(f(zi) + 1) * 1/3

+ 1 * 1/3

E(f(zi)) = E(f(zlow(i)) + E(f(zi-1) + 3

24

Decompose: multiple “0”s

110101010101

110001010001

100001

111001110001

101001111001

111111111101

110101010101

110001100001

111001110001

110101

010001

101001111001111101111111

110111010111

110111100111

111111110111

111101

111001

101001111001111101111111

110111 111111

111101 111111

Start

Sat assign.

25

Recap Chain Formula Results Adding implied constraints capturing long-range

dependencies speeds random walk on 2-Chain to near linear time.

Certain long-range dependencies in 3-SAT lead to poly-time convergence of random walks.

Can we take advantage of these results on practical problem instances?

Yes! (next)Our methodology: Identify, analyze, and “exploit” special tractable structure in large practical reasoning problems.

26

Empirical Results

SSS-SAT-1.0 instances (Velev 1999). 100 total. level of redundancy added (20% near optimal)

Formulas(redun. level)

<40 sec <400 sec

<4000 sec

= 0.0 15 26 42

= 0.2 85 98 100

= 1.0 13 33 64

28

Probabilistic Reasoning Previously, we asked “does statement s hold

in world A (represented by a set of clauses)?”

A s (¬s) ^ A unsatisfiable Now, what is the probability that statement s

holds in world A?

Pr = #( s ^ A ) / #A

29

Probabilistic Reasoning Close connection between counting and samp

ling (Jerrum et al, 1986). Bayesian Net Queries can be encoded as #SAT.

(Kautz, 2004) How to Sample:

By repeated counting using DPLL algorithms Monte Carlo Markov Chain Method Use state-of-the-art local search methods Note: Random Walk not uniform

30

Characteristics of Solution space: Solution Clustering Visualization with multi-dimensional scaling (MDS)

Solutions to specific 75 variable, 325 clause 3-SAT instances

75 dimensional solution projected to two dimensions Distance between points approximates hamming distance

31-10 -5 0 5 10 15 20 25-25

-20

-15

-10

-5

0

5

10

15

20SA on SAT-75 - Multi-dimensional Scaling Using Hamming Distance Between Solutions

25 Most Often Found Solutions25 Next Most Often Found SolutionsMiddle Solutions25 Next Least Often Found 25 Least Often Found Solutions

Empirically determined each solution’s probability (uf75-01 - 75 variable, 325 clause 3-SAT instance)

WalkSat finds every solution, but with very large range of probabilities (1:104)

Probability Clusters

Solution Probability Using WalkSat Algorithm

32

Probability Ranges in Different Domains

Instance Runs Hits Rarest

Hits Common

Common-to -Rare

Ratio

Random 50*106 53 9*105 1.7*104

Logistics 1* 106 84 4*103 50

33

Improving the Uniformity of Sampling - mixing sampling strategy

To reduce the range of probabilities, we propose a hybrid local search algorithm: With probability p, the algorithm makes a biased

random walk move With probability 1-p, the algorithm makes a SA (si

mulated annealing) move In our experiments, we used

50% WalkSat + 50% SA at a fixed temperature

34

Results of the Hybrid Approach

Our key figure.

35

Solution Clusters

Results on a random 3-SAT instance (70 vars, 301 clauses, 2531 solutions).

36

Summary

1) WalkSAT does sample all solutions.2) But, sampling can be highly biased.3) Using a new hybrid strategy, we can obtain effective near-uniform sampling.

Lesson: Hybrid of SA and biased walk, is a promising alternative to MCMC methods for sampling.

Idea: Use SAT solvers to sample solutions from a combinatorial space.Findings:

37

Research Directions

Can we exploit techniques from DPLL solvers, such as learned clauses (inferred structure) in local search?

Can we identify other classes of structure in real-world problem that can further accelerate random walk style SAT solvers?

Can we design better SAT encodings based on our insights about structure?

38

Research Directions, cont. Solution sampling

Main Challenge: moving between clusters evaluation tools.

Randomize DPLL to sample solution space Early indicator of solution counts.

Compare with Bayesian/probabilistic state-of-the-art inference methods. Can we outperform them?