LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it...
Transcript of LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it...
![Page 1: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/1.jpg)
LECTURE 11 & 12
CONSTRAINT-BASED LOCAL SEARCHCONSTRAINT-BASED LOCAL SEARCH
![Page 2: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/2.jpg)
Constraint-based Local Search
• Problem given in CSP form :
– a set of variables V={V1, V2,…, Vn}
– a set of constraints C={C1, C2,…, Ck}
i.e. arithmetic or symbolic relations over V
• Use Local Search to solve the constraints
• Cost function to minimize:
number of violated constraints
• Naturally copes with Over-Constraint problems
![Page 3: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/3.jpg)
Local Search for Constraint Solving
(v1 ... vn)
(v’1 ... v’n)
(w1 ... wn)
solving as optimization :Objective function to minimizee.g. number of unsatisfied constraints
Solution
![Page 4: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/4.jpg)
Ressources
• P. Van Hentenryck and L. Michel
Constraint-based Local Search
MIT Press 2005
• COMET System 2.1 • COMET System 2.1
http://dynadec.com/support/downloads/
• Adaptive Search Algorithm (C library)
http://cri-dist.univ-paris1.fr/diaz/adaptive/
![Page 5: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/5.jpg)
Adaptive Search
• Basic Idea from GSAT
(Local Search for SAT)
[Selman 92, Selman/Kautz 94]
• Extension: n-ary domains + arbitrary constraints
• Iterative repair
at each iteration: 1 variable selected, value is changed
• min-conflict [Minton 92]
• Adaptive memory (cf. Tabu Search)
• Based on variables + constraints information
![Page 6: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/6.jpg)
Adaptive search:
Error Functions
constrainte.g. X = Y
error function which measures how much the constraint is satisfied
e.g. | X- Y |
projection on each variable
![Page 7: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/7.jpg)
Intuitively :
• Search guided by the structure of the problem
– at the variable level
– at the constraint level
– constraints give structure to the problem– constraints give structure to the problem
& variables link them together
• Convergence on variables with « bad » values
• Any kind of constraint can be used (if appropriate error functions is defined)
![Page 8: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/8.jpg)
| V1 - V2 |
max (0, 1 + V1 - V2 )
min (V1 - v), v V2∈
V1= V2
V1 < V2
V1 V2∈
Examples of Error Functions
max (fC1, fC2
)
Card (Vi=Vk, i < k)
max fC1(V1), V1 V2∈
C1 C2
alldiff (V1 … Vn )
V1 V2 , C (V1)
∧
∀ ∈
Increasing (V1 ... Vn ) i < j < nV i > V j
∑ Vi
− Vj
![Page 9: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/9.jpg)
Basic Algorithm
• Start with random configuration
• Repeat
compute the errors on constraints
combine the errors on each variable
select the worst variable X (except if “tabu”)select the worst variable X (except if “tabu”)
compute cost of configurations with other values of X
if a better configuration is found, move to this config.
else mark X tabu & move randomly
• Until
a solution is found
or a maximum number of iterations is reached
![Page 10: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/10.jpg)
Generic Algorithm
• Parametrized by :
– score function for constraints
– Combination function for variables
– cost function for configurations
• Tabu list
– each var. leading to local minimum is marked and
cannot be chosen for a few iterations
• Plateau(x):
– when selected variable has no improving
alternative value (but equal global cost)
![Page 11: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/11.jpg)
Escaping Plateaux
• with prob p : choose a neighbor on the plateau
• with probability 1-p : escape from plateau
(mark current variable as Tabu and randomly
choose another variable) choose another variable)
• empirical best value of p ∈ [0.85,0.95]
• example:
– Magic Square
– p=94%
– About 10 times speed-up !
![Page 12: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/12.jpg)
Also …
• (partial) Resets
• Different from « Restarts »
• Triggered by the number of Tabu variables
• Then reset a given % of variables• Then reset a given % of variables
• Usually about 20%
• Can be specialized for a given problem
• C library available as freeware since 2003
http://cri-dist.univ-paris1.fr/diaz/adaptive
![Page 13: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/13.jpg)
![Page 14: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/14.jpg)
N queens
• N variables
• Configurations = permutation of {1,…,n}
• Constraints (diagonals):
∀ i ∈ [1,N], ∀j ∈ [i+1,N],
Qi+i ≠ Qj +j Qi-i ≠ Qj -jQi+i ≠ Qj +j Qi-i ≠ Qj -j
• Error : 0 if constraint satisfied
1 if constraint violated (Qi = Qj)
• combination on vars. : sum
• cost : sum of errors
• move : swap 2 queens
![Page 15: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/15.jpg)
Performance on N-Queens
N Adaptive Search
(time in seconds)
1000 0.00
10000 0.52
30000 4.64
60000 19.05
100000 55.58
Surprisingly, runtime results are very stable for any given n
![Page 16: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/16.jpg)
Magic Square
• place all the numbers in {1, 2, . . . ,N2} on a
NxN square s.t. sum of the numbers in all
rows, columns and two diagonals are equal
• Constraints are equations
of the form: Σi xi = M
(mean: M = N*(N+1)/2 )
• Plus one all_different constraint
![Page 17: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/17.jpg)
Adaptive Search formulation
• N2 Variables: permutation of {1, 2, . . . ,N2}
• Constraints of the form: Σi xi = M
error fonction for equation: given X=Y error is X-Y
• Combination: sum of errors
(could be: sum of absolute values)
• cost : sum of absolute values
• move : swap 2 values
• all_different constraint is implicit
because moves are swaps of values
![Page 18: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/18.jpg)
Magic Square with AS
![Page 19: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/19.jpg)
Comparison with other solvers
• Local search solver COMET (Dynatec)
![Page 20: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/20.jpg)
All-Intervals series
• Find a permutation σ of {1 ,…, n} s.t.
|σi+1 – σi| is a permutation of {1, … , n-1}
• In fact, this problem comes from musical exercices :
– For n=12, one has musical series
(all notes of the chromatic scale appear exactly once)
where intervals between two consecutive notes are all differentwhere intervals between two consecutive notes are all different
• Constraints :
all_different({ |σi+1 – σi| , i ∈ [1,n-1] } )
• A solution for n=8:
![Page 21: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/21.jpg)
Performance on All-Interval Series
N Adaptive Search
(time in seconds)
100 0.22
150 1.83
200 5.87
300 23.53
![Page 22: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/22.jpg)
Perfect Squares
![Page 23: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/23.jpg)
Performance on Perfect Squares
N Adaptive Search
(time in seconds)
1 2.16
2 4.87
3 6.30
4 6.42
5 22.35
![Page 24: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/24.jpg)
How to do Performance Analysis
• The execution time of LS algorithms (e.g. AS)
varies from one run to another
• Difficult to analyze performance and compare
• A simple thing is to consider mean values• A simple thing is to consider mean values
– Also can add min, max and standard deviation
• But is it enough ?
• In fact larger class of algorithms:
– Las Vegas algorithms
![Page 25: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/25.jpg)
Las Vegas Algorithms
• Proposed by [Babai 79] as extension of Monte Carlo
algorithms
• Any algorithm for which the execution time varies for
different runs on the same input :
– Local Search algorithms & Metaheuristics
– Quicksort with pivot element chosen at random
– Randomized algorithms
![Page 26: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/26.jpg)
Heavy Tailed Distribution
• In the value distribution of a random variable,
some (a few) values can be very big
– distribution is not bounded
• [Gomes 2000]: backtrack search in combinatorial • [Gomes 2000]: backtrack search in combinatorial
problems exhibit Heavy-Tailed distribution
– e.g. 80% runs solve the problem in 1000 iterations
but 5% in more than 1,000,000 iterations
• Similar behavior for Local Search
• Thus motivates the use of restarts
– better performance
![Page 27: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/27.jpg)
Power Law Decay
Heavy Tailed vs. Standard
Standard Distribution(finite mean & variance)
Exponential Decay
[from Gomes 2004]
![Page 28: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/28.jpg)
Histogram of runtime for
Magic Square 100x100
with no restart limit
(bins = 1 second)
![Page 29: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/29.jpg)
Zooming on the histogram
of runtime for
Magic Square 100x100
(4 big runs excluded)
![Page 30: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/30.jpg)
Histogram of runtime for
Magic Square 100x100
with restart limit set
at 100 000 iterations
(± 20 seconds)
![Page 31: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/31.jpg)
Statistical Analysis
• Consider the runtime of the algorithm as a
random variable X
• Use statistical tools to analyze this variable
• Try to find if it matches some well-known • Try to find if it matches some well-known
probability distribution, e.g.:
– Exponential distribution
– Normal (gaussian) distribution
– Lognormal distribution
• Approximate the runtime by this probability
distribution to “predict” runtime behavior
![Page 32: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/32.jpg)
Back to Local Search
• [Hoos / Stutzle 98] proposed to use runtime
distributions to characterize running time of
LS algorithms for combinatorial optimization
• [Aiex / Resende / Ribeiro 02] showed • [Aiex / Resende / Ribeiro 02] showed
(experimentally) that the GRASP
metaheuristic has exponential runtime
distribution on many classical O.R. problems
• [Hoos 97, 98] conjecture that LS solvers for
SAT have exponential runtime behavior
![Page 33: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/33.jpg)
Notations
• Let Y be the runtime of a Las Vegas algorithm
(or the number of iteration of a LS algorithm)
• Cumulative distribution:
– Distribution (probability density function):
= derivative of cumulative distribution
• Expectation:
![Page 34: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/34.jpg)
Basic Exponential Distribution
• Random variable X ~ Exp(λ):
distribution (pdf) cumulative distribution
![Page 35: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/35.jpg)
(Shifted)
Exponential
Distribution
![Page 36: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/36.jpg)
Time-To-Target Plots
• For optimization problems, one can consider the
runtime of the algorithm to produce a solution of
a given target value:
– Notion of time-to-target– Notion of time-to-target
– Time-To-arget plots
• tttplots system [Aiex / Resende / Ribero 2005]
– Hypothesis: runtime distributions can be
approximated by (shifted) exponential distributions
– λ and x0 computed automatically
(λ is close to the mean value of runtimes)
![Page 37: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/37.jpg)
tttplots for AS
• As we are considering satisfaction problems, our
target value is zero (for objective function)
• Can draw tttplots for runtime execution of AS• Can draw tttplots for runtime execution of AS
• Can check if behavior of runtime of AS algorithm
is exponential (or close)
• It depends on the problem !
– e.g. for N-queens runtimes are (approx.) constant …
![Page 38: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/38.jpg)
All-Interval 300
x0 = -1.14
λ = 28.43
![Page 39: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/39.jpg)
Magic square
200x200
restart limit
set at mean
(500 000 iter.)
x0 = -20.57
λ = 353.85λ = 353.85
![Page 40: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/40.jpg)
Q-Q Plots
• Quantile-Quantile plots (Q-Q plots) can be used
to better see how an experimental distribution
matches a theoretical one
• Quantile = point below which a certain fraction • Quantile = point below which a certain fraction
of distribution lies
– E.g. the 0.3 quantile is the point at which 30% falls
below and 70% above
• Q-Q plot points : (theo. quantile, exp. quantile)
• points on the diagonal = perfect match
• Example: runtime versus exponential distribution
![Page 41: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/41.jpg)
Q-Q plot for
Magic Square 100x100
with no restart limit
![Page 42: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/42.jpg)
Q-Q plot for
Magic Square 100x100
with restart limit at mean
![Page 43: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/43.jpg)
Q-Q plot for
Magic Square 100x100
with restart limit at mean/2
![Page 44: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/44.jpg)
Parallel Local Search Parallel Local Search
![Page 45: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/45.jpg)
Parallel Local Search
• Interest since the 90’s in using parallel machines
for Local Search
• [Aarts / Verhoeven 95] proposed two simple • [Aarts / Verhoeven 95] proposed two simple
ways to exploit parallelism in LS :
– single-walk parallelism
exploit // within a single search trajectory
– multi-walk parallelism
exploit // with different search trajectories
(= multi-start, = portfolio algorithms for SAT solving)
![Page 46: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/46.jpg)
Parallel Local Search :
Single-Walk Parallelism
Search Spaceneighborhood
current configuration
![Page 47: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/47.jpg)
Parallel Local Search :
Multi-Walk Parallelism
Search Space
initial configurations
![Page 48: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/48.jpg)
Motivation
• Local-Search methods are efficient for
Combinatorial Optimization Problems
• Can solve bigger problems with more
computing power …computing power …
e.g. multi-cores CPUs & massively parallel computers
• But can // be exploited efficiently ?
• What about massively parallel computers ?
e.g. > 10 000 processors
![Page 49: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/49.jpg)
Massively Parallel Machines
K Computer at RIKEN, Kobe
700000 cores
(number 1 at the top500 in 2011)
Jugene: IBM Blue Gene/P
at Jurlich, Germany
300000 cores
(EU PRACE machine)
![Page 50: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/50.jpg)
… at University of Tokyo
Fujitsu Oakleaf FX10
77000 cores
Hitachi HA8000
15000 cores
![Page 51: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/51.jpg)
Top500 Supercomputer
Performance Development
![Page 52: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/52.jpg)
From Supercomputer to Desktop…
Intel MIC architecture NVIDIA Tesla Kepler GPU
50
cores
2496
cores
![Page 53: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/53.jpg)
Issues with Parallelism …
• Good method/algorithm ≠ good performance
• Very different to develop a good method for
– a few cores (shared memory)– a few cores (shared memory)
– a few tens/hundreds of cores (e.g. Grid platforms)
– a few thousands of cores (e.g. supercomputers)
• The best sequential algorithm may not give the
best parallel performances
![Page 54: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/54.jpg)
Existing Parallel Implementations
• Several implementations for shared memory
multi-core architectures
– Local search, SAT, CSP
• A few implementations on PC clusters• A few implementations on PC clusters
• Now some parallel implementations for GPU– Genetic algorithms, Tabu Search, Pattern Search
• A few implementation on Grids
– e.g. Branch & Bound: Grid’BnB
![Page 55: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/55.jpg)
Parallel Implementation
of Adaptive Search
• On HA8000 machine at University of Tokyo
• Parallel Method: independent multi-walks
– start // search from different initial configurations– start // search from different initial configurations
– no communication
• C-code encapsulated using MPI
• No optimization done wrt architecture (nodes)
• Same code runs on Grid5000 platform
![Page 56: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/56.jpg)
![Page 57: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/57.jpg)
Results
![Page 58: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/58.jpg)
Influence of
Problem Size:
Magic
SquaresSquares
![Page 59: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/59.jpg)
Costas Array Problem
• Proposed by Costas in the 60’s
• Sonar / Radio applications
• Active community for 40 years
• Constructive methods exist• Constructive methods exist
– but not valid for all n
• Very combinatorial, solutions are very scarce
– n=29 has only 164 solutions (23 unique) out of 29!
• still several open problems …
– Does it exist costas arrays for n=32 ?
![Page 60: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/60.jpg)
Linear speedups for large instances
Speedups for CAP 22
![Page 61: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/61.jpg)
Costas Array, up to 8192 cores
![Page 62: LECTURE 11 & 12 CONSTRAINT -BASED LOCAL SEARCHwebia.lip6.fr/~codognet/PSAI/11-CBLS.pdf– Does it exist costas arrays for n=32 ? Linear speedups for large instances Speedups for CAP](https://reader035.fdocuments.in/reader035/viewer/2022071410/61053ae0a0b3f30a1432c63f/html5/thumbnails/62.jpg)
Current Research
• Develop new models for // Local Search
• Communication between search engines
– dependent multi-walks– dependent multi-walks
• What to communicate / when to communicate?
• Idea of a pool of (elite) solutions
– …