NOMAD, a blackbox optimization software · Parallel Versions of the MADS Algorithm for Black-Box...

Blackbox optimization MADS NOMAD References

NOMAD, a blackbox optimization software

Sebastien Le Digabel, Charles Audet, VivianeRochon-Montplaisir, Christophe Tribes

EUROPT, 2018–07–12

NOMAD: Blackbox Optimization 1/27


Presentation outline

Blackbox optimization

The MADS algorithm

The NOMAD software package

References




The MADS algorithm


References



Blackbox optimization problems

I Optimization problem:

minx∈Ω

f(x)

I Evaluations of f (the objective function) and of the functionsdefining Ω are usually the result of a computer code (ablackbox).

I n variables, m general constraints.

I Ω =x ∈ X : cj(x) ≤ 0, j ∈ 1, 2, . . . ,m

⊆ Rn.

I X : Bounds and/or nonquantifiable constraints.




The MADS algorithm


References



[0] Initializations (x0, ∆0 )[1] Iteration k

[1.1] Searchselect a finite number of mesh pointsevaluate candidates opportunistically

[1.2] Poll (if Search failed)construct poll set Pk = xk + ∆kd : d ∈ Dksort(Pk)evaluate candidates opportunistically

[2] Updatesif success

xk+1 ← success pointincrease ∆k

elsexk+1 ← xk

decrease ∆k

k ← k + 1, stop or go to [1]

From the MADS original paper [Audet and Dennis, Jr., 2006].



Poll illustration (successive fails and mesh shrink)

∆k = 1

rxk

rp1

rp2

r

p3

trial points=p1, p2, p3




∆k = 1

rxk

rp1

rp2

r

p3


∆k+1 = 1/4

rxkr

p4 rp5

r

p6

= p4, p5, p6




∆k = 1

rxk

rp1

rp2

r

p3


∆k+1 = 1/4

rxkr

p4 rp5

r

p6

= p4, p5, p6

∆k+2 = 1/16

rxk

rp7

r p8rSSp9

= p7, p8, p9



MADS extensions

I Constraints handling with the Progressive Barriertechnique [Audet and Dennis, Jr., 2009].

I Surrogates [Talgorn et al., 2015].

I Categorical variables [Abramson, 2004].

I Granular and discrete variables [Audet et al., 2018].

I Global optimization [Audet et al., 2008a].

I Parallelism [Le Digabel et al., 2010, Audet et al., 2008b].

I Multiobjective optimization [Audet et al., 2008c].

I Sensitivity analysis [Audet et al., 2012].




The MADS algorithm


References



NOMAD (Nonlinear Optimization with MADS)I C++ implementation of MADS.

I Standard C++, no other package needed.

I Parallel versions with MPI.

I Runs on Linux, Mac OS X and Windows.

I MATLAB versions; Multiple interfaces (Python, Excel, etc.)

I Command-line and library interfaces.

I Distributed under the LGPL license.

I Complete user guide available in the package.

I Doxygen documentation available online.

I Download at https://www.gerad.ca/nomad.


https://www.gerad.ca/nomad

https://www.gerad.ca/nomad


NOMAD: History and team

I Developed since 2000.

I Current version: 3.9 (June 2018).

I Algorithm designers:I M. Abramson, C. Audet, J. Dennis, S. Le Digabel, C. Tribes,

V. Rochon-Montplaisir.

I Developers:I Versions 1 and 2: G. Couture.I Version 3 (2008): S. Le Digabel, C. Tribes.I Version 4 (2019): C. Tribes, V. Rochon-Montplaisir.

I Support at [email protected].

I Related article in TOMS [Le Digabel, 2011].


mailto:[email protected]


More than 9,000 certified downloads since 2008.

0

2000

4000

6000

8000

10000

12000

2008-10-21 2010-03-05 2011-07-18 2012-11-29 2014-04-13 2015-08-26 2017-01-07 2018-05-22

Softpedia

Sourceforge

Gerad

Total

0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018



Users from Sept. 2014 to Sept. 2015

(downloads from the GERAD website only)

I Industries: Hydro-Quebec (IREQ), Rio Tinto, Bombardier,Airbus, Boeing, United Airlines, Siemens, Schneider Electric,GHD Toronto, Energen, Skyconseil, German AerospaceCenter, Moscow Power Engineering Institute, NewmericalTechnologies, Disney Research, US Federal Reserve Board,Softree, Aditazz, Essar Steel, Tatura Milk Industries,Westrock, Market Appeal, Corona Insights, The ClimateCorporation.

I National Labs: Argonne, Sandia, Los Alamos.

I Universities.



Main functionalities (1/2)

I Single or biobjective optimization.

I Variables:I Continuous, integer, binary, granular, categorical.I Periodic.I Fixed.I Groups of variables.

I Searches:I Latin-Hypercube (LH).I Variable Neighborhood Search (VNS).I Quadratic models.I sgtelib: Stand-alone library of surrogates.I User search.


https://github.com/bastientalgorn/sgtelib


Main functionalities (2/2)

I Constraints treated with 4 different methods:I Progressive Barrier (default).I Extreme Barrier.I Progressive-to-Extreme Barrier.I Filter method.

I Several dense direction types:I Coordinate directions.I LT-MADS.I OrthoMADS.I Hybrid combinations.

I Sensitivity analysis.

(all items correspond to published or submitted papers).



NOMAD installation

I Pre-compiled executables are available for Windows and Mac.

I Installation programs copy these executables.

I On Unix/Linux, after download, launch an installation script.

I Two ways to use NOMAD: batch mode or library mode.



Blackbox conception (batch mode)

I Command-line program that takes as argument a filecontaining x, and displays the values of f(x) and the cj(x)’s.

I Can be coded in any language.

I Typically: > bb.exe x.txt displays f c1 c2 (objectiveand two constraints).



Important parameters

I Necessary parameters: Blackbox characteristics (dimension n,number of constraints, etc.), starting point (x0).

I All algorithmic parameters have default values. The mostimportant are:

I Maximum number of blackbox evaluations,I Starting point (more than one can be defined),I Types of directions (more than one can be defined),I Initial mesh size,I Constraint types,I Surrogate searches,I Seeds.

I See the user guide for the description of all parameters, or usethe nomad -h option.



Run NOMAD> nomad parameters.txt



Advanced functionalities (library mode)

I No system calls: the code executes faster.

I The user can program a custom search strategy.

I The user can pre-process all evaluation points before they areevaluated.

I The user can decide the priority in which trial points areevaluated.

I The user can define callback functions that will be called atsome specific events (new success, new iteration, new MADSrun in bi-objective optimization)



Examples included in the NOMAD packageSimple examples:

I How to run NOMAD.

I Usage of parallellism.

I Categorical variables.

I MATLAB/Python interfaces.

Advanced examples to illustrate additional possibilities:I Multi-start from points generated with LH sampling.

I Problems used in library mode and coded as:I A Windows DLL.I A GAMS program.I A CUTEst problem.I A FORTRAN code.

I A GUI prototype in JAVA.



Other MADS distributions

I MATLAB version within the Opti Toolbox package.http://www.i2c2.aut.ac.nz/Wiki/OPTI.

I Available in the MATLAB Optimization Toolbox.Old version, not maintained.

I NOMADM by M. Abramson. Old version, not maintained.

I Excel with the OpenSolver tool.http://www.opensolver.org (GPLv3).


http://www.i2c2.aut.ac.nz/Wiki/OPTI/index.php/Solvers/NOMAD

http://www.i2c2.aut.ac.nz/Wiki/OPTI

http://www.mathworks.com/matlabcentral/linkexchange/links/1820-mesh-adaptive-direct-search-optimization-toolbox

http://opensolver.org/non-linear-nomad-integration/

http://www.opensolver.org


Summary

I Blackbox optimization motivated by industrial applications.

I Algorithmic features backed by mathematical convergenceanalyses and published in optimization journals.

I NOMAD: Software package implementing MADS.

I Open source; LGPL license.

I Features: Constraints, biobjective, global optimization,surrogates, several types of variables, parallelism.

I Fast support at [email protected].

I NOMAD can be customized through collaborations.


mailto:[email protected]

n


Applications

For a nice application of NOMAD, come see Frederic Messine’splenary talk on Friday, 10am.




The MADS algorithm


References



References I

Abramson, M. (2004).

Mixed variable optimization of a Load-Bearing thermal insulation system using a filter pattern searchalgorithm.Optimization and Engineering, 5(2):157–177.

Audet, C., Bechard, V., and Le Digabel, S. (2008a).

Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search.Journal of Global Optimization, 41(2):299–318.

Audet, C. and Dennis, Jr., J. (2006).

Mesh Adaptive Direct Search Algorithms for Constrained Optimization.SIAM Journal on Optimization, 17(1):188–217.

Audet, C. and Dennis, Jr., J. (2009).

A Progressive Barrier for Derivative-Free Nonlinear Programming.SIAM Journal on Optimization, 20(1):445–472.

Audet, C., Dennis, Jr., J., and Le Digabel, S. (2008b).

Parallel Space Decomposition of the Mesh Adaptive Direct Search Algorithm.SIAM Journal on Optimization, 19(3):1150–1170.

Audet, C., Dennis, Jr., J., and Le Digabel, S. (2012).

Trade-off studies in blackbox optimization.Optimization Methods and Software, 27(4–5):613–624.



References II

Audet, C., Digabel, S. L., and Tribes, C. (2018).

The mesh adaptive direct search algorithm for granular and discrete variables.Technical Report G-2018-16, Les cahiers du GERAD.

Audet, C., Savard, G., and Zghal, W. (2008c).

Multiobjective Optimization Through a Series of Single-Objective Formulations.SIAM Journal on Optimization, 19(1):188–210.

Le Digabel, S. (2011).

Algorithm 909: NOMAD: Nonlinear Optimization with the MADS algorithm.ACM Transactions on Mathematical Software, 37(4):44:1–44:15.

Le Digabel, S., Abramson, M., Audet, C., and Dennis, Jr., J. (2010).

Parallel Versions of the MADS Algorithm for Black-Box Optimization.In Optimization days, Montreal. GERAD.Slides available at https://www.gerad.ca/Sebastien.Le.Digabel/talks/2010_JOPT_25mins.pdf.

Talgorn, B., Le Digabel, S., and Kokkolaras, M. (2015).

Statistical Surrogate Formulations for Simulation-Based Design Optimization.Journal of Mechanical Design, 137(2):021405–1–021405–18.


https://www.gerad.ca/Sebastien.Le.Digabel/talks/2010_JOPT_25mins.pdf

NOMAD, a blackbox optimization software · Parallel Versions of the MADS Algorithm for Black-Box...

Documents

Transcript of NOMAD, a blackbox optimization software · Parallel Versions of the MADS Algorithm for Black-Box...