1

Surrogate Response Surface Methods for Optimization,

Parameter Estimation, and Uncertainty Quantification of

Multi-modal, Computationally Expensive Models

(With Environmental Applications)

Christine A. Shoemaker Civil and Environmental Engineering. &

Operations Research and Information Engineering.

Cornell University With results from joint work with others as noted

NSF Institute for Mathematical Applications

“Large-Scale Inverse Problems and Uncertainty Quantification” Workshop

June 5-10, 2011

(References given at end of the slides; contact cas12@cornell.edu for codes)

2

Acknowledgements

• This work has been a collaboration with others including: R. Regis, S. Wild , Y. Wang (Optimization) A.Singh, A. Espinet and P. Mugunthan (Environmental Applications)N. Bliznyuk and D. Ruppert (uncertainty quantification),

Funding by NSF from CISE, Math & Phys Science, Geoscience, and Engineering Directorates

3

Outline of Talk on Analysis for Computationally Expensive

Nonlinear Simulation Models

• Background

• Global Optimization with Surrogates

• Integration of global optimization into Uncertainty Quantification

4

Calibration Optimization

• Our goal is to find the

minimum of f(x)

where x є D.• Let Fmax be the maximum number of function

evaluations (e.g. simulations)• We want Fmax to be small because f(x) is “costly” to

evaluate

This can be a measure of error between model prediction and observations

X can be parameter values

5

Context (i.e. Is this of Interest to You?)

• We are dealing with general models for which one can make few mathematical assumptions.

• The models are NOT assumed to be linear, Gaussian, or unimodal, and we do not have bounds on amplitude of derivatives. (Almost none of our real applications have simulation models that satisfy any of these criteria.)

• The primary assumptions are that – the simulation models and decision variables are continuous – the simulation computation time (minutes to days/run) is significantly

longer than is required to compute a response surface approximation (seconds)

– the decision variables are in a closed (bounded) space.• Our methods can be applied without derivatives if they are not available.

6

Global Optimization versus Local OptimizantiMulti-Modal Problems have Multiple local minima

F(x)

X (parameter value)

Local minimum

Global minimum

7

IDEAS: Local Derivative Methods have difficulty when Objective Function can be a Rough Surface because of Numerical Simulation and Embedded Equations

Figures below give Objective Function Versus Parameter Value for a Real Watershed Model called SWAT

8

Context--Continued• As a result the lack of assumptions (on unimodality, etc.) and

the cost of the simulations, we have no evidence (yet) that we can solve problems with more than 40 decision variables.

• For problems with more variables, it is sometimes possible to do a hierarchical approach, – a subset of the variables (lower level) can justifiably be

computed from a nonlinear or linear programming solution as a function of the remaining (upper level variables).

– Then the global optimization problem has only the upper level variables. This greatly reduces the computational effort.

9

Objective Function and Simulation Model

• Let G(x) = simulation model (computationally expensive) – e.g. a PDE model of groundwater contamination

• Let Ω= the objective function, which depends on output from the simulation. – For example, Ω could be the sum of squared errors

between data and output from G(x).

• Then the function to be minimized is f(x)= Ω(G(x)).

• We build our response surface on f(x).

10

Ideas: Nonlinear Simulation-Optimization is usually a Global Optimization Problem

• If objective function is a nonlinear equation of output from a nonlinear simulation model, the resulting optimization problem is multimodal and hence needs a global optimization algorithm.

• Hence most nonlinear simulation-optimization problems require global optimization.

11

Why Do Nonlinear Simulation Models Lead to Global Optimization Problems?

2 2( ) , ( ) ( )i ii

G x x y SSE y D

2 2( ) ( ( )) ( )i ii

f x G x SSE x D

Assume simulation model G(x) is nonlinear, and objective function (e.g. an error function) Ω (y) is nonlinear. The simplest nonlinear form is that both of them are quadratic

Then the function to be minimized is a fourth order polynomial

A fourth order polynomial can be multimodal and hence it can have multiple local minima. In this case, it requires a global optimization method to find the best solution. This is the simplest form of nonlinearity, and in multiple dimensions with more nonlinear terms, we can expect many local minima .

12

Global Optimization versus Local OptimizantiMulti-Modal Problems have Multiple local minima

F(x)

X (parameter value)

Local minimum

Global minimum

13

Response Surface Approximation for Surrogate

• Different types of splines can be used for creating the response surface.

• We use Radial Basis Functions (RBF) plus a linear polynomial for our spline because it works well with scattered data.

• Some other applications have also used a kriging surface.

14

IDEAS: Response Surface for Objective Function

• It has been relatively common for years to create an approximation A-G(x) of the simulation model of G(x) (often called a surrogate model or a low fidelity model).– Often this is done either by making the mesh for the PDE

model coarser or by using something like a neural net approximation of G(x).

– Then an optimization method is applied to A-G(x), so it is feasible to do many evaluations of the cheap model A-G(x).

• I believe it is much better to make the response surface of the objective function f(x)=Ω(G(x)) since f’s sensitivity to changes in x can be different from A-G(x)’s sensitivities.

15

Experimental Design with Symmetric Latin Hypercube (SLHD)

• To fit the first Surrogate Approximation we need to have evaluated the function at several points.

• We use a symmetric Latin Hypercube (SLHD) to pick these initial points.

16

x (parameter value-one dimensional example)

Objective

Function

f(x)

measure of error

One Dimensional Example of Experimental Design to Obtain Initial Surrogate Approximation

Costly Function Evaluation (e.g. over .5 hour CPU time for one evaluation).

17

x (parameters)

f(x)

Surrogate Approximation with Initial Points from Experimental Design

In real applications x is multidimensional since there are many parameters (e.g. 10).

18

x (parameters)

f(x)

Surrogate Approximation with Initial Points from Experimental Design

Two goals in picking next point: small value of surrogate and away from previously evaluated points. (We select based on weighted sum of these two factors.)

19

x (parameter value)

f(x)

Update in Surrogate Approximation with New Evaluation

Update done in each iteration for Surrogate Approximation for each algorithm expert.

Surrogate Approximation is a guess of the function value of f(x) for all x.

new

Graph by Stefan Wild 20

Example: Cubic RBF Surrogate

8 3

1 2 1 3 221

,k j jj

m x s s y v v s v s

cubic

Graph by Stefan Wild, ANL-DOE 21

8 2

1 2 1 3 2,2

1

1k j jj

m x s s y v v s v s

Example: Multiquadric RBF Surrogate

Multiquadric

22

Why Use Surrogate Approximation Methods?

• A Surrogate Approximation R(x) can be used as part of an efficient parallel optimization algorithm in order to reduce the number of points at which we evaluate f(x), and thereby significantly reduce computational cost.

• Our Surrogate Approximation algorithm searches for the global minimum.

23

Idea: Use previous Simulations

• One of the advantages of this approach is that you are making use of all previous simulation results (since they are incorporated in the response surface). A great deal of time has been invested in these previous simulations.

• Contrast this with derivative-based optimization, which is based entirely on the derivative at the current search point and does not use any of the values from previous function evaluations

24

Outline of Stochastic RBF (LMSRBF) Method1

• 1. Use a “space filling” experimental design (e.g. SLHD) to select a limited number of evaluation points.

• 2. Make an approximation of the function (e.g. Radial Basis Functions) based on experimental design points over the entire domain.

• 3. Use our algorithm to select the next function evaluation point from a set of random candidate points (Note: selection could be based on Surrogate Approximation and location of previously evaluated points; and can use different probability distributions)

• 4. Construct a new Surrogate Approximation that incorporates the newly evaluated point and all prior points.

• 5. Stop if reach maximum number of iterations. Otherwise go to Step 3

1 Reference: (Regis and Shoemaker, INFORMS JOC, 2007)

25Regis and Shoemaker, Informs Jn. of Computing, 2007

Stochastic RBF Algorithm Converges Almost Surely

26

So What Are the Components of a Surrogate Optimization Algorithm after initialization?

1. Type of surrogate (e.g. polynomial, kriging, radial basis function (RBF-cubic, Gaussian, etc.)

2. Method of search on surrogate model (math programming, heuristic, random search)

3. Objective function used in the search (lowest surrogate surface value, maximize expected uncertainty, or weighted or constrained objective that considers both the surrogate model value and other criteria to promote global search.

27

Applications for Optimization

• The following applications focus on the use of Stochastic RBF published in 2007 in INFORMS Jn of Computing and a modification for parallel computation published in INFORMS Jn of Computing in 2009.

• There are three subsurface applications discussed including bioremediation, groundwater remediation and carbon sequestration.

• The method is general and can be used on other applications (e.g. the watershed application discussed later in the talk).

28

Application to Groundwater Bioremediation at Cape Canaveral Field Site

• This example requires solution of a highly nonlinear systems of partial differential equations and is a truly “computationally expensive function” requiring up to 2.5 hours for one evaluation.

• I show here application to a “hypothetical” example that has the same equations as the 2.5 hour simulation model, but a smaller grid since it was applied to many other algorithms

• The results for the full 2.5 hour model are given in the paper by Mugunthan, Shoemaker and Regis, Water Resource Research, 2005.

29

Why Does Simulation Take So Long?

• Partial differential equation has 22 species at each node in finite difference model and a total of 1448 time steps.

• There are in 4800 nodes 3 dimensions.• The total number of unknowns over all time steps in

this nonlinear system is about 153 million.• The biological reactions are very nonlinear (biggest

factor)• Many groundwater problems are larger than this.

30

Algorithms Used for Comparison of Optimization Performance on Calibration

• Stochastic Greedy Algorithm– Neighborhood defined to make search global– Neighbors generated from triangular distribution around current solution.

Moves only to a better solution.• Evolutionary Algorithms

– Derandomized evolution strategy DES with lambda = 10 and b1 = 1/n and b2 = 1/n0.5 (Ostermeier et al. 1992)

– Binary or Real Genetic algorithm GA, population size 10, one point cross-over, mutation probability 0.1, crossover probability 1

• RBF Surrogate Approximation Algorithms– RBF Gutmann- radial basis function approach, with cycle length five, SLH

space filling designGlobal Stochastic RBF-Cornell radial basis function approach

• FMINCON– derivative based optimizer in Matlab with numerical derivatives

• 10 trials of 100 function evaluations were performed for heuristic and Surrogate Approximation algorithms for comparison

31

Comparison of Algorithm Performance on Hypothetical Aquifer – CNS

Experimental Design for RBF algorithms gets over at 28

Lowest curve is best—note log scale

32

Mean and Standard deviation of best solution produced after 100 function evaluations – Hypothetical Example

•Algorithm with a lowest mean and lowest standard deviation is desirable

•Based on 10 trials

-1

0

1

2

3

4

5

6

7

DESRGA

BiGA GS

FA-Gutm

ann

FA-RS

FMINCON

-[A

ve

rag

e N

S ±

S.D

.]

ours

33

Conclusions on Bioremediation Example

• Our Surrogate Approximation algorithm generally outperformed the alternative algorithms considered.

• This performance was based on a limited number of function evaluations.

• The Surrogate Approximation algorithm was robust in that it had very few bad results of 10 trials.

34

Types of Global Optimization Methods (all of which are used in later comparisons)

• Heuristics (genetic algorithms, simulated annealing, particle swarm, etc.)

• Response surface methods (that utilize mathematical analysis associated with approximation of continuous functions by other continuous functions)

• Coupling a local optimization method with a “multi-start method” that does another local search with a new starting point each time the local search finds a local minimum.

35

Stochastic RBF (our algorithm) Results On Many Global Optimization Test Problems (2007)

• We solved 17 global optimization test problems and one small groundwater problem.

• All methods repeated in 30 trials and values shown are the means over these 30 trials.

• (Local) Stochastic RBF is the best among the eight algorithms considered on higher dimensional problems and is as good or better on lower dimensional test problems.

(Regis and Shoemaker, INFORMS Jn of Computing, 2007)

36

14 Dimensional Schoen 14.100 Global Test Function

Ours

(LMSRBF)

From INFORMS JOC, Regis and Shoemaker, 2007

37

Conclusions on our Surrogate Approximation Optimization with RBF from INFORMS JOC 2007 paper

• Our local metric stochastic RBF outperformed other algorithms for dimensions (6 to 14) when looking at the suite of global optimization test problems with limited numbers of simulations. We did not consider dimensions over 14

• Contrary to popular belief, using multistart with an excellent local optimizer is not the most effective method for global optimization

• If the number of simulations are not limited, other algorithms than a surrogate like Stochastic RBF might find a more accurate solution.

38

Current Research: Groundwater Remediation Site (Umatilla) History

• 19,728-acre military reservation in Oregon• Established in 1941• From 1950’s until 1965, depot operated an onsite

explosives washout plant• Wash water disposed in two unlined lagoons• Estimated 85 million gallons of water was discharged

during the 15 year stretch• Two contaminants

– RDX (Hexahydro-1,3,5-trinitro-1,3,5-triazine)– TNT(2,4,6-Trinitrotoluene)

Results (Umatilla) : This plot shows that Stochastic RBF is most efficient of the tested algorithms (lowest curve)

(From copyrighted PhD thesis of my student Amandeep Singh, 2011)

Convergence Plot (note log scale)

39

Conclusions for comparison of Algorithms on Umatilla Groundwater Remediation Problem

• Response Surface Algorithm (Stochastic RBF) was more efficient than heuristic and non-linear optimization methods for an expensive remediation model.

• Response Surface Algorithm produced good solutions with lowest mean and least spread on Umatilla.

• Response Surface based methods were robust to the two different functions Umatilla and Blaine.

• Recommend the use of Response Surface based methods as an optimization tool for computationally expensive groundwater models.

40

41

CO2 Plume Estimation by Automatic Calibration of TOUGH2 Models for Carbon

Sequestration in Geological Formations(current research)

Antoine Espinet & Christine ShoemakerCornell University

Christine DoughtyDOE Berkeley Lab

42

Current Research: CO2 Plume Estimation by Automatic Calibration of TOUGH2 Models for Carbon Sequestration in Geological

Formations

• Carbon sequestration: storage of super critical carbon dioxide in geological formations

43

Example Based on Frio Field Site

• We generated multiple realizations that were modeled on a DOE Field Site for Carbon Sequestration.

• The parameter to be estimated was permeability.

44

From PHD thesis in progress for my student Antoine Espinet

45

Optimization to Estimate CO2 Plume by Estimating Spatial Distribution of Permeabilities.

• Data locations are sparse because of expense of drilling wells over 1000 m deep.

• Insufficient data to estimate all 60 permeabilities• Reduction of parameters to 7 parameters gave good

results.• Fortunately knowing precise permeabilities for high

permeability regions was not important.

46

Approximated Permeabilities with 7 ParametersFrom PHD thesis in progress (2011) for my student Antoine Espinet

47

Results for Carbon Sequestration Problem(from PhD thesis in progress by my student Antoine Espinet)

1. Current forecast (based on pressure data up to 1.5 years from two wells [one inside plume and one outside plume plus injection well] has R2 =.88.2. Future forecast (to 7 years into the future with no data beyond 1.5 years) has R2 = .85

These are excellent results for deep subsurface estimates.

Results are only slightly better when CO2 saturation is also measured.

7-parameter, t=7.5 yr, (R2=0.85), layer 4

48

Based on simulation with true parameters

Based on simulation with parameters estimated with limited data

Based on simulation with parameters generated randomly

49

Comparison of Optimization Algorithms for Geological Carbon Sequestration

• One simulation takes up to 3 hours• 7 parameters, the permeabilities• Heterogeneous, 3D geological model• Averaged over 3 trials• 120 evaluations• Multiple Local minima so requires global

optimization

50

Algorithm comparison: notice the log scale on the Y-axisLow curves are best so local algorithms did poorly

The derivative based local algorithms Our surrogate algorithms

Number of simulations

Obj

ectiv

e Fu

nctio

n

51

Conclusions on Carbon Sequestration

• Managing Risks for Carbon Sequestration requires solving a large scale nonlinear inverse problem in a reasonable amount of time, which we could do with our Stochastic RBF surrogate method and with our global optimization trust region RBF method GORBIT (Wild , PhD thesis, 2009), which uses our local RBF algorithm Orbit (Wild et SIAM Sci Comp. 2007 & SIAM Jn of Optim., in press)

• The surrogate global optimization methods work much better than the local optimizers, presumably because the problem has multiple local minima.

• Future work: we are applying our uncertainty method SOARS to this GCS problem currently.

New Method Global Optimization Method

• My postdoc Yilun Wang and I are now developing a new method that continues to use the RBF surrogate approximation but which changes the objective function and the way candidate points are selected.

• This method changes the procedure for randomly selecting perturbation size, and incorporates derivatives (cheap to obtain) on the RBF surface to generate candidate points.

• The results are very promising for higher dimension (up to 40 dimensional problems tested).

5252

53

Keane Function (30 Dimensional)

Our earlier Stochastic RBF method

Our new RBF method in blue(lowest curve) is best (from Y. Wang and C. Shoemaker, manuscript in draft, 2011)

Conclusions

• Stochastic RBF is a very effective global optimization method.

• Further improvements appear to be possible, especially for higher dimensions.

5454

55

Future Work on Global Optimization

• I just got positive notice about an NSF CISE award to develop an asynchronous parallel global optimization algorithm that incorporates some of the improvements above and some newer ideas to give faster results for high dimensional problems.1

• 1 We published an earlier paper on a synchronous parallel algorithm based on an extension of Stochastic RBF (Regis and Shoemaker, Informs Jn. of Computing, 2009)

56

Next Idea: Optimization + Response Surface in Uncertainty Quantification for

Computationally Expensive Models

• Typically uncertainty analysis (e.g. Monte Carlo Sampling or MCMC) requires a huge number of samples (e.g. simulations) in the domain. This makes them infeasible for computationally expensive simulation models.

• I thought the use of optimization in the sampling process combined with response surfaces can greatly reduce the number of simulations and give accurate results.

57

SOARS: An Uncertainty Analysis Combining Statistical and Optimization Analysis and Response Surfaces

for Computationally Expensive Models

National Science Foundation Project(Jointly Funded by Mathematical Science (Statistics) and

Geoscience Programs—now expired)

C. Shoemaker and D. Ruppert- PIs

Students and Postdocs: N. Blizniouk, R. Rommel, S. Wild, D. Cowen. P. Mugunthan, Yilun Wang, Amy Li

(This presentation covers 2 papers from this project)

58

MCMC Markov Chain Monte Carlos

• MCMC is a Bayesian and statistically rigorous method for estimating uncertainty.

• Our SOARS method reduced computational effort by a factor of 65 (e.g. 150 simulations rather than 10,000 simulations for conventional MCMC) and

• SOARS is also a statistically rigorous method without requiring any linear or unimodal assumptions.

59

Paper 1 on SOARS

• We have several papers we have on SOARS (one published, one in review , and two in draft).

• I will first discuss a statistics journal paper:

• Blizniouk, N., D. Ruppert, C.A. Shoemaker, R. G. Regis, S. Wild, P. Mugunthan, “Bayesian Calibration of Computationally Expensive Models Using Optimization and Radial Basis Surrogate Approximation.” Journal of Computational and Graphical Statistics, July 2008.

60

Our Objective Function

• The objective is the likelihood function, which we want to maximize.

• The likelihood function is based on generalized least squares and includes the basic model parameters as well as parameters for transformations to convert non normal random variables into normal.

• We want our method to be able to address multi modal functions.

61

Role of Optimization

• We want to approximate the objective function (e.g. the likelihood function) in parameter space.

• We use a derivative free optimization search to find the global maxima (MLE) and important local maxima.

• We do additional evaluations around local maxima to get a more accurate description of the function.

• We build an RBF Surrogate Approximation of the objective function based on this information

62

Find Maximum Likelihood and Other Modes in Likelihood Functions

(picture is one dimensional example)

Objective

(Likelihood)

High probability region

Low probability region

Parameter value

63

Overview of Methodology

There are three steps:

1.Use optimization to search for the maximum likelihood and construct Surrogate Approximation (with radial basis functions) of the likelihood function with the simulation results from this search. If likelihood is possibly multimodal then use surrogate global optimization method.

2. Locate the region(s) of high posterior density and do more simulations to get an improved (RBF) Surrogate Approximation of the likelihood function using both optimization and extra simulations.

3. Do MCMC and standard Bayesian analysis using the approximate

posterior density-likelihood based on RBF. (This is the major step to reduce computational effort.)

64

Find Maximum Likelihood and Other Modes in Likelihood Functions

(picture is one dimensional example)

Objective

(Likelihood)

High probability region

Low probability region

Parameter value

65

SOARS Computational Effort

• Based on the “costly” function evaluations done in the optimization search and additional function evaluations around the local optimizers, we build a Surrogate Approximation of the likelihood function. (100’s of expensive simulations.)

• We then apply MCMC (Markov Chain Monte Carlo) to the Surrogate Approximation of the likelihood function – This generates the joint posterior density function) of

the parameter values (requires 10,000+ evaluations of the Inexpensive approximation function).

66

•There is a chemical spill of mass M into a long narrow channel of water at both locations marked in red.

•The system is described by advection diffusion equation.

•We want to estimate the mass M, the time t and location L of the second spill, and the diffusion coefficent D, which are the model parameters.

•The output from the model we want is average pollutant concentration over time at the end of the channel.

Paper 1 Application to Chemical Spill Problem

67

The equations describing the concentration C as a function of location s and time t and the parameters M,D,L, and tau are:

Because this equation can be solved analytically, we can afford to do the 10,000 or more model evaluations necessary for conventional MCMC and compare it to our SOARS method which requires far fewer model evaluations. General goal is for non-analytical models.

68

Joint Posterior Density

• Y0 - vector of observed data;

• η – parameter vector in the (joint) statistical model (the final solution is called β)

• [list1|list2] - conditional density of random variables in list1 given list2, e.g.

is the conditional density of η given the data Y0.

The term η is a vector of parameter values,

[ | ]OY

69

Kernel estimates of the marginal posterior densities by a) (solid line) exact joint posterior obtained from con- ventional MCMC Analysis with 10,000 function evaluations & b) (dashed lines) with our Surrogate Approximation method with 150 function evaluations. One graph for each parameter.

70

Significance of Results

• To obtain the pdf by MCMC normally requires 10,000 or more”costly” function evaluations.

• These results indicated we were able to get good results with much less (by almost two orders of magnitude) computational effort by– Doing some “costly” function evaluations (a few

hundreds) and then fitting a Surrogate Approximation to the likelihood function and

– Evaluating 10,000 points on the inexpensive Surrogate Approximation surface with the MCMC

71

SOARS Paper 3

• A third paper is not yet submitted:

Shoemaker, C.A., D. Cowen, N. Bliznyuk, D. Ruppert, J. Woodbury, X. Lin, “Application of SOARS for Uncertainty Analysis of Computationally Expensive Hydrologic Models with Application to Cannonsville Basin”

72

How Do We Protect This Water From Pollution?

(New York City water supply)

Problem is that phosphorous from surrounding Watershed can result in need for an

$ 8 Billion Water Plant for NYC!

73

SWAT2000 Model Discretization of Cannonsville Watershed (1200 km2)

27

38

29

1

6

43

4

11

19

3

9

26

728

41

10

2

8

18

305

16

32

40

24

33

37

2012

13

23

21

34

2214

39

31

15

17

363542

25

43 subbasins758 HRUsAvg HRU = 1.6 km2

Using a spatially distributed model helps us evaluate management options.

Reservoir Lake

Watershed land

74

Rough Surface of Numerical SimulationError Function Versus Parameter Value

Parameter 1 Parameter 14

ERRO

R

Roughness and multiple local minima exist making solution difficult.

75

Computed by SOARSMarginal posterior Densities of the parameters (βi) for

[ | ]OY

These are the marginal posterior densities for two of the parameters in the analysis of the SWAT model for the Cannonsville Watershed. The analysis considered a larger number of parameters.

It is important to observe the asymmetry in the densities. It is obvious this is not a normally distributed density .

76

Uncertainty in Unmeasured Output during the Calibration Period

• Preceding slide shows posterior joint densities of the parameters for the Cannonsville example, (which depend on data Yo during the calibration).

• The next slide shows quantiles of model output that is not directly measured (such as evapotranspiration or groundwater recharge) as shown in the following slide.

• This is only possible with a “process” model and is not possible with a polynomial regression model or neural net or genetic programming model for rainfall runoff.

[ | ]OY

77

Quantiles, Means, Standard Dev.

Output (mm) Mean S. Dev.5% 25% 75% 95%

Surface Runoff 54.6 55.7 56.8 59.9 56.9 1.6Lateral Flow 341.9 349.4 354.8 364.8 354.2 7.2Groundwater Flow 275.0 280.3 284.8 295.5 284.8 6.3ET 434.9 436.4 437.5 441.3 437.8 2.0Revap 11.8 11.9 11.9 12.0 11.9 0.1Yield 690.5 693.0 694.4 697.4 694.2 2.1

Quantiles

These are based on statistically rigorous analysis with transformations to account for non normal data obtained with a small fraction of the number of simulations required by other methods including MCMC and GLUE

78

What Have We Achieved?

• Applied modern statistical tools to calibration of environmental engineering models, including transformations.

• Implemented a Bayesian method of uncertainty analysis

• Substantially reduced the number of simulations (by one or two orders of magnitude) .

• This reduction makes the method feasible for computationally expensive models for which no other existing method is computationaly feasible.

79

Overall Conclusions

• Surrogate Response Surfaces in combination with global optimization and uncertainty quantification algorithms can enable us to do analysis on computationally expensive simulation models that would not otherwise be possible.

80

Some References Regis, R.G., C.A. Shoemaker, “A Stochastic Radial Basis Function Method for the Global

Optimization of Expensive Functions”, INFORMS Journal of Computing, 19: 497-509, 2007.

Wild, Stefan, Regis, R.G.., Shoemaker, C.A., “ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions,” SIAM Journal on Scientific Computing 30(6) 3197-3219, 2007

Bliznyuk, N., D. Ruppert, C.A. Shoemaker, R. G. Regis, S. Wild, P. Mugunthan, “Bayesian

Calibration of Computationally Expensive Models Using Optimization and Radial Basis Function Approximation.” Journal of Computational and Graphical Statistics, 17 (2), 270-295, 2008

Regis, R.G., C.A. Shoemaker, “Parallel Stochastic Global Optimization Using Radial Basis

Functions,” INFORMS Jn. of Computing 21(3), 411-426, 2009

Wild, S.M. and C.A. Shoemaker, Global Convergence of Radial Basis Functions Trust Region Derivative-Free Algorithms, SIAM Jn. Optimization (in press).

81

Additional References

Mugunthan, P., C.A. Shoemaker, R. G. Regis "Comparison of Function Approximation, Heuristic and Derivative-based Methods for Automatic Calibration of Computationally Expensive Groundwater Bioremediation Models," Water Resources Research Vol. 41, W11427

Regis, R.G. and Shoemaker, C.A. Parallel radial basis function methods for the global

optimization of expensive functions. European Journal of Operational Research 182(2), 514-535 doi: 10.1016/j.ejor.2006.08.040, 2007

Regis, R.G. and Shoemaker, C.A. Improved strategies for radial basis function methods for global

optimization. Journal of Global Optimization, Vol. 37, No. 1, pp. 113-135, doi : 10.1007/s10898-006-9040-1, 2007.

82

End

• Contact Christine Shoemaker cas12@cornell.edu for information on open access for available codes.