Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration

under contract DE-AC04-94AL85000.

Optimization and Model InsightResearch Directions at

Sandia National Laboratories

Scott A. Mitchell

INFORMS Chicago ChapterCUSTOM

Managing Risk in an Uncertain World

Intro

• Take home messages

– Why we’re doing what we’re doing, not much of the how• Pose the questions, not all the answers

– Sandia environment• Breadth of SNL mission• Unique applications

– Physics variety

– Extreme computations and simulations

– Risks to manage, uncertainties to assess

– Our response• Current R&D activities• New research directions

• ( My “not-to-do” list

– Policy and political issues )

Acknowledgements

• Optimization and Uncertainty Estimation dept. – http://www.cs.sandia.gov/departments/9211/index.htm

• Thanks to Urmila for invitation• Thanks to department staff…

– Tim Trucano– Tony Giunta– Mike Eldred– Bart van Bloemen Waanders– Roscoe Bartlett

• … and others throughout Sandia, from whose viewgraphs I have borrowed liberally – Marty Pilch– John Aidun

– Garth Reese & Kendall Pierson

Sandia Mission Breadth

• http://www.sandia.gov/about/vision/index.html

• My division:– Nuclear Weapons: ensuring the stockpile is safe, secure,

reliable, and can support the United State’s deterrence policy.

• Optimization (Opt) and Uncertainty Quantification (UQ) are key to 2 out of 3!

Risk

Unc

erta

in W

orld

Types of SNL problems

• Every year, Labs’ director signs off on stockpile– Says testing not needed (so far)

• Standards< 1 in 10^6 accidents result in “any” nuclear reaction

• Conservatism in design– Almost no performance data

• Very few controlled tests. Systems rarely actually used or in accidents (that’s good, but makes data scarce!)

– Tests designed to demonstrate performance, not to test limits of failure. Rarely designed for parameter studies. (Aim for mid-points of parameter ranges, not extremes.)

• Contrast to automotive industry– Imagine building the Ford Expedition fleet and meeting

individual and aggregate specs, if last car fleet actually driven on roads to failure were 10 model T’s.

Pulsed Power & Inertial Confinement Fusion (ICF)

• ICF is a goal at Sandia National Labs

• Pulsed Power Technique using Z-machine

•Wire arrays explode,creating a plasma sheath, which implodes and stagnates.

•X-rays hit capsule, generating fusion

•Capsule performance very sensitive to variations!

To fulfill our National Security mission, we develop systems and components designed to perform

Sandia: The Extreme Engineering Lab

extreme applications, under adverse conditions.

The duty cycle can include damage, crush and failure;

melting; decomposition – or just plain old age.

Extreme sports athlete Dave Mirra

Example Applications I

Extremely Tough:• Earth penetrator bombs• Radiation-hard

microelectronics• Explosive Destruction

System

Extremely Sensitive:• Sensors• Explosives Sniffer

Extremely Powerful:• Ferroelectric ceramic power

supply – explosively driven• Pulsed Power Z-Machine

HE charge

Electrical Load

Extreme Insult Resistant:• Aluminum Honeycomb

energy absorber• Engineered Stress Profile

Glass • Architectural Surety

Extreme Effectiveness:• Environmental remediation• Decontamination Foam

Example Applications II

Aluminum Honeycomb

Competitive Cs and Sr sorption on clay mineral

Example Applications III

Extremely Small & Efficient:• Microsystems & Nanotechnology• Solid State Lighting• Next Generation NG

Extremely Reliable:• Electrical Connectors• Bonds

– Solder Joints– Adhesives Joints– Brazes and Welds

World’s smallest linear accelerator

StructuralDynamics

Incompressible Fluid flow

Heat transfer

Shock Physics

Compressiblefluid flow

Geophysics

Solid mechanics

Fire

Electrical

Design Tools:optimizationsensitivity

uncertainty analysis

Motivation: Target Simulation Codes

Spin the wheel:our example du jour

Large Structural Dynamics Sandia Calculations

• Example Large Sandia Calculations– Gordon Bell Prize won by ‘Salinas’ Structural

Dynamics Code team in “special” category at SuperComputing 2002

• Sandia has had many wins in the past• Only non Earth Simulator winner in 2002• Sandia Director of Engineering Sciences

Tom Bickel: first time a “production code” won– C++– Math and solver libraries– Systems of equations like

• Production eigen solver based on ARPACK, Ax=Bx• Linear statics, dynamics, Ax=b• Nonlinear statics, dynamics, A1x1=b1, A2x2=b2,… Anxn=bn

Northrop Grumman Newport News Full Ship Model Description

•Large-scale detailed model–2 million equations–Shell & beam elements– Interior structure

•60 processors (Baby Q)•20 Modes/1.5 hours

Sway

Blast

Increasing Levels of Model Complexity

Explosion in computer hardware and software technologies allows higher levels of structural dynamics modeling sophistication.

15 years ago:Shellshock 2.5D

200 equations, Pre-Cray

Recent Past:NASTRAN

Electronics package30,000 equations,

Cray, Vector Supercomputer

Electronics package model 2 yrs ago

400,000 equations,Parallel

Supercomputer

Increasing Levels of Model Complexity

Now, 10+ million equation (modeling at circuit board level)

Electronics package model 1 yr ago: 800,000 equations

Cubit: Sandia Meshing Tool

Gordon Bell SC2002: 0.5M equations, 18 minutes on 128 processors of ASCI Red

Sandia Capabilities

• Sandia considers itself (needing to be an) expert in– Advanced Manufacturing– Biosciences– Chemical and Earth Sciences– Computer Information Sciences– Electronics– Engineering– Homeland Security– Materials and Process– Microsystems– Nanotechnology– Pulsed Power

• Modeling and Simulation of the above!• My dept’s role: tools for design optimization,

reliability assessment, of the above in uncertain environments.

Computational Focus

• Limitations of testing(I’m talking about non-nuclear engineering testing here… Even so, for some things, we wouldn’t test even if we could…)

– Too many possible designs / scenarios• 20d param space

– Not going to do 2^20 tests, but could explore via sampling, SAND (later)

– Could you instrument an experiment adequately?

– Worst case scenarios only identifiable by computational tools

• paradigm shift from “engineering intuition”

Limits of Computation

• In the past, • Calibrate computational models

to observed experiments

– Codes used for interpolating between tested points

– “Verification” criteria is approx. curve fit up to “view graph norm” after calibration

– Underlying physics unimportant, any function that curve-fits is ok

• In the future, want • Predictive capability

– Codes used for extrapolating to unknown designs / environments

– “Verification” criteria is error bars in computed answer matching error bars in experimental answer

– Confidence in underlying physics, “validation” that important phenomena are modeled

Key Challenges of this Approach

• How much credibility is sufficient?– Variabilities and uncertainties must be

acknowledged, and their impact in the decision context quantified

• Historical context– Reactor safety, WIPP, Yucca Mountain

– Sandia was leader in developing, applying, and defending methodologies

– Decade of peer review at the highest levels

Sandia vs. Textbook Problems

• A few details…

Functional Form

• Textbook… – Functional form known

• Differentiable, analytic derivatives, or reliable finite differences

– Fast to calculate• ¼ second on a single-

processor workstation

• Sandia functions…– Function is FEM solution

over large grid• May be transient

simulation involving multi-physics

• Simulation may crash at some parameter values

– Worse than discontinuous, non-existent value!

– noisy response may make finite differences meaningless

– Slow to calculate• Several days on 1000s of

processors on one of the world’s top 10 MPP supercomputers

Sandia Functions

• Functional form unknown• Except: if function is FEM solution, then sometimes

functional form may be implicitly given by the PDE• Foreshadow Sensitivities and SAND methods

• Local analytic derivates obtainable by hard work

Incompressiblefluid-flow codePremoEuler,

Discretized Euler

Note: huge graph of these eq.s, not just single scalar

Sandia vs. Textbook Functions

Typical Textbook

Optimization Problem

CTH Shock Physics Application

• inexpensive to evaluate• gradients exist and are

accurate

1.0

0.3

0.41.0

x1

f(x)

x2

1.0

0.0

• very expensive to evaluate• gradients do not always

exist and may be inaccurate

Sandia vs. Textbook Functions

Typical Textbook

Optimization Problem

ALEGRA ICF Capsule Implosion Study

• Single minimum

• Variability near optimum not important

• Multiple minima• Variability near optimum is

important• Range of sensitivity to variables

-3.0E+07

-2.0E+07

-1.0E+07

0.0E+00

Impl

osio

nV

eloc

ity(c

m/s

)

0.001

0.002

0.003

Fuel Density (g/cm 3) 0.10

0.120.14

Capsule Radius (cm)

DtoA - Function Evaluation

• It’s worse than that– The prior slides assumed that it was possible to run

a simulation at an arbitrary design point

Expanded Design Optimization View

Geometry creation & cond.

Mesh generation

Model preparation & mgmt.

Mesh decomposition

Parametric model changes

Design

Design to Analysis Simulation to DesignSimulation

Results filtering & decomp.

Parallel output

Visualization

Parametric optimization

Geometric optimization

Design evaluation

Equation assembly

Solution

Error estimation

Mesh refinement

Load rebalancing

“Analysis”

D2A S2DEmbedded

SET = D2A + S2D + some embedded libraries

Many steps currently require human intervention or interpretation, either initially (bother) or in the loop (oh dear!)

D2A (DtoA)

• Problem set-up a key issue– Set it up once

• Usually tedious

– Re-set it up / close the optimization loop • Can be very difficult, beyond current technology• Varying material properties fairly easy

– Just a continuous variable in an input file

• Simulation interfacing ok, a development and design,not conceptual, challenge

Problem set-up a key issue

• Varying geometry is difficult

– Occasional shape-optimization, well controlled changes, ok

– Unstructured hexahedral (brick) mesh dependence, hard• Brittle hex meshing structure, 2.5D algorithms

– Several people-months for initial “meshing strategy” script

– Many constraints flow all the way forward in the process

• Geometry movement may cause discrete event in decomposed model, even if not in initial model!

symm plane

position

“foam”“steel case”

“impact”

Material Bound Cond Init Cond

Problem set-up a key issue: Varying Geometry Fidelity Difficult

• Designer: parametric design changes retain “idealized model” characteristics

Decompositions

Add detail

Parametric change

Analysis BC’s

• Analyst: parameterize DSM->ASM so that ASM can be updated automatically after parametric DSM changes

Design Solid Model (DSM) Analysis Solid Model (ASM) Data

Idealized Model

Design

er chan

ges

AnalystResponse

Other ASM data:•Detail suppression•Mesh scheme•Mesh size

“block_screw.asy”

“block”

“screw”

“block_screw.asy”

“block”

“screw”

“he”“primer”

“block_screw.asy”

“block”

“screw”

“he”“primer”

My Dept’s Response

•Our response to these challenges– Programmatic / Organizational

– Technical

Mission (from Center web pages)

• Traditionally– Opt and UQ over normal-sized weapon components. Focus on:

• Complex systems, full system response multiphysics: electrical, radiation, thermal, fluid flow and shock, radiation, magneto-hydrodynamics

– Generation of and reaction to these physics

• Reliability in “normal environments”– Vibration and shock of launch, re-entry– Electronic and mechanical component

performance during impact

• Safety in “abnormal environments”– Weapon in a fire, pool effects

• Expanding to Nano, MEMS, bio (not combinatoric DNA problems, but modeling chemistry at sub-cellular level)

• Also ties to Emerging Threats– Discrete and agent-based

logistics and battlefield system simulations

Scope – dept’s local focus

Dept. Activity Map

ApplicationsApplications

Technology/Methodology

Technology/Methodology

ResearchResearch

Optimization & Algorithms- Multi-fidelity- Surrogates- Trust regions- Sensitivity Analysis- SAND- Multi-disciplinary- Parallel strategies

DAKOTA

Optimization- Sandia Eng Sci- LLNL A-Div- Homeland security inversion & control- Premo, Xyce- SNL Designers

DAKOTA/UQMOOCHOTSFCore

Validation Methods & Assessments

UQ (Uncertainty Quantification) & OUU (Opt. Under Uncertainty)- Sandia Eng Sci- Sandia Pulsed Power/RES/ICF

Validation Applications- Sandia Eng Sci- ALEGRA- MPSalsa reaction-diffusion eq.- Multi-scale modeling- Nano systems

Uncertainty & Algorithms- Sampling- Inference- Characterization- Calibration- Probability, Stats Analytic Reliability

Validation Science- Credibility Quantification- Optimal decisions (OR)- QMU- “Predictability”

Push/Pull

Push/Pull

Push/Pull

Push/Pull

Technical Themes

Technical themes to meet these challenges1. Validation & Verification

• To trust the simulations

2. Sensitivity-based (intrusive) methods• Expensive simulations and/or large design spaces• Seven levels of intrusion• Large scale problem motivation• Inversion and homeland security projects

3. DAKOTA• Noisy and/or expensive simulations• Levels of parallelization• Strategies and methods• SBO, OUU -> SBOUU

1. Validation & Verification Theme

• Hamming – “The purpose of computing is insight…” (?)

• ASCI – the purpose of computing is to provide “high-performance, full-system, high-fidelity-physics predictive codes to support weapon assessments, renewal process analyses, accident analyses, and certification.” (DOE/DP-99-000010592)

• Philip Holmes – “…a huge simulation of the ‘exact’ equations…may be no more enlightening than the experiments that led to those equations…Solving the equations leads to a deeper understanding of the model itself. Solving is not the same as simulating.” (SIAM News, June, 2002)

Useful quotes to keep in mind.

ASCI Program

• ASCI Applications developing suite of simulation codes– Milestones like “simulate phenomena X”– Major supporter of DAKOTA / our dept’s activities

• ASCI Validation & Verification program– To verify those codes, and validate the model

applicationso we have confidence in our answer

– Uncertainty Quantification seen as a key technology for validating codes, by overlapping computational and experimental error bars.

– DAKOTA is the tri-labs delivery vehicle for UQ technology

• Validation definition– Is the computational model an accurate

representation of reality (the reality I care about)?• Depends on phenomena of interest• Depends on decision you need to make

• Verification definition– Given the computational model, does the code

produce the right answer?• Depends on SQE, accurate solvers, allowing code to

run to convergence, etc.

Validation and Verification

• Hint: Validation is a “physics problem.”

• Hint: Verification is a “math problem.”

Consider the following “validation” exercise:

1

2

3

5

6

4

Incident Angle

p

r/

pin

c

35 40 45 5030

Experiment + Error Bar

Analytic

ALEGRA Calculation

This is physics.

This is math.

• So, fundamentally, what does this comparison mean?• Please note that the calculation is not converged.• Stringently, “verification” for numerical PDE codes

basically means:

Demonstrate convergence to the correct answer.

• …if not for the “code” at least for the particular calculation(s).

• Since it is unlikely that we will establish convergence and since we don’t know what the correct answer is this is quite a problem.

There is at least one essential problem with the previous comparison – there are no numerical error bars.

• Uncertainty in verification arises from:– (Recall complexity of simulations = functions we hope to

optimize over)– Software implementation errors – BUGS

• Code crashes are the least of our problems.• Mutually reinforcing errors are also “easily” detectable.• Mutually canceling errors are of greater concern.

– Inadequate algorithms• No amount of resolution will solve the problem.

– Inadequate resolution• Resolution “solves” the problem but is probably

unavailable.

• The issue of “verifying” ASCI Level 1 milestones is becoming prominent.

“Uncertainty In Verification”

Is Probabilistic Software Reliability (PSR) useful for computational science software?

• We are test fixated in building software, properly so:

“Based on the software developer and user surveys, the national annual costs of an inadequate infrastructure for software testing is estimated to range from $22.2 to $59.5 billion.” (“The Economic Impacts of Inadequate Infrastructure for Software Testing,” NIST report, 2002.)

• If we can’t test software perfectly, then testing alone does not solve the verification problem.

A view of software “reliability” is decreasing # of “failures” and increasing # of “users” and they are correlated.

A notional “reliability” diagram for a PDE code thus looks something like the following:

Development and test Capability I Capability II Capability etc

Fai

lure

Rat

e

Application DecisionsApplication Decisions

# of Users

1st use / Validation

(WH

AT

IS

A F

AIL

UR

E?

)

• I don’t know what the error is with certainty.

• I still need to apply the calculation.

• The alternative is analysis paralysis (in particular, NUMERICAL analysis paralysis).

– “Global warming” is an example; keep it in mind.

The bottom line in the previous example can be generalized: numerical accuracy is an important uncertainty.

Qualitative: “I’m uncertain what the accuracy of this calculation is.”

Qualitative: “I’m uncertain what the accuracy of this calculation is.”

Quantitative leap: “I need to apply probabilistic language to describe my understanding of the

accuracy of this calculation”

Quantitative leap: “I need to apply probabilistic language to describe my understanding of the

accuracy of this calculation”

Are Probabilistic Error Models (PEM) useful for computational science software?

• Suppose that we can neither “verify codes” nor “verify calculations.”

– “When quantifying uncertainty, one cannot make errors small and then neglect them, as is the goal of classical numerical analysis; rather we must of necessity study and model these errors.”

– “…most simulations of key problems will continue to be under resolved, and consequently useful models of solution errors must be applicable in such circumstances.”

– “…an uncertain input parameter will lead not only to an uncertain solution but to an uncertain solution error as well.”

• These quotes reflect a new view of “numerical error” expressed in B. DeVolder, J. Glimm, et al. (2001), “Uncertainty Quantification for Multiscale Simulations,” Los Alamos National Laboratory, LAUR-01-4022.

– “All models are wrong-but some models are useful,” statistician George P. E. Box.

2. Sensitivities Theme

Sensitivities Project

• Sensitivities (Derivatives, Jacobians, Hessians, etc.) can dramatically speed up optimization over large PDE-based codes

• NAND• SAND approach names

– PDE-Constrained Optimization, orSimultaneous Analysis and Design – SAND, orall-at-once-approach

Large Scale PDE Constrained Optimization

Optimizer

PDE simulationInput Output

PDE simulationInput Outputoptimizer

BlackBox

SANDMOOCHO

Idea: add PDE equations as constraints to optimization problem

PDE equations as constraints

• PDE-constrained optimization formulation

u = design vars, nu moderatey = PDE FEM state vars, ny huge

1E-13

1E-12

1E-11

1E-10

1E-09

1E-08

1E-07

1E-06

1E-05

0.0001

0.001

0.01

0.1

1

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

CPU (seconds)

||c

(y)|

| /

||c

(y0

)|| Explicit

Implicit : no prec, MF mat-vec

Implicit: FD prec, MF mat-vec

Impliict: AD prec, MF mat-vec

Implicit: AD prec, AD mat-vec

Sensitivities

• Solution methods

– PDE’s are constraints

– NAND: Nested Analysis and Design• Eliminate or solve PDE equations, • Then optimize keeping them eliminated / solved• Bonus: sensitivities speed up NAND

– SAND: Simultaneous Analysis and Design• Start infeasible• Then solve as you converge• Alternative gradient formulations, SAND / NAND

– Alternative sensitivity formulations - Next slidesPremo solution time using sensitivities

Gradients

• NAND Solve

– By eliminating / solving c, get

• Use gradient based optimization

T

NAND Solve ----------

• Grad based opt, 3 choices, keep c solvedLevel-1: Finite differences

– Exact gradients, solve either

Level-2

Level-3 or

Pre-compute adjoint sensitivity matrix, independent of u

compute direct sensitivity matrix,depends on u, so compute product each step

How to get sensitivity matrices? Paper and pencil? AD?

SAND Solve -----------

• Truly solve simultaneously

– Level-4, use direct sensitivities

– Level-5, use adjoint sensitivities• Transpose of Jacobians

– Level-6, assemble & solve full KKT system (or QP subproblem)• Second derivatives

• Get the report! bartv@sandia.gov

Moretypes ofsensitivities

Numerical Results6 inlet CVD problem

Black Box: 22 hrs

SAND/rSQP++ : 1.56 hrs

* PIII 500 Mhz, Linux Redhat 6.1 MPICH

Solution: Smoothed norm of deviation = 2.547 % @ velocity = 13.989 cm/s

Problem Size: solving for velocities (x,y,z), temperature, pressure, 3 species for a total of 31,992 unknowns

Payoff: run-time approximately independent of number of design variables (PDE solution dominates, done about once).

3. DAKOTA Theme

Surrogate Based Optimization

4 Levels of Parallel ComputationExample: MINLP

• Sophisticated parallel efficiency

1. Strategy: concurrent exploration of design space– Parallel branching cases due to MINLP branching

2. Different simulation code runs– Objective or constraints require different physics codes, e.g.

constrain vibration and temperature

3. Multiple simulation runs at nearby values– For finite differences or pattern search

4. Parallel simulation codes– E.g. Salinas, PRONTO inherent parallelism

Background: Surrogate-Based Optimization

• To meet Sandia’s challenge: surrogate based optimization– algorithms that perform optimization on a low fidelity “surrogate model” with

periodic corrections from a high fidelity model.• Two motivations

– Low fidelity model can be computationally inexpensive– Low fidelity model can be more smooth

• Surrogate model types:– Multifidelity simulation models having different mesh densities and/or physical

accuracy. (Inexpensive)– Multidimensional surface fitting methods that smooth out “noise” in the

simulation data. (Inexpensive & more smooth)• polynomial regression, spline interpolation, radial basis functions, etc.

• Ad hoc SBO algorithms have been used by engineers for decades, but these often failed without explanation.

• Provably-convergent and heuristic SBO algorithms in DAKOTA.– Provably convergent to a local minimum when gradients are available

(journal article in review)

– Insight into methods for proving convergence when gradients are unavailable(collaboration with Prof. Luis Vicente – Univ. Coimbra, Portugal)

– Recent 2nd order correction results for TR-SBOUU

1.0

0.3

0.41.0

x1

f(x)

x2

1.0

0.0

Surface Fitting Functions as Surrogate Models

• Inexpensive, more smooth

• Many smoothing function types:– polynomial regression– radial basis functions– kriging interpolation– neural networks

• Usually employ data sampling or statistical design of experiments methods.

• Practical limit of O(101-102) independent variables.– sampling becomes inefficient for

high dimensional problems– but, many engineering design

problems are low dimensional

Area smoothing

function is applied.

From SBO to SBOUU

From SBO to SBOUU:• SBO is provably convergent with trust region

globalization– 1st order consistency (e.g., beta correction)– verification of approx. steps

• Extensions to SBOUU– 1st order consistency, assuming a worthwhile

stoch. gradient– verification of stats. in relative sense:

nonoverlapping confidence intervals (rigorous or stoch. approx.)

Sequence of trust regions

Optimization under Uncertainty with Surrogates

Opt

UQ

Sim

{d} {Su}

{u} {Ru}

Opt

UQ

Sim

{d} {Su}

{u} {Ru}

Data Fit

{d} {Su}

Opt

UQ

{d} {Su}Data Fit

{d} {Su}

Sim

{u} {Ru}

Data Fit/Hier

{d} {Ru}{u}

Opt

UQ

Sim

{d} {Su}

{u} {Ru}

Data Fit/Hier

{d} {Ru}{u}

Formulations 2 & 4 amenable to trust-region approachesGoals: maintain quality of results, provable convergence (for a selected confidence level)

DakotaModel

Single Layered Nested

Data Fit Hierarchical

Nested model: internal iterators/models execute a complete iterative study as part of every evaluation.

Layered model: internal iterators/models used for periodic update and verification of data fit (global/local/multipoint) or hierarchical (variable fidelity) surrogates.

Nestings/Layerings can recurse

Formulation 1: Nested

Formulation 2: Layered containing Nested

Formulation 3: Nested containing Layered

Formulation 4: Layered containing Nested containing Layered

TR-SBOUU Results• Direct nested OUU is expensive and requires seed reuse

• SBOUU expense much lower (up to 300x), but unreliable.

• TR-SBOUU maintains quality of results and reduces expense

– Form 4 = double fit, Form 2 = single fit– Ex. 1: formulation 4 with TR 2-3x less expensive than direct nesting

– Ex. 2: formulation 4 with TR 8-10x less expensive than direct nesting

– ICF Ex.: formulation 2 with TR located solution vicinity in a single cycle

• Additional benefits:– Navigation of nonsmooth engineering problems– Less sensitive to seed reuse: variable patterns OK and often helpful

– Less sensitive to starting point: data fit SBO provides some global ident.

• One part of a larger OUU picture.

L = 100”

w

tX

YMinimize f + pfail_r1 + pfail_r3

Subject to gi 0, for i = 1,2,3

r2 + 3r2 1.6e5

Conference papers (3): AIAA MA&O, SIAM CS&E, USNCCMJournal (1): Eldred, M.S., Giunta, A.A., Wojtkiewicz, S.F., Jr., and Trucano, T.G., "Formulations for Surrogate-Based Optimization Under Uncertainty," (in preparation) SIAM Journal on Scientific Computing, special issue on UQ.

OUU/SBOUU Problem Formulation

• Technical Challenge: Given a fixed computing budget, is it possible to identify minima in OUU problems? (It is possible w/ unlimited budget.)

Improvement is easy to discern when confidence intervals do not overlap, e.g., “At the 95% confidence level, F(x2) is an improvement over F(x1).”

F(x, u)

x

x1 x2

OUU/SBOUU Problem Formulation

• Technical Challenge: Given a fixed computing budget, is it possible to identify minima in OUU problems? (It is possible w/ unlimited budget.)

F(x, u)

x

x3x2

Improvement is harder to discern when confidence intervals overlap, e.g., “At the 95% confidence level, F(x3) is not significantly better than F(x2).”

New Themes

New Research Areas

• SBOUU (ongoing)• Computational Credibility,

quantitative answers for V&V– How can a given credibility measure be optimized given

constrained resources? – How much validation is sufficient? – How can credibility best be characterized for

extrapolation beyond the validation domain? – How are credibility measures best applied for decisions

with uncertainty?

• Computational Predictability– QMU (now three NNSA milestones!)

• Calibration Under Uncertainty

Technical Challenges

• Technical challenges– What Opt or UQ can be done if only one or two

calculations can fit in Red Storm?• Surrogates for expensive, noisy simulations

• SAND for large design spaces, expensive simulations

– Error bars for calculations • Epistemic uncertainty

• These are Cultural challenges, too!

Take-Home Messages

• Showed– Breadth of SNL activities– Unique simulations at SNL– New research directions we're pursuing

• Hope you say “quite an interesting world at SNL, in terms of applications and techniques needed to address”

• Anything sparked your interest? – Always interested in collaboration, visitors

• CSRI program

• samitch@sandia.gov • http://www.cs.sandia.gov/departments/9211/index.htm

The End

Questions?