Post on 24-Jan-2016
description
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Optimization and Model InsightResearch Directions at
Sandia National Laboratories
Scott A. Mitchell
INFORMS Chicago ChapterCUSTOM
Managing Risk in an Uncertain World
Intro
• Take home messages
– Why we’re doing what we’re doing, not much of the how• Pose the questions, not all the answers
– Sandia environment• Breadth of SNL mission• Unique applications
– Physics variety
– Extreme computations and simulations
– Risks to manage, uncertainties to assess
– Our response• Current R&D activities• New research directions
• ( My “not-to-do” list
– Policy and political issues )
Acknowledgements
• Optimization and Uncertainty Estimation dept. – http://www.cs.sandia.gov/departments/9211/index.htm
• Thanks to Urmila for invitation• Thanks to department staff…
– Tim Trucano– Tony Giunta– Mike Eldred– Bart van Bloemen Waanders– Roscoe Bartlett
• … and others throughout Sandia, from whose viewgraphs I have borrowed liberally – Marty Pilch– John Aidun
– Garth Reese & Kendall Pierson
Sandia Mission Breadth
• http://www.sandia.gov/about/vision/index.html
• My division:– Nuclear Weapons: ensuring the stockpile is safe, secure,
reliable, and can support the United State’s deterrence policy.
• Optimization (Opt) and Uncertainty Quantification (UQ) are key to 2 out of 3!
Risk
Unc
erta
in W
orld
Types of SNL problems
• Every year, Labs’ director signs off on stockpile– Says testing not needed (so far)
• Standards< 1 in 10^6 accidents result in “any” nuclear reaction
• Conservatism in design– Almost no performance data
• Very few controlled tests. Systems rarely actually used or in accidents (that’s good, but makes data scarce!)
– Tests designed to demonstrate performance, not to test limits of failure. Rarely designed for parameter studies. (Aim for mid-points of parameter ranges, not extremes.)
• Contrast to automotive industry– Imagine building the Ford Expedition fleet and meeting
individual and aggregate specs, if last car fleet actually driven on roads to failure were 10 model T’s.
Pulsed Power & Inertial Confinement Fusion (ICF)
• ICF is a goal at Sandia National Labs
• Pulsed Power Technique using Z-machine
•Wire arrays explode,creating a plasma sheath, which implodes and stagnates.
•X-rays hit capsule, generating fusion
•Capsule performance very sensitive to variations!
To fulfill our National Security mission, we develop systems and components designed to perform
Sandia: The Extreme Engineering Lab
extreme applications, under adverse conditions.
The duty cycle can include damage, crush and failure;
melting; decomposition – or just plain old age.
Extreme sports athlete Dave Mirra
Example Applications I
Extremely Tough:• Earth penetrator bombs• Radiation-hard
microelectronics• Explosive Destruction
System
Extremely Sensitive:• Sensors• Explosives Sniffer
Extremely Powerful:• Ferroelectric ceramic power
supply – explosively driven• Pulsed Power Z-Machine
HE charge
Electrical Load
Extreme Insult Resistant:• Aluminum Honeycomb
energy absorber• Engineered Stress Profile
Glass • Architectural Surety
Extreme Effectiveness:• Environmental remediation• Decontamination Foam
Example Applications II
Aluminum Honeycomb
Competitive Cs and Sr sorption on clay mineral
Example Applications III
Extremely Small & Efficient:• Microsystems & Nanotechnology• Solid State Lighting• Next Generation NG
Extremely Reliable:• Electrical Connectors• Bonds
– Solder Joints– Adhesives Joints– Brazes and Welds
World’s smallest linear accelerator
StructuralDynamics
Incompressible Fluid flow
Heat transfer
Shock Physics
Compressiblefluid flow
Geophysics
Solid mechanics
Fire
Electrical
Design Tools:optimizationsensitivity
uncertainty analysis
Motivation: Target Simulation Codes
Spin the wheel:our example du jour
Large Structural Dynamics Sandia Calculations
• Example Large Sandia Calculations– Gordon Bell Prize won by ‘Salinas’ Structural
Dynamics Code team in “special” category at SuperComputing 2002
• Sandia has had many wins in the past• Only non Earth Simulator winner in 2002• Sandia Director of Engineering Sciences
Tom Bickel: first time a “production code” won– C++– Math and solver libraries– Systems of equations like
• Production eigen solver based on ARPACK, Ax=Bx• Linear statics, dynamics, Ax=b• Nonlinear statics, dynamics, A1x1=b1, A2x2=b2,… Anxn=bn
Northrop Grumman Newport News Full Ship Model Description
•Large-scale detailed model–2 million equations–Shell & beam elements– Interior structure
•60 processors (Baby Q)•20 Modes/1.5 hours
Sway
Blast
Increasing Levels of Model Complexity
Explosion in computer hardware and software technologies allows higher levels of structural dynamics modeling sophistication.
15 years ago:Shellshock 2.5D
200 equations, Pre-Cray
Recent Past:NASTRAN
Electronics package30,000 equations,
Cray, Vector Supercomputer
Electronics package model 2 yrs ago
400,000 equations,Parallel
Supercomputer
Increasing Levels of Model Complexity
Now, 10+ million equation (modeling at circuit board level)
Electronics package model 1 yr ago: 800,000 equations
Cubit: Sandia Meshing Tool
Gordon Bell SC2002: 0.5M equations, 18 minutes on 128 processors of ASCI Red
Sandia Capabilities
• Sandia considers itself (needing to be an) expert in– Advanced Manufacturing– Biosciences– Chemical and Earth Sciences– Computer Information Sciences– Electronics– Engineering– Homeland Security– Materials and Process– Microsystems– Nanotechnology– Pulsed Power
• Modeling and Simulation of the above!• My dept’s role: tools for design optimization,
reliability assessment, of the above in uncertain environments.
Computational Focus
• Limitations of testing(I’m talking about non-nuclear engineering testing here… Even so, for some things, we wouldn’t test even if we could…)
– Too many possible designs / scenarios• 20d param space
– Not going to do 2^20 tests, but could explore via sampling, SAND (later)
– Could you instrument an experiment adequately?
– Worst case scenarios only identifiable by computational tools
• paradigm shift from “engineering intuition”
Limits of Computation
• In the past, • Calibrate computational models
to observed experiments
– Codes used for interpolating between tested points
– “Verification” criteria is approx. curve fit up to “view graph norm” after calibration
– Underlying physics unimportant, any function that curve-fits is ok
• In the future, want • Predictive capability
– Codes used for extrapolating to unknown designs / environments
– “Verification” criteria is error bars in computed answer matching error bars in experimental answer
– Confidence in underlying physics, “validation” that important phenomena are modeled
Key Challenges of this Approach
• How much credibility is sufficient?– Variabilities and uncertainties must be
acknowledged, and their impact in the decision context quantified
• Historical context– Reactor safety, WIPP, Yucca Mountain
– Sandia was leader in developing, applying, and defending methodologies
– Decade of peer review at the highest levels
Sandia vs. Textbook Problems
• A few details…
Functional Form
• Textbook… – Functional form known
• Differentiable, analytic derivatives, or reliable finite differences
– Fast to calculate• ¼ second on a single-
processor workstation
• Sandia functions…– Function is FEM solution
over large grid• May be transient
simulation involving multi-physics
• Simulation may crash at some parameter values
– Worse than discontinuous, non-existent value!
– noisy response may make finite differences meaningless
– Slow to calculate• Several days on 1000s of
processors on one of the world’s top 10 MPP supercomputers
Sandia Functions
• Functional form unknown• Except: if function is FEM solution, then sometimes
functional form may be implicitly given by the PDE• Foreshadow Sensitivities and SAND methods
• Local analytic derivates obtainable by hard work
Incompressiblefluid-flow codePremoEuler,
Discretized Euler
Note: huge graph of these eq.s, not just single scalar
Sandia vs. Textbook Functions
Typical Textbook
Optimization Problem
CTH Shock Physics Application
• inexpensive to evaluate• gradients exist and are
accurate
1.0
0.3
0.41.0
x1
f(x)
x2
1.0
0.0
• very expensive to evaluate• gradients do not always
exist and may be inaccurate
Sandia vs. Textbook Functions
Typical Textbook
Optimization Problem
ALEGRA ICF Capsule Implosion Study
• Single minimum
• Variability near optimum not important
• Multiple minima• Variability near optimum is
important• Range of sensitivity to variables
-3.0E+07
-2.0E+07
-1.0E+07
0.0E+00
Impl
osio
nV
eloc
ity(c
m/s
)
0.001
0.002
0.003
Fuel Density (g/cm 3) 0.10
0.120.14
Capsule Radius (cm)
DtoA - Function Evaluation
• It’s worse than that– The prior slides assumed that it was possible to run
a simulation at an arbitrary design point
Expanded Design Optimization View
Geometry creation & cond.
Mesh generation
Model preparation & mgmt.
Mesh decomposition
Parametric model changes
Design
Design to Analysis Simulation to DesignSimulation
Results filtering & decomp.
Parallel output
Visualization
Parametric optimization
Geometric optimization
Design evaluation
Equation assembly
Solution
Error estimation
Mesh refinement
Load rebalancing
“Analysis”
D2A S2DEmbedded
SET = D2A + S2D + some embedded libraries
Many steps currently require human intervention or interpretation, either initially (bother) or in the loop (oh dear!)
D2A (DtoA)
• Problem set-up a key issue– Set it up once
• Usually tedious
– Re-set it up / close the optimization loop • Can be very difficult, beyond current technology• Varying material properties fairly easy
– Just a continuous variable in an input file
• Simulation interfacing ok, a development and design,not conceptual, challenge
Problem set-up a key issue
• Varying geometry is difficult
– Occasional shape-optimization, well controlled changes, ok
– Unstructured hexahedral (brick) mesh dependence, hard• Brittle hex meshing structure, 2.5D algorithms
– Several people-months for initial “meshing strategy” script
– Many constraints flow all the way forward in the process
• Geometry movement may cause discrete event in decomposed model, even if not in initial model!
symm plane
position
“foam”“steel case”
“impact”
Material Bound Cond Init Cond
Problem set-up a key issue: Varying Geometry Fidelity Difficult
• Designer: parametric design changes retain “idealized model” characteristics
Decompositions
Add detail
Parametric change
Analysis BC’s
• Analyst: parameterize DSM->ASM so that ASM can be updated automatically after parametric DSM changes
Design Solid Model (DSM) Analysis Solid Model (ASM) Data
Idealized Model
Design
er chan
ges
AnalystResponse
Other ASM data:•Detail suppression•Mesh scheme•Mesh size
“block_screw.asy”
“block”
“screw”
“block_screw.asy”
“block”
“screw”
“he”“primer”
“block_screw.asy”
“block”
“screw”
“he”“primer”
My Dept’s Response
•Our response to these challenges– Programmatic / Organizational
– Technical
Mission (from Center web pages)
• Traditionally– Opt and UQ over normal-sized weapon components. Focus on:
• Complex systems, full system response multiphysics: electrical, radiation, thermal, fluid flow and shock, radiation, magneto-hydrodynamics
– Generation of and reaction to these physics
• Reliability in “normal environments”– Vibration and shock of launch, re-entry– Electronic and mechanical component
performance during impact
• Safety in “abnormal environments”– Weapon in a fire, pool effects
• Expanding to Nano, MEMS, bio (not combinatoric DNA problems, but modeling chemistry at sub-cellular level)
• Also ties to Emerging Threats– Discrete and agent-based
logistics and battlefield system simulations
Scope – dept’s local focus
Dept. Activity Map
ApplicationsApplications
Technology/Methodology
Technology/Methodology
ResearchResearch
Optimization & Algorithms- Multi-fidelity- Surrogates- Trust regions- Sensitivity Analysis- SAND- Multi-disciplinary- Parallel strategies
DAKOTA
Optimization- Sandia Eng Sci- LLNL A-Div- Homeland security inversion & control- Premo, Xyce- SNL Designers
DAKOTA/UQMOOCHOTSFCore
Validation Methods & Assessments
UQ (Uncertainty Quantification) & OUU (Opt. Under Uncertainty)- Sandia Eng Sci- Sandia Pulsed Power/RES/ICF
Validation Applications- Sandia Eng Sci- ALEGRA- MPSalsa reaction-diffusion eq.- Multi-scale modeling- Nano systems
Uncertainty & Algorithms- Sampling- Inference- Characterization- Calibration- Probability, Stats Analytic Reliability
Validation Science- Credibility Quantification- Optimal decisions (OR)- QMU- “Predictability”
Push/Pull
Push/Pull
Push/Pull
Push/Pull
Technical Themes
Technical themes to meet these challenges1. Validation & Verification
• To trust the simulations
2. Sensitivity-based (intrusive) methods• Expensive simulations and/or large design spaces• Seven levels of intrusion• Large scale problem motivation• Inversion and homeland security projects
3. DAKOTA• Noisy and/or expensive simulations• Levels of parallelization• Strategies and methods• SBO, OUU -> SBOUU
1. Validation & Verification Theme
• Hamming – “The purpose of computing is insight…” (?)
• ASCI – the purpose of computing is to provide “high-performance, full-system, high-fidelity-physics predictive codes to support weapon assessments, renewal process analyses, accident analyses, and certification.” (DOE/DP-99-000010592)
• Philip Holmes – “…a huge simulation of the ‘exact’ equations…may be no more enlightening than the experiments that led to those equations…Solving the equations leads to a deeper understanding of the model itself. Solving is not the same as simulating.” (SIAM News, June, 2002)
Useful quotes to keep in mind.
ASCI Program
• ASCI Applications developing suite of simulation codes– Milestones like “simulate phenomena X”– Major supporter of DAKOTA / our dept’s activities
• ASCI Validation & Verification program– To verify those codes, and validate the model
applicationso we have confidence in our answer
– Uncertainty Quantification seen as a key technology for validating codes, by overlapping computational and experimental error bars.
– DAKOTA is the tri-labs delivery vehicle for UQ technology
• Validation definition– Is the computational model an accurate
representation of reality (the reality I care about)?• Depends on phenomena of interest• Depends on decision you need to make
• Verification definition– Given the computational model, does the code
produce the right answer?• Depends on SQE, accurate solvers, allowing code to
run to convergence, etc.
Validation and Verification
• Hint: Validation is a “physics problem.”
• Hint: Verification is a “math problem.”
Consider the following “validation” exercise:
1
2
3
5
6
4
Incident Angle
p
r/
pin
c
35 40 45 5030
Experiment + Error Bar
Analytic
ALEGRA Calculation
This is physics.
This is math.
• So, fundamentally, what does this comparison mean?• Please note that the calculation is not converged.• Stringently, “verification” for numerical PDE codes
basically means:
Demonstrate convergence to the correct answer.
• …if not for the “code” at least for the particular calculation(s).
• Since it is unlikely that we will establish convergence and since we don’t know what the correct answer is this is quite a problem.
There is at least one essential problem with the previous comparison – there are no numerical error bars.
• Uncertainty in verification arises from:– (Recall complexity of simulations = functions we hope to
optimize over)– Software implementation errors – BUGS
• Code crashes are the least of our problems.• Mutually reinforcing errors are also “easily” detectable.• Mutually canceling errors are of greater concern.
– Inadequate algorithms• No amount of resolution will solve the problem.
– Inadequate resolution• Resolution “solves” the problem but is probably
unavailable.
• The issue of “verifying” ASCI Level 1 milestones is becoming prominent.
“Uncertainty In Verification”
Is Probabilistic Software Reliability (PSR) useful for computational science software?
• We are test fixated in building software, properly so:
“Based on the software developer and user surveys, the national annual costs of an inadequate infrastructure for software testing is estimated to range from $22.2 to $59.5 billion.” (“The Economic Impacts of Inadequate Infrastructure for Software Testing,” NIST report, 2002.)
• If we can’t test software perfectly, then testing alone does not solve the verification problem.
A view of software “reliability” is decreasing # of “failures” and increasing # of “users” and they are correlated.
A notional “reliability” diagram for a PDE code thus looks something like the following:
Development and test Capability I Capability II Capability etc
Fai
lure
Rat
e
Application DecisionsApplication Decisions
# of Users
1st use / Validation
(WH
AT
IS
A F
AIL
UR
E?
)
• I don’t know what the error is with certainty.
• I still need to apply the calculation.
• The alternative is analysis paralysis (in particular, NUMERICAL analysis paralysis).
– “Global warming” is an example; keep it in mind.
The bottom line in the previous example can be generalized: numerical accuracy is an important uncertainty.
Qualitative: “I’m uncertain what the accuracy of this calculation is.”
Qualitative: “I’m uncertain what the accuracy of this calculation is.”
Quantitative leap: “I need to apply probabilistic language to describe my understanding of the
accuracy of this calculation”
Quantitative leap: “I need to apply probabilistic language to describe my understanding of the
accuracy of this calculation”
Are Probabilistic Error Models (PEM) useful for computational science software?
• Suppose that we can neither “verify codes” nor “verify calculations.”
– “When quantifying uncertainty, one cannot make errors small and then neglect them, as is the goal of classical numerical analysis; rather we must of necessity study and model these errors.”
– “…most simulations of key problems will continue to be under resolved, and consequently useful models of solution errors must be applicable in such circumstances.”
– “…an uncertain input parameter will lead not only to an uncertain solution but to an uncertain solution error as well.”
• These quotes reflect a new view of “numerical error” expressed in B. DeVolder, J. Glimm, et al. (2001), “Uncertainty Quantification for Multiscale Simulations,” Los Alamos National Laboratory, LAUR-01-4022.
– “All models are wrong-but some models are useful,” statistician George P. E. Box.
2. Sensitivities Theme
Sensitivities Project
• Sensitivities (Derivatives, Jacobians, Hessians, etc.) can dramatically speed up optimization over large PDE-based codes
• NAND• SAND approach names
– PDE-Constrained Optimization, orSimultaneous Analysis and Design – SAND, orall-at-once-approach
Large Scale PDE Constrained Optimization
Optimizer
PDE simulationInput Output
PDE simulationInput Outputoptimizer
BlackBox
SANDMOOCHO
Idea: add PDE equations as constraints to optimization problem
PDE equations as constraints
• PDE-constrained optimization formulation
u = design vars, nu moderatey = PDE FEM state vars, ny huge
1E-13
1E-12
1E-11
1E-10
1E-09
1E-08
1E-07
1E-06
1E-05
0.0001
0.001
0.01
0.1
1
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
CPU (seconds)
||c
(y)|
| /
||c
(y0
)|| Explicit
Implicit : no prec, MF mat-vec
Implicit: FD prec, MF mat-vec
Impliict: AD prec, MF mat-vec
Implicit: AD prec, AD mat-vec
Sensitivities
• Solution methods
– PDE’s are constraints
– NAND: Nested Analysis and Design• Eliminate or solve PDE equations, • Then optimize keeping them eliminated / solved• Bonus: sensitivities speed up NAND
– SAND: Simultaneous Analysis and Design• Start infeasible• Then solve as you converge• Alternative gradient formulations, SAND / NAND
– Alternative sensitivity formulations - Next slidesPremo solution time using sensitivities
Gradients
• NAND Solve
– By eliminating / solving c, get
• Use gradient based optimization
T
NAND Solve ----------
• Grad based opt, 3 choices, keep c solvedLevel-1: Finite differences
– Exact gradients, solve either
Level-2
Level-3 or
Pre-compute adjoint sensitivity matrix, independent of u
compute direct sensitivity matrix,depends on u, so compute product each step
How to get sensitivity matrices? Paper and pencil? AD?
SAND Solve -----------
• Truly solve simultaneously
– Level-4, use direct sensitivities
– Level-5, use adjoint sensitivities• Transpose of Jacobians
– Level-6, assemble & solve full KKT system (or QP subproblem)• Second derivatives
• Get the report! bartv@sandia.gov
Moretypes ofsensitivities
Numerical Results6 inlet CVD problem
Black Box: 22 hrs
SAND/rSQP++ : 1.56 hrs
* PIII 500 Mhz, Linux Redhat 6.1 MPICH
Solution: Smoothed norm of deviation = 2.547 % @ velocity = 13.989 cm/s
Problem Size: solving for velocities (x,y,z), temperature, pressure, 3 species for a total of 31,992 unknowns
Payoff: run-time approximately independent of number of design variables (PDE solution dominates, done about once).
3. DAKOTA Theme
Surrogate Based Optimization
4 Levels of Parallel ComputationExample: MINLP
• Sophisticated parallel efficiency
1. Strategy: concurrent exploration of design space– Parallel branching cases due to MINLP branching
2. Different simulation code runs– Objective or constraints require different physics codes, e.g.
constrain vibration and temperature
3. Multiple simulation runs at nearby values– For finite differences or pattern search
4. Parallel simulation codes– E.g. Salinas, PRONTO inherent parallelism
Background: Surrogate-Based Optimization
• To meet Sandia’s challenge: surrogate based optimization– algorithms that perform optimization on a low fidelity “surrogate model” with
periodic corrections from a high fidelity model.• Two motivations
– Low fidelity model can be computationally inexpensive– Low fidelity model can be more smooth
• Surrogate model types:– Multifidelity simulation models having different mesh densities and/or physical
accuracy. (Inexpensive)– Multidimensional surface fitting methods that smooth out “noise” in the
simulation data. (Inexpensive & more smooth)• polynomial regression, spline interpolation, radial basis functions, etc.
• Ad hoc SBO algorithms have been used by engineers for decades, but these often failed without explanation.
• Provably-convergent and heuristic SBO algorithms in DAKOTA.– Provably convergent to a local minimum when gradients are available
(journal article in review)
– Insight into methods for proving convergence when gradients are unavailable(collaboration with Prof. Luis Vicente – Univ. Coimbra, Portugal)
– Recent 2nd order correction results for TR-SBOUU
1.0
0.3
0.41.0
x1
f(x)
x2
1.0
0.0
Surface Fitting Functions as Surrogate Models
• Inexpensive, more smooth
• Many smoothing function types:– polynomial regression– radial basis functions– kriging interpolation– neural networks
• Usually employ data sampling or statistical design of experiments methods.
• Practical limit of O(101-102) independent variables.– sampling becomes inefficient for
high dimensional problems– but, many engineering design
problems are low dimensional
Area smoothing
function is applied.
From SBO to SBOUU
From SBO to SBOUU:• SBO is provably convergent with trust region
globalization– 1st order consistency (e.g., beta correction)– verification of approx. steps
• Extensions to SBOUU– 1st order consistency, assuming a worthwhile
stoch. gradient– verification of stats. in relative sense:
nonoverlapping confidence intervals (rigorous or stoch. approx.)
Sequence of trust regions
Optimization under Uncertainty with Surrogates
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Data Fit
{d} {Su}
Opt
UQ
{d} {Su}Data Fit
{d} {Su}
Sim
{u} {Ru}
Data Fit/Hier
{d} {Ru}{u}
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Data Fit/Hier
{d} {Ru}{u}
Formulations 2 & 4 amenable to trust-region approachesGoals: maintain quality of results, provable convergence (for a selected confidence level)
DakotaModel
Single Layered Nested
Data Fit Hierarchical
Nested model: internal iterators/models execute a complete iterative study as part of every evaluation.
Layered model: internal iterators/models used for periodic update and verification of data fit (global/local/multipoint) or hierarchical (variable fidelity) surrogates.
Nestings/Layerings can recurse
Formulation 1: Nested
Formulation 2: Layered containing Nested
Formulation 3: Nested containing Layered
Formulation 4: Layered containing Nested containing Layered
TR-SBOUU Results• Direct nested OUU is expensive and requires seed reuse
• SBOUU expense much lower (up to 300x), but unreliable.
• TR-SBOUU maintains quality of results and reduces expense
– Form 4 = double fit, Form 2 = single fit– Ex. 1: formulation 4 with TR 2-3x less expensive than direct nesting
– Ex. 2: formulation 4 with TR 8-10x less expensive than direct nesting
– ICF Ex.: formulation 2 with TR located solution vicinity in a single cycle
• Additional benefits:– Navigation of nonsmooth engineering problems– Less sensitive to seed reuse: variable patterns OK and often helpful
– Less sensitive to starting point: data fit SBO provides some global ident.
• One part of a larger OUU picture.
L = 100”
w
tX
YMinimize f + pfail_r1 + pfail_r3
Subject to gi 0, for i = 1,2,3
r2 + 3r2 1.6e5
Conference papers (3): AIAA MA&O, SIAM CS&E, USNCCMJournal (1): Eldred, M.S., Giunta, A.A., Wojtkiewicz, S.F., Jr., and Trucano, T.G., "Formulations for Surrogate-Based Optimization Under Uncertainty," (in preparation) SIAM Journal on Scientific Computing, special issue on UQ.
OUU/SBOUU Problem Formulation
• Technical Challenge: Given a fixed computing budget, is it possible to identify minima in OUU problems? (It is possible w/ unlimited budget.)
Improvement is easy to discern when confidence intervals do not overlap, e.g., “At the 95% confidence level, F(x2) is an improvement over F(x1).”
F(x, u)
x
x1 x2
OUU/SBOUU Problem Formulation
• Technical Challenge: Given a fixed computing budget, is it possible to identify minima in OUU problems? (It is possible w/ unlimited budget.)
F(x, u)
x
x3x2
Improvement is harder to discern when confidence intervals overlap, e.g., “At the 95% confidence level, F(x3) is not significantly better than F(x2).”
New Themes
New Research Areas
• SBOUU (ongoing)• Computational Credibility,
quantitative answers for V&V– How can a given credibility measure be optimized given
constrained resources? – How much validation is sufficient? – How can credibility best be characterized for
extrapolation beyond the validation domain? – How are credibility measures best applied for decisions
with uncertainty?
• Computational Predictability– QMU (now three NNSA milestones!)
• Calibration Under Uncertainty
Technical Challenges
• Technical challenges– What Opt or UQ can be done if only one or two
calculations can fit in Red Storm?• Surrogates for expensive, noisy simulations
• SAND for large design spaces, expensive simulations
– Error bars for calculations • Epistemic uncertainty
• These are Cultural challenges, too!
Take-Home Messages
• Showed– Breadth of SNL activities– Unique simulations at SNL– New research directions we're pursuing
• Hope you say “quite an interesting world at SNL, in terms of applications and techniques needed to address”
• Anything sparked your interest? – Always interested in collaboration, visitors
• CSRI program
• samitch@sandia.gov • http://www.cs.sandia.gov/departments/9211/index.htm
The End
Questions?