A Mutual Information -Based Framework to Use High-Fidelity ...Jul 02, 2017 · Kathleen Schmidt,...
Transcript of A Mutual Information -Based Framework to Use High-Fidelity ...Jul 02, 2017 · Kathleen Schmidt,...
Ralph C. SmithDepartment of Mathematics
North Carolina State University
Support: DOE Consortium for Advanced Simulation of LWR (CASL)
National Science Foundation (NSF)NNSA Consortium for Nonproliferation Enabling Capabilities (CNEC)
Air Force Office of Scientific Research (AFOSR)
A Mutual Information-Based Framework to Use High-Fidelity Codes to Calibrate Low-Fidelity Codes (Hi2Lo)
97 97.5 98 98.5 990
1
2
3
4
5
6
7
8 yIteration 3Iteration 7Iteration 15
Example 1: Pressurized Water Reactors (PWR) -- CASL
Models:• Involve neutron transport, thermal-hydraulics, chemistry.
• Inherently multi-scale, multi-physics.
CRUD Measurements: Consist of low resolution images at limited number of locations.
Example: Pressurized Water Reactors (PWR)Thermo-Hydraulic Equations: Mass, momentum and energy balance for fluid
Hi2Lo Goals: • Use experimental design to guide use of high-fidelity codes and experiments to
“optimally” calibrate low-fidelity models.
• Use high-fidelity codes to improve closure relations for low-fidelity codes
Example: Shearon Harris outside Raleigh
@
@t(↵f⇢f ) +r · (↵f⇢f vf ) = -�
↵f⇢f@vf
@t+ ↵f⇢f vf ·rvf +r · �R
f + ↵fr · �+ ↵frpf
= -F R - F + �(vf - vg)/2 + ↵f⇢f g
@
@t(↵f⇢f ef ) +r · (↵f⇢f ef vf + Th) = (Tg - Tf )H + Tf�f
-Tg(H - ↵gr · h) + h ·rT - �[ef + Tf (s⇤ - sf )]
-pf
✓@↵f
@t+r · (↵f vf ) +
�
⇢f
◆
Example: Pressurized Water Reactors (PWR)Thermo-Hydraulic Equations: Mass, momentum and energy balance for fluid
• Brian Williams (LANL)• Brian Adams, Vince Mousseau, Chris Jones, Natalie Gordon, Lindsay Gilkey,
Laura Swiler, Kathryn Maupin (Sandia) • Bob Salko, Kevin Clarno (ORNL)• Yixing Sung, Emre Tatli (Westinghouse)
Note: Large collaborative effort
Example: Shearon Harris outside Raleigh
@
@t(↵f⇢f ) +r · (↵f⇢f vf ) = -�
↵f⇢f@vf
@t+ ↵f⇢f vf ·rvf +r · �R
f + ↵fr · �+ ↵frpf
= -F R - F + �(vf - vg)/2 + ↵f⇢f g
@
@t(↵f⇢f ef ) +r · (↵f⇢f ef vf + Th) = (Tg - Tf )H + Tf�f
-Tg(H - ↵gr · h) + h ·rT - �[ef + Tf (s⇤ - sf )]
-pf
✓@↵f
@t+r · (↵f vf ) +
�
⇢f
◆
Example 2: Radiation Source Localization in Urban Setting CNEC: Consortium for Nonproliferation Enabling Capabilities
Photofrom[Ştefănescu,etal.,2016],courtesyofJ.Hite.
Kathleen Schmidt, Razvan Stefanescu, Jared Cook: NCSU MathematicsIsaac Michaud: NCSU StatisticsJason Hite, John Mattingly: NCSU Nuclear Engineering
Example 2: Radiation Source Localization in Urban Setting Model: i th detector
Notes: • Simplified Boltzmann transport model.
• Represent source and detectors as points.
• Consider only uncollided flux.
• Ray tracing used to quantify detector responses.
�i = E[Ri(I0, r0)] =�ti✏iAi I0
4⇡|ri - r0|2exp
-NiX
ni=1
�ni sni
!
+ E[Bi�ti ]
10 Detectors
Goal: Use experimental design to guide fixed and moving sensor strategies. Parameters: ✓ = [x0, y0, I0]
Bayesian Calibration: Nuclear Power Plant ApplicationExample: Dittus—Boelter Relation
i.e., [0, 0.046], [0, 1.6], [0,0.8]
Industry Standard: Conservative, uniform, bounds
Bayesian Analysis: Employ conservative bounds as priors
Note: • Substantial reduction in parameter uncertainty
• Quantifies correlation between parameters
Nu = 0.023Re0.8Pr 0.4
2� ⇡ 0.0035 2� ⇡ 0.06 2� ⇡ 0.03
✓1
✓2
✓2
✓3
Prediction Intervals: Nuclear Power Plant Application Strategy: Propagate parameter uncertainties through COBRA-TF to determine
uncertainty in maximum fuel temperature.
Ramifications: • Temperature uncertainty reduced from 40 degrees to 5 degrees.
• Can run plant 20 degrees hotter, which significantly improves efficiency.
• Warrants continued calibration of closure relations.
• Accommodates disparate data sets.
Prediction Intervals: Nuclear Power Plant Application Strategy: Propagate parameter uncertainties through COBRA-TF to determine
uncertainty in maximum fuel temperature.
Ramifications: • Temperature uncertainty reduced from 40 degrees to 5 degrees.
• Can run plant 20 degrees hotter, which significantly improves efficiency.
• Warrants continued calibration of closure relations.
• Accommodates disparate data sets.
High-to-Low Framework:• Objective 1: Use synthetic data generated from validated high-fidelity
codes to calibrate lower-fidelity codes.• Objective 2: Employ high-fidelity codes to improve closure relations and
reduce model discrepancy in low-fidelity codes.• Goal: Using information-theoretic framework, calibrate parameters in
the low-fidelity code using as few high-fidelity simulations as possible.
Scope of UQ: Nuclear Power Plant Analysis
Code Requirements:• CFD simulations must reflect same physics
and design configurations as CTF.
• CFD errors/uncertainties should be quantified through mesh analysis and statistical validation.
• CFD simulations must be performed for designs where code has been verified and validated.
• One must formulate statistical models for QoI with probabilistic or bound-delineated errors.
STAR−CCM+i
+ Calibration ExperimentValidation Experiment
Inputs: θ
Inputs: θ
χ2
χ3
ε iErrorsMeasurement
χ1
δ(χ i ,θ̂ )
S i
Inputs: θ or g(θ)
Low−Fidelity
VerificationHigh−Fidelity
COBRA−TF; e.g., DB
Simulation Codes
ExperimentsPhysical
Thermal−Hydraulic
n
Gaussian Process
Response Surface
Simulation−Based Models
ni S i ε iyi = f(χ i ,θ)+ + + +δ(χ i ,θ̂ )
+
ValidationRegime
+ + +
+
++ +
+
ValidationCalibration
Design of ExperimentsPrediction Intervals
Low−Fidelity Statistical ModelHigh−Fidelity
Physical Models
Scope of UQ: Nuclear Power Plant Analysis
STAR−CCM+i
+ Calibration ExperimentValidation Experiment
Inputs: θ
Inputs: θ
χ2
χ3
ε iErrorsMeasurement
χ1
δ(χ i ,θ̂ )
S i
Inputs: θ or g(θ)
Low−Fidelity
VerificationHigh−Fidelity
COBRA−TF; e.g., DB
Simulation Codes
ExperimentsPhysical
Thermal−Hydraulic
n
Gaussian Process
Response Surface
Simulation−Based Models
ni S i ε iyi = f(χ i ,θ)+ + + +δ(χ i ,θ̂ )
+
ValidationRegime
+ + +
+
++ +
+
ValidationCalibration
Design of ExperimentsPrediction Intervals
Low−Fidelity Statistical ModelHigh−Fidelity
Physical Models
Code Requirements:• Computational budgets dictate that a
limited number of STAR simulations will be available to generate synthetic data to inform or calibrate CTF.
– Necessitates efficient experimental design
Example 1: Motivation and Strategy
Statistical Models:
Notation:
Low-Fidelity Model:
Friction Factor: f = 64/Re
dn = d`(✓, ⇠n) + �(⇠n) + "n(⇠n)
edn = dh(⇠n) + e"n(⇠n)
• ⇠n: Design conditions; e.g., Reynolds numbers
• �(⇠n): Model discrepancy
• ✏n(⇠n): Random observation or discretization errors
f (✓) = ✓1Re✓2
• d`(✓, ⇠n): Low-fidelity model; e.g., friction factor
• dh(⇠n): High-fidelity model; e.g., Hydra
• ✓: Low-fidelity model parameters
High-Fidelity Model: Hydra computing Poiseulle flow
Design Algorithm
Existing data:
Calibrate parameters of low-fidelity model:
Choose new design to reduce uncertainty in
Evaluate high-fidelity model at
Delayed Rejection Adaptive Metropolis (DRAM)
kNN or ANN Estimate of Mutual Information
dn = d`(✓, ⇠n) + �(⇠n) + "n(⇠n)
edn = dh(⇠n) + e"n(⇠n) Dn-1 = {ed1, ed2, · · · , edn-1}
[(⇠1, ed1), (⇠2, ed2), · · · , (⇠n-1, edn-1)]
d`(✓, ⇠n)
⇠n ✓
⇠n : edn = dh(⇠n) + e"n(⇠n)
Delayed Rejection Adaptive Metropolis (DRAM)
1. Determine q0 = arg min
q
NX
i=1
[�i - f (ti , q)]2
2. For k = 1, · · · , M(a) Construct candidate q⇤ ⇠ N(qk-1
, V )
(b) Compute likelihood
SSq⇤ =NX
i=1
[�i - f (ti , q⇤)]2
⇡(�|q) =1
(2⇡�2)n/2
e-SSq/2�2
(c) Accept q⇤with probability dictated by likelihood
Algorithm: [Haario et al., 2006] – MATLAB, Python, QUESO-Dakota
Mutual Information
Note:
Bayesian Framework: Quantifies change in knowledge due to new data
Goal:
Mutual Information: Two random variables I(X;Y)• Measure of variables’ mutual dependence• I(X,Y) quantifies reduction in uncertainty in X that knowing Y provides
Marginal Entropies
Set Analogy
I(X ; Y ) = H(X ) + H(Y )- H(X , Y )
= H(X )- H(X |Y )
= H(Y )- H(Y |X )
p(✓|Dn) =p(Dn|✓)p(✓)
p(Dn)=
p(edn, Dn-1|✓)p(✓)p(edn, Dn-1)
Provide framework to optimize information in
edn based on design ⇠n
Mutual Information
Physical Entropy:
Shannon Entropy: Quantifies unpredictability of information content
Mutual Information:
Note:
Utility Function
S = -kB
X
i
pi log pi • pi : Probability that system is in i thmicrostate
I(X ; Y ) =
Z
X,Y
p(x , y) log
p(x , y)
p(x)p(y)dxdy
H(X ) = -
Z
X
p(x) log p(x)dx
I(X ; Y ) =
Z
X,Y
p(x , y) log
p(x , y)p(x)p(y)
dxdy
=
Z
XY
p(x , y) log
p(x , y)p(x)
dxdy -
Z
XY
p(x , y) log p(y)dxdy
=
Z
XY
p(x)p(y |x) log p(y |x)dxdy -
Z
XY
p(x , y) log p(y)dxdy
= -
Z
X
p(x)H(Y |X )dx -
Z
Y
log p(y)p(y)dy
= -H(Y |X ) + H(Y )
Mutual Information
Utility Function:
Strategy:
• Marginalize over set of unknown future observations to compute average amount of information provided by design :
• Choose design condition that yields largest mutual information
• Experimental Design: [Terejanu et al., 2012]
• Implementation Issue: Efficient evaluation of mutual information
• Solution: Employ kth nearest neighbor (kNN) algorithm [Kraskov et al., 2004]
U(dn, ⇠n) =
Z
⌦p(✓|dn, Dn-1
) log p(✓|dn, Dn-1
)d✓-
Z
⌦p(✓|Dn-1
log p(✓|Dn-1
d✓
⇠n
I(✓; dn|Dn-1, ⇠n) =
Z
D
U(dn, ⇠n)p(dn|Dn-1, ⇠n)ddn
⇠⇤n = arg max
⇠n2⌅I(✓; dn|Dn-1
, ⇠n)
Example 1: Hydra
Example: Laminar Poiseuille flow to verify Hydra and low-fidelity model
Design Variable:
Friction Factor: f = 64/Re
Low-Fidelity Model:True Parameters:
0 500 1000 1500 20000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1
234 5 6 7 8 9
Reynolds Number
Fric
tion
Fact
or
Low−FidelityHigh−FidelityInitial Points
60 70 800
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Parameter a
−1.1 −1 −0.90
100
200
300
400
500
600
Parameter b
Stage 9Stage 5Stage 1
Stage 9Stage 5Stage 1
f (✓) = ✓1Re✓2
✓ = [64,-1]
⇠ = Re
Example 1: Hydra
0 500 1000 1500 20000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1
234 5 6 7 8 9
Reynolds Number
Fric
tion
Fact
or
Low−FidelityHigh−FidelityInitial Points
60 70 800
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Parameter a
−1.1 −1 −0.90
100
200
300
400
500
600
Parameter b
Stage 9Stage 5Stage 1
Stage 9Stage 5Stage 1
Model ComparisonDensity Evolution
Hydra Verification
Hydra-TH Values Analytic ValuesRe dp/dz Vavg f dp/dz Vavg f100 0.1851 0.0038 0.6782 0.1870 0.0039 0.6400200 0.3752 0.0075 0.3392 0.3740 0.0077 0.3200300 0.5623 0.0112 0.2269 0.5611 0.0116 0.2133400 0.7494 0.0150 0.1705 0.7481 0.0154 0.1600500 0.9366 0.0187 0.1365 0.9351 0.0193 0.1280
Example 2: Turbulent Mixing in COBRA-TF (CTF)
0.0 0.2 0.4 0.6 0.8 1.0
240
260
280
300
320
Scaled Input
Ts18
BETAExPRESTINGINAFLUX
Problem Setup:• Configuration (Design) Variables in STAR-
CCM+
– ExPRES: Initial pressure of fluid domain
– TIN: Initial temperature in fluid domain
– GIN: Inlet mass flow rate
– AFLUX: Average linear heat rate per rod
• Calibration Variable in CTF
– BETA: Turbulent mixing factor
• Experimental Data from WEC
– 22 mixing tests each of which produce 36 outlet temperatures
Dakota Workflow: Kathryn Maupin, Laura Swiler, Brian Adams, Brian Williams
Surrogate Required for COBRA-TF (CTF)
Ts1 Ts2 Ts3 Ts4 Ts5 Ts6
Range 114.2526 113.2762 112.8341 112.8384 113.2961 114.2797
PRESS 0.737077 0.639302 0.67627 0.679073 0.621807 0.759441
PRESS/Range 0.006451 0.005644 0.005993 0.006018 0.005488 0.006645
RMSE 0.457771 0.485425 0.518064 0.517842 0.484804 0.465511
RMSE/Range 0.004007 0.004285 0.004591 0.004589 0.004279 0.004073
Mutual Information Estimation in Dakota:• Requires 5000 independent samples
• MCMC with 20% burn-in removed and subsampling rate of 3 requires minimum of 18,750 iterations
• This necessitates construction and verification of fast surrogate for CTF
• Gaussian process (GP) surrogate trained and verified for all 36 subchannels
Surrogate Verification: PRESS (1-fold cross validation) and RMSE (50 test runs)
STAR-CCM+ Calibration of BETA in CTF
Hi2Lo Workflow and Results:• Calibrate BETA to initial simulations and/or
experiments.
• Generate predictions of HiFi code output using LoFi code – or its surrogate – at each candidate design.
• Estimate MI between calibrated BETA samples and HiFi predictions at each candidate.
– Select candidate with largest MI.
• Run HiFi code at optimal candidate, add calibration dataset, and recalibrate BETA.
– Repeat process until MI is sufficiently small or design budget is exhausted.
0.00 0.02 0.04 0.06 0.08 0.10
020
4060
80100
120
140
BETA Posteriors
BETA
1234567891011
BETA Posteriors
5 10 15 20
1.4
1.6
1.8
2.0
2.2
2.4
2.6
Current Design Size
Mut
ual I
nfor
mat
ion
●
●
●
●
●
●
●
● ●
●
● ● ●
●
●●
● ●
●
●
●
Mut
ual I
nfor
mat
ion
Design Step
Order of STAR-CCM+ Evaluations Selected by MI
ExPRES TIN GIN AFLUX MI ExPRES TIN GIN AFLUX MI1475 406.5 16.24 4.231 2390 465.5 18.60 4.584 2.17
1500 398.8 16.56 4.804 2.71 2320 551.1 14.87 3.702 2.19
2320 521.0 18.49 4.434 2.53 1475 406.6 16.24 4.214 2.182390 590.5 18.51 4.535 2.48 2390 556.9 22.19 5.645 2.092390 556.5 18.44 4.462 2.66 2375 553.0 14.85 3.735 2.162320 556.3 18.48 4.544 2.60 1465 518.5 22.26 3.780 2.192320 518.0 22.14 5.681 2.47 1465 514.5 18.58 3.202 2.232320 520.0 14.76 3.710 2.62 2320 593.7 18.38 4.621 2.232390 518.5 22.13 5.637 2.40 2320 553.8 22.23 5.661 1.992390 518.5 14.77 3.719 2.39 2320 591.5 22.11 5.564 1.482390 520.0 18.42 4.560 2.30 2390 591.4 22.26 5.641 1.32
Range of TIN essentiallycovered in 3 iterations
Example 3: Radiation Source Localization in Urban Setting Model: i th detector
Notes: • Simplified Boltzmann transport model.
• Represent source and detectors as points.
• Consider only uncollided flux.
• Ray tracing used to quantify detector responses.
�i = E[Ri(I0, r0)] =�ti✏iAi I0
4⇡|ri - r0|2exp
-NiX
ni=1
�ni sni
!
+ E[Bi�ti ]
10 Detectors
Goal: Use experimental design to guide fixed and moving sensor strategies. Parameters: ✓ = [x0, y0, I0]
Example 3: Radiation Source Localization in Urban Setting Algorithm:1.Set N equal to the number of samples to be used in kNN algorithm
2.Define the set of M possible discrete sensor locations
3.Obtain initial estimates for (x,y,I) using sensors placed at 3 locations from set of discrete sensor locations.
4.This leaves M-3 possible sensor locations.
10 Sensor Locations
5. For j = 4:M-1
a) Employ MCMC algorithm DRAM to construct 3xN matrix of parameter chains that are sent to kNN algorithm.
b) Employ kNN algorithm to determine design conditions that optimizes MI and hence indicates where 1 of 3 sensors should be moved.
c) Append location and response to data set.
d) Remove that location from design conditions.
0 50 100 150 20020
40
60
80
100
120
140
160
1
2 3
4
5
6
7
8
9
10
1112 13
14
15
16
17
18
1920
21
22
23
24
25 26
Example 3: Radiation Source Localization in Urban Setting Results: Order of chosen detector locations
Example 3: Radiation Source Localization in Urban Setting Results: Convergence of Posterior Distributions:
157.6 157.8 158 158.2 158.40
2
4
6
8
10 xIteration 3Iteration 7Iteration 15
3.2137 3.2138 3.2139 3.214 3.2141 3.2142 3.2143#109
0
1
2
3
4
5
6
7
8 #10-6 IIteration 3Iteration 7Iteration 15
97 97.5 98 98.5 990
1
2
3
4
5
6
7
8 yIteration 3Iteration 7Iteration 15
Concluding RemarksConclusions: • Mutual information (MI) framework can guide where in the design space to generate data using validated high-fidelity codes or experiments.
• MI framework can also be employed to guide sensor strategies for radiation detection.
• Computation of MI is natural in a Bayeisan framework. 0 500 1000 1500 20000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1
234 5 6 7 8 9
Reynolds Number
Fric
tion
Fact
or
Low−FidelityHigh−FidelityInitial Points
Future Work: • Extension to continuous design variables.
• Extend theory to accommodate highly correlated responses.
• Complete implementation of Hi2Lo framework and workflow in Dakota.
• Complete the CASL STAR-CCM+/CTF Hi2Lo analysis and implementation via Dakota.
• Experimentally test sensor placement strategy. 97 97.5 98 98.5 990
1
2
3
4
5
6
7
8 yIteration 3Iteration 7Iteration 15