MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 1 Roland Preuss and Udo v....
-
Upload
hannah-terry -
Category
Documents
-
view
218 -
download
2
Transcript of MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 1 Roland Preuss and Udo v....
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 1
Roland Preuss and Udo v. Toussaint
Comparison of
Numerical Methods for
Evidence Calculation
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 2
• Evidence computation crucial for model comparison
• Evidence is normalization constant in posterior computation
High-dimensional integral over parameter space is required
How?
Evidence Computation
IMDp
IMpIMDpIMDp
,|
,|,,|,,|
IMpIMDpIDMp |,|,|
IMpIMDpdIMDp ,|,,|,|
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 3
Variety of techniques:
-Analytical methods:
- Laplace approximation (saddle-point approximation)
- Variation principle
-Deterministic methods
- Numerical quadrature
-Probabilistic methods
Evidence Computation
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 4
Evidence Computation: Test cases 1&2
|xpxdITest cases:
2exp
2
1|
xx
xpT
N
03.0
3.0
2
1
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 5
Evidence Computation: Test cases 3&4
222
211 expexp
2
1|
dxdxdxdx
xpTT
N
03.0
3.0
2
1
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 6
Evidence Computation: Test cases 5&6
222
211
2expexp10
2210
1|
dxdxdxdx
xp
TT
NN
03.0
3.0
2
1
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 7
Evidence Computation: Test cases 5&6
Methods:
1. Laplace Approximation
2. Quadrature: Trapezoidal Rule
3. Importance Sampling
4. VEGAS Algorithm
5. Thermodynamic Integration
6. Nested Sampling
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 8
Evidence Computation: Laplace approximation
Laplace Approximation: |2
1exp| *
00 xpxxHxxAxp T
H
xpxdxpxdN
det
2|| *
Only for unimodal distributions!
Pay attention to the integration region (eg. >0): Improvements eg by von der Linden, McKay,…
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 9
Evidence Computation: Trapezoidal rule
Quadrature eg by trapezoidal rule:
Curse of dimension: Number of support points per dimension: N(N=15,150)
Number of dimensions: K
Total number of support points:
KN Dimension 1 2 4 8
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 10
Evidence Computation: Importance Sampling
Importance Sampling: xgxg
xpxdxpxdI
Samples are generated from simpler function g(x) easy to sample from. Usually a gaussian with widths from variances from previous MCMC runs (Metropolis algorithm).
Dimension 1 2 4 8 16
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 11
Evidence Computation: Vegas
VEGAS algorithm:
(Peter Lepage)
Samples are generated from a separable function g(x)
Dimension 1 2 4 8 16
• Algorithm freely available from Numerical Recipes
• Appropriate coordinate system important
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 12
Evidence Computation: Thermodynamic IntegrationThermodynamic Integration: Slowly introduce likelihood structure into prior:
(Stat. Physics, Neal 1993, Skilling 97)
Z(0)=1 and Z(1)=Evidence
xxxdZ
1
0
1
0
lnln
ln
xxxdd
ZdI
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 13
Evidence Computation: Thermodynamic IntegrationThermodynamic Integration: Slowly introduce likelihood structure into prior:
xxxdZ
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 14
Evidence Computation: Nested Sampling
Nested Sampling: Sample from prior within likelihood constraint:
Evidence=
Probabilistic measure of the >active< prior volume
kk k
kk Lz
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 15
Evidence Computation: Nested Sampling
Nested Sampling:
Some recent developements: arXiv.org/abs/0704.3704 (Farhan Feroz)
arXiv.org/astro-ph/0701867 (R. Shaw)
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 16
Evidence Computation: Conclusion
Conclusion: - None of the methods is a foolproof black box
Use more than one method!
Outlook: - More methods are available, eg. Perfect Tempering
- Still lack of extensive and systematic studies about properties
- Multimodal (nongaussian) analytical tracktable examples needed
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 17
Evidence Computation: Conclusion
Conclusion: - None of the methods is a foolproof black box
Use more than one method!
Outlook: - More methods are available, eg. Perfect Tempering
- Still lack of extensive and systematic studies about properties
- Multimodal (nongaussian) analytical tracktable examples needed
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 18
Fusion Research
Example: W7-X
Thomson scattering
ECE
Magnetics
Langmuir Probes
Interferometry
Bolometry
Video
H
Calorimetry
Neutral gas
Vis. spectroscopy
CXRS
SPRED
Soft-X ray
Thermography
Flux surface measurements
Diagnostic beam
Neutron detectors
Diagnostics Software(data analysis)
General purpose (stellarator) software
Integrated data analysis
Modelling
Infrastructure &management
Spectroscopy
Imaging
Tomography
DedicatedDiagnostics(misc.)
W7-X DiagnosticsSoftware
2-D diagnostics essential!
BES
Zeff
CO-Monitor
(by A. Dinklage)
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 19
Fusion Research: Tomography
Wide variety of tomographic configurations: experimental requirements, costs,…
Goal: best possible reconstruction of 2-D profile
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 20
Fusion Research: Tomography
•Tomographic reconstruction: underdetermined & ill-posed
• Prior information available
standard inversion techniques
mostly perform poorly
??
?
??
? ??
?
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 21
Thin-Plate Splines
-Transport along magnetic field lines very fast: smooth profiles usually favored:
curvature only when enforced by the data
- 1D- case: cubic splines
- 2D- case: Minimize curvature IB of f(x,y) (can be generalized )
222 2 yyxyxxB fffdydxI
Fundamental solution[1]:
22 ln rrrU
[1] G. Wahba, Spline models for Observational Data, SIAM, 1990
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 22
Thin-Plate Splines
Interpolating function z=f(x,y):
3,||,,||, 01
nyaxaayxyxUwyxfz yxii
n
ii
Coefficients wi, a are given by the solution of
00
z
a
w
P
PKT
with Kij=U(||(xi,yi)-(xj,yj)||), Pi=(1,xi,yi)
The bending energy is given by zrwwKwwI TB ,,
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 23
Model
Model: jj
iji gCm , Cij: response matrix
gj : emissivity on grid space
Likelihood: 2
2
2,,|
i
iiii
mdIgdp
with ),( yxfdxdygjA
j
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 24
Prior Distributions
• Number of support points n on grid with N places:
)|(,|),,|()|( InpInRpInRzpIMp
• Curvature : Testable information
)),(),(exp(1
,,| zRwKzRwZ
nRzp T
• Hyperparameter :
MaxMinIp ,/1)|(
• Model prior factorizes:
!)3(
!)!()|,(
NN
nnNInRp
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 25
IpIRzpdIRzDpzdIMp
IRzpIRzDpzdIMpIMDpIMpIDMp
|,,|,,||
,|,,||,||,|
Posterior
• Posterior for Model Mi :
• Evidence approximation:
• Simple sampling in N,R
• 1-D search in : *
2/1**** )det(,|,,,,|,| HIMzpIMzDpIMDp
)det(
)2(,,|,,,||
,,|,,||,,|
0
**
HIMzpIMzDpIp
IMzpIMDpzdIpIMDp
n
• Optimization preferred: Laplace Approximations…
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 26
Thin-Plate Splines: Results
• Reconstruction of challenging (but realistic) 2-d emission profile:
• Result of evidence weighted average
Mock profile: Reconstruction: Difference:
Key features, shape and absolut intensity recovered
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 27
Thin-Plate Splines: Results
Location of z-optimized support points:
Open issue: Support points outside of region of interest beneficial?
Reconstruction within the uncertainty
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 28
Thin-Plate Splines: Outlook
• Extension to f(x,y,z): Viewing cone
• Extension to f(x,y,(z),t):
time correlated data
Connection to morphing applications
Drawback: Huge amounts of data to be processed
• Online (monitoring) requirements:Use as input for Bayesian Neural Networks
• Check approximations with MCMC
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 29
Thin-Plate Splines: Conclusion
Thank you!
MaxEnt 2007, Udo v. Toussaint, July 2007, [email protected] 30
W7-X: Design
• 50+36SL-Coils
• Optimized shape
• stability without current-drive
• Size:
major: 5.5m
minor: 0.53m
• Magnetic field:
3.3T (2.5T reg)
12kA, 38MW/m2
• Heating:
10 MW ECR
4 MW ICR
5 MW NBI