Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical...
Transcript of Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical...
![Page 1: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/1.jpg)
1
Best fit mehods
Least squares
Maximum likelihood
![Page 2: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/2.jpg)
2
The method of the Least Squares
ii
ii
ii
ii
i
yyL
2
2
2
2 )(Min
)(
2
1exp
2
1MaxMax
![Page 3: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/3.jpg)
3
From probabilitycalculus
![Page 4: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/4.jpg)
4
Probability Intervals
v = uv
u
![Page 5: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/5.jpg)
5
This band contains 68.3 % of x values
This band contains 68.3 % of x values
The ellipse contains 68.3 % of (x,y) points
![Page 6: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/6.jpg)
6
Least SquaresCase I
![Page 7: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/7.jpg)
7
From Statistics
When N is fixed theDoF are K-1
![Page 8: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/8.jpg)
8
Shaded areas:probability to be wrong if we reject the model(type I error)
0.5
0.5
![Page 9: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/9.jpg)
9
Least SquaresCase II
s
![Page 10: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/10.jpg)
10
Case II
![Page 11: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/11.jpg)
11
Least SquaresCase III
![Page 12: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/12.jpg)
12
Case III
![Page 13: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/13.jpg)
13
Standard case
f(x) is the correlation funtion
Common in labmeasurements
![Page 14: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/14.jpg)
14
Some warnings
![Page 15: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/15.jpg)
15
Some warnings
![Page 16: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/16.jpg)
16
Some warnings
![Page 17: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/17.jpg)
17
Linear Least Squares
leads
(x, ) = F
![Page 18: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/18.jpg)
18
Linear Least Squares
![Page 19: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/19.jpg)
21
Theorems on Least Squares
estimations
![Page 20: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/20.jpg)
Binned and unbinned likelihood
22
k
i
ii
ni
k
i
pnnL
pL i
1
1
)](ln[),(ln
)()(
Binned likelihood
Unbinned likelihood
N
i
ixpL1
),()(
k bins
N points
![Page 21: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/21.jpg)
Unbinned likelihood
24
N
n
nfi
n
MpT)L(M
p
1
),,(,
momenta4
theareif
MEASURED
)( nfi pLipsTW
First method: generate MC events
following phase space, weight them
with T=|<f|T|i>|2 and compare with
binned data
Second method, unbinned likelihood:
fill the transition matrix with
the measured momenta
and maximize
![Page 22: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/22.jpg)
Unbinned likelihood
25
DCxwmxA
DCAwmL
n
N
n
n ),,(Breit
),,,,(
1
![Page 23: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/23.jpg)
26
The extended likelihoodi
i
en
nLN
i
n
i
1 !),(
i
)()](ln[),(ln11
k
i
i
k
i
iinnL
k
i
iin
i
k
i
pnnLpL i
11
)](ln[),(ln)()(
)()()(),( NpNpN iiiiSince
If there is no functional relation between N and
the result is the same as for the non extended likelihood
when N is a function of as in the case of a detector efficiency,
![Page 24: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/24.jpg)
27
Binomialp=0.5
Gaussian=70=10
p= 0.522 0.015
p= 0.528 0.017
=70.09 0.31=9.73 0.22
=69.97 0.31=9.59 0.22
L
![Page 25: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/25.jpg)
Lagrange multipliers
28
A function f(x,y) to be minimized with the constraint G(x,y)=0
G(x,y)=0
f
||fG
The constrained minimum condition is ||f = 0 . Hence:
andparameters free allw.r.t.
G f S
minimizetoimplies0),(constraintthewith0|| yxGGff
G
![Page 26: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/26.jpg)
Degrees of Freedom
29
)()(DOF)),((
2
2
2 anynxafy
i i
ii
0DOF)(
2
2
2ii
i i
ii yaay
Constraint with an internal function:
Constraint with with an external function:
)()()]()([)()(DOF
)()(),()(
0
2
2
2
znnznnanyn
anynzaay
k
kk
i i
ii
Unconstrained 2 does not work:
![Page 27: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/27.jpg)
30
Minimization withconstraints
![Page 28: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/28.jpg)
31
Minimization withConstraints:example
![Page 29: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/29.jpg)
32
Minimization withConstraints:example
![Page 30: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/30.jpg)
33
![Page 31: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/31.jpg)
34
Kinematic fit: degrees offreedom
0,0),(
),(2)()(3
1
3
1
2
dDpp
ppppWpp
u
u
fit
jmeasjij
fit
i
N
i
measi
N
j
Degrees of freedom:
N(p) means number of the p variables
satisfiednotaresconstraint
fitwithout
fulfilledaresconstraint
worksfitthe.)()()(
N(constr.))N(pN(p)
N(constr.))N(pN(p)
constrNpNpN
u
u
u
)(.).( upNeqconstrN
![Page 32: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/32.jpg)
35
Kinematic fit: examples
A
p
Unmeasured: 1Constraints: 4p
3
2
3
3C
Total: 7CTotal: 4C
![Page 33: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/33.jpg)
36
Kinematic fit: examples
4C
1C
4C
5C
![Page 34: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/34.jpg)
37
Kinematic fit: goodness of fit
2
0
2
2
d),(1 p
cut
)1,0(d)(0
UxxpCX
![Page 35: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/35.jpg)
38
Non linear fits(MINUIT)
![Page 36: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/36.jpg)
39
Non linear fits(MINUIT)
![Page 37: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/37.jpg)
41
![Page 38: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/38.jpg)
Luca Lista Statistical Methods for Data
Analysis
42
Minuit function MIGRAD• Purpose: find minimum
**********
** 13 **MIGRAD 1000 1
**********
(some output omitted)
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM MIGRAD STATUS=CONVERGED 31 CALLS 32 TOTAL
EDM=2.36773e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 mean 8.84225e-02 3.23862e-01 3.58344e-04 -2.24755e-02
2 sigma 3.20763e+00 2.39540e-01 2.78628e-04 -5.34724e-02
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 3.338e-04
3.338e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00430 1.000 0.004
2 0.00430 0.004 1.000
Parameter values and approximate errors reported by MINUIT
Error definition (in this case 0.5 for a likelihood fit)
Progress information,watch for errors here
![Page 39: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/39.jpg)
Luca Lista Statistical Methods for Data
Analysis
43
Minuit function MIGRAD• Purpose: find minimum
**********
** 13 **MIGRAD 1000 1
**********
(some output omitted)
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM MIGRAD STATUS=CONVERGED 31 CALLS 32 TOTAL
EDM=2.36773e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 mean 8.84225e-02 3.23862e-01 3.58344e-04 -2.24755e-02
2 sigma 3.20763e+00 2.39540e-01 2.78628e-04 -5.34724e-02
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 3.338e-04
3.338e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00430 1.000 0.004
2 0.00430 0.004 1.000
Approximate Error matrix
And covariance matrix
Value of 2 or likelihood at minimum
(NB: 2 values are not divided by Nd.o.f)
![Page 40: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/40.jpg)
44
![Page 41: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/41.jpg)
Luca Lista Statistical Methods for Data
Analysis
45
Minuit function MIGRAD• Purpose: find minimum
**********
** 13 **MIGRAD 1000 1
**********
(some output omitted)
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM MIGRAD STATUS=CONVERGED 31 CALLS 32 TOTAL
EDM=2.36773e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 mean 8.84225e-02 3.23862e-01 3.58344e-04 -2.24755e-02
2 sigma 3.20763e+00 2.39540e-01 2.78628e-04 -5.34724e-02
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 3.338e-04
3.338e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00430 1.000 0.004
2 0.00430 0.004 1.000
Status: Should be ‘converged’ but can be ‘failed’
Estimated Distance to Minimumshould be small O(10-6)
Error Matrix Qualityshould be ‘accurate’, but can be ‘approximate’ in case of trouble
![Page 42: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/42.jpg)
Luca Lista Statistical Methods for Data
Analysis
46
Minuit function HESSE• Purpose: calculate error matrix from 2
2
dp
Ld
**********
** 18 **HESSE 1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM HESSE STATUS=OK 10 CALLS 42 TOTAL
EDM=2.36534e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 mean 8.84225e-02 3.23861e-01 7.16689e-05 8.84237e-03
2 sigma 3.20763e+00 2.39539e-01 5.57256e-05 3.26535e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 2.780e-04
2.780e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00358 1.000 0.004
2 0.00358 0.004 1.000
Symmetric errors calculated from 2nd
derivative of –ln(L) or 2
![Page 43: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/43.jpg)
Luca Lista Statistical Methods for Data
Analysis
47
Minuit function HESSE
• Purpose: calculate error matrix from **********
** 18 **HESSE 1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM HESSE STATUS=OK 10 CALLS 42 TOTAL
EDM=2.36534e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 mean 8.84225e-02 3.23861e-01 7.16689e-05 8.84237e-03
2 sigma 3.20763e+00 2.39539e-01 5.57256e-05 3.26535e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 2.780e-04
2.780e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00358 1.000 0.004
2 0.00358 0.004 1.000
Error matrix (Covariance Matrix)
calculated from1
2 )ln(
ji
ijdpdp
LdV
2
2
dp
Ld
![Page 44: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/44.jpg)
Luca Lista Statistical Methods for Data
Analysis
48
Minuit function HESSE
• Purpose: calculate error matrix from **********
** 18 **HESSE 1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM HESSE STATUS=OK 10 CALLS 42 TOTAL
EDM=2.36534e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 mean 8.84225e-02 3.23861e-01 7.16689e-05 8.84237e-03
2 sigma 3.20763e+00 2.39539e-01 5.57256e-05 3.26535e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 2.780e-04
2.780e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00358 1.000 0.004
2 0.00358 0.004 1.000
Correlation matrix ij
calculated from
ijjiijV
2
2
dp
Ld
![Page 45: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/45.jpg)
Luca Lista Statistical Methods for Data
Analysis
49
Minuit function HESSE
• Purpose: calculate error matrix from **********
** 18 **HESSE 1000
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=257.304 FROM HESSE STATUS=OK 10 CALLS 42 TOTAL
EDM=2.36534e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 mean 8.84225e-02 3.23861e-01 7.16689e-05 8.84237e-03
2 sigma 3.20763e+00 2.39539e-01 5.57256e-05 3.26535e-01
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 2 ERR DEF=0.5
1.049e-01 2.780e-04
2.780e-04 5.739e-02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2
1 0.00358 1.000 0.004
2 0.00358 0.004 1.000
Global correlation vector:correlation of each parameter
with all other parameters
2
2
dp
Ld
![Page 46: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/46.jpg)
Luca Lista Statistical Methods for Data
Analysis
50
Wouter Verkerke, NIKHEF
Minuit function MINOS
• Error analysis through nll contour finding**********
** 23 **MINOS 1000
**********
FCN=257.304 FROM MINOS STATUS=SUCCESSFUL 52 CALLS 94 TOTAL
EDM=2.36534e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER PARABOLIC MINOS ERRORS
NO. NAME VALUE ERROR NEGATIVE POSITIVE
1 mean 8.84225e-02 3.23861e-01 -3.24688e-01 3.25391e-01
2 sigma 3.20763e+00 2.39539e-01 -2.23321e-01 2.58893e-01
ERR DEF= 0.5
Symmetric error
(repeated result from HESSE)
MINOS errorCan be asymmetric
(in this example the ‘sigma’ error is slightly asymmetric)
![Page 47: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/47.jpg)
Luca Lista Statistical Methods for Data
Analysis
51
Mitigating fit stability problems • Strategy I – More orthogonal choice of parameters
– Example: fitting sum of 2 Gaussians of similar width
),;()1(),;(),,,;( 221121 msxGfmsxfGssmfxF
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL [ f] [ m] [s1] [s2]
[ f] 0.96973 1.000 -0.135 0.918 0.915
[ m] 0.14407 -0.135 1.000 -0.144 -0.114
[s1] 0.92762 0.918 -0.144 1.000 0.786
[s2] 0.92486 0.915 -0.114 0.786 1.000
HESSE correlation matrix
Widths s1,s2
strongly correlatedfraction f
![Page 48: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/48.jpg)
52
Mitigating fit stability problems – Different parameterization:
– Correlation of width s2 and fraction f reduced from 0.92 to 0.68
– Choice of parameterization matters!
• Strategy II – Fix all but one of the correlated parameters
– If floating parameters are highly correlated, some of them may be redundant
and not contribute to additional degrees of freedom in your model
),;()1(),;( 2212111 mssxGfmsxfG
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL [f] [m] [s1] [s2]
[ f] 0.96951 1.000 -0.134 0.917 -0.681
[ m] 0.14312 -0.134 1.000 -0.143 0.127
[s1] 0.98879 0.917 -0.143 1.000 -0.895
[s2] 0.96156 -0.681 0.127 -0.895 1.000
![Page 49: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/49.jpg)
53
![Page 50: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/50.jpg)
54
![Page 51: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/51.jpg)
55
Non linear fits(MINUIT)
![Page 52: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/52.jpg)
56
END
![Page 53: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/53.jpg)
57
![Page 54: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/54.jpg)
58
![Page 55: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/55.jpg)
59
Case III
![Page 56: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/56.jpg)
60
Fit stability with polynomials
• Warning: Regular parameterization of polynomials
a0+a1x+a2x2+a3x
3 nearly always results in strong correlations
between the coefficients ai.
– Fit stability problems, inability to find right solution common at higher
orders
• Solution: Use existing parameterizations of polynomials that have
(mostly) uncorrelated variables
– Example: Chebychev polynomials
![Page 57: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/57.jpg)
Luca Lista Statistical Methods for Data
Analysis
61
Browsing fit results
• As fits grow in complexity (e.g. 45 floating parameters),
number of output variables increases
– Need better way to navigate output that MINUIT screen dump
• RooFitResult holds complete snapshot of fit results
– Constant parameters
– Initial and final values of floating parameters
– Global correlations & full correlation matrix
– Returned from RooAbsPdf::fitTo() when “r” option is supplied
• Compact & verbose printing modefitres->Print() ;
RooFitResult: min. NLL value: 1.6e+04, est. distance to min: 1.2e-05
Floating Parameter FinalValue +/- Error
-------------------- --------------------------
argpar -4.6855e-01 +/- 7.11e-02
g2frac 3.0652e-01 +/- 5.10e-03
mean1 7.0022e+00 +/- 7.11e-03
mean2 1.9971e+00 +/- 6.27e-03
sigma 2.9803e-01 +/- 4.00e-03
Alphabeticalparameter
listing
Compact Mode
Constantparametersomitted in
compact mode
![Page 58: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/58.jpg)
Luca Lista Statistical Methods for Data
Analysis
62
Browsing fit resultsfitres->Print(“v”) ;
RooFitResult: min. NLL value: 1.6e+04, est. distance to min: 1.2e-05
Constant Parameter Value
-------------------- ------------
cutoff 9.0000e+00
g1frac 3.0000e-01
Floating Parameter InitialValue FinalValue +/- Error GblCorr.
-------------------- ------------ -------------------------- --------
argpar -5.0000e-01 -4.6855e-01 +/- 7.11e-02 0.191895
g2frac 3.0000e-01 3.0652e-01 +/- 5.10e-03 0.293455
mean1 7.0000e+00 7.0022e+00 +/- 7.11e-03 0.113253
mean2 2.0000e+00 1.9971e+00 +/- 6.27e-03 0.100026
sigma 3.0000e-01 2.9803e-01 +/- 4.00e-03 0.276640
Verbose printing mode
Constant parameterslisted separately
Initial,final value and global corr. listed side-by-side
Correlation matrix accessed separately
![Page 59: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/59.jpg)
63
Browsing fit results• Easy navigation of correlation matrix
– Select single element or complete row by parameter name
–
• RooFitResult persistable with ROOT I/O
– Save your batch fit results in a ROOT file and navigate
your results just as easy afterwards
r->correlation("argpar","sigma")
(const Double_t)(-9.25606412005910845e-02)
r->correlation("mean1")->Print("v")
RooArgList::C[mean1,*]: (Owning contents)
1) RooRealVar::C[mean1,argpar] : 0.11064 C
2) RooRealVar::C[mean1,g2frac] : -0.0262487
C
3) RooRealVar::C[mean1,mean1] : 1.0000 C
4) RooRealVar::C[mean1,mean2] : -
0.00632847 C
5) RooRealVar::C[mean1,sigma] : -0.0339814
C
![Page 60: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/60.jpg)
64
![Page 61: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/61.jpg)
65
SL
(15% error)
![Page 62: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/62.jpg)
66
1.9 + 5.8 x2 - 0.6 x3
![Page 63: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/63.jpg)
67
Applications of Least Squares
![Page 64: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/64.jpg)
68
Search forcorrelations
-
![Page 65: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/65.jpg)
69
Search forcorrelations
![Page 66: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/66.jpg)
70
Search forcorrelations
![Page 67: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/67.jpg)
71
-0.95 + 1.03 x2
![Page 68: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/68.jpg)
72
Minimization withconstraints
= p measured
= q non measured
![Page 69: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/69.jpg)
Kinematic fit
73
![Page 70: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/70.jpg)
74
Kinematic fit
![Page 71: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/71.jpg)
Kinematic fit
75
0,0),(
),(2)()(3
1
3
1
2
dDpp
ppppWpp
u
u
fit
jmeasjij
fit
i
N
i
measi
N
j
Look at measured and unmeasured variables!
![Page 72: Best fit mehods Least squares Maximum likelihoodrotondi/BestfitBf.pdf · Luca Lista Statistical Methods for Data Analysis 51 Mitigating fit stability problems • Strategy I –More](https://reader033.fdocuments.in/reader033/viewer/2022042123/5e9e4f7f43e11c091a7f2c7d/html5/thumbnails/72.jpg)
76
Least Squaresproperties