Center for Medical Statistics, Informatics and Intelligent ...
Transcript of Center for Medical Statistics, Informatics and Intelligent ...
The multiple faces of shrinkageGeorg HeinzeCenter for Medical Statistics, Informatics and Intelligent SystemsSection for Clinical [email protected]
Partly supported by Austrian Science Fund FWF, Project I2276-N33
The multiple faces of shrinkage
β’
β’
Dunkler, Sauerbrei and Heinze, JStatSoft 2016
β’
Puhr, Heinze, Nold, Lusa and Geroldinger, StatMed 2017
Post-estimation shrinkage methods
β’
β’ π½
β’
β’
β’ π
β’ π½ π½(βπ)
β’ ππ = π π₯ππ π½π
(βπ)
β’ π
Use of the shrinkage factors
β’
β’
β’
β’
β’
β’
π¦πππ€ = π½0 + π π₯ππππ€ π½
β’
Sauerbreiβs (1999) βparameterwise shrinkage factorsβ
β’ π½ π½(βπ)
β’
partial πππ = π₯ππ π½π(βπ)
β’ ππ
β’
Dunklerβs (2016) extension of parameterwise shrinkage
β’ ππ
β’
β’
β’
β’
β’ πΊ πππ = πβπ½ππ₯ππ
π½π(βπ)
π = 1, β¦ , πΊ
β’ πππ ππ, π = 1, β¦ , πΊ
β’ π½(βπ) β π½ β π·πΉπ΅πΈππ΄π
From bias reduction to shrinkage and beyondJoint work with Rainer Puhr, Angelika Geroldinger, Sander Greenland
Firthβs penalization for logistic regression
πΏβ π½ = πΏ π½ det( πΌ π½ )1/2,
πΌ π½ πΏ π½
β’ π½,
β’
β’
Firthβs penalization for logistic regression
πΏβ π½ = πΏ π½ det(ππ‘ππ)1/2
π = diag expit Xiπ½ (1 β expit Xiπ½ )
= diag(ππ 1 β ππ )
β’
π ππ =1
2π½ = 0
β’1
2,
β’
Example of Greenland 2010
320
32
346 6 352
=32
352= 0.091 =
33
354= 0.093
= 2.03 = 2.73
321
33
346.5 6.5 354
logF(1,1) prior (Greenland and Mansournia, 2015)
β’
πΏ π½ β = πΏ π½ β βπ
π½π2
1+ππ½π
.
β β ββ β ββ β ββ β ββ β ββ β ββ β β
β β ββ β ββ β ββ β ββ β ββ β ββ β β
Simulation of bivariable log reg models
β’ π1, π2~Bin(0.5) π = 0.8, π = 50
β’ π½1 = 1.5 π½2 = 0.1 π
π
π½1
π½1
π½2
π½2
π·π
FLAC: Firthβs Logistic regression with Added Covariate
π=1
π
π¦π β ππ π₯ππ + βπ
1
2β ππ π₯ππ = 0; π = 0, β¦ , π
βπ π» = π1
2π πβ²ππ β1ππ1/2
π=1
π
π¦π β ππ π₯ππ +
π
π
βπ
1
2β ππ π₯ππ =
=
π=1
π
π¦π β ππ π₯ππ +
π=1
πβπ
2(π¦π β ππ) +
π=1
πβπ
2(1 β π¦π β ππ) = 0
FLAC: Firthβs Logistic regression with Added Covariate
β’
π=1
π
π¦π β ππ π₯ππ +
π=1
πβπ
2π¦π β ππ π₯ππ +
π=1
πβπ
21 β π¦π β ππ π₯ππ = 0
βπ/2 βπ/2
FLAC: Firthβs Logistic regression with Added Covariate
β’
π=1
π
π¦π β ππ π₯ππ +
π=1
πβπ
2π¦π β ππ π₯ππ +
π=1
πβπ
21 β π¦π β ππ π₯ππ = 0
βπ/2 βπ/2
Other methods for accurate prediction
β’
πΏ π½ β = πΏ π½ det(ππ‘ππ)π, π = 0.1,
β’
β’
β’
Comparison
FLAC
β’
β’
β’
β’
β’
β’
β’
Bayesian methods (CP, logF)
β’
β’ m m m
m m
β’ m
β’
β’
Ridge
Conclusion
Part 1: Prediction under model uncertainty
β’
β’
β’
β’
β’
β’
β’
Part 2: Prediction under sparsity (fixed model)