Smoothing mortality rates using R Gary Brown & Julie Mills.
-
Upload
herbert-bennett -
Category
Documents
-
view
220 -
download
4
Transcript of Smoothing mortality rates using R Gary Brown & Julie Mills.
Smoothing mortality rates using R
Gary Brown & Julie Mills
Overview
• Introduction• Context• Methodology• Implementation• Results• Summary• What’s next?
Introduction
• Future mortality rates are published every two years – Until 2004, by the Government Actuary’s
Department (GAD)– Since 2006, by ONS (with GAD consultants)
• The methodology was designed by GAD in the 1990s and runs in Excel
• In 2010, ONS reviewed the current process– Implementation and testing still ongoing
Context
• Mortality rates, estimated 75 years into the future, are a key factor in National Population Projections (others: births and net migration)– Natural change (births – deaths) accounts for 1/3
of total population change
• Population projections used as inputs/control totals for other government projections, such as numbers of school children or pensioners
• Robustness of mortality rates is crucial
Methodology - current
Mortality rate = deaths/pop
Methodology - current
Mortality rate = deaths/popconstrained survivor ratio
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
constrained survivor ratio
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
constrained survivor ratio
extrapolate to age 120
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
constrained survivor ratio
extrapolate to age 120
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
… further adjustments
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
… further adjustments
Methodology - new
• Replace two-stage smoothing process
• Smooth mortality rates surface simultaneously over ages and years
• Estimate improvement rate using existing smoothed years – ie do not estimate T+1
• Requires longer path to T+26 opinions!
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
• Best advice - read “Smoothing and forecasting mortality rates”, Currie et al, 2004!
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
Implementation
• Difficult to understand (and explain) … but easy to implement!
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
Mort2Dsmooth(x=ages,y=years,Z=deaths,offset=log(pop))
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
• Smoothed values = 21st entry in list of R output
Mort2Dsmooth(x=ages,y=years,Z=deaths,offset=log(pop))
Results - testing
Males Females
Sensitivity – adding annual data
Ages 0 - 1031961 – 2003
1961 – 2004
1961 – 2005
1961 – 2006
1961 – 2007
1961 – 2008
1961 – 2009
Ages 0 - 1041961 – 2003
1961 – 2004
1961 – 2005
1961 – 2006
1961 – 2007
1961 – 2008
1961 – 2009
Sensitivity
- adding years of age
1961 – 20090 – 90
0 – 100
0 – 103
1961 – 2005 1961 - 2009 0 – 104 0 – 90
0 – 105 0 – 100
0 – 106 0 – 104
0 – 105
0 – 106
Results - testing
0
1
2
3
4
5
6
102
%
Age
0 20 40 60 8010 30 50 70 90
61-04
61-05
61-06
61-09
Mortality improvement rates by age, 2003/04
61-07
61-08
Results – mortality rates in the base year
Issues• New method does not project rates forward to
base year• Edge effects
Solution• Step back 2 years into the data set
2010 mortality rates = 2007 mortality rates x (1 – 2006-07 improvement rates/100) ^
3
Results – mortality rates in the base year
Year
Age
100
52 yrs
49
48 48 yrs
0
1961………………………………………………….. 2007 2008 2009 2010
Past improvements in smoothed mortality rates, males – old method
Past improvements in smoothed mortality rates, males – new method
Past improvements in smoothed mortality rates, Scotland males – new method
Past improvements in smoothed mortality rates, females – old method
Past improvements in smoothed mortality rates, females – new method
Comparison of projected smooth % changes in death rates by age, UK 2009-10
Males
-1
0
1
2
3
4
5
6
0 10 20 30 40 50 60 70 80 90 100 110
Age
Per
cent
age
redu
ctio
n
2008-basedprojections
Proposed 2010-based projections
Comparison of projected smooth % changes in death rates by age, UK 2009-10
Females
0
1
2
3
4
5
6
0 10 20 30 40 50 60 70 80 90 100 110
Age
2008-basedprojections
Proposed 2010-based projections
Comparison of actual and projected expectation of life at birth
65
70
75
80
85
90
1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050 2055 2060 2065 2070 2075 2080 2085
Old 08
New 08
Old 08
New 08
Females
Males
Summary
• New smoothing method used to produce the 2010-based ‘proposed’ mortality assumptions
• Introduced in the 2010-based consultation with devolved administrations and government departments
What’s next?
• More testing/evaluation:Over-smoothingAdding 2010 dataDerivation of base year rates
• Using R to project mortality rates