Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts
description
Transcript of Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts
![Page 1: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/1.jpg)
Skill Variability Diagnosis for Discriminating Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal ForecastsUse of CPC Long-Lead Seasonal Forecasts
Bob Livezey and Marina TimofeyevaNOAA/NWS/OCWWS/Climate Services Division
Climate Prediction Applications Science WorkshopMarch 9-11, 2004Tallahassee, FL
![Page 2: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/2.jpg)
OutlineOutline
• IntroductionIntroduction• CPC Skill Graphs and Some Other CPC Skill Graphs and Some Other
StratificationsStratifications• ResultsResults• Conclusions and LessonsConclusions and Lessons
![Page 3: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/3.jpg)
IntroductionIntroduction
• Users should only care about the performance of forecasts that can potentially benefit their decision process
• Livezey (1990): There are non-random subsets of seasonal forecasts that were skillful enough to be useful
• After 1997-98 a common presumption was
that forecasts are generally skillful enough to be useful
Seasonal Temperature Forecast Skill Seasonal Temperature Forecast Skill 1960s to 80s1960s to 80s
All Seasons 8.3
Winter 12.6Spring 8.6Summer 9.3Fall 2.8
![Page 4: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/4.jpg)
Introduction (Cont.)Introduction (Cont.)
• This talk will make the point :
– That was made by Livezey (1990)– That there are many non-random subsets of
forecasts that do not have useful skill – That it is critical for this information to be shared
with potential users– That skill analyses with different stratifications are
highly informative while CPC’s web displays are not
![Page 5: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/5.jpg)
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
![Page 6: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/6.jpg)
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
Stratification by Lead and Stratification by Lead and Seasons: TemperatureSeasons: Temperature
Heidke Skill Scores for All Years
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Seasons Cold Seasons WarmSeasons
![Page 7: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/7.jpg)
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Temp.ENSO vs Other Years: Temp.
Heidke Scores for Cold Seasons(DJF, JFM, FMA)
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
![Page 8: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/8.jpg)
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Temp.ENSO vs Other Years: Temp.
Heidke Scores for Warm Seasons(MAM - NDJ)
-10
0
10
20
30
40
50
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
![Page 9: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/9.jpg)
Displays and StratificationsDisplays and Stratifications
• CPC Seasonal Forecasts– For 3-equally probable temperature and precipitation classes at 102
Climate Divisions– Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month
leads
• Skill Measure: Modified Heidke Skill Score of Categorized Forecasts
• Displays and Stratifications– CPC: Summed over all forecasts for each lead and displayed with
times series for this lead– Here:
• Summed over all forecasts for each lead and all leads displayed together• Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM
to NDJ)• Stratified further by strong ENSO years vs. other years• Stratified by region
![Page 10: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/10.jpg)
![Page 11: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/11.jpg)
Stratification by Lead and Stratification by Lead and Regions: Temp.Regions: Temp.
Heidke Skill Scores for All Seasons / All Years
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Heid
ke S
S .
Western States Eastern States Allstates
![Page 12: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/12.jpg)
ResultsResults
• Seasonal Temperature:– Useable skill confined to strong ENSO years and mainly
at short to medium leads– Otherwise skill is dominantly level with lead (derived
from biased climatologies, ie long-term trend)– Best forecasts are for strong-ENSO cold seasons at
very short leads– Worst forecasts are for cold seasons at longer leads for
strong ENSOs and at very-short leads for other years– Skill is substantially higher than average in the West,
and substantially lower in the East– Short-lead forecasts are better now than for the 1960s-
80s, ~13 vs ~8 overall, ~20 vs ~13 for the winter
![Page 13: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/13.jpg)
Stratification by Lead and Seasons: Stratification by Lead and Seasons: PrecipitationPrecipitation
Heidke Skill Scores for All Years
-10
-5
0
5
10
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Seasons Cold Seasons WarmSeasons
![Page 14: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/14.jpg)
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Precip.ENSO vs Other Years: Precip.
Heidke Scores for Cold Seasons(DJF, JFM, FMA)
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
.
All Years 1997-2000 Other Years
![Page 15: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/15.jpg)
Further Stratification by Strong-Further Stratification by Strong-ENSO vs Other Years: Precip.ENSO vs Other Years: Precip.
Heidke Scores for Warm Seasons(MAM - NDJ)
-10
-5
0
5
10
15
20
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Hei
dke
SS
All Years 1997-2000 Other Years
![Page 16: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/16.jpg)
![Page 17: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/17.jpg)
Stratification by Lead and Stratification by Lead and Regions: Precip.Regions: Precip.
Heidke Skill Scores for All Seasons / All Years
-2
-1
0
1
2
3
4
0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5
Lead (month)
Heid
ke S
S .
Northern States Southern States Allstates
![Page 18: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/18.jpg)
ResultsResults
• Seasonal Precipitation:
– Barely useable skill entirely confined to strong ENSO years in short to medium leads
– Otherwise skill is either negative or statistically indistinguishable from zero
– Best forecasts are for strong-ENSO cold seasons at short leads
– Skill is a little higher than average in the South, and a little lower in the North
– Short-lead forecasts overall seem to be no better now than for the 1960s-80s
![Page 19: Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts](https://reader035.fdocuments.in/reader035/viewer/2022070404/56813ac8550346895da2df9c/html5/thumbnails/19.jpg)
Conclusions and LessonsConclusions and Lessons
• There are non-random subsets of seasonal forecasts that are skillful enough to be useful
• There are many non-random subsets of forecasts that do not have useful skill (should we be issuing them?)
• It is critical for this information to be shared with potential users
• Skill analyses with different stratifications are highly informative while CPC’s web displays are not
• Skills mainly reflect ENSO and trend signals• Temperature forecasts are getting better