Subjective Probability Forecasts at the NWS Storm Prediction Center
Probability Forecasts and Verification
-
Upload
kiayada-hayes -
Category
Documents
-
view
11 -
download
0
description
Transcript of Probability Forecasts and Verification
![Page 1: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/1.jpg)
Probability Forecasts and Verification
![Page 2: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/2.jpg)
Uncertainty
• Forecasts have uncertainty that should be taken into account:– Observational Uncertainty (initial conditions)
• Instrument errors, bias in frequency of measurements, representativeness, reporting errors, random errors, precision errors, conversion errors …
– Model Uncertainty• Insufficient spatial resolution, truncation errors,
parameterization errors, coding errors, initialization errors …
![Page 3: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/3.jpg)
Deterministic and Probabilistic Forecasts
• Deterministic Forecast: Specific Outcome is Predicted
• Probabilistic Forecast: Range of Possible Outcomes Accompanied by Confidence Estimate
![Page 4: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/4.jpg)
Ex: Tercile Probabilities
![Page 5: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/5.jpg)
Probability Forecast
![Page 6: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/6.jpg)
![Page 7: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/7.jpg)
![Page 8: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/8.jpg)
![Page 9: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/9.jpg)
![Page 10: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/10.jpg)
Probabilistic Verification
• Requires Several Forecast Cases – Cannot Verify a Single Probabilistic Forecast
• Compare Forecast Distribution to a Distribution of Observations
• Rank Probability Score (RPS)– Rank Probability Skill Score (RPSS)
• Relative Operating Characteristics (ROC)• Reliability (Not Really Verification)
![Page 11: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/11.jpg)
Rank Probability Score
![Page 12: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/12.jpg)
![Page 13: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/13.jpg)
Rank Probability Skill Score
![Page 14: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/14.jpg)
July 1 startDJF prec forecastRPSS
NMME
CFSv2
![Page 15: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/15.jpg)
Relative Operating Characteristics
![Page 16: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/16.jpg)
Relative Operating Characteristics
Hit Rate = hits/(hits+misses)
False Alarm Rate = false alarms/(hits+false alarms)
![Page 17: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/17.jpg)
Relative Operating Characteristics
For Each Threshold Build aContingency Table SummingOver All Forecasts
![Page 18: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/18.jpg)
Relative Operating Characteristics
![Page 19: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/19.jpg)
![Page 20: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/20.jpg)
Reliability
• Prediction System is Reliable if Predicted Probabilities Agree with Observed Frequencies– Ex: Climatological Probability has Perfect
Reliability (33%, 33%, 33%)
• Reliability Diagram Measure whether the Prediction System is Reliable, Overconfident or Underconfident
• Reliability Diagram also Shows Resolution and Sharpness
![Page 21: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/21.jpg)
Reliability Diagram
![Page 22: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/22.jpg)
Reliability Diagram
![Page 23: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/23.jpg)
![Page 24: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/24.jpg)
![Page 25: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/25.jpg)
![Page 26: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/26.jpg)
![Page 27: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/27.jpg)
![Page 28: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/28.jpg)
T2m October Starts JFM Verification
![Page 29: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/29.jpg)
T2m October Starts JFM Verification
![Page 30: Probability Forecasts and Verification](https://reader030.fdocuments.in/reader030/viewer/2022032607/568130ae550346895d96bc73/html5/thumbnails/30.jpg)
Probabilistic Verification
• Requires Several Forecast Cases – Cannot Verify a Single Probabilistic Forecast
• Compare Forecast Distribution to a Distribution of Observations
• Rank Probability Score (RPS)– Rank Probability Skill Score (RPSS)
• Relative Operating Characteristics (ROC)• Reliability (Not Really Verification)