Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major,...
-
Upload
kellie-cannon -
Category
Documents
-
view
216 -
download
0
Transcript of Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major,...
![Page 1: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/1.jpg)
Taking Uncertainty Into Taking Uncertainty Into Account:Account:
Bias Issues Arising from Uncertainty in Risk Bias Issues Arising from Uncertainty in Risk ModelsModels
John A. Major, ASAJohn A. Major, ASAGuy Carpenter & Company, Inc.Guy Carpenter & Company, Inc.
![Page 2: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/2.jpg)
N=20 observationsN=20 observations T = sample mean; T = sample mean; =1 true mean=1 true mean MLE EP curve:MLE EP curve:
q-exceedance point (PML, VaR)q-exceedance point (PML, VaR)
XX.01.01 = 4.605 actual = 4.605 actual
Example: Exponential Example: Exponential DistributionDistribution
TxxQ exp)(ˆ
)ln(ˆ qTX q
![Page 3: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/3.jpg)
Sampling Distribution of TSampling Distribution of T
![Page 4: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/4.jpg)
Estimated PDFsEstimated PDFs
![Page 5: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/5.jpg)
Client QuestionsClient Questions
What is the 1 in 100-yr PML (1% VaR)?What is the 1 in 100-yr PML (1% VaR)? What is probability of exceeding 4.605?What is probability of exceeding 4.605? Can you give me an EP curve to answer Can you give me an EP curve to answer
these and similar questions?these and similar questions? Does sampling error affect the answer?Does sampling error affect the answer? Can I get unbiased answers?Can I get unbiased answers?
![Page 6: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/6.jpg)
3 Kinds of Bias3 Kinds of Bias
““dollar” or X-bias:dollar” or X-bias: the average of PML dollar estimatesthe average of PML dollar estimates
““probabilistic” or P-bias:probabilistic” or P-bias: the average the average truetrue exceedance probability of exceedance probability of estimatedestimated PML points PML points
““exceedance” or Q-bias:exceedance” or Q-bias: the average estimated exceedance the average estimated exceedance
probabilityprobability
qq vsXE Xˆ
qvsXQE qˆ
qvsQE qXˆ
![Page 7: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/7.jpg)
Exponential MLE is X-Exponential MLE is X-unbiasedunbiased
TE
qq qqTEXE X)ln()ln(ˆ
![Page 8: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/8.jpg)
Exponential MLE is X-Exponential MLE is X-unbiasedunbiased
![Page 9: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/9.jpg)
for small qfor small q Expected actual risk is greater than Expected actual risk is greater than
nominalnominal Uncertainty increases risk!Uncertainty increases risk!
Exponential MLE is P-biasedExponential MLE is P-biased
qXQE q ˆ
![Page 10: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/10.jpg)
Exponential MLE is P-biasedExponential MLE is P-biased
![Page 11: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/11.jpg)
Correcting for P-biasCorrecting for P-bias
Predictive distributionPredictive distribution ““Prediction interval” in regressionPrediction interval” in regression
Mix randomness and uncertaintyMix randomness and uncertainty integrate model pdf over parameter integrate model pdf over parameter
distributiondistribution Exponential model:Exponential model: Predictive result:Predictive result:
TxxQ exp)(
n
nT
xxQ
1)(
![Page 12: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/12.jpg)
Predictive vs. Model DensityPredictive vs. Model Density
![Page 13: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/13.jpg)
Which to use?Which to use? MLE curve is X-unbiasedMLE curve is X-unbiased
no uncertainty adjustment, but...no uncertainty adjustment, but... on average, gets right $ answeron average, gets right $ answer
Predictive curve is P-unbiasedPredictive curve is P-unbiased ““takes uncertainty into account” and...takes uncertainty into account” and... on average, reflects true exceedance pron average, reflects true exceedance pr
But they disagree...But they disagree... and it gets worse...and it gets worse...
![Page 14: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/14.jpg)
for small qfor small q Expected estimated risk is greater than Expected estimated risk is greater than
the true risk (at the specified threshold)the true risk (at the specified threshold) Uncertainty now causes risk to be Uncertainty now causes risk to be overoverstated!stated!
Exponential MLE is Q-biasedExponential MLE is Q-biased
qQE q Xˆ
![Page 15: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/15.jpg)
Exponential MLE is Q-biasedExponential MLE is Q-biased
![Page 16: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/16.jpg)
Correcting for Q-biasCorrecting for Q-bias
Minimum Variance Unbiased EstimatorMinimum Variance Unbiased Estimator standard procedure in classical statisticsstandard procedure in classical statistics
Rao-Blackwell TheoremRao-Blackwell Theorem Expectation of unbiased estimator, Expectation of unbiased estimator,
conditional on sufficient statisticconditional on sufficient statistic Exponential model:Exponential model: MVUE result:MVUE result:
TxxQ exp)(
1
1)(
n
nT
xxQ
![Page 17: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/17.jpg)
MVUE vs. Model DensityMVUE vs. Model Density
![Page 18: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/18.jpg)
ParadoxParadox
Say we get an estimated T=1 (correct)Say we get an estimated T=1 (correct) MLE says XMLE says X.01.01=4.605, Pr{X>4.605}=1%=4.605, Pr{X>4.605}=1%
Predictive: XPredictive: X.01.01=5.179 is p-unbiased=5.179 is p-unbiased risk is greater than MLE answer because risk is greater than MLE answer because
impact of uncertaintyimpact of uncertainty MVUE: Pr{X>4.605}=.69% is q-unbiasedMVUE: Pr{X>4.605}=.69% is q-unbiased
risk is less because MLE tends to overstate risk is less because MLE tends to overstate exceedance probabilityexceedance probability
![Page 19: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/19.jpg)
How the Paradox ArisesHow the Paradox Arises
![Page 20: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/20.jpg)
ConclusionsConclusions
Uncertainty induces bias in estimatorsUncertainty induces bias in estimators Biases operate in different directionsBiases operate in different directions
depends on the question being askeddepends on the question being asked There is no monolithic “fix” for taking There is no monolithic “fix” for taking
uncertainty into accountuncertainty into account Predictive distribution fixes p-bias, Predictive distribution fixes p-bias, while making q-bias worsewhile making q-bias worse
![Page 21: Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649db35503460f94aa2f4c/html5/thumbnails/21.jpg)
RecommendationsRecommendations
First: Show modal estimates (MLE etc.)First: Show modal estimates (MLE etc.) Second: Show effect of uncertaintySecond: Show effect of uncertainty
Keep uncertainty distinct from randomnessKeep uncertainty distinct from randomness Sensitivity testing w.r.t. parameters Sensitivity testing w.r.t. parameters Confidence intervals on estimatorsConfidence intervals on estimators
Third: Adjust for bias only as necessaryThird: Adjust for bias only as necessary Carefully attend to the question askedCarefully attend to the question asked Advise that bias adjustment is equivocalAdvise that bias adjustment is equivocal