Post on 14-Jan-2015
description
1
Black Swan(s) – the Fat Tail Issue -
Guy Lion
December 2009
2
The Black Swan Paradigm
Taleb stated in “The Black Swan” that stock prices are more volatile than the Normal distribution entails. Thus, he relegated to “Mediocristan” all models of modern finance that rely on the Normal distribution. He referred to the intractable volatility of the real world as “Extremistan.”
3
Benoit Mandelbrot
Taleb’s argument is based on Benoit Mandelbrot’s work. The latter, a French mathematician, uncovered that stock price returns are more dispersed than the Normal distribution. He suggested the Cauchy distribution fits the tails of stock returns much better.
4
The Cauchy Distribution
The Cauchy distribution is not well known. Even university classes on statistical distributions may not cover it. And, quantitative software don’t pre-program it. Fortunately, the construction of its cumulative distribution function is not too difficult as it relies on two simple parameters:1) The Median;
2) The difference between the 75th and 25th percentile divided by 2 (called Gamma). The above are the rough equivalent of the Mean and the Standard Deviation for the Normal distribution.
5
Cauchy Cumulative Distribution formula
xo = Median
= (75th percentile – 25th percentile)/2
Excel includes the arctan formula (it calls it ATAN).
6
The Cauchy Conundrum
Even though Mandelbrot uncovered the benefit of the Cauchy distribution (fatter tail) near the onset of the development of modern finance that relied on the Normal distribution, the Cauchy distribution has remained mysterious. Why is that?
7
Distribution Analysis
To investigate the fat tail benefit of the Cauchy distribution, I looked at daily S&P 500 returns since 1950 (15,085 trading days) and compared the overall fit and the left tail fit of the Cauchy distribution vs the Normal distribution and the Student’s t distribution (best fit per Crystal Ball software).
8
An optical illusion
The above frequency distribution curves in % of total daily stock returns do seem similar for all three distributions vs Actuals.
Frequency distributions of daily stock returns
-10%
0%
10%
20%
30%
40%
50%
60%
-10% -8% -6% -4% -2% 0% 2% 4% 6% 8% 10%
Daily stock returns
% o
f to
tal d
aily
sto
ck r
etu
rns
Actual
Student's t
Normal
Cauchy
The Cauchy tails go up because at this point the graph captures all returns beyond + or – 10%
9
Source of Radical Divergence
The Cauchy distribution has fewer observations centered around the Mean. Those are redistributed in the tails. But, it is difficult to see proportions of <2%. As a result, this customary frequency distribution graph is somewhat misleading.
Frequency distributions of daily stock returns
-10%
0%
10%
20%
30%
40%
50%
60%
-10% -8% -6% -4% -2% 0% 2% 4% 6% 8% 10%
Daily stock returns
% o
f to
tal d
aily
sto
ck
retu
rns
Actual
Student's t
Normal
Cauchy
10
Overall Fit
A complete pictureThe table allocates the 15,085 trading day stock returns since January 1, 1950 in various daily stock returns bins ranging from -24% or worse to + 24% or higher. As shown, the fit between the Actual data series and the Student’s t distribution is excellent. This is confirmed by the high Chi Square p value of 0.89 between the two data sets.
The Normal distribution fit is not so good as it misses the 38 worse returns and the 39 best returns.
However, surprisingly the Cauchy distribution fit is also poor as its tails are way too fat.
Dailyreturns Student's Normal Cauchy
Bins Actual t dist. dist. dist.
=<-24% - - - 89 -22% - - - 8 -20% 1 - - 10 -18% - 1 - 12 -16% - - - 15 -14% - - 0 19 -12% - 1 0 26 -10% - - 0 36
-8% 4 3 0 54 -6% 10 8 0 89 -4% 23 27 0 177 -2% 268 265 266 512 0% 6,827 6,855 7,074 6,015 2% 7,629 7,629 7,432 6,926 4% 284 261 313 547 6% 31 27 0 183 8% 5 5 0 92
10% 1 2 - 55 12% 2 - - 37 14% - - - 26 16% - - - 20 18% - - - 15 20% - 1 - 12 22% - - - 10
= >24% - - - 100
15,085 15,085 15,085 15,085
Chi Square p value 0.89 0.00 0.00
11
Assessing Risk Frequency
The table at the top shows that the Cauchy distribution resulted in 134 days (out 15,085) having a negative monthly return of -16% or worse vs only 1 month in the actual data. Thus, the Cauchy distribution overstated this risk frequency by 134 times as shown in the second table.
This table shows the # of months in each return bucket on a cumulative basis. Thus, in the Actual data there were 38 days with a negative return of – 4% or lower.
Cumulative frequency distribution in # of monthsDaily
returns Student's Normal CauchyBins Actual t dist. dist. dist.
-16% 1 1 - 134 -8% 5 5 0 269 -6% 15 13 0 359 -4% 38 40 0 535
Overstating vs understating risk frequencyDaily
returns Student's Normal CauchyBins Actual t dist. dist. dist.
-16% 1.0 1.0 - 134.5 -8% 1.0 1.0 0.0 53.9 -6% 1.0 0.9 0.0 23.9 -4% 1.0 1.1 0.0 14.1
12
Risk Frequency narrativeOverstating vs understating risk frequency
Dailyreturns Student's Normal Cauchy
Bins Actual t dist. dist. dist.
-16% 1.0 1.0 - 134.5 -8% 1.0 1.0 0.0 53.9 -6% 1.0 0.9 0.0 23.9 -4% 1.0 1.1 0.0 14.1
As shown, the Student’s t distribution captures the left tail risk at all levels almost perfectly. At every return cut off points, the number of days captured by this distribution is very close to the actual data (resulting in a multiple close to 1).
Meanwhile, the Normal distribution completely misses out the entire left tail as the -4% return threshold is already over 4 standard deviation away (multiple of 0).
For the Cauchy distribution it is the opposite problem. The tails are way too fat and it overstates the risk frequency at every cut off point by a factor ranging from 14 times to 134 times the actual risk frequency.
13
Risk SeverityRanking
worst Student's Normal Cauchyreturns Date Actual t dist. dist. dist.
1 10/19/1987 -20.5% -19.2% -3.7% -2202.0%2 10/15/2008 -9.0% -12.5% -3.5% -1095.7%3 12/1/2008 -8.9% -9.9% -3.4% -727.7%4 9/29/2008 -8.8% -8.2% -3.3% -547.0%5 10/26/1987 -8.3% -8.0% -3.3% -436.2%6 10/9/2008 -7.6% -8.0% -3.2% -363.6%7 10/27/1997 -6.9% -7.5% -3.2% -311.4%8 8/31/1998 -6.8% -7.3% -3.1% -272.4%9 1/8/1988 -6.8% -7.2% -3.1% -242.0%
10 11/20/2008 -6.7% -6.8% -3.1% -218.0%11 5/28/1962 -6.7% -6.5% -3.0% -198.3%12 9/26/1955 -6.6% -6.4% -3.0% -181.9%13 10/13/1989 -6.1% -6.0% -3.0% -167.8%14 11/19/2008 -6.1% -5.8% -3.0% -155.6%15 10/22/2008 -6.1% -5.7% -3.0% -145.2%16 4/14/2000 -5.8% -5.7% -2.9% -136.1%17 10/7/2008 -5.7% -5.4% -2.9% -128.2%18 6/26/1950 -5.4% -5.3% -2.9% -121.0%19 1/20/2009 -5.3% -5.1% -2.9% -114.7%20 11/5/2008 -5.3% -5.1% -2.9% -109.0%21 11/12/2008 -5.2% -5.1% -2.9% -103.8%22 10/16/1987 -5.2% -5.1% -2.8% -99.1%23 11/6/2008 -5.0% -5.0% -2.8% -94.8%24 9/17/2001 -4.9% -4.9% -2.8% -90.9%25 2/10/2009 -4.9% -4.8% -2.8% -87.2%26 9/11/1986 -4.8% -4.8% -2.8% -83.8%27 9/17/2008 -4.7% -4.6% -2.8% -80.7%28 9/15/2008 -4.7% -4.6% -2.8% -77.9%29 3/2/2009 -4.7% -4.6% -2.8% -75.2%30 2/17/2009 -4.6% -4.3% -2.7% -72.6%31 4/14/1988 -4.4% -4.3% -2.7% -70.3%32 3/12/2001 -4.3% -4.2% -2.7% -68.1%33 4/20/2009 -4.3% -4.1% -2.7% -66.1%34 3/5/2009 -4.3% -4.1% -2.7% -64.1%35 11/30/1987 -4.2% -4.1% -2.7% -62.3%36 11/14/2008 -4.2% -4.1% -2.7% -60.5%37 9/3/2002 -4.2% -4.1% -2.7% -58.9%38 10/2/2008 -4.0% -4.1% -2.7% -57.3%
The table shows the left-tail consisting of the 38 worst daily returns out of 15,085 trading days since January 1st, 1950. It also shows the corresponding 38 worst values for the Student’s t -, Normal - , and Cauchy – distributions. You can see how the Student’s t distribution matches the actual data very well. The Normal distribution misses out all 38 values as its very worst value (-3.7%) is still higher than the actual data’s 38th worst value of 4.0%. On the other hand, the Cauchy distribution value are so much worst than the actual data as to be meaningless.
14
Risk Severity multipleRanking
worst Student's Normal Cauchyreturns Date t dist. dist. dist.
1 10/19/1987 0.9 0.2 107.6 2 10/15/2008 1.4 0.4 121.3 3 12/1/2008 1.1 0.4 81.5 4 9/29/2008 0.9 0.4 62.1 5 10/26/1987 1.0 0.4 52.7 6 10/9/2008 1.0 0.4 47.7 7 10/27/1997 1.1 0.5 45.4 8 8/31/1998 1.1 0.5 40.0 9 1/8/1988 1.1 0.5 35.8
10 11/20/2008 1.0 0.5 32.5 11 5/28/1962 1.0 0.5 29.7 12 9/26/1955 1.0 0.5 27.5 13 10/13/1989 1.0 0.5 27.4 14 11/19/2008 0.9 0.5 25.4 15 10/22/2008 0.9 0.5 23.8 16 4/14/2000 1.0 0.5 23.4 17 10/7/2008 0.9 0.5 22.3 18 6/26/1950 1.0 0.5 22.5 19 1/20/2009 1.0 0.5 21.7 20 11/5/2008 1.0 0.5 20.7 21 11/12/2008 1.0 0.6 20.0 22 10/16/1987 1.0 0.6 19.2 23 11/6/2008 1.0 0.6 18.9 24 9/17/2001 1.0 0.6 18.5 25 2/10/2009 1.0 0.6 17.8 26 9/11/1986 1.0 0.6 17.4 27 9/17/2008 1.0 0.6 17.1 28 9/15/2008 1.0 0.6 16.5 29 3/2/2009 1.0 0.6 16.1 30 2/17/2009 0.9 0.6 15.9 31 4/14/1988 1.0 0.6 16.1 32 3/12/2001 1.0 0.6 15.8 33 4/20/2009 1.0 0.6 15.4 34 3/5/2009 1.0 0.6 15.1 35 11/30/1987 1.0 0.6 14.9 36 11/14/2008 1.0 0.6 14.5 37 9/3/2002 1.0 0.6 14.2 38 10/2/2008 1.0 0.7 14.2
This table divides the distribution return vs actual return. Doing so, shows that the Cauchy distribution overstates the worst return by a multiple of 107.6 times calculated as follows:
-2202%/-20.5% = 107.6 times
Meanwhile, the Normal distribution understates this return by 80%:
-3.7%/-20.5% = 0.2
While the Normal distribution pretty much misses out this 38 worst observation left-tail risk and the Cauchy distribution overstates it by a factor of 14 to 121 times, the Student’s t distribution gets it just about right through the entire range (from very worst to the 38th worst return).
15
How often would a VAR model blow up?
Let’s say you use a Value-at-risk model as a risk management tool. You use a Normal distribution because it is a lot more transparent than any other distributions. And after all, it works 99.75% of the time [(15,085 – 38)/15,085]. How often would such a VAR model have blown up since 1950? And, when…
As shown on the table, this VAR model would have blown up in 14 different years a total of 38 times. But, it would have blown up 15 times in 2008 alone!
# of daily returns missed by Normal dist.
Year #
1950 11955 11962 11986 11987 41988 21989 11997 11998 12000 12001 22002 12008 152009 6
38
16
99.75% of the time is not good enough!
As reviewed a VAR model relying on the Normal distribution that works 99.75% of the time would have still blown up in 14 out of the past 59 years (or in 24% of the years). And, it would have also blown up 15 times in 2008 alone.
The blown-up proofing remedy: use the Student’s t distribution.