1

Modeling and Calibration Errors in

Measures of Portfolio Credit Risk

Nikola Tarashev and Haibin Zhu

Bank for International Settlements

April 2007

The views expressed in this paper need not represent those of the BIS.

2

Motivation ASRF model: a well-known model of portfolio credit risk Main reason for the model’s popularity: “portfolio invariance” of capital

• Assumption 1: perfect granularity of the portfolio

• Assumption 2: single common factor of credit risk

Does ASRF allow for “bottom-up” calculation of economic capital ?

Not really: estimating ρi requires a global approach !!!

To take full advantage of portfolio invariance off-the-shelf values for ρi:

Pillar 1 of Basel II: ρi = ρIRB( PDi )

ASRF-based capital measures are subject to:

• misspecification errors: ie, violated assumptions of the model

• calibration errors: eg, off-the-shelf values for ρi.

3

Related literature (BCBS WP No 15)

Misspecification of the ASRF model:

• Granularity assumption:• Martin and Wilde (2002), Vasicek (2002)

Emmer and Tasche (2003), Gordy and Luetkebohmert (2006)

• Sector concentration and the number of common factors• Pykhtin (2004), Duellmann (2006), Garcia Cespedes et al (2006)

Duellmann and Masschelein (2006)

• Both assumptions:

• Heitfield et al (2006), Duellman et al (2006)

Flawed calibration:

• Loeffler (2003), Morinaga and Shiina (2005)

4

This paper

Decomposes the wedge between target and shortcut capital measures.

Exhaustive and non-overlapping components due to:

Misspecification of the ASRF model

• Multifactor effect

• Granularity effect Flawed calibration the ASRF model

• Correlation dispersion effect

• Correlation level effect

• Plausible estimation errors

Takes seriously the overall distribution of risk factors Gaussian versus fatter-tailed distributions

The paper does not:

(i) Discuss calibration of PDs or LGDs; (ii) address time variation in risk parameters

5

Main results

Applying ASRF model

Large deviations from target capital for realistic bank portfolios

The main drivers of these deviations are calibration errors

• noise in the estimated value of ρi

• wrong distributional assumptions

Misspecification of the ASRF model has a much smaller impact

• exception: granularity effect in small portfolios

6

Roadmap

The ASRF model (overview)

Alternative sources of error in capital measures (intuition)

Empirical methodology

Findings

7

The ASRF model (an overview)

PDi, LGDi, ρi and weights wi are all known

Assets (driven by a single common factor) drive defaults

M ~ F (0,1), Z ~ G (0,1), V ~ H (0,1): weak restrictions

Perfectly fine granularity:

Idiosyncratic risk is diversified away (low wi & many exposures)

Pairwise correlations: ρi ρk

8

To attain solvency with probability (1- α):

The ASRF model (implications)

Portfolio invariance

If V, M, Z are all normal and α = 0.001:

which underpins the IRB approach of Basel II

.999

9

Errors in calculated capital charges

The granularity effect (specification error 1)

• Stylized homogeneous portfolio: PD=1%, LGD=45%, ρ2 = 10%

• Granularity ASRF capital undershoots target capital

target capital

ASRF capital

10

The multi-factor effect (specification error 2)

• Stylized homogeneous portfolio: PD=1%, LGD=45%• Two sectors, weight = ω. Intra-sector correlation = 0.2; inter-sector = 0

• Owing to its single-CF structure, ASRF model undershoots target

Errors in calculated capital charges (cont’d)

11

Correlation level effect (calibration error 1)

• Stylized homogeneous portfolio: PD=1%, LGD=45%

• Higher correlation raises the capital measure

Errors in calculated capital charges (cont’d)

12

Correlation dispersion effect (calibration error 2)

• Start with a homogeneous portfolio: PD=1%, LGD=45%

ρ varies within

• Then, let PDs vary too, within [0.5%,1.5%]

Errors in calculated capital charges (cont’d)

13

Methodology. Decomposing the gap

Select a portfolio:

• Realistic distribution of exposures across industrial sectors

• representative portfolio of large US banks, Heitfield et al. (2006)

• Size

• Large: 1000 exposures (homogeneous weights)

• Small: 200 exposures (homogeneous weights)

For each portfolio, quantify the difference between two extremes:

• Target capital

• Shortcut capital, implied by off-the-shelf calibration of ASRF model

Three additional capital calculations bring up the

(i) multifactor (ii) granularity

(iii) correlation-level (iv) correlation-dispersion effects

14

Target capitalN firms, (σ) + Monte Carlo simulations

Shortcut capitalN = , + ASRF model

Methodology: decomposing the four effects

N firms, R() (1-factor best fit) + copula

Multi-factor effect

For all measures:homogeneous portfolios,

the same {PDi}iN and LGD,

Gaussian distributions

15

R() is the “one-factor approximation” of (σ)

Loadings (i): allowed to differ across exposures

Approximation matches well average correlation Approximation could miss correlation dispersion

Fitting a single-factor structure

16

Target capitalN firms, (σ) + Monte Carlo simulations

N firms, R() (one-factor best fit) + copula

Shortcut capitalN = , + ASRF model

Multi-factor effect

Methodology: decomposing the four effects

N = , R() + ASRF model

Granularity effect

N = , average + ASRF model

Correlation dispersion effectFor a

ll m

easu

res:

hom

ogeneo

us p

ortfo

lios,

the

sam

e {P

D i} iN a

nd LG

D,

Gau

ssia

n dis

tribu

tions

Correlation level effect

17

Data

10,891 non-financial firms worldwide:

• 40 industrial sectors

• mostly unrated firms

Risk parameters:

• from Moody’s KMV,

• for July 2006

• two sets of mutually consistent estimates:

• EDF: 1-year physical PD, at exposure level

• Global Correlation (GCORR) asset return correlations

Estimated based on a multi-factor loading structure

LGD = 45%

18

Match sectoral distribution of the representative portfolio of US wholesale banks. Heitfield et al. (2006)

Two sizes: large (1000 exposures) and small (200 exposures)

3000 simulations (for each size): to average out sampling errors

mean (%)

Large Small

Average PD 2.42 2.28

Std dev of PD 5.16 5.05

Median PD 0.26 0.24

Average correlation 9.78 10.49

Std dev of loadings 9.33 10.54

Correl.(PD, loading) -20.0 -19.8

Simulated Portfolios

19

Findings

Dissecting deviations from target capital:

Mean (%) Add-on (%) 95% interval (%)

Target capital

Multi-factor effect

Granularity effect

Correlation dispersion effect

Correlation level effect

Shortcut capital (correlation = 12%)

Total difference

2.95-0.03

-0.11

0.35

0.55

3.71

0.76

-1.02

-3.73

11.86

18.64

25.76

[2.64, 3.27]

[-0.09, 0]

[-0.14, -0.09]

[0.27, 0.43]

[0.44, 0.66]

[3.37, 4.06]

Correlation level effect if:

correlation = 6%

correlation = 18%

correlation = 24%

-0.96

2.01

3.47

-32.54

68.14

117.63

[-1.11, -0.83]

[1.84, 2.18]

[3.23, 3.72]

20

For small portfolios: similar results, except for the granularity effect

Large Mean (%)

Small Mean (%)

Target capital

Multi-factor effect

Granularity effect

Correlation dispersion effect

Correlation level effect

Shortcut capital

Correlation level effect if

correlation = 6%

correlation = 18%

correlation = 24%

2.95

-0.03

-0.11

0.35

0.55

3.71

-0.96

2.01

3.47

3.35

-0.04

-0.53

0.38

0.36

3.52

-1.07

1.76

3.15

add-on: large -3.73%;small -15.8%

Findings, continued

21

Findings, continued Discrepancy: shortcut minus target capital

Explaining the variation of the discrepancy across simulated portfolios

22

Delving into calibration errors

Small-sample estimation errors in calibrated correlations

Conduct the following exercise

• Design true model: 2 = 9.78%, PD = 1%, Gaussian variables, N equal exposures

• Draw from the true model: T periods of asset returns

• Adopt the point of view of a user (has data, estimates parameters)

• Construct sample correlation matrix (PD, etc: known)

• Fit a one-factor model and calculate ASRF-implied capital

Repeat 1000 times for each (N,T)

23

Small-sample estimation errors are significant

Meaningful reduction of errors requires unrealistic sample sizes

Delving into calibration errors, contd.

24

Estimation errors affect substantially capital charges

• Bias

• Noise

Delving into calibration errors, contd.

25

The importance of distributional assumptions

Stylized fact: fat tails of asset returns

Gaussian (thin tails) assumption too low capital measures

Quantify the bias:

• Student t distributions

• General ASRF formula:

26

Alternative distributions

Gaussian assumption negative bias in measured capital

27

Conclusions

Large deviations from target capital for realistic portfolios

The main drivers of the deviations:

• calibration errors

• not model misspecification

Challenges for risk managers and supervisors