2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

70
Bayesian Methods for Modern Statistical Analysis Milovan Krnjaji´ c School of Mathematics, Statistics & Applied Mathematics National University of Ireland, Galway Whitaker Institute for Innovation and Societal Change 26-March-2013

description

Dr Milovan Krnjajic, School of Mathematics, NUI Galway, presented this inaugural workshop on Modern Statistical Analysis using Bayesian Methods as part of the launch of the Social Sciences Computing Hub at the Whitaker Institute, NUI Galway on 26th March 2013.

Transcript of 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Page 1: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian Methods forModern Statistical Analysis

Milovan Krnjajic

School of Mathematics, Statistics & Applied MathematicsNational University of Ireland, Galway

Whitaker Institute for Innovation and Societal Change26-March-2013

Page 2: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Contents

1 Classical approach to statistical analysis

2 Features of Bayesian approach

3 Bayesian MCMC computation engine

4 An application of Bayesian modelling

5 Bayesian hierarchical models

6 A model for analysis of financial risk

Page 3: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Statistics and uncertainty

Statistics studies uncertainty (quantify, interpret and present, use)

Uncertainty is unavoidable in everyday life, science, economy → importanceof statistics

Information uncertain or incomplete, causes unknown, events random: growth/fall

of a company stock, or an index; financial risk in company merge; portfolio risk;

size of the insurance premium; dynamics of electric power consuption; direction

and intensity of the spread of an epidemics; demand for products and how would

new product lines change it;

Statistical analysis: quantification of uncertainty in order to learn (gaininsight in) a problem (phenomenon) of interest; this learning contributes todecreasing of the uncertainty

Milovan Krnjajic (NUIG) 1 / 60

Page 4: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Statistics and uncertainty

Probability Theory as a formal apparatus for systematic uncertaintyquantification

Statistical analysis comprises:I Gathering data samples and describing data propertiesI Specifying a model (a formal specification of unknown parameters) and

combine it with the data in order to derive inferences about theparameters

I Inference about the nature of the data generating process (answersquestions about the causes)

I generalizing sample properties to statements about populationI make predictions (with estimates of uncertainty bands)I Decision making based on the inference and prediction(s) regarding the

available actions with the goal of choosing the optimal ones.

Milovan Krnjajic (NUIG) 2 / 60

Page 5: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Classical statistical analysis

Also called ”frequentist” since it adopts the frequentist interpretationof probability (as a relative frequency of event occurrences in a longsequence of repeated observations).

Model parameters are unknown fixed quantities (constants).

Probabilistic statements about unknown parameters make no sense(parameters are not repeatable in any way).

Randomness in the stats model is assumed for the data set generatedby the sampling process, which in turn may be (imagined to be)repeated indefinitely.

Data has a sampling probability distribution p(y |θ). Data exibitrandom variability, which refers to aleatory uncertainty.

Parameters are not random, but are unknown (uncertain) as a resultof a lack of information (knowledge), which is epistemic uncertainty(not expressible by frequentist probability)

Milovan Krnjajic (NUIG) 3 / 60

Page 6: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Classical statisticians: K. Pearson, R. Fisher, J. Neyman

Karl Pearson (1857 – 1936 ) Ronald Fisher (1890 – 1961) Jerzy Neyman (1894 – 1981 )

Milovan Krnjajic (NUIG) 4 / 60

Page 7: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Classical statistics: Inference

Neyman-Pearson hypothesis testing (HT)

HT error probabilities, power of test, most powerful tests, unbiasedand efficient tests.

Fisher’s maximum likelihood based point estimation (ML)

P-values, Likelihood ratio tests

Inferential results depend on the Central Limit Theorem and largesample sizes

Frequentist interpretation of probability makes repeatability a centralconcept in model development and statistical inference. In (many)cases where there is no repeatability at all it needs to be imagined inorder to interpret results probabilisticaly.

Milovan Krnjajic (NUIG) 5 / 60

Page 8: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Problem: Interpretation of confidence interval (CI)

Assume Xi ∼ N(µ, σ2), with sample mean X = (X1 + X2 + ...+ Xn)/n.

Then X ∼ N(µ, σ2/n) so that

P(µ− 1.96σ/√

n < X < µ+ 1.96σ/√

n) = 0.95

Therefore

P(X − 1.96σ/√

n < µ < X + 1.96σ/√

n) = 0.95

95% confidence interval (CI) is(X ± 1.96σ/

√n)

The above probability statements are good only before the sample is taken.Once we know that X = x it makes no sense to compare probabilisticallytwo constants, unknown (yet constant) µ and x .

Is P(µ ∈ 95%CI) = 0.95? The answer is NO!

As an unknown but fixed, the true µ either is or is not in the CI. If weimagined repeated sampling, then µ would be in about 95% of the CI-s.

Milovan Krnjajic (NUIG) 6 / 60

Page 9: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Problem: Interpretation of confidence interval (CI)

Figure: Is P(µ ∈ 95%CI) = 0.95? The answer is NO! Being an unknown but fixed quantity, the true mean either is or isnot in the CI. However, if we imagine repeated sampling, then µ would be in the CI about 95% of time.

Milovan Krnjajic (NUIG) 7 / 60

Page 10: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Problem: Hypothesis testing, p-values

Figure: H0 vs. H1, α fixed in advance, β depends on the sample size

Milovan Krnjajic (NUIG) 8 / 60

Page 11: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Problem: Hypothesis testing, p-values

Hypothesis testing rests on a decision procedure not on a probabilitystatement.

Testing procedure indirect and convoluted.

Rejecting H0 at level α = 5% does not mean that there is only 5%probability that H0 is correct.

Rejecting H0 at level α = 5% does not mean that only 5% of data(collected repeatedly) would come from H0

P-value is not the probability that the H0 is correct.P-value = the probability of observing data points such as the oneobserved or more extreme, assuming that the H0 is correct. What extrememeans depends on the formulation of the null and alternative hypotheses

Milovan Krnjajic (NUIG) 9 / 60

Page 12: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Violation of the Likelihood Principle

Likelihood Principle (LP): Inference about θ should depend on thesample but not on the many samples which might have been obtained.

Freq. based analysis violates the LP in hypothesis testing and ininterpretation of the confidence intervals.

Example: Sample of size n with k successes, where P( success ) = θis unknown.

I (a) fixed n, sample exactly n points;P(X = k) =

(nk

)θk(1− θ)n−k , hence unbiased θ = k/n.

I (b) fixed k, sample as many points as necessary to obtain k successes.P(X = k) =

(n−1k−1

)θk(1− θ)n−k but here the unbiased θ = k/(n − 1).

Frequentist claims two different sampling procedures (hence twodifferent distributions and estimators).

Yet the type of the sampling procedure does not convay anyinformation on θ

Problem: what if neither n nor k were fixed?

Milovan Krnjajic (NUIG) 10 / 60

Page 13: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Classical analysis: a summary of problems

Freq. interp. of probability imposes an assumption of repeatedestimation procedures to derive prob. statements about the estimates.

These are not direct prob. statements about the probable values ofjust obtained estimate (e.g. a CI for mean or variance), but ”proxy”statements based on imagined sampling which is never done.

Hypothesis testing based on a convoluted decision procedure whichconsiders the scenario of indefinite repetitions of sampling

CI-s, p-values and results of hypotheses testing often misinterpretedby non-specialists.

Likelihood principle often violated or not observed.

Conceptual difficulty in dealing with inherently non-repeatable eventsand data sets which are not random samples: cross-national data ineconomics, or political science using national data banks of the OECDglobal repository. Analysis of the main factors leading to civil wars inXX century (say) based on a comprehensive data account. What isthe meaning of ”statistical significance” here?

Milovan Krnjajic (NUIG) 11 / 60

Page 14: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Classical analysis: Hallmarks

Interpretation of probability as long term relative frequency

Model parameters unknown fixed quantities whereas a data sample isconsidered random

Inference based on the assumption that the sampling procedure isrepeated indefinitely

Distributional properties of estimators based on asymptotics (large n)and an appeal to CLT (assumptions of normality)

Taught to undergraduates at an introductory level

Wide variety of powerful (industrial grade) software packages such asSAS, SPSS, STATA, SPLUS, Minitab

Milovan Krnjajic (NUIG) 12 / 60

Page 15: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesians: T. Bayes, P.S. Laplace, B. de Finetti

Thomas Bayes (1701 – 1761 ) P.S. Laplace (1749 – 1827) B. de Finetti (1906 – 1985 )

Milovan Krnjajic (NUIG) 13 / 60

Page 16: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Thomas Bayes (1702–1761)

T. Bayes(1763) An essay towards solving a problem in the doctrine ofchances. Phil Trans Roy. Soc. 53370-418

Interested in causal relationships; Observing a phenomenon we askabout the cause(s), for example observing symptoms we want todiagnose the disease, that is, find the causes.

Bayes was the first to consider causal connections in terms ofconditional (inverse) probability

Conditional probability: P(A | B) = P(A i B)/P(B), where A and Bare true or false statements

P(A = effect | B = cause), easier to consider thanP(B = cause | effect), (for example, A = symptom, B = illness)

Inversion of conditioning:

Bayes theorem : P(B | A) =P(B) P(A | B)

P(A)

Milovan Krnjajic (NUIG) 14 / 60

Page 17: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit.

9 99

Negat.

1 891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 18: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit.

9 99

Negat.

1 891

10

990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 19: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit.

9 99

Negat.

1 891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 20: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit.

9 99

Negat.

1

891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 21: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit.

9

99

Negat.

1

891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 22: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit. 9 99

Negat.

1

891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 23: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit. 9 99

Negat. 1 891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 24: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit. 9 99

Negat. 1 891

10 990

Answer: Pr(Ill | Posit.) =

9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 25: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: An example

Problem: A random sample of 1,000 persons selected for a medicaltest to identify those having illness XYZ

Test characteristics:I Pr( Posit. | Ill ) = 90%I Pr( Negat. | Healthy ) = 90%I Pr( Illness ) = 1%

Question: Pr( Illness. | Posit. ) = ?

Ill Healthy

Posit. 9 99

Negat. 1 891

10 990

Answer: Pr(Ill | Posit.) = 9/(9 + 99) = 8.3%

Milovan Krnjajic (NUIG) 15 / 60

Page 26: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Conditional probability: Bayes theorem

The medical test example:

P(B | +) =P(+ | B)P(B)

P(+ | B)P(B) + P(+ | Z)P(Z)=

(0.9)(0.01)

(0.9)(0.01) + (0.1)(0.99)= 8.3%

Meaning of the theorem:

P(unknown | data) =P(unknown) P( unknown | data)

P(data)

Theorem holds for real numbers, where unknown = θ, and knowndata are in the sample y = (y1, y2, ..., yn):

p(θ | y) =p(θ) p(y | θ)

p(y),

where p(θ | y) i p(y | θ) conditional probability densities and p(θ) ip(y) marginal densities for θ i y

Milovan Krnjajic (NUIG) 16 / 60

Page 27: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Knowledge synthesis

Bayes theorem modifies p(θ), the uncertainty about θ, using the infoin the sample, p(y | θ) = L(θ | y):

p(θ | y) ∝ p(θ)× L(θ | y),

p(θ) = prior; info. about θ outside the sample (external)

L(θ | y) = likelihood; info about θ within the sample (internal)

p(θ | y) = posterior; updated info about θ

On the log. scale, lnp(θ | y) = lnp(θ) + lnL(θ | y), so the Bayestheorem formalizes synthesis of knowledge:(posterior

total info. on θ

)=

(prior

info outside sample

)+

(likelihood

info within sample

)The theorem emphasizes the sequential nature of knowledgeacquisition, before and after obtaining the sample:

Milovan Krnjajic (NUIG) 17 / 60

Page 28: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian analysis

Classical stats interprets the probability as a limiting frequency in aseries of experiments repeated many times (impossible to formulateprobability of inherently unrepeatable events).In Bayesian statistics the probability is a degree of the subjectivebelief (of a rational person) in the truth of a (true/false) statement.This is subjective probability.Science objective? It aspires to be so, and to avoid subjectivejudgments, yet in every area of science there is a controversy anddifferences of opinion over topics of current interest.Every statistical model (Bayesian or otherwise) ever developed isbased on a series of subjective judgment calls and assumptions. InBayesian stats we make these explicit (as priors).Hallmark of Bayesian approach: Uncertainty is directly related torandomness such that any unknown or uncertain quantity is treatedas a random variable with the corresponding prior distribution.In function L(θ) = p(y | θ) the parameter vector θ is unknown and isconsequently treated as a random variable.

Milovan Krnjajic (NUIG) 18 / 60

Page 29: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Additional information, prior distribution

One of the hallmarks of the Bayesian approach, treating unknowns asrandom, allows us to incorporate in the model the information that isnot present in the data but can be expressed probabilistically.

This is the prior distribution, which encodes information obtainedfrom sources other than the sample (can be specified before or afterthe data sample is obtained).

This gives a potentially great advantage to Bayesian methods interms of the ability to integrate information in the data set along withall other available information.

Significant role in this plays the subjective interpretation of probability.

This has been a source of controversy, disagreements andmisunderstandings.

There are some problems in this approach, but the benefits of beingable to use additional information in a consistent and systematicmanner by far outweigh any difficulties

Milovan Krnjajic (NUIG) 19 / 60

Page 30: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Experts, impostors, and priors

Three experiments involving tea, music sheets, and cards, and threeexperts, a tea taster, a music sheet reader, a drunk (based on anexample from L. Savage).

Experts claim extreme ability to recognize properties of objects

Assume 10 correct answers out of 10 attempts for each expert

Hypothesis H0: the expert correct no more than expected by chanceHypothesis H1: the expert performs better than chance (based on aspecial ability)

There is an equally obvious evidence against H0 in each case (thesame data), thus the conclusions are the same.

But should they be? A Bayesian analysis can include prior knowledge,different in each case, affecting the final conclusions in a reasonableway.

Milovan Krnjajic (NUIG) 20 / 60

Page 31: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Prior distribution, tea taster

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

02

46

810

12

Tea taster

PriorLikelihoodPosterior

Figure:

Milovan Krnjajic (NUIG) 21 / 60

Page 32: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Prior distribution, music expert

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

02

46

810

12

Music expert

PriorLikelihoodPosterior

Figure:

Milovan Krnjajic (NUIG) 22 / 60

Page 33: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Prior distribution, drunk

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

02

46

810

12

Drunk

PriorLikelihoodPosterior

Figure:

Milovan Krnjajic (NUIG) 23 / 60

Page 34: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Expert posterior distributions

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

02

46

810

12

Posteriors

Tea tasterMusic expertDrunk

Figure:

Milovan Krnjajic (NUIG) 24 / 60

Page 35: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Additional information, prior distribution

This example shows the value of incorporating additional information,even if it may be subjective (in the sense of one’s making a judgmentcall, but then, as rational beings, we do it often)In Bayesian modelling approach we express the additional informationin terms of the probability distributionThe second message from this example is that different priors mayhave quite different impact on the posteriorIn the time of subjectivity controversy Bayesians developed a class ofpriors with minimal info on which anyone would agree, attempting toformulate ”objective ignorance” (weak, diffuse, non-informative,objective priors).Informative priors extremely useful; Elicitation, the process ofobtaining substantive info from the (application) experts andencoding it in terms of prob. dist.Studying the impact of priors on the posterior is a regular exerciseduring development of a Bayesian model. This is sensitivity analysisand it also includes comparison of alternative models

Milovan Krnjajic (NUIG) 25 / 60

Page 36: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

A conjugate Bayesian model

Sample y = (y1, y2, ..., yn), yi ∈ {0, 1}, where n = 300, nd = 50 loansare delinquent (1) and (n − nd) = 250 are not (0).

Goal: what is the probability of loan delinquency in the populationand how uncertain are we about the estimate.

Loan, yi : Bernoulli (binary) random variable (r.v.):

yi =

{1, with probability θ,0, with probability 1− θ

(1)

P(yi = b) = θb(1− θ)1−b, b = 1 ili b = 0

All r.v.-s are independent and identically distributed (IID)

The r.v. θ is unknown parameter of the Bernoulli dist.

Goal: obtain the posterior information about θ

Milovan Krnjajic (NUIG) 26 / 60

Page 37: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Beta-Bernoulli Conjugate Bayesian model

Bayesian model:

(yi | θ)IID∼ Bernoulli(θ), i = 1, . . . , n

θ ∼ p(θ)

Prior prob. p(θ) = Beta(a, b) ∝ θa−1(1− θ)b−1

Likelihood = p(sample | θ) ∝ θnd (1− θ)n−nd

Posterior prob. p(θ | sample) ∝ prior prob.× likelihood

post dist. p(θ | sample) = Beta(a + nd , b + n − nd) =Beta(53, 270), for a = 3, b = 20

Posterior and prior from same family of dist. – prior dist. conjugatewith the likelihood

Post. mean of θ, E (θ | sample) = 0.164; SD(θ | sample) ≈ 0.021;95%CI ≈ (0.164± 2× 0.021) = (0.12, 0.20)

Milovan Krnjajic (NUIG) 27 / 60

Page 38: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Beta-Bernoulli Conjugate Bayesian model

0.0 0.1 0.2 0.3 0.4

05

1015

20

θ

Pro

babi

lity

dens

ity

a priorilikelihooda posteriori

Milovan Krnjajic (NUIG) 28 / 60

Page 39: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian highest density interval

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Beta(3, 9) Highest density interval

θ

95% credible interval = (.060, .518)95% highest densty int. = (.04, .48)

Figure: 95%CI = (.06, .518), HDI 95%CI = (.04, .48 ); P(θ ∈ 95% CI ) = .95 = P(θ ∈ HDI 95% CI)

Milovan Krnjajic (NUIG) 29 / 60

Page 40: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian Computation

Figure: Cmputing power increases exponentially; from about 1 MIPS for Motorola 68000 (1980) to 17 peta FLOPS (2012,

LLNL, Sequoia, IBM BlueGene PowerPC processors, 98,000 nodes). 1 peta = 1015 = 1 million billion floating point operationsper second. 1 sequoia aprox. 1 billion MC68000

Milovan Krnjajic (NUIG) 30 / 60

Page 41: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Monte Carlo (MC) sampling

In practice, p(θ | y) = p(θ)× L(θ | y), complex and dim(θ) large

Problem: no closed form for integrals; no numerical integration;

A solution: Can learn anything about a probabilitydistribution from a large sample.

If xi ∼ p(x) then 1n

∑ni=1 g(xi ) → E [g(X )] =

∫X g(x)p(x)dx

Instead of maths – simulation: generate independent samples fromthe joint posterior distribution

Milovan Krnjajic (NUIG) 31 / 60

Page 42: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Rejection method for direct MC sampling

0 50 100 150 200

0.2

0.4

0.6

0.8

Samples of θ from Accept − Reject procedure

Histogram of θ samples from Accept − Reject procedure

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

Figure: Goal: generate samples from a p.d. with density f (θ) = −4θlog(θ), where θ ∈ (0, 1).Rejection method: generate a θi from Unif(0,1) and accept it w.p. (θ)/1.5. The acceptedsamples are from the p.d. with density f (x) = −4θlog(θ).

Milovan Krnjajic (NUIG) 32 / 60

Page 43: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Markov Chain Monte Carlo(MCMC)

IID sampling from u multidim. dist. very difficult and inefficient.

Markov Chain Monte Carlo (MCMC): a class of algorithms thatgenerate samples in a way that depends on the previous sample (notIID)

Ergodic theorem: MCMC sequence (chain) converges to the desireddistribution.

Difficult to prove convergence in practice, instead can show evidence

Milovan Krnjajic (NUIG) 33 / 60

Page 44: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Inventors of MCMC

A. A. Markov (1857 – 1936 ) John von Neumann, Stanislav Ulam, Nicholas Metropolis

Milovan Krnjajic (NUIG) 34 / 60

Page 45: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

MCMC: Metropolis algorithm

Metropolis algorithm: generate samples from any prob. dist. p(θ | y)

Let g(θ) be a symetric p.d. with the same support as p((θ) | y)

Metropolis algorithm (step t + 1):I Generate θ∗ iz g(θ)I Accept θ∗ as a new sample (that is, θt+1 = θ∗) with probability

min(α, 1), where

α =p(θ∗ | y)

p(θt | y)

I If θ∗ is not accepted then θt+1 = θt

The set of sample points (θ1, θ2, . . . , θN) has p.d. p(θ | y)

Metropolis algorithm works for any distribution, however, it can bevery inefficient

Milovan Krnjajic (NUIG) 35 / 60

Page 46: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

MCMC: Gibbs algorithm

Goal: generate samples from the joint p.d. p(θ1, θ2, θ3 | y)

The joint p.d. p(θ1, θ2, θ3 | y) is uniquely determined withp(θ1 | θ2, θ3, y), p(θ2 | θ1, θ3, y) i p(θ3 | θ1, θ2, y) (so called fullconditionals)

Gibbs algorithm generates samples from full conditionals. A set ofsuch points has the joint p.d. p(θ1, θ2, θ3 | y)

starting values: θ01, θ

02, θ

03

Step t + 1 of the Gibbs:

θt+11 ∼ p(θ1 | θt

2, θt3, y)

θt+12 ∼ p(θ2 | θt+1

1 , θt3, y)

θt+13 ∼ p(θ3 | θt+1

1 , θt+12 , y)

Sampling from full cond. can be complex; high auto-correlation

Milovan Krnjajic (NUIG) 36 / 60

Page 47: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian decision theory

Bayesian decision theory: rational and coherent decisions are madebased on maximization of the expected utility function.

Let A be a set of possible actions, U(a, θ) the utility function, actiona, where the unknown info is θ;

The optimal actions a∗ maximizes the expectation of U:

E(θ|y) [U(a, θ)]

This approach has been succesfully used in many areas, such asbusiness management, econometrics, engineering, health care,medicine, etc.

Milovan Krnjajic (NUIG) 37 / 60

Page 48: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian methodology (+/-)

, Natural way of combining apriori knowledge with observations;Sequential learning, in an analogy to acquiring of the scientificknowledge.

, Basis for the theory of optimal decision making, Inference based on posterior distributions, does not depend on

asymptotics and approximations of CLT type., Interpretation of CI-s for estimators direct and intuitive., Hypothesis testing without p-values, Enables systematic mapping of complex problems and structured data

into corresponding model specifications based on hierarchies ofconditional prob. distributions.

, Uses unique probabilistic framework for model development./ Requires careful choice and spec of the prior distributions./ Requires sensitivity (robustness, stability) analysis of inference with

respect to the apriori assumptions./ Development of the model spec and MCMC implementation

non-trivial; requires support of substantial computing power.Milovan Krnjajic (NUIG) 38 / 60

Page 49: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Future directions for modern statistical analysis

Both paradigms have their strenghts and weaknesses. In practicalwork, as staitsticians use any tools which solve the problem.In the future research and application of modelling methodology,focus on the advantages of both paradigms; bring them togethermaximizing the gain and also minimizing the impact of weak features.Use Bayesian approach when constructng the model, bringing in allthe relevant information and take advantage of (1) full probabilisticframework for inference (2) and flexibility of hierarchical specificationsto model structured data,Interestingly, the fequentist ideas and statistical results based onfrequentist interpretation of probability remain valid for analyzing thesampling output of the MCMC, which is extremely important toexamine and establish evidence of convergence.When checking and using the model it is important to assess itsperformance in a frequentist way, how many times does the modelpredict well when used in practice.With the available computing power which is continuing to rise, theBayesian approach to modelling becomes more efficient andincreasingly feasible for tackling the real-world problems of growingcomplexity.

Milovan Krnjajic (NUIG) 39 / 60

Page 50: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Adaptive clinical trials

Clinical trials are prospective studies with the goal of evaluating the effect of amedical treatment (a drug or a procedure) administered to human patients undercontrolled conditions.

Phase I: considers safety from toxicity and includes dose finding; Phase II: looks fordrug efficacy, having to protect against both toxicity and futility (continuing a trialunlikely to produce positive results, i.e. the drug has no effect) even if all availablepatients are enrolled in the trial. In such case it is prudent to stop the trial early;Phase III: randomized controlled multi-center confirmatory trials on large groups ofpatients; The basis for making definitive assessment of drug efficacy.

A standard approach to design and analysis of the clinical trials uses methods ofclassical or frequentist statistics.

Inference closely follows the structure of the trial and the protocol must be strictlymaintained throughout the trial.

Suboptimal in terms of finishing early (either confirming the success or declaringnon-efficacy of a drug or a procedure).

Patients must be randomized in two groups; the number of patients in eithergroup sometimes larger than necessary.

Any future adaptations or interventions in the trial must be spelled out in advancein order for the inference to work.

Milovan Krnjajic (NUIG) 40 / 60

Page 51: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Adaptive clinical trials (2)

A modern alternative approach uses Bayesian modelling and design methodologies.

Bayesian approach results in flexible trial design and analysis where experimentsand corresponding interventions can be changed during the course of trials.Furthermore, various sources of information can be included in the analysis as thenew information becomes available and expert opinion can be incorporated in finalinferences and conclusions. Also, the methods of decision theory can be seemlesslycombined with the results of Bayesian analysis when making final decisions.

A leading criterion in freqentist design of trials is the control of the rate of Type Ierrors (false positives). Any change in the protocol of the trial which affectsstopping boundaries is deemed adaptive if it also keeps Type I error constant.

Bayesian inference, does not depend on the particular features of the design of thetrial such as selection of the sample sizes in advance, for example. In Bayesianapproach, besides choosing the stopping rules during trial, one is free to makeassumptions in the form of prior probability distributions. Trial sample sizes can bedetermined while the trials are in progress.

A goal of Bayesian approach to design and analysis of trials is to maximize theusage of information (which may become sequentially available during the trial),minimize number of patients involved along with the trial duration. Historical datacan be useful in this approach, especially for testing medical devices.

Milovan Krnjajic (NUIG) 41 / 60

Page 52: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Adaptive clinical trials (3)

For Immediate Release: Feb. 5, 2010 Media Inquiries: Karen Riley, 301-796-4674;[email protected] Consumer Inquiries: 888-INFO-FDAFDA Issues Guidance to Help Streamline Medical Device Clinical Trials. Agency says Bayesianstatistical methods could trim costs, boost efficiency.The U.S. Food and Drug Administration today issued guidance on Bayesian statistical methodsin the design and analysis of medical device clinical trials that could result in less costly andmore efficient patient studies.The Bayesian statistical method applies an algorithm that makes it possible for companies tocombine data collected in previous studies with data collected in a current trial. The combineddata may provide sufficient justification for smaller or shorter clinical studies.The final guidance describes use of Bayesian methods, design and analysis of medical deviceclinical trials, the benefits and difficulties with the Bayesian approach, and comparisons withstandard statistical methods. The guidance also presents ideas for using Bayesian methods inpost-market studies.Health care payers are also contemplating the role Bayesian methods could play in makingcoverage decisions. In a June 2009 public meeting, a Medicare Advisory Committee encouragedMedicare policymakers to consider Bayesian approaches when reviewing trials or technologyassessments during the national coverage analysis process.

Milovan Krnjajic (NUIG) 42 / 60

Page 53: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Modelling of Count Data

Example: Count data — Bayesian parametric Poisson based model vs.Bayesian nonparametric (BNP) with a Dirichlet process prior.

Fixed-effects Poisson model, (for i = 1, . . . , n),

(yi |θ)ind∼ Poisson[exp(θ)]

(θ|µ, σ2)iid∼ N(µ, σ2)

(µ, σ2) ∼ p(µ, σ2).

(2)

This uses a Lognormal prior for λ = eθ rather than conjugate Gamma choice;the two families are similar, and the Lognormal generalizes more readily.

Data often exhibit heterogeneity resulting in (extra-Poisson variability),variance-to-mean ratio, VTMR > 1

Milovan Krnjajic (NUIG) 43 / 60

Page 54: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Parametric Random-Effects Poisson (PREP) Model

Random-effects Poisson model (PREP):

(yi |θi )ind∼ Poisson[exp(θi )]

(θi |G )iid∼ G

G ≡ N(µ, σ2)(µ, σ2) ∼ p(µ, σ2),

(3)

assuming a parametric CDF G (the Gaussian) for the latent variablesor random effects θi .

Distribution, G , in the population to which it’s appropriate togeneralize may be multimodal or skewed, which a single Gaussiancan’t capture; if so, this PREP model can fail to be valid.

Milovan Krnjajic (NUIG) 44 / 60

Page 55: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Dirichlet Process Mixture Model

Remove the assumption of a specific parametric family (normal) forthe mixing dist. G , allowing G to be random and specifying a priormodel on the spqace of {G}, that may be centered on N(µ, σ2), butpermits adaptation/learning.

We use Dirichlet process (DP), G ∼ DP(α,G0),

Poisson DP mixture model (PDPM):

yi | θiind∼ Poisson(eθi )

θi | Giid∼ G

G ∼ DP(α,G0),

(4)

where G0 ≡ N(·;µ, σ2) and i = 1, ..., n.

yi | Gind∼

∫Poisson(yi ; e

θ)dG (θ), (5)

with random mixing d.f. G ∼ DP(α,G0), and G0 = N(µ, σ2).

Milovan Krnjajic (NUIG) 45 / 60

Page 56: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Financial crisis, liquidity risk, credit risk

Financial crisis, from 2008 until now, the worst in last 80 years.

Former giant financial power houses got bankrupt or were liquidated (LehmanBrothers), some were sold (Bear Stearns, Merrill Lynch), and some investmentbanks became commercial (Morgan Stanley, Goldman Sachs). Big companies alsogot bankrupt (Gen. Motors) and the governments of several countries had to makeunpopular decisions to salvage the banking systems from total collapse.

The root causes and the dynamics of the crisis will be studied for long time.

A few decisive factors: (1) unprincipled credit rating of the companies heavilyinvested in complicated financial products including large positions in collateralizedloan obligations. (2) absence of proper regulation, the CDS markets grew huge,contracts made in private, no market transparency; No proper regulation in thesecuritization of loans and other financial instruments (3) massive financing ofresidential loans made available to population with no ability to service the loans.(4) extremely complex and innovative financial instruments and their deirvatives(5) turning the financial market into a giant casino with the betting system usingderivatives and syntetic CDS creating a pyramid of bets several hundred timeslarger than the total value of underlying real financial structure.

Scandalous corruption at highest levels of core financial institutions (NASDAQ)

In a situation like this, the serious errors in risk assessment unavoidable

Milovan Krnjajic (NUIG) 46 / 60

Page 57: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Analysis of liquidity risk and credit risk (1)

Goal: Main modelling ideas and results of an analysis of the roles of liquidityrisk and credit risk in the financial crisis 2007-2009. (Garry Koop,Department of Economics, Univ of Stratchlyde, UK)

Joint analysis of the CDS spread between LIBOR and OIS. LIBOR = London

Interbank Offered Rate; OIS = Overnight Index Swap rates; CDS = Credit default swap.

A swap transfering credit exposure of fixed income products between parties. Payments

are made to the seller of the swap. In return, the seller agrees to pay off a third party debt

if this party defaults on the loan. A CDS is considered insurance against non-payment.

Separated dynamics of credit risk and liquidity risk.

Liquidity Risk (LR): stems from inability to buy or sell a security quicklyenough to prevent or minimize a loss.

Credit Risk (CR): possibility that debtor stops servicing his obligations, forexample, paying off the loan.

Milovan Krnjajic (NUIG) 47 / 60

Page 58: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Analysis of liquidity risk and credit risk (2)

LIBOR: A dayly interest rate at which banks can borrow funds from eachother in the London interbank market.

OIS: (overnight indexed swap) is a way for two financial institutions to exchange interest

rates which they pay (to ithers), a fixed is exchanged for a variable. OIS is a measure of

the investor expectation of the effective federal funds rate and should NOT reflect credit

or liquidity risk. High OIS means that the banks are unwilling to borrow, whereas a low

OIS means that the liquidity is good.

”LIBOR-OIS spread” (difference between LIBOR and OIS, is an indication of the

condition of credit markets. Higher and increasing spread means that the banks consider

that credit risk is high implying a possibility of economic decline. Lower and decreasing

spread means that the banks consider lending the money less risky indicating prospect of

economic growth.)

Milovan Krnjajic (NUIG) 48 / 60

Page 59: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Analysis of liquidity risk and credit risk (3)

LIBOR-OIS spread has two components due to liquidity risk and credit risk.(typically about 10bp; compare with 364bp 10/2008, about 100bp 01/2009).

A typical analysis of LIBOR-OIS spread based on a single aggregate timeseries.

This analysis of LIBR-OIS spread is based on cross-bank, cross-currency andcross term time series from a number of different banks, different currencies,and terms (long and short term spreads).

The goal is to disentangle credit and liquidity risks from each other andanalyze the evolution of good/bad combinations of each risks during thecrisis.

Milovan Krnjajic (NUIG) 49 / 60

Page 60: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Analysis of liquidity risk and credit risk (4)

Gary Koop (et al.), Department of Economics, University of Stratchlyde, Understanding

liquidity and credit risks in the financial crisis (Oct. 2010). Analyzes LIBOR-OIS time

series in three dimensions: {banks, currency, terms} in order to understand the dynamics

and impact of two risks (liquidity and credit).

Data: LIBOR-OSI from the banks such as (Barclays, JP Morgan, Citibank,

Deutchebank, RBS, HSBC, Rabobank, LLoyds, UBS); terms of (1, 3, 12) months;

currencies (EUR, USD, GBP)

Basic elements of the model:I SijktSijktSijkt = LIBOR-OIS;I DjtDjtDjt = CDS rate for bank j ;I LktLktLkt = liquidity risk (unknown);I KtKtKt = aggregate credit risk for all banks (unknown);

Meaning of the indexes:I term: iii = {1, 2, 3}I bank: jjj = {1, 2, ..., 12}I currency: kkk = {1, 2, 3}I time (days): ttt = {1, 2, ..., 727}

Milovan Krnjajic (NUIG) 50 / 60

Page 61: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian structural model of dynamic factors (1)

A simple form of state space model, or dynamic linear model (DLM)

Yt = Ftθt + vt , vt ∼ Nm(0,Vt)

θt = Gtθt−1 + wt , wt ∼ Np(0,Wt)

where Ft is m × p matrix, θ is p × 1 vector of states, vt i wt is a sequenceof independent Gaussian r.v.-s Vt i Wt are covariance (dispersion) matriceswhereas Yt are observations.

Latent variables, θ, change according to the Markov chain:

P(θt | θ1, ..., θt−1) = P(θt | θt−1)

If (θ∗1 , . . . , θ∗k ) are discrete, we have the so called switching states model, or

hidden Markov model, where P(i | j) = P(θt = θ∗i | θt−1 = θ∗j )

0 25 50 75 100 125 150 175 200 225 250 275 300 325 350 375 400 425 450 475 500−5

0

5

10

15

20

time (s)

respiration

!"#$% &' ()%*+ ,-.#/% 0$%*1!$2+!-3 4-$ %6 -4 2 12+!%3+ 7!+) *.%%1 213%2 8#$!3" +7- 3-39 -3+!3#-#* +!/%*%"/%3+* -4 +)% *2/% 3!")+ 0/%2*#$%/%3+* *2/1.%8 2+ : ;<6= 026 >$2!3!3" 82+2= ?13%2 !* )2$2 +%$!<%8@A %B+%38%8 1%$!-8* -4 */2.. ,2$!2@!.!+A !3 )%*+ ,-.#/%C 4-..-7%8 @A @#$*+* 0"2*1!3"6= ;%$% 7% *%% *# )@%)2,!-#$ 2$-#38 D :EFC 4-..-7%8 @A 3-$/2. $)A+)/! @$%2+)!3"= 0@6 >%*+ 82+2= G3 +)!* *%"/%3+ 7% H38*%,%$2. !3*+23 %* -4 213%2 238 23 211$-B!/2+%.A $)A+)/! $%"!-3= 0>)% +)! I .!3%* 2+ +)% @-++-/ -4 %2 )1.-+ 2$% %B1.2!3%8 !3 +)% /2!3 +%B+=6JK

Milovan Krnjajic (NUIG) 51 / 60

Page 62: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian structural model of dynamic factors (2)

Joint model for S (LIBOR-OIS) spread and D (CDS, credit default swap)accross different banks, terms and currencies:

Sijkt = λSijkLkt + ψS

ij Kt + β′

ikXt + εSijkt

Djt = ψKj Kt + γ

′Zt + εDjt

where εSijktIID∼ N(0, σ2

ijkS) and εDjtIID∼ N(0, σ2

jD)

Assumptions:

I coefficients (of L i K ) vary overbanks (j) and terms (i)I D, the CDS bank rate does not depend directly on liquidity riskI λS > 0, ψS > 0, ψK > 0, growth of L i K increases S (LIBOR-OIS) and

D (CDS rate)I Credit risk impacts S equally for all terms i and currencies k.

Milovan Krnjajic (NUIG) 52 / 60

Page 63: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Bayesian structural model of dynamic factors (3)

Model for liquidity risk Lkt and credit risk Kt :

Lkt = φk0 + φk1Lk,t−1 + σkLvkt

Kt = η0 + η1Kt−1 + σKwt

Markov state space model (st):

Lkt = φLk0(s

Lt ) + φL

k11(sLt )Lk,t−1 + σkL(s

Lt )vL

kt

Kt = φK0 (sK

t ) + φ1K (sKt )Kt−1 + σK (sK

t )vKt

sKt ∈ {GK ,BK} and sL

t ∈ {GL,BL} where ukt and vt are from N(0,1) dist.

Result: transition probability matrices for all states (probability of goingfrom state i to state j)

st is same for all currencies, so all liquidity factors are connected through st .

Milovan Krnjajic (NUIG) 53 / 60

Page 64: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Posterior mean of latent credit and liquidity risks

Figure: Credit risk (dark blue); Liquidity risk (EUR, USD, GBP); CR: mainly grows until 12/008 with two abrupt dips,larger from 12/2007 to 01/2008 (result of forced re-evaluation and writedown), and a smaller one in the summer of 2008;otherwise the changes are fairly smooth. LR: changes much more abruptly with three major upsurges of LR 08/2007, 12/2007,and in 10/08 (time of the worst crisis); LR similar accross currencies, yet LR in USD is the largest of the three (10/2008, peakof crisis). CR changes slower than LR, whereas LR has abrupt changes and it is easier to relate these with regulatoryinterventions and other events during the crisis.

Milovan Krnjajic (NUIG) 54 / 60

Page 65: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Probabilities of states (GL, GC )

Milovan Krnjajic (NUIG) 55 / 60

Page 66: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Probabilities of states (BL, GC )

Milovan Krnjajic (NUIG) 56 / 60

Page 67: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Probabilities of states (GL, BC )

Milovan Krnjajic (NUIG) 57 / 60

Page 68: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Probabilities of states (BL, BC )

Milovan Krnjajic (NUIG) 58 / 60

Page 69: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Variability components of LIBOR-OIS (Euro)

1 Month 3 Months 12 MonthsVar L Var C Var L Var C Var L Var C

barclays 0.8953 0.0000 0.6521 0.1986 0.2183 0.5940

btmufj 0.8509 0.0154 0.6115 0.2391 0.2308 0.5804

citibank 0.8573 0.0045 0.6327 0.2208 0.2288 0.5784

deutschebank 0.8600 0.0000 0.5914 0.2377 0.2786 0.5370

hbos 0.8276 0.0000 0.7510 0.0077 0.5575 0.0610

hsbc 0.8616 0.0003 0.6458 0.2079 0.2908 0.5170

jpmc 0.9018 0.0000 0.6562 0.2013 0.2787 0.5280

lloyds 0.8918 0.0000 0.6309 0.2220 0.2037 0.6015

rabobank 0.8816 0.0015 0.6043 0.2301 0.2736 0.5323

rboscotland 0.8773 0.0003 0.6107 0.2350 0.1847 0.6213

ubs 0.8353 0.0005 0.5832 0.2578 0.2277 0.5860

westlb 0.8539 0.0043 0.5929 0.2518 0.2337 0.5835

Figure: Variability conomponents for teh predictive distribution; 1-month terms, over 80% ofvariability belongs to LR, 20% to CR; 3-month terms (about) 60-70%; 12-month terms – creditrisk more significant.

Milovan Krnjajic (NUIG) 59 / 60

Page 70: 2013.03.26 An Introduction to Modern Statistical Analysis using Bayesian Methods

Thank you for your attention!

Milovan Krnjajic (NUIG) 60 / 60