The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum Likelihood

Maximum likelihood with Flexible Probabilities

• Topic: parametric estimation of the invariants distribution based onthe Maximum Likelihood approach

• We introduce the Maximum Likelihood with FlexibleProbabilities estimate

• We consider the special case of invariants whose distribution belongs toan exponential family

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Canonical Maximum Likelihood

Intuition behind Maximum Likelihood estimation

Suppose that the number of observations t is small, or the number of simul-taneous invariants ı is large.

A parametric assumption on the distribution of the invariants is

εt ∼ fε ∈ {fθ}θ∈Θ (2a.58)

where• θ ≡ (θ1, . . . , θl)

′, l × 1 vector

• Θ ⊆ Rl, discrete or continuous

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Canonical Maximum Likelihood

Given the realized time series of the invariants, the likelihood is

fθ (i) =∏tt=1fθ (εt) (2a.60)

where i represents the realization of the information I ≡ (ε1| . . . |εt| . . . |εt).

The Maximum Likelihood estimate is

fMLε ≡ f

θMLε

, θML

ε ≡ argmaxθ∈Θ

{1

t

∑tt=1 ln fθ (εt)} (2a.61)

The Maximum Likelihood distribution is unequivocally determinedby the corresponding Maximum Likelihood parameters

fMLε ⇔ θ

ML

ε (2a.62)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Canonical Maximum Likelihood

Given the realized time series of the invariants, the likelihood is

fθ (i) =∏tt=1fθ (εt) (2a.60)

where i represents the realization of the information I ≡ (ε1| . . . |εt| . . . |εt).

The Maximum Likelihood estimate is

fMLε ≡ f

θMLε

, θML

ε ≡ argmaxθ∈Θ

{1

t

∑tt=1 ln fθ (εt)} (2a.61)

The Maximum Likelihood distribution is unequivocally determinedby the corresponding Maximum Likelihood parameters

fMLε ⇔ θ

ML

ε (2a.62)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Properties of the ML estimation

Equivalently, the Maximum Likelihood parameters are defined as

θML

ε = argminθ∈Θ E(fHistε ||fθ) (2a.63)

where E(fHistε ||fθ) is the relative entropy (25.32) and fHist

ε the historicaldistribution (2a.22).

Consistency: the parametric assumption (2a.59) holding true, θML

ε

approximates the true, unknown, parameters θε ∈ Θ of the invariants(fε ≡ fθε), and

limt→∞ θML

ε = θε ⇔ limt→∞ fMLε = fε (2a.64)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Example 2a.17. Consistency of the maximum likelihoodparameter

• Invariants: εt ∼ t(3, 0, 2)

• Number of observations: 500

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Generalization with Flexible Probabilities

Consider Flexible Probabilities {pt}tt=1 rather than the equal probabilityweights pt ≡ 1/t.

The Maximum Likelihood with Flexible Probabilities (MLFP)estimate is

fMLFPε ≡ f

θMLFPε

, θMLFP

ε ≡ argmaxθ∈Θ

{∑tt=1pt ln fθ (εt)} (2a.65)

The MLFP distribution is unequivocally determined by the corre-sponding MLFP parameters

fMLFPε ⇔ θ

MLFP

ε (2a.66)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Properties of the MLFP estimation

Equivalently, the Maximum Likelihood with Flexible Probabilitiesparameters are defined as

θMLFP

ε = argminθ∈Θ E(fHFPε ||fθ) (2a.67)

where E(fHFPε ||fθ) is the relative entropy (25.32) and fHFP

ε the Historicalwith Flexible Probability distribution (2a.24).

Consistency: the parametric assumption (2a.58) holding true, andprovided the Effective Number of Scenarios (2a.21) is ens(p1, . . . , pt) ≈ t

θMLFP

ε ≈ θε ⇔ fMLFPε ≈ fε (2a.68)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Extracting PropertiesUnder the parametric assumption, a generic property sε ≡ S{εt} reads

sε = hS(θε) (2a.69)

for some function hS.How to estimate the properties of fε?

Maximum Likelihood with Flexible Probabilities (MLFP) es-timate

sMLFPε ≡ SMLFP{ε} ≡ hS(θ

MLFP

ε ) (2a.73)See also Example 2a.18

θMLFP

ε ⇔ fMLFPε ≈

ens(p)→∞θε ⇔ fε

hS

y yhS

sMLFPε ≈

ens(p)→∞sε

(2a.74)

where ens(p) is the Effective Number of Scenarios (2a.21).

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodFrom Maximum Likelihood to Flexible Probabilities

Extracting PropertiesUnder the parametric assumption, a generic property sε ≡ S{εt} reads

sε = hS(θε) (2a.69)

for some function hS.How to estimate the properties of fε?

Maximum Likelihood with Flexible Probabilities (MLFP) es-timate

sMLFPε ≡ SMLFP{ε} ≡ hS(θ

MLFP

ε ) (2a.73)See also Example 2a.18

θMLFP

ε ⇔ fMLFPε ≈

ens(p)→∞θε ⇔ fε

hS

y yhS

sMLFPε ≈

ens(p)→∞sε

(2a.74)

where ens(p) is the Effective Number of Scenarios (2a.21).ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodExponential family invariants

Exponential family invariants

Consider ML estimation from a sample {εt, pt}tt=1 in the exponential family(22.109)

fθ(εt) = h(εt)eθ′φ(εt)−ψ(θ) (2a.76)

The log-likelihood maximization (2a.64) becomes

θMLFP

≡ argmaxθ{θ′ηHFPε − ψ(θ) +

∑tt=1pt lnh(εt)} (2a.77)

where ηHFPε is the Historical with Flexible Probabilities (HFP) estimator

(2a.37) of the expected features (22.112)

ηHFPε ≡

∑tt=1ptφ(εt) (2a.78)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodExponential family invariants

Exponential family invariants

From an information geometry perspective, ηHFPε is the estimator of the

expectation parameters, or m-coordinates (25.57).

The estimators θMLFP

ε and ηHFPε are related as follows

θMLFP

ε ≡ (∇θψ)−1(ηHFPε ) (2a.79)

where ψ is the log-partition function (22.110).

See also Example 2a.20

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

Goal: Use Maximum Likelihood to estimate tails modeled via ExtremeValue Theory

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Extreme value theory quantile estimation

• Invariant: realized residual of GARCH(1, 1) (2.88)-(2.89) fit ofCSCO’s return

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail modeling: Extreme Value Theory (EVT)

zoom on left tail(losses)

1 Conditional excess distribution (CED)

fε−X|X≤ε (x) =fX(ε− x)

FX(ε)(3a.88)

2 Generalized Pareto distribution (GDP)

fξ,σ (x) ≡

1x≥0

σ1/ξ

(σ + ξx)1+1/ξif ξ ≥ 0

10≤x≤−σ/ξσ1/ξ

(σ + ξx)1+1/ξif ξ < 0

(3a.89)

fξ,σξ→0−→ exponential distr.

Theorem of Extreme Value Theory

fε−X|X≤ε (x) ≈ fξ∗,σ∗ (x) (3a.90)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail modeling: Extreme Value Theory (EVT)

zoom on left tail(losses)

1 Conditional excess distribution (CED)

fε−X|X≤ε (x) =fX(ε− x)

FX(ε)(3a.88)

2 Generalized Pareto distribution (GDP)

fξ,σ (x) ≡

1x≥0

σ1/ξ

(σ + ξx)1+1/ξif ξ ≥ 0

10≤x≤−σ/ξσ1/ξ

(σ + ξx)1+1/ξif ξ < 0

(3a.89)

fξ,σξ→0−→ exponential distr.

Theorem of Extreme Value Theory

fε−X|X≤ε (x) ≈ fξ∗,σ∗ (x) (3a.90)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail modeling: Extreme Value Theory (EVT)

zoom on left tail(losses)

1 Conditional excess distribution (CED)

fε−X|X≤ε (x) =fX(ε− x)

FX(ε)(3a.88)

2 Generalized Pareto distribution (GDP)

fξ,σ (x) ≡

1x≥0

σ1/ξ

(σ + ξx)1+1/ξif ξ ≥ 0

10≤x≤−σ/ξσ1/ξ

(σ + ξx)1+1/ξif ξ < 0

(3a.89)

fξ,σξ→0−→ exponential distr.

Theorem of Extreme Value Theory

fε−X|X≤ε (x) ≈ fξ∗,σ∗ (x) (3a.90)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail estimation: Maximum Likelihood with Flexible Probabilities (MLFP)

1 MLFP Pareto parameters

(ξMLFP , σMLFP ) ≡ argmaxσ>0,ξ

{∑εt≤ε

pt ln fξ,σ(ε− εt)} (3a.91)

2 MLFP cdf

FMLFPε (x) =

∫ x

−∞fMLFPε (x)dx (3a.92)

≡ fξMLFP ,σMLFP

3 MLFP quantile

qMLFPε (c) = (ε− σMLFP

ξMLFP(

(c

c

)−ξMLFP

− 1), c ≤ c ≡ FMLFPε (ε) (3a.93)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail estimation: Maximum Likelihood with Flexible Probabilities (MLFP)

1 MLFP Pareto parameters

(ξMLFP , σMLFP ) ≡ argmaxσ>0,ξ

{∑εt≤ε

pt ln fξ,σ(ε− εt)} (3a.91)

2 MLFP cdf

FMLFPε (x) =

∫ x

−∞fMLFPε (x)dx (3a.92)

≡ fξMLFP ,σMLFP

3 MLFP quantile

qMLFPε (c) = (ε− σMLFP

ξMLFP(

(c

c

)−ξMLFP

− 1), c ≤ c ≡ FMLFPε (ε) (3a.93)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update

The “Checklist” > 3a. Estimation: Flexible Probabilities > Maximum LikelihoodTails: Extreme Value Theory

Tails: Extreme Value Theory

• Tail estimation: Maximum Likelihood with Flexible Probabilities (MLFP)

1 MLFP Pareto parameters

(ξMLFP , σMLFP ) ≡ argmaxσ>0,ξ

{∑εt≤ε

pt ln fξ,σ(ε− εt)} (3a.91)

2 MLFP cdf

FMLFPε (x) =

∫ x

−∞fMLFPε (x)dx (3a.92)

≡ fξMLFP ,σMLFP

3 MLFP quantile

qMLFPε (c) = (ε− σMLFP

ξMLFP(

(c

c

)−ξMLFP

− 1), c ≤ c ≡ FMLFPε (ε) (3a.93)

ARPM - Advanced Risk and Portfolio Management - arpm.co This update: Apr-05-2017 - Last update