On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc...
Transcript of On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc...
![Page 1: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/1.jpg)
1
On Heavy-Tailed Rare Event Analysis
Chang-Han Rhee
Centrum Wiskunde & Informatica, Amsterdam
![Page 2: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/2.jpg)
2
Rare Events
![Page 3: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/3.jpg)
2
Rare Events
![Page 4: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/4.jpg)
2
Rare Events
![Page 5: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/5.jpg)
2
Rare Events
Although rare, rare events matter.
Need for understanding ‘how often?’ & ‘why?’
![Page 6: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/6.jpg)
3
Rare Events depend on “Tail Behaviors”
Light-Tailed Distributions
• Extreme Values are Very Rare
• Normal, Exponential, etc
Heavy-Tailed Distributions
• Extreme Values are Frequent
• Power Law, Weibull, etc
Structural difference in the way systemwide rare events arise.
![Page 7: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/7.jpg)
3
Rare Events depend on “Tail Behaviors”
Light-Tailed Distributions
• Extreme Values are Very Rare
• Normal, Exponential, etc
Heavy-Tailed Distributions
• Extreme Values are Frequent
• Power Law, Weibull, etc
Structural difference in the way systemwide rare events arise.
![Page 8: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/8.jpg)
3
Rare Events depend on “Tail Behaviors”
Light-Tailed Distributions
• Extreme Values are Very Rare
• Normal, Exponential, etc
Systemwide rare events
arise because
EVERYTHING goes wrong.
(Conspiracy Principle)
Heavy-Tailed Distributions
• Extreme Values are Frequent
• Power Law, Weibull, etc
Structural difference in the way systemwide rare events arise.
![Page 9: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/9.jpg)
3
Rare Events depend on “Tail Behaviors”
Light-Tailed Distributions
• Extreme Values are Very Rare
• Normal, Exponential, etc
Systemwide rare events
arise because
EVERYTHING goes wrong.
(Conspiracy Principle)
Heavy-Tailed Distributions
• Extreme Values are Frequent
• Power Law, Weibull, etc
Systemwide rare events
arise because of
A FEW Catastrophes.
(Catastrophe Principle)
Structural difference in the way systemwide rare events arise.
![Page 10: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/10.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’,
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
![Page 11: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/11.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ’Why?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
![Page 12: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/12.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ‘Most likely scenario?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
![Page 13: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/13.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ‘Most likely scenario?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
In fact, ‘The only plausable scenario!’
![Page 14: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/14.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ‘Most likely scenario?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
In fact, ‘The only plausable scenario!’
![Page 15: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/15.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ‘Most likely scenario?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
In fact, ‘The only plausable scenario!’
![Page 16: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/16.jpg)
4
Light-tailed rare-event analysis has a long & successful history.
• Large Deviations Theory answers ‘How rare?’, ‘Most likely scenario?’
• Design of Scheduling Algorithms, Simulation Algorithms, etc
Varadhan won Abel Prize in 2007
Heavy-tailed rare events are NOT understood well.
In fact, ‘The only plausable scenario!’
![Page 17: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/17.jpg)
5
But, Heavy Tails are Everywhere:Heavy tails are everywhere!
Computer systems Finance
delays, files, … losses
1 𝑃 𝑠𝑖𝑧𝑒 > 𝑛 ≈ 1
𝑛𝛼
Social networks Energy Systems
popularity, contagion blackouts
B. Zwart (CWI) Heavy tails 4 / 43
![Page 18: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/18.jpg)
6
For Example: Open Problem Posed by Whitt (2000)
Congestion of Multiple Server Queue:
d
2
1
How many big jobs are needed to create large queue lengths?
In case of Weibull tails, NOT EVEN a conjecture!
![Page 19: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/19.jpg)
6
For Example: Open Problem Posed by Whitt (2000)
Congestion of Multiple Server Queue:
d
2
1
How many big jobs are needed to create large queue lengths?
In case of Weibull tails, NOT EVEN a conjecture!
![Page 20: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/20.jpg)
7
Goal: Systematic Tools for Heavy-Tailed Systems
In many applications, one can write the rare event of interest as
{Sn ∈ A}
where Sn is the whole trajectory of a random walk.
We want to understand P(Sn ∈ A) and P(Sn ∈ ·|Sn ∈ A)
for as general A as possible.
![Page 21: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/21.jpg)
7
Goal: Systematic Tools for Heavy-Tailed Systems
In many applications, one can write the rare event of interest as
{Sn ∈ A}
where Sn is the whole trajectory of a random walk.
We want to understand P(Sn ∈ A) and P(Sn ∈ ·|Sn ∈ A)
for as general A as possible.
How rare?
![Page 22: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/22.jpg)
7
Goal: Systematic Tools for Heavy-Tailed Systems
In many applications, one can write the rare event of interest as
{Sn ∈ A}
where Sn is the whole trajectory of a random walk.
We want to understand P(Sn ∈ A) and P(Sn ∈ ·|Sn ∈ A)
for as general A as possible.
How rare? Most likely scenario?
![Page 23: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/23.jpg)
7
Goal: Systematic Tools for Heavy-Tailed Systems
In many applications, one can write the rare event of interest as
{Sn ∈ A}
where Sn is the whole trajectory of a random walk.
We want to understand P(Sn ∈ A) and P(Sn ∈ ·|Sn ∈ A)
for as general A as possible.
sample path︷ ︸︸ ︷How rare? Most likely scenario?
![Page 24: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/24.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
yolo
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
![Page 25: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/25.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
By Law of Large Numbers, P(Sn/n > ε)→ 0.
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 26: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/26.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• By Law of Large Numbers, P(Sn/n > ε)→ 0.
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 27: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/27.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• By Law of Large Numbers, P(Sn/n > ε)→ 0.
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 28: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/28.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• By Law of Large Numbers, P(Sn/n > ε)→ 0.
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 29: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/29.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 30: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/30.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
(t)00S5
0 5
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 31: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/31.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
00S5(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 32: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/32.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
0S10(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 33: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/33.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
0S30(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 34: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/34.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
S100(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 35: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/35.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
S500(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 36: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/36.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
t
0 S∞(t)
0 1
• P(Sn ∈ A)→ 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
↙Typical Sn
Heavy-Tailed
![Page 37: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/37.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 if A does not contain 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 38: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/38.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 if A does not contain 0
Goal
- How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 39: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/39.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 if A does not contain 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 40: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/40.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 if A does not contain 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 41: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/41.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 if A does not contain 0
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 42: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/42.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 for “general” A⊆ D
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 43: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/43.jpg)
8
Goal: Systematic Tool for Heavy-Tailed Systems
• Sk , X1 + · · ·+Xk.
• Xi: iid, EXi = 0,
P(Xi ≥ x) = x−(α+1), α > 0 or P(Xi ≥ x) = exp(−xα), α ∈ (0,1).
• Sn(t), Sbntc/n, t ∈ [0,1] (scaled process)
• P(Sn ∈ A)→ 0 for “general” A⊆ D
Goal - How fast? P(Sn ∈ A)∼ ?
- What is the most likely scenario? P(Sn ∈ ·|Sn ∈ A)→?
- How to compute? P(Sn ∈ A) = ?
Heavy-Tailed
![Page 44: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/44.jpg)
9
Outline
Part 1. Large Deviations for Power Law Tails
R., Blanchet, Zwart (2016)
Under second round review at Annals of Probability
Part 2. Heavy-Tailed Rare Event Simulation
Chen, Blanchet, R., Zwart (2017)
Mathematics of Operations Research
Part 3. Large Deviations for Weibull Tails
Bazbha, Blanchet, R., Zwart (2017)
Submitted to Annals of Applied Probability
![Page 45: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/45.jpg)
10
Part 1. Large Deviations for Power Law Tails
i.e., P(Xi ≥ x) = x−(α+1), α > 0
![Page 46: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/46.jpg)
11
What’s already known: principle of a single big jump
“In heavy-tailed systems, rare events arise due to one big anomaly.”
![Page 47: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/47.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A)→ 0α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↙ Typical Sn
![Page 48: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/48.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A)→ 0→ 0α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↙ Typical Sn
![Page 49: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/49.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↙ Typical Sn
![Page 50: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/50.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
![Page 51: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/51.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
![Page 52: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/52.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
![Page 53: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/53.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
![Page 54: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/54.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ ?α−1 (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
![Page 55: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/55.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ cn−αsupt∈[0,1] (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ ?
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
![Page 56: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/56.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ cn−αsupt∈[0,1] (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
![Page 57: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/57.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
U
Z
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ cn−αsupt∈[0,1] (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
![Page 58: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/58.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
U
Z
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ cn−αsupt∈[0,1] (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
Principle of a single big jump
![Page 59: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/59.jpg)
12
What’s already known: principle of a single big jump
t
f (t)
0 1
a
U
Z
• A = { f ∈ D : f crosses level a on [0,1]}
• P(Sn ∈ A) ∼ cn−αsupt∈[0,1] (Ruin Probability of Insurance Firm)
• P(Sn ∈ ·|Sn ∈ A)→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
Hult, Lindskog, Mikosch, Samorodnitsky (2005)
↖This is typical Sn|Sn ∈ A
Principle of a single big jump
![Page 60: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/60.jpg)
13
Are all heavy-tailed rare events due to a single big jump?
No, by no means, absolutely not:
• Multiple server queues
• Queueing networks
• Re-insured insurance line
• Down-and-in barrier option
• Many more
![Page 61: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/61.jpg)
13
Principle of a single big jump is just a tip of the iceberg!
![Page 62: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/62.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
b?
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) ?∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) ?→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 63: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/63.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
b
> b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) ?∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) ?→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 64: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/64.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
b
> b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) ?→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 65: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/65.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
b
> b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 66: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/66.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
b
≤ b
≤ b
≤ b
≤ b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 67: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/67.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
bb
b
b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 68: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/68.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
bb
b
b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α ?⇒ (cn−α)3
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·) for some r.v.-s Z and U
![Page 69: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/69.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
bb
b
b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α ?⇒ (cn−α)3
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·)?⇒ P
(∑
3i=1 Zi1[Ui≤t] ∈ ·
)
![Page 70: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/70.jpg)
14
Illustration: What if Large Claims are Reinsured?
t
f (t)
0 1
a
bb
b
b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A) 6∼ cn−α ?⇒ (cn−α)3
• P(Sn ∈ ·|Sn ∈ A) 6→ P(Z1[U≤t] ∈ ·)?⇒ P
(∑
3i=1 Zi1[Ui≤t] ∈ ·
)
Principle of multiple big jumps?
![Page 71: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/71.jpg)
15
Exact Asymptotics for Heavy-tailed Random Walks
P(Xi ≥ x) = x−(α+1), α > 0
Theorem (R., Blanchet, Zwart, 2017)
For “general” A⊆ D
C(A◦)≤ liminfn→∞
P(Sn ∈ A)n−αJ (A)
≤ limsupn→∞
P(Sn ∈ A)n−αJ (A)
≤ C(A−).
• J (A): min #jumps for step functions to be inside A
• C(·): a measure
![Page 72: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/72.jpg)
15
Exact Asymptotics for Heavy-tailed Random Walks
P(Xi ≥ x) = x−(α+1), α > 0
Theorem (R., Blanchet, Zwart, 2017)
For “general” A⊆ D
P(Sn ∈ A)∼ n−αJ (A)
• J (A): min #jumps for step functions to be inside A
• C(·): a measure
![Page 73: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/73.jpg)
15
Exact Asymptotics for Heavy-tailed Random Walks
P(Xi ≥ x) = x−(α+1), α > 0
Theorem (R., Blanchet, Zwart, 2017)
For “general” A⊆ D
P(Sn ∈ A)∼ n−αJ (A)
• J (A): min #jumps for step functions to be inside A
• C(·): a measure
↙ LD power index
![Page 74: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/74.jpg)
16
Back to Our Reinsurance Example: Conjecture Confirmed!
t
f (t)
0 1
a
bb
b
b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A)∼ n−da/beα
• P(Sn ∈ ·|Sn ∈ A)→ P(
∑da/bei=1 Zi1[Ui≤t] ∈ ·
)
J (A) = da/be= 3
![Page 75: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/75.jpg)
17
Conspiracy vs Catastrophy
A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
t
f (t)
0 1
a
b
t
f (t)
0 1
a
bb
b
b
Light-Tailed Claim Size
Reinsurance makes no difference.
Heavy-Tailed Claim Size
Reinsurance helps!
![Page 76: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/76.jpg)
17
Conspiracy vs Catastrophy
A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
t
f (t)
0 1
a
b
t
f (t)
0 1
a
bb
b
b
Light-Tailed Claim Size
Reinsurance makes no difference.
Heavy-Tailed Claim Size
Reinsurance helps!
![Page 77: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/77.jpg)
17
Conspiracy vs Catastrophy
A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
t
f (t)
0 1
a
b
t
f (t)
0 1
a
bb
b
b
Light-Tailed Claim Size
Reinsurance makes no difference.
Heavy-Tailed Claim Size
Reinsurance helps!
![Page 78: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/78.jpg)
18
Two-sided Random Walks
P(Xi ≥ x) = x−(α+1), α > 0 and P(Xi ≤−x) = x−(β+1), β > 0.
Theorem (R., Blanchet, Zwart, 2017)
For “general” A⊆ D,
P(Sn ∈ A)∼ n−{αJ (A)+βK(A)}.
• J (A): # of upward jumps
• K(A): # of downward jumps
of step functions that minimize the cost of staying inside A
↙ Cost of Jumps︷ ︸︸ ︷
![Page 79: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/79.jpg)
19
Connection to Impulse Control Problem
The “rate of decay” is determined by a discrete optimization problem:
αJ (A)+βK(A) = minj,k
αj+βk
subject to ( j,k) ∈ Z2+
Dj,k∩A 6= /0
where
Dj,k = {step functions w/ j upward jumps and k downward jumps}
Different from variational calculus that arises in light-tailed case!
![Page 80: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/80.jpg)
20
Example: Barrier Option
t
f (t)
0 1
a
b
• A = { f ∈ D : f is below b at some point & end up above a}
• P(Sn ∈ A)∼ ?n−(α+β )
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(Z11[U1≤t]−Z21[U2≤t] ∈ ·
)
K(A) = 1
J (A) = 1
![Page 81: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/81.jpg)
20
Example: Barrier Option
t
f (t)
0 1
a
b
• A = { f ∈ D : f is below b at some point & end up above a}
• P(Sn ∈ A)∼ ?n−(α+β )
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(Z11[U1≤t]−Z21[U2≤t] ∈ ·
)
K(A) = 1
J (A) = 1
![Page 82: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/82.jpg)
20
Example: Barrier Option
t
f (t)
0 1
a
b
• A = { f ∈ D : f is below b at some point & end up above a}
• P(Sn ∈ A)∼ ?n−(α+β )
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(Z11[U1≤t]−Z21[U2≤t] ∈ ·
)
K(A) = 1
J (A) = 1
![Page 83: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/83.jpg)
20
Example: Barrier Option
t
f (t)
0 1
a
b
• A = { f ∈ D : f is below b at some point & end up above a}
• P(Sn ∈ A)∼ n−(1α+1β )
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(Z11[U1≤t]−Z21[U2≤t] ∈ ·
)
K(A) = 1
J (A) = 1
![Page 84: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/84.jpg)
20
Example: Barrier Option
t
f (t)
0 1
a
b
• A = { f ∈ D : f is below b at some point & end up above a}
• P(Sn ∈ A)∼ n−(1α+1β )
• P(Sn ∈ ·|Sn ∈ A)→ P(Z11[U1≤t]−Z21[U2≤t] ∈ ·
)
K(A) = 1
J (A) = 1
![Page 85: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/85.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = ?
K(A) = ?
![Page 86: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/86.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = ?
K(A) = ?
![Page 87: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/87.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 88: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/88.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 89: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/89.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 90: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/90.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 91: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/91.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 92: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/92.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 93: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/93.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 1
K(A) = ?
![Page 94: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/94.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 95: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/95.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 96: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/96.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 97: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/97.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 98: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/98.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 99: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/99.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 100: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/100.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 101: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/101.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 102: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/102.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 103: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/103.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 104: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/104.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 105: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/105.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 106: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/106.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 107: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/107.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 108: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/108.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 109: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/109.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 110: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/110.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 111: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/111.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 112: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/112.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 113: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/113.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 114: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/114.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 115: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/115.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 116: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/116.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 117: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/117.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A) = ?
![Page 118: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/118.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A)≥ 1
![Page 119: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/119.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A)≥ 1
![Page 120: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/120.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A)≥ 2
![Page 121: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/121.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 2
K(A)≥ 2
![Page 122: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/122.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 3
K(A)≥ 2
![Page 123: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/123.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A)≥ 3
K(A)≥ 2
![Page 124: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/124.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = 3
K(A) = 2
![Page 125: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/125.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ ?n−αn−β
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = 3
K(A) = 2
![Page 126: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/126.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ n−(3α+2β )
• P(Sn ∈ ·|Sn ∈ A)→ ?
P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = 3
K(A) = 2
![Page 127: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/127.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ n−(3α+2β )
• P(Sn ∈ ·|Sn ∈ A)→ P(
∑3j=1 Z+
j 1[U+j ≤t]−∑
2k=1 Z−k 1[U−k ≤t] ∈ ·
)
J (A) = 3
K(A) = 2
![Page 128: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/128.jpg)
21
Example: Sausage
t
f (t)
0 1
• A = { f ∈ D : f stays between the two curves}
• P(Sn ∈ A)∼ exp(−nI∗)
I∗: sol’n of optimization problem over absolutely continuous functions.
in case Xi’s are light tailed
![Page 129: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/129.jpg)
21
Example: Sausage
Conspiracy
Connection to variational problems
(continuous optimization)
Catastrophy
Connection with impulse control
(discrete optimization)
![Page 130: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/130.jpg)
22
Levy Processes and Multidimensional Processes
• Same results for Levy processes.
• Similar results for vector valued processes with independentcomponents as well (for both Levy processes and random walks).
Analysis of Many Server Queues and even Queueing Networks!
![Page 131: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/131.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 132: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/132.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 133: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/133.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 134: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/134.jpg)
23
Example - Stochastic Fluid Network
0.8
0.1
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 135: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/135.jpg)
23
Example - Stochastic Fluid Network
0.8
0.720.1
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 136: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/136.jpg)
23
Example - Stochastic Fluid Network
0.8
0.72Congestion!
(if µ3 < 1+0.8+0.72 = 2.52)
Cost = α1 = 1
0.1
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 137: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/137.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 138: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/138.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 139: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/139.jpg)
23
Example - Stochastic Fluid Network
0.8
0.8
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 140: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/140.jpg)
23
Example - Stochastic Fluid Network
0.8
0.8Congestion!
(if µ3 < 1+0.8+0.8 = 2.6)
Cost = α1 +α2 = 2
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 141: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/141.jpg)
23
Example - Stochastic Fluid Network
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 142: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/142.jpg)
23
Example - Stochastic Fluid Network
Congestion!
Cost = α3 = 3
Queue 1
Queue 2
Queue 3µ1 = 1
µ2 = 1
µ3
80%
80%
10%
10%
ρ1 = 0.8
ρ2 = 0.8
ρ3 = 1
• Job size distribution: Pareto with α1 = 1, α2 = 1, α3 = 3
• Queue 3 experiences congestion because of what?
• Optimization problem reduces to a knapsack type problem
![Page 143: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/143.jpg)
24
Part 2. Heavy-Tailed Rare Event Simulation
Computing P(Sn ∈ A) for finite n
![Page 144: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/144.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Head)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of head
• Divide by 100 and report the number
Should be reasonably close to 1/2
![Page 145: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/145.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge
• Divide by 100 and report the number
Should be roughly 1/2
![Page 146: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/146.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge : most likely to be 0
• Divide by 100 and report the number
Should be roughly 1/2
![Page 147: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/147.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge : most likely to be 0
• Divide by 100 and report the number : and hence, likely to be 0
Should be roughly 1/2
![Page 148: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/148.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge : most likely to be 0
• Divide by 100 and report the number : and hence, likely to be 0
Is 0 a useful answer?
No.
e.g., Nuclear Meltdown, Large-Scale Blackout, Large Financial Loss
![Page 149: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/149.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)
Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge : most likely to be 0
• Divide by 100 and report the number : and hence, likely to be 0
Is 0 a useful answer? No.
e.g., Nuclear Meltdown, Large-Scale Blackout, Large Financial Loss
![Page 150: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/150.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)Suppose≈ 10−6
• Flip the coin 100 times
• Count the number of Edge
• Divide by 100 and report the number
Should be roughly 1/2
![Page 151: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/151.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)Suppose≈ 10−6
• Flip the coin 100 a few million times
• Count the number of Edge
• Divide by the total number of flips and report the number
Should be roughly 1/2
![Page 152: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/152.jpg)
25
Challenge of Rare Event Simulation
Monte Carlo simulations as repetitive random experiments:
e.g. Coin flip: want to estimate P(Edge)Suppose≈ 10−6
• Flip the coin 100 a few million times
• Count the number of Edge
• Divide by the total number of flips and report the number
Much harder than P(Head)
![Page 153: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/153.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 154: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/154.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 155: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/155.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 156: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/156.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 157: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/157.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 158: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/158.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)
Finding a good alternative universe Q is crucial.
![Page 159: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/159.jpg)
26
Importance Sampling
for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)as an estimate of P(Edge)
Finding a good alternative universe Q is crucial.
![Page 160: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/160.jpg)
26
Importance Sampling for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Q
• Perform experiments there
- i.e., Sample I(1)Edge, . . . ,I(m)Edge from Q
i.e., as well as the likelihood ratio(
dPdQ
)(1), . . . ,
(dPdQ
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i)Edge(
dPdQ
)(i)as an estimate of P(Edge)
Finding a good alternative universe Q is crucial.
![Page 161: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/161.jpg)
26
Importance Sampling for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Qn
• Perform experiments there
- i.e., Sample I(1){Sn∈A}, . . . ,I(m)
{Sn∈A} from Qn
i.e., as well as the likelihood ratio(
dPdQn
)(1), . . . ,
(dP
dQn
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i){Sn∈A}
(dP
dQn
)(i)as an estimate of P(Sn ∈ A)
Finding a good alternative universe Qn is crucial.
![Page 162: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/162.jpg)
26
Importance Sampling for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Qn
• Perform experiments there
- i.e., Sample I(1){Sn∈A}, . . . ,I(m)
{Sn∈A} from Qn
i.e., as well as the likelihood ratio(
dPdQn
)(1), . . . ,
(dP
dQn
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i){Sn∈A}
(dP
dQn
)(i)as an estimate of P(Sn ∈ A)
Finding a good alternative universe Qn is crucial.
︸ ︷︷ ︸I{Sn∈A}
dPdQn
: IS estimator
![Page 163: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/163.jpg)
26
Importance Sampling for P(Sn ∈ A)
• Construct an alternative universe
- i.e., Consider an ”importance distribution” Qn
• Perform experiments there
- i.e., Sample I(1){Sn∈A}, . . . ,I(m)
{Sn∈A} from Qn
i.e., as well as the likelihood ratio(
dPdQn
)(1), . . . ,
(dP
dQn
)(m)
• Recover the true probability using the relationship between the twoparallel universes
- i.e., Report1m
m
∑i=1
I(i){Sn∈A}
(dP
dQn
)(i)as an estimate of P(Sn ∈ A)
Finding a good alternative universe Qn is crucial.
︸ ︷︷ ︸I{Sn∈A}
dPdQn
: IS estimator
![Page 164: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/164.jpg)
27
Goal: Strongly Efficient IS Estimator
I{Sn∈A}dP
dQnis a strongly efficient estimator for P(Sn ∈ A), if
EQn
(I{Sn∈A}
dPdQn
)2
∼ P(Sn ∈ A)2
⇒ Number of simulation runs required remains bounded.
Notoriously Hard for Heavy-Tailed Processes.
![Page 165: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/165.jpg)
27
Goal: Strongly Efficient IS Estimator
I{Sn∈A}dP
dQnis a strongly efficient estimator for P(Sn ∈ A), if
EQn
(I{Sn∈A}
dPdQn
)2
∼ P(Sn ∈ A)2
⇒ Number of simulation runs required remains bounded.
Notoriously Hard for Heavy-Tailed Processes.
Error 2
![Page 166: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/166.jpg)
27
Goal: Strongly Efficient IS Estimator
I{Sn∈A}dP
dQnis a strongly efficient estimator for P(Sn ∈ A), if
EQn
(I{Sn∈A}
dPdQn
)2
∼ P(Sn ∈ A)2
⇒ Number of simulation runs required remains bounded.
Notoriously Hard for Heavy-Tailed Processes.
Error 2Target Quantity2
![Page 167: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/167.jpg)
27
Goal: Strongly Efficient IS Estimator
I{Sn∈A}dP
dQnis a strongly efficient estimator for P(Sn ∈ A), if
EQn
(I{Sn∈A}
dPdQn
)2
∼ P(Sn ∈ A)2
⇒ Number of simulation runs required remains bounded.
Notoriously Hard for Heavy-Tailed Processes.
Error 2Target Quantity2
![Page 168: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/168.jpg)
27
Goal: Strongly Efficient IS Estimator
I{Sn∈A}dP
dQnis a strongly efficient estimator for P(Sn ∈ A), if
EQn
(I{Sn∈A}
dPdQn
)2
∼ P(Sn ∈ A)2
⇒ Number of simulation runs required remains bounded.
Notoriously Hard for Heavy-Tailed Processes.
Error 2Target Quantity2
![Page 169: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/169.jpg)
28
What is a good alternate universe Qn for P(Sn ∈ A)?
General principle for making I{Sn∈A}dP
dQnan efficient estimator:
- Choose Qn(·) as close to P(·|Sn ∈ A) as possible.
- Make sure that dPdQn
does not blow up.
![Page 170: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/170.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
![Page 171: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/171.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
dPdQn
not too big
↑
![Page 172: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/172.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
dPdQn
not too big
↑ Qn(·) close to P(·|Sn ∈ A)↗
![Page 173: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/173.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
dPdQn
not too big
↑ Qn(·) close to P(·|Sn ∈ A)↗
![Page 174: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/174.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
dPdQn
not too big
↑ Qn(·) close to P(·|Sn ∈ A)↗
![Page 175: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/175.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
dPdQn
not too big
↑ Qn(·) close to P(·|Sn ∈ A)↗
![Page 176: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/176.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ )
![Page 177: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/177.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A)
![Page 178: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/178.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ}
⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A) n−αJ (Bγ )
![Page 179: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/179.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A) n−αJ (Bγ )
![Page 180: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/180.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A) n−αJ (Bγ )
![Page 181: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/181.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2
∼(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A) n−αJ (Bγ )
![Page 182: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/182.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
n−αJ (A\Bγ ) n−αJ (A) n−αJ (Bγ )
![Page 183: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/183.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
![Page 184: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/184.jpg)
29
First All-Purpose Simulation Scheme for Heavy-Tails
• Bγ , {paths w/ at least J (A) jumps of size > γ} ⇒ J (A) = J (Bγ)
• Fix w ∈ (0,1) and define
Qn( ·), wP( ·)+(1−w)P( · | Sn ∈ Bγ)
• If we choose γ so that J (A\Bγ)≥ 2J (A),
EQn
(1{Sn∈A}
dPdQn
)2
≤ 1w
P(Sn ∈ A\Bγ)+P(Sn ∈ A) ·P(Sn ∈ Bγ)
∼(n−αJ (A))2 ∼
(P(Sn ∈ A)
)2
Zn , 1{Sn∈A}dP
dQnis strongly efficient for P(Sn ∈ A)!
Chen, Blanchet, R., and Zwart (2017)
![Page 185: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/185.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
![Page 186: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/186.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
∈ A
![Page 187: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/187.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
∈ Bγ
![Page 188: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/188.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
/∈ A\Bγ
![Page 189: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/189.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
/∈ A\Bγ
![Page 190: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/190.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
b≥≤ γ
≤ γ
≤ γ
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
/∈ A\Bγ
![Page 191: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/191.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
≤ γ
≤ γ
≤ γ
≤ γ
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
∈ A\Bγ
![Page 192: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/192.jpg)
30
How to ensure J (A\Bγ)≥ 2J (A): Reinsurance Example
t
f (t)
0 1
a
b
γ≤ b
≤ b
≤ γ
≤ γ
≤ γ
≤ γ
• A = {paths that cross level a on [0,1] & jump sizes ≤ b}
• Bγ = {paths with at least 3 jumps of size > γ}
• A\Bγ = {paths that cross level a on [0,1]& all jump sizes ≤ b & at most 2 jumps of size > γ}
7 = J (A\Bγ)> 2J (A) = 6
![Page 193: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/193.jpg)
31
Numerical Experiments for Reinsurance Example
2000 4000 6000 8000 10000
0.08
0.10
0.12
0.14
0.16
0.18
Numerical results − reinsurance policy
n
leve
l of p
reci
sion
β=1.2
β=1.4
β=1.6
β=2.0
![Page 194: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/194.jpg)
32
Numerical Results for Queueing Network
2000 4000 6000 8000 10000 12000
0.10
0.15
0.20
0.25
Numerical results − fluid queue
n
leve
l of p
reci
sion
β=(1.2,1.2,2.3)
β=(1.6,1.4,3.1)
β=(1.8,2.0,3.0)
β=(2.3,2.0,3.5)
![Page 195: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/195.jpg)
33
Part 3. Large Deviations for Weibull Tails
P(Xi ≥ x) = exp(−xα), α ∈ (0,1)
![Page 196: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/196.jpg)
34
Rate function depends on the size of the jump
Theorem (Bazhba, Blanchet, R., Zwart 2017)
Suppose that P(Xi ≥ x) = exp(−xα), α ∈ (0,1). Then
P(Sn ∈ A)∼ exp(−nα inff∈A
I( f ))
where
I( f ) =
{∑t(
f (t)− f (t−))α
, if f is a nondecreasing step function
∞, otherwise.
![Page 197: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/197.jpg)
35
Back to our old example
t
f (t)
0 1
a
bb
b
a−2b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A
)∼ exp
(−nα I∗
)where I∗ = bα +bα +(a−2b)α .
![Page 198: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/198.jpg)
35
Back to our old example
t
f (t)
0 1
a
bb
b
a−2b
• A = { f ∈ D : f crosses level a on [0,1] & jump sizes ≤ b}
• P(Sn ∈ A
)∼ exp
(−nα I∗
)where I∗ = bα +bα +(a−2b)α .
Jump Sizes Matter
![Page 199: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/199.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
d
2
1
Tail Asymptotics? Most likely scenario?
So far, NOT EVEN a reasonable conjecture!
![Page 200: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/200.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
d
2
1
Tail Asymptotics? Most likely scenario?
So far, NOT EVEN a reasonable conjecture!
![Page 201: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/201.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
P(Queue length exeeds n)∼ exp(−c∗nα)
where
c∗ = mind
∑i=1
xαi s.t.
λ s−d
∑i=1
(s− xi)+ ≥ 1 for some s ∈ [0,γ],
x1, ...,xd ≥ 0 .
• Special case of Lα -norm minimization problem.
• We have explicit solution.
Solution to Open Problem Posed by Whitt (2000)
![Page 202: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/202.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
P(Queue length exeeds n)∼ exp(−c∗nα)
where
c∗ = mind
∑i=1
xαi s.t.
λ s−d
∑i=1
(s− xi)+ ≥ 1 for some s ∈ [0,γ],
x1, ...,xd ≥ 0 .
• Special case of Lα -norm minimization problem.
• We have explicit solution.
Solution to Open Problem Posed by Whitt (2000)
![Page 203: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/203.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
P(Queue length exeeds n)∼ exp(−c∗nα)
where
c∗ = mind
∑i=1
xαi s.t.
λ s−d
∑i=1
(s− xi)+ ≥ 1 for some s ∈ [0,γ],
x1, ...,xd ≥ 0 .
• Special case of Lα -norm minimization problem.
• We have explicit solution.
Solution to Open Problem Posed by Whitt (2000)
![Page 204: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/204.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
P(Queue length exeeds n)∼ exp(−c∗nα)
where
c∗ = mind
∑i=1
xαi s.t.
λ s−d
∑i=1
(s− xi)+ ≥ 1 for some s ∈ [0,γ],
x1, ...,xd ≥ 0 .
• Special case of Lα -norm minimization problem.
• We have explicit solution.
Solution to Open Problem Posed by Whitt (2000)
Nontrivial Tradeoff between
Size and Number of Jumps
![Page 205: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/205.jpg)
36
Queue Length Asymptotics for the Weibull GI/GI/d Queue
In case service time distribution is Weibull, i.e., P(S > x) = exp(−xα),
P(Queue length exeeds n)∼ exp(−c∗nα)
where
c∗ = mind
∑i=1
xαi s.t.
λ s−d
∑i=1
(s− xi)+ ≥ 1 for some s ∈ [0,γ],
x1, ...,xd ≥ 0 .
• Special case of Lα -norm minimization problem.
• We have explicit solution.
Solution to Open Problem Posed by Whitt (2000)
Nontrivial Tradeoff between
Size and Number of Jumps
![Page 206: On Heavy-Tailed Rare Event Analysis · Design of Scheduling Algorithms, Simulation Algorithms, etc Varadhan won Abel Prize in 2007 Heavy-tailed rare events are NOT understood well.](https://reader033.fdocuments.in/reader033/viewer/2022042915/5f5236c626c49c096450dca2/html5/thumbnails/206.jpg)
37
Summary
• Systematic tools for rare-event analysis of heavy-tailed systems
• The principle of multiple (least expensive) big jumps
• Solved longstanding open problems in simulation and queueing theory
• New connections btwn rare event analysis and (discrete) optimization
e.g., impulse control, knapsack problem, Lα -norm minimization α < 1