Approximations of Stochastic Networks - IIT Bombay · Routing structure Approximations of...

Approximations of Stochastic NetworksK. Ramanan, Brown University

() Approximations of Stochastic Networks 1 / 35

What is a Stochastic Network?

For the purposes of this tutorial, a stochastic network is a collection ofstatic interacting nodes (queues/stations) whose state evolvesrandomly, with its dynamics determined by:

External arrival processesNature of processing at each node(service policy/scheduling discipline)Routing structure


Stochastic Networks

Stochastic networks arise in a myriad applications:manufacturing, telecommunication systems, service systems (callcentres, health care operations) ...The state dynamics are typically too complex to be amenable toexact analysis

Scaling limits provide useful insight(early pioneers: Kushner, Harrison, ...)


Stochastic Networks

Stochastic networks arise in a myriad applications:manufacturing, telecommunication systems, service systems (callcentres, health care operations) ...The state dynamics are typically too complex to be amenable toexact analysisScaling limits provide useful insight(early pioneers: Kushner, Harrison, ...)


Approximations of Stochastic Networks

Different classes of stochastic networks give rise to differentscaling limits

Proving scaling limit theorems is useful only if the limit process ismore tractable than the original process describing the networkThe study of stochastic networks has spurred the development ofnew mathematical tools for the study of these scaling limitsThe mathematics developed has typically been useful in theanalysis of a broader class of stochastic processes



Different classes of stochastic networks give rise to differentscaling limitsProving scaling limit theorems is useful only if the limit process ismore tractable than the original process describing the network

The study of stochastic networks has spurred the development ofnew mathematical tools for the study of these scaling limitsThe mathematics developed has typically been useful in theanalysis of a broader class of stochastic processes



Different classes of stochastic networks give rise to differentscaling limitsProving scaling limit theorems is useful only if the limit process ismore tractable than the original process describing the networkThe study of stochastic networks has spurred the development ofnew mathematical tools for the study of these scaling limits

The mathematics developed has typically been useful in theanalysis of a broader class of stochastic processes



Different classes of stochastic networks give rise to differentscaling limitsProving scaling limit theorems is useful only if the limit process ismore tractable than the original process describing the networkThe study of stochastic networks has spurred the development ofnew mathematical tools for the study of these scaling limitsThe mathematics developed has typically been useful in theanalysis of a broader class of stochastic processes


What this Tutorial is About ...

In this tutorial we will ...

Discuss several concrete examples of stochastic networksIntroduce related scaling limitsDescribe some mathematical tools that were developed to analyzethese scaling limitsDiscuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limitDiscuss some open problems




Discuss several concrete examples of stochastic networks

Introduce related scaling limitsDescribe some mathematical tools that were developed to analyzethese scaling limitsDiscuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limitDiscuss some open problems




Discuss several concrete examples of stochastic networksIntroduce related scaling limits

Describe some mathematical tools that were developed to analyzethese scaling limitsDiscuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limitDiscuss some open problems




Discuss several concrete examples of stochastic networksIntroduce related scaling limitsDescribe some mathematical tools that were developed to analyzethese scaling limits

Discuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limitDiscuss some open problems




Discuss several concrete examples of stochastic networksIntroduce related scaling limitsDescribe some mathematical tools that were developed to analyzethese scaling limitsDiscuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limit

Discuss some open problems




Discuss several concrete examples of stochastic networksIntroduce related scaling limitsDescribe some mathematical tools that were developed to analyzethese scaling limitsDiscuss qualitative insight gained into the original stochasticnetwork from analysis of the scaling limitDiscuss some open problems


Outline of Lecture 1

Background on basic probability limit theoremsA simple single-server queueScaling limits: reflected processesSolution of a related optimization problemNext lecture:

1 Generalizations and related mathematical questions2 The second example involves a measure-valued process


Fundamental Limit Theorems in Probability

{Xn} is a sequence of independent and identically distributed randomvariables with E[|Xn|] <∞, e.g., Bernoulli random variables

P(Xn = 1) = P(Xn = −1) =12

Sn =∑n

i=1 Xi is a “simple random walk”

Theorem ( Strong Law of Large Numbers (SLLN))As n→∞

Sn

n→ E [X1] a.s.


Fundamental Limit Theorems in Probability

Theorem (Central Limit Theorem (CLT))

Given a sequence {Xn}n∈N with E[|X1|2] <∞, and Var(X1) > 0,

Sn − E [Sn]√nVar(X1)

⇒ Z

where Z ∼ N (0,1) is a standard Gaussian random variable:

P(Z ∈ A) =1√2π

∫A

e−x2/2dx

and⇒ represents “convergence in distribution”:

Zn ⇒ Z implies P(Zn ∈ A)→ P(Z ∈ A)

for all Borel sets A


Functional Versions (FSLLN)

Theorem (SLLN)Sn

n→ E[X1] a.s.

Define a sequence of “random functions” on [0,1]

B(n)

(kn

)=

Sk

n=

∑ki=1 Xi

n, k = 1,2, . . . ,

and extend to all of [0,1] by linear interpolation.

Theorem (FSLLN)Almost surely,

sups∈[0,1]

∣∣∣B(n)(s)− E[X1]s∣∣∣→ 0


Functional Versions (FCLT)

Theorem (CLT)Assume Var(X1) = 1

Sn − nE[X1]√n

⇒ Z ∼ N (0,1)

Define a sequence of centered, renormalized “random functions” on[0,1]

B(n)

(kn

)=

Sk − kE[X1]√n

=

∑ki=1(Xi − E[Xi ])√

nfor k = 1,2, . . . , and extend to all of [0,1] by linear interpolation

n = 20

n

0 1

B


Functional Versions (FCLT)

Theorem (CLT)Assume Var(X1) = 1

Sn − nE[X1]√n

⇒ Z ∼ N (0,1)

Note

B(n)

(kn

)=

Sk − kE[X1]√n

=

√kn

(Sk − kE[X1]√

k

)Sending k ,n→∞ such that k/n→ t , we see that

B(n)(t) ∼√

t

(Sbntc − bntcE[X1]√

bntc

)⇒√

tZ ∼ N (0, t)

In fact, something stronger is true.


Brownian Motion

B(n)(·)⇒ B(·)

where⇒ represents “weak convergence” andB(·) is a standard Brownian motion (BM) on [0,1]:

B is the unique random function with the property that1 B(t) ∼ N (0, t) for every t ∈ [0,1]

2 B has continuous paths (a.s.)3 B has stationary, independent increments


Weak Convergence

Definition of Weak ConvergenceGiven a complete separable metric (Polish) space (X ,d), andX -valued random elements {Xn} and {X}, Xn converges weakly to X ,denoted Xn ⇒ X , if and only if for all bounded continuous functions fon X ,

E[f (Xn)]→ E[f (X )]

Weak convergence is a generalization of convergence indistribution that applies to random elements that do notnecessarily take values in Rd .Let Y be another Polish space and suppose that the mapH : X 7→ Y is continuous, and Yn = H(Xn), Y = H(X ) then

Xn ⇒ X ⇒ Yn ⇒ Y

Continuous Mapping Theorem


Weak Convergence


E[f (Xn)]→ E[f (X )]

Weak convergence is a generalization of convergence indistribution that applies to random elements that do notnecessarily take values in Rd .

Let Y be another Polish space and suppose that the mapH : X 7→ Y is continuous, and Yn = H(Xn), Y = H(X ) then




Weak Convergence


E[f (Xn)]→ E[f (X )]

Weak convergence is a generalization of convergence indistribution that applies to random elements that do notnecessarily take values in Rd .Let Y be another Polish space and suppose that the mapH : X 7→ Y is continuous, and Yn = H(Xn), Y = H(X ) then




Single-Server Queue: A Simple Illustration

µλ (t)(t)(t)φ

Dynamics

dϕdt

=

{λ(t)− µ(t) if ϕ(t) > 0

[λ(t)− µ(t)] ∨ 0 if ϕ(t) = 0

The ODE has a discontinuous right-hand sideFalls outside the classical theory


An ODE with Discontinuous Dynamics

Dynamics

dϕdt

=

{λ(t)− µ(t) if ϕ(t) > 0

[λ(t)− µ(t)] ∨ 0 if ϕ(t) = 0

Idea: Representation in terms of Unconstrained Dynamics

Define ψ as follows: ψ(0) = φ(0) and

Unconstrained Dynamicsdψdt

= λ(t)− µ(t)

Note that1 ϕ lies in R+

2 η = ϕ− ψ increases only at times when ϕ(t) = 0


The One-Dimensional Skorokhod Map

Definition of the 1-D Skorokhod ProblemGiven càdlàg ψ : [0,∞) 7→ R with ψ(0) ≥ 0, find a càdlàgφ : [0,∞) 7→ R such that for every t ∈ [0,∞),

1 φ(t) = ψ(t) + η(t) ≥ 0;2 η(0) = 0 and η is non-decreasing;3 “η increases only when φ is at zero”:

∫∞0 I{φ(s)>0}dη(s) = 0

Theorem (1-D Skorokhod Map, Skorokhod, ’61)For every càdlàg ψ with ψ(0) ≥ 0, the càdlàg function

φ(t) = ψ(t) + sups∈[0,t]

[−ψ(s)]+, t ≥ 0,

is the unique solution to the Skorokhod problem for ψ.The map Γ : ψ 7→ φ is referred to as the Skorokhod map


The One-Dimensional Skorokhod Map1-D Skorokhod Map

1 φ(t) = ψ(t) + η(t) ≥ 0;2 η(0) = 0 and η is non-decreasing;3 “η increases only when φ is at zero”:

∫∞0 I{φ(s)>0}dη(s) = 0

Γ(ψ)(t) = φ(t) = ψ(t) + sups∈[0,t]

[−ψ(s)]+, t ≥ 0,

t0

φ

ψ


Representing the Solution in terms of the SkorokhodMap

Dynamics

dϕdt

=

{λ(t)− µ(t) if ϕ(t) > 0

[λ(t)− µ(t)] ∨ 0 if ϕ(t) = 0

Unconstrained Dynamicsdψdt

= λ(t)− µ(t)

Note that1 ϕ lies in R+

2 ϕ− ψ increases only at times when ϕ(t) = 0Therefore,

φ = Γ(ψ)


Heavy Traffic Limit Theorem

Goal: To estimate performance measures, e.g. average work in queueor waiting time of customer in queueFocus: on the case when the system is in “heavy traffic”:

“average rate of arriving work” ≈ “average service rate”

Approach: Embed the particular system of interest into a sequence ofsystems that are tending towards heavy traffic or instability


A Single Server Queue (GI/G/N) in Heavy Traffic

Description of the n-th system

• Renewal customer arrival process:A(n)(t) = cumulative number of customers arrived in [0, t ]

=: max{

k : S(n)k ≤ t

}where S(n)

k =∑k

i=1 τ(n)i , with τ (n)

1 , τ(n)2 , . . . – IID positive random

variables being the inter-arrival times

Example: A(n) is a Poisson process (when τ (n)1 ∼ exponential)




• Renewal customer arrival process:A(n)(t) = cumulative number of customers arrived in [0, t ]

=: max{

k : S(n)k ≤ t

}where S(n)

k =∑k

i=1 τ(n)i , with τ (n)

1 , τ(n)2 , . . . – IID positive random

variables being the inter-arrival timesExample: A(n) is a Poisson process (when τ (n)

1 ∼ exponential)




• Renewal customer arrival process:A(n)(t) = cumulative number of customers arrived in [0, t ]For example, A(n)(t) is a Poisson process

•Work arrival process:V (n)(k) = cumulative work brought by the first k customers

=:∑k

i=1 v (n)i ,

with v (n)1 , v (n)

2 , . . . – IID positive random variables being theservice times




• Renewal customer arrival process:A(n)(t) = cumulative number of customers arrived in [0, t ]For example, A(n)(t) is a Poisson process•Work arrival process:V (n)(k) = cumulative work brought by the first k customers

=:∑k

i=1 v (n)i ,

with v (n)1 , v (n)

2 , . . . – IID positive random variables being theservice times


The workload arrival process

(n)

(A(n)

(t))VW

(n)

(t)

v(n)

2

(n)

2τ

W (n)(t): amount of unfinished work (in the queue) at time tAssume that server processes work at rate 1 and is work-conserving:“server is never idle when there is unfinished work in the queue”


The netput process

(n)

(A(n)

(t))VW

(n)

(t)

v(n)

2

(n)

2τ

• Netput process:

N(n)(t) =: V (n)(A(n)(t))− t =

A(n)(t)∑i=1

v (n)i

Amount of work in queue if server is always processing at rate 1(ignores times when queue is empty and server is idling)


Unfinished work process

(n)

(A(n)

(t))VW

(n)

(t)

v(n)

2

(n)

2τ

• Netput process:

N(n)(t) =: V (n)(A(n)(t))− t

• Unfinished work process:W (n)(t) = Γ0(N(n))(t) = N(n)(t) + sup

s∈[0,t][−N(n)(s)]

where Γ0 is the reflection map or Skorokhod map on [0,∞)


Heavy Traffic Assumption

•λ(n) =: 1/Eτ (n)i −− Arrival rate •µ(n) =: 1/Ev (n)

i −− Service rate

HT Assumption: For some θ > 0,

ρ(n) =: λ(n)/µ(n) = 1− θ√n

• N(n)(t) =: V (n)(A(n)(t))− t W(n) = Γ0(N(n))• Same relation applies to scaled netput and workload:

N(n)(t) =: 1√n N(n)(nt), W (n)(t) =: 1√

n W (n)(nt)

W(n) = Γ0(N(n))

•Well known: N(n) ⇒ B∗, B∗(t) = σ2B(t)− θt , BM with drift

Theorem (RBM Heavy Traffic Limit for the Single Server Queue)

W (n) ⇒W ∗ =: Γ0(B∗) where B∗ is a Brownian motion with drift −θ


Some properties of reflected diffusions

W (n)(t) =:1√n

W (n)(nt)

HT condition ρ(n) =: λ(n)/µ(n) = 1− θ√n

W (n)(t) = θ−1(1− ρ(n))W (n)((θ(1− ρ(n))−2)t)

Theorem (Heavy Traffic Limit)

W (n) ⇒W ∗ =: Γ0(B∗) where B∗ is a Brownian motion with drift −θ

Stationary DistributionIf θ > 0 (i.e. negative drift), the reflected Brownian motion (RBM) W ∗

has a stationary distribution that is exponentially distributed with meanσ2/2|θ|, where σ2 is the unit variance of the Brownian motion B∗


A Size-Based Assignment Model (P. Glynn, M. Harchol-Balter

and K.R.

λ arrival rate of jobs to router, cΛ is the coeff. of variation ofjob-size distributiond servers, server i processes at a rate ai and

∑di=1 ai = 1

The router chooses cutoffs 0 = b0 < b1 < . . . < bd−1 < bd =∞and dispatches jobs with sizes in [bi−1,bi) to the i th server

EXOGENOUS

ARRIVALS

ROUTER

1

2

3

4

FCFS

FCFS

FCFS

FCFS

a1

a

a

a

2

3

4

λ


A Size-Based Assignment Model: An OptimizationProblem

GoalChoose cutoffs {bi} in a “congested” system so as to minimize theaverage sojourn time of a job in the system

Notation:Xi is the time it would take a rate 1 server to process job iX1,X2, . . . forms an i.i.d. sequencef (·) is the density of the distribution of Xi

The system load is defined to be ρ = λE[X1](recall λ is the arrival rate of jobs to the router)Assume ρ = 1 (heavy traffic)


An Intuitive Observation

An Observation:In heavy traffic it is clear that the optimal cutoffs should be0 = q0 < q1 < . . . < qd =∞ such that

λ

∫ qi

qi−1

xf (x)dx = ai

i.e., arrival rate to the i th server is equal to its capacityIn other words, each individual server is in heavy traffic

A Natural Conjecture:For heavily loaded systems (i.e. with ρ close to 1), the optimal cutoffshould be a suitable “perturbation” of the qi


A family of systems indexed by the load ρ

The ρth system has arrival rate λ, but has job density

fρ(u) =1ρ

f (u/ρ)

The “balanced” cutoffs qi(ρ), which are defined by

λ

∫ qi (ρ)

qi−1(ρ)xf (x)dx = ρai

then satisfy qi(ρ) = ρqi

Heavy traffic theory of the single server queue suggests thatoptimal cutoffs would be of the form

bi(ρ).

= ρqi + (1− ρ)ri

for some ri ∈ R to be determined


What the heavy traffic limit theorem yields

Define Wi(ρ, t) to be the unfinished work in the i th buffer at time tin the ρth system.Let

W ρi (t) = Wi(ρ, (1− ρ)−2t)

Theorem (Consequence of the Heavy Traffic Limit:)As ρ ↑ 1,

(1− ρ)W ρi ⇒ Zi

where Zi is a reflected Brownian motion (RBM) with drift θi andvariance σ2

i given explictly in terms of the problem data

by

θi = λqi f (qi)ri − λqi−1f (qi−1)ri−1 − qi

and

σ2i = λ

∫ qi

qi−1

x2f (x)dx +1λ

a2i (c2

Λ − 1)


What the heavy traffic limit theorem yields

Define Wi(ρ, t) to be the unfinished work in the i th buffer at time tin the ρth system.Let

W ρi (t) = Wi(ρ, (1− ρ)−2t)

Theorem (Consequence of the Heavy Traffic Limit:)As ρ ↑ 1,

(1− ρ)W ρi ⇒ Zi

where Zi is a reflected Brownian motion (RBM) with drift θi andvariance σ2

i given explictly in terms of the problem data by

θi = λqi f (qi)ri − λqi−1f (qi−1)ri−1 − qi

and

σ2i = λ

∫ qi

qi−1

x2f (x)dx +1λ

a2i (c2

Λ − 1)


An approximate solution to the optimization problem

The corresponding approximation for the expected sojourn time canthen be expressed explicitly in terms of the unknowns ri :

11− ρ

d∑1=1

d∑i=1

αi

ai + γi−1ri−1 − γi ri,

for some known constants αi and γi .The explicit minimisation problem can then be solved explicitly

r∗i =1γi

i∑j=1

aj −∑i

j=1 α12j∑d

k=1 α12k

for 1 ≤ i ≤ d − 1.

• HT approximation agrees well with exact values, e.g., when d = 2


Summary

Study of multi-class queueing networks with FCFS(first-come-first-serve) service within each class leads to the studyof deterministic and stochastic dynamics which displaydiscontinuitiesPerformance measures for queueing networks can often beapproximated by functionals involving reflected Brownian motionsThe study of stochastic networks has led to the development of atheory for• uniqueness of a class of ODEs with discontinuous drift• reflected diffusions in non-smooth domains


References: RBM approximations of stochasticnetworks

Weak Convergence• Billingsley, Convergence of Probability Measures, ’99• Parthasarathy, Probability Measures on Metric Spaces, ’67.Properties of the Skorokhod Map• K. Burdzy, K. Ramanan and W. Kang, “The Skorokhodproblem in a time-dependent interval”, Stochastic Processes andtheir Applications 119 2009 428–454• L. Kruk, J. Lehoczky, K.Ramanan and S. Shreve, “An explicitformula for the Skorokhod map on [0,a]”, Ann. Probab., 18 20081:22–58• K.Ramanan “Reflected diffusions defined via the extendedSkorokhod map”, EJP, 11, 2006, 934–992.• Ramasubramanian “A subsidy-surplus model and theSkorokhod problem in an orthant.” Math. Oper. Res. 25 2000,3:509–538.


References: RBM approximations of stochasticnetworks (contd.)

Properties of the Skorokhod Map (contd.)• P. Dupuis and K.Ramanan, “Convex Duality and the SkorokhodProblem,” Probab. Theor. Rel. Fields, 115, 1999, Part I: 153–195,Part II: 197–236.• Dupuis and Ishii, “Lipschitz continuity of the solution mappingot the Skorokhod problem”, Stochastics, 35 1991, 35–62.• Harrison and Reiman, “Reflected Brownian motion in theorthant,” Annals of Probab., 9 1981, 302–308.Books on Heavy traffic limits of queueing networks• Kushner Heavy traffic analysis of controlled queueing andcommunication networks, ’01• Chen and Yao Fundamentals of Queueing Networks, ’01• Harrison Brownian Motion and Stochastic Flow Systems, ’85.


Approximations of Stochastic Networks - IIT Bombay · Routing structure Approximations of...

Documents

Transcript of Approximations of Stochastic Networks - IIT Bombay · Routing structure Approximations of...