Stochastic processing networks:steady-state 23 diffusion approximations

Jim Dai

School of Operations Research and Information EngineeringCornell University

andInstitute for Data and Decision Analytics (iDDA)

Chinese University of Hong Kong, Shenzhen

May 14, 2018

Jim Dai 1 / 37

Collaborators

Masakiyo Miyazawa, Department of Information Sciences, Tokyo Universityof Science

Anton Braverman, Kellogg School of Business, Northwestern University

Chang Cao, Department of Statistical Science, Cornell University

Xiangyu Zhang, School of ORIE, Cornell University

Jim Dai 2 / 37

An example of generalized Jackson networks (GJNs)

ExternalArrivalsλr

Re,1(t)L1 Rs,1

L2 Rs,2

L3 Rs,30.7

0.3

0.5

0.5

P =

0 .5 .51 0 0.7 0 0

Jim Dai 3 / 37

Indirect method: limit interchange for GJN

L(t) L(∞)

L(r)(t) L(r)(∞)

r → 0 r → 0

t →∞

t →∞

Harrison-Williams (1987)

Reiman (1984)L is an SRBM

Down-Meyn (1994)

Gamarnik-Zeevi (2006)

Jim Dai 4 / 37

Gamarnik-Zeevi (2006): indirect method

Budhiraja and Lee (2009): second moments and uniform integrability

Gurvich (2014), multiclass queueing networks

Ye-Yao (2015), (head-of-line) bandwidth sharing networks

There is a growing literature: Tezcan (2008), Zhang-Zwart (2008), Katsuda(2010), Gamarnik-Stolyar (2012), D-Dieker-Gao (2014), and more...

Jim Dai 5 / 37

Basic adjoint relationship (BAR): three direct methods

Stein’s method; Gurvich (2014), Braverman-Dai (2017)Strongest results, but difficult for general systems

Moment generating function (mgf)-BAR approach: Miyazawa (2015),Braverman-Dai-Miyazawa (2017) for GJN

Drift method (Quadratic-BAR approach): Eryilmaz and Srikant (2012,Maguluri-Srikant (2016), Wang-Maguluri-Srikant-Ying (2017)

Surprisingly successful for single-pass systems,

Jim Dai 6 / 37

A two-link bandwidth sharing network

S1B3

B1S2

B2

class 3departures

class 1departures

class 2departures

class 1arrivals

class 2arrivals

class 3arrivals

Insensitivity of jobsize distributions in heavy traffic.Full version, beyond moments, of the conjecture remains open a problem.

Jim Dai 7 / 37

Jim Dai 8 / 37

Today’s topics

TopicsMgf approach for multiclass queueing networks under priority policies:Braverman, Dai and Miyazawa.

Strong state space collapse: Cao, Dai, Miyazawa, Zhang.

OutlineAn illustrative theorem for a re-entrant line

Mgf approach for GI/GI/1 queues

Proof sketches

An open problemSelf-contained tools

Jim Dai 9 / 37

A 2-station, 3-class reentrant line

S1B1 B2 S2

B3class 1arrivals

class 3departures

{(Te(i),Ts,1(i),Ts,2(i),Ts,3 (i)

), i ≥ 1

}is an iid sequence with mean

(1,m1,m2,m3) and finite second moment.The interarrival times are Te(i), satisfying the heavy traffic condition

λ = 1− r , m1 + m3 = 1, m2 = 1, r ↓ 0.

Load parameters

βi = λmi , i = 1, 2, 3, ρ1 = β1 + β3 = λ < 1, ρ2 = β2 = λ < 1.

Jim Dai 10 / 37

System dynamics

Assuming (preemptive-resume) LBFS policy, one has the Markovianrepresentation: X = {X (t)}t≥0 is Markov process, where state at time t is

X (t) = (L(t),Re(t),Rs(t)).

Li (t) = number of customers in buffer i at time t (including the one inservice)Re(t) = residual time until next exogenous arrivalRs,i (t) = residual time until next service completion in buffer iThe reentrant in the figure has a 7-dimensional representation:(

L1(t),Re(t),Rs,1(t)),(L2(t),Rs,2(t)

),(L3(t),Rs,3(t)

)

Jim Dai 11 / 37

A sample result

When λ = 1− r < 1, Markov process X is positive Harris recurrent (Dai1995): as t →∞,

Lr (t) =⇒ Lr (∞) = (Lr1(∞), Lr

2(∞), Lr3(∞)).

Theorem 1

rLr (∞) =⇒(

L∗1(∞), L∗2(∞), 0)

as r → 0.

rLr3(∞)⇒ 0, state space collapse (SSC).(

rLr1(∞), rLr

2(∞))

=⇒(

L∗1(∞), L∗2(∞)).

Jim Dai 12 / 37

State space collapse

The pre-limit is a K -dimensional problem, and the limit is a d-dimensionalproblem, where K is the number of buffers and d is the number of stations.

IMPL

AN

T_O

X

LT

O

OX

IDE

_1

BA

RR

IER

_OX

POL

Y_D

EP

POL

Y_D

OPE

INT

ER

GA

TE

CR

IT_D

EV

VW

R_O

VE

N

ME

D_C

UR

RE

NT

_IM

P

NO

NC

RIT

_DE

V

PEA

KR

EFL

OW

BPS

G

MA

TR

IX

HIG

H_C

UR

RE

NT

_IM

P

85 S

TA

ION

S2

PRO

DU

CT

SPR

OD

UC

T1:

210

ST

EPS

PRO

DU

CT

2: 2

45 S

TE

PSIN

TH

E F

IGU

RE

:

16

ST

AT

ION

S

48 S

TE

PS

OR

IGIN

AL

:

Jim Dai 13 / 37

Strong SSC

In proving a version of Theorem 1 for general multiclass queueing networksunder priority policies, we need the following strong SSC:

rEν [Lr3(∞)]→ 0.

Theorem 2Assume interarrival and service times have phase-type distributions. Assume thatthe critically loaded fluid model has SSC. Then

supr∈(0,r0]

∑i∈H

Eν [Lri (∞)]p <∞ for p ≥ 1,

which implies the strong SSC.

Jim Dai 14 / 37

Fluid model SSC

S1B1 B2 S2

B3class 1arrivals

class 3departures

Fluid model (L1(0) can be ∞ )

L1(t) = L1(0) + t − µ1T1(t),L2(t) = L2(0) + µ1T1(t)− µ2T2(t),L3(t) = L3(0) + µ2T2(t)− µ3T3(t),

where µi = 1/mi is service rate for class i jobs.H = {3}. When L3(t) > 0, T3(t) = 1, which implies

L3(t) = µ2T2(t)− µ3 ≤ µ2 − µ3 < 0.

L3(t) = 0 for t ≥ L3(0)/(µ3 − µ2).Jim Dai 15 / 37

LBFS priority policy?

- -

-

-

-

-

m1 m2

m3 m4

m5

Station 1 Station 2α1

Under the LBFS-FBFS priority policy and λ(m2 + m5) > 1,

Time0 4000 8000 12000

Job

coun

t

0

1000

2000

3000

4000Station 1Station 2

Jim Dai 16 / 37

Miyazawa’s mgf approach

For a family of nonnegative random vector L(r)(∞) ∈ Rd+,

L(r)(∞)⇒ L(∞) for some random vector L(∞)

if and only if the mgf converges

φ(r)(θ) = E[e〈θ,L(r)(∞)〉]→ E[e〈θ,L(∞)〉] for all θ ≤ 0.

The family {L(r)(∞)} is tight iff for any sequence rn → 0,

limn→∞

φ(rn)(θ)→ φ(θ) ∀θ ≤ 0 implies that

φ(0−) = φ(0−, . . . , 0−) = 1. (1)

Equation (1) says φ(·) is left continuous at 0.

Jim Dai 17 / 37

GI/GI/1 queue

S1B1 departuresarrivals

{Te(i), i ≥ 1} iid interarrival times; λ = 1/E[Te(i)];{Ts(i), i ≥ 1} iid service times; µ = 1/E[Ts(i)].Heavy traffic condition

λ = µ− r with r ↓ 0.

X = {X (t), t ≥ 0} is a Markov process, where

X (t) = (L(t),Re(t),Rs(t)),

whereL(t) is the number of customers in system,Re(t) is remaining interarrival times,Rs(t) is the remaining service times.

Jim Dai 18 / 37

Piecewise deterministic Markov process (PDMP)

The process X = (L,Re ,Rs) is a piecewise deterministic Markov process(PDMP); Davis (1981)A sample path of a PDMP is composed of two parts, deterministic andcontinuous sections and (random) jumps due to expiration of remainingtimes.

0t

Te(1) Te(2) Te(3) Te(4)

Ts(4)

Ts(3)

Ts(2)

Te(5)

Ts(1)

L(t)

Re(t)Rs(t)

Figure: A sample path of remaining times for GI/G/1 queue

Jim Dai 19 / 37

Change of variables for PDMP

Consider function f (x) = f (z , u, v) : Z+ × R+ × R+ → R.Define the jump size ∆f (X (s)) = f (X (s))− f (X (s−)),

f (X (t))− f (X (0)) =∫ t

0

dds f (X (s)) ds +

∑si∈(0,t]

∆f (X (si ))

= −∫ t

0

[ ∂∂u f (X (s))− 1{Z (s) > 0} ∂

∂v f (X (s))]ds (continuous)

+[∫ t

0∆f (X (s))dNA(s) +

∫ t

0∆f (X (s))dND(s)

](jumps)

NA – arrival process.ND – departure process.

Jim Dai 20 / 37

Full BAR in GI/GI/1 setting

Assume X has stationary distribution ν.

Basic Adjoint Relationship (BAR)

0 =tEν[− ∂

∂u f (X (0))− 1{Z (0) > 0} ∂∂v f (X (0)

]+ (continuous)

+Eν

[∫ t

0∆f (X (s))dNA(s) +

∫ t

0∆f (X (s))dND(s)

](jumps)

Jump terms are intractable; getting rid of arrival-jump term requires

ETe

[f (L + 1,Te ,Rs)− f (L, 0,Rs)

]= 0 (2)

for any given L and Rs .Similarly,

ETs

[f (L− 1,Re ,Ts)− f (L,Re , 0)

]= 0 (3)

for any given L ≥ 1 and Re .Jim Dai 21 / 37

Exponential functions (I)

Fix a θ ≤ 0, take

f (θ; z , u, v) = eθz+a(θ)u+bθv .

Then

E[f (z + 1,Te , v)− f (z , 0, v)] = 0,Eeθ(z+1)+a(θ)Te+b(θ)v = eθz+b(θ)v ,

E[eθ+a(θ)Te

]= 1.

For each θ ≤ 0, find a = a(θ) such that

E[ea(θ)Te

]= e−θ. (4)

Jim Dai 22 / 37

Exponential functions (II)

Similarly, for each θ ≤ 0, find b(θ) such that

E[eb(θ)Ts ] = eθ. (5)

Then

E[f (z − 1, u,Ts)− f (z , u, 0)] = 0 for z ≥ 1.

Jim Dai 23 / 37

Exponential functions (III): Summary

Define a(θ) and b(θ) via (4) and (5). Set

f θ; z , u, v) = eθz+a(θ)u+b(θ)v .

Then,

∂

∂u f (θ; z , u, v) = a(θ)f (θ; z , u, v),

∂

∂v f (θ; z , u, v) = b(θ)f (θ; z , u, v).

Thus, BAR becomes

Eν[− a(θ)f (θ; X (0))− b(θ)1{L(0)>0}f (θ; X (0))

]= 0,

or equivalently

[a(θ) + b(θ)]Eν [f (θ; X (0))]− b(θ)Eν[1{L(0)=0}f (θ; X (0))

]= 0. (6)

Jim Dai 24 / 37

Prelimit (restricted) BAR

From (6), for θ ≤ 0,

[a(θ) + b(θ)]Eν [f (θ; X (0))]− b(θ)Pν(L(0) = 0)Eν[f (θ; X (0))|L(0) = 0

]= 0.

[a(θ) + b(θ)]Eν [f (θ; X (0))]− b(θ)(1− λ/µ)Eν[f (θ; X (0))|L(0) = 0

]= 0.

Scaling: changing θ to rθ for any θ ≤ 0 to get pre-limit BAR,

[a(rθ) + b(rθ)]φ(r)(θ)− b(rθ)(1− λ/µ)φ(r)0 (θ) = 0, (7)

where

φr (θ) ≡ E[erθLr (0)+a(rθ)Re(0)+b(rθ)Rs (0)] ≈ E[eθ(

rLr (0))],

φr0(θ) ≡ Eν

[f (rθ; X (0))|L(0) = 0

]= E[ea(rθ)Te(0)+b(rθ)Ts (0)] ≈ 1.

Jim Dai 25 / 37

Asymptotic expansion of coefficients in (7)

As θ → 0,

a(θ) ≈ −λ(θ + c2

e θ2/2), (8)

where

λ = 1E(Te) , c2

e = Var(Te)(E(Te))2 .

Fix θ ≤ 0. Using (8), as r → 0,

a(rθ) + b(rθ) ≈ −λ(rθ + r2θ2c2e /2)− µ(−rθ + r2θ2c2

s /2)= (µ− λ)rθ − r2θ2(λc2

e + µc2s )/2

= r2θ − r2θ2(λc2e + µc2

s )/2≈ r2θ − r2θ2µ(c2

e + c2s )/2. ( using λ = µ− r)

Also

− b(rθ)(1− λ/µ) ≈ (−µrθ + 1/2r2θ2µc2s )(1− λ/µ)

≈ (−µrθ)(1− λ/µ) = −r2θ.

Jim Dai 26 / 37

Limit BAR

Recall prelimit BAR (7)

[a(rθ) + b(rθ)]φ(r)(θ)− b(rθ)(1− λ/µ)φ(r)0 (θ) = 0.

Assume that as r → 0,

φr → φ, φr0 → φ0 = 1.

Dividing (7) by r2 and taking limit as r → 0, one has the following limit BARfor φ and φ0:[

θ2(λc2e + µc2

s )/2− θ]φ(θ) + θφ0(θ) = 0 for θ ≤ 0. (9)

Jim Dai 27 / 37

Analysis from the limit BAR (9)

Thus, for θ < 0, [θµ(c2

e + c2s )/2− 1

]φ(θ) + φ0(θ) = 0,

or

φ(θ) = φ0(θ)[θµ(c2

e + c2s )/2− 1

] = 1[θµ(c2

e + c2s )/2− 1

] ,Take θ ↑ 0, one has φ(0−) = φ0(0−) = 1, concluding that {rLr (0)} is tight.

Furthermore, as r → 0,

rLr (0)⇒ exponential with mean µ(c2e + c2

s )/2.

Jim Dai 28 / 37

Mgf approach works for generalized Jackson networks

ExternalArrivals

Re,1L1 Rs,1

L2 Rs,2

L3 Rs,30.7

0.3

0.5

0.5

P =

0 .5 .51 0 0.7 0 0

Braverman-Dai-Miyazawa (2017)

Jim Dai 29 / 37

A sketch for proving Theorem 1

S1B1 B2 S2

B3class 1arrivals

class 3departures

f (θ,X (0)) = exp( 3∑

i=1θiLi (0) + a(θ1)Re(0)

+ b1(θ1 − θ2)Rs,1(0) + b2(θ2 − θ3)Rs,2(0) + b3(θ3)Rs,3(0)).

E[ea(θ1)Te

]= e−θ1 , E[eb1(θ1−θ2)Ts,1 ] = eθ1−θ2 ,

E[eb2(θ2−θ3)Ts,2 ] = eθ2−θ3 , E[eb3(θ3)Ts,3 ] = eθ3 .

Jim Dai 30 / 37

Pre-limit BAR

Restricted BAR

a(θ1)E[f (θ,X )] + b1(θ1 − θ2)E[f (θ,X )1{L1>0,L3=0}

]+ b2(θ2 − θ3)E

[f (θ,X )1{L2>0}

]+ b3(θ3)E

[f (θ,X )1{L3>0}

]= 0,

which is equivalent to[a(θ1) + (1− β3)b1(θ1 − θ2) + b2(θ2 − θ3) + β3b3(θ3)

]E[f (θ,X )]

+[b1(θ1 − θ2)− b3(θ3)

](E[f (θ,X )1{L3=0}

]− (1− β3)E

[f (θ,X )

])(10)

− b1(θ1 − θ2)E[f (θ,X )1{L1=0,L3=0}

]− b2(θ2 − θ3)E

[f (θ,X )1{L2=0}

]= 0.

Replacing θ by rθ, define

φr (θ) = E[f (rθ,X )], φr1(θ) = E

[f (rθ,X )|L1 = 0, L3 = 0

],

φr2(θ) = E

[f (rθ,X )|L2 = 0

].

Jim Dai 31 / 37

State space collapse

Choosing θ3 to aid the removal of (10).Conduct expansion

b1(r(θ1 − θ2)

)≈ µ1

(r(θ1 − θ2)− r2(θ1 − θ2)2c2

s,1/2)),

b3(rθ3) ≈ µ3

(rθ3 − r2θ2

3c2s,3/2

).

Choose

θ1 ≤ 0, θ2 ≤ 0, θ3 = µ1µ3

(θ1 − θ2) ≤ 0. (11)

There are “enough” points θ = (θ1, θ2, θ3) satisfying (11).

Jim Dai 32 / 37

Limit BAR

Assume that, for any θ1 ≤ θ2 ≤ 0, and θ3 = (µ1/µ3)(θ1 − θ2),(φr (θ), φr

1(θ), φr2(θ)

)→(φ(θ1, θ2, θ3), φ1(θ2), φ2(θ1, θ3)

).

The limit satisfies

γ(θ1, θ2, θ3)φ(θ1, θ2, θ3) + (θ2 − θ3)φ2(θ1, θ3) + µ3θ3φ1(θ2) = 0

where

γ(θ1, θ2, θ3) = µ1(θ1 − θ2) + (θ2 − θ3) + quadratic term of θ.

Jim Dai 33 / 37

Tightness

Assume for any θ1 ≤ θ2 ≤ 0, and θ3 = (µ1/µ3)(θ1 − θ2). Ignoring the quadraticterm,

(θ2 − θ3)(φ2(θ1, θ3)− φ(θ1, θ2, θ3)

)+ µ3θ3

(φ1(θ2)− φ(θ1, θ2, θ3)

)= 0.

Setting θ2 = u ↑ 0, θ3 = −u2, and

θ1 = u − µ3µ1

u2 < 0,

one has

φ(0−, 0−, 0−) = φ2(0−, 0−).

Setting θ2 = 0 and using φ1(0) = 1, one has

φ(0−, 0, 0−)− φ2(0−, 0−) + µ3(1− φ(0−, 0, 0−)) = 0,

which implies that φ(0−, 0−, 0−) = φ2(0−, 0−) = φ(0−, 0, 0−) = 1.

Jim Dai 34 / 37

BAR for a 5-class reentrant line

- -

-

-

-

-

m1 m2

m3 m4

m5

Station 1 Station 2α1

a(θ1)E[f (θ; X (0)

]+ b1(θ1 − θ2)E

[f (θ; X (0)1{L1>0,L3=0,L5=0}

]+ b2(θ2 − θ3)E

[f (θ; X (0)1{L2>0,L4=0}

]+ b3(θ3 − θ4)E

[f (θ; X (0)1{L3>0,L5=0}

]+ b4(θ4 − θ5)E

[f (θ; X (0)1{L4>0}

]+ b5(θ5)E

[f (θ; X (0)1{L5>0}

]= 0.

Jim Dai 35 / 37

Choice of θ3, θ4 and θ5 to aid SSC

µ1(θ1 − θ2) = µ3(θ3 − θ4) = µ5θ5, µ2(θ2 − θ3) = µ4(θ4 − θ5). (12)

“Not enough points θ < 0” satisfying (12).

Truncation on L3, L4, L5 is needed to allow positive values of θ3, θ4, θ5. Forexample, even if θ3 > 0,

erθ3Lr3(0) ≤ eθ3 if L3(0) < 1/r .

Sufficient to have

sup(r∈(0,r0])

E[Lr3(0)] <∞.

Jim Dai 36 / 37

Open problem: not using Theorem 2

In proving a version of Theorem 1 for general multiclass queueing networksunder priority policies, we need the following strong SSC:

rEν [Lr3(∞)]→ 0.

When all distributions are exponential, the mean drift

E[Lr3(n + 1)]2|Lr

3(n) = x ]− x2 ≤ C1 − C2x ,

where C1,C2 > 0 do not depend on r . Thus,

E[Lr3(∞)] ≤ C1

C2.

SSC

rLr3(∞)⇒ 0.

can be proved using the BAR for this case. General?

Jim Dai 37 / 37