Stochastic vs. Deterministic Models for

Systems with Delays

H.T. Banks∗Jared Catenacci

∗∗Shuhua Hu

∗∗∗

∗ Center for Research in Scientific Computation, North Carolina StateUniversity, Raleigh, NC 27695-8212 (e-mail:htbanks@ncsu.edu)

∗∗ (jwcatena@ncsu.edu)∗∗∗ (shu3@ncsu.edu)

Abstract: We consider population models with nodal delays which result in dynamical systemswith delays. For small population models the appropriate models are discrete stochastic systemswith delays. We consider these delay systems and present new theoretical and computationalresults for such systems. In particular, in this note we summarize results on the effects ofdifferent types of delays (a fixed delay and a random delay) on the dynamics of stochasticsystem as well as their relationship with each other in the context of a just-in-time networkmodel. In addition, we numerically explore the corresponding deterministic approximations forthe stochastic systems with these two different types of delays.

Keywords: Stochastic Markov chains, ordinary differential equations, delay equations

1. INTRODUCTION

Continuous time Markov Chain models are widely usedto model physical and biological processes (e.g., see [1,3]).These models are typically used when dealing with dy-namic systems involving low species count. However, be-cause simulations are quite expensive for large populationstochastic systems, in such models one often wishes toknow whether or not the stochastic system can be approx-imated by a deterministic one when the population size issufficiently large. Theory established by Kurtz (e.g. [11–15]) gives a way to construct a deterministic system toapproximate density dependent continuous time MarkovChains as the population size grows large (this resultis often called Kurtz’s limit theorem). In general, deter-ministic systems are much easier to analyze compared tostochastic systems. Techniques, such as parameter estima-tion methods, are well developed for deterministic systems,whereas parameter estimation is much more difficult in astochastic framework. For example, in [16], the authorsdeveloped a method for estimating parameters in dynamicstochastic (Markov Chain) models based on Kurtz’s limittheory coupled with inverse problem methods developedfor deterministic dynamical systems and illustrated theseideas in the context of disease dynamics. The methodologyrelies on finding an approximate large-population behaviorof an appropriate scaled stochastic system. The approachdetailed there leads to a deterministic approximation ob-tained as solutions of rate equations (ordinary differentialequations) in terms of the large sample size average oversample paths or trajectories (limits of pure jump Markov

⋆ This research was supported in part by Grant Number NIAIDR01AI071915-10 from the National Institute of Allergy and Infec-tious Diseases, in part by the Air Force Office of Scientific Researchunder grant number AFOSR FA9550-12-1-0188, and in part by theNational Science Foundation under Research Training Grant (RTG)DMS-0636590.

processes). Using the resulting deterministic model theauthors discussed how to select parameter subset combina-tions that can be estimated using an ordinary-least-squares(OLS) or generalized-least-squares (GLS) inverse problemformulation with a given data set. The authors illustratedthe ideas with a stochastic model for the transmissionof vancomycin-resistant enterococcus (VRE) in hospitalsand VRE surveillance data from an oncology unit. Thedynamics of the VRE colonization of patients in a hospitalunit are modeled as a continuous time Markov Chain(MC) with discrete state space embedded in R3. Moreimportantly the transition parameters in such models areprecisely the coefficients in the large population mean dy-namics. This leads to rather obvious parameter estimationtechniques for continuous time Markov Chain models.

Delays occur and are important in many physical andbiological processes, but especially in modeling of just-in-time networks with transport delays. For example, recentstudies show that delayed-induced stochastic oscillationscan occur in certain elements of gene regulation networks[10]. In addition, delays are of practical importance ingeneral supply networks for investigation of a wide rangeof perturbations (either accidental or deliberate) such asa node being rendered inoperable due to bad weather,and technical difficulties in communication system. Hence,continuous time Markov chain models with delays incorpo-rated (simply referred to stochastic models with delays inthis note) have enjoyed considerable research attention inthe past decade, especially the efforts on the developmentof algorithms to simulate such systems (e.g., [2,8,10,17]).However, it appears that there is only minimal effort onthe convergence of stochastic solutions to deterministicsystems with delays as the sample size goes to infinity (thatis, the analogy of the Kurtz’s limit theorem). We foundtwo works is this spirit given in [9] and [19]. Specifically,Bortolussi and Hillston [9] extended the Kurtz’s limit the-

orem to the case where fixed delays are incorporated into adensity dependent continuous time Markov chain. Schlichtand Winkler [19] showed that if all the transition ratesare linear, then the mean solution of stochastic systemwith random delays can be described by deterministicdifferential equations with distributed delays. However, toour knowledge, there is still no theoretical results on theconvergence of the solution to a general stochastic systemwith random delays; that is, there is not yet an analog ofthe Kurtz’s limit theorem for a general stochastic systemwith random delays. In this note we discusses these issuesin the context of a pork production supply network. Thismodel was originally developed in [4], where four nodes ofproduction are considered: sows, nurseries, finishers, andslaughterhouses. The movement of pigs from one node tothe next is assumed to occur only in the forward direction.That is, from sows to nurseries, from nurseries to finishersand from finishers to slaughterhouses

2. THE PORK PRODUCTION NETWORK MODELWITH A FIXED DELAY

The assumption made on the original pork productionnetwork model derived and studied in [4] was that the tran-sition from one node to the next is made instantaneously. Itwas made clear in [4] that this is a simplifying assumption,and that incorporating delays would give a more realisticmodel due to the possible long distance between the nodesor bad weather or some other disruptions/interruptions.Presented here is a first attempt to account for delaysin the following way. Assume that all transitions occurinstantaneously except for the arrival of pigs transitioningfrom Node 1 to 2. That is, the pigs leave Node 1 immedi-ately, but the time of arrival at Node 2 is delayed. We onlyconsider a delay in one of the transitions for simplicity, butdepending on the physical proximity of the nodes, it maybe a reasonable assumption to have delays in not all of thetransition.

X1(t) = X1(0) − Y1

t∫

0

λ1(X(s))ds

+Y4

t∫

0

λ4(X(s))ds

X2(t) = X2(0) − Y2

t∫

0

λ2(X(s))ds

+Y1

t∫

0

λ1(X(s− τ))ds

Xi(t) = Xi(0) − Yi

t∫

0

λi(X(s))ds

+Yi−1

t∫

0

λi−1(X(s))ds

i = 3, 4.

(1)

Here {Yj(t), t ≥ 0}, j = 1, 2, 3, 4 are independent stan-dard Poisson processes, and transition rates λj(x) =kjxj(Lj+1−xj+1)+, j = 1, 2, 3 and transition rate λ4(x) =k4 min(x4, Sm), where (z)+ = max(z, 0), ki and Li respec-tively denote the service rate and capacity constraint atNode i, and Sm is the maximal exit constraint at Node 4.

2.1 The Stochastic Model With A Fixed Delay

As a first consideration, we take the delayed time of arrivalat node 2, τ , to be a fixed value. If we assume that theprocess starts from t = 0 (that is, X(t) = 0 for t < 0),then this results in a stochastic model with a fixed delaygiven by system (1). Note that λ1 only depends on thestate. Hence, the assumption of X(t) = 0 for t < 0 leadsto λ1(X(t)) = 0 for t < 0. The interpretation of stochasticmodel (1) with a fixed delay is as follows. When any ofthe transitions λ2, λ3 or λ4 fires at time t, the systemis updated accordingly at time t. When the transition λ1

fires at time t, one unit is subtracted from the first node.Since the completion of the transition is delayed, at timet+ τ the unit is added onto Node 2.

2.2 The Corresponding Deterministic System For TheStochastic Model With A Fixed Delay

In [9], Bortolussi and Hillston extended the Kurtz’s limittheorem to a scenario with fixed delays incorporatedinto a density dependent continuous time Markov chain,where the convergence is in the sense of convergencein probability. We will use our pork production model(1) to illustrate this theorem (referred to as BH limittheorem). An approximating deterministic system canbe constructed based on the BH limit theorem for ascaled stochastic system with a fixed delay. Let C(t) =X(t)/N with X(t) described by (1). This approximatingdeterministic system is given by

c1(t) = −κ1c1(t)(l2 − c2(t))+ + κ4 min(c4(t), sm)c2(t) = −κ2c2(t)(l3 − c3(t))+

+κ1c1(t− τ)(l2 − c2(t− τ))+c3(t) = −κ3c3(t)(l4 − c4(t))+ + κ2c2(t)(l3 − c3(t))+c4(t) = −κ4min(c4(t), sm) + κ3c3(t)(l4 − c4(t))+,

(2)

where (z)+ = max(z, 0), κ4 = k4, sm = SM/N andκi = Nki, i = 1, 2, 3. We note that this approximating de-terministic system is not a system of ordinary differentialequations, but rather a system of delay (ordinary) differ-ential equations with a fixed delay. The delay differentialequation is a direct result of the delay term present in(1). Since there is a delay term, the system is dependenton the previous states, for this reason it is necessary tohave some past history functions as initial conditions. Itshould be noted that past history functions should not bechosen in an arbitrary fashion as they should capture thelimit dynamics of the scaled stochastic system with a fixeddelay.

Now we illustrate how to construct the initial conditionsfor the delay differential equation (2). Noted that inthe interval [0, τ ] the delay term has no affect (X(t) =0, t < 0 !), thus we can ignore the delay term in thisinterval. This yields a stochastic system with no delays, theconcentration of which can be approximated by a systemof ODE’s. This gives the deterministic system (as in [4])

c1(t) = −κ1c1(t)(l2 − c2(t))+ + κ4 min(c4(t), sm)c2(t) = −κ2c2(t)(l3 − c3(t))+c3(t) = −κ3c3(t)(l4 − c4(t))+ + κ2c2(t)(l3 − c3(t))+c4(t) = −κ4min(c4(t), sm) + κ3c3(t)(l4 − c4(t))+c(0) = c0

(3)

for t ∈ [0, τ ], and let Φ(t) denote the solution to (3). Thuswe have that C(t) converges to Φ(t) as N → ∞ on theinterval [0, τ ].

In the interval [τ, tf ], where tf is the final time, the delayhas an affect, so we approximate with the DDE system(2), and the solution Φ(t) to the ODE system (3) on theinterval [0, τ ] serves as the initial function. Explicitly thesystem can be written as

c1(t) = −κ1c1(t)(l2 − c2(t))+ + κ4 min(c4(t), sm)c2(t) = −κ2c2(t)(l3 − c3(t))+

+κ1c1(t− τ)(l2 − c2(t− τ))+c3(t) = −κ3c3(t)(l4 − c4(t))+ + κ2c2(t)(l3 − c3(t))+c4(t) = −κ4min(c4(t), sm) + κ3c3(t)(l4 − c4(t))+c(s) = Φ(s), s ∈ [0, τ ].

(4)

The BH limit theorem indicates that C(t) converges inprobability to the solution of (4) as N → ∞.

2.3 Comparison Of The Stochastic Model With A FixedDelay And Its Corresponding Deterministic System

In this section we compare the results of the stochasticsystem (1) with a fixed delay to its corresponding deter-ministic system (in terms of number of pigs, i.e., Nc(t)with c(t) being the solution to (4)). The stochastic systemwith a fixed delay (1) was simulated using an algorithmgiven in detail in [6], and the deterministic system (4)was solved numerically using a linear spline approximationmethod (e.g., see [5,7] for details).

All parameter values and initial conditions are taken from[6]. The value of the delay was set to be τ = 5. Two samplesizes were considered, N = 100 and N = 10,000, for thestochastic system. In Figures 1 and 2 the deterministicapproximation (Nc(t) with c(t) being the solution to (4))is compared to five typical sample paths of solution tostochastic system (1) as well as to the mean solution forthe stochastic system (1), where the mean solution wascalculated by averaging 10,000 sample paths. It is clearthat the trajectories of the stochastic simulations followthe solution of its corresponding deterministic system,and the variance among sample paths of the stochasticsolution decreases as the sample size increases. It is alsoseen that as the sample size increases the mean solutionof the stochastic system become closer to that of itscorresponding deterministic system. Convergence rates inNodes 3 and 4 are even more rapid as is depicted in [6].

3. THE PORK PRODUCTION NETWORK MODELWITH A RANDOM DELAY

Now we return to the issue of how to implement randomdelays into the original pork production model. In theprevious section we assumed that the delay was fixed. Theinterpretation of this is that every transition from Node1 to Node 2 was delayed by the same amount of time.Now, we want to consider the amount of delayed timevarying at each transition. The motivation for doing sois that in practice we would expect that the amount oftime it takes to travel from Node 1 to Node 2 will varybased on a number of conditions, e.g. weather, traffic, roadconstruction, etc. In this case it may not be a reasonableassumption that every transition is delayed by the same

0 10 20 30 40 500

5

10

15

20

25

30N = 100

Time (Days)

Pig

s

X1 (S)

X1 (D)

0 10 20 30 40 505

10

15

20

25

30N = 100

Time (Days)

Pig

s

X1 (S)

X1 (D)

(a) (b)

0 10 20 30 40 50500

1000

1500

2000

2500

3000N = 10,000

Time (Days)

Pig

s

X1 (S)

X1 (D)

0 10 20 30 40 50500

1000

1500

2000

2500

3000N = 10,000

Time (Days)

Pig

s

X1 (S)

X1 (D)

(c) (d)

Fig. 1. Node 1 obtained by the stochastic system with afixed delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a fixed delayvs. the solution of its corresponding deterministicsystem, and (b) and (d) the mean solution of thestochastic system with a fixed delay vs. the solutionof its corresponding deterministic system.

0 10 20 30 40 5010

15

20

25

30

35N = 100

Time (Days)

Pig

s

X2 (S)

X2 (D)

0 10 20 30 40 5014

16

18

20

22

24

26

28N = 100

Time (Days)P

igs

X2 (S)

X2 (D)

(a) (b)

0 10 20 30 40 501400

1600

1800

2000

2200

2400

2600

2800N = 10,000

Time (Days)

Pig

s

X2 (S)

X2 (D)

0 10 20 30 40 501400

1600

1800

2000

2200

2400

2600

2800N = 10,000

Time (Days)

Pig

s

X2 (S)

X2 (D)

(c) (d)

Fig. 2. Node 2 obtained by the stochastic system with afixed delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a fixed delayvs. the solution of its corresponding deterministicsystem, and (b) and (d) the mean solution of thestochastic system with a fixed delay vs. the solutionof its corresponding deterministic system.

amount of time, but rather may vary for each transitionthat occurs. One way to implement this variation of delaytimes is to consider the delay to be a random variable that

will be sampled for every transition. The resulting systemis called a stochastic model with a random delay. For thismodel, we still assume here, for ease in exposition, that alltransitions occur instantaneously except for the arrival ofpigs transitioning from Node 1 to 2, and the process startsfrom t = 0, that is, X(t) = 0 for t < 0.

3.1 The Corresponding Deterministic System For TheStochastic Model With A Random Delay

In [19], Schlicht and Winkler showed that if all the tran-sition rates are linear, then the mean solution of thestochastic system with random delays can be describedby a system of deterministic differential equations witha distributed delay, where the delay kernel is the prob-ability density function of the given distribution for therandom delay. Even though the transition rates in ourpork production model are nonlinear, we still would liketo explore whether or not such deterministic system canbe used as a possible corresponding deterministic systemfor our stochastic system with random delay and explorethe relationship between them.

Let G(t) be the probability density function of the randomdelay. Then the corresponding deterministic system for ourstochastic system with delay is given by

c1(t) = −κ1c1(t)(l2 − c2(t))+ + κ4 min(c4(t), sm)c2(t) = −κ2c2(t)(l3 − c3(t))+

+

t∫

−∞

G(t− s)κ1c1(s)(l2 − c2(s))+ds

c3(t) = −κ3c3(t)(l4 − c4(t))+ + κ2c2(t)(l3 − c3(t))+c4(t) = −κ4min(c4(t), sm) + κ3c3(t)(l4 − c4(t))+ci(0) = ci0, i = 1, 2, 3, 4,ci(s) = 0, s < 0, i = 1, 2, 3, 4,

(5)

We remark that numerically solving a system of the form(5) may prove to be difficult due to the distributed delayterm. However, if we make additional assumptions on thedelay kernel, we can transform a system with a distributeddelay into a system of ODE’s. Specifically, if we assumethat the delay kernel has the form

G(u;α, n) =αnun−1e−αu

(n− 1)!(6)

with α > 0 and n being a positive integer number, thatis, G is the probability density function of a Gammadistributed random variable with mean being n/α and thevariance being n/α2, then by way of the linear chain trick(e.g., see [18] and the references therein) we can transformthe system (5) into a system of ODE’s. For example, forthe case n = 1, if we let

c5(t) =

t∫

−∞

αe−α(t−θ)κ1c1(θ)(l2 − c2(θ))+dθ,

then this substitution yields the following system of ODE’s

c1(t) = −κ1c1(t)(l2 − c2(t))+ + κ4 min(c4(t), sm)c2(t) = −κ2c2(t)(l3 − c3(t))+ + c5(t)c3(t) = −κ3c3(t)(l4 − c4(t))+ + κ2c2(t)(l3 − c3(t))+c4(t) = −κ4 min(c4(t), sm) + κ3c3(t)(l4 − c4(t))+c5(t) = ακ1c1(t)(l2 − c2(t))+ − αc5(t)ci(0) = ci0, i = 1, 2, 3, 4,

c5(0) =

0∫

−∞

αeαθκ1c1(θ)(l2 − c2(θ))+dθ = 0,

(7)

which is equivalent to (5).

The advantage of introducing the linear chain trick istwo fold. The resulting system of ODE’s is much easierto solve compared to the system (5) where there is adistributed delay. In addition, we can use the Kurtz’s limittheorem to construct a corresponding stochastic systemwhich converges to the resulting system of ODE’s.

3.2 Comparison Of The Stochastic Model With A RandomDelay And Its Corresponding Deterministic System

All parameter values and initial conditions remain as in[6], i.e., the same as in [4]. For the probability densityfunction G(u;α, n), n was taken to be 1 and α to be 0.2,which implies that the mean value of the random delayis 5 and its variance is 25.0. Sample sizes of N = 100and N = 10,000 were considered for the stochastic systemwith a random delay. As before, the stochastic system wassimulated for 10,000 trials, and the mean solution wascomputed.

Figures 3 through 6 compare the solution of deterministicsystem for the nodes (in terms of number of pigs, i.e.,Nc(t) with c(t) being the solution to (7)) and the resultsof the stochastic system with a random delay. From thesefigures we observe that the trajectories of the stochasticsimulations follow the solution of deterministic system,and the variation of sample paths of solution to stochasticsystem with a random delay decreases as the sample sizeincreases. It is also seen that as the sample size increasesthe mean solution of stochastic system with a randomdelay become closer to the solution of deterministic sys-tem. Hence, not only are the sample paths of the solutionto stochastic system with a random delay showing lessvariation for larger sample sizes, but the expected value ofthe solution is better approximated by the solution of de-terministic system for large sample sizes. Thus, the deter-ministic system (7) (or deterministic differential equationwith a distributed delay (5)) could be used to serve asa reasonable corresponding deterministic system for thisparticular stochastic system with a random delay (withthe given parameter values and initial conditions).

4. CONCLUSION

In this note we extended the stochastic pork productionmodel in [4] to incorporate delay to account for the phe-nomenon that movement from one node to the next isoften not instantaneous in practice due to the physical dis-tance and/or some unexpected disruptions/interruptions.We considered two different types of delays, a fixed de-lay and a random delay, and numerically explored thecorresponding deterministic approximations for these two

0 10 20 30 40 505

10

15

20

25

30N = 100

Time (Days)

Pig

s

X1 (S)

X1 (D)

0 10 20 30 40 505

10

15

20

25

30N = 100

Time (Days)

Pig

s

X1 (S)

X1 (D)

(a) (b)

0 10 20 30 40 50500

1000

1500

2000

2500

3000N = 10,000

Time (Days)

Pig

s

X1 (S)

X1 (D)

0 10 20 30 40 50500

1000

1500

2000

2500

3000N = 10,000

Time (Days)

Pig

s

X1 (S)

X1 (D)

(c) (d)

Fig. 3. Node 1 obtained by the stochastic system with a ran-dom delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a randomdelay vs. the solution of its corresponding determinis-tic system, and (b) and (d) the mean solution of thestochastic system with a random delay vs. the solutionof its corresponding deterministic system.

0 10 20 30 40 5014

16

18

20

22

24

26

28N = 100

Time (Days)

Pig

s

X2 (S)

X2 (D)

0 10 20 30 40 5019

19.5

20

20.5

21

21.5

22

22.5

23N = 100

Time (Days)

Pig

s

X2 (S)

X2 (D)

(a) (b)

0 10 20 30 40 501800

1900

2000

2100

2200

2300

2400N = 10,000

Time (Days)

Pig

s

X2 (S)

X2 (D)

0 10 20 30 40 501900

1950

2000

2050

2100

2150

2200

2250

2300N = 10,000

Time (Days)

Pig

s

X2 (S)

X2 (D)

(c) (d)

Fig. 4. Node 2 obtained by the stochastic system with a ran-dom delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a randomdelay vs. the solution of its corresponding determinis-tic system, and (b) and (d) the mean solution of thestochastic system with a random delay vs. the solutionof its corresponding deterministic system.

resulting stochastic models. Numerical results show thatwhen the sample size is sufficiently large the stochasticmodel with a fixed delay can be well approximated by a

0 10 20 30 40 5045

50

55

60

65

70N = 100

Time (Days)

Pig

s

X3 (S)

X3 (D)

0 10 20 30 40 5045

50

55

60

65

70N = 100

Time (Days)

Pig

s

X3 (S)

X3 (D)

(a) (b)

0 10 20 30 40 504500

5000

5500

6000

6500

7000N = 10,000

Time (Days)

Pig

s

X3 (S)

X3 (D)

0 10 20 30 40 504500

5000

5500

6000

6500

7000N = 10,000

Time (Days)

Pig

s

X3 (S)

X3 (D)

(c) (d)

Fig. 5. Node 3 obtained by the stochastic system with a ran-dom delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a randomdelay vs. the solution of its corresponding determinis-tic system, and (b) and (d) the mean solution of thestochastic system with a random delay vs. the solutionof its corresponding deterministic system.

0 10 20 30 40 500

1

2

3

4

5

6N = 100

Time (Days)

Pig

s

X4 (S)

X4 (D)

0 10 20 30 40 500

1

2

3

4

5

6N = 100

Time (Days)P

igs

X4 (S)

X4 (D)

(a) (b)

0 10 20 30 40 500

100

200

300

400

500

600N = 10,000

Time (Days)

Pig

s

X4 (S)

X4 (D)

0 10 20 30 40 500

100

200

300

400

500

600N = 10,000

Time (Days)

Pig

s

X4 (S)

X4 (D)

(c) (d)

Fig. 6. Node 4 obtained by the stochastic system with a ran-dom delay (S) and by the corresponding deterministicsystem (D), with (a) and (b) N = 100, and (c) and(d) N = 10,000. (a) and (c) five typical sample pathsof the solution to stochastic system with a randomdelay vs. the solution of its corresponding determinis-tic system, and (b) and (d) the mean solution of thestochastic system with a random delay vs. the solutionof its corresponding deterministic system.

system of deterministic differential equations with a fixeddelay. This confirms with the recent theoretical resultspresented in [9]. We also numerically showed that the

mean solution of the stochastic model with a Gammadistributed random delay can be well approximated by thesolution of a system of deterministic differential equationswith a Gamma distributed delay when the sample sizeis sufficiently large. Hence, the system of deterministicdifferential equations with a Gamma distributed delay canbe used as a possible corresponding deterministic one forthis particular stochastic model with a Gamma distributedrandom delay (with the given parameter values and initialconditions).

In additional efforts [6], we have compared the stochasticmodel with a Gamma distributed random delay to thestochastic system constructed based on the Kurtz’s limittheorem from a system of deterministic differential equa-tions with a Gamma distributed delay. Even though thesame system of deterministic differential equations with aGamma distributed delay can be used as the correspondingdeterministic ones for these two stochastic systems, itwas found that with the same sample size the histogramplots of the state solutions to the constructed stochasticsystem appear more flattened than the corresponding onesobtained for the stochastic model with a random delay.However, there is more agreement between the histogramsof these two stochastic systems as the variance of therandom delay decreases. We also found that with the samevariance for the random delay the histogram plots forthe stochastic model with a random delay are symmetricfor all the sample size investigated, while those for theconstructed stochastic system are asymmetric when thesample size is small, but becomes more symmetric as thesample size increases.

Finally in [6] we compared the histogram plots of the statesolutions to the stochastic model with a fixed delay tothose obtained for the stochastic model with a randomdelay, where the value of the fixed delay is chosen asthe mean value of the random delay. Numerical resultsshow that for those states affected by the delay most theirhistogram plots obtained for the stochastic system with afixed delay have similar unimodal shapes and dispersionas the corresponding ones for the stochastic system witha random delay, but their mode values become smaller(i.e., shifted more to the left side as compared to thecorresponding ones obtained for the stochastic systemwith a random delay) as the sample size increases. Wealso found that when the variance of the random delayis sufficiently small, the histograms of state solutions tothe stochastic model with a fixed delay agree well withthe corresponding ones obtained for the stochastic modelwith a random delay regardless of the sample size. Detaileddescriptions of all these results can be found in [6].

REFERENCES

[1] Allen, L.J.S. (2011). An Introduction to StochasticProcesses with Applications to Biology. Chapman &Hall/CRC, Boca Raton, FL.

[2] Anderson, D. (2007). A modified next reaction methodfor simulating chemical systems with time dependentpropensities and delays. The Journal of ChemicalPhysics , volume 127, 214107-1–214107-10.

[3] Andersson, H. and Britton, T. (2000). Stochastic Epi-demic Models and Their Statistical Analysis, Springer-Verlag, New York.

[4] Bai, P., Banks, H.T., Dediu, S., Govan, A.Y., Last, M.,Lloyd, A.L., Nguyen, H.K., Olufsen, M.S., Rempala,G., and Slenning, B.D. (2007). Stochastic and deter-ministic models For agricultural production networks,Mathematical Biosciences and Engineering, volume 4,373–402.

[5] Banks, H.T. (2012). A Functional Analysis Frameworkfor Modeling, Estimation and Control in Science andEngineering, CRC Press/Taylor and Frances Publish-ing, Boca Raton, FL.

[6] Banks, H.T., Catenacci, J., and Hu, S. (2013). Acomparison of stochastic systems with different typesof delays, in preperation.

[7] Banks, H.T., and Kappel, F. (1979). Spline approxi-mations for functional differential equations. Journalof Differential Equations, volume 34, 496–522.

[8] Barrio, M., Burrage, K., Leier, A. and Tian, T. (2006).Oscillatory regulation of Hes1: discrete stochastic delaymodeling and simulation. PLoS Computational Biol-ogy, volume 2, 1017–1030.

[9] Bortolussi, L. and Hillston, J. (2012). Fluid approxima-tions of CTMC with deterministic delays. QuantitativeEvaluation of Systems (QEST), 2012 Ninth Interna-tional Conference on Sept 17-20, 53–62.

[10] Bratsun, D. Volfson, D., Tsimring, L.S., and Hasty, J.(2005). Delayed-induced stochastic oscillations in generegulation. PNAS, volume 102, 14593–14598.

[11] Ethier, S.N., and Kurtz, T.G. (1986). Markov Pro-cesses: Characterization and Convergence, J. Wiley &Sons, New York.

[12] Kurtz, T.G. (1970). Solutions of ordinary differentialequations as limits of pure jump Markov processes. J.Appl. Prob., volume 7, 49–58.

[13] Kurtz, T.G. (1971). Limit theorems for sequences ofpure jump Markov processes approximating ordinarydifferential processes. J. Appl. Prob., volume 8, 344–356.

[14] Kurtz, T. G. (1978). Strong approximation theoremsfor density dependent Markov Chains. StochasticProcesses and their Applications, volume 6, 223–240.

[15] Kurtz, T.G. (1981). Approximation of PopulationProcesses, SIAM, Pennsylvania.

[16] Ortiz, A.R., Banks, H.T., Castillo-Chavez, C., Chow-ell, G., and Wang, X. (2011). deterministic methodol-ogy for estimation of parameters in dynamic Markovchain models. CRSC-TR10-07, N.C. State University,May, 2010. J. Biological Systems, volume 19, 71–100.

[17] Roussel, M., and Zhu, R. (2006). Validation of analgorithm for delay stochastic simulation of transcrip-tion and translation in prokaryotic gene expression.Physical Biology, volume 3, 274–284.

[18] Ruan, S. (2006). Delay differential equations in singlespecies dynamics. in Delay Differential Equations andApplications, O. Arino et al. (eds.), Springer, Berlin,477–517.

[19] Schlicht, R. and Winkler, G.(2008). A delay stochas-tic process with applications in molecular biology.Journal of Mathematical Biology, volumne 57, 613–648.