Basic Theory For SDE’stroy/vlad/doering.pdf · Basic Theory For SDE’s W. C. Troy∗ July 11,...

Basic Theory For SDE’s

W. C. Troy∗

July 11, 2008Table of Contents(1.) Background: general properties.

(a.) The Cauchy-Schwarz, Chebychev and Markov inequalities.

(b.) How to get the ODE associated with PX ≤ x+ t|X > t = P0 < X ≤ x.(c.) Stationary and WSS (wide sense stationary) random processes.

(d.) If X(t) is a WSS process then E (X(t)) = µ is constant.

(e.) The autocorellation function: if X(t) is WSS then E(X(t)X(s)) = RX(|t− s|).(f.) Important Example: white noise.

(g.) If X(t) is WSS then V ar (X(t)) = constant = σ2 = RX(0) − µ2.

(h.) Random processes with independent increments.

(i.) X(t) is mean square continuous at t ⇐⇒ RX(t, s) is continuous at s = t.

(j.) If X(t) is m. s. continuous at t then X(t) is continuous in probability at t.

(k.) First derivative properties of a random process.

(l.) Stochastic integral properties of a random process.

(m.) How to see where the greatest uncertainty of a process occurs?

(n.) The pdf’s for Z = Y +X and Z = Y −X.

(o.) Definition and computation of FX(x|a < X ≤ x).

(p.) Proof that ρ(x, t|y, t) = δ(x − y) when X(t) is continuous.

(q.) The interpretaion of∫

R1

∫

R2ρ(x′, t1, x′′, t2)dx′dx′′.

(r.) Principle X: Prob a < X ≤ b, c < Y ≤ d =∫ d

cProba < X ≤ b|Y = η fY (η)dη.

(2.) The Poisson process

(a.) Basic definition.

(b.) Proof that E(X(t)) = λt, E(X2(t)) = λt+ λ2t2 and V ar(X(t)) = λt.

(c.) Application: estimate the charge on the electron.

(d.) Proof that E(X(t)X(s)) = λmin(t, s) + λ2ts and Cov(X(t), X(s)) = λmin(t, s).

∗Department of Mathematics, University of Pittsburgh, Pittsburgh, PA 15260, U.S.A.

1

(e.) Variation, and coefficients of correlation and variation.

(f.) Proof that CV=1 for T1.

(g.) Proof that fTN (t) = λe−λt (λt)N−1

(N−1)! and E(T kN ) = N(N+1)..(N+k−1)

λk .

(h.) Principle X: Proba < X ≤ b, c < Y ≤ d =∫ d

c Prob a < X ≤ b|Y = η fY (η)dη.

(i.) Proof that ZN = TN − TN−1 is an exponential random variable.

(j.) If Z = TN − TM , N > M then fZ(z) = λN

(N−1−M)!e−λzzN−1−M .

(k.) Proof that Z2 = T2 − T1 and Z1 = T1 are independent.

(l.) Proof that Z = TN − TM , N > M and TM are independent.

(m.) Two proofs that E(y(t)) = λ for the spike train y(t) =∑∞

k=1 δ(t− Tk).

(n.) Two proofs that the spike train y(t) satisfies E(y(t)y(s)) = λ2 when s > t.

(o.) Proof that the spike train satisfies E(y(t)y(s)) = λδ(t − s) + λ2

(p.) Proof that y(t) satisfies E(y2(t)) = ∞ and V ar(y(t)) = ∞.

(q.) Proof that Probone spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s).

(3.) Spiking Theory

(a.) The Definition of Input Current.

(b.) Derivation of the correlation coefficient c.

(c.) The LIF model - Benji’s notes.

(d.) How to solve τ dVdt = (µ− V ) + σ dζ

dt .

(e.) How to analyze E(X(t)X(t+ τ) when X(t) solves an SDE?

(f.) Gardner’s eigenfunction approach to find ρ(x, t).

(g.) How to use the eigenfunction expansion to find ρ(x, t|x0, 0).

(h.) How to find the autocorrelation function from the eigenfunction expansion.

(i.) Example 1: the Ornstein-Uhlenbeck process.

(j.) The linear integrate and fire model.

(4.) The Wiener process W (t).

(a.) Start with conditional, then construct the single time and joint distributions of W (t).

(b.) Important properties ρ(w, 0) = δ(w) and ρ(w′, t|w, t) = δ(w′ − w).

(c.) Proof that ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ w(d.) Proof that E(W (t)W (s)) = min(t, s).

(e.) Proof that W (t) has independent increments.

(f.) Proof that W (t) −W (s) ∼√t− sN(0, 1) when t > s.

(g.) W (t) is continuous in mean square and in probability at time t.

(h.) Proof that W (t) does not have a mean square first derivative.

(i.) Proof that the generalized derivative of W (t) satisfies RW ′(t, s) = σ2δ(t− s).

(j.) Properties of the mean square integral Y =∫ t

t0W (α)dα.

(k.) Y (t) =∫ t

0X(η)dη is a Wiener process if X(t) is normal white noise.

(l.) Scaling: dWτ =√ǫdWt if τ = ǫt. (i.e. they have the same pdf’s)

2

(m.) dXt = f(Xt)dt+B(Xt)dWt becomes ǫdXτ = f(Xt)dτ +√ǫB(Xt)dWτ if τ = ǫt.

(5.) Doering’s derivation of the Fokker-Planck equation.

(a.) The stochastic ODE.

(b.) Evaluation of E(∆X(t)|X(t) = x) and E(

(∆X(t))2|X(t) = x)

.

(c.) The Chapman-Kolmogorov equation.

(d.) Use of the Chapman-Kolmogorov eq. to derive the Fokker-Planck eq.

(e.) Use of Ito’s Lemma to derive the Fokker-Planck eq.

(6.) Derivation of the Backwards Kolmogorov Equation.

(7.) Examples of the Fokker - Planck equation.

(a.) wienerSDEscalar.m solves the Wiener process SDE W ′ = σζ(t).

(b.) The Fokker-Planck pde for the Wiener process SDE.

(c.) The Ornstein-Uhlenbeck SDE X ′ = −γX + σζ(t).

(d.) The autocorellation and power functions for the Ornstein-Uhlenbeck process.

(e.) How to use the Ornstien-Uhlenbeck SDE to find ρ(x, t|y, s) (monahan.pdf notes)

(f.) The Fokker-Planck pde for systems of SDE’s.

(g.) How to find ρ(y, t|y0, 0) for dYdt = µ+ σζ(t), Y (0) = y0.

(8.) Ito’s Lemma for scalar equations and examples.

(a.) Informal proof of Ito’s Lemma.

(b.) Example 1. Solve dx(t)dt = λx+ µxζ(t) using Ito’s Lemma.

(c.) ItoExample1.m solves dx(t)dt = λx+ µxζ(t) and compares with the true sol.

(d.) Linear stability for dx(t)dt = λx+ µxζ(t)

(e.) higham3.m solves dx(t)dt = x+ .5xζ(t) 1000 times and graphs 5 solutions.

(f.) ItoExample2.m solves dx(t)dt = λx + µζ(t) and compares with the true sol.

(g.) Solve dx(t)dt = µx− x2 + σxζ(t) and find the Fokker-Planck eq. .

(h.) The Brownian Bridge SDE dx = k−x1−t + dW (t), x(0) = A

(i.) Numerical sol.and the Fokker-Planck pde for dx(t)dt = α− λx+ σ

√xζ(t)

(j.) Proof that∫ t

0 WNdW (s) = W N+1

N+1 − N2

∫ t

0 WN−1(s)ds and E

(

∫ t

0 WNdW (s)

)

= 0.

(k.) Proof that∫ t

0 g(s)dW (s) = g(t)W (t) −∫ t

0 W (s)dg(s) and E(

∫ t

0 g(s)dW (s))

= 0.

(l.) How to find ρ(V, t|v, 0) numerically from the SDE

(9.) A power law associated with dX = µXdt+ σXdW,X(0) = x0

(a.) Transform dX = µXdt+ σXdW,X(0) = x0 into dYdt = µ+ σζ(t), Y (0) = y0.

(b.) Find ρ(y, t|y0, 0) for dYdt = (µ− σ2

2 )dt+ σζ(t), Y (0) = y0.

(c.) Use moment generating functions to derive the power law.

(d.) The generalized power law.

(e.) The generalized power law when N=2

(f.) The generalized power law when N=3

3

(g.) A power law associated with the Gompertz equation.

(h.) The Gompertz equation with a delay.

(10.) Ito’s Lemma for systems and examples.

(a.) Stament of Ito’s Lemma for systems.

(b.) Example 1.∫ t

0etAdB(s) = etAB(t) −

∫ t

0AetAB(s)ds.

(c.) Example 2. Solve LQ′′ + RQ′ + 1CQ = G+ σζ(t)

(d.) Find the system of SDE’s satisfied by eix(t), where x(t) is the Wiener process.

(11.) Exit time problems following Oksendal

(a.) Use of Ito’s Lemma to derive the generator A of X(t).

(b.) Dynkin’s formula.

(c.) The exit time from a ball in RN for Brownian motion (exit1.m).

(d.) The Backward Kolmogorov Equation - Oksendal version.

(e.) The Backward Kolmogorov Equation - Wikipedia version.

(f.) The Feynman-Kac formula - Wikepedia version.

(g.) The Feynman-Kac formula - Oksendal version.

(12.) The first passage time - Gardner version

(a.) Basic ODE for first passage time. Exercises.

(b.) Basic ODE for Nth moment.

(c.) Example I. 1st and 2nd moments for diffusion.

(d.) Estimating integrals by the method of steepest descent.

(e.) Escape over a double-well potential barrier.

(f.) Why escape time is asymptotically an exponential random variable.

(g.) The probability, and mean time, of exit thru a particular end of [a, b].

(h.) Probability of exit thru x = 0 or x = 1 in dU = γU(1 − U)dt+ σ√

U(1 − UdW.

(i.) Example: finite time extinction in dx(t)dt = µx− x2 + σxpζ(t).

(k.) Stochastic Resonance - Benzi paper ideas (benzipaper1.pdf).

(13.) Large deviation theory - Freidlin-Wentzell theory

(a.) Maier’s notes on exit time asymptotics: santafewentzellnotes.pdf and maier2.pdf

(b.) Links between the Hamiltonian, Lagrangian and Action integral.

(c.) Example: dXt = 2λXt +√

2N b(X)dWt

(d.) Muratov - 2 emails, and paper Deville.pdf

(d.) Barbara Gentz estimates for large deviations - linuxmail.org

(14.) Continuum limits of Master equations.

(a.) Derivation of Ct = DCxx from C(x, t + τ) = λrC(x − ∆x, t) + λlC(x + ∆x, t).

4

(b.) Derivation of the Fokker-Planck equation for dU = γU(1 − U)dt+ σ√

U(1 − UdW.

(15.) Time of first passage problems for discrete problems.

(a.) Random walk: how to find the probability that XN = 0 for some N > 0.

(14.) White noise

(a.) Definition of white noise.

(b.) Example: uniform white noise.

(c.) Computing RX(τ) and SX(ω) for uniform white noise.

(16.) Research ideas and papers to get

(a.) Feynman extension.

(b.) Oscillations in predator-prey systems.

(c.) Neuro papers: Brunel-Hakim, Amit-Brunel, Brune

(d.) alternans

(17.) Use of the steady state pde

(18.) Population growth in a randomly varying environement.

(a.) The problem.

(b.) Put the extinction problem into the Central Limit Theorem setting.

(c.) Conditions which imply that E(Xt) → ∞, yet extinction occurs.

(d.) Proof that(

∏ti=1 li

)1/t

≤ 1t

∑ti=1 li when li ≥ 0.

(e.) A periodic sequence li such that ρ < 1 < λ and (Xt, Xt) → (0, 0) and as t→ ∞.

(f.) A periodic sequence li such that ρ = 1 < λ, Xt is periodic, Xt → constant as t→ ∞.

(19.) A noisy genetic swith model.

(20.) Coherence resonance in SIR (Kuske et al [13])

(a.) The problem.

(b.) Properties of the Bivariate normal distribution.

(c.) Properties of the Ornstein-Uhlenbeck process

1 Background: general properties.

(a.) The Cauchy-Schwarz, Chebychev and Markov inequalities.

The Cauchy-Schwarz inequality. Let X and Y be random vbariables. Then

|E (XY ) |2 ≤ E(X2)E(Y 2). (1.1)

5

To prove this we consider 0 ≤ E(

[X − sY ]2)

= E(X2) − 2sE(XY ) + s2E(Y 2). The first derivative

with respect to s of the right side is zero when s = E(XY )E(Y 2) . At this value we conclude that

0 ≤ E(X2) − (E(XY ))2

E(Y 2). (1.2)

From this a manipulation gives (3.461). It will be used below in part (h.).

Chebychev’s inequality. Let X be a random variable. If E(X) = µX and σ2 = V ar(X) exist then

P (|X − µX | ≥ a) ≤ σ2X

a2=V ar(X)

a2for each a > 0. (1.3)

The proof (Problem 2.35, p. 66 of Schaum’s) starts with

P (|X − µX | ≥ a) =

∫ µX−a

−∞fX(x)dx +

∫ ∞

µX+a

fX(x)dx =

∫

|x−µX |≥a

fX(x)dx. (1.4)

from the definition of σ2 we conclude that

σ2 =

∫ ∞

−∞(x− µx)2fX(x)dx ≥

∫

|x−µX |≥a

(x− µx)2fX(x)dx ≥ a2

∫

|x−µX |≥a

fX(x)dx. (1.5)

Combining (1.4) and (1.5) gives (1.3). It will be used below in part (i.).

Markov’s inequality. Let X be a random variable If fX = 0 for x ≤ 0 then

P (X ≥ a) ≤ µX

afor each a > 0. (1.6)

The proof (Problem 2.34, p. 66 of Schaum’s) starts with

µX =

∫ ∞

0

xfX(x)dx ≥∫ ∞

a

fX(x)dx ≥ a

∫ ∞

a

fX(x)dx. (1.7)

Thus,∫ ∞

a

fXdx = P (X ≥ a) ≤ µX

a. (1.8)

(b.) How to get the ODE associated with PX ≤ x+ t|X > t = P0 < X ≤ x.This follows [26], p. 73 - no. 2.53. Then PX ≤ x+ t|X > t = P0 < X ≤ x is equivalent to

PX ≤ x+ t and X > tPX > t = P0 < X ≤ x. (1.9)

which in turn leads to

PX ≤ x+ t and X > t = PX > tP0 < X ≤ x. (1.10)

Assume that X ≥ 0 and let FX(x) = P0 < X ≤ x. Then (1.10) becomes FX(x + t) − FX(x) =

(1 − FX(t))FX(x). Divide both sides by x and get

FX(x+ t) − FX(x)

x= (1 − FX(t))

FX(x)

x. (1.11)

6

Use the fact that FX(0) = 0 (since X ≥ 0) and let x→ 0 to get the ODE

F ′X(t) = F ′

X(0)(1 − FX(t)). (1.12)

Set µ = F ′X(0), solve this ODE and find that FX(t) and the associatd pdf fX(t) are

FX(t) = 1 − e−µt and fX(t) = F ′X(t) = µe−µt ∀t ≥ 0. (1.13)

(c.) Stationary and WSS (wide sense stationary) random processses.

A random process X(t) is a stationary (Schaum’s, p. 163) process if, for each set ti|i = 1, 2, .., n,

fX(x1, t1, x2, t2, .., xn, tn) = fX(x1, t1 + γ, x2, t2 + γ, .., xn, tn + γ) ∀γ. (1.14)

If the stationary condition holds only for n ≤ k, then the process X(t) is stationary to order k. If X(t)

is stationary to order 2, then X(t) is called a WSS (wide sense stationary) process, and therefore

fX(x1, t1) = fX(x1, t1 + τ) and fX(x1, t1, x2, t2) = fX(x1, t1 + η, x2, t2 + η) ∀t1, t2, τ and η (1.15)

Setting τ = −t1 and η = −t2 reduces (1.15) to

fX(x1, t1) = fX(x1, 0) and fX(x1, t1, x2, t2) = fX(x1, t1 − t2, x2, 0) ∀t1 and t2. (1.16)

Alternatively, we could have let η = −t1, in which case the second part of (1.15) reduces to

fX(x1, t1, x2, t2) = fX(x1, 0, x2, t2 − t1) ∀t1, t2. (1.17)

Thus, we conclude that if X(t) is a WSS process then

fX(x1, t1, x2, t2) = fX(x1, x2, |t2 − t1|) ∀t1, t2. (1.18)

(d.) If X(t) is a WSS process then E(X(t)) = µ is constant.

It follows from the first part of (1.17), and the definition of mean, that

E(X(t) =

∫ ∞

−∞x1fX(x1, 0)dx = µ ∀t. (1.19)

(e.) The autocorellation function: if X(t) is WSS then E(X(t)X(s)) = RX(|t− s|)

It follows from (1.18) that the autocoreelation function RX(t, s) satisfies

RX(t, s) = E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞x1x2f(x1, x2, |t− s|)dx1dx2 = RX(|t− s|) ∀t, s. (1.20)

This and (1.20) give the following result which we will use:

RX(t, t+ τ) = E (X(t)X(t+ τ)) = RX(τ) ∀t, τ. (1.21)

7

A continuous time WSS random variable X(t) is called white noise if

µX(t) = 0 and RX(τ) = σ2δ(τ) ∀τ ∈ R. (1.22)

In this case the frequency power spectrum SX(ω) satisfies

SX(ω) =

∫ ∞

−∞σ2δ(τ)dτ = σ2 ∀ω ∈ R. (1.23)

Since SX(ω) ≡ σ2, all frequencies contribute equally to power, hence X(t) is white noise. Section 19

gives further details about the connection between the uniform distribution and white noise.

(g.) If X(t) is WSS then V ar (X(t)) = constant = σ2 = RX(0) − µ2.

The variance is defined by

V ar (X(t)) = E(

[(X(t) − E(X(t))]2)

= E(

[X(t)]2)

− (E(X(t))2. (1.24)

Setting τ = 0 in (1.21), and using (1.19), we obtain

V ar (X(t)) = RX(0) − µ2 = constant = σ2 ∀t. (1.25)

(h.) Processes with independent, stationary increments.

Definition 1. A random process X(t), t ≥ 0 has independent increments if whenever 0 < t1 < t2 <

.. < tN , then X(0), X(t1) −X(0), X(t2) −X(t1), .., X(tN ) −X(tN−1) are independent.

Definition 2. If a random process X(t), t ≥ 0 has independent increments, and if X(t) − X(s)

has the same probability distribution as X(t + h) −X(s + h) ∀s, t, h ≥ 0, then X(t) is said to have

stationary independent increments (see Schaum’s, p. 163).

Basic properties. First, the assumption that X(t) has stationary, independent increments means

that if t > s then X(t) − X(s) has the same pdf as X(t + h) − X(s + h) ∀h. Setting h = −s, we

conclude that X(t) −X(s) and X(t− s) −X(0) have exactly the same pdf.

Next, if X(t), t ≥ 0 has stationary independent increments, and X(0) = 0, then

E (X(t)) = µX t ∀t ≥ 0, (1.26)

V ar (X(t)) = σ2t ∀t ≥ 0 and V ar (X(t) −X(s)) = V ar (X(t)) − V ar (X(s))∀t > s ≥ 0. (1.27)

Remarks. It follows from (1.19), (1.25), (1.26) and (1.27) that a process with stationary independent

increments is not a WSS process. The process with stationary independent increments that we are

interested in is the Wiener process which is described in the next section.

The proofs of (1.26)-(1.27) are from Problems 5.22 and 5.23, p. 181-182 of Schaum’s. First, define

f(t) = E (X(t)) = E (X(t) −X(0)) ∀t ≥ 0. (1.28)

8

Then

f(t+ s) = E (X(t+ s) −X(0)) = E (X(t+ s) −X(s) +X(s) −X(0)) (1.29)

The linearity of E implies that

f(t+ s) = E (X(t+ s) −X(s)) + (X(s) −X(0)) . (1.30)

Because X(t) has has stationary independent increments, this reduces to

f(t+ s) = E (X(t) −X(0)) + (X(s) −X(0)) = f(t) + f(s). (1.31)

The solution of f(t+ s) = f(t) + f(s) is f(t) = ct. Setting c = f(1) = E(X(1)) = µX gives

f(t) = E (X(t))) = µXt ∀t ≥ 0. (1.32)

Next, to prove (1.27) we define

g(t) = V ar (X(t)) = V ar (X(t) −X(0)) ∀t ≥ 0. (1.33)

Then

g(t+s) = V ar (X(t+ s) −X(s) +X(s) −X(0)) = V ar (X(t+ s) −X(s))+V ar (X(s) −X(0)) t > s ≥ 0.

(1.34)

This used the independent stationary increment property that X(t+ s) −X(s) and X(s)−X(0) are

independent, the property that V ar(Y +Z) = V ar(Y )+V ar(Z) if Y and Z are independent (Problem

4.43, pp 149-150 of Schaum’s). Since X(t+ s) −X(s) = X(t) −X(0), then (1.34) reduces to

g(t+ s) = V ar (X(t) −X(0)) + V ar (X(s) −X(0)) = g(t) + g(s) t > s ≥ 0. (1.35)

Thus, g(t + s) = g(t) + g(s) and as before it follows that g(t) = kt where k is a constant. Thus

V ar(X(t)) = kt, hence

V ar (X(t)) = σ2t, σ2 = V ar(1) ∀t ≥ 0. (1.36)

(i.) X(t) is m. s. continuous at t ⇐⇒ RX(t, s) is continuous at s = t.

Definition 1. A random process X(t) is mean square continuous at t ( Schaum’s, p. 209) if

limh→0

E([X(t+ h) −X(t)]2) = 0. (1.37)

Start with (Problem 6.1, p. 219 Schaum’s)

E(

[X(t+ h) −X(t)]2)

= E(

X2(t+ ǫ))

− 2E (X(t+ ǫ)X(t)) + E(

X2(t))

= RX(t+ ǫ, t+ ǫ) − 2RX(t+ ǫ, t) +RX(t, t).(1.38)

If RX(t, s) is continuous at s = t, then the right side of (1.38) tends to 0 as ǫ→ 0, hence (1.37) holds.

9

Next, suppose that X(t) is mean square continuous at t. Then

RX(t+ ǫ1, t+ ǫ2) −RX(t, t) = E (X(t+ ǫ1)X(t+ ǫ2)) − E(

X2(t))

= E ([X(t+ ǫ1) −X(t)] [X(t+ ǫ2) −X(t)])

+ E ([X(t+ ǫ1) −X(t)]X(t)) + E ([X(t+ ǫ2) −X(t)]X(t)) .

(1.39)

The Cauchy Schwarz inequality |E(XY )|2 ≤ E(X2)E(Y 2) applied to (1.39) implies that

|RX(t+ ǫ1, t+ ǫ2) −RX(t, t)|2 ≤ E(

[X(t+ ǫ1) −X(t)]2)

E(

[X(t+ ǫ2) −X(t)]2)

+ E(

[X(t+ ǫ1) −X(t)]2)

E(

X2(t))

+ E(

[X(t+ ǫ2) −X(t)]2)

E(

X2(t))

.(1.40)

The right side of (1.40) tends to 0 as (ǫ1, ǫ2) → 0, hence RX(t, t) is continuous.

(j.) If X(t) is m. s. continuous at t then X(t) is continuous in probability.

Definition. A process X(t) is continuous in probability (Schaum’s, Problem 5.62) at t if ∀ǫ > 0,

limh→0

P (|X(t+ h) −X(t)| > ǫ) = 0. (1.41)

Proof. We follow Problem 6.5, p. 220 of Schaum’s. First, recall Chebychev’s inequality (Problem

2.36, p. 66 of Schaum’s outline): Let X be a random variable such that µX and V ar(X) exist. Then

P (|X − µ| > a) ≤ σ2X

a2=V ar(X)

a2for each a > 0. (1.42)

Letting a = ǫ, we conclude from (1.42) that

P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t))| > ǫ) ≤ V ar(X(t+ h) −X(t))

ǫ2. (1.43)

To proceed further we need the following observation: if Z is a random variable with mean µ then

P (|Z − µ| > ǫ) =

∫ µ−ǫ

−∞f(z)dz +

∫ ∞

µ+ǫ

f(z)dz. (1.44)

Note that −|µ| ≤ µ and µ ≤ |µ|. Thus,

∫ −|µ|−ǫ

−∞f(z)dz +

∫ ∞

|µ|+ǫ

f(z)dz ≤∫ µ−ǫ

−∞f(z)dz +

∫ ∞

µ+ǫ

f(z)dz (1.45)

Translating (1.45) into probability terms gives

P (|Z| > |µ| + ǫ) ≤ P (|Z − µ| > ǫ) . (1.46)

Setting Z = X(t+ h) −X(t) and µ = µ(t+ h) + µ(t) in (1.46), we obtain

P (|X(t+ h) −X(t)| > |µ(t+ h) − µ(t)| + ǫ) ≤ P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t)| > ǫ) . (1.47)

To reduce this further we use the observe that if 0 < λ < ǫ then

P (|Z| > 2ǫ) ≤ P (|Z| > λ+ ǫ) . (1.48)

10

Let h > 0 be small enough that |µ(t+ h) − µ(t)| < ǫ. It then follows from as in (1.44)-(1.46) that

P (|X(t+ h) −X(t)| > 2ǫ) ≤ P (|X(t+ h) −X(t)| > |µ(t+ h) − µ(t)| + ǫ) . (1.49)

Combining (1.49) with (1.47), we conclude that

P (|X(t+ h) −X(t)| > 2ǫ) ≤ P (|X(t+ h) −X(t) − (µ(t+ h) − µ(t)| > ǫ) . (1.50)

Finally, it follows from (1.43) and (1.48) that

P (|X(t+ h) −X(t)| > 2ǫ) ≤ V ar(X(t+ h) −X(t))

ǫ2. (1.51)

Replacing ǫ with ǫ2 in (1.51) gives

P (|X(t+ h) −X(t)| > ǫ) ≤ 4[V ar(X(t+ h) −X(t))]

ǫ2. (1.52)

Note that

V ar(X(t+ h) −X(t)) = E([X(t+ h) −X(t)]2) − (µ(t+ h) − µ(t))2 (1.53)

The continuity of µ(t), and the fact that X(t) is mean square continuous at t, imply that the right

side of (1.53) appoaches zero as h→ 0. This implies that

limh→0

P (|X(t+ h) −X(t)| > ǫ) = 0. (1.54)

(k.) First derivative properties of a random process

Definition. A random process X(t) has a mean square first derivative X ′(t) (p. 209 Schaum’s) if

limǫ→0

E

(

[

X(t+ ǫ) −X(t)

ǫ−X ′(t)

]2)

= 0. (1.55)

Property 1. If a random process X(t) has a mean square first derivative then

E (X ′(t)) = µ′X(t) ∀t ≥ 0. (1.56)

To prove this we follow (Schaum’s, p. 221, Problem 6.7) and obtain

E (X ′(t)) = limǫ→0E(

X(t+ǫ)−X(t)ǫ

)

= limǫ→0µX (t+ǫ)−µX (t)

ǫ = µ′X(t) ∀t ≥ 0.

(1.57)

Property 2. If a random process X(t) has a mean square first derivative then

RXX′(t, s) =∂RX(t, s)

∂s(1.58)

To prove this we follow Problem 6.7, p. 221 of Schaum’s and obtain

RXX′(t, s) = limǫ→0E(

X(t)X(s+ǫ)−X(s)ǫ

)

= limǫ→0E[X(t)X(s+ǫ)]−E[X(t)X(s)]

ǫ = ∂RX (t,s)∂s .

(1.59)

11

Property 3. If a WSS process X(t) has a mean square first derivative then

RXX′(t, t+ τ) =∂RX(s− t)

∂s|s=t+τ =

dRX(τ)

dτ. (1.60)

Property 4. If X(t) has a mean square first derivative then RX′(t, s) = ∂2RX(t,s)∂t∂s .

Remark. This property will be used to show that the Wiener process does not have a m. s. derivative.

To prove that RX′(t, s) = ∂2RX (t,s)∂t∂s start with Problem 6.7, p. 221 of Schaum’s and obtain

RX′(t, s) = E (X ′(t)X ′(t)) = limǫ→0

E

(

X(t+ ǫ) −X(t)

ǫX ′(t)

)

. (1.61)

This reduces to

RXX′(t, s) = limǫ→0E[X(t+ǫ)X′(s)]−E[X(t)X′(s)]

ǫ

= limǫ→0RXX′ (t+ǫ,s)−RXX′ (t,s)

ǫ |s=t = ∂2RX(t,s)∂t∂s .

(1.62)

Property 5. If a WSS process X(t) has a mean square first derivative then

RX′(t, t+ τ) =∂2RX(s− t)

∂t∂s|s=t+τ =

d2RX(τ)

dτ2. (1.63)

Property 6. If ∂2RX (t,s)∂t∂s |s=t exists then X(t) has a mean square first derivative.

The proof is in Porblem 6.6, p. 220 of Schaum’s.

Remarks. Our main application of these properties will be to the Wiener process W (t) which we will

show (in the next section) does not have a mean square first derivative. However, as we will see, W (t)

it will have a generalized function first derivative given by the delta function.

(l.) Stochastic integral properties of a random process.

Definition. Y (t) =∫ t

t0X(α)dα (p. 210 Schaum’s) is the mean square integral of X(t) if

lim∆ti→0

E

[

∫ t

t0

X(α)dα −∑

i

X(ti)∆ti

]2

= 0. (1.64)

Problem 6.11, p. 223 of Schaum’s proves that the mean square integral

∫ t

t0

X(α)dα exists ⇐⇒∫ t

t0

∫ t

t0

RX(α, β)dαdβ exists. (1.65)

(m.) How to see where the greatest uncertainty of a process occurs?

The greatest uncertainty occurs at the value of t where V ar(x(t)) is a maximum. For example, when

we solve the Brownian Bridge SDE, we find that if we impose x(0) = x(1) = 0 then E(x(t)) = 0 ∀t,and V ar(x(t)) = E(x2(t)) = t(1 − t). Thus, the maximum uncertainty occurs at t = 1/2.

(n.) The pdf’s for Z = Y +X and Z = Y −X.

12

First we follow ([26], p.123) and consider general Z = g(X,Y ). For a given z ∈ R define

DZ(z) = (x, y)|g(x, y) ≤ z. (1.66)

Let f(x, y) be the joint pdf for X and Y. Then the cdf FZ(z) is defined by

ProbZ ≤ z = FZ(z) =

∫ ∫

DZ(z)

f(x, y)dydx. (1.67)

If Z = Y +X then

DZ(z) = (x, y)| −∞ < x <∞, y ≤ z − x (1.68)

and

Prob Z ≤ z = FZ(z) =

∫ ∫

DZ (z)

f(x, y)dydx =

∫ ∞

−∞

∫ z−x

−∞f(x, y)dydx. (1.69)

The corresponding pdf for Z is the convolution

fZ(z) =d

dzProbZ ≤ z =

d

dzFZ(z) =

∫ ∞

−∞f(x, z − x)dx. (1.70)

If Z = Y −X then the pdf for Z is

fZ(z) =d

dzFZ(z) =

∫ ∞

−∞f(x, z + x)dx. (1.71)

Suppose that Y and X are functions of t and that Z = Y (t2) −X(t1). Then

fZ(z) =d

dz

∫ ∞

−∞

∫ z−x

−∞f(x, t1, y, t2)dydx =

∫ ∞

−∞f(x, t1, z + x, t2)dx. (1.72)

Remark. In order to make use of this formula we sometimes (see the Wiener process part) change

the order in the integrand and get

fZ(z) =

∫ ∞

−∞f(z + x, t2, x, t1)dx =

∫ ∞

−∞f(z + x, t2|x, t1)fX(x, t1)dx. (1.73)

(o.) Definition and computation of FX(x|a < X ≤ b).

This is from Schaum’s Outline, Ch. 2 - problems (2.50)-(2.53)). Recall that

P (X ≤ x) = FX(x) =

∫ x

−∞fX(η)dη, (1.74)

P (X ∈ (a, b]) = FX(b) − FX(a) =

∫ b

a

fX(η)dη, (1.75)

and the basic definition

FX(x|a < X ≤ b) = P (X ≤ x|a < X ≤ b) =P (X ≤ x and a < X ≤ b)

P (a < X ≤ b). (1.76)

To compute the right side of (1.76), start with

X ≤ x and a < X ≤ b ⇐⇒ X ∈ (−∞, x] ∩ (a, b]. (1.77)

13

Next, note that

(−∞, x] ∩ (a, b] =

φ, x ≤ a,

(a, x], a < x ≤ b,

(a, b], x > b.

(1.78)

Therefore,

X ∈ (−∞, x] ∩ (a, b] ⇐⇒

X ∈ φ, x ≤ a,

X ∈ (a, x], a < x ≤ b,

X ∈ (a, b], x > b.

(1.79)

From this we conclude that

P (X ∈ (−∞, x] ∩ (a, b]) =

P (X ∈ φ) = 0, x ≤ a,

P (X ∈ (a, x]) = FX(x) − FX(a), a < x ≤ b,

P (X ∈ (a, b]) = FX(b) − FX(a), x > b.

(1.80)

Finally, combine (1.76), (1.75) and (1.80), and get

FX(x|a < X ≤ b) =

0, x ≤ a,

FX (x)−FX(a)FX (b)−FX(a) , a < x ≤ b,

FX (b)−FX(a)FX (b)−FX(a) , x > b.

(1.81)

(p.) Proof that ρ(x, t|y, t) = δ(x− y) when X(t) is continuous

Assume that X(t) is a continuous random variable, and let φ ∈ C∞0 (R) be a test function with compact

support on an interval (a, b). To prove the result we need to show that

∫ ∞

−∞φ(x)ρ(x, t|y, t)dx =

∫ b

a

φ(x)ρ(x, t|y, t)dx = φ(y). (1.82)

Using Riemann sums, it follows that

∫ b

a

φ(x)ρ(x, t|y, t)dx = lim∆x→0

∑

i

φ(xi)ρ(xi, t|y, t)∆x. (1.83)

Consider the particular Riemann sum with the points xi defined by

x0 = y,

xi = y + i∆x, i 6= 0.

(1.84)

Then

14

∫ b

a φ(x)ρ(x, t|y, t)dx = lim∆x→0

∑i=Mi=−L φ(xi)ρ(xi, t|y, t)∆x

= φ(y)ρ(y, t|y, t)∆x+ lim∆x→0

∑i=−1i=−L φ(xi)ρ(xi, t|y, t)∆x

+ lim∆x→0

∑i=Mi=1 φ(xi)ρ(xi, t|y, t)∆x,

(1.85)

where M and L are large enough to insure that the interval (a, b) is contained in [y−L∆T, y+M∆t].

When i 6= 0 we observe that

φ(xi)ρ(xi, t|y, t)∆x = φ (xi) Prob

X(t) ∈ (xi − ∆x2 , xi + ∆x

2 )|X(t) = y

= 0, (1.86)

since X(t) = y /∈ (y + i∆x− ∆x2 , y + i∆x+ ∆x

2 ) when i 6= 0. We conclude that

∫∞−∞ φ(x)ρ(x, t|y, t)dx = lim∆x→0

∑i=Mi=−L φ(xi)ρ(xi, t|y, t)∆x

= lim∆x→0 φ(y)ρ(y, t|y, t)∆x

= lim∆x→0 φ (y) Prob

X(t) ∈ (y − ∆x2 , y + ∆x

2 )|X(t) = y

= φ(y)

(1.87)

since the condition X(t) = y implies that

Prob

X(t) ∈ (y − ∆x

2, y +

∆x

2)|X(t) = y

= 1 ∀∆x > 0.

(q.) The interpretation of∫

R1

∫

R2ρ(x′, t1, x′′, t2)dx′dx′′

According to Gardner [11], Ch. 5, we have the following interpreation:∫

R1

∫

R2ρ(x′, t1, x′′, t2)dx′dx′′ is the probability the particle is in R1 when t = t1, and the particle is

in R2 when t = t2.

(r.) Principle X: Proba < X ≤ b, c < Y ≤ d =∫ d

c Prob a < X ≤ b|Y = η fY (η)dη.

This principle provides a first step for proving results about Prob a < X ≤ b, c < Y ≤ d , which is

typically hard to do. For example, in the next section we use it to prove that, for the Poisson process,

(i) the interspike interval for the Poisson process is an exponential random,

(i) TN − TM and TM are independent random variables when N > M.

The proof of Principle X uses two basic concepts - Riemann sum and conditional probability:

Step I. Express Prob a < X ≤ b, c < Y ≤ d as a sum.

Let ∆η = (d−c)N > 0 and define ηi = c+ i∆η, 1 ≤ i ≤ N. Then

Proba < X ≤ b and c < Y ≤ d = Prob

a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η](1.88)

15

Step II To get a Riemann sum we use conditional probability and conclude that (2.29) reduces to


a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b|Y ∈ [c, c+ i∆η]ProbY ∈ [c, c+ i∆η]

≈∑

i≥1 Proba < X ≤ b|Y = ηi fY (ηi)∆η

(1.89)

The last expression is a Riemann sum. Thus, in the limit as ∆t→ 0, we obtain the integral formula

Prob[a < X ≤ b and c < Y ≤ d] =

∫ d

c

Prob a < X ≤ b|Y = η fY (η)dη (1.90)

16

2 The Poisson Process

(a.) Basic definition This follows [26], pp. 170-171. A counting process X(t)|t ≥ 0 satisfies

(i) X(t) ≥ 0 ∀t ≥ 0, X(t) is integer valued, and X(0) = 0.

(ii) X(t) is the number of events in [0, t) ∀t ≥ 0, X(s) < X(t), 0 ≤ s < t

A counting process X(t) is a Poisson process with rate λ > 0 if

(iii) X(t) has stationary, independent increments, and

(iv) The number of X(t) in any interval (s, t+ s) of length t satisfies

Prob[X(s+ t) −X(s) = n] = e−λt (λt)n

n!∀t, s ≥ 0.

Example I. Let 0 < t < s. If ∆t, ∆s are small enough then

X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)

are independent random variables. Thus

Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = Prob X(t+ ∆t) −X(t) = 1Prob X(s+ ∆s) −X(s) = 1(2.1)

Evaluation of the right side of (2.1) gives

Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1 = 0 = λ2∆t∆se−λ(∆t+∆s). (2.2)

Similarly,

ProbX(t+ ∆t) −X(t) = 0 and X(s+ ∆s) −X(s)) = e−λ(∆t+∆s. (2.3)

(b.) Proof that E(X(t)) = λt, E(X2(t)) = λt+ λ2t2 and V ar(X(t)) = λt.

It follows from the basic definition that

E(X(t)) =∑∞

k=0 ke−λt (λt)k

k!

=∑

k≥1 e−λt (λt)k

(k−1)! = λt∑

k≥1 e−λt (λt)k−1

(k−1)!

= λt∑

j≥0 e−λt (λt)j

j! = λt.

(2.4)

Similarly,

E(X2(t)) =∑∞

k=0 k2e−λt (λt)k

k!

= λt∑

k≥1 ke−λt (λt)k−1

(k−1)! = λt∑

j≥0(1 + j)e−λt (λt)j

(j)!

= λt(

∑

j≥0 e−λt (λt)j

j! +∑

j≥0 je−λt (λt)j

j!

)

= λt(1 + λt).

(2.5)

17

Finally, combining (2.4) and (2.5), we obtain

V ar(X(t)) = E(X2(t)) − E2(X(t)) = λt(1 + λt) − λ2t2 = λt. (2.6)

(c.) Application: estimate the charge on the electron.

Let X denote the number of electrons passing a given point in time inteval [0, t]. Assume that X is a

Poisson random variable with λ > 0 denoting the mean value of X in [0, t]. Current is defined by

I = eX, (2.7)

where e is the charge on a single electron. Thus, current is a random variable which is a constant

multiple of the Poisson random variable X. We conclude that

E(I) = eλ and V ar(I) = V ar(eX) = e2λ.

This leads to the formula

e =V ar(I)

E(I).

Thus, the charge on the elctron can be computed by measuring the mean current and its variation at

a given spatial point.

Below we will show that the spike train for the Poisson process is

y(t) =

∞∑

k=1

δ(t− Tk), (2.8)

where Tk is a gamma random variable parameters λ and k. The mean of the spike train is

E(y(t)) =∞∑

k=1

E(δ(t− Tk)) =∞∑

k=1

∫ ∞

0

δ(η − Tk)fTk(η)dη =

∞∑

k=1

fTk(t). (2.9)

Since Tk is a gamma random variable parameters λ and k, we conclude that

E(y(t)) =∞∑

k=1

λ

(k − 1)!e−λt)(λt)k−1 = λ (2.10)

The Poisson process X(t) satisfies

X(t) =

∫ t

0

∞∑

k=1

δ(t− Tk) =

∫ t

0

y(η)dη. (2.11)

This implies that

X ′(t) =

∞∑

k=1

δ(t− Tk) = y(η). (2.12)

We conclude that the derivative of the Poisson process is the spike train, and that

E(y(t)) =d

dtE(X(t)) = E(X ′(t)) =

∞∑

k=1

E(δ(t− Tk)) = λ. (2.13)

18

Thus, for the current defined above in (2.7), we conclude that

dI

dt= eX ′(t) = e

∞∑

k=1

δ(t− Tk) = ey(η). (2.14)

(d.) Proof that E(X(t)X(s)) = λmin(t, s) + λ2ts and Cov(X(t), X(s)) = λmin(t, s).

Let t > s ≥ 0 and recall that X(t) −X(s) and X(t− s) have the same pdf. Thus,

E([X(t) −X(s)]2) = E([X(t− s)]2) = λ(t− s) + λ2(t− s)2. (2.15)

Also, an expansion gives

E([X(t) −X(s)]2) = E(X2(t)) + E(X2(s)) − 2E(X(t)X(s))

= λt+ λ2t2 + λs+ λ2s2 − 2E(X(t)X(s)).

(2.16)

Combining (2.15) and (2.16) gives the autocorrelation

E(X(t)X(s)) = λmin(t, s) + λ2ts. (2.17)

Next, from the definition of covariance we obtain

Cov(X(t), X(s)) = E(X(t)X(s)) −E(X(t))E(X(s)) = λmin(t, s) + λ2ts− λ2ts = λmin(t, s). (2.18)

(e.) The variation, and coefficients of correlation and variation.

From (2.17) it follows that E(X2(t)) = λt− λ2t2. Thus, the varaiation of X(t) is

σ2 = V ar(X(t)) = E(X2(t)) − E2(X(t)) =(

λt+ λ2t2)

− (λt)2 = λt. (2.19)

The coefficient of variation of two random variables X and Y is

ρ =Cov(X,Y )

√

V ar(X)√

V ar(Y ). if 0 < s ≤ t. (2.20)

According to Wikepedia, the covariance between two random variables X and Y is

Cov(X,Y ) = E(XY ) − E(X)E(Y ), (2.21)

and

Cov(aX, bY ) = abCov(X,Y ). (2.22)

According to Wikepedia, the autocovariance for the two random variables X(t) and X(s) is

KXX(t, s) = E(X(t)X(s)) − E(X(t))E(X(s)) if 0 < s ≤ t. (2.23)

For a wide sense stationary process, µ = E(X(t)) and

KXX(τ) = E(X(t)X(t+ τ)) − E(X(t))E(X(t+ τ)) = RXX(τ) − µ2 (2.24)

19

According to Wikepedia, cross correlation and Cov(X,Y ) are sometimes referred to as the same thing.

According to Wikepedia, the coefficient of variation between two random variables X and Y is

ρXY =Cov(X,Y )

σXσY, σ2

X = V ar(X). (2.25)

A nice discussion is in http : //www.tufts.edu/ gdallal/corr.htm where they tell you to always look at

scatter plots. The site http : //en.wikipedia.org/wiki/Image : Oldfaithful3.png has an interesting

scatterplot showing two different waiting times for the old faithful geyser eruption time.

According to Wikepedia, the sample correlation is

rXY =N∑

xiyi − xy√

N∑

x2i − (

∑

xi)2√

N∑

y2i − (

∑

yi)2(2.26)

According to http : //www.vias.org/tmdatanaleng/cccorrcoeff.html, for a sample correlation we

need the following assumptions on the random variables x and y are:

*(i) linear relationship between x and y

(ii) continuous random variables and both variables must be normally distributed

(iii) x and y must be independent of each other

According to Wikepedia, the coefficient of variation is

CV =

√

V ar(X(t)

E(X(t))=

√λt

λt=

1√λt

if 0 < s ≤ t. (2.27)

(f.) Proof that CV=1 for T1.

Recall that T1 is an exponential random variable with

fT1(t) = λexp(−λt),

hence

E(T1) =

∫ ∞

0

tfT1(t)dt =1

λand E(T 2

1 ) =

∫ ∞

0

t2fT1(t)dt =2

λ2.

Therefore,

CV =

√

V ar(T1)

E(T1)= 1 (2.28)

(g.) Proof that fTN (t) = λe−λt (λt)N−1

(N−1)! and E(T kN ) = N(N+1)..(N+k−1)

λk .

(i) The N-th passage time is the random variable TN which is defined by

TN = inft > 0|X(t) > N ∀N ≥ 1.

To find the pdf of TN , we let t ≥ 0 and compute

Prob[TN > t] = Prob[X(t) ≤ N − 1] =N−1∑

i=0

e−λt (λt)i

i!.

20

Thus,

Prob[TN ≤ t] = 1 −N−1∑

i=0

e−λt (λt)i

i!,

and therefore

fTN (t) =d

dtProb[TN ≤ t] = λe−λt (λt)N−1

(N − 1)!.

(ii) To show that E(T kN ) = N(N+1)..(N+k−1)

λk we use the moment generating function and compute

E(eζTN ) =

∫ ∞

0

eζtfTN (t)dt =

∫ ∞

0

eζtλe−λt (λt)N−1

(N − 1)!dt

This gives

E(eζTN ) =λN

Γ(N)

∫ ∞

0

tN−1e(ζ−λ)tdt =

(

λ

ζ − λ

)N

, ζ < λ.

Divide top and bottom by λ, use the binomial expansion and obtain

E(eζTN ) =

∞∑

i=0

N(N + 1)..(N + k − 1)

λk

ζk

k!

Fromn this we conclude that

E(T kN) =

N(N + 1)..(N + k − 1)

λk

(h.) Principle X: Prob a < X ≤ b, c < Y ≤ d =∫ d

c Proba < X ≤ b|Y = η fY (η)dη.

This principle, which is repeated from the previous section, provides a very important and useful

first step for proving results about Prob a < X ≤ b, c < Y ≤ d , which is typically hard to do. For

example, below we use it to prove that, for the Poisson process,

(i) the interspike interval for the Poisson process is an exponential random,

(i) TN − TM and TM are independent random variables when N > M.

The proof of Principle X uses two basic concepts - Riemann sum and conditional probability:

Step I. Express Prob a < X ≤ b, c < Y ≤ d as a sum.

Let ∆η = (d−c)N > 0 and define ηi = c+ i∆η, 1 ≤ i ≤ N. Then


a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η](2.29)

Step II To get a Riemann sum we use conditional probability and conclude that (2.29) reduces to


a < X ≤ b and Y ∈ ∪Ni=1[c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b and Y ∈ [c, c+ i∆η]

=∑

i≥1 Proba < X ≤ b|Y ∈ [c, c+ i∆η]ProbY ∈ [c, c+ i∆η]

≈∑i≥1 Proba < X ≤ b|Y = ηi fY (ηi)∆η

(2.30)

21

The last expression is a Riemann sum. Thus, in the limit as ∆t→ 0, we obtain the integral formula

Prob[a < X ≤ b and c < Y ≤ d] =

∫ d

c

Prob a < X ≤ b|Y = η fY (η)dη (2.31)

(i.)Proof that ZN = TN − TN−1 is an exponential random variable.

Our goal is to prove that

Prob ZN = TN − TN−1 ≤ z = 1 − e−λz ∀z ≥ 0.

The proof uses (i.) the method of proof of Principle X, and (ii.) the key ”probability link”

ProbTN − TN−1 > z|TN−1 = τ = Prob X(z + τ) −X(τ) = 0 = e−λz. (2.32)

First, we follow the method of proof of Principle X and obtain and integral formula for ProbZN ≤ z .Let ∆τ > 0 be small and define

τi = i∆τ, i ≥ 0.

Note that the events ZN > z and ZN > z and TN−1 ∈ [0,∞)] are equivalent. Therefore

ProbZN > z = ProbZN > z and TN−1 ∈ [0,∞)]

= Prob ZN > z and TN−1 ∈ ∪i≥0[τi, τi+1)

=∑

i≥0 Prob ZN > z and TN−1 ∈ [τi, τi+1)

=∑

i≥0 Prob ZN > z|TN−1 ∈ [τi, τi+1)Prob TN−1 ∈ [τi, τi+1)

≈∑i≥0 Prob ZN > z|TN−1 = τi fTN (τi)∆τ

≈∫∞0

ProbZN > z|TN−1 = τ fTN (τ)dτ

(2.33)

Thus, in the limit as ∆t→ 0, we obtain the integral formula

ProbZN > z =

∫ ∞

0

ProbZN > z|TN−1 = τ fTN (τ)dτ (2.34)

From this and (2.32) we conclude that

Prob ZN > z =∫∞0 ProbZN > z|TN−1 = τ fTN (τ)dτ

=∫∞0

Prob X(z + τ) −X(τ) = 0 fTN (τ)dτ

=∫∞0 e−λzfTN (τ)dτ = e−λz

∫∞0 fTN (τ)dτ

= e−λz

(2.35)

From this we conclude that ProbZN ≤ z = 1 − e−λz as claimed.

22

(j.) If Z = TN − TM , N > M then fZ(z) = λN−M

(N−1−M)!e−λzzN−1−M .

The first step is to obtain a formula for ProbZ > z . Using the same method as in the previous

subsection, we conclude that

Prob Z > z = ProbZ > z and TM ∈ [0,∞)] =∫∞0

ProbZ > z|TM = τ fTM (τ)dτ.

(2.36)

Next, we observe the key probability equivalence

ProbTN − TM > z|TM = τ =

N−1−M∑

i=0

ProbX(z + τ) −X(τ) = i =

N−1−M∑

i=0

e−λz (λz)i

i!. (2.37)

Combining (2.36) and (2.37) gives

ProbZ > z =

∫ ∞

0

N−1−M∑

i=0

e−λz (λz)i

i!fM (τ)dτ =

N−1−M∑

i=0

e−λz (λz)i

i!

∫ ∞

0

fM (τ)dτ. (2.38)

This, and the fact that∫∞0 fM (τ)dτ = 1, imply

ProbZ ≤ z = 1 −N−1−M∑

i=0

e−λz (λz)i

i!

∫ ∞

0

fM (τ)dτ = 1 −N−1−M∑

i=0

e−λz (λz)i

i!. (2.39)

Finally, it follows from (2.39) that

fZ(z) =d

dzProb Z ≤ z =

λN−M

(N − 1 −M)!e−λzzN−1−M . (2.40)

(k.) Proof that Z2 = T2 − T1 and Z1 = T1 are independent.

Our goal is to prove the following result: if x > 0 and y > 0 then

Prob Z2 ≤ x, T1 ≤ y = Prob Z2 ≤ xProbT1 ≤ y .

The proof of this property consists of two steps: I. use the method of proof of Principle X to develop

an integral formula involving a product of a conditional probability and a simpler probability density

function, and II. reduce the conditional probability part of the integrand to a function which is

independent of the variable of integration so that it can can be taken outside of the integral

Step I. First, note that the range of T1 is 0 ≤ T1 ≤ y. Thus, as in the proof of Principle X, we

subdivide the interval [0, y] into subintervals. Let N > 1 be large and set ∆y = yN . Define

τi = i∆y, 0 ≤ i ≤ N.

23

We observe that the events [Z2 ≤ x and T1 ≤ y] and [Z2 ≤ x and T1 ∈ [0, y]] are equivalent. Then

Prob[Z2 ≤ x and T1 ≤ y] = Prob[Z2 ≤ x and T1 ∈ [0, y]]

= Prob[Z2 ≤ x and T1 ∈ ∪1≤i≤N [τi−1, τi)]

=∑

1≤i≤N Prob[Z2 ≤ x and T1 ∈ [τi−1, τi)]

=∑

1≤i≤N Prob[Z2 ≤ x|T1 ∈ [τi−1, τi)]Prob[T1 ∈ (τi−1, τi]]

≈∑

1≤i≤N Prob[Z2 ≤ x|T1 = τi−1]fT1(τi−1)∆τ

≈∫ y

0Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ

(2.41)

Thus, in the limit as ∆t→ 0, we obtain the integral formula

Prob[Z2 ≤ x and T1 ≤ y] =

∫ y

0

Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ. (2.42)

Step II. From this integral formula we conclude that

Prob[Z2 ≤ x and T1 ≤ y] =∫ y

0Prob[Z2 ≤ x|T1 = τ ]fT1(τ)dτ

=∫ y

0 Prob[T2 − T1 ≤ x|T1 = τ ]fTN (τ)dτ

=∫ y

0(1 − Prob[T2 − T1 > x|T1 = τ ]) fTN (τ)dτ

=∫ y

0 (1 − e−λx)fTN (τ)dτ

= (1 − e−λx)∫ y

0fTN (τ)dτ

= (1 − e−λx)(1 − e−λy) = Prob[Z2 ≤ x]Prob[T1 ≤ y].

(2.43)

(l.) Proof that Z = TN − TM , N > M and TM are independent.

Our goal is to prove the following result: if x > 0 and y > 0 then

ProbZ ≤ x, 0 ≤ TM ≤ y = Prob Z ≤ xProb0 ≤ TM ≤ y .

The proof uses (i.) Principle X, and (ii.) the fact that the Ti’s increase as i increases implies the key

”probability link”

Prob TN − TM > x|TM = τ =

N−1−M∑

i=0

Prob X(z + τ) −X(τ) = i =

N−1−M∑

i=0

e−λz (λz)i

i!. (2.44)

First, we conclude from Principle X that

Prob Z ≤ x and 0 ≤ TM ≤ y =

∫ y

0

ProbZ ≤ x|TM = τ fTM (τ)dτ. (2.45)

24


ProbZ ≤ x and 0 ≤ TM ≤ y =∫ y

0ProbZ ≤ x|TM = τ fTM (τ)dτ

=∫ y

0 ProbTN − TM ≤ x|TM = τ fTM (τ)dτ

=∫ y

0(1 − ProbTN − TM > x|TM = τ) fTM (τ)dτ

=∫ y

0

(

1 −∑N−1−Mi=0 ProbX(z + τ) −X(τ) = i

)

fTM (τ)dτ

=(

1 −∑N−1−Mi=0 e−λz (λz)i

i!

)

∫ y

0 fTM (τ)dτ

= Prob Z ≤ xProb0 ≤ TM ≤ y .

(2.46)

(m.) Two proofs that E(y(t)) = λ for the spike train y(t) =∑∞

k=1 δ(t− Tk).

Method I. This method relies on knowing the exact form of the pdf’s of the random variables Tk.

The spike train for the Poisson process is

y(t) =

∞∑

k=1

δ(t− Tk), (2.47)

where Tk is a gamma random variable parameters λ and k. The mean of the spike train is

E(y(t)) =∞∑

k=1

E(δ(t− Tk)) =∞∑

k=1

∫ ∞

0

δ(t− η)fTk(η)dη =

∞∑

k=1

fTk(t). (2.48)

Since Tk is a gamma random variable parameters λ and k, we conclude that

E(y(t)) =

∞∑

k=1

λ

(k − 1)!e−λt(λt)k−1 = λ (2.49)

Method II. This method relies on knowing the exact formula for E(X(t)). The Poisson process X(t)

satisfies E(X(t)) = λt. Furthermore,

X(t) =

∫ t

0

∞∑

k=1

δ(t− Tk) =

∫ t

0

y(η)dη, (2.50)

hence

X ′(t) =

∞∑

k=1

δ(t− Tk) = y(η). (2.51)

We conclude that the derivative of the Poisson process is the spike train, and that

E(y(t)) =d

dtE(X(t)) = E(X ′(t)) = λ =

∞∑

k=1

E(δ(t− Tk)). (2.52)

(n.) Two proofs that the spike train y(t) satisfies E(y(t)y(s)) = λ2 when s > t.

25

Method I. This method relies on knowing the exact formula for E(X(t)X(s)). For the Poisson process

we recall from (2.17) that

E(X(t)X(s)) = λmin(t, s) + λ2ts. (2.53)

From this it follows that

E(y(t)y(s)) = E(X ′(t)X ′(s)) =∂2

∂t∂sE(X(t)X(s)) = λ2. (2.54)

Method II. This method proceeds in two steps: step 1 notes thatX(t) has independent, stationary

increments and step 2 uses these two properties to formulate a reasonable approach to compute

E(X ′(t)X ′(s)).

Remarks:

(i) Is this a reasonable approach to use to compute the autocorellation E(y(t)y(t + τ)) for a spike

train associated with a solution of a LIF SDE?

(ii) Is it true that the arguments given below imply that E(y(t)y(t+ τ)) = E(y(t))E(y(t + τ))

First, since X(t) is the Poisson process, we conclude that if 0 < t < s and ∆t, ∆s are small enough

then

X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)

are independent random variables.

Secondly, this fact allows us to compute E(X ′(t)X ′(s)). That is,

E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)

E

((

X(t+ ∆t) −X(t)

∆t

)(

X(s+ ∆s) −X(s)

∆s

))

(2.55)

becomes

E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)

E

((

X(t+ ∆t) −X(t)

∆t

))

E

((

X(s+ ∆s) −X(s)

∆s

))

. (2.56)

Because the Poisson process is stationary and X(0) = 0, this further reduces to

E(y(t)y(s)) = E(X ′(t)X ′(s)) = lim(∆t,∆s)→(0,0)

E

((

X(∆t)

∆t

))

E

((

X(∆s)

∆s

))

= λ2. (2.57)

26

(o.) Proof that the spike train satisfies E(y(t)y(s)) = λδ(t− s) + λ2

The first step is to recall that

E(X(t)X(s)) =

λs+ λ2ts, if t > s,

λt+ λ2ts, if t < s,(2.58)

and that

E(y(t)y(s)) = E

(

dX(t)

dt

dX(s

ds

)

. (2.59)

Thus,

E (X(t)y(s)) = E

(

X(t)dX(s

ds

)

=∂

∂sE (X(t)X(s)) . (2.60)

Combining (2.58) with (2.60) gives

E(X(t)y(s)) =

λ+ λ2t, if t > s,

0 + λ2t, if t < s.(2.61)

This can be written as

E (X(t)y(s)) = λH(t− s) + λ2t, (2.62)

Where H(t−s) is the heaviside function. From (2.63), and the fact that H ′(t−s) = δ(t−s), it follows

that

E (y(t)y(s)) = λδ(t− s) + λ2. (2.63)

27

(p.) Proof that E(y2(t)) = ∞ and V ar(y(t)) = ∞

The spike train stisfies

y(t) =d

dtX(t) = lim

∆t→0

(

X(t+ ∆t) −X(t)

∆t

)

Therefore,

E(y2(t)) = lim∆t→0

E

(

(

X(t+ ∆t) −X(t)

∆t

)2)

= lim∆t→0

1

(∆t)2E(

(X(t+ ∆t) −X(t))2)

(2.64)

Because X(t) is stationary this further reduces to

E(y2(t)) = lim∆t→0

1

(∆t)2E(

(X(∆t))2)

= lim∆t→0

λ∆t

(∆t)2= lim

∆t→0

λ

∆t= ∞ (2.65)

Thus, to compute V ar(y(t), we combine (2.52) and (2.65), and obtain

E(y2(t)) − E2(y(t)) = ∞− λ2 = ∞. (2.66)

(q.) Proof that Probone spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s).

Let 0 < t < s. If ∆t, ∆s are small enough then

X(t+ ∆t) −X(t) and X(s+ ∆s) −X(s)

are independent random variables. Thus,

Prob one spike in (t, t+ ∆t) and one spike in (s, s+ ∆s)

is equivalent to

Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1

= ProbX(t+ ∆t) −X(t) = 1Prob X(s+ ∆s) −X(s) = 1(2.67)

Evaluation of the right side of (2.67) gives

Prob X(t+ ∆t) −X(t) = 1 and X(s+ ∆s) −X(s) = 1 = λ2∆t∆se−λ(∆t+∆s). (2.68)

Thus,

Prob one spike in (t, t+ ∆t) and one spike in (s, s+ ∆s) = λ2∆t∆se−λ(∆t+∆s). (2.69)

28

3 Spiking Theory

(a.) The definition of input current.

This follows de la Rocha et al [2]. Corellated fluctuating currents resembling synaptic activity are

injected into the somata of twenty unconnected cortical neurons. The total input current to cell i is

Ii(t) = µi + σi

(√1 − cζi(t) +

√cζc(t)

)

, (3.1)

where µi is temporal average of the current. The term

σi

(√1 − cζi(t) +

√cζc(t)

)

represents Gaussian fluctuations with a temporal structure consistent with that in-vivo (how are they

consistent?), and is composed of two weighted factors: ζi(t), which is independent for each cell, and

ζc(t), which is common to all cells. The parameter σi sets the variance of the total input current. The

weight of the shared fluctuations between adjacent pairs of cells is set by the ”corelation coefficient”

c, where 0 ≤ c ≤ 1.

Q1. Are the cells adjacent in reality?

(b.) Derivation of the correlation coefficient c.

This follows the derivation in the first part of the Methods Section of [2]. Let k denote trial number,

with 1 ≤ k ≤ 100. Then total input current to cell i is

Iki (t) = µi + σi

(√cζk

c (t) +√

1 − cζki (t)

)

(3.2)

The cross correlation function of two random variables X(t) and Y (t) is, according to p. 211 of

Schaums’,

E (X(t)Y (t+ τ)) , τ ∈ (−∞,∞). (3.3)

They define cross correlation function of a pair of currents by

Gkk′

ij = E(

Iki (t)Ik′

j (t+ τ))

− µiµj . (3.4)

Q2. This is slightly differently from (3.3) since they subtract off µiµj . Why do they define Gkk′ij as a

cross correlation?

When the right side of (3.4) is computed, the independence of ζki (t), ζk

j (t) and ζkc (t) force several

terms to be zero, and they get

Gkk′

ij = δkk′σiσj [δij + (1 − δij)c]E(

ζkc (t)ζk′

c (t+ τ))

, (3.5)

where δij is the Kronecker delta. This formula seems right.

Q3. What is the range of t? Is t ≥ 0 since t represents time?

29

Let k = k′ and i 6= j. Then (3.5) becomes

Gkkij = σiσjcE

(

ζkc (t)ζk′

c (t+ τ))

= σiσjcδ(τ). (3.6)

If k = k′ and i = j then (3.5) becomes

Gkkii = σiσiδ(τ). (3.7)

Q4. Are these calculations right?

They claim the corellation coefficient of a pair of input currents is

cij =

∫∞−∞Gkk

ij (τ)dτ∫∞−∞Gkk′

ii (τ)dτ= c. (3.8)

However, I seem to get

cij =

∫∞−∞Gkk

ij (τ)dτ∫∞−∞Gkk

ii (τ)dτ=σiσj

σiσic =

σj

σic. (3.9)

Q5. Which of these two answers is right? Also, why are they integrating? The definition

of corelatation coefficient does not seem to involve an integral.

To get c from the computation should (3.9) be

cij =

∫∞−∞Gkk

ij (τ)dτ(

∫∞−∞Gkk

ii (τ)dτ)1/2 (

∫∞−∞Gkk

jj (τ)dτ)1/2

=σiσj

σiσjc = c. (3.10)

30

(c.) The LIF model - Benji’s notes

These notes are a summary of our discussions about Beji Lindner’s thesis starting on Feb. 6, 2008.

The Leaky Integrate and Fire (LIF) model with trasmembrane constant τ is the equation

τdV

dt= (µ− V ) + σ

dζ

dt, (3.11)

together with the reset condition

V (t−) = vT and V (t+) = vR. (3.12)

Remark. If σ =√

2D then we get Benji’s equation

D =σ2

2. (3.13)

We set τ = 1 and study the simpler equation

dV

dt= (µ− V ) + σ

dζ

dt, . (3.14)

Remarks.

(i) Typical values for vT and vR are

vT = 1 and vR = 0. (3.15)

(ii) If µ < 1 and σ = 0 then there can be no spike since V (t) just increases to µ as t→ ∞.

(iii) If µ > 1 and σ = 0 then we have a deterministic train of equally spaced spikes.

(iv) The law of large numbers links the mean of the interspike intervals (a deterministic number) with

a limiting sum of random variables as follows: Let T be the random variable which denotes the time

the solution V takes to go from V = vR to V = vT , i.e. T is a first passage time. Also, let ti denote

the spike times (random variables). The quantities ti − ti−1 are i.i.d. random variables representing

the interspike intervals, each with mean E(ti − ti−1) = E(T ). Because of the law of large numbers,

Benji (equation (2.10)) can conclude that

E(T ) = limN→∞

1

N

N∑

i=0

(ti − ti−1). (3.16)

The Fokker-Planck equation associated with problem (3.14)-(3.23) is

∂P

∂t=

∂

∂V

(

(V − µ) +σ2

2

∂

∂V

)

P + ν(t)δ(V − vR). (3.17)

31

Question 1. Does Benji set V (0) = 0 ? If yes, then P (V, t) = (V, t|0, 0) is the solution of the

Fokker-Planck equation? Brent says that he himself assumes that P (V, 0) = δ(V ), and therefore his

P (V, t) = (V, t|0, 0). He also pointed out that Benji is only interested in the stationary pdf case, and

therefore Benji does not assume a specific intitial value for V (0).

Let P (V, t) denote the solution. Since the range of values of solutions of (3.14)-(3.23) is −∞ < V (t) <

vT , then P (V, t) satisfies∫ vT

−∞P (V, t)dV = 1 ∀t ≥ 0. (3.18)

Question 2. Do we interpret (3.18) as

Prob−∞ < V (t) < vT =

∫ vT

−∞P (V, t)dV = 1? (3.19)

Next, since we need the integral in (3.18) to converge, then it is reasonable to assume that

limV →−∞

P (V, t) = 0. (3.20)

The absorbing boundary condition at V = vT is

P (vT , t) = 0. (3.21)

32

(d.) How to solve τ dVdt = (µ− V ) + σ dζ

dt .

The Leaky Integrate and Fire (LIF) model with trasmembrane constant τ is the equation

τdV

dt= (µ− V ) + σ

dζ

dt, (3.22)


V (t−) = vT and V (t+) = vR. (3.23)

Next, multiply (3.22) by dt/τ and get

dV =(µ− V )

τdt+ σ

√

1

τdW. (3.24)

The numerical solution. We let vR = 0, vT = 1 and assume that V (0) = 0. The time step is

h = dt. The numerical Euler algorithm to solve (3.24) in matlab is

V (i+ 1) = V (i) +(µ− V (i))

τh+ σ

√

h

τrandn, V (1) = 0, (3.25)

A reasonable neuronal value for τ is τ = .1 However, we let τ = 1, and (3.25) becomes

V (i+ 1) = V (i) +(µ− V (i))

τh+ σ

√hrandn, V (1) = 0, (3.26)

together with the rest condition. We use time step h = .01 for experimental purposes, and h = .001

for publication.

The refractory period. After a spike, when we reset V to vR we need to keep V (t) = vR = 0

for a short interval of t values called the refractory period. We set taurefractory = .1 in our matlab

program.

33

(e.) How to analyze E (X(t)X(t+ τ)) when X(t) solves an SDE?

Our goal is to develp a useful formula for the autocorellation function E (X(t)X(t+ τ)) . We do this

in two parts:

I. Basic properties of E (X(t)X(s)) when s ≥ 0 and t > s.

II. The derivation of the formula for E (X(t)X(s)) .

Part I. Basic properties of E (X(t)X(s)) when s ≥ 0 and t > s.

Our goal here is to analyze basic properties of E (X(t)X(s)) when X(t) solves the SDE

dX

dt= f(X) + σζ(t), X(s) = x, and t > s ≥ 0. (3.27)

where ζ(t) is white noise.

The Fokker-Planck equation for the conditional probability density function associated with (3.27) is

∂

∂tρ(x′, t|x, s) = − ∂

∂x′(f(x′)ρ(x′, t|x, s)) +

σ2

2

∂2

∂x′2ρ(x′, t|x, s), t > s ≥ 0. (3.28)

When s = t the solution ρ(x′, s|x, s) is required to satisfy

ρ(x′, s|x, s) = δ(x′ − x) (3.29)

The function E (X(t)X(s)) is defined by

E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, t;x, s)dx′dx. (3.30)

Note that ρ(x′, t;x, s) can be written as

ρ(x′, t;x, s) = ρ(x′, t|x, s)ρ(x, s) (3.31)

Substituting (3.31) into (3.30) transforms (3.30) into the more amenable form

E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, t|x, s)ρ(x, s)dx′dx. (3.32)

Note that the function ρ(x′, t|x, s) in the integrand is the solution of problem (3.28)-(3.29).

Question 1. On the right side of the SDE (3.27) the only place that t appears is in the white noise

function ζ(t). Does the property allow us to ”shift to the left” by amount s and conclude that

ρ(x′, t|x, s) = ρ(x′, t− s|x, 0) = ρ(x′, τ |x, 0), τ = t− s. (3.33)

If (3.33) holds then substitution of (3.33) into (3.32) gives

E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, t− s|x, 0)ρ(x, s)dx′dx. (3.34)

Part II. The derivation of the formula for E (X(t)X(s)) .

34

Replace t and s in (3.34) with t+ τ and t, respectively, and transform (3.34) into

E (X(t+ τ)X(t)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, τ |x, 0)ρ(x, t)dx′dx. (3.35)

The function ρ(x′, τ |x, 0) solves the the Fokker-Planck equation

∂

∂tρ(x′, τ |x, 0) = − ∂

∂x′(f(x′)ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0), τ > 0. (3.36)

At τ = 0 the function ρ(x′, τ |x, 0) satisfies the ususal condition

ρ(x′, 0|x, 0) = δ(x′ − x). (3.37)

Also, the function ρ(x′, τ |x, 0) is pdf for the solution X(τ) of the SDE

dX

dτ= f(X) + σζ(τ), X(0) = x, and τ ≥ 0. (3.38)

Question 2. Next, we need to know what problem the function ρ(x, t) solves. I think that

ρ(x, t) = ρ(x, t|0, 0), (3.39)

which is the Fokker-Planck pdf for the solution of

dX(t)

dt= f(X(t)) + σζ(t), X(0) = 0, and t > 0. (3.40)

Substitute (3.39) into (3.35) and get

E (X(t+ τ)X(t)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, τ |x, 0)ρ(x, t|0, 0)dx′dx. (3.41)

This formula allows us to see the contributions from both τ and t to the of the autocorellation function

E (X(t+ τ)X(t)) .

Remarks. In order to use (3.41) to analyze the behavior of E (X(t+ τ)X(t)) when t >> 1, we need

to determine the behavior of ρ(x, t|0, 0) when t is large. One possibility is that ρ(x, t|0, 0) approaches

a stationary solution as t → ∞. A stationary solution of the Fokker-Planck equation (3.28) is a time

independent solution ρS(x′) of

0 = − d

dx′(f(x′)ρS(x′)) +

σ2

2

d2

dx′2ρS(x′), (3.42)

where ρS(x′) is bounded, ρS ∈ C2(R), and∫ ∞

−∞ρS(x′)dx′ = 1. (3.43)

Note that (3.42) can be written as

0 = − d

dx′J(x′), (3.44)

where J(x′) is the time independent flux

J(x′) = f(x′)ρS(x′) − σ2

2

d

dx′ρS(x′). (3.45)

35

(f.) Gardner’s eigenfunction approach to finding ρ(x, t).

Goal. We develop Gardner’s eigenfunction exapansion method ([11], pp. 129-131) to construct solu-

tions of the Fokker-Planck equation associated with the SDE

dX

dt= A(X) +

√

B(x)ζ(t), X(s) = x0, (3.46)

where ζ(t) is white noise. The associated Fokker-Planck equation is

∂

∂tρ(x, t) = − ∂

∂x(A(x)ρ(x, t)) +

1

2

∂2

∂x2(B(x)ρ(x, t)) . (3.47)

We assume that ρ(x, t) solves (3.47) on the interval a < xb, that a < b are finite, and that ρ(x, t)

satisfies the reflective boundary conditions

0 = −A(x)ρ(x, t) +1

2

∂

∂x(B(x)ρ(x, t)) when x = a, x = b. (3.48)

Remark. In the next subsection we show how to use the expansion method to construct ρ(x, t|x0, 0),

the particular solution associated with the SDE (3.46).

Step I. Find the stationary solution ρs(x), which solves

0 = − ∂

∂x(A(x)ρs(x)) +

1

2

∂2

∂x2(B(x)ρs(x)) . (3.49)

Assuming that ρs(x) satisfies reflective boundary conditions implies that

0 = −A(x)ρs(x) +1

2

∂

∂x(B(x)ρs(x)) ∀x ∈ [a, b]. (3.50)

This equation plays a key role in Step IV below.

Step II. Define the function q(x, t) associated with ρs(x) and ρ(x, t) by setting

ρ(x, t) = ρs(x)q(x, t). (3.51)

Combining (3.47), (3.50) and (3.51) shows that q(x, t) satisfies the backwards equation

∂

∂tq(x, t) = A(x)

∂

∂xq(x, t) +

B(x)

2

∂2

∂x2q(x, t). (3.52)

Step III. Find the eigenvalues and eigenfunctions associated with ρ(x, t) and q(x, t). These satisfy

ρ(x, t) = Pλ(x)e−λt and q(x, t) = Qλ′ (x)e−λ′t. (3.53)

It is important for Step IV below to observe that, if λ = λ′, then

Pλ(x) = ρs(x)Qλ(x). (3.54)

Next, it is easily verified that Pλ(x) and Qλ′ (x) satisfy the ODE’s

−λPλ(x) = − d

dx(A(x)Pλ(x)) +

1

2

d2

dx2(B(x)Pλ(x)) , (3.55)

36

and

−λ′Qλ′ (x) = A(x)

d

dxQλ′ (x) +

B(x)

2

d2

dx2Qλ′ (x). (3.56)

The boundary conditons astisfied by Pλ(x) are

0 = −A(x)Pλ(x) +1

2

d

dx(B(x)Pλ(x)) when x = a, x = b. (3.57)

When λ = λ′= 0 the eigenfunctions are

P0(x) = ρs(x) and Q0(x) = 1 ∀x ∈ (a, b). (3.58)

Furthermore, by scaling ρs(x), we conclude that

∫ b

a

P0(x)Q0(x)dx =

∫ b

a

ρs(x)dx = 1 (3.59)

Step IV. Show that the eigenfunctions satisfy the orthognality conditions

∫ b

a

Pλ(x)Qλ′ (x)dx 6= 0 when λ = λ′, (3.60)

and∫ b

a

Pλ(x)Qλ′ (x)dx = 0 when λ 6= λ′. (3.61)

First, when λ = λ′we see that

∫ b

a

Pλ(x)Qλ(x)dx =

∫ b

a

ρs(x) (Qλ(x))2dx 6= 0. (3.62)

This is consistent with (3.59).

Next, multiply (3.55) by Qλ′ (x), and (3.56) by Pλ(x), subtract the two equations, and integrate the

resulting equation from x = a to x = b to get

(λ′ − λ)

∫ b

a

Pλ(x)Qλ′ (x)dx =

[

Qλ′ (x)

(

−A(x)Pλ(x) +1

2(B(x)Pλ(x))′

)

− 1

2B(x)Pλ(x)Q′

λ′ (x)

]b

a(3.63)

To prove (3.61) we nee to show that the right side of (3.63) is zero when λ 6= λ′.

It follows from (3.57) that the first term on the right side of (3.63) is zero. This reduces (3.63) to

(λ′ − λ)

∫ b

a

Pλ(x)Qλ′ (x)dx =

[

−1

2B(x)Pλ(x)Q′

λ′ (x)

]b

a

(3.64)

It remains to show that the right side of (3.64) is zero when λ 6= λ′. For this we develop an expression

for B(x)Pλ(x)Q′λ′ (x). First, it follows from (3.54) that, if λ is replaced by λ

′, then

ρs(x) =Pλ′ (x)

Qλ′ (x)and ρ′s(x) =

P ′λ′ (x)

Qλ′ (x)− Pλ′ (x)

Q2λ′ (x)

Q′λ′ (x). (3.65)

37

Recall from (3.50) that

0 = −A(x)ρs(x) +1

2

∂

∂x(B(x)ρs(x)) ∀x ∈ [a, b]. (3.66)

Substitute (3.65) into (3.66) and obtain

0 =

(

−A(x) +1

2B′(x)

)

Pλ′ (x)

Qλ′ (x)+B(x)

2

(

P ′λ′ (x)

Qλ′ (x)− Pλ′ (x)

Q2λ′ (x)

Q′λ′ (x)

)

. (3.67)

Multiplying (3.67) by Q2λ′ leads, after an algerbraic mainipulation, to

1

2B(x)Pλ′ (x)Q′

λ′ (x) =

(

−A(x) +1

2B′(x)

)

Pλ′ (x)Qλ′ (x) +B(x)

2P ′

λ′ (x)Qλ′ (x) (3.68)

Rearranging terms gives

1

2B(x)Pλ′ (x)Q′

λ′ (x) =

(

−A(x)Pλ′ (x) +1

2(B(x)Pλ′ (x))

′)

Qλ′ (x) (3.69)

Finally, multiply (3.69) by Pλ(x) and divide by Pλ′ (x), and get

1

2B(x)Pλ(x)Q′

λ′ (x) =Pλ(x)

Pλ′(x)

(

−A(x)Pλ′ (x) +1

2(B(x)Pλ′ (x))′

)

Qλ′ (x) (3.70)

Recasll from (3.57) that Pλ′ (x) satisfies reflective boundary condtions, hence

[

1

2B(x)Pλ(x)Q′

λ′ (x)

]b

a

=

[

Pλ(x)

Pλ′(x)

(

−A(x)Pλ′ (x) +1

2(B(x)Pλ′ (x))

′)

Qλ′ (x)

]b

a

= 0. (3.71)

This completes the proof that the right side of (3.64) is zero, and therefore the proof of of the

orthogonality condition (3.61) is complete.

Step V. We now develop the eigenfunction exapsnion for a solution of the Fokker-Planck equa-

tion (3.47) with reflective boundary conditions. The expansion is

ρ(x, t) =∑

λ

AλPλ(x)e−λt. (3.72)

To find a specific Aλ′ , multiply (3.72) by Qλ′ (x), integate from x = a to x = b, and get

∫ b

a

ρ(x, t)Qλ′ (x)dx =∑

λ

Aλ

∫ b

a

Pλ(x)Qλ′ (x)dxe−λt. (3.73)

Because of the orthogonality conditions (3.61)-(3.62), it immediately follows from (3.74) that

Aλ′ =

∫ b

a

ρ(x, 0)Qλ′ (x)dx. (3.74)

(g.) How to use the eigenfunction expansion to find ρ(x, t|x0, 0).

38

As above assume that ρ(x, t|x0, 0) satisfies reflective boundary conditions. Recall that ρ(x, 0|x0, 0) =

δ(x− x0). Then (3.74) reduces to

Aλ′ =

∫ b

a

ρ(x, 0|x0, 0)Qλ′ (x)dx =

∫ b

a

δ(x− x0)Qλ′ (x)dx = Qλ′ (x0). (3.75)

In turn, this implies that ρ(x, t|x0, 0) is given by

ρ(x, t|x0, 0) =∑

λ

Qλ(x0)Pλ(x)e−λt. (3.76)

(h.) How to find the autocorrelation function from the eigenfunction expansion.

As above assume that ρ(x, t|x0, 0) satisfies reflective boundary conditions, so that

ρ(x, t|x0, 0) =∑

λ

Qλ(x0)Pλ(x)e−λt. (3.77)

The autocorrelation function is

RX(τ) =

∫ b

a

∫ b

a

xx0ρ(x, τ |x0, 0)ρs(x0)dxdx0e−λτ =

∑

λ

∫ b

a

∫ b

a

xx0Qλ(x0)Pλ(x)ρs(x0)dxdx0e−λτ .

(3.78)

Recall that

Pλ(x) = ρs(x)Qλ(x). (3.79)


RX(τ) =∑

λ

(

∫ b

a

xQλ(x)ρs(x)dx

)(

∫ b

a

x0Qλ(x0)ρs(x0)dx0

)

e−λτ . (3.80)

From this, and the fact that Q0(x) = 1, we conclude that

RX(τ) =

(

∫ b

a

x(x)ρs(x)dx

)2

+∑

λ6=0

(

∫ b

a

xQλ(x)ρs(x)dx

)2

e−λτ . (3.81)

It follows from (3.81) that

limτ→∞

RX(τ) =

(

∫ b

a

xρs(x)dx

)2

. (3.82)

Remark 1. If a = −b < 0, and ρs(x) is an even function, then

RX(τ) =∑

λ6=0

(

∫ b

a

xQλ(x)ρs(x)dx

)2

e−λτ , (3.83)

and therefore

limτ→∞

RX(τ) = limτ→∞

∑

λ6=0

(

∫ b

a

xQλ(x)ρs(x)dx

)2

e−λτ = 0 (3.84)

39

Remark 2. This property will hold for the Ornstein-Uhlenbeck process where a = −∞, b = ∞, and

ρs(x) = exp(−kx2/2). Here, as Gardner shows ([11], pp.),

ρ(x, t|x0, 0) =∑

λ

Hλ(x)Pλ(x)e−λt, (3.85)

where λ = n ≥ 0, and Hλ(x) is the nth order Hermite polynomial. Thus, since H0(x) = 1, then

P0(x) = ρs(x) and we conclude that

limt→∞

ρ(x, t|x0, 0) = H0(x)P0(x) = ρs(x). (3.86)

Recall that Pλ(x) = Qλ(x)ρs(x). This and the orthogonality condition imply that

∑

λ

∫ ∞

−∞xQλ(x)ρs(x)dx =

∑

λ

∫ ∞

−∞H1(x)Pλ(x)dx =

∫ ∞

−∞H1(x)P1(x)dx =

∫ ∞

−∞x2ρs(x)dx. (3.87)

This implies that RX(τ) has the simplified form

RX(τ) =

(∫ ∞

−∞x2ρs(x)dx

)2

e−κτ , (3.88)

where κ > 0 is a constant (see the next section).

(i.) The autocorrelation function for the Ornstein-Uhlenbeck process.

The Ornstein-Uhlenbeck procsess is modelled by the SDE

dX(t)

dt= −γX + σζ(t), X(0) = x, and t ≥ 0. (3.89)

The Fokker-Planck problem for this process consists of the pdf

∂

∂tρ(x′, t|x, 0) = γ

∂

∂x′(x′ρ(x′, t|x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, t|x, 0), τ > 0. (3.90)

with initial condition

ρ(x′, 0|x, 0) = δ(x′ − x). (3.91)

Stationary solution. Setting f(x′) = −γx in (3.42) gives the ODE

0 =d

dx′(γx′ρS(x′)) +

σ2

2

d2

dx′2ρS(x′) (3.92)

An integration reduces (3.92) to the first order equation

c1 = γx′ρS(x′) +σ2

2

d

dx′ρS(x′). (3.93)

We assume that ρS(x′) satisfies the conditions

lim|x′|→∞

x′ρS(x′) = lim|x′|→∞

d

dx′ρS(x′) = 0. (3.94)

40

Substitution of (3.94) into (3.93) shows that c1 = 0, and therefore (3.93) simplifies to

0 = γx′ρS(x′) +σ2

2

d

dx′ρS(x′). (3.95)

Solving (3.95) subject to conditions (3.43) and (3.94), we obtain the stationary solution

ρS(x′) =1

σ

√

γ

πexp

(

− γ

σ2x′2)

. (3.96)

We use (3.96) to study the behavior of E (X(t+ τ)X(t)) when t >> 1. We assume that

limt→∞

ρ(x, t|0, 0) = ρS(x). (3.97)


limt→∞

E (X(t+ τ)X(t)) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.98)

where the function ρ(x′, τ |x, 0) solves the Fokker-Planck equation

∂

∂tρ(x′, τ |x, 0) =

∂

∂x′(γx′ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0), τ > 0, (3.99)

and

ρ(x′, 0|x, 0) = δ(x′ − x). (3.100)

Finally, since

RX(τ) = limt→∞

E (X(t+ τ)X(t)) , (3.101)

we conclude that

RX(τ) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.102)

Properties of RX(τ).

Our goal here is to show how to use (3.102) to determine the behavior of RX(τ) at τ = 0 and τ = ∞,

and also to obtain a closed form formula for RX(τ).

First, It follows from (3.102) that

RX(0) =

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, 0|x, 0)ρS(x)dx′dx. (3.103)

Substituting ρ(x′, 0|x, 0) = δ(x− x′) into (3.103), we obtain

RX(0) =

∫ ∞

−∞x2ρS(x)dx. (3.104)

Combining (3.96) with (3.104), we obtain the well known ’fluctuation-dissipation’ equation

RX(0) =1

σ

√

γ

π

∫ ∞

−∞x2exp

(

− γ

σ2x2)

dx =σ2

2γ. (3.105)

41

Next, to determine the behavior of RX(τ) as τ → ∞, we need to assume that

limτ→∞

ρ(x′, τ |x, 0) = ρS(x′). (3.106)


limτ→∞

RX(τ) =

∫ ∞

−∞

∫ ∞

−∞x′xρS(x)ρS(x′)dx′dx =

∫ ∞

−∞x′ρS(x′)dx′

∫ ∞

−∞xρS(x)dx = 0, (3.107)

since ρS is an even fucntion.

Derivation of the ODE satisfied by RX(τ).

To obtain a closed form formula for RX(τ) we derive and solve an associated ODE initial value

problem. First, a differentiation of both sides of (3.102) with respect to τ gives

d

dτRX(τ) =

∫ ∞

−∞

∫ ∞

−∞x′x

∂

∂τρ(x′, τ |x, 0)ρS(x)dx′dx (3.108)

Combining (3.99) with (3.108), we obtain

d

dτRX(τ) =

∫ ∞

−∞

∫ ∞

−∞x′x

(

∂

∂x′(γx′ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0)

)

ρS(x)dx′dx (3.109)

Integration by parts reduces (3.109) to

d

dτRX(τ) = −γ

∫ ∞

−∞

∫ ∞

−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx. (3.110)

This, together with (3.102), further reduces (3.110) to

d

dτRX(τ) = −γRX(τ). (3.111)

Solving (3.111), with the initial condition RX(0) = σ2

2γ , gives the well known autocorellation function

RX(τ) =σ2

2γe−γτ . (3.112)

42

(j.) Example 2: the LIF model.

The Leaky Integrate and Fire (LIF) model consists of the equation

dX

dt= (µ−X) + σ

dζ

dt, X(0) = x ∈ (−∞, vT ), (3.113)


X(t−) = vT and X(t+) = vR. (3.114)

Typical values for vT (threshold) and vR (reset) are

vT = 1 and vR = 0. (3.115)

Observations.

(i) If µ < vT and σ = 0 then there can be no spike since X(t) just increases to µ as t→ ∞.

(ii) If µ > vT and σ = 0 then we have a deterministic train of equally spaced spikes.

The Fokker-Planck problem associated with problem (3.113)-(3.114) consists of the pde

∂

∂tρ(x′, t|x, 0) =

∂

∂x′((x′ − µ)ρ(x′, t|x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, t|x, 0) + ν(t)δ(x′ − vR), (3.116)


ρ(x′, 0|x, 0) = δ(x′ − x). (3.117)

In addition, since the range of values of X(t) is −∞ < X(t) <∞, the normalizing condition is

∫ vT

−∞ρ(x′, t|x, 0)dx′ = 1. (3.118)

Stationary solution. A stationary solution of (3.116) is a solution ρS(x′) of

0 =d

dx′((x′ − µ)ρS(x′)) +

σ2

2

d2

dx′2ρS(x′) + νsH

′(x′ − vR), (3.119)

where we have used the fact that the Heaviside function H(x′ − vR) satisfies H ′(x′ − vR) = δ(x′ − vR)

.

Aside. Note that (3.42) can be written as

0 = − d

dxJ(x′), (3.120)

where J(x′) is the time independent flux

J(x′) = (µ− x′)ρS(x) − σ2

2

d

dx′ρS(x′) − νsH(x′ − vR). (3.121)

The stationary solution must satisfy

limx′→−∞

ρS(x′) = limx′→−∞

(x′ − µ)∂

∂x′ρS(x′) = 0 and

∫ vT

−∞ρS(x′)dx′ = 1. (3.122)

43

Our approach is to find functions ρ1(x′) and ρ2(x

′) such that

ρS(x′) =

ρ1(x′), −∞ < x′ < vR,

ρ2(x′), vR ≤ x′ < vT .

(3.123)

First, we let x′ < vR where ρS(x′) = ρ1(x′) and (3.119) reduces to

0 =d

dx′((x′ − µ)ρ1(x

′)) +σ2

2

d2

dx′2ρ1(x

′) + νsH′(x′ − vR), x′ < vR. (3.124)

An integration of (3.124) from −∞ to x′, together the first two conditions in (3.122), gives

(x′ − µ)ρ1(x′) +

σ2

2

d

dx′ρ1(x

′) = 0, x′ < vR. (3.125)

The solution of this equation is

ρ1(x′) = k1exp

(

− (x′ − µ)2

σ2

)

, x′ < vR, (3.126)

where k1 is a constant that remains to be determined.

Next, we let vR < x′ < vT where ρS(x′) = ρ2(x′) and (3.119) reduces to

0 =d

dx′((x′ − µ)ρ2(x

′)) +σ2

2

d2

dx′2ρ2(x

′) + νsH′(x′ − vR), vR < x′ < vT . (3.127)

An integration of (3.127) gives

(x′ − µ)ρ2(x′) +

σ2

2

d

dx′ρ2(x

′) = −νS + k2, vR < x′ < vT . (3.128)

An integration of (3.128) from x′ to vT , subject to the condition ρ2(vT ) = 0, gives

ρ2(x′) =

2

σ2(νS − k2)exp

(−(x′ − µ)2

σ2

)∫ vT

x′exp

(

(η − µ)2

σ2

)

dη, vR < x′ < vT . (3.129)

We need to determine the constants k1 and k2 in (3.126) and (3.129). For this we have make use of

two matching conditions at x′ = vR. The first is the continuity condtion

ρ1(v−R) = ρ2(v

+R). (3.130)

Combining (3.126), (3.129) and (3.130), we find that k1 and k2 satisfy

k1 =2

σ2(νS − k2)

∫ vT

vR

exp

(

(η − µ)2

σ2

)

dη. (3.131)

Next, we derive the jump condition at vR which is satisfied by ρ′1(vR) and ρ′2(vR). For this we let

ǫ > 0 and integrate (3.119) from vR − ǫ to vR + ǫ, and find that the stationary solution ρS satisfies

(vR + ǫ− µ)ρS(vR + ǫ) +σ2

2ρ′S(vR + ǫ) − (vR − ǫ− µ)ρS(vR − ǫ) − σ2

2ρ′S(vR − ǫ) = −νS. (3.132)

44

Letting ǫ→ 0+ reduces (3.133) to the jump condition

ρ′S(v+R) − ρ′S(v−R) = −2νS

σ2. (3.133)

Thus, in terms of ρ1 and ρ2, this becomes

ρ′2(v+R) − ρ′1(v

−R) = −2νS

σ2. (3.134)

From (3.125) we get

ρ′1(v−R) = − 2

σ2(vR − µ)ρ1(v

−R ). (3.135)

Likewise, from (3.128) we conclude that

ρ′2(v+R) =

2

σ2(−νS + k2) −

2

σ2(vR − µ)ρ2(v

+R). (3.136)

We substitute (3.136) into (3.135), and make use of (3.130), to conclude that

2

σ2(−νS + k2) = −2νS

σ2. (3.137)

Thus, it must be the case that k2 = 0. This and (3.131) imply that

k1 =2νS

σ2

∫ vT

vR

exp

(

(η − µ)2

σ2

)

dη and k2 = 0. (3.138)

Substituting these values into (3.126) and (3.129) gives

ρ1(x′) =

2νS

σ2exp

(

− (x′ − µ)2

σ2

)∫ vT

vR

exp

(

(η − µ)2

σ2

)

dη, −∞ < x′ < vR, (3.139)

and

ρ2(x′) =

2νS

σ2exp

(−(x′ − µ)2

σ2

)∫ vT

x′exp

(

(η − µ)2

σ2

)

dη, vR < x′ < vT . (3.140)

Substituting (3.139) and (3.140) into (3.123), we obtain the stationary solution

ρS(x′) =

2νS

Nσ2 exp(

− (x′−µ)2

σ2

)

∫ vT

vRexp

(

(η−µ)2

σ2

)

dη, −∞ < x′ < vR,

2νS

Nσ2 exp(

−(x′−µ)2

σ2

)

∫ vT

x′ exp(

(η−µ)2

σ2

)

dη, vR ≤ x′ < vT ,(3.141)

where N is the ’normalizing’ constant

N =∫ vR

−∞2νS

σ2 exp(

− (x′−µ)2

σ2

)

∫ vT

vRexp

(

(η−µ)2

σ2

)

dηdx′

+∫ vT

vR

2νS

σ2 exp(

−(x′−µ)2

σ2

)

∫ vT

x′ exp(

(η−µ)2

σ2

)

dηdx′.(3.142)

Remarks. In our derivation of the ODE satisfied by RX(τ) we may need the following insights about

the behavior of ρ1(x′) and ρ2(x

′). Setting k2 = 0 in (3.128), we find that

ρ′2(vR) =2

σ2(−νS + (µ− vR)ρ2(vR)) , ρ2(vT ) = 0 and ρ′2(vT ) = −2νS

σ2. (3.143)

45

Properties of RX(τ).

Following the method of the previous subsection, we find (correct?) that

RX(τ) =

∫ vT

−∞

∫ vT

−∞x′xρ(x′, τ |x, 0)ρS(x)dx′dx, (3.144)

where ρS(x) is the stationary solution defined above in (3.141), and ρ(x′, τ |x, 0) solves the Fokker-

Planck equation

∂

∂τρ(x′, τ |x, 0) =

∂

∂x′((x′ − µ)ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0) + νSδ(x

′ − x), τ > 0, (3.145)

subject to the initial condition

ρ(x′, 0|x, 0) = δ(x′ − x). (3.146)

Integrating (3.145) gives

∫ vT

−∞

∂

∂τρ(x′, τ |x, 0)dx′ =

σ2

2ρx′(vT , τ |x, 0) + νS . (3.147)

Note: the ’flux’ J(x′, τ) satisfies

.∂

∂τρ(x′, τ |x, 0) = − ∂

∂x′J(x′, τ) (3.148)

and is given by

J(x′, τ) = −(

(x′ − µ)ρ(x′, τ |x, 0) +σ2

2

∂

∂x′ρ(x′, τ |x, 0) + νSH(x′ − x)

)

(3.149)

Derivation of the ODE satisfied by RX(τ).

To obtain a closed form formula for RX(τ) we derive and solve an associated second order ODE.

First, a differentiation of both sides of (3.144) with respect to τ gives

d

dτRX(τ) =

∫ vT

−∞

∫ vT

−∞x′x

∂

∂τρ(x′, τ |x, 0)ρS(x)dx′dx. (3.150)

Substituting (3.145) into (3.150) gives

R′X(τ) =

∫ vT

−∞

∫ vT

−∞x′(

∂

∂x′((x′ − µ)ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0) + νSδ(x

′ − x)

)

xρS(x)dx′dx.

(3.151)

Integration the first term by parts, and evaluating the third term, shows that

R′X(τ) = −

∫ vT

−∞∫ vT

−∞ x′ρ(x′, τ |x, 0)xρS(x)dx′dx+ µ∫ vT

−∞

(

∫ vT

−∞ ρ(x′, τ |x, 0)dx′)

xρS(x)dx

+ σ2

2

∫ vT

−∞∫ vT

−∞ x′ ∂2

∂x′2 ρ(x′, τ |x, 0)xρS(x)dx′dx+ νS

∫ vT

−∞ x2ρS(x)dx.

(3.152)

Using the definition of RX(τ) given in (3.144), and the fact that

∫ vT

−∞ρ(x′, τ |x, 0)dx′ = 1, (3.153)

46

we find that (3.152) reduces to

R′X(τ) = −RX(τ) + µ

∫ vT

−∞ xρS(x)dx + νS

∫ vT

−∞ x2ρS(x)dx

+ σ2

2

∫ vT

−∞∫ vT

−∞ x′ ∂2

∂x′2 ρ(x′, τ |x, 0)xρS(x)dx′dx.

(3.154)

We need to simplify the double integral in (3.154). An integration by parts gives

∫ vT

−∞x′

∂2

∂x′2ρ(x′, τ |x, 0)dx′ = vT

∂

∂x′ρ(vT , τ |x, 0) −

∫ vT

−∞

∂

∂x′ρ(x′, τ |x, 0)dx′. (3.155)

Since∫ vT

−∞

∂

∂x′ρ(x′, τ |x, 0)dx′ = ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0, (3.156)

then (3.155) becomes∫ vT

−∞x′

∂2

∂x′2ρ(x′, τ |x, 0)dx′ = vT

∂

∂x′ρ(vT , τ |x, 0). (3.157)

Substitution of (3.157) into (3.154) leads to

R′X(τ) = −RX(τ) + µ

∫ vT

−∞ xρS(x)dx + νS

∫ vT

−∞ x2ρS(x)dx

+ vT σ2

2

∫ vT

−∞ ρx′(vT , τ |x, 0)xρS(x)dx.

(3.158)

To proceed further we need to derive a second helpful identity. For this we integrate (3.145) from −∞to vT and obtain

∫ vT

−∞

∂

∂τρ(x′, τ |x, 0)dx′ =

∫ vT

−∞

(

∂

∂x′((x′ − µ)ρ(x′, τ |x, 0)) +

σ2

2

∂2

∂x′2ρ(x′, τ |x, 0) + νSδ(x

′ − x)

)

dx′.

(3.159)

Interchanging the integral and derivative on the left (hopefully this is legal), we get

∂

∂τ

∫ vT

−∞ρ(x′, τ |x, 0)dx′ = (vT − µ)ρ(vT , τ |x, 0) +

σ2

2ρx′(vT , τ |x, 0) + νS . (3.160)

Using the facts that∫ vT

−∞ ρ(x′, τ |x, 0)dx′ = 1 and ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0, this reduces to

0 =σ2

2ρx′(vT , τ |x, 0) + νS ∀τ ≥ 0, (3.161)

hence

ρx′(vT , τ |x, 0) = −2νS

σ2∀τ ≥ 0. (3.162)

Hopefully, this allows us to conclude that

ρx′τ (vT , τ |x, 0) = 0 ∀τ ≥ 0. (3.163)

Next, a differentiation of (3.154) gives the second order equation

R′′X(τ) = −R′

X(τ) +σ2

2

∫ vT

−∞

∫ vT

−∞x′

∂2

∂x′2ρτ (x′, τ |x, 0)xρS(x)dx′dx. (3.164)

47

An integration by parts reduces this equation to

R′′X(τ) = −R′

X(τ) +σ2

2

∫ vT

−∞vTρx′τ (vT , τ |x, 0)xρS(x)dx − σ2

2

∫ vT

−∞

∫ vT

−∞ρx′τ (x′, τ |x, 0)xρS(x)dx′dx.

(3.165)

Another integration by parts gives us (hopefully correct)

∫ vT

−∞x′ρx′τ (x′, τ |x, 0)dx′ = vT ρτ (vT , τ |x, 0) −

∫ vT

−∞ρτ (x′, τ |x, 0)dx′ (3.166)

We should be able to conclude that ρτ (vT , τ |x, 0) = 0 ∀τ ≥ 0 since ρ(vT , τ |x, 0) = 0 ∀τ ≥ 0. Using

this fact, and substituting (3.166) into (3.165) gives

R′′X(τ) = −R′

X(τ) +σ2

2

∫ vT

−∞vT ρx′τ (vT , τ |x, 0)xρS(x)dx

σ2

2

∫ vT

−∞

∫ vT

−∞ρτ (x′, τ |x, 0)xρS(x)dx′dx.

(3.167)

48

(k.) The linear integrate and fire model.

This follows Mattia and Giudic [20]. and Doiron, Rinzel and Reyes [6]. The linear integrate and fire

model consists of the SDEdV

dt= µ(t) + σ(t)

dζ

dt, (3.168)


V (0) =

v ∈ (vL, vT ) if vL > −∞,

v ∈ (−∞, vT ) if vL = −∞.(3.169)

A ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immediately reset

to V = vR, where vR ∈ [vL, vT ). Thus, we impose the reset condition

V (t−) = vT and V (t+) = vR. (3.170)

It is important to note that either

vL < vR < vT , (3.171)

or

vL = vR < vT . (3.172)

We first assumme that (3.171) holds in order to properly derive the boundary conditons satisfied by

solutions of the Fokker-Planck equation associated with (3.168). This will allow us to use a limiting

argument to obtain the correct boundary conditions when (3.172) holds. Later we follow [6, 20] and

assume that

vR = vL = 0 and vT > 0. (3.173)

Remarks.

(i) We follow [6, 20] and assume for simplicity that σ(t) = σ > 0 is a constant. However, recent

experiments show that interesting results can occur when σ(t) is a step function [7].

(ii) Suppose that µ(t) = µ is a constant. If µ < 0, σ > 0 and vL = −∞ then

E(V (t)) = v + µt→ −∞ as t→ ∞. (3.174)

Thus, the probability of a spike occuring decreases as t increses. If µ > 0, σ > 0 and vL ≥ −∞, then

spikes occur. If µ > 0, vL ≥ −∞ and σ = 0 then a deterministic train of equally spaced spikes occurs.

(iii) An interesting phenomenon occurs when µ(t) is a step function as in [6, 20]. This causes oscil-

lations in the ’emission rate” function ν(t), which appears below in the Fokker-Planck equation. Our

main goal is to understand the analytical techniques developed in [6, 20] which show how oscillations

in ν(t) arise when µ(t) is a step function.

The Fokker-Planck equation associated with equation (3.168) consists of the pde

∂

∂tρ(V, t) = −µ(t)

∂

∂Vρ(V, t) +

σ2(t)

2

∂2

∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.175)

49

with the ’absorbing’ boundary condition

ρ(vT , t) = 0 ∀t ≥ 0, (3.176)

and normalizing condition∫ vT

vL

ρ(V, t)dV = 1 ∀t ≥ 0. (3.177)

The ’probability current’ is the function

Sρ(V, t) = µ(t)ρ(V, t) − σ2(t)

2

∂

∂Vρ(V, t). (3.178)

When vL < vR < vT solutions satisfy the reflctive boundary condition Sρ(vL, t) = 0, i. e.

µ(t)ρ(vL, t) −σ2(t)

2

∂

∂Vρ(vL, t) = 0 when vL < vR < vT . (3.179)

We assume that the functions µ(t) and σ(t) are piecewise continuous in t and V so that

ρ(V, t) is continuous in t and V, (3.180)

and∂

∂tρ(V, t) and

∂

∂Vρ(V, t) are piecewise continuous in t and V. (3.181)

Remark (iv) The specific solution ρ(V, t|v, 0) of (3.175) which is associated with the SDE initial

value problem (3.168)-(3.170) satisfies the additional condition

ρ(V, 0|vR, 0) = δ(V − vR). (3.182)

The emission rate function. We noted above in Remark (iii) that the function ν(t) in (3.175) is

called the ’emission rate’ function. Our goal is to analyze ν(t) in two distinct parameter regimes. The

first regime is when vL < vR < vT . We will derive the identities

ν(t) = −σ2(t)

2

∂

∂Vρ(vT , t) ∀t ≥ 0 when vL < vR < vT . (3.183)

and∂

∂Vρ(vT , t) =

∂

∂Vρ(v+

R , t) −∂

∂Vρ(v−R , t) ∀t ≥ 0 when vL < vR < vT . (3.184)

The second regime is vL = vR < vT . Here we let vL → v−R and show how (3.184) reduces to

∂

∂Vρ(vT , t) =

∂

∂Vρ(v+

R , t) −∂

∂Vρ(v−R , t) ∀t ≥ 0 when vL = vR < vT . (3.185)

Case I: vL < vR < vT . The first step is to integrate (3.175) from v = vL to v = vT and obtain

∂

∂t

∫ vT

vL

ρ(V, t)dV = −µ(t)

∫ vT

vL

∂

∂Vρ(V, t)dV +

∫ vT

vL

σ2(t)

2

∂2

∂V 2ρ(V, t)dv + ν(t)

∫ vT

vL

δ(V − vR)dV.

(3.186)

50

This reduces to

∂

∂t

∫ vT

vL

ρ(V, t)dV = µ(t)ρ(vL, t)− µ(t)ρ(vT , t) +σ2(t)

2

∂

∂Vρ(vT , t)−

σ2(t)

2

∂

∂Vρ(vL, t) + ν(t). (3.187)

Combining (3.187) with conditions (3.176), (3.177) and (3.179) reduces (3.187) to (3.183).

Next, we derive a second identity satisfied by ν(t). For this we integrate (3.175) from v = vR − ǫ to

v = vR + ǫ, and obtain

∂

∂t

∫ vR+ǫ

vR−ǫ

ρ(V, t)dV = −µ(t)

∫ vR+ǫ

vR−ǫ

∂

∂Vρ(V, t)dV+

∫ vR+ǫ

vR−ǫ

σ2(t)

2

∂2

∂V 2ρ(V, t)dv+ν(t)

∫ vR+ǫ

vR−ǫ

δ(V−vR)dV.

(3.188)

This reduces to

∂

∂t

∫ vR+ǫ

vR−ǫ

ρ(V, t)dV = −µ(t) (ρ(vR + ǫ, t) − ρ(vR − ǫ, t))+σ2(t)

2

(

∂

∂Vρ(vR + ǫ, t) − ∂

∂Vρ(vR − ǫ, t)

)

+ν(t).

(3.189)

Letting ǫ→ 0+ in (3.189), and using the continuity conditions (3.180) and (3.181), we conclude that

ν(t) = −σ2(t)

2

(

∂

∂Vρ(v+

R , t) −∂

∂Vρ(v−R , t)

)

∀t ≥ 0 when vL < vR < vT . (3.190)

Combining (3.183) and (3.190) gives the identity

∂

∂Vρ(vT , t) =

∂

∂Vρ(v+

R , t) −∂

∂Vρ(v−R , t) ∀t ≥ 0 when vL < vR < vT . (3.191)

Case II: vL = vR < vT . Recall from (3.179) that

µ(t)ρ(vL, t) −σ2(t)

2

∂

∂Vρ(vL, t) = 0 ∀t ≥ 0 when vL < vR < vT . (3.192)

Letting vL → v−R reduces this to

µ(t)ρ(v−R , t) −σ2(t)

2ρV (v−R , t) = 0 ∀t ≥ 0 when vL = vR < vT . (3.193)

The continuity property (3.180) implies that ρ(v−R , t) = ρ(vR, t) ∀t ≥ 0. This, together with (3.193),

imply that∂

∂Vρ(v−R , t) =

2µ(t)

σ2(t)ρ(vR, t) ∀t ≥ 0 when vL = vR < vT . (3.194)

Finally, we substitute (3.194) into (3.191) and obtain the identity

∂

∂Vρ(vT , t) =

∂

∂Vρ(v+

R , t) −2µ(t)

σ2(t)ρ(vR, t) ∀t ≥ 0 when vL = vR < vT . (3.195)

Remark (v). We can remove the term ν(t)δ(V −vR) from (3.175) when vL = vR, and instead impose

the additional condition (3.195). We do this below where we restrict our attention to the parameter

regime vL = vR = 0 and θ = vT > 0.

51

Eigenfunctions: Basic Theory. Following [20], we assume that

vL = vR = 0 and vT = θ > 0. (3.196)

Our goals are to answer the following questions:

(I.) What is the ODE problem satisfied by eigenfunctions φ(V, t) of the Fokker-Planck equation?

(II.) What is the related ’adjoint’ ODE problem?

(III.) How to find eigenvalues λ and eigenfunctions φλ(V, t) and ψλ(V, t)?

(IV.) How to compute the eigenvalues λ and eigenfunctions φλ(V, t)?

(V.) How to find ψλ(V, t) and ∂∂µψλ(V, t) for the adjoint problem?

(VI.) Stationary solution: how to find φ0(V, t) and ψ0(V, t) when λ = 0?

(I.) We look for a solution of the Fokker-Planck equation (3.175) which has the form

ρ(v, t) = φ(V, t)eλt. (3.197)

The eigenfunction φ(V, t) satisfies

λφ(V, t) = −µ(t)∂

∂Vφ(V, t) +

σ2(t)

2

∂2

∂V 2φ(V, t), (3.198)

with absorbing condition

φ(θ, t) = 0. (3.199)

Because of (3.196), we need to impose boundary condition (3.195), namely

∂

∂Vφ(θ, t) =

∂

∂Vφ(0, t) − 2µ(t)

σ2(t)φ(0, t) ∀t ≥ 0. (3.200)

We will see below that the eigenvalues λ are complex when µ(t) is a positive constant, and they are

real and negative when µ(t) ≤ 0 is zero or a negative constant

(II.) To understand how the adjoint ODE problem arises, we define two linear operators. The first is

the operaor L defined by

Lφ(V, t) = −µ(t)∂

∂Vφ(V, t) +

σ2(t)

2

∂2

∂V 2φ(V, t). (3.201)

Thus, an eigenfunction φ(V, t), and its eigenvalue λ, satisfy

Lφ(V, t) = λφ(V, t). (3.202)

Define the inner product

(ψ(V, t), φ(V, t)) =

∫ θ

0

ψ(V, t), φ(V, t)dV. (3.203)

52

The second linear operator, the ’adjoint operator’ L+, is defined by

(

L+ψ(V, t), φ(V, t))

= (ψ(V, t), Lφ(V, t)) , (3.204)

and satisfies

L+ψ(V, t) = µ(t)∂

∂Vψ(V, t) +

σ2(t)

2

∂2

∂V 2ψ(V, t). (3.205)

For each eigenfunction φ, and an associated eigenvalue λ, of L, there is a corresponding eigenfunction

ψ(V, t) of L+ which satisfies

λψ(V, t) = µ(t)∂

∂Vψ(V, t) +

σ2(t)

2

∂2

∂V 2ψ(V, t). (3.206)

It follows from (3.204) applied to φ and ψ, and conditions (3.199) and (3.200), that

ψ(0) = ψ(θ) and∂

∂Vψ(0, t) = 0. (3.207)

It also follows from these properties that if ψ(V, t) and φ(V, t) are eigenfunctions with eigenalues λ′

and λ, respectively, then ψ(V, t) and φ(V, t) satisfy the orthogonality condition

(ψ(V, t), φ(V, t)) = 0 if λ 6= λ′, (3.208)

and the normalizing property

(ψ(V, t), φ(V, t)) = 1 if λ = λ′. (3.209)

(III.) How to find the eigenvalues λ and eigenfunctions φλ(V, t) and ψλ(V, t)

The first step is to substitute

φλ(V, t) = exp

(

mV

θ

)

(3.210)

into

λφλ(V, t) = −µ(t)∂

∂Vφλ(V, t) +

σ2(t)

2

∂2

∂V 2φλ(V, t), (3.211)

and obtain the algebra problem

m2 − 2µθ

σ2m− 2λθ2

σ2= 0. (3.212)

Note that the variable t appears in m through µ(t) or σ(t). Solving (3.212) gives

m = ζ ± γ, (3.213)

where

ζ =µθ

σ2and γ =

θ

σ2

√

µ2 + 2λσ2 (3.214)

Thus, the general solution of (3.198) is

φλ(V, t) = exp

(

ζV

θ

)(

c1 exp

(

γV

θ

)

+ c2 exp

(

−γVθ

))

. (3.215)

53

It follows from (3.215), the boundary condition φλ(θ, t) = 0, and an algebraic manipulation, that

φλ(V, t) = cλ exp

(

ζV

θ

)

sinh

(

γ(θ − V )

θ

)

, (3.216)

where cλ is a constant which is possibly complex, and is determined below in (3.245)-(3.246).

The next step is to substitute (3.243) into the second boundary condition

∂

∂Vφλ(θ, t) =

∂

∂Vφλ(0, t) − 2µ(t)

σ2(t)φλ(0, t) ∀t ≥ 0. (3.217)

This gives the ’characteristic equation’ from which the eigenvalues are computed, namely

γ exp(ζ) = γ cosh(γ) + ζ sinh(γ), (3.218)

where we recall that

where ζ =µθ

σ2and γ =

θ

σ2

√

µ2 + 2λσ2.

Finally, it follows from an integration of (3.211) from V = 0 to V = θ, and the boundary conditions

φλ(θ, t) = 0,∂

∂Vφλ(θ, t) =

∂

∂Vφλ(0, t) − 2µ(t)

σ2(t)φλ(0, t),

that∫ θ

0

φλ(V, t)dV = 0 when λ 6= 0. (3.219)

Note. We will show below that, if λ = 0, then

∫ θ

0

φ0(V, t)dV = 1. (3.220)

54

(IV.) How to compute the eigenvalues λ and eigenfunctions φλ(V, t)

Recall that the eigenvalues λ satisfy

ζ =µθ

σ2and γ =

θ

σ2

√

µ2 + 2λσ2. (3.221)

and that γ satsifies

γ exp(ζ) = γ cosh(γ) + ζ sinh(γ), (3.222)

Substituting γ = γ1 + iγ2 into (3.221) and separating real and imaginary parts gives the system

γ1 exp(ζ) − γ1 cosh(γ1) cos(γ2) + γ2 sinh(γ1) sin(γ2) − ζ sinh(γ1) cos(γ2) = 0,

γ2 exp(ζ) − γ2 cosh(γ1) cos(γ2) − γ1 sinh(γ1) sin(γ2) − ζ cosh(γ1) sin(γ2) = 0.(3.223)

We asssume that λ is a function of µ, hence γ = γ1 + iγ2 is also a function of µ. Our goal is to derive

ODE’s satisfied by γ1(µ) and γ2(µ). We then use (3.221) to find the values of λ.

The first step is to find the initial values for γ1(µ) and γ2(µ). Mattia et al [20] show that

λN = −σ2

θ22N2π2, N ≥ 1 when µ = 0. (3.224)

Thus, from (3.221) and (3.224) it follows that

γ1(0) = 0 and γ2(0) = 2Nπ, N ≥ 1 when µ = 0. (3.225)

Next, differentiate (3.221) with respect to µ and obtain

dγ

dµ

(

eζ − (1 + ζ) cosh(γ) − γ sinh(γ))

=θ

σ2

(

sinh(γ) − γeζ)

. (3.226)

We will make use of the identities

sinh(γ) = sinh(γ1) cos(γ2) + i cosh(γ1) sin(γ2), (3.227)

cosh(γ) = cosh(γ1) cos(γ2) + i sinh(γ1) sin(γ2), (3.228)

andγ sinh(γ) = γ1 sinh(γ1) cos(γ2) − γ2 cosh(γ1) sin(γ2)

+ i (γ2 sinh(γ1) cos(γ2) + γ1 cosh(γ2) sin(γ2)) .(3.229)

We write

eζ − (1 + ζ) cosh(γ) − γ sinh(γ) = A + Bi, (3.230)

where

A = eζ − (1 + ζ) cosh(γ1) cos(γ2) − γ1 sinh(γ1) cos(γ2) + γ2 cosh(γ1) sin(γ2),

B = −(1 + ζ) sinh(γ1) sin(γ2) − γ2 sinh(γ1) cos(γ2) − γ1 cosh(γ1) sin(γ2).(3.231)

55

Next, we writeθ

σ2

(

sinh(γ1) − γeζ)

= C + Di, (3.232)

whereC = θ

σ2

(

sinh(γ1) cos(γ2) − eζγ1

)

,

D = θσ2

(

cosh(γ1) sin(γ2) − eζγ2

)

.(3.233)

Combining (3.230), (3.230), (3.231), (3.232) and (3.233), we obtain

γ′1(µ) = CA+DBA2+B2 , γ1(0) ≈ constant

√µ,

γ′2(µ) = DA−CBA2+B2 , γ2(0) ≈ 2Nπ.

(3.234)

After solving (3.234), we substitute γ1 and γ2 into (3.221) and get

λN =σ2

2θ2(

γ21 − γ2

2

)

− µ2

2σ2− i

σ2

2θ2γ1γ2. (3.235)

To find the γ and λ values numerically, and reproduce Figures 1 and 2 of Mattia et al, we use the

programs described below.

The programs lifeigenvalues.ode, twodnewton.m, f1.m, f2.m These can be found at

www.math.pitt.edu/∼troy/lif/

Goal: use lifeigenvalues.m, twodnewton.m, f1.m and f2.m to find the eigenvalues, and to reproduce

Figure 1 in Mattia et al. The results are shown below in Figure 1. In the xpp program lifeigenvalues.ode

we set

µ = start = .1001, σ = 1, θ = 1, N = 1, dt = .0005

and continue the solution forward to µ=5, and backwards to µ = 0. The the variable t represents

µ. The intial conditions are found by the two dimensional Newton iteration program twodnewton.m

which finds the gamma values to 12 decimal accuracy.

The matlab programs mattifig2eigenvalues.m, f1a., f2a.m These can be found at

www.math.pitt.edu/∼troy/lif/mattiafigure2/

Goal: use mattiafig2eigenvalues.m, f1a.m, f2a.m to find the eigenvalues as a first step in reproducing

the deterministic firing rate diagram in Figure 2 of Mattia et al (see Figures 2 and 4). The program

mattiafig2eigenvalues.m solves eqs (3.223) by two dimensional newton iteration when

µ = 25, σ = 1, θ = 1.

These are the values used by Mattia et al. The left sides of eqs. (3.223) are defined in f1a.m and f2a.m

Further details are given in a later section.

We now continue the analysis and show how to find the eigenfunction ψλ(V, t). Substitute

ψλ(V, t) = exp

(

mV

θ

)

(3.236)

56

-100

-50

0

50

100

0 1 2 3 4 5

Figure 1: Imaginary part of the eigenvalues vs. µ The programs for these computations are are

lifeigenvalues.m, twodnewton.m, f1.m, f2.m

into

λψλ(V, t) = µ(t)∂

∂Vψλ(V, t) +

σ2(t)

2

∂2

∂V 2ψλ(V, t), (3.237)

and obtain the algebra equation

m2 +2µθ

σ2m− 2λθ2

σ2= 0. (3.238)

Solving (3.238) gives

m = −ζ ± γ, (3.239)

where

ζ =µθ

σ2and γ =

θ

σ2

√

µ2 + 2λσ2. (3.240)

Thus, the general solution of (3.237) is

ψλ(V, t) = exp

(

−ζVθ

)(

c1 exp

(

γV

θ

)

+ c2 exp

(

−γVθ

))

. (3.241)

Substituting the boundary condtions

∂

∂Vψλ(0, t) = 0 and ψλ(0, t) = ψλ(θ, t) (3.242)

57

into (3.241), we find that the eigenfunction ψλ(V, t) is given by

ψλ(V, t) = exp

(

−ζVθ

)(

γ cosh

(

γV

θ

)

+ ζ sinh

(

γV )

θ

))

. (3.243)

It remains to find the value cλ such that the ’normalizing’ condition holds, i.e

∫ θ

0

cλφλ(V, t)ψλ(V, t)dV = 1. (3.244)

Substitution of (3.243) and (3.243) into (3.244) gives

cλ

∫ θ

0

sinh

(

γ(θ − V )

θ

)(

γ cosh

(

γV

θ

)

+ ζ sinh

(

γV )

θ

))

dV = 1. (3.245)

An integration shows that

cλ =2γ

θ [ζγ cosh(γ) + (γ2 − ζ2) sinh(γ)](3.246)

Finally, we have

∂

∂µψλ(V, t) = exp

(

−ζVθ

)[

γ cosh

(

γV

θ

)

+ ζ sinh

(

γV

θ

)]

. (3.247)

(VI.) Stationary solution: how to find φ0(V, t) and ψ0(V, t) when λ = 0?

To find a stationary solution we set λ = 0 in (3.198), and look for a solution φ0(V, t) of

0 = −µ(t)∂

∂Vφ0(V, t) +

σ2(t)

2

∂2

∂V 2φ0(V, t) ∀t ≥ 0, (3.248)

with boundary conditions

φ0(θ, t) = 0 (3.249)

and∂

∂Vφ0(θ, t) =

∂

∂Vφ0(0, t) −

2µ(t)

σ2(t)φ0(0, t) ∀t ≥ 0, (3.250)

and normalizing condition∫ θ

0

φ0(V, t)dV = 1 ∀t ≥ 0, (3.251)

Note that φ0(V, t) candepend on t since µ(t) in (3.250) may be a function of t.

The eigenfunction ψ0(V, t) is given by

ψ0(V, t) = 1 ∀ V ∈ [0, θ], (3.252)

hence∫ θ

0

φ0(V, t)ψ0(V, t)dV =

∫ θ

0

φ0(V, t)dV = 1 ∀ V ∈ [0, θ]. (3.253)

Thus, φ0 satisfies the normalizing condition

∫ θ

0

φ0(V, t)dV = 1 ∀t ≥ 0. (3.254)

58

Note. The exact formula for φ0 depends on whether µ = 0 or µ 6= 0.

Case 1: the Eigenfunctions and eigenvalues when µ(t) = 0 ∀t ≥ 0.

Our goals are to address the following issues:

(I.) When µ(t) = 0 ∀t ≥ 0 prove that the solutions φ0(V ) and ψ0(V ) are given by

φ0(V ) =2

θ2(θ − V ) and ψ0(V ) = 1, 0 ≤ V ≤ θ. (3.255)

(II.) When µ(t) = 0 ∀t ≥ 0 prove that

λN = −σ2

θ22N2π2, φN (V, t) = sin

(

2NπV

θ

)

, ψN = cos

(

2NπV

θ

)

, N ≥ 1. (3.256)

(III.) When µ(t) = 0 ∀t ≥ 0 prove that

∫ θ

0

φN (V, t)dV = 0 and

∫ θ

0

φN (V, t)ψN (V, t)dV = 0, N ≥ 1. (3.257)

(IV.) When µ(t) = 0 ∀t ≥ 0 is there a formula for ρ(V, t|0, 0)? In particular we will show that there

is a major problem with getting the property ρ(V, 0|0, 0) = δ(V ) to hole when µ = 0. The difficulty is

that ∈θ0 φN (V )ψN (V )dV = 0 when µ = 0.

(V.) When µ(t) = 0 ∀t ≥ 0 is there a formula for the firing rate ν(t)?

Proof of (I.) We set µ(t) = 0 in (3.248) and investigate the existence of a solution of

∂2

∂V 2φ0(V ) = 0, (3.258)


φ0(θ) = 0 and∂

∂Vφ0(θ) =

∂

∂Vφ0(0), (3.259)

and normalizing condition∫ θ

0

φ0(V )dV = 1. (3.260)

Note that φ0(V ) is independent of t since µ(t) = 0 ∀t ≥ 0.

The general solution of (3.259) is

φ0(V ) = c1V + c2, 0 ≤ V ≤ θ. (3.261)

Combining (3.261) with (3.259) and (3.260 shows that

φ0(V ) =2

θ2(θ − V ) , 0 ≤ V ≤ θ. (3.262)

As shown earlier the eigenfunction ψ0(V ) is given by

ψ0(V ) = 1, 0 ≤ V ≤ θ. (3.263)

59

Proof of (II.) To find the non-zero eigenvalues and eigenfunctions when µ(t) = 0, we solve

∂2

∂V 2φ(V ) − 2λ

σ2φ(V ) = 0, (3.264)


φ(θ) = 0 and∂

∂Vφ(θ) =

∂

∂Vφ(0) (3.265)

The eigenvalues and eigenfunctions are

λN = −σ2

θ22N2π2 and φN (V ) = sin

(

2NπV

θ

)

, N ≥ 1. (3.266)

Next, we solve the related adjoint boundary value problem

∂2

∂V 2ψ(V ) − 2λN

σ2ψ(V ) = 0, (3.267)

with boundary conditions∂

∂Vψ(θ) = 0 and ψ(0) = ψ(θ) (3.268)

The solution of (3.267)-(3.268) is

ψN (V ) = cos

(

2Nπ

θV

)

. (3.269)

Proof of (III.) When µ(t) = 0 ∀t ≥ 0 the eigenfunctions satisfy

∫ θ

0

φN (V )dV =

∫ θ

0

sin

(

2NπV

θ

)

dV = 0, N ≥ 1, (3.270)

and∫ θ

0

φN (V )ψN (V )dV =

∫ θ

0

sin

(

2NπV

θ

)

cos

(

2Nπ

θV

)

dV = 0, N ≥ 1. (3.271)

Proof of (IV.) Our goal is to derive a formula for ρ(V, t|0, 0) when µ(t) = 0 ∀t ≥ 0.

We follow the methods in Gardner [11] and assume that ρ(V, t|0, 0) exists, and that there are values

AN |N ≥ 1 such that

ρ(V, t|0, 0) = φ0(V ) +

∞∑

N=1

ANφN (V )e−λN t, (3.272)

where

φ0(V ) =2

θ2(θ − V ) , λN = −σ

2

θ22N2π2 and φN (V ) = sin

(

2NπV

θ

)

, N ≥ 1 (3.273)

Recall that ρ(V, 0|0, 0) = δ(V ). Thus, setting t = 0 in (3.272) gives

δ(V ) =2

θ2(θ − V ) +

∞∑

N=1

AN sin

(

2NπV

θ

)

. (3.274)

60

Note that δ(V ) is required to satisfy

δ(V ) = 0, 0 < V ≤ θ. (3.275)

From (3.274) it follows that

δ(0) =2

θand δ

(

θ

2

)

=1

θ. (3.276)

But this seems to contradict (3.275). If so, then do we conclude there the pdf ρ(V, t|0, 0) does not

satisfy ρ(V, 0|0, 0) = δ(V )? One possibility is that ρ(V, t|0, 0) satisfies

ρ(V, t|0, 0) ≡ φ0(V ) =2

θ2(θ − V ) , 0 ≤ V ≤ θ and t ≥ 0. (3.277)

(V.) To find the firing rate ν(t) when µ(t) = 0 ∀t ≥ 0, we begin with the definition

ν(t) = −σ2(t)

2

∂

∂Vρ(θ, t|0, 0). (3.278)

Suppose that (3.277) holds. Then combining (3.277) with (3.278), we obtain

ν(t) =σ2(t)

θ2, t ≥ 0. (3.279)

This all seems illegitimate.

Case 2: the eigenvalues and eigenfunctions when µ(t) 6= 0 for all t ≥ 0.

Our goal here is to prove the following:

(I.) When µ(t) 6= 0 prove that the solutions φ0(V ) and ψ0(V ) exist and are given by

φ0(V, t) =c

µ(t)

(

1 − exp(−2ζ(θ − V )

θ)

)

, (3.280)

where

c =

[

σ2(t)

2µ2(t)

(

2µ(t)θ

σ2(t)− 1 + exp

(

−2µ(t)θ

σ2(t)

))]−1

and ζ =µ(t)θ

σ2(t). (3.281)

(II.) When µ(t) 6= 0 the non-zero eigenvalues and their eigenfunctions are

(III.) When µ(t) 6= 0 the eigenfunctions satisfy

∫ θ

0

φN (V, t)ψN (V, t)dV = 1, N = 0,±1,±2.. (3.282)

Proof of (I.) Mattia et al [20] analyze the case µ(t) 6= 0 and show that the stationary solution

of (3.248)-(3.249)-(3.250)-(3.254) exists and is given by

φ0(V, t) =c

µ(t)

(

1 − exp(−2ζ(θ − V )

θ)

)

, (3.283)

where

c =

[

σ2(t)

2µ2(t)

(

2µ(t)θ

σ2(t)− 1 + exp

(

−2µ(t)θ

σ2(t)

))]−1

and ζ =µ(t)θ

σ2(t). (3.284)

61

Furthermore, it is easily verified that

limµ(t)→0+

φ0(V, t) =2

θ2(θ − V ) , 0 ≤ V ≤ theta, (3.285)

in agreeement with (3.280)

The firing rate ν(t) when µ(t) 6= 0 and ρ(V, t|0, 0) ≈ φ0(V, t).

Here we assume the Fokker-Planck solution reaches equilibrium, i.e. ρ(V, t) ≈ φ0(V, t). It follows

from (3.183) and (3.196) that ν(t) satisfies

ν(t) = −σ2(t)

2

∂

∂Vφ0(θ, t) ∀t ≥ 0. (3.286)

Substitution of (3.283)-(3.284) into (3.286) gives

ν(t) = c =

[

σ2(t)

2µ2(t)θ

(

2µ(t)θ

σ2(t)− 1 + exp

(

−2µ(t)θ

σ2(t)

))]−1

and ζ =µ(t)θ

σ2(t). (3.287)

An analysis of (3.287) shows that

ν(t) ≈ µ(t) when µ(t) is large. (3.288)

Interestingly, when µ(t) = 0 and no stationary solution exists, formula (3.286) indicates that the firing

rate is given by

ν(t) =σ2

θ2when µ(t) = 0. (3.289)

We compare this formula with that derived from first passage techniques (this assumes that the pdf

ρ(V, t|0, 0) exists), beginning with the definition

ν(t) =1

E(T (0)). (3.290)

Here E(T (0)) denotes the mean first passage time for V (t) to reach V (t) = θ from initial value

V (0) = vR = 0. To find E(T (0)) we set µ(t) = 0 and solve the ODE problem

σ2

2M ′′(V ) = −1, M ′(0) = M(θ) = 0. (3.291)

The solution of (3.291) is

M(V ) =θ2 − V 2

σ2. (3.292)

When V = 0 this reduces to

M(0) = E(T (0)) =θ2

σ2. (3.293)

Substituting (3.293) into (3.290) gives ν(t) = σ2

θ2 , in agreement with (3.289).

How to find the firing rate ν(t) when µ(t) 6= 0.

62

Recall from (3.183) that the firing rate ν(t) satisfies by

ν(t) = −σ2(t)

2

∂

∂Vρ(vT , t) ∀t ≥ 0. (3.294)

We assume that

ρ(V, t) =∑

N

aN(t)φN (V, t), −∞ < N <∞. (3.295)

To find each ak(t) multiply (3.295) by ψk(V, t) and integrate from V = 0 to V = θ to get

∫ θ

0

ψk(V, t)ρ(V, t)dV =

∫ θ

0

ψk(V, t)

(

∑

N

aN (t)φN (V, t)

)

dV, −∞ < N <∞ (3.296)

This reduces to

ak(t) = (ψk(V, t), ρ(V, t)) =

∫ θ

0

ψk(V, t)ρ(V, t)dV (3.297)

since

(ψk(V, t), φj(V, t)) = 0 when k 6= j, and (ψk(V, t), φk(V, t)) = 1. (3.298)

Thus, for each N we have

aN (t) = (ψN (V, t), ρ(V, t)) =

∫ θ

0

ψN (V, t)ρ(V, t)dV. (3.299)

Differentiating both sides of (3.299) with respect to t gives

d

dtaN (t) =

(

ψN (V, t),∂

∂tρ(V, t)

)

+dµ(t)

dt

(

∂

∂µψN (V, t), ρ(V, t)

)

. (3.300)

This reduces to

d

dtaN(t) = (ψN (V, t), Lρ(V, t)) +

dµ(t)

dt

(

∂


)

, (3.301)

henced

dtaN (t) =

(

L+ψN (V, t), ρ(V, t))

+dµ(t)

dt

(

∂


)

, (3.302)

and therefored

dtaN (t) = (λNψN (V, t), ρ(V, t)) +

dµ(t)

dt

(

∂


)

, (3.303)

Combining (3.304) with (3.295) and (3.299) gives

d

dtaN (t) = λNaN (t) +

dµ(t)

dt

∑

m

am(t)

(

∂

∂µψN (V, t), φm(V, t)

)

, (3.304)

The hard part is to find all the eigenvalues λN ’s and eigenfunctions φN ’s.

63

How to compute the ‘deterministic’ and population firing rate functyions.

Our goal here is to complete the computation of Figure 2 (left panel) of Mattia et al.

The deterministic firing rate. The non-noisy deterministic firing rate is shown in the upper left

panel of Figure 2. For this computation we use several matlab programs, all of which can be found at

www.math.pitt.edu/∼troy/lif/mattiafigure2/

(i) mattiafig2eigenvalues.m, f1a., f2a.m are used to compute the eigenvalues by means of a two dimen-

sional Newton iteration scheme when

µ = 25, σ = 1, θ = 1.

(ii) mattiafigure2driver.m, mattiafig2psi.m, mattiafig2odes.m, mattiafig2phiprime.m, are used to-

gether with the matlab ODE45 program to find the aN (t) functions, and subsequently to construct

the deterministic firing rate function. The program mattiafig2odes.m uses the command load mat-

tiafig2gammavalues.dat to construct the lambda values which appear on the right side of the aN (t)

ODES.

The population (i.e. noisy) firing rate. We consider a population of N uncoupled (i.e. in-

dependent) cells and obtain the ”noisy” firing rate function shown in Figure 2 (upper right panel).

Each cell is modelled by the linear integrate and fire SDE

dV

dt= µ(t) + σ(t)

dζ

dt, V (0) = 0. (3.305)

We require (see Figure 4) that the solution V (t) satisfy

0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.306)

Recall that a ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immedi-

ately reset to V = vR = 0. The reset condition

V (t−) = vT and V (t+) = vR. (3.307)

The procedure to obtain the firing rate is as follows: for each i ∈ [1, N ] solve (3.305)-(3.306)-(3.307)

and obtain the corresponding spike train

yi(t) =∑

j

δ(t− tij), (3.308)

where the values tij are the spike times. The firing rate ν(t) is defined by

ν(t) = limN→∞

1

N

N∑

i=1

∑

j

δ(t− tij)

. (3.309)

64

0 100 200 4000

30

60

t (msec)

V

0 100 200 4000

30

60

t (msec)

V

0 100 200 4000

30

60

t (msec)

V

0 100 200 4000

30

60

t (msec)

V

Figure 2: Upper left: “deterministic ” firing rate function computed from solving the Fokker-Planck

equation. Upper right: noisy firing rate function. Lower left: deterministic and noisy firing rate

functions superimposed. Lower right: the dashed curves denote the deterministic firing rate plus or

minus 2.5, where the value 2.5 is estimated from the finite size effect formula (??). Parameters are

µ = 25 and θ = σ = 1.

We let ∆ > 0 be small and approximate ν(t) by

ν(t) ≈ 1

N

N∑

i=1

∑

j

σij(t, t+ ∆)

∆

, (3.310)

where

σij(t, t+ ∆) =

1 if tij ∈ (t, t+ ∆),

0 otherwise.(3.311)

Matlab program mattiafigure2noisyfiringrate.m This program can be found at

65

0 100 200 4000

30

60

t (msec)

V

0 0.1 0.2 0.3 0.40

10

20

30

40

50

60

Figure 3: Left: deterministic firing rate function computed with matlab program ODE45. Right:

computation of the deterministic firing rate using the explicit Euler method. The ODE45 method gives

a better fit to the SDE generated population firing rate shown in Figure 2. The matlab driver programs

for these computations are www.math.pitt.edu/∼troy/lif/mattiafigure2/mattiafig2drivernew.m and

mattiafig2eulerdriver.m

0 500 10000

0.5

1

t (msec)

V

Figure 4: A solution of (3.305)-(3.306)-(3.307) for parameter values µ = 25, θ = 1, σ = 1. The problem

is solved for 0 ≤ t ≤ 1 second. The t values are multiplied by 1000 to illustrate milliseconds.

www.math.pitt.edu/∼troy/lif/mattiafigure2/mattiafig2noisyfiringrate.m

The main goal of mattiafigure2noisyfiringrate.m is to obtain the approximation for ν(t) given by (3.310)-

(3.311) (see Figure 2).

Step I. In the first part of the program we create a set of spike times. For this define N (the number

of uncoupled cells in the population) and ∆ by

N = 10000 and ∆ = .001 (3.312)

66

Define the define the step size for the solution of (3.305)-(3.306)-(3.307) by

dt =∆

100=.001

100= .000001 (3.313)

The number of time steps used to solve (3.305)-(3.306)-(3.307) for one cell is

K =1

dt= 100000. (3.314)

Next, set V (0) = 0, and initialize a row containing the spike times by the commands

V(0)=0; spiketimes=[]

Use the two for loops described below to solve (3.305)-(3.306)-(3.307) N=10000 times using Euler’s

method. During each run save the spike times in the row spiketimes.

for i=1:N (i.e. the number of cells is N)

for k=1:K (i.e. the number of time steps per solution is K)

V ((k + 1) ∗ dt) = V (k ∗ dt) + µ ∗ dt+ σ ∗√dt ∗ randn

Check for a spike: if V ((k + 1) ∗ dt) = 1 then V ((k + 1) ∗ dt) = 0 (reset condition) and add the

number t=(k+1)*dt to the row containing spike times. To do this use the command

spiketimes=[spiketimes (k+1)*dt];

end; end;

Step II. Sort through the row of spiketimes and create the function

n(t, t+ ∆t) = the number of spikes in (t, t+ ∆t), (3.315)

This function will then be used to obtain

ν(t) =n(t, t+ ∆t)

N∆t. (3.316)

To obtain ν(t, t+ ∆) we subdivide [0,1] into 1/∆ subintervals of (0,1]. We define the vector envalues,

which is the approximation to n(t, t+ ∆), by

envalues(kk) = the number of spikes in ((kk − 1)∆, kk∆] , 1 ≤ kk ≤ 1

∆= 1000. (3.317)

It takes 20 minutes to compute envalues. We then construct the noisy firing rate with the command

firingrate = envalues/(numberofcells ∗ ∆),

where numberofcells = N = 10000, ∆ = .001 and save it externally with the commands

tt = linspace(0, 1, 1/delta);

aa = [tt firingrate];

67

save mattiafig2noisyfiringrate.dat − ascii aa

Matlab program sortingexample.m This program can be found at

www.math.pitt.edu/∼troy/lif/mattiafigure2/sortingexample.m

The program sortingexample.m gives a simple example showing the algorithm used to create the vector

envalues by sorting a row of 12 spiketimes according to the number of spikes in 1/∆ subintervals of

[0,1] of length ∆.

Step III. In the third part of the program mattiafigure2noisyfiringrate.m we draw the four firing rate

pictures in Figure 2. To do this use the load command to get the externally saved data for both the

deterministic non noisy firing rate, and the noisy firing rate.

To load the detereministic firing rate data use the two commands

loadmattiafig2drivernewfiringrate.dat;

nonnoisyfiringrate = mattiafig2drivernewfiringrate;

To load the noisy firing rate data use the two commands

loadnoisyfiringrate.dat;

noisyfiringratedata = noisyfiringrate;

We use these data sets to obtain Figure 2.

68

How to solve the basic firing rate step problem.

We consider a population of N statistically identical, independent neuronal cells. Our goal is to

compute the deterministic and population firing rates ν∞(t) and νN (t) in “layer I”in response to a

step input from “layer 0.” Our step input example is

µ(t) =

0 if 0 ≤ t < 1,

25 if t ≥ 1.(3.318)

Each cell is modelled by the linear integrate and fire SDE

dV

dt= µ(t) + σ

dζ

dt, V (0) = 0, (3.319)

with the constraint that

0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.320)

Recall that a ’spike’ occurs if V (t) = vT at some finite t value, at which point the solution is immedi-

ately reset to V = vR = 0. The reset condition is

V (t−) = vT and V (t+) = vR. (3.321)

In our computations we set

vR = 0, vT = θ = 1 and σ = 1. (3.322)

The Fokker-Planck problem associated with (3.318)-(3.322) consists of the pde

∂

∂tρ(V, t) = −µ(t)

∂

∂Vρ(V, t) +

σ2(t)

2

∂2

∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.323)

with the ’absorbing’ boundary condition

ρ(vT , t) = 0 ∀t ≥ 0, (3.324)


vL

ρ(V, t)dV = 1 ∀t ≥ 0. (3.325)

We let ρ(V, t|0, 0) denote the particular solution of (3.323)-(3.324)-(3.325) such that

ρ(V, 0|0, 0) = δ(V ). (3.326)

The first theoretical difficulty is due to the fact that when µ(t) = 0 over [0, 1) we could not find a

solution ρ(V, t|0, 0) of the full problem (3.323)-(3.324)-(3.325)-(3.326). However, there is a stationary

solution which we might be able to use, namely

ρS(V ) =2

θ2(θ − V ). (3.327)

69

0 500 1000 15000

0.5

1

t (msec)

V

0 500 10000

30

60

t (msec)

νN

0 500 10000

1

3

5

t (msec)

νN

1000 14000

30

60

t (msec)

νN

Figure 5: Upper left: solution of the SDE problem (3.319)-(3.322). Parameters: σ = θ = 1,

µ(t) = 0 when t ∈ [0, 1), and µ(t) = 25 when t ∈ [1, 2]. Time has been translated to msec.

Upper right: population firing rate νN (t). Lower left: blowup of the firing rate diagram over

the first 1000 msec. shows that νN (t) quickly reaches the stationary firing rate νs = 1 as pre-

dicted by the theory in (3.329). Lower right: blowup of the firing rate diagram over the sec-

ond 1000 msec. shows the ringing effect. Computations were done with the matlab program

www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemfiringrate.m

The deterministic firing rate function ν∞(t) is defined by

ν∞(t) = −σ2

2

∂

∂Vρ(vT , t|0, 0) ∀t ≥ 0. (3.328)

Since there is only a stationary pdf when 0 ≤ t < 1 and µ(t) = 0, we conjecture that ρ(V, t|0, 0) =

ρS(V ) when 0 ≤ t < 1, and that

ν∞(t) = −σ2

2ρ′S(θ) =

σ2

θ2= 1 ∀t ∈ [0.1) (3.329)

70

0 500 10000

30

60

t (msec)

ν∞

0 500 10000

30

60

t (msec)

νN

0 500 10000

1

3

5

t (msec)

νN

1000 14000

30

60

t (msec)

νN

Figure 6: Upper left: infinite size firing rate ν∞(t). Parameters: σ = θ = 1, µ(t) = 0

when t ∈ [0, 1), and µ(t) = 25 when t ∈ [1, 2]. Time has been translated to msec. Up-

per right: population firing rate νN (t) and deterministic firing rate ν∞(t). Lower left: blowup

of the firing rate diagram over the first 1000 msec. Lower right: blowup over the second

1000 msec. shows the ringing effect. Computations were done with the matlab program

www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemnoisyfiringrate.m

Our numerical experiments illustrated in Figure 5 (upper right and lower left panels) indicate that

the prediction given in (3.329) does hold.

Next, we determine ρ(V, t) and ν∞(t) when t ≥ 1. We assume that ρ(V, t) has the form

ρ(V, t) =

∞∑

n=−∞an(t)φn(V, t), t ≥ 1 (3.330)

where φn(V, t) is the eigenfunction associated with the eigenvalue λn when µ = 25. Thus, we get

ν∞(t) = −σ2

2

∂

∂Vρ(θ, t) ∀t ≥ 1. (3.331)

71

0 500 10000

1

3

5

t (msec)

νN

1000 1500 200024

26

29

t (msec)

νN

Figure 7: Left panel: blowup of the first 1000 msec of the firing rate diagram shown in Figure 5.

Right panel: blowup of the second 1000 msec. The same vertical and horizontal scales in each panel

illustrate the significance of the finite size effect due to ν∞(t) changing from ν∞(t) = 1 (left panel) to

ν∞(t) = 25 (right panel) in the formula νN (t) = ν∞(t) +√

ν∞(t)N ζ(t).


ν∞(t) = −σ2

2

∞∑

n=−∞an(t)

∂

∂Vφn(θ, t) ∀t ≥ 1. (3.332)

To find each ak(t) multiply (3.295) by ψk(V, t) and integrate from V = 0 to V = θ to get

∫ θ

0

ψk(V, t)ρ(V, t)dV =

∫ θ

0

ψk(V, t)

( ∞∑

n=−∞an(t)φn(V )

)

dV, t ≥ 1. (3.333)

This reduces to

ak(t) = (ψk(V, t), ρ(V, t)) =

∫ θ

0

ψk(V, t)ρ(V, t)dV (3.334)

since

(ψk(V, t), φj(V, t)) = 0 when k 6= j, and (ψk(V, t), φk(V, t)) = 1. (3.335)

Thus, for each n we have

an(t) = (ψn(n, t), ρ(V, t)) =

∫ θ

0

ψn(V, t)ρ(V, t)dV, t ≥ 1. (3.336)

When t ≥ 1 the function an(t) satisfies the initial value ODE problem

a′n(t) = λNan, t > 1, (3.337)

and

an(1) =

∫ θ

0

ψN (V, 1+)ρ(V, 1−|0, 0)dV. (3.338)

72

Substituting ρ(V, 1−|0, 0) = ρS(V ) = 2θ2 (θ − V ) reduces (3.338) to

an(1) =

∫ θ

0

ψN (V, 1+)2

θ2(θ − V ) dV (3.339)

I. The computation of ν∞(t). The first step in computing ν∞(t) is to generate the gamma values

and the eigenvalues. For this use the matlab programs f1a.m, f2a.m amd mattiafig2eigenvalues.m in

www.math.pitt.edu/∼ troy/lif/stepproblem/

In all three programs the parameters are set to µ = 25, σ = 1 and θ = 1. Remove the per-

cent sign in front of the two save commands in mattiafig2eigenvalues.m and give the names mat-

tiafig2gammavaluesnew.dat and mattiafig2eigenvaluesnew.dat to the two data sets that contain the

gamma values and the eigenvalues. Type mattiafig2eigenvalues at the matlab prompt and the gamma

values and eigenvalues will be computed and saved in the files mattiafig2gammavaluesnew.dat and

mattiafig2eigenvaluesnew.dat

Next, to find ν∞(t) itself use the programs stepproblemdriver.m which is on the website

www.math.pitt.edu/∼ troy/lif/stepproblem/

Type stepproblemdriver at the matlab prompt ν∞(t) will be computed and drawn. The matlab

programs called by stepproblemdriver.m are also on this website.

II. The computation of νN (t). The procedure to obtain νN (t) is similar to that of the last subsection.

First, for each i ∈ [1, N ] solve (3.318)-(3.322) and obtain the corresponding spike train

yi(t) =∑

j

δ(t− tij), (3.340)

where the values tij are the spike times. The deterministic firing rate ν(t) satisfies

ν∞(t) = limN→∞

1

N

N∑

i=1

∑

j

δ(t− tij)

. (3.341)

We let ∆ > 0 be small and approximate ν∞(t) by the population firing rate

νN (t) =1

N

N∑

i=1

∑

j

σij(t, t+ ∆)

∆

, (3.342)

where

σij(t, t+ ∆) =

1 if tij ∈ (t, t+ ∆),

0 otherwise.(3.343)

Matlab program www.math.pitt.edu/∼troy/lif/stepproblem/stepproblemnoisyfiringrate.m

73

This program computes the approximation for νN (t) given by (3.342)-(3.343) (see Figure 5).

The first part of the program creates the spike times. For this define N (the number uncoupled cells

in the population) and ∆ by

N = 10000 and ∆ = .001 (3.344)

Define the the step size for the solution of (3.318)-(3.322) by

dt =∆

100=.001

100= .000001 (3.345)

The number of time steps used to solve (3.318)-(3.322) for one cell (Figure 5, upper left panel) is

K =2

dt= 200000. (3.346)

Next, set V (0) = 0, and initialize a row containing the spike times by the commands

V(0)=0; spiketimes=[]

Use the two for loops described below to solve the SDE=10000 times using Euler’s method. During

each run save the spike times in the row spiketimes.

for i=1:N (i.e. the number of cells is N)

for k=1:K (i.e. the number of time steps per solution is K)

V ((k + 1) ∗ dt) = V (k ∗ dt) + µ ∗ dt+ σ ∗√dt ∗ randn

Check for a spike: if V ((k + 1) ∗ dt) = 1 then V ((k + 1) ∗ dt) = 0 (reset condition) and add the

number t=(k+1)*dt to the row containing spike times. To do this use the command

spiketimes=[spiketimes (k+1)*dt];

end; end;

Next, sort through the row of spiketimes and create the function

n(t, t+ ∆t) = the number of spikes in (t, t+ ∆t), (3.347)

This function is then used to obtain

νN (t) =n(t, t+ ∆t)

N∆t. (3.348)

To obtain νN (t, t+∆) we subdivide [0,2] into 2/∆ subintervals of (0,2]. We define the vector envalues,

which is the approximation to n(t, t+ ∆), by

envalues(kk) = the number of spikes in ((kk − 1)∆, kk∆] , 1 ≤ kk ≤ 2

∆= 2000. (3.349)

It takes 20 minutes to compute envalues. We then construct νN (t) with the command

firingrate = envalues/(numberofcells ∗ ∆),

74

where numberofcells = N = 10000, ∆ = .001 and save it externally with the commands

tt = linspace(0, 1, 1/delta);

aa = [tt firingrate];

save steproblemfiringrate.dat − ascii aa

In the final part of stepproblemnoisyfiringrate.m we draw the firing rate diagrams in Figure 5. To do

this use the load command to get the externally saved data.

75

How to solve the basic feed forward problem.

This follows Doiron et [6]. We consider M populations of N cells which, in layer 1, are statistically

independent and identical. Our goals are to compute (i) the ‘finite size effect’ population firing rates

ν1,N (t), ν2,N (t), ν3,N (t), and (ii) the ‘infinite size limit’ firing rates ν1,∞, ν2,∞, ν3,∞ in layers 1, 2

and 3 in response to noise, and a step input from layer 0 of the form

ν0(t) =

0 if 0 ≤ t < t∗ (msec),

µ if t ≥ t∗.(3.350)

Each cell in layer m (m ≥ 1) receives input from exactly CEx excitatory cells and CIn inhibitory cells

in layer m-1, and also from a bath of noise present in layer m. The transmembrane potential Vi,m(t)

of cell i in layer m satisfies the ODE

dVi,m

dt= −β + Ii,m(t), Vj(0) = 0, (3.351)

where β > 0 is a constant, and Ii,m(t) is the input to cell i in layer m from layer m-1, and also from

the bath of noise in layer m. The positive constant β guarantees that (3.351) has a ‘stable rest state’

in the absence of the input Ii,m(t). To see this, remove Ii,m(t) so that (3.351) reduces to

dVi,m

dt= −β. (3.352)

If vL = vR = 0 and 0 ≤ V (0) ≤ vT then

Vi,m(t) → vL = vR = 0 as t→ tmin for some first tmin ≥ 0. (3.353)

Thus, vL = 0 is a stable rest state for (3.351) when β > 0. If β = 0 and Vi,m(0) > 0, then Vi,m(t) ≡constant, hence (3.353) does not hold and there is no stable rest state. Doiron et al [6] set β = .5

The input Ii,m(t) has the form

Ii,m(t) = JEx

N∑

j=1

KEx,m(i, j)yj(t) + JIn

N∑

j=1

KIn,m(i, j)yj(t) + σζi,m(t), (3.354)

where yj(t) is a spike train generated by cell j in layer m-1, KEx,m and KIn,m are N by N matrices,

σ2 is the variation of the noise in each layer, JEx ≥ 0 is the magnitude of postsynaptic current due to

a single excitatory presynaptic input, and JIn ≤ 0 is the magnitude of postsynaptic current due to a

single inhibitory presynaptic input. We assume that JEx and JIn have the form

JEx =JEx

CExand JIn =

JIn

CIn, (3.355)

where

JEx ∈ 0, 1, 2, 3 and JIn ∈ 0,−1,−2,−3. (3.356)

76

Thus, (3.354) becomes

Ii,m(t) =JEx

CEx

N∑

j=1

KEx,m(i, j)yj(t) +JIn

CIn

N∑

j=1

KIn,m(i, j)yj(t) + σζi,m(t). (3.357)

To understand how CEx and CIn are determined, define

NEx = the number of excitatory cells that can potentially connect to another ‘downstream’ cell,

pEx = the probability that an excitatory cell will connect to a downstream cell,

and

CEx = the number of excitatory cells that can potentially connect to another ‘downstream’ cell times

the probability that an excitatory cell will connect to a downstream cell.

Thus,

CEx = NEx ∗ pEx. (3.358)

Similarly,

NIn = the number of inhibitory cells that can potentially connect to another ‘downstream’ cell,

pIn = the probability that an inhibitory cell will connect to a downstream cell,

and

CEIn = the number of excitatory cells that can potentially connect to another ‘downstream’ cell times

the probability that an excitatory cell will connect to a downstream cell.

Thus,

CIn = NIn ∗ pEx. (3.359)

To determine the values of NEx and NIn we combine two observations. First,

NEx +NIn = N = the total number of cells in the population. (3.360)

Secondly, experimental evidence indicates that

NEx = 4NIn. (3.361)

Thus, if N = 1000, we conclude that

NEx = .8 ∗N = 800 and NIn = .2 ∗N = 200. (3.362)

Experimental evidence suggests that pIn = 4pEx, and that a reasonable value for pEx is pEx = .125 [6].

Therefore, pIn = .5 and we conclude that

CEx = NEx ∗ pEx = 800 ∗ .125 = 100 and CIn = 200 ∗ pIn = 200 ∗ .5 = 100. (3.363)

Next, we need to understand the role of the N by N matrices KEx,m(i, j) and KIn,m(i, j).

77

The matrix KEx,m(i, j) consists of 0’s and 1’s, with exactly CEx randomly distributed 1’s in each row.

The interpretation of 0’s and 1’s in row i of KEx,m(i, j) is that

ProbKEx,m(i, j) = 1 = pEx and Prob KEx,m(i, j) = 0 = 1 − pEx. (3.364)

The definition of the N by N matrix KIn(i, j) is similar and omitted for the sake of brevity.

How to estimate values for JEx and JIn in the LIF model.

Let V If (t) and V Exp(t) denote the theoretical (i.e LIF) transmembrane potential and experimental

transmembrane potential, repectively. The experimental rest state and threshold values are

V ExpR = −65 mv and V Exp

T = −55 mv. (3.365)

The theoretical rest state and threshold values are

V IfR = 0 mv and V If

T = 1 mv. (3.366)

We assume that V If (t) and V Exp(t) are related linearly by an equation of the form

V IfR (t) = aV Exp(t) + b. (3.367)

Substituting (3.365) and (3.366) gives

V IfR (t) =

1

10

(

V Exp(t) + 65)

. (3.368)

Next, let JEx denote the height of the response of a synapse to an excitatory change in voltage, and

let JIn denote the height of the response of a synapse to an inhibitory change in voltage. We believe

the experimental observation that an excitatory kick causes an increase of .05 mv. in transmembrane

potential from −65 mv. to −64.5 mv. Substituting this into (3.368) gives

JEx =1

10(−64.5 + 65) = .05 mv. (3.369)

Similarly, we believe the experimental observation that an inhibitory kick causes a dcrease in trans-

membrane potential from −65 mv. to −66 mv. Substituting this into (3.368) gives

JIn =1

10(−66 + 65) = −.1 mv. (3.370)

Note that |JIn| = 2|JEx|. That is, an inhibitory kick has double the strength of an excitatory kick.

One possible case to study is the ’balanced’ setting, in which it is required that

JExCEx + JInCIn = 0. (3.371)

Substituting (3.369) and (3.369) into (3.369) gives

CEx = 2CIn. (3.372)

78

The ubalanced case requires that

JExCEx + JInCIn 6= 0. (3.373)

Properties of layer 0. In layer 0 let Xi(t) denote the number of spikes generated by cell number i

in time interval (0, t). It is assumed that each Xi(t) is a Poisson random variable with the same firing

rate

ν0(t) =

0 Hz if 0 ≤ t < t∗ (msec),

40 Hz if t ≥ t∗.(3.374)

Thus, each cell generates spike trains which are statisitically identical.

Properties of layer 1, 2 and 3. The transmembrane potential Vi,m(t) of cell i in layer m (m ≥ 1)

satisfies the SDEdVi,m

dt= −β + Ii,m(t), Vj(0) = 0, (3.375)

where the input Ii,m(t) has the form

Ii,m(t) = JEx

N∑

j=1

KEx,m(i, j)yj(t) + JIn

N∑

j=1

KIn,m(i, j)yj(t) + σζi,m(t). (3.376)

We assume that

JEx =JEx

CExand JIn =

JIn

CIn, (3.377)

hence (3.375) becomes

dVi,m

dt= −β +

JEx

CEx

N∑

j=1

KEx,m(i, j)yj(t) +JIn

CIn

N∑

j=1

KIn,m(i, j)yj(t) + σζi,m(t), Vj(0) = 0. (3.378)

The spike trains yj(t) are generated by cell j in layer 0, and are all statically identical. Because of

this, we follow Doiron et al [6] and assume that the cells in layer 1 are statistically independent and

identical. To understand what ‘statistically independent’ means let Vk(t) denote the transmembrane

potential of cell no. k, and let ρm(Vk, t) denote its pdf (probability distribution function). If i 6= j let

ρ(Vi, Vj , t) denote the joint pdf of Vi(t) and Vj(t). Then ρm(Vi, t) is obtained from ρ(Vi, Vj , t) by the

formula

ρm(Vi, t) =

∫ vT

vL

ρ(Vi, Vj , t)dVj . (3.379)

Cells number i and j are statistically independent if

ρ(Vi, Vj , t) = ρm(Vi, t)ρm(Vj , t), i 6= j. (3.380)

The population of cells is statistically independent if (3.380) extends to any finite combination of cells

in the population. The cells are ‘identical’ if they are identically distributed, i.e.

ρm(Vi, t) = ρm(Vj , t), i 6= j. (3.381)

79

Because the cells in layer 1 are statistically independent and identical, each cell in layer 1 is modelled

by diffusion approximation to (3.375), namely

dV

dt= −β +

(

JEx + JIn

)

ν0(t) +

√

(

J2Ex

CEx+J2

In

CEx

)

ν0(t) + σ2ζ(t), V (0) = 0, (3.382)

where

0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞). (3.383)

If a ’spike’ occurs if V (t) = vT at some finite t value then the solution is immediately reset to

V = vR = 0. The reset condition is

V (t−) = vT and V (t+) = vR. (3.384)

In our computations we set

vR = 0 and vT = θ = 1. (3.385)

The computations for Figures 8, 9 and 12 We follow Doiron et al and use parameter values

N = 1000, β = −.5, σ = .63, CEx = 100, JEx = 2, CIn = 48, JIn = −.96 (3.386)

For the values given in (3.386) the SDE (3.382) becomes

dV

dt= −.5 + 1.04ν0(t) +

√

.00592ν0(t) + .632ζ(t), V (0) = 0, (3.387)

where

ν0(t) =

0 Hz if 0 ≤ t < 500 (msec),

40 Hz if 500 (msec) ≤ t ≤ 900 (msec).(3.388)

In layer 1 we solve (3.382)-(3.388) 1000 times.

For layers 2 and 3 we substitute the values given in (3.386) into (3.378) and obtain

dVi,m

dt= −.5 + .02

N∑

j=1

KEx,m(i, j)yj(t) − .02

N∑

j=1

KIn,m(i, j)yj(t) + .63ζi,m(t), Vj(0) = 0, (3.389)

We solve (3.389) 1000 times for m=2 (layer 2) and m=3 (layer 3). When N=1000 note that KEx(i, j)

and KIn(i, j) are 1000 by 1000 matrices with 1 million entries. To simplify the computations we take

advantage of their sparse nature by generating smaller N by CEx and N by CIn (i.e. 1000 by 100

and 1000 by 48) matrices: the rows of the bigger matrix consist of 100 different randomly distributed

integers between 1 and 1000, and the rows of the smaller consist consist of 48 different randomly

distributed integers between 1 and 1000, This is done with the program /lif/fforward2/ kmatrix.m

All three figures exhibit oscillations during the ‘transient phase.’ During the transient phase the

functions ν1,N(t), ν2,N (t), ν3,N (t) oscillate with period ≈ 24.33 msec. This result is obtained with

the formula

Period ≈ 1 sec.

(−β + (JEx + JIn)ν0(t)) cycles=

1000 msec.

41.1 cycles= 24.33

msec

cycle. (3.390)

80

700 750 8000

50

100

200

t (msec)

Layer 3ν3,N

layer3noisyfiringrate4a.eps

1000 10750

50

100

t (msec)

νN

Layer 3

700 750 8000

50

100

200

t (msec)

Layer 2ν2,N

layer2noisyfiringrate4a.eps

1000 10750

50

100

t (msec)

νN

Layer 2

0 500 10000

50

100

t (msec)

νN

Layer 1

1000 10750

50

100

t (msec)

νN

Layer 1

Figure 8: Propagation of ringing. Computations for layer 1 were done by solving (3.382)-(3.388) 1000

times. Computations for for layers 2 and 3 were done by solving (3.389) 1000 times for m=1 (layer 1)

and m=2 (layer 2). Error in computaions for layer 1: in [0,500] we used σ = 1.66 instead of the cor-

rect value σ = .63 The computations were done in /lif/feedforweard3/ with layer1noisyfiringrate4.m,

layer2noisyfiringrate2.m, and layer3noisyfiringrate4.m.

Next, we descrivbe how to find ν1,∞(t) from the Fokker-Planck equation

∂

∂tρ(V, t) = (β − µ(t))

∂

∂Vρ(V, t) +

σ2(t)

2

∂2

∂V 2ρ(V, t) + ν(t)δ(V − vR), (3.391)

where

µ(t) = (JEx + JIn)ν0(t), (3.392)

81

700 7500

50

100

t (msec)

Layer 3ν

3,N

700 750 8000

50

100

t (msec)

Layer 2ν2,N

700 750 8000

50

100

t (msec)

Layer 1ν1,N

Figure 9: Propagation of ringing as in Figure 8 but now we use the correct for creating spike-

trains in layer 1. The computations were done in /lif/feedforweard3/ with layer1noisyfiringrate5.m,

layer2noisyfiringrate5.m, and layer3noisyfiringrate5.m.

and

σ2(t) =

√

(

J2Ex

CEx+J2

In

CEx

)

ν0(t) + σ2 (3.393)

82

700 750 8000

50

100

t (msec)

Layer 1 N=1000Cex=100 Cin=48ν

1,N

700 750 8000

50

100

t (msec)


1,N

700 750 8000

50

100

t (msec)


1,N

Figure 10: Layer 1 ringing. Top row: N=1000 CEx = 100 and CIn = 48. Row 2: N=1000, CEx = 100

and CIn = 48 repeat the experiment 10 times. Row 3: N=10000 and CEx = 1000, CIn = 480 Layer 0

input: ν0(t) = 0 Hz for t ∈ [0, 500), and ν0(t) = 40 Hz for t ∈ [500, 900]. Parameters: σ = .63, JEx =

2, JIn = −.96, CIn = 48. (upper and middle panels) Computations done in /lif/feedforweard3/ with

layer1noisyfiringrate5a.m, layer1noisyfiringrate5b.m, layer1noisyfiringrate5bb.m.

83

ρ(V, t) satisfies the ’absorbing’ boundary condition

ρ(vT , t) = 0 ∀t ≥ 0, (3.394)


vL

ρ(V, t)dV = 1 ∀t ≥ 0. (3.395)

We let ρ(V, t|0, 0) denote the particular solution of (3.323)-(3.324)-(3.325) such that

ρ(V, 0|0, 0) = δ(V ). (3.396)

As in the last section, because µ(t) = 0 over [0, .5), we could not find ρ(V, t|0, 0). However, there is a

stationary solution, namely

ρS(V ) =2

θ2(θ − V ). (3.397)

The ‘infinite size limit’ firing rate ν∞(t) is defined by

ν∞(t) = −σ2

2

∂

∂Vρ(vT , t|0, 0) ∀t ≥ 0. (3.398)

Since there is only a stationary pdf when 0 ≤ t < t∗ and µ(t) = 0, we conjecture that ρ(V, t|0, 0) =

ρS(V ) when 0 ≤ t < t∗, and that

ν∞(t) = −σ2

2ρ′S(θ) =

σ2

θ2∀t ∈ [0, t∗) (3.399)

Our numerical experiments described below indicate that the prediction given in (3.399) does hold.

To compute ν2,∞(t) we need to solve the Fokker-Planck equation corresponding to the SDE

dV

dt= µ2(t) + σ2(t)ζ(t), V (0) = 0, (3.400)

where

µ2(t) = µ2 + (JExCEx + JInCIn) ν1,∞(t) and σ22,∞(t) = σ2 +

(

J2ExCEx + J2

InCIn

)

ν1,∞(t). (3.401)

In Figure ?? we assume that

µ2 = 0, JEx =JEx

CEx=

1

CExand JIn =

JIn

CIn= 0, (3.402)

and therefore (3.401) reduces to

µ2(t) = ν1,∞(t) and σ22,∞(t) = σ2 +

ν1,∞(t)

CEx= σ2 +

ν1,∞(t)

100(3.403)

Substituting (3.403) into (3.400) gives the SDE

dV

dt= ν1,∞(t) +

(

σ2 +ν1,∞(t)

100

)

ζ(t), V (0) = 0. (3.404)

84

Feedforward propagation of finite width pulses.

In the unbalanced setting we assume that

JEx =JEx

CExand JIn =

JIn

CIn. (3.405)

The parameters for our balanced setting experiments are

dt = .00001, β = −.5, σ = .63, JEx = 2, CEx = 100, JIn = −.96, CIn = 48. (3.406)

The firing rate in Figures 11 and 12 is

ν0(t) =

0 Hz if 0 ≤ t < 750 (msec),

41.1 Hz if 750 (msec) ≤ t < 800 (msec),

0 Hz if 800 (msec) ≤ t < 900 (msec).

(3.407)

700 750 8000

50

100

t (msec)


1,N

700 750 8000

50

100

t (msec)


1,N

700 750 8000

50

100

t (msec)


1,N

Figure 11: Upper right: repeat the experiment in upper left 10 times. Lower: increase N to N=10000

and preserve CEx

N = .1 and CIn

N = .048. /lif/feedforward4/layer1noisyfiringratetwobump4a.m,

layer1noisyfiringratetwobump4aa.m and layer1noisyfiringratetwobumpaaa.m

In the balanced setting We assume that

JEx =JEx√CEx

and, JIn =JIn√CIn

and JEx

√

CEx = JIn

√

CIn (3.408)

Question. In the figures do we see memory in the background noise after the input?

85

700 750 8000

50

100

200

t (msec)

Layer 3ν3,N

700 750 8000

50

100

200

t (msec)

Layer 2ν2,N

700 750 8000

50

100

200

t (msec)

Layer 1ν1,N

Figure 12: Unbalanced setting propagation of one-bump ring. Layer 0 input: ν0(t) = 0 Hz for t ∈[0, 750), ν0(t) = 41.1 Hz for t ∈ [750, 775], ν0(t) = 0 Hz for t ∈ [775, 900].Parameters: dt = .00001, β =

−.5, σ = .63, JEx = 2, JIn = −.96, CIn = 48, CEx = 100. The ‘background’ noise does not propa-

gate to layers 2 and 3. Computations in /lif/feedforweard4/ with layer1noisyfiringrateonebump4a.m,

layer2noisyfiringrateonebumpb.m, and layer3noisyfiringrateonebump4a.m.

86

525

50

100

t (msec)

Layer 2

95 spikes

300 525 8000

50

100

t (msec)

Layer 1

Cex=Cin=50

Jex=−Jin=.35

N=1000

914 spikes

β=.5, σ=.5

ν1,N

Figure 13: Propagation failure in the balanced setting . Parameters: JEx = −JIn = −.35, CEx =

CIn = 50 dt = .00001, β = .5, σ = .63, Dashed curve is the layer 0 input: ν0(t) = 0 Hz for

t ∈ [0, 500), ν0(t) = 80 Hz for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900] Computations done in

/lif/feedforweard4/ with layer1fff.m and layer2fff.m

87

300 500 6000

15

50

100

t (msec)

Layer 3

886 spikesν

3,N

300 500 6000

10

20

Blowupν3,N

300 500 6000

50

100

t (msec)

Layer 2

1122 spikesν

2,N

300 500 6000

10

20

Blowupν2,N

300 500 6000

15

50

100

t (msec)

Layer 1

Cex=Cin=50

Jex=−Jin=.7

N=1000

1418 spikes

β=.5, σ=.63

ν1,N

300 500 6000

10

20

Blowupν

1,N

Figure 14: Balanced setting. Propagation failure. Parameters: N = 1000, β = .5, σ = .63, CEx =

CIn = 50, JEx = −JIn = .7 Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz

for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. Computations done in /lif/feedforweard4/

with layer1mmm1.m, lyer1mmm1blowup.m, layer2mmm1, layer2mmm1blowup.m, layer3mmm1.m,

layer3mmmblowup.m

88

300 525 800

50

100

t (msec)

Layer 2

739 spikes

300 525 8000

50

100

t (msec)

Layer 1

Cex=CIn

=50

JEx

=−JIn

=1

N=1000

453 spikes

ν1,N

Figure 15: Propagation success in the balanced case. Dashed curve is the layer 0 input: ν0(t) = 0 Hz

for t ∈ [0, 500), ν0(t) = 4 Hz for t ∈ [500, 550] and ν0(t) = 0 Hz for t ∈ [550, 900]. Parameters: dt =

.00001, β = .5, σ = .63, JEx = 1, JIn = −1. Computations done in /lif/feedforward4/layer1ccc.m,

layer2ccc.m, layer2ddd.m

89

300 525 8000

50

100

t (msec)

Layer 1

Cex=CIn

=50

JEx

=−JIn

=1

N=1000

453 spikes

ν1,N

300 500 8000

50

100

t (msec)

Layer 1

N=20000

3800 spikes

ν1,N

300 500 8000.5

4

8

Blowup

N=20000

ν1,N

Figure 16: Balanced setting. Parameters: dt = .00001, β = .5, σ = .63, CEx = CIn = 50 and

JEx = 1, JIn = −1. Dashed curve is the layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 4 Hz for

t ∈ [500, 550] and ν0(t) = 0 Hz for t ∈ [550, 900] Middle row: repeat the experiment in top row 20

times. Bottom row: Blowup of figure in middle row. Computations done in /lif/feedforweard4/ with

layer1ccc.m and layer1ccca.m

90

500 600

50

100

t (msec)

Layer 3

6410 spikesν

3,N

500 6000

5

10

Blowupν3,N

500 600

50

100

t (msec)

Layer 2

4357 spikesν

2,N

500 6000

5

10

Blowupν2,N

300 500 6000

15

50

100

t (msec)

Layer 1

Cex=Cin=50

Jex=−Jin=1

N=1000

2922 spikes

β=.5, σ=.63

ν1,N

500 6000

5

10

Blowupν1,N

Figure 17: Balanced setting. Progation success. Parameters: N = 1000, β = .5, σ = .63, CEx =

CIn = 50, JEx = −JIn = 1. Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz

for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. The layer 0 firing rate successefully propagates

to higher layers. Computations done in /lif/feedforweard4/ with matlab programs layer1mmm.m,

lyer1mmmblowup.m, layer2mmm.m, layer2mmmblowup.m, layer3mmm.m, layer3mmmblowup.m

91

5000 100000

15

50

100

t (msec)

Layer 1 N=1000

Cex=Cin=50

Jex=−Jin=1

4229 spikes

β=.5, σ=.63

ν1,N

500 6000

2

5

t (msec)

Blowup 1ν1,N

2000 2400 28000

2

5

t (msec)

Blowup 2ν1,N

9200 9600 100000

2

5

t (msec)

Blowup 3ν1,N

Figure 18: Balanced setting layer 1 computations to illustrate what happens to the post-stimulus

noise shown in third row of Figure 17. Layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500), ν0(t) = 15 Hz

for t ∈ [500, 600], ν0(t) = 0 Hz for t ∈ [600, 10000]. Parameters CEx = CIn = 50, JEx = −JIn = 1

and β = .1, σ = .1 Upper left: layer 1 firing rate curve. Upper right: blowup of the upper left panel

indicates that the post-stimulus noise in the firing rate in the time interval [600,800] has increased

when compared with the pre-stimulus noise in [0,500]. Lower panels: blowups over [2000,2800] and

[9200,10000] seems to indicate that the evolves into the stationary state. Comparison with the next

figure gives further credence to this conjecture. Computations done in /lif/feedforweard4/ with matlab

prograam layer1mmm5.m,

In Figure 18 we investigate what happens to the post-stimulus noise that is observed in the third row

of Figure 17. For this we solve the SDE on the extended time interval [0,10000]. It seems that the

post-stimulus immediastely enteres the stationary state.

92

5000 100000

15

50

100

t (msec)

Layer 1 N=1000

Cex=Cin=50

Jex=−Jin=1

4229 spikes

β=.5, σ=.63

ν1,N

500 6000

2

5

t (msec)

Blowup 1ν1,N

2000 2400 28000

2

5

t (msec)

Blowup 2ν1,N

9200 9600 100000

2

5

t (msec)

Blowup 3ν1,N

Figure 19: Balanced setting layer 1 computations to extend those in Figure 18 and illustrate what

happens to the noise when no stimulus is present. Layer 0 input is set equal to zero: ν0(t) = 0 Hz ∀t ≥0. ν0(t) = 15 Hz Parameters CEx = CIn = 50, JEx = −JIn = 1 and β = .1, σ = .1 Upper left: layer 1

firing rate curve. Upper right: blowup of the upper left panel shows that the noise in the firing rate

increases. Lower panels: blowups over [2000,2800] and [9200,10000] indicates that the noise evolves

into the stationary state. The firing rate over [9200,10000] apopears to be the same as in Figure 18.

Computations done in /lif/feedforweard4/ with matlab prograam layer1mmm5a.m,

In Figure 19 we remove the stimulus and de,onstrate how the noise evolves into the stationary state.

93

300 500 600

50

100

t (msec)

Layer 3

552 spikes

ν1,N

300 500 6000

5

10

Blowupν1,N

500 600

50

100

t (msec)

Layer 2

352 spikes

ν1,N

500 6000

5

10

Blowupν1,N

300 500 6000

50

100

t (msec)

Layer 1

Cex=Cin=50

Jex=−Jin=1

N=1000

221 spikes

β=.5, σ=.63

ν1,N

500 6000

5

10

Blowupν1,N

Figure 20: Balanced setting. Sucess of propagation. Parameters: N = 1000, β = .5, σ = .63,

CEx = CIn = 50, JEx = −JIn = .35 Dashed curve is layer 0 input: ν0(t) = 0 Hz for t ∈ [0, 500),

ν0(t) = 1 Hz for t ∈ [500, 600] and ν0(t) = 0 Hz for t ∈ [600, 900]. The small layer 0 firing rate successe-

fully propagates to higher layers. Computations done in /lif/feedforweard4/ with matlab programs

layer1ppp.m, layerpppblowup.m, layer2ppp.m, layer2pppblowup.m, layer3ppp.m, layer3pppblowup.m

94

200 300 550 6500

15

50

100

t (msec)

Layer 1 N=1000

Cex=Cin=50 Jex=−Jin=1

β=.5, σ=.63

ν1,N

/lif/feedforward4/layer1mmm3.m layer1mmm31.eps

200 300 550 6500

15

50

100

t (msec)

Layer 1 N=125000


β=.5, σ=.63

spikes: .26 2779 64.2 2852 66.2spikes: .26 2779 64.2 2852 66.2

Diff=73 Rel. Change=2.7%

ν1,N

/lif/feedforward4/layermmm3.m layer1mmm331.eps

200 300 550 6500

15

50

100

t (msec)

Layer 1 N=1000


β=.1, σ=.1

ν1,N

/lif/feedforward4/layermmm4.m layer1mmm314.eps

200 300 550 6500

15

50

100

t (msec)

Layer 1 N=1000


spikes: 0 2672 3 2832 4

Diff=160 Rel. Change=6%

β=.1, σ=.1

ν1,N

Figure 21: Balanced setting. Layer 1 computations. Dashed curve is layer 0 input: ν0(t) = 0 Hz

for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ [300, 550], ν0(t) = 15 Hz for

t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ [650, 900]. Parameters: CEx = CIn = 50, JEx = −JIn = 1.

Upper row: β = .5, σ = .63 gives an increase of 74 more spikes (2.76 percent) in [550,650] than in

[200,300]. Lower row: β = .1, σ = .1 gives an increase of 153 more spikes (5.6 percent) Computations

done in /lif/feedforweard4/ with matlab prograams layer1mmm3.m, layer1mmm4.m

Basic assumptions on a the pool of cells.

We consider a collection, of size N, of neuronal cells which are stastically identical and sparsely

connectd. For fixed i ∈ [1,N] define the counting process

Xi(t) = the number of spikes emitted by cell i in (0, t). (3.409)

We assume that the cells are statistically identical (Figure 24) and sparsely connected (Figure 25). To

say that they are statistically identical means that the pdf (probability distribution function) of Xi(t)

95

0 200 300 550 650

50

100

t (msec)

Layer 2 N=1000

spikes: 0 3427 18 3787 24

Diff=360 Rel. Change=10.5%

ν2,N

200 300 550 650 9000

1

2

3

4

t (msec)

Layer 2

Blowup

ν2,N

200 300 550 6500

15

50

100

t (msec)

Layer 1 N=1000


spikes: 0 2672 3 2832 4

Diff=160 Rel. Change=6%

β=.1, σ=.1

ν1,N

0 200 300 550 6500

0.5

1

V

t (msec)

Layer 2

Figure 22: Balanced setting. Layer 1 and layer 2 computations. Dashed curve is layer 0 input:

ν0(t) = 0 Hz for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ [300, 550],

ν0(t) = 15 Hz for t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ [650, 900]. Parameters: CEx = CIn = 50,

JEx = −JIn = 1 and β = .1, σ = .1 Lower left: layer 1 firing rate shows that there is an increase

of 160 more spikes (6 percent) in [550,650] than in [200,300]. Upper left: layer 2 shows an increase

of 360 more spikes (10.5 percent) in [550,650] than in [200,300]. Upper right: blowup of layer 2 to

see the noise between stimulus regions. Lower right: typical layer 2 solution. Computations done in

/lif/feedforweard4/ with matlab prograams layer1mmm4.m, layer2mmm4.m

is the same for 1 ≤ i ≤ N. To say that the cells are sparsely connected means that the probability of

two cells receiving input from the same third cell is small. To develop a mathematical formualtion of

the concept of ‘sparse connectivity’ we define

Probi→ k = The probability that cell i is connected to cell k (3.410)

96

250 600 9500

15

50

100

t (msec)

Layer 1 N=1000


β=.1, σ=.1

ν1,N

250 600 9500

15

50

100

t (msec)

Layer 1 N=25000


spikes: 0 2717 1.6 2872 1.9 2867 1

Diff1=155 Rel. Change=5.7%

β=.1, σ=.1

ν1,N

Figure 23: Balnced setting 3 stimulus computations. Dashed curve is the layer 0 input: ν0(t) = 0 Hz

for t ∈ [0, 200), ν0(t) = 15 Hz for t ∈ [200, 300], ν0(t) = 0 Hz for t ∈ (300, 550), ν0(t) = 15 Hz for

t ∈ [550, 650] and ν0(t) = 0 Hz for t ∈ (650, 900] Parameters: dt = .00001, β = .1, σ = .1, JEx = 1,

JIn = −1. Lower panel repeats the N=1000 experiment 25 times and shows an average increase of

152 spikes from 1st to 2nd stimulus, no increase from 2nd to 3rd stimulus. Computations done in

/lif/feedforward4/layer1mmm6.m

97

ν(t)

Figure 24: Schematic of a population of cells that are statistically identical. This diagram was provided

by Brent Doiron.

and

Probi→ k, j → k = The probability that cells i and j are both connected to cell k. (3.411)

If the cells are independent then

Probi→ k, j → k = Probi→ kProbj → k (3.412)

Set

Probi→ k = p ∈ (0, 1). (3.413)

The cells sparsely connected if they are independent and

Probi→ k, j → k = p2 << 1. (3.414)

Figure 25 illustrates the results of Oswald et al [24] who give experimental evidence for the sparse

connectivity of a pool of pyramidal cells. The upper panel graphs p, the probability that two cells are

connected, vs. the distance between cells. Note that 0 < p < .2 so that p2 < .04 is small, as required

for sparse connectivity.

98

A

B

0.3

0.2

0.1

0.0 Prob

abilit

y of

Con

nect

ion

140 120 100 80 60 40 20

Distance between Somas (μm)

x y

-150 -100 -50 0 50 100 150

Distance X (μm)

-100

-50

50

100

Dis

tanc

e Y

(μm

)

0

-150

150

Tested Connected Presynaptic Cell

L1

L2/3

Figure 25: Upper: probability p that two cell pyramidal cells are connected vs. distance (µm). Note

that 0 < p < .2 so that p2 < .04 Lower panel: This diagram was provided by Brent Doiron.

Assumptions on the current entering a cell.

I. We consider a population of N neuronal cells which are stastically identical and indpependent. For

each i ∈ [1,N] and t > 0 let

Xi(t) = the number of spikes emitted by cell i in [0, t). (3.415)

We assume that the Xi(t) satisfy Xi(0) = 0, and are independent Poisson processes with parameter

ν∞. The firing rate dXi

dt of cell i is the spike train yi(t) defined by

yi(t) =dXi

dt=∑

j

δ(t− tij). (3.416)

From (3.416) it follows that

E (yi(t)) = ν∞ ∀i ∈ [1, N ]. (3.417)

Since the Xi(t) are independent then yi(t) and yk(t′) are independent when i 6= k, hence

E (yi(t)yk(t′)) = E (yi(t))E (yk(t′)) = ν2∞ ∀i 6= k ∈ [1, N ], and t, t′ ≥ 0. (3.418)

99

Because each Xi(t) is Poisson process, we conclude that

E (yi(t)yi(t′)) = ν∞δ(t− t′) + ν2

∞ ∀i ∈ [1, N ], and t ≥ t′ ≥ 0. (3.419)

II. The current Ij(t) which enters cell j is assumed to have the form

Ij(t) = µ+ JN∑

i=1

yi(t) + σζj(t), (3.420)

where µ ∈ R and σ > 0 are constants, and ζj(t) is white noise. The cells are excitatory if J > 0, and

inhibitory of J < 0.

III. Goal. Our goal is to find constants µ ∈ R and σ > 0 such that Ij(t) ≈ Ij(t) where

Ij(t) = µ+ σζj(t), (3.421)

and ζj(t) is white noise. In particular, we show below that

µ = µ+ JNν∞ and σ2 = σ2 + J2Nν∞, (3.422)

and therefore

Ij(t) = µ+ JNν∞ +√

σ2 + J2Nν∞ζj(t). (3.423)

Remark. It is interesting to observe that Var(Ij)(t) = Var(Ij)(t) = ∞ for every value of N.

IV. To derive the values of µ and σ2 given in (3.422) we require that two conditions hold, namely

E (Ij(t)) = E(Ij(t)) and E (Ij(t)Ij(t′)) = E(Ij(t)Ij(t

′)). (3.424)

The first step is to conclude from (3.421) that

E(

Ij(t))

= µ, (3.425)

and

E(

Ij(t)Ij(t′))

= µ2 + σ2δ(t− t′). (3.426)

Next, repeat these calculations for the form of Ij(t) assumed in (3.420). First, we get

E (Ij(t)) = µ+ J

N∑

i=1

E (yi(t)) = µ+ JNν∞(t). (3.427)

Equating the right sides of (3.425) and (3.427) gives

µ = µ+ JNν∞. (3.428)


E(

Ij(t)Ij(t′))

= µ2 + 2µJNν∞ + J2N2ν2∞ + σ2δ(t− t′). (3.429)

100

Next, observe that

E (Ij(t)Ij(t′)) = E

([

µ+ J

N∑

i=1

yi(t) + σζj(t)

][

µ+ J

N∑

k=1

yk(t′) + σζj(t′)

])

. (3.430)

We need to evaluate the right side of (3.430). First, note that

E(

µ2)

= µ2, E(

µσζj(t))

= E(

µσζj(t′))

= 0, (3.431)

and

E

(

µJN∑

i=1

yi(t)

)

= E

(

µJN∑

i=1

yi(t′)

)

= µJNν∞. (3.432)

Since the spike trains are independent of the white noise terms, it follows that

E(

µJyiζj(t))

= E (µJyi)E(

ζj(t))

= 0 and E(

µJζj(t′))

= E (µJyi)E(

ζj(t′))

= 0. (3.433)

Because ζj(t) is white noise we conclude that

E(

σ2ζj(t)ζj(t′))

= σ2δ(t− t′). (3.434)

Combining (3.430) with (3.431), (3.432), (3.433) and (3.434) gives

E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + σ2δ(t− t′) + J2

N∑

i=1

N∑

k=1

E(yi(t)yk(t′)). (3.435)

Next, writeN∑

i=1

N∑

k=1

E(yi(t)yk(t′)) =

N∑

i=1

E(yi(t)yi(t′)) +

∑

i6=k

E(yi(t)yk(t′)). (3.436)


N∑

i=1

E(yi(t)yi(t′)) = Nν∞δ(t− t′) +Nν2

∞. (3.437)

From (3.418) we conclude that

∑

i6=k

E(yi(t)yk(t′)) = N (N − 1) ν2∞. (3.438)


N∑

i=1

N∑

k=1

E(yi(t)yk(t′)) = Nν∞δ(t− t′) +N2ν2∞. (3.439)

Substituting (3.439) into (3.435), we obtain

E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + σ2δ(t− t′) + J2

(

Nν∞δ(t− t′) +N2ν2∞)

. (3.440)

101

This reduces to

E (Ij(t)Ij(t′)) = µ2 + 2µJNν∞ + ν2

∞J2N2 + σ2δ(t− t′) + J2Nν∞δ(t− t′) (3.441)

Finally, we equate the right sides of (3.429) and (3.441), integrate from t− ǫ to t+ ǫ, and get

σ2 = σ2 + J2Nν∞. (3.442)

This completes the proof of (3.422).

102

The relationship between the population firing rate and the determinitic firing rate.

Our first goal in this section is to show that

νN (t) ≈ ν(t) +

√

ν(t)

Nζ(t) when N >> 1. (3.443)

where νN (t) denotes the population firing rate, ν(t) = ν∞(t) is the deterministic firing rate and ζ(t)

is white noise.

Our second goal is to prove that

Prob 1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t when ∆t > 0 is small. (3.444)

I. To prove (3.443) we use the Central Limit Theorem. Let Y1, Y2, .., YN be identically distributed

random variables ( i.e. they all have the same probability distribution function), with

E(Yi) = µ and V ar(Yi) = σ2, 1 ≤ i ≤ N. (3.445)

The Central Limit Theorem states that∑N

i=1 Yi −Nµ

σ√N

≈ N(0, 1) when N >> 1. (3.446)

Next, recall that for fixed i ∈ [1,N],

Xi(t) = the number of spikes emitted by cell i in (0, t). (3.447)

Thus,

∆Xi(t) = Xi(t+ ∆t) −Xi(t) = the number of spikes emitted by cell i in (t, t+ ∆t). (3.448)

For fixed t and small ∆t we assume that ∆Xi(t) is a Poisson random variable parameter ν(t). Then

E (∆Xi(t)) = V ar (∆Xi(t)) = ν(t)∆t, 1 ≤ i ≤ N. (3.449)

To use the Central Limit Theorem we set Yi = ∆Xi(t), µ = ν(t)∆t and σ =√

ν(t)∆t in (3.446) and

conclude that∑N

i (∆Xi) −Nν(t)∆t√

ν(t)∆t√N

≈ N(0, 1) when N >> 1. (3.450)


N∑

i

(∆Xi) −Nν(t)∆t ≈√

ν(t)∆t√NN(0, 1) when N >> 1. (3.451)

Dividing both sides of (3.452) by N∆t gives

1

N

N∑

i

(

∆Xi

∆t

)

≈ ν(t) +

√

ν(t)

N∆tN(0, 1) when N >> 1. (3.452)

103

The population firing rate νN (t) satisfies

νN (t) =1

N

N∑

i=1

dXi

dt≈ 1

N

N∑

i

(

∆Xi

∆t

)

. (3.453)


νN (t) ≈ ν(t) +

√

ν(t)

N∆tN(0, 1) when N >> 1. (3.454)

To link this with white noise ζ(t) we make use of the property

ζ(t) ≈ ∆W

∆t=N(0, 1)√

∆t(3.455)

Combining( 3.454) and (3.455) gives

νN (t) ≈ ν(t) +

√

ν(t)

Nζ(t) when N >> 1. (3.456)

II. Our second goal is to show that

Prob1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t. (3.457)

First, observe that

Prob1 spike occurs in(t, t+ ∆t) = Prob Xi(t+ ∆t) −Xi(t) = 1 . (3.458)

From the assumption that ∆Xi(t) is a Poisson random variable with parameter ν(t) it follows that

Prob 1 spike occurs in(t, t+ ∆t) = Prob∆Xi(t) = 1 = e−ν(t)∆tν(t)∆t. (3.459)

Since e−ν(t)∆tν(t)∆t ≈ ν(t)∆t when ∆t > 0 is small, we conclude that

Prob1 spike occurs in(t, t+ ∆t) ≈ ν(t)∆t. (3.460)

104

How to solve the SDE problem dVdt = f(V ) + Jδ(t− T ) + σζ(t), V (0) = V0 ?

Goal: develop an Euler numerical scheme to solve

dV

dt= f(V ) + Jδ(t− T ) + σζ(t), V (0) = V0, (3.461)

where f(V ) is a function of V, J is a real parameter which can be positive, negative or zero, σ ≥ 0 is

the standard deviation of the white noise term, and V0 is the initial value of the solution.

The numerical Euler scheme to solve (3.461) is

V (tk+1) = V (tk) + f(V (tk))dt + Jδ(tk − T )dt+ σrandn√dt, V (t1) = V0. (3.462)

To implement (3.462) the first step is to define the total integration time and step size, e.g.

totaltime = 10 and dt = .001 (3.463)

Next, define K, the number of integration steps, by

K = totaltime/dt = 10/.001 = 10000 (3.464)

Define the integation times by

tk+1 = tk + dt, k = 1, 2, ...,K − 1 (3.465)

In the theoretical solution we have

∫ tk+1

tk

δ(t− T )dt =

1 if T ∈ [tk, tk+1),

0 if T /∈ [tk, tk+1).(3.466)

In order that the numerical solution be consistent with this property we set

δ(tk − T ) =

1dt if T ∈ [tk, tk+1),

0 if T /∈ [tk, tk+1).(3.467)

From this we obtain the approximation

∫ tk+1

tk

δ(t− T )dt ≈ δ(tk − T )dt =

(

1dt

)

dt = 1 if T ∈ [tk, tk+1),

0 if T /∈ [tk, tk+1).(3.468)

Combining (3.462) with (3.468) gives V (t1) = V0 and

V (tk+1) =

V (tk) + f(V (tk))dt+ J + σrandn√dt, if T ∈ [tk, tk+1),

V (tk) + f(V (tk))dt+ σrandn√dt if T /∈ [tk, tk+1).

(3.469)

Remark. It follows from (3.469) that the term δ(t− T ) in the model provides a ‘kick’ of magnitude

J (positive, negative or zero) to the solution at t = T, i.e when T ∈ [tk, tk+1].

105

How to solve the SDE problem dVdt = f(V ) + J

∑

j δ(t− Tj) + σζ(t), V (0) = V0 ?

Goal: develop an Euler numerical scheme to solve

dV

dt= f(V ) + J

∑

j

δ(t− Tj) + σζ(t), V (0) = V0, (3.470)

where f(V ) is a function of V, J is a real parameter, σ ≥ 0 is the standard deviation of the white

noise term, and V0 is the initial value of the solution. The term

∑

j

δ(t− Tj) (3.471)

is a spike train. In the simplest setting it is assumed that the spike train is externally generated, with

Tj > 0 ∀j ≥ 1. To accurately describe the numerical solution we define the set of spike times

S = T1, T2, ... (3.472)

The numerical Euler scheme to solve (3.461) is

V (tk+1) = V (tk) + f(V (tk))dt+ J

∑

j

δ(tk − Tj)

dt+ σrandn√dt, V (t1) = V0. (3.473)

To implement (3.473) the first step is to define the total integration time, the step size dt, and K, the

total number of time steps. For example,

totaltime = 10, dt = .001, K = totaltime/dt = 10000 (3.474)

Define the integation times by

tk+1 = tk + dt, k = 1, 2, ...,K − 1 (3.475)

In the theoretical solution we have

∫ tk+1

tk

δ(t− Tj)dt =

1 if Tj ∈ [tk, tk+1),

0 if Tj /∈ [tk, tk+1).(3.476)

In order that the numerical solution (3.473) be consistent with this property we set

δ(Tk − T ) =

1dt if Tj ∈ [tk, tk+1),

0 if Tj /∈ [tk, tk+1).(3.477)

From this we obtain the approximation

∫ tk+1

tk

δ(t− Tj)dt ≈ δ(tk − Tj)dt =

(

1dt

)

dt = 1 if Tj ∈ [tk, tk+1),

0 if Tj /∈ [tk, tk+1).(3.478)

106

Observe that over a given subinterval [tk, tk+1) it is possible that several spikes lie in [tk, tk+1), and

that each such spike will provide an extra ‘kick’ to the solution. Thus, we define

γk = the number of spikes in S which lie in [tk, tk+1), k ≥ 1. (3.479)

Note that either γk = 0 (i.e. no spikes lie in [tk, tk+1)), or else γk > 0 (i.e. γk > 0 spikes lie in

[tk, tk+1)). To account for these possibilities we set

∑

j

δ(tk − Tj)

dt = γk, k ≥ 1. (3.480)

Combining (3.473) with (3.480) gives V (t1) = V0 and

V (tk+1) =

V (tk) + f(V (tk))dt + Jγk + σrandn√dt, if γk > 0,

V (tk) + f(V (tk))dt + σrandn√dt if γk = 0.

(3.481)

Remarks.

(i) It follows from (3.481) that the term J∑

j δ(t−Tj) in the model provides a ‘kick’ of magnitude Jγk

to the solution. Note that the kick can be either positive (J > 0), negative (J < 0), or zero (J = 0).

(ii) At the k-th step it is necessary to do a search of the set S of spike times to precisely determine

γk, the number of spikes that lie in [tk, tk+1).

107

The feed forward model of Doiren et al [6]

In layer I it is assumed that there N cells, they re statistically identical and uncorellated (i.e. indep-

nedent). The tansmembrane potential of each cell is modelled by the same linear integrate and fire

SDEdV

dt= µ(t) + σ(t)

dζ

dt, V (0) = 0, (3.482)

0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞), (3.483)

with reset condition

V (t−) = vT = 0 and V (t+) = vR = 1. (3.484)

The system receives a step input after a time t∗ when the firing rate has become stationary. Each cell

generates a spike train

yi(t) =∑

j

δ(t− tij). (3.485)

They compute the population firing rate νN (t), and also the deterministic firing rate ν∞.

In layer II it is assumed that the cells are uncorellated (i.e. independent). They are assumed to receive

input from the cells in layer I, and also from noise in their surroundings. The tansmembrane potential

of cell j in layer II is modelled by

dVj

dt= µ+ J

N∑

i=1

Kijyi(t) + σdζ

dt, V (0) = 0, (3.486)

where yi(t) denotes the spike train generated by cell i in layer I, J > 0 if the cells in layer I are

excitatory, and J < 0 if the cells in layer I are inhibitory. A typical values of J is

J =J

Nwhere J = 1, 2 or 3. (3.487)

The connectivity matrix is assumed to be sparse and is defined by

Kij =

1 if cell i is connected to cell j,

0 if cell i is not connected to cell j.(3.488)

Anne Marie’s data [24] indicates that

Prob(Kij = 1) = .15,

Prob(Kij = 0) = .85(3.489)

Therefore .15 of the entries of Kij are ones and .85 of the entries of Kij are zeros. Define

p = Prob (cell i is connected to cell j) , (3.490)

108

and

Kj =N∑

i=1

Kij . (3.491)

Then

E(Kj) = Np = the expected number of cells connected to cell j (3.492)

Define

C = Np = the expected number of cells connected to cell j (3.493)

For example, p = .15 according to Anne Marie’s data. This, if a N=5000 then

C = .15 ∗ 5000 = 750 = the expected number of cells connected to cell j (3.494)

Note that Kij is a 5000 by 5000 sparse matrix a 750 by 750 submatrix of ones, and the rest of the

entries are zeros.

109

How to compute the spectrum in Figure 3 of Mattia et al.

We solve the linear integrate and fire SDE

dV

dt= µ(t) + σ(t)

dζ

dt, V (0) = 0, (3.495)

0 ≤ V (t) ≤ vT = 1 ∀t ∈ [0,∞), (3.496)

with reset condition

V (t−) = vT = 0 and V (t+) = vR = 1. (3.497)

The parameters are

µ = 20, σ = θ = 1. (3.498)

To compute the spectrum we consider N=1000 cells, and for each of these cells solve (3.496)-(3.496)-

(3.497) over 0 ≤ t ≤ 60 seconds. Since this is the drift dominant setting we should get about 20

spikes per second times 20 seconds, which gives approximately 400 spikes per cell. Because we have

1000 cells we should end up with about 400,000spikes. During each run only keep the spikes after a

transient period of about 500 msec during which ρ(V, t|0, 0) settles into the stationary state ρS(V ),

and the the firing rate function ν(t) ≈ νS ≈ 20.

First example. Figure 26 demonstrates the ringing phenomenon when we do this numerical experiment

over a 1 second time interval, for two values of σ, namely σ = 1( left panel) and σ = .5 (right panel).

Here we keep all spikes bcause we want to see the transient period behavior.

0 100 200 4000

30

60

t (msec)

νN µ=20, θ

N=1000 cells

=1, σ=1

0 100 200 4000

30

60

t (msec)

νN

N=1000 cells

µ=20, θ=1, σ=.5

Figure 26: Left: ringing (i.e. oscillations) in the population firing rate function for parameter

values µ = 20, σ = 1, θ = 1 and N = 1000 cells. Right: the ringing phenomenon becomes

more pronounced when σ is lowered to σ = .5 The matlab code to obtain these diagrams is at

www.math.pitt.edu/∼troy/lif/mattiafigure3/mattiafigure3ring.m.m

110

4 The Wiener Process W (t).

(a.) The conditional, single and joint probability distributions of W (t).

The random process W (t), t ≥ 0 is a Wiener process (Schaum’s, p.172) if

(1.) W (0) = 0, E(W (t)) = 0 ∀t ≥ 0, and W(t) has stationary independent increments.

(2.) When t > s the increment W (t) −W (s) is normally distributed.

Implications. The assumption that W (t) has stationary, independent increments means that if t > s

then W (t)−W (s) has the same cdf and pdf as W (t+h)−W (s+h) ∀h (see [26], pp. 163-164). Setting

h = −s, we conclude, since W (0) = 0, that W (t) −W (s) and W (t− s) have the same cdf, i.e.

P (W (t) −W (s) ≤ w) = P (W (t− s) ≤ w) =

∫ w

−∞

1√

2π(t− s)e−

η2

2(t−s)dη, (4.1)

and a differentiaiton of both sides of (4.1) with respect to w gives the pdf

ρ(w, t− s) =1

√

2π(t− s)e−

w2

2(t−s) . (4.2)

Our goal here is to show how to derive the distribution (4.2) as well as the joint pdf ρ(w, t;w′, t′).

Step I. Assume that ρ(w, 0) = δ(w), and that the conditional probability density function is

ρ(w, t|w′, s) =1

√

2π(t− s)e−

(w−w′)22(t−s) , t > s. (4.3)

Step II. To derive the single time distribution ρ(w, t) we follow the fundamental principle of beginning

with an integral equation representation for ρ(w, t) :

ρ(w, t) =

∫ ∞

−∞ρ(w, t;w′, 0)dw′. (4.4)

Next, substitute ρ(w, t;w′, 0) = ρ(w, t|w′, 0)ρ(w′, 0) into the right side of (4.4) and get

ρ(w, t) =

∫ ∞

−∞ρ(w, t|w′, 0)ρ(w′, 0)dw′ =

∫ ∞

−∞ρ(w, t|w′, 0)δ(w′)dw′. (4.5)

From (4.3), the definition of the delta function and (4.5) we obtain

ρ(w, t) =

∫ ∞

−∞

1√2πt

e−(w−w′)2

2t δ(w′)dw′ =1√2πt

e−w2

2t . (4.6)

Replace t with t − s in (4.6) to get (4.2). Finally, we obtain the joint probability distribution

ρ(w, t;w′, t′) by setting

ρ(w, t;w′, t′) = ρ(w, t|w′, t′)ρ(w′, t′) =1

√

2π(t− t′)e− (w−w′)2

2(t−t′)1√2πt′

e−w′22t′ (4.7)

Note: W (t) is not stationary since ρ(w, t, w′, s) 6= ρ(w, t− s;w′, 0).

For the N-variable case, suppose that t1 < t2.. < tN . Then the n-variable joint distribution is

ρ(wN , tN , wN−1, tN−1, .., w1, t1) =

(

2∏

i=1

1√

2π(ti − ti−1)e− (wi−wi−1)2

2(ti−ti−1)

)

1√2πt1

e−w2

12t1 (4.8)

111

Mean and variance. From (4.6) it follows that the mean and variance of W (t) are

E(W (t)) =

∫ ∞

−∞

w√2πt

e−w2

2t dw = 0 and E(W 2(t)) =

∫ ∞

−∞

w2

√2πt

e−w2

2t dw = t ∀t ≥ 0. (4.9)

(b.) Important properties: ρ(w, 0) = δ(w) and ρ(w′, t|w, t) = δ(w′ − w).

Both of these properties are used later in the derivation of the Fokker-Planck equation.

Let g(w) be an L1 function on (−∞,∞). Then the delta function δ(x) satisfies

∫ ∞

−∞g(w)δ(w)dw = g(0). (4.10)

Thus, to show that ρ(w, 0) = δ(w), we need to prove that

∫ ∞

−∞g(w)ρ(w, 0)dw = g(0), (4.11)

First, observe that∫ ∞

−∞g(w)ρ(w, t)dw =

∫ ∞

−∞g(w)

1√2πt

e−w2

2t dw. (4.12)

Let w = t1/2y. Then (4.12) becomes

∫ ∞

−∞g(w)ρ(w, t)dw =

1√2π

∫ ∞

−∞g(√ty)e−

y2

2 dy. (4.13)

Next, we let t→ 0+ on both sides of (4.13) and conclude that

∫ ∞

−∞g(w)ρ(w, 0)dw =

1√2π

∫ ∞

−∞g(0)e−

y2

2 dy = g(0)1√2π

∫ ∞

−∞e−

y2

2 dy = g(0), (4.14)

since 1√2π

∫∞−∞ e−

y2

2 dy = 1. Thus, we have shown that

∫ ∞

−∞g(w)ρ(w, 0)dw = g(0), (4.15)

From this and (4.10) it follows that ρ(w, 0) = δ(w).

Proof that ρ(w′, t|w, t) = δ(w′ − w). The function δ(w′ − w) satisfies

∫ ∞

−∞g(w′)δ(w′ − w)dw′ = g(w). (4.16)

Thus, to show that ρ(w′, t|w, t) = δ(w′ − w), it suffices to prove that

∫ ∞

−∞g(w′)ρ(w′, t|w, t)dw′ = g(w). (4.17)

Proceeding as in part (b.), we observe that

∫ ∞

−∞g(w′)ρ(w′, t|w, s)dw′ =

∫ ∞

−∞g(w′)

1√

2π(t− s)e−

(w′−w)2

2(t−s) dw′, t > s. (4.18)

112

Let w′ = w + y√t− s. Then (4.18) becomes

∫ ∞

−∞g(w′)ρ(w′, t|w, s)dw′ =

1√2π

∫ ∞

−∞g(w + y

√t− s)e−

y2

2 dy. (4.19)

Next, we let s→ t− on both sides of (4.19), and conclude that

∫ ∞

−∞g(w′)ρ(w′, t|w, t)dw′ =

1√2π

∫ ∞

−∞g(w)e−

y2

2 dy = g(w)1√2π

∫ ∞

−∞e−

y2

2 dy = g(w), (4.20)

since 1√2π

∫∞−∞ e−

y2

2 dy = 1. Thus, we have shown that (4.17) holds, as required.

Examples. Doering makes use (p. 20, eq. (4.8)) of ρ(x′, t|x, t) = δ(x′ − x) to obtain

∫ ∞

−∞x′ρ(x′, t|x, t)dx′ =

∫ ∞

−∞x′δ(x′ − x)dx′ = x, (4.21)

∫ ∞

−∞f(x′, t)∆tρ(x′, t|x, t)dx′ =

∫ ∞

−∞f(x′, t)∆tδ(x′ − x)dx′ = f(x, t)∆t. (4.22)

(c.) Proof that ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ wFirst we recall from above how to find the pdf for general Z = g(X,Y ). For a given z ∈ R define

DZ(z) = (x, y)|g(x, y) ≤ z. (4.23)

Let f(x, y) be the joint pdf for X and Y. Then the cdf FZ(z) is defined by

FZ(z) =

∫ ∫

DZ(z)

f(x, y)dydx. (4.24)

Example I. If Z = Y +X then

DZ(z) = (x, y)|y ≤ z − x (4.25)

and

FZ(z) =

∫ ∫

DZ (z)

f(x, y)dydx =

∫ ∞

−∞

∫ z−x

−∞f(x, y)dydx. (4.26)

The corresponding pdf for Z is the convolution

fZ(z) =d

dzFZ(z) =

∫ ∞

−∞f(x, z − x)dydx. (4.27)

Example II. If Z = Y −X then the pdf for Z is

fZ(z) =d

dzFZ(z) =

∫ ∞

−∞f(x, z + x)dydx. (4.28)

Example III. Suppose that Y and X are functions of t and that Z = Y (t2) −X(t1). Then

fZ(z) =d

dz

∫ ∞

−∞

∫ z+x

−∞f(x, t1, y, t2)dydx =

∫ ∞

−∞f(x, t1, z + x, t2)dx. (4.29)

113

Now let z = w, Y (t) = W (t+ h) and X(t) = W (t). Then t2 = t+ h, t1 = t, and we get

fZ(w) =d

dw

∫ ∞

−∞

∫ w+w1

−∞ρ(w1, t+ h,w2, t)dw2dw1 =

∫ ∞

−∞ρ(w1, t, w + w1, t+ h)dw1. (4.30)

Thus, since ρ(w1, t, w + w1, t+ h) = ρ(w + w1, t+ h,w1, t), we have

fZ(w) =

∫ ∞

−∞ρ(w + w1, t+ h,w1, t)dw1. (4.31)

where ρ(w + w1, t+ h,w1, t) is the joint pdf for the Wiener process, and

ρ(w + w1, t+ h,w1, t) = ρ(w + w1, t+ h|w1, t)ρ(w1, t). (4.32)

Recall that, if t′ > t′′, then

ρ(w′, t′|w′′, t′′) =1

√

2π(t′ − t′′)e− (w′′−w′)2

2(t′−t′′) , and ρ(w′, t′) =1√2πt′

e−(w′)22t′ . (4.33)


ρ(w + w1, t+ h,w1, t) =1√2πh

e−(w)2

2h1√2πt

e−(w1)2

2t . (4.34)


fZ(w) =d

dwProbW (t+ h) −W (t) ≤ w =

∫ ∞

−∞

1√2πh

e−(w)2

2h1√2πt

e−(w1)2

2t =1√2πh

e−(w)2

2h . (4.35)

Finally, since W (0) = 0, we note that

d

dwProbW (h) −W (0) ≤ w =

d

dwProbW (h) ≤ w =

d

dw

∫ w

−∞

1√2πh

e−(w)2

2h =1√2πh

e−(w)2

2h .

(4.36)

Thus, we conclude that

ProbW (t+ h) −W (t) ≤ w = ProbW (h) −W (0) ≤ w, (4.37)

and the proof is complete.

(d.) Proof that E(W (t)W (s)) = min(t, s).

First, recall that if X is a normal random variable then its probability density is

fX(x) =1√2πσ

e−(x−µ)2

2σ2 dx, (4.38)

where the mean, µ, and variance σ2, satisfy

E(X) = µ and E(X2) − [E(X)]2 = σ2. (4.39)

Let t > s. It follows from (4.7) that the autocorellation E(W (t)W (s)) is

E (W (t)W (s)) =

∫ ∞

−∞

∫ ∞

−∞ww′ 1

√

2π(t− s)e−

(w−w′)22(t−s)

1√2πs

e−w′22s dwdw′. (4.40)

114

Rearranging terms gives

E (W (t)W (s)) =

∫ ∞

−∞

1√2πs

w′e−w′22s

(

∫ ∞

−∞w

1√

2π(t− s)e−

(w−w′)22(t−s) dw

)

dw′. (4.41)

Using (4.38)-(4.39), we see that the inner integral reduces to

∫ ∞

−∞w

1√

2π(t− s)e−

(w−w′)22(t−s) dw = w′. (4.42)

Combining (4.41), (4.42) and a second application of (4.38)-(4.39) gives

E (W (t)W (s)) =1√2πs

∫ ∞

−∞w′2e−

w′22s dw′ = s = min(t, s). (4.43)

(e.) Proof that W (t) has independent increments.

We need to prove that

E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = E (W (t2) −W (t1))E (W (t1) −W (t0)) , 0 ≤ t0 < t1 < t2.

(4.44)

Expanding the left side of (4.44), and using the property E(W (t)W (s)) = min(t, s), we obtain

E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = E(W (t2)W (t1))−E(W 2(t1))−E(W (t2)W (t0))+E(W (t1)W (t0))

(4.45)

This reduces to

E ([W (t2) −W (t1)][(W (t1) −W (t0)]) = t1 − t1 + t0 − t0 = 0. (4.46)

Similarly, an expansion of the right side of (4.44) gives

E (W (t2) −W (t1))E (W (t1) −W (t0)) = (E(W (t2)) − E(W (t1))) (E(W (t1)) − E(W (t0))) = 0,

(4.47)

since E(W (t)) = 0 ∀t ≥ 0.

(f.) Proof that W (t) −W (s) ∼√t− sN(0, 1) when t > s.

This property is at the core of why matlab uses dW = (√

∆t)(randn) in the solution of stochastic

ode’s. We need to prove that the cdf’s P (W (t) −W (s) ≤ w) and P (√t− sX ≤ w) have the same

pdf, where X = N(0, 1) denotes the normal random variable. These cdf’s have the same pdf if

d

dwP (

√t− sX ≤ w) =

d

dwP (

√t− sX ≤ w) =

1√

2π(t− s)e−

w2

2(t−s) . (4.48)

First, we know from (4.1) that

P (W (t) −W (s) ≤ w) =

∫ w

−∞

1√

2π(t− s)e−

η2

2(t−s) dη, (4.49)

115

henced

dwP (W (t) −W (s) ≤ w) =

1√

2π(t− s)e−

w2

2(t−s). (4.50)

Next,

P (√t− sX ≤ w) = P

(

X ≤ w√t− s

)

=

∫ w√t−s

−∞

1√2πe−

η2

2 dη, (4.51)

and therefored

dwP (

√t− sX ≤ w) =

1√

2π(t− s)e−

w2

2(t−s). (4.52)

Thus, (4.48) now follows from (4.50) and (4.52) and the proof is complete.

(g.) W (t) is m.s. continuous, and continuous in probability at t.

Definition A random process X(t) is mean square continuous at t if

limh→0

E([X(t+ h) −X(t)]2) = 0. (4.53)

We now show that W (t) is mean square continuous at t. First, because of the linearity of E, we have

E(

[W (t+ ∆t) −W (t)]2)

= E(

W 2(t+ ∆t))

− 2E (W (t+ ∆t)W (t)) + E(

W 2(t))

. (4.54)

This, and (4.43), imply that

E(

[W (t+ ∆t) −W (t)]2)

= t+ ∆t− 2t+ t = ∆t. (4.55)

From (4.55) we conclude that

lim∆t→0

E(

[W (t+ ∆t) −W (t)]2)

= 0, (4.56)

hence W (t) is continuous in mean square at t.

In Section 1 (part h) it was proved that a randon process that is mean square continuous at t is also

continuous in probability at t. Thus, W (t) is continuous in probability at t.

(h.) Proof that W (t) does not have a mean square first derivative.

Recall from part (j.), property 4 of the previous section that if a random process X(t) has a mean

square first derivative then RX′(t, s) = ∂2RX (t,s)∂t∂s ∀t, s ≥ 0. Therefore, to prove that W (t) does not

have a mean square derivative, we’ll prove that

∂2RW (t, s)

∂t∂s|s=t does not exist. (4.57)

First, recall that the autocorrelation function for the Wiener process satisfies For this start by recalling

that

RW (t, s) = σ2min(t, s) (4.58)

116


lim∆s→0RW (t, s+ ∆s) −RW (t, s)

∆s= H(t− s), (4.59)

Where H is the heaviside function. Therefore

∂2

∂s∂tRW (t, s) does not exist. (4.60)

This completes the proof that W (t) does not have a mean square first derivative.

(i.) The generalized first derivative of W satisfies RW ′ (t, s) = δ(t− s)

In the last section it was proved that

lim∆s→0RW (t, s+ ∆s) −RW (t, s)

∆s= H(t− s), (4.61)

Thus (handwaving, admittedly), if W is a Wiener process with parameter σ2, then

RW ′(t, s) = E(W ′(t)W ′(s)) =∂2

∂s∂tE(W (t)W (s)) =

∂2

∂s∂tRW (t, s) = σ2δ(t− s). (4.62)

(j.) Properties of the mean square integral Y =∫ t

t0W (α)dα.

Definition. Recall from Section 1, part (k) that the mean square integral

∫ t

t0

X(α)dα exists ⇐⇒∫ t

t0

∫ t

t0

RX(α, β)dαdβ exists. (4.63)

The Wiener process with parameter σ has autocorellation function RW (t, s) = σ2min(t, s), and it

follows that∫ t

0

∫ t

0RW (α, β)dαdβ exists. Thus, Y (t) =

∫ t

0W (α)dα exists.

Property 1. The mean of Y (t) is

µY (t) = E (Y (t)) =

∫ t

0

E (W (α)) dα = 0. (4.64)

Property 2. The variance of Y (t) is

V ar(Y (t)) = E(

Y 2(t))

=∫ t

0

∫ t

0 E (W (α)W (β)) dαdβ =∫ t

0

∫ t

0 (min(α, β) dαdβ

=∫ t

0

(

∫ β

0αdα+

∫ t

ββdα

)

dβ =∫ t

0β2

2 dβ.+∫ t

ββ(t− β)dβ = σ2t3

3 .(4.65)

Property 3. Let t > s. Then the autocorellation function RY (t)(t, s) satisfies

RY (t)(t, s) =1

6σ2s2(3t− s). (4.66)

The proof follows Problem 6.12, p. 224 of Schaum’s.The big insight is to write Y (t) as

Y (t) =

∫ s

0

W (α)dα +

∫ t

s

W (α)dα = Y (s) +

∫ t

s

(W (α) −W (s)) dα+ (t− s)W (s). (4.67)

117

He does this to make use of the ”independent increment ” property of W (t). Next,

RY (t)(t, s) = E(

Y 2(s))

+∫ t

sE (W (α) −W (s))Y (s)dα + (t− s)E(W (s)Y (s))

= σ2s3

3 +∫ t

s E(

[W (α) −W (s)]∫ s

0 W (β))

dβdα+ (t− s)∫ s

0 E(W (s)W (β))dβ

= σ2s3

3 +∫ t

s

∫ s

0E ([W (α) −W (s)]W (β)) dβdα + σ2(t− s)

∫ s

0min(s, β)dβ

= σ2s3

3 +∫ t

s

∫ s

0 E ([W (α) −W (s)][W (β) −W (0)]) dβdα+ σ2(t− s)∫ s

0 βdβ

= σ2s3

3 +∫ t

s

∫ s

0E (W (α) −W (s))E (W (β) −W (0)) dβdα+ σ2(t− s) s2

2 ,

(4.68)

since 0 < β < s < α implies that W (α)−W (s) and W (β) −W (0) are independent. The definition of

W (t) guarantees that E(W (α) −W (s)) = E(W (β) −W (0)) = 0, hence (4.68) reduces to

RY (t)(t, s) =σ2s3

3+ σ2(t− s)

s2

2=σ2s2

6(3t− s). (4.69)

(k.) Y (t) =∫ t

0 X(η)dη is a Wiener process if X(t) is normal white noise.

To prove this result we follow Problem 6.22, p. 230 of Schaum’s. Let Y (t) =∫ t

0X(η)dη, where

X(t) is a normal white noise process. Then Y (t) is also a normal process (see, Problem 6.10, p. 223

of Schaum’s). Therefore, to conclude that Y (t) is a Wiener process we need to verify that their

autocorellation functions are equal, i.e. RY (t, s) = RW (t, s) = min(t, s). We start with

RY (t, s) = σ2

∫ t

0

∫ s

0

RX(α, β)dβdα = σ2

∫ t

0

∫ s

0

δ(α− β)dαdβ. (4.70)

Note that if z > 0 then

σ2

∫ u

0

δ(α− z)dα = σ2H(u− z), (4.71)

where H is the Heaviside function. Combining (4.70) and (4.71) shows that (4.70) reduces to

RY (t, s) = σ2

∫ t

0

H(s− β)dβ. (4.72)

Now consider two cases: if s > t then s > β, H(s− β) = 1 and (4.70) becomes

RY (t, s) = σ2

∫ t

0

dβ = σ2t = min(t, s). (4.73)

Next, suppose that s < t. Then

RY (t, s) = σ2

∫ s

0

H(s− β)dβ + σ2

∫ t

s

H(s− β)dβ = σ2

∫ s

0

H(s− β)dβ = σ2s = min(t, s). (4.74)

The case s = t is similar. Thus, we conclude that RY (t, s) = min(t, s) = RW (t, s), as required.

(l.) Scaling: dWτ =√ǫdWt if τ = ǫt. (i.e. they have the same pdf’s)

Proof: First, define

dWτ = W (τ + ∆τ) −W (τ) and dWt = W (t+ ∆t) −W (t). (4.75)

118

Then

P (dWτ ≤ w) = P (W (τ + ∆τ) −W (τ) ≤ w) (4.76)

Substitute τ = ǫt anad ∆t = ǫ∆t into (4.76) and get

P (W (τ + ∆τ) −W (τ) ≤ w) = P (W (ǫt+ ǫ∆t) −W (ǫt) ≤ w) (4.77)

The stationary property implies that the right side reduces to

P (W (ǫt+ ǫ∆t) −W (ǫt) ≤ w) = P (W (ǫ∆t) −W (0) ≤ w) = P (W (ǫ∆t) ≤ w) (4.78)

Next, prove that

P (W (ǫ∆t) ≤ w) = P(√ǫW (∆t) ≤ w

)

(4.79)

First, note that

P (W (ǫ∆t) ≤ w) =

∫ w

−∞

1√2πǫ∆t

exp(− x2

2ǫ∆t)dx (4.80)

Now let x = u√ǫ and transform (4.80) into

P (W (ǫ∆t) ≤ w) =

∫ w√ǫ

−∞

1√2π∆t

exp(− u2

2∆t)du (4.81)

Thus,

P (W (ǫ∆t) ≤ w) = P

(

W (∆t) ≤ w√ǫ

)

= P(√ǫW (∆t) ≤ w

)

(4.82)

Now work backwards from the right side. That is,

P(√ǫW (∆t) ≤ w

)

= P(√ǫ [W (t+ ∆t) −W (t)] ≤ w

)

= P(√ǫdWt ≤ w

)

(4.83)

This imnplies that dWτ =√ǫdWt (i.e. they have the same pdf’s).

(m.) dXt = f(Xt)dt+B(Xt)dWt becomes ǫdXτ = f(Xt)dτ +√ǫB(Xt)dWτ if τ = ǫt.

Proof. Multiply both sides of

dXt = f(Xt)dt+B(Xt)dWt (4.84)

by ǫ and get

ǫdXt = f(Xt)ǫdt+B(Xt)ǫdWt. (4.85)

From the previous section we have dWτ =√ǫdWt. Also, dτ = ǫdt. Substitute these into (4.85) and

obtain the required equation, i.e.

ǫdXτ = f(Xτ )dτ +B(Xτ )√ǫdWτ (4.86)

119

5 Doering’s derivation of the Fokker-Planck equation

(a.) The Stochastic ODE.

The Langevin stochastic differential equation for a first order Markov process is

dX(t)

dt= f(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (5.1)

where ζ(t) is white noise. Our goal is to derive the Fokker-Planck equation (i.e. Kolmogorov’s forward

equation) for the associated conditional probability density function, namely

∂

∂tρ(x, t|y, s) = − ∂

∂x(f(x, t)ρ(x, t|y, s)) +

1

2

∂2

∂x2

(

g(x, t))2ρ(x, t|y, s))

. (5.2)

At s = t the solution must satisfy

ρ(x, s|y, s) = δ(x− y) (5.3)

Remarks. (i) The interpretation of the product ρ(x, t|y, s)dx is

ρ(x, t|y, s)dx = P (X(t) ∈ (x, x+ dx)|X(s) = y) . (5.4)

(ii) If f = 0 and g = 1 then the pde (5.2) becomes

∂

∂tρ(x, t|y, s) =

1

2

∂2

∂x2ρ(x, t|y, s), (5.5)

which is satified by the conditional probability ρ(w, t|w′, s) = 1√2π(t−s)

e−(w−w′)22(t−s) , t > s, defined above

in (4.3) for the Wiener process W (t). The transition probability density function ρ(w, t|w′, s) is the

pdf for the solution W (t) of the stochastic ODE problem

W ′ = ζ(t), t > s, such that W (s) = w′. (5.6)

(iii) Let t > s. Then the mean of g(X(t)), given that X(s) = y, is

E (g(X(t))|X(s) = y) =

∫ ∞

−∞g(x′)ρ(x′, t|y, s)dx′. (5.7)

(iv) Important example: if s = t and y = x then “the mean of g(X(t)), given thatX(s) = x, ” is

E (g(X(t))|X(t) = x) =

∫ ∞

−∞g(x′)ρ(x′, t|x, t)dx′ =

∫ ∞

−∞g(x′)δ(x′ − y)dx′ = g(x). (5.8)

(v) More generally, for any continuous random variable X(t) it seems that it ought to be true that

E (g(X(t))|X(t) = x) = g(x). (5.9)

One way to get this is

E (g(X(t))|X(t) = x) =

∫ ∞

−∞g(x)fX(x′, t)dx′ = g(x)

∫ ∞

−∞fX(x′, t)dx′ = g(x). (5.10)

120

(vi) Even more generally, it seems that if we have two random variables X(t) and Y (t) then

E (g(X(t))Y (s)|X(t) = x) = E(g(x)Y (s)) = g(x)E(Y (s)). (5.11)

(b.) Evaluation of E(∆X(t)|X(t) = x) and E(

(∆X(t))2|X(t) = x)

.

To derive (5.2) Doering’s method is to let t > s and analyze the behavior of ρ on a small interval

(t, t + ∆t). For this he assumes that X(t) = x and analyzes the behavior of solutions of the related

discrete SDE

∆X(t) = X(t+ ∆t) −X(t) = f(X(t), t)∆t+ g(X(t), t)∆W (t), X(t) = x, (5.12)

where

∆W (t) = W (t+ ∆t) −W (t). (5.13)

Goal I. Prove that

E (∆X(t)|X(t) = x) =

∫ ∞

−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′ = f(x, t)∆t. (5.14)

This result will be used in the derivation of the Fokker-Planck equation. The first step in the proof is

to use linearity to conclude that

E (∆X(t)|X(t) = x) = E (X(t+ ∆t)|X(t) = x) − E (X(t)|X(t) = x) . (5.15)

Note that

E (X(t+ ∆t)|X(t) = x) =

∫ ∞

−∞x′ρ(x′, t+ ∆t|x, t)dx′ (5.16)

and

E (X(t)|X(t) = x) =

∫ ∞

−∞x′ρ(x′, t|x, t)dx′ =

∫ ∞

−∞x′δ(x′ − x)dx′ = x. (5.17)

Now combine property (5.17) with the requirement that∫∞−∞ ρ(x′, t+ ∆t|x, t)dx′ = 1 and get

E (X(t)|X(t) = x) = x = x

∫ ∞

−∞ρ(x′, t+ ∆t|x, t)dx′ =

∫ ∞

−∞xρ(x′, t+ ∆t|x, t)dx′. (5.18)

Finally, from (5.15), (5.16) and (5.18) it follows that

E (∆X(t)|X(t) = x) =

∫ ∞

−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′. (5.19)

The next step is to substitute the right side of (5.12) into E (∆X(t)|X(t) = x) for ∆X(t), which gives

E (∆X(t)|X(t) = x) = E (f(X(t), t)∆t|X(t) = x) + E (g(X(t), t)∆W (t)|X(t) = x) . (5.20)

The first term on the right side of (5.20) becomes

E (f(X(t), t)∆t|X(t) = x) =

∫ ∞

−∞f(x′, t)∆tρ(x′, t|x, t)dx′ =

∫ ∞

−∞f(x′, t)∆tδ(x′ − x)dx′ = f(x, t)∆t,

(5.21)

121

again because of (5.8). Next, since we are given that X(t) has the fixed value X(t) = x, then it seems

to make sense to conclude that X(t) and g(X(t)) do not depend on any other random variable, in

particular ∆W = W (t+ ∆t)−W (t). Also, ∆W is computed without the use of knowledge of X(t) or

g(X(t)). Therefore, it seems to make sense that ∆W and g(X(t)) are independent and that

E (g(X(t), t)∆W (t)|X(t) = x) = E (g(X(t), t)|X(t) = x)E (∆W (t)|X(t) = x) = 0. (5.22)

Here, we use the property

E (∆W |X(t) = x) = E (∆W ) = E(W (t+ ∆t)) − E(W (t)) = 0, (5.23)

since E(W (t)) = E(W (t+ ∆t)) = 0. Thus, it follows from (5.20), (5.21) and (5.22) that

E (∆X(t)|X(t) = x) = f(x, t)∆t. (5.24)

Finally, we combine (5.19) with (5.24) and obtain∫ ∞

−∞(x′ − x)ρ(x′, t+ ∆t|x, t)dx′ = f(x, t)∆t. (5.25)

This completes the proof of (5.14).

Goal II. Prove that

E(

(∆X(t))2|X(t) = x)

=

∫ ∞

−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′ = g2(x, t)∆t + f2(x, t) (∆t)

2. (5.26)

This result will be used in the derivation of the Fokker-Planck equation.

The first step in the proof is to observe that

E(

(∆X(t))2|X(t) = x)

= E(

X2(t+ ∆t)|X(t) = x)

−2E (X(t+ ∆t)X(t)|X(t) = x)+E(

X2(t)|X(t) = x)

.

(5.27)

This translates into the integral form

E(

(∆X(t))2|X(t) = x)

=

∫ ∞

−∞(x′)2ρ(x′, t+∆t|x, t)dx′−2

∫ ∞

−∞xx′ρ(x′, t+∆t|x, t)dx′+

∫ ∞

−∞(x′)2ρ(x′, t|x, t)dx′

(5.28)

The second integral has x in the integrand since X(t) = x is given. Using (5.8), we conclude that

∫ ∞

−∞(x′)2ρ(x′, t|x, t)dx′ =

∫ ∞

−∞(x′)2δ(x′ − x)dx′ = x2 (5.29)

Because∫∞−∞ ρ(x′, t+ ∆t|x, t)dx′ = 1, it follows that

x2 = x2

∫ ∞

−∞ρ(x′, t+ ∆t|x, t)dx′ =

∫ ∞

−∞x2ρ(x′, t+ ∆t|x, t)dx′. (5.30)

We combine (5.28), (5.29) and (5.30) and obtain

E(

(∆X(t))2|X(t) = x)

=

∫ ∞

−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′. (5.31)

122

Next, we proceed as in the previous case and use the ode to conclude that

E(

(∆X(t))2|X(t) = x)

= E(

[f(X(t), t)∆t+ g(X(t), t)∆W ]2 |X(t) = x

)

(5.32)

The right side of (5.32) expands to

E(

[f(X(t), t)∆t]2 |X(t) = x

)

+2E (f(X(t), t)∆tg(X(t), t)∆W |X(t) = x)+E(

[g(X(t), t)∆W ]2 |X(t) = x

)

.

(5.33)

Translating the first term to integral form, and using (5.8) gives

E(

[f(X(t), t)∆t]2 |X(t) = x

)

=

∫ ∞

−∞(f(x′, t)∆t)2ρ(x′, t|x, t)dx′ =

∫ ∞

−∞(f(x′, t)∆t)2δ(x′−x)dx′ = f2(x, t)(∆t)2.

(5.34)

The second term on the right side of (5.35) becomes

2E (f(X(t), t)∆tg(X(t), t)∆W |X(t) = x) = 2E (f(X(t), t)∆tg(X(t), t)|X(t) = x)E (∆W |X(t) = x) = 0

(5.35)

Similarly, the third term on the right side of (5.35) becomes

E(

[g(X(t), t)∆W ]2 |X(t) = x

)

= E(

[g(X(t), t)]2 |X(t) = x

)

E(

[∆W ]2)

= g2(x, t)E(

[∆W ]2)

,

(5.36)

where

E([∆W ]2) = E([W (t+ ∆t)]2) − 2E(W (t+ ∆t)W (t)) + E(W 2(t)) = t+ ∆t− 2t+ t = ∆t. (5.37)

Combining (5.320, (5.35), (5.34), (5.36) and (5.37), we obtain

E(

(∆X(t))2|X(t) = x)

= g2(x, t)∆t+ f2(x, t)[∆t]2. (5.38)

Finally, we combine (5.31) with (5.38) and obtain∫ ∞

−∞(x′ − x)2ρ(x′, t+ ∆t|x, t)dx′ = g2(x, t)∆t + f2(x, t)[∆t]2. (5.39)

We make use of this identity below in part (e.) where the Fokker-Planck equation is derived.

(c.) The Chapman-Kolmogorov equation

To derive the Chapman-Kolmogorov equation we start with

ρ(x1, t1|x3, t3) =ρ(x1, t1, x3, t3)

ρ(x3, t3)=

1

ρ(x3, t3)

∫ ∞

−∞ρ(x1, t1, x2, t2, x3, t3)dx2 (5.40)

An algebraic manipulation gives

ρ(x1, t1, x2, t2, x3, t3)

ρ(x3, t3)=ρ(x1, t1, x2, t2, x3, t3)

ρ(x2, t2, x3, t3)

ρ(x2, t2, x3, t3)

ρ(x3, t3). (5.41)

Using conditional probability notation, we write (5.41) in the form

ρ(x1, t1, x2, t2, x3, t3)

ρ(x3, t3)= ρ(x1, t1|x2, t2, x3, t3)ρ(x2, t2|x3, t3). (5.42)

123

The Markov property implies that the first term on the right side of (5.42) reduces to

ρ(x1, t1|x2, t2, x3, t3) = ρ(x1, t1|x2, t2). (5.43)

Combining (5.43) with (5.42) gives the important identity

ρ(x1, t1, x2, t2, x3, t3)

ρ(x3, t3)= ρ(x1, t1|x2, t2)ρ(x2, t2|x3, t3). (5.44)

Finally, we substitute (5.44) into (5.40) and obtain the Chapman-Kolmogorov equation

ρ(x1, t1|x3, t3) =

∫ ∞

−∞ρ(x1, t1|x2, t2)ρ(x2, t2|x3, t3)dx2. (5.45)

(d.) Use of the Chapman-Kolmogorov eq. to derive the Fokker-Planck eq.

We make use the following version of (5.45):

ρ(x, t+ ∆t|y, s) =

∫ ∞

−∞ρ(x, t+ ∆t|z, t)ρ(z, t|y, s)dz. (5.46)

Our goal is to show how to use (5.46) to derive the Fokker-Planck pde

∂

∂tρ(x, t|y, s) +

∂

∂x[f(x, t)ρ(x, t|y, s)] − ∂2

∂x2

[

g2(x, t)ρ(x, t|y, s)]

= 0. (5.47)

First, it follows from an integration of (5.46) that

∫ ∞

−∞R(x)ρ(x, t + ∆t|y, s)dx =

∫ ∞

−∞R(x)

∫ ∞

−∞ρ(x, t+ ∆t|z, t)ρ(z, t|y, s)dzdx, (5.48)

where R(x) is a C∞ function. Changing the order of integration gives∫ ∞

−∞R(x)ρ(x, t + ∆t|y, s)dx =

∫ ∞

−∞ρ(z, t|y, s)

(∫ ∞

−∞R(x)ρ(x, t + ∆t|z, t)dx

)

dz. (5.49)

Next, expanding R(x) in a power Taylor series centered at x = z gives∫ ∞

−∞ρ(x, t+∆t|y, s)dx =

∫ ∞

−∞ρ(z, t|y, s)

∫ ∞

−∞

[

R(z) +R′(z)(x− z) +R′′(z)

2(x− z)2 + ...

]

ρ(x, t+∆t|z, t)dxdz.(5.50)

Keep terms up to second order (this is WLOG), and regroup them to obtain

∫∞−∞R(x)ρ(x, t+ ∆t|y, s)dx =

∫∞−∞ ρ(z, t|y, s)R(z)dz

+∫∞−∞ ρ(z, t|y, s)R′(z)

∫∞−∞(x− z)ρ(x, t+ ∆t|z, t)dxdz

+∫∞−∞ ρ(z, t|y, s)R′′(z)

2

∫∞−∞(x − z)2ρ(x, t+ ∆t|z, t)dxdz.

(5.51)

Subtituting (5.25) and (5.39) into the right side of (5.51) reduces (5.51) to

∫∞−∞R(x)ρ(x, t + ∆t|y, s)dx =

∫∞−∞ ρ(z, t|y, s)R(z)dz

+∫∞−∞ ρ(z, t|y, s)R′(z)f(z, t)∆tdz

+∫∞−∞ ρ(z, t|y, s)R′′(z)

2

[

g2(z, t)∆t+ f2(z, t)[∆t]2]

dz.

(5.52)

124

Next, replace z with x on the right side of (5.53) and obtain

∫∞−∞R(x)ρ(x, t + ∆t|y, s)dx =

∫∞−∞ ρ(x, t|y, s)R(x)dx

+∫∞−∞ ρ(x, t|y, s)R′(x)f(x, t)∆tdx

+∫∞−∞ ρ(x, t|y, s)R′′(x)

2

[

g2(x, t)∆t+ f2(x, t)[∆t]2]

dx.

(5.53)

Put all terms on one side of the equation, divide by ∆t, and get

0 =∫∞−∞R(x)ρ(x,t+∆t|y,s)−ρ(x,t|y,s)

∆t dx−∫∞−∞ ρ(x, t|y, s)R′(x)f(x, t)dx

−∫∞−∞ ρ(x, t|y, s)R′′(x)

2

[

g2(x, t) + f2(x, t)∆t]

dx.(5.54)

Let ∆t→ 0 and this equation reduces to

0 =

∫ ∞

−∞R(x)

∂

∂tρ(x, t|y, s)dx−

∫ ∞

−∞R′(x)ρ(x, t|y, s)f(x, t)dx −

∫ ∞

−∞R′′(x)

1

2ρ(x, t|y, s)g2(x, t)dx.

(5.55)

Integrate the second and third integrals by parts and obtain

0 =

∫ ∞

−∞R(x)

∂

∂tρ(x, t|y, s)dx+

∫ ∞

−∞R(x)

∂

∂x[f(x, t)ρ(x, t|y, s)] dx−

∫ ∞

−∞R(x)

1

2

∂2

∂x2

[

g2(x, t)ρ(x, t|y, s)]

dx.

(5.56)

Combining all three integrals together gives

0 =

∫ ∞

−∞R(x)

(

∂

∂tρ(x, t|y, s) +

∂

∂x[f(x, t)ρ(x, t|y, s)] − 1

2

∂2

∂x2

[

g2(x, t)ρ(x, t|y, s)]

)

dx. (5.57)

Finally, since R(x) is arbitrary, it follows that ρ(x, t|y, s) satisfies the Fokker-Planck pde

∂

∂tρ(x, t|y, s) +

∂

∂x[f(x, t)ρ(x, t|y, s)] − 1

2

∂2

∂x2

[

g2(x, t)ρ(x, t|y, s)]

= 0 (5.58)

This completes the derivation of the Fokker-Planck equation.

(e.) Use of Ito’s Lemma to derive the Fokker-Planck eq.

Assume that the SDE for a first order Markov process X(t) satisfying X(s) = y is

dX(t)

dt= a(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (5.59)

To keep with the notation in various books, we changed f(X(t), t) to a(X(t), t) in the SDE. Our goal

is to derive the associated the conditional pdf ρ(x, t|y, s). To use Ito’s Lemma we let f ∈ C∞(−∞,∞)

be arbitrarily chosen and assume that f has compact support. Then, by Ito’s Lemma, f(X(t)) satisfies

df(X(t)) =

(

a(X(t), t)fx(X(t)) +g2(X(t), t)

2fxx(X(t))

)

dt+ g(X(t), t)fx(X(t))dW (t). (5.60)

Integrate both sides of (5.60) from s to t and get

f(X(t)) = f(x) +

∫ t

s

(

a(X(η), η)fx +g2(X(η), η)

2fxx

)

dη +

∫ t

s

g(X(η), η)fxdW (η). (5.61)

125

Next, integrate both sides of (5.61) against the conditional pdf ρ(x, t|y, s) and get

E(f(X(t)) = f(x) +

∫ t

s

E

(

a(X(η), η)fx(X(η)) +g2(X(η), η)

2fxx(X(η))

)

dη, (5.62)

where E(h(X(τ), τ))) is defined by

E(h(X(τ), τ)) =

∫ ∞

−∞h(x′, τ)ρ(x′, τ |y, s)dx′. (5.63)

Take the first derivative of both sides of (5.62) with respect to t and get

d

dtE(f(X(t))) = E(a(X(t), t)fx(X(t))) +E(

g2(X(t), t)

2fxx(X(t))). (5.64)

The left side of (5.64) reduces to

d

dtE(f(X(t))) =

d

dt

∫ ∞

−∞f(x′)ρ(x′, t|y, s)dx′ =

∫ ∞

−∞f(x′)ρt(x

′, t|y, s)dx′ (5.65)

Integration by parts shows that

E(a(X(t), t)fx(X(t))) =∫∞−∞ a(x′, t)fx(x′)ρ(x′, t|y, s)dx′

= −∫∞−∞ f(x′) ∂

∂x′ (a(x′, t)ρ(x′, t|y, s))dx′

(5.66)

andE(g2(X(t),t)

2 fxx(X(t))) =∫∞−∞

g2(x′,t)2 fxx(x′)ρ(x′, t|y, s)dx′

=∫∞−∞ f(x′) ∂2

∂x′2 (g2(x′,t)2 ρ(x′, t|y, s))dx′

(5.67)

Finally, equate the right sides of (5.65), (5.66) and (5.67) and get

∫ ∞

−∞f(x′)ρt(x

′, t|y, s)dx′ =

∫ ∞

−∞f(x′)(− ∂

∂x′(a(x′, t)ρ(x′, t|y, s)) +

∂2

∂x′2(g2(x′, t)

2ρ(x′, t|y, s)))dx′

(5.68)

Since f(x) is arbitrary then the integrands are equal, which shows that the transition density ρ(x, t|y, s)satisfies the Fokker-Planck equation

ρt(x, t|y, s) = − ∂

∂x(a(x′, t)ρ(x, t|y, s)) +

∂2

∂x2(g2(x, t)

2ρ(x, t|y, s)) (5.69)

126

6 Derivation of the Backwards Kolmogorov Equation.

The stochastic differential equation is

dX(t)

dt= f(X(t), t) + g(X(t), t)ζ(t), X(s) = y, and t > s. (6.1)

Our goal is to derive the Backwards Kolmogorov Equation

∂

∂sρ(x, t|y, s) = −f(y, s)

∂

∂yρ(x, t|y, s) − g2(y, s)

2

∂2

∂y2ρ(x, t|y, s), ρ(x, s|y, s) = δ(x− y). (6.2)

The starting point is

∂

∂sρ(x, t|y, s) = lim

∆s→0

1

∆s(ρ(x, t|y, s+ ∆s) − ρ(x, t|y, s)) . (6.3)

We use the following version of the Chapman-Kolmogorov equation (5.45):

ρ(x, t|y, s) =

∫ ∞

−∞ρ(x, t|z, s+ ∆s)ρ(z, s+ ∆s|y, s)dz. (6.4)

Substitution of (6.4) into (6.3) gives

∂


∆s→0

1

∆s

(

ρ(x, t|y, s+ ∆s) −∫ ∞

−∞ρ(x, t|z, s+ ∆s)ρ(z, s+ ∆s|y, s)dz

)

. (6.5)

Next, observe that∫ ∞

−∞ρ(z, s+ ∆s|y, s)dz = 1. (6.6)

Combining (6.5) and (6.6 ), we obtain

∂


∆s→0

∫ ∞

−∞

ρ(z, s+ ∆s|y, s)∆s

(ρ(x, t|y, s+ ∆s) − ρ(x, t|z, s+ ∆s)) dz. (6.7)

Expanding (be careful here) ρ(x, t|z, s+ ∆s) around z = y gives

ρ(x, t|z, s+ ∆s) ≈ ρ(x, t|y, s+ ∆s) + ρy(x, t|y, s+ ∆s)(z − y) +1

2ρyy(x, t|y, s+ ∆s)(z − y)2. (6.8)

Substitute (6.8) into the right side of (6.7) and get

∂

∂sρ(x, t|y, s) = − lim

∆s→0

∫ ∞

−∞

ρ(z, s+ ∆s|y, s)∆s

(

∂

∂yρ(x, t|y, s+ ∆s)(z − y) +

1

2

∂2

∂y2ρ(x, t|y, s+ ∆s)(z − y)2

)

dz.

(6.9)

We need to evaluate the right side of (6.9). First, observe that

∫ ∞

−∞ρ(z, s+ ∆s|y, s) ∂

∂yρ(x, t|y, s+ ∆s)(z − y)dz =

∂

∂yρ(x, t|y, s+ ∆s)

∫ ∞

−∞ρ(z, s+ ∆s|y, s)(z − y)dz

(6.10)

It follows exactly as in the derivation of (5.25) that

∂

∂yρ(x, t|y, s+ ∆s)

∫ ∞

−∞ρ(z, s+ ∆s|y, s)(z − y)dz =

∂

∂yρ(x, t|y, s+ ∆s)f(y, s)∆s. (6.11)

127


∫ ∞

−∞ρ(z, s+ ∆s|y, s) ∂

∂yρ(x, t|y, s+ ∆s)(z − y)dz =

∂

∂yρ(x, t|y, s+ ∆s)f(y, s)∆s. (6.12)

Likewise, following the same steps as in the derivation of (5.39), we obtain

∫ ∞

−∞ρ(z, s+∆s|y, s)1

2

∂2

∂y2ρ(x, t|y, s+∆s)(z−y)2dz =

1

2

∂2

∂y2ρ(x, t|y, s+∆s)

(

g2(y, s)∆s+ f2(y, s)[∆s]2)

.

(6.13)

Combining (6.9), (6.12) and (6.13), we conclude that

∂

∂sρ(x, t|y, s) = − lim

∆s→0

(

f(y, s)∂

∂yρ(x, t|y, s+ ∆s) + (g2(y, s) + f2(y, s)∆s)

1

2

∂2

∂y2ρ(x, t|y, s+ ∆s)

)

.

(6.14)

Finally, taking the limit, we obtain the backwards Kolmogorov equation

∂

∂sρ(x, t|y, s) = −f(y, s)

∂

∂yρ(x, t|y, s) − g2(y, s)

1

2

∂2

∂y2ρ(x, t|y, s). (6.15)

Important special case. When f and g are independent of t, the backwards Kolmogorov equation

becomes∂

∂sρ(x, t|y, s) = −f(y)

∂

∂yρ(x, t|y, s) − g2(y)

1

2

∂2

∂y2ρ(x, t|y, s). (6.16)

Suppose that s = 0 and that we want to derive the associated backwards equation satisfied by

ρ(x, t|y, 0). Gardner ([11], p. 137) notes that

ρ(x, t|y, 0) = ρ(x, 0|y,−t). (6.17)

Then ρ(x, t|y, 0) satisfies

∂

∂τρ(x, t|y, 0) = f(y)

∂

∂yρ(x, t|y, 0) + g2(y)

1

2

∂2

∂y2ρ(x, t|y, 0), ρ(x, 0|y, 0) = δ(x− y). (6.18)

Remarks: (i) We’ll use this form of the backwards equation in the analysis of exit times.

(ii) It is easily verified that the Wiener process conditional pdf (4.7) and the Ornstein-Uhlenbeck

process pdf (7.24) both satisfy (6.17).

128

7 Examples of the Fokker - Planck equation.

(a.) wienerSDEcalar.m solves the Wiener process SDE W ′ = σζ(t).

We show how to obtain a numerical “realization” of the Wiener process. The SDE is

dW (t)

dt= σζ(t), (7.1)

where ζ(t) is white noise. The site http://www.engineering.ucsb.edu/ moehlis/APC591/tutorials/tutorial7/node2.html

has a description of the matlab programs wienerSDEscalar.m (scalar version) and wienerSDEvec-

tor.m (vectorized version) which solve this equation. On google type wiener process in matlab and

click on the entry “ A Standard Wiener Process”. These programs are also the 1st two programs in

Higham’s article [12]. The “scalar” discretization of (7.1) is

W (ti+1) = W (ti) + σ(√

∆t)

randn, (7.2)

where√

∆t = ti+1 − ti, and randn is the command to obtain an element of N(0, 1). Here

(∆t)ζ(t) = (∆t)∆W

∆t= ∆W =

(√∆t)

randn. (7.3)

To see where the square root term comes we recall from (4.55) that

E(

[W (t+ ∆t) −W (t)]2)

= t+ ∆t− 2t+ t = ∆t. (7.4)

Thus E((∆W )2) = ∆t, and from this we get the average value ∆W =√

∆t in (12.58) and (7.3).

The program wienerSDEscalar.m below gives the “scalar” solution W ′(t) = σζ(t)

T = 1; N = 500; dt = T/N; sigma=2

dW=zeros(1,N); W=zeros(1,N); dW(1) = sqrt(dt)*randn; W(1) = dw(1);

sum=zeros(1,N); sum(1)=W(1); meanvalue=zeros(1,N); meanvalue(1)=W(1);

for j = 2:N

dW(j) = sigma*sqrt(dt)*randn; W(j) = W(j-1) + sigma*dW(j); sum(j)=W(j)+sum(j-1); end

for j=2:N

meanvalue(j)=sum(j)/j; end;

plot([0:dt:T],[0,meanvalue],[0:dt:T],[0,W],’r-’); xlabel(’t’,’FontSize’,16); ylabel(’W(t)’,’FontSize’,16,’Rotation’,0)

legend(’blue curve is the mean’,’red jagged vurve is W(t)’,2)

The program wienerSDEvector.m below gives the vectorized solution of W ′(t) = σζ(t).

T = 1; N = 500; dt = T/N; sigma=2; dW = sqrt(dt)*randn(1,N); W =σ cumsum(dW);

Note: the matlab command cumsum computes the integral∫ t

0dW (t)

plot([0:dt:T],[0,W],’r-’); xlabel(’t’,’FontSize’,16); ylabel(’W(t)’,’FontSize’,16,’Rotation’,0)

(b.) The Fokker-Planck equation for the Wiener process SDE

129

Recall that the Langevin stochastic differential equation for a first order Markov process X(t) is

dX(t)

dt= f(X(t), t) + g(X(t), t)ζ(t), (7.5)

where ζ(t) is white noise. The associated Fokker-Planck pde for the conditional probability density is

∂

∂tρ(x, t|y, s) =

[

− ∂

∂xf(x, t) +

1

2

∂2

∂x2(g(x, t))2

]

ρ(x, t|y, s). (7.6)

The intial condition is

ρ(x, s|y, s) = δ(x− y) (7.7)

The stochastic differential equation for the Wiener process is

dX(t)

dt= σζ(t), (7.8)

where ζ(t) is white noise. Thus, f(x, t) = 0 and g(x, t) = σ, and the Fokker-Planck equation (7.6) is

∂

∂tρ(x, t|y, s) =

σ2

2

∂2

∂x2ρ(x, t|y, s). (7.9)

Solution by the Fourier transform method. We let σ = 1 and solve the single time pde

∂

∂tu(x, t) =

1

2

∂2

∂x2u(x, t), (7.10)


u(x, 0) = δ(x). (7.11)

Taking the Fourier transform of both sides of (7.10), we obtain

∂

∂th(ω, t) = −ω

2

2h(ω, t), (7.12)

where

h(ω, t) =

∫ ∞

−∞e−iωxρ(x, t, y, s)dx. (7.13)

The Fourier transform of (7.11) gives

h(ω, 0) =

∫ ∞

−∞e−iωxδ(0)dx = 1. (7.14)

The solution of (7.12) which satisfies (7.14) is

h(ω, t) = e−ω2

2 t. (7.15)

Now consider the function v = be−ax2

. The Fourier transform of v is

v =

∫ ∞

∞

e−iωxbe−ax2

dx = be−ω2

4a

∫ ∞

−∞e−a(x−i ω2

2a )2dx =b√ae−

ω2

4a√π (7.16)

130

Let a = 12t and b = 1√

2πt. Then v = h(ω, t) and the corresponding functions u(x, t) and v(x, t) satisfy

u(x, t) = v(x, t) =1√2πt

e−x2

2t (7.17)

To obtain the conditional probability solution we replace the t with t− s and x with x− y to obtain

ρ(x, t|y, s) =1

√

2π(t− s)e−

(x−y)2

2(t−s) . (7.18)

Finally, the joint probability distribution ρ(x, t, y, s) is given by the product

ρ(x, t, y, s) =1

√

2π(t− s)e−

(x−y)2

2(t−s)1√2πs

e−y2

2s . (7.19)

(c.) ornstein.m solves the Ornstein-Uhlenbeck SDE dX(t)dt = −γX + σζ(t).

The matlab program ornstein.m computes the solution of the SDE dX(t)dt = −γX+σζ(t). The method

is to select points t1, t2, ..., tN all equally spaced, with ∆t = ti+1 − ti. The discretization is

X(ti+1) −X(ti) = −γX(ti) + σ(√

∆t)

randn. (7.20)

Here the lst term on the right is due to the observation that

(∆t)ζ(t) = (∆t)∆W

∆t= ∆W =

(√∆t)

randn, (7.21)

where W denotes the Wiener process. Note: Section 5 (d.) shows how to apply the Ito Lemma

to the Ornstein process. The program ItoExample2.m computes the true solution, solves the SDE

numerically using Euler’s method, and then compares the true and numerical solution.

The Focker-Planck equation. Substituting f(x, t) = −γx and g(x, t) = σ into

∂

∂tρ(x, t|y, s) =

[

− ∂

∂xf(x, t) +

1

2

∂2

∂x2(g(x, t))2

]

ρ(x, t|y, s) (7.22)

gives the pde

∂

∂tρ(x, t|y, s) = γρ(x, t|y, s) + γx

∂

∂xρ(x, t|y, s) +

σ2

2

∂2

∂x2ρ(x, t|y, s). (7.23)

Below, in part (d.) Monahan [16] shows how to use the SDE to derive the conditional probability

ρ(x, t|y, s) =1

√

2πg(t− s)e−

(x−ye−γ(t−s))2

2(g(t−s) , (7.24)

where

g(t) =σ2

2γ

(

1 − e−2γt)

. (7.25)

Letting t→ ∞ gives the steady state probability function

ρstat(x) =

√

γ

πσ2e−

γx2

σ2 (7.26)

131

Doering claims that “the covariance in the steady state is”

E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞ρ(x, t|y, s)ρstat(y)dxdy =

σ2

2γe−γ|t−s|. (7.27)

To verify (7.27) we begin with

E (X(t)X(s)) =

∫ ∞

−∞

∫ ∞

−∞

xy√

2πg(t− s)e−


2(g(t−s)

√

γ

πσ2e−

γy2

σ2 dxdy (7.28)

Assume that t > s. Reordering the terms in the integration gives

E (X(t)X(s)) =

∫ ∞

−∞y

√

γ

πσ2e−

γy2

σ2

(

∫ ∞

−∞

x√

2πg(t− s)e−


2(g(t−s) dx

)

dy (7.29)

This immeditately reduces to

E (X(t)X(s)) =

∫ ∞

−∞y

√

γ

πσ2e−

γy2

σ2

(

ye−γ(t−s))

dy. (7.30)

This further reduces to

E (X(t)X(s)) = e−γ(t−s)

∫ ∞

−∞y2

√

γ

πσ2e−

γy2

σ2 dy =σ2

2γe−γ|t−s|. (7.31)

Substituting s = t into (7.31) gives the “fluctuation-dissipation” equation

E([X(t)]2) =σ2

2γ. (7.32)

Note that (7.32) is independent of t. A second way to derive (7.32)is to use the steady state pdf

ρstat(x) and compute

E([X ]2) =

∫ ∞

−∞x2ρstat(x)dx =

σ2

2γ. (7.33)

The fluctuation-dissipation equation appears in the analysis when red noise is used to model cli-

mate [26].

(d.) Autocorellation and power functions for the Ornstein-Uhlenbeck SDE.

Doering does’nt say if ρ(x, t|y, s)ρstat(y) in (7.28) is the joint probability function ρ(x, t, y, s). In

Monahana’s and Doering’s notes it looks like the joint probability function is

ρ(x, t, y, s) =1

√

2πg(t− s)e−


2(g(t−s)

√

γ

πσ2e−

γy2

σ2 , (7.34)

where g(t) = σ2

2γ

(

1 − e−2γt)

. Recall from (7.31) that

E (X(t)X(s)) =σ2

2γe−γ|t−s|. (7.35)

From this we obtain the autocorellation function

RX(τ) = E (X(t)X(t+ τ)) =σ2

2γe−γ|τ |. (7.36)

132

The “power” is given by

Power = RX(0) =σ2

2γ, (7.37)

and the “power function” is given by

S(ω) =1

2π

∫ ∞

−∞e−iωτ σ

2

2γe−γ|τ |dτ =

σ2

2π

1

ω2 + γ2. (7.38)

(e.) How to analyze the Ornstein-Uhlenbeck SDE to obtain ρ(x, t|y, s).

This follows the monahan.pdf notes and shows how to derive the pdf ρ(x, t|y, s) from the SDE

dX(t)

dt= −γX + σζ(t). (7.39)

The first step is to multiply (7.39) by eγt, integrate the result from 0 to t, and obtain

X(t) = e−γtX(0) + σ

∫ t

0

e−γ(t−t′)ζ(t′)dt′, ∀t ≥ 0. (7.40)

Monahan remarks (p. 4, top line of monahan.pdf ) that X(t) is a gaussian process (presumably since

the integral of white noise is gaussian - problem 6.22 of Schaum’s gives part of the proof). Therefore,

we only need to figure out the mean and variance for X(t), then substitute them into the right spot

in the formula for the pdf of a gaussian process. The mean is given by

E(X(t)) = E(e−γtX(0)) + σ

∫ t

0

e−γ(t−t′)E (ζ(t′)) dt′ = e−γtX(0) ∀t ≥ 0, (7.41)

since E(ζ(t)) = 0 ∀t ≥ 0. Next, to find the variance, start with the covariance

E ([X(t) − E(X(t))][X(t+ s) − E(X(t+ s))]) = σ2

∫ t

0

∫ t+s

0

e−γ(t−t′)eγ(t+s−t′)E (ζ(t′)ζ(t′′)) dt′′dt′.

(7.42)

There were four other terms in this computaion, but two cancel each other and two must be zero since

E(ζ(t)) = 0 ∀t ≥ 0. Because E(ζ(t′)ζ(t′′)) = δ(t′ − t′′), then (7.42) becomes

E ([X(t) − E(X(t))][X(t+ τ) − E(X(t+ τ))]) = σ2

∫ t

0

e−γ(t−t′)

(∫ t+τ

0

e−γ(t+s−t′′)δ(t′ − t′′)dt′′)

dt′.

(7.43)

Evaluation of the inner integral using the deltal function reduces (7.43) to

E ([X(t) − E(X(t))][X(t+ τ) − E(X(t+ τ))]) = σ2

∫ t

0

e−γ(2t+τ−2t′)dt′ =σ2

2γe−γ(2t+τ)

(

e2γt − 1)

.

(7.44)

Finally, letting τ = 0 reduces this to the variance formula

V ar(X(t)) = E(

[X(t) − E(X(t))]2)

=σ2

2γ

(

1 − e−2γt)

. (7.45)

(f.) The Fokker-Planck pde for systems of SDE’s

133

Systems (p.34 of Doering’s paper) of SDE’e have the form

dXi(t)

dt= fi(X1, .., XN) +

N∑

j=1

gi,j(X1, .., XN )ζj(t), i = 1, .., N (7.46)

Let G be the N by N matrix G = (gi,j). The matrix D = GGT satisfies

Di,j =

N∑

k=1

gi,kgj,k for 1 ≤ i, j ≤ N. (7.47)

The Fokker-Planck equation associated with the system (7.46) is

∂

∂tρ =

(

− ∂

∂xifi +

1

2

∂

∂xi

∂

∂xjDi,j

)

ρ. (7.48)

Here the subscript notation implies summation. That is, equation (7.48) is really

∂

∂tρ =

−N∑

i=1

∂

∂xifi +

1

2

N∑

i,j=1

∂

∂xi

∂

∂xjDi,j

ρ. (7.49)

Doering notes that the Fokker-Planck equation (7.48) can be written in the form

∂

∂tρ+

∂

∂xiJi = 0, (7.50)

where J is the current vector field with components

Ji(x, t) =

(

fi −1

2

∂

∂xiDi,j

)

ρ(x, t) (7.51)

The summation forms of (7.50) and (7.51) are

∂

∂tρ+

N∑

i=1

∂

∂xiJi = 0, (7.52)

Ji(x, t) =

fi −N∑

j=1

1

2

∂

∂xjDi,j

ρ(x, t) (7.53)

The current vector J gives the flow of probability in state space.

Open problem. Doering claims that finding ways to solve ∇ · J = 0 is an unresolved open problem.

134

8 Ito’s Lemma and applications

(a.) Informal proof of Ito’s Lemma

In google type Ito’s Lemma - Wikipedia to find all of this. Consider the stochastic ODE

dx(t)

dt= a(x, t) + b(x, t)ζ(t), (8.1)

where ζ(t) is normal white noise. Multiply both sides by dt and get the Ito process

dx(t) = a(x, t)dt + b(x, t)dW (t). (8.2)

Ito’s Lemma. Let f(x, t) be a C2 function. Then f(x(t), t) is an Ito process and

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.3)

The first step in proving (8.3) is to consider the expansion

f(x+ ∆x, t+ ∆t) = f(x, t) +∂f

∂x∆x+

∂f

∂t∆t+

1

2

∂2f

∂x2(∆x)2 + · · · (8.4)

Next, in (8.2) replace dx(t), dt and dW (t) with ∆x, ∆t and ∆W and get

∆x = a(x, t)∆t + b(x, t)∆W. (8.5)

Substitute (8.5) into (8.4), set (∆W )2 = ∆t (i.e. repalce (∆W )2 with its average value E(∆W )2 = ∆t)

and keep the lowest order terms. This gives

∆f =

(

a(x, t)∂f

∂x+∂f

∂t+

(b(x, t))2

2

∂2f

∂x2

)

∆t+ b(x, t)∂f

∂x∆W. (8.6)

where ∆f = f(x+ ∆x, t+ ∆t)− f(x, t). Replace ∆f, ∆t and ∆W with df, dt and dW, and get (8.3).

(b.) Example 1. Solve dx(t)dt = λx+ µxζ(t) using Ito’s Lemma.

The solution of dx(t)dt = λx + µxζ(t) using Ito’s Lemma is on pp. 59-61 of [23]. First, recall the SDE

formsdx(t)

dt= a(x, t) + b(x, t)ζ(t), (8.7)

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.8)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.9)

To put dx(t)dt = λx + µxζ(t) into the right form multiply by dt and get

dx(t) = λxdt + µxdW (t). (8.10)

Here a(x, t) = λx and b(x, t) = µx, and therefore (8.9) becomes

df(x(t), t) =

[

λx∂f

∂x+∂f

∂t+

1

2(µx)2

∂2f

∂x2

]

dt+ µx∂f

∂xdW (t). (8.11)

135

To apply Ito’s Lemma we let f(x, t) = ln(x), x > 0. Substituting ∂f∂x = 1

x ,∂f∂t = 0 and ∂2f

∂x2 = − 1x2

into (8.11) gives

df =

(

λ− µ2

2

)

dt+ µdW (t). (8.12)

Integration from 0 to t gives

ln(x(t)) = ln(x(0)) +

(

λ− µ2

2

)

t+ +µW (t) t ≥ 0. (8.13)

From this we obtain

x(t)) = x(0)e

((

λ−µ2

2

)

t+µW (t)

)

, t ≥ 0. (8.14)

Ocksendal[23], p. 62, points out that W (t) obeys the bound

lim supt→∞W (t)

√

2tlog(logt)= 1 a.s. (8.15)

From (8.14) and (8.15) Ocksendal., on p.60, makes the following conclusions:

if λ > µ2

2 then x(t) → ∞ as t→ ∞,

if λ < µ2

2 then x(t) → 0 as t→ ∞,

if λ = µ2

2 then x(t) flucutates between arbitrarily large and small values as t→ ∞.

(8.16)

Conclusions. When µ = 0 and λ > 0 the solution grows exponentially fast without bound. When

we add the multiplicative white noise term µxζ(t) to the equation the effect is to thwart the growth

because of the extra term −µ2

2 t that appears in the answer. This term won’t be present in section

(d.) where we solve dx(t)dt = λx + µζ(t). The difference is that the multiplicative white noise term is

replaced with the “aditive” white noise term µζ(t) in the SDE.

(c.) ItoExample1.m solves dx(t)dt = λx + µxζ(t) and compares with the true sol.

The program ItoExample1.m solves the stochastic ODE numerically and compares the numerical solu-

tion with the exact solution - www.engineering.ucsb.edu/ moehlis/APC591/tutorials/tutorial7/node3.html

describes the numerical method - in google type wiener process in matlab, and click on white noise.

From (8.14) above we have the exact solution x(t) = x(0)e

((

λ−µ2

2

)

t+µW (t)

)

, t ≥ 0.

The program ItoExample1.m is below.

lambda = 2; mu = 1; Xzero = 1; T = 1; N = 28; dt = 1/N ;

dW = sqrt(dt)*randn(1,N); W = cumsum(dW); Here W (t) =∫ t

0ζ(t′)dt′.

Xtrue = Xzero*exp((lambda-0.5*mu2)*([dt:dt:T])+mu*W);

plot([0:dt:T],[Xzero,Xtrue],’m-’), hold on

Xem = zeros(1,N); Xem(1) = Xzero + dt*lambda*Xzero + mu*Xzero*dW(1);

for j=2:N

136

Xem(j) = Xem(j-1) + dt*lambda*Xem(j-1) + mu*Xem(j-1)*dW(j); end

plot([0:dt:T],[Xzero,Xem],’b–*’), hold off

xlabel(’t’,’FontSize’,12); ylabel(’x’,’FontSize’,16,’Rotation’,0,’HorizontalAlignment’,’right’)

emerr = abs(Xem(end)-Xtrue(end))

(d.) Linear stability for dx(t)dt = λx+ µxζ(t)

According to Higham [12], pp. 539-541, the three kinds of linear stability are deterministic stability,

asymptotic stability and mean square stability. First, to compute the asymptotic stability fo the SDE,

we recall that the exact solution is x(t) = x(0)e

((

λ−µ2

2

)

t+µW (t)

)

, t ≥ 0. From this it follows that if

µ = 0 then x = 0 is deterministically stable if Re(λ) < 0. When µ 6= 0 we have the solution x = 0 is

asymtotically stable if Re(λ) < µ2

2 . This is because

|x(t)| → 0 as t→ ∞ ⇐⇒ Re(λ) <µ2

2. (8.17)

The solution x = 0 is mean square stable if E(x2(t)) → 0 as t→ ∞. For this we compute

E(x2(t)) = E(

x2(0)e((2λ−µ2)t+2µW (t)))

= x2(0)e(2λ−µ2)tE(

e2µW (t))

. (8.18)

Note that

E(

e2µW (t))

=

∫ ∞

−∞

e2µw

√2πt

e−w2

2t dt =

∫ ∞

−∞

e2µ2t

√2πt

e−(w−2µt)2

2t dt = e2µ2t. (8.19)

Combining (8.18) and (8.19), we conclude that

E(x2(t)) = x2(0)e(2λ+µ2)t → 0 as t→ ∞ ⇐⇒ Re(λ) < −µ2

2. (8.20)

Note that mean square stability implies asymptotic stability, but asymptotic stability does not neces-

sarily imply mean square stability. More details are in Higham’s article [12].

(e.) higham3.m solves dx(t)dt = x+ .5xζ(t) 1000 times and graphs 5 solutions

This is a very impiortant exercise. The program solves dx(t)dt = x + .5xζ(t) 1000 times, computes the

sample mean for all 1000 solutions, gropahs the first 5 solutions, and also graphs the mena of the 1000

solutions. The program higham3.m is below.

randn(’state’,100); clf; T=1; N=500; dt=T/N; t=[dt:dt:1];M=1000; dW=sqrt(dt)*randn(M,N); W=cumsum(dW,2);

dW is an M ( no. of W paths) row by N ( t1, .., tN values) column matrix of randn nos.

cumsum(dW,2) computes the cumulative sum across each row of dW.

U=exp(repmat(t,[M,1]) + 0.5*W);

The command repmat(t,[M,1]) creates an M by 1 matrix M rows. Each element of M is the row

t = [t1, t2, .., tN ] It’s easy to see if we let M=7, N=10, dt=T/N; t=[dt:dt:T] Then repmat(t,[M,1])

creates a 7 by 1 matrix - a column matrix, each entry of which is the row [.1, .2, .., .9, 1]

137

Umean=mean(U);

Umean is a single row, with N elements, corresponding to t1, .., t10 The first entry of Umean = (sum

of column 1 of U)/M the j-th entry is (sum of col j of U)/M, etc

figure(1); plot([0,t],[1,Umean],’b-’), hold on; plot([0,t],[ones(5,1),U(1:5,:)],’r–’); axis([0 1 0 5.5])

averr=norm( (Umean-exp(9*t/8)),’inf’)

(f.) ItoExample2.m solves dx(t)dt = λx+ µζ(t) and compares with the true sol.

First, recall the general SDE forms

dx(t)

dt= a(x, t) + b(x, t)ζ(t), (8.21)

dx(t) = a(x, t)dt + b(x, t)dW (t). (8.22)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.23)

To put dx(t)dt = λx + µζ(t) into the right form multiply by dt and get

dx(t) = λxdt + µdW (t). (8.24)

Here a(x, t) = λx and b(x, t) = µ, and therefore (8.23) becomes

df(x(t), t) =

[

λx∂f

∂x+∂f

∂t+

1

2µ2 ∂

2f

∂x2

]

dt+ µ∂f

∂xdW (t). (8.25)

Let f(x, t) = xe−λt. Substituting ∂f∂x = e−λt, ∂f

∂t = −λxe−λt and ∂2f∂x2 = 0 into (8.25) gives

df = µeλtdW (t). (8.26)


x(t)e−λt = x(0) + µ

∫ t

0

e−λsdW (s) t ≥ 0. (8.27)

From this we obtain the “true” solution

x(t) = x(0)eλt + µ

∫ t

0

eλ(t−s)dW (s) t ≥ 0. (8.28)

ItoExample2.m matlab program is below.

clf; lambda = 1 ;mu =2; Xzero = 1;T = 1; N = 28; dt = 1/N;

dW = sqrt(dt)*randn(1,N); These are the Brownian increments

The command to compute∫ t

0e−λsdW

integral =cumsum(exp(-lambda*[dt:dt:T]).*dW);

Next, compute the true solution x(t) = x(0)eλt + µ∫ t

0eλ(t−s)dW (s) is on the next line

Xtrue = Xzero*exp(lambda*([dt:dt:T]))+ mu*exp(lambda*[dt:dt:T]).* integral;

Plot the true solutions

138

figure(1); plot([0:dt:T],[Xzero,Xtrue],’m-’), hold on

Below, compute the ” numerical euler method solution” and plot it.

Xem = zeros(1,N); Xem(1) = Xzero + dt*lambda*Xzero + mu*dW(1);

for j=2:N Xem(j) = Xem(j-1) + dt*lambda*Xem(j-1) + mu*dW(j); end

plot([0:dt:T],[Xzero,Xem],’b–*’), hold off

xlabel(’t’,’FontSize’,12)

ylabel(’x’,’FontSize’,16,’Rotation’,0,’HorizontalAlignment’,’right’)

emerr = abs(Xem(end)-Xtrue(end))

(g.) Solve dx(t)dt = µx− x2 + σxζ(t). and find the Fokker-Planck eq.

First, recall the general SDE forms

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.29)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.30)

To put dx(t)dt = µx− x2 + σxζ(t) into the right form multiply by dt and get

dx(t) = (µx− x2)dt+ σxdW (t). (8.31)

Here a(x, t) = µx− x2 and b(x, t) = σx, and therefore (8.30) becomes

df(x(t), t) =

[

(µx − x2)∂f

∂x+∂f

∂t+

1

2σ2x2 ∂

2f

∂x2

]

dt+ σx∂f

∂xdW (t). (8.32)

Let f(x, t) = ln(x). Substituting ∂f∂x = 1

x∂f∂t = 0 and ∂2f

∂x2 = − 1x2 into (8.32) gives

df =

(

µ− σ2

2− x

)

dt+ σdW. (8.33)


x(t) = x(0)e

((

µ−σ2

2

)

t−∫

t

0x(η)dη+σW (t)

)

. (8.34)

To obtain the “exact” solution let v =∫ t

0x(η)dη. Then (8.34) becomes

v′ev = x(0)e

((

µ−σ2

2

)

t+σW (t))

. (8.35)

Solve this equation and get

e

∫ t

0x(η)dη

= 1 + x(0)

∫ t

0

e

((

µ−σ2

2

)

s+σW (s))

ds. (8.36)

Finally, substitute (8.36) into (8.34) and obtain the exact solution

x(t) =x(0)e

((

µ−σ2

2

)

t+σW (t))

1 + x(0)∫ t

0e

((

µ−σ2

2

)

s+σW (s))

ds. (8.37)

139

Remarks. The program verhulst.m solves this eq. and shows that solutions go extinct at finite t,

possibly because x(t) = 0 at a finte t. It seems that the solution in (8.37) won’t allow x(t) = 0 at a

finite value of t. A proof is needed.

The Fokker - Planck equation. Let f(x, t) = µx− x2 and g(x, t) = σx in

∂

∂tρ(x, t|y, s) =

[

− ∂

∂xf(x, t) +

1

2

∂2

∂x2(g(x, t))2

]

ρ(x, t|y, s) (8.38)

and get the Fokker-Planck pde

∂

∂tρ(x, t|y, s) =

∂

∂x

(

(x2 − µx)ρ(x, t|y, s))

+σ2

2

∂2

∂x2

(

x2ρ(x, t|y, s))

. (8.39)

Steady state, time independent solutions satisfy

d

dx

(

(x2 − µx)ρ(x))

+σ2

2

d2

dx2

(

x2ρ(x))

= 0. (8.40)

This reduces to the first order ODE

(x2 − µx)ρ(x) +σ2

2

d

dx

(

x2ρ(x))

= 0. (8.41)

This further reduces to2

σ2+

2(σ2 − µ)

σ2x+ρ′

ρ= 0. (8.42)

Because X(t) > 0 ∀t ≥ 0, we assume that ρ(x) = 0 ∀x ∈ (−∞, 0]. An integration of (8.42) gives

ρ(x) = Nx2( µ

σ2 −1)e−2x

σ2 ∀x ≥ 0. (8.43)

The requirement 1 =∫∞−∞ ρ(x)dx =

∫∞0ρ(x)dx shows that

N =

(∫ ∞

0

x2( µ

σ2 −1)e−2xσ2 dx

)−1

(8.44)

The substitution t = 2xσ2 leads to

N =

(

σ2

2

)

2µ

σ2 −1(∫ ∞

0

t2(µ

σ2 −1)e−tdt

)−1

. (8.45)

Finally, observe that∫ ∞

0

t2(µ

σ2 −1)e−tdt <∞ if 0 < σ < 2µ, (8.46)

∫ ∞

0

t2(µ

σ2 −1)e−tdt = ∞ if σ ≥ 2µ. (8.47)

Thus, when σ ≥ 2µ it follows that N = 0, hence ρ(x) = 0 ∀x and the most probable state is X = 0.

This is the basis of Doering’s explanation of why solutions eventually become extinct when σ ≥ 2µ.

(h.) The Brownian bridge SDE dx = k−x1−t + dW (t), x(0) = A

140

Our goals are (i) solve the SDE, and (ii) prove that E(x(t)) = A(1−t)+kt ∀t, and E((x(t)−k)2) → 0 as

t→ 1− The numerical solution is in BrownianBridge.m The websites en.wikipedia.org/wiki/Brownianbridge

and http://math.la.asu.edu/ kawski/MATLAB/other/wiener.m also discuss the brownian bridge. This

problem is from [23], p. 72. First, recall the SDE forms

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.48)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.49)

Here a(x, t) = k−x1−t and b(x, t) = 1. Let f(x, t) = x

1−t . Substitute fx = 11−t , ft = x

(1−t))2 and fxx = 0

into (8.49) and get

df(x(t), t) =

(

k − x

(1 − t)2+

x

(1 − t)2+ 0

)

dt+1

1 − tdW (t). (8.50)


df(x(t), t) =

(

k

(1 − t)2

)

dt+1

1 − tdW (t). (8.51)

Integrate both sides from 0 to t, substitute f(x, t) = x1−t , and obtain

x

(1 − t)= A+

(

kt

(1 − t)

)

dt+

∫ t

0

1

1 − sdW (s). (8.52)

Multiply both sides by (1 − t) and arrive at the author’s answer

x(t) = A(1 − t) + kt+ (1 − t)

∫ t

0

1

1 − sdW (s), (8.53)

From (8.53) it follows that E(x(t)) = A(1 − t) + kt.

Next, we show that E((x(t) − k)2) → 0 as t→ 1− it follows from (8.53) that

E(

(x(t) − k)2)

= (A+ k)2(1 − t)2 + 2(A+ k)(1 − t)∫ t

01

1−sE(ζ(s))ds

+ (1 − t)2∫ t

0

∫ t

01

1−t′1

1−t′′E(ζ(t′)ζ(t′′))dt′dt′′.(8.54)

Since E(W (s)) = 0 and E(ζ(t′)ζ(t′′)) = δ(|t′ − t′′|), this reduces to

E(

(x(t) − k)2)

= (A+ k)2(1 − t)2 + (1 − t)2∫ t

0

1

(1 − t′)2dt′, (8.55)

hence

E(

(x(t) − k)2)

= (A+ k)2(1 − t)2 + (1 − t)t→ 0 as t→ 1− (8.56)

From (12.1) it follows that if A = k = 0 then the maximum uncertainty of x(t) is at t = 12 .

(i.) Numerical sol. and Fokker-Planck pde for dx(t)dt = α− λx+ σ

√xζ(t).

This equation models price assets and is discussed in [12], pp. 544-546. First, recall the SDE forms

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.57)

141

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.58)

To put dx(t)dt = α− λx + σ

√xζ(t) into the right form multiply by dt and get

dx(t) = (α− λx)dt + σ√xdW (t). (8.59)

Thus, a(x, t) = α− λx and b(x, t) = σ√x, and (8.58) becomes

df(x(t), t) =

[

(α− λx)∂f

∂x+∂f

∂t+

1

2σ2x

∂2f

∂x2

]

dt+ σ√x∂f

∂xdW (t). (8.60)

Let f(x, t) =√x. Substituting ∂f

∂x = 12√

x∂f∂t = 0 and ∂2f

∂x2 = − 14x3/2 into (8.60) gives

df =

(

4α− σ2

8√x

− λ

2

√x

)

dt+ σdW, (8.61)

or equivalently,

df =

(

4α− σ2

8f− λ

2f

)

dt+ σdW, (8.62)

Integrate (8.61) from 0 to t. let f =√x and get the exact solution

√

x(t) =√

x(0) +

∫ t

0

(

4α− σ2

8√

x(s)− λ

2

√

x(s)

)

ds+ σW (t). (8.63)

Numerical solutions: chain.m and chain1.m solve (8.59) and (8.62) and compare the solutions.

The Fokker-Planck equation. To derive the the Fokker-Planck pde we recall the basic forms

dx(t)

dt= f(x(t), t) + g(x(t), t)ζ(t), (8.64)

where ζ(t) is white noise. The associated Fokker-Planck pde for the conditional probability density is

∂

∂tρ(x, t|y, s) =

[

− ∂

∂xf(x, t) +

1

2

∂2

∂x2(g(x, t))2

]

ρ(x, t|y, s). (8.65)

For our equation f = α− λx and g = σ√x. Substituting these functions into (8.65), gives

∂

∂tρ(x, t|y, s) =

[

− ∂

∂x(α− λx) +

1

2

∂2

∂x2σ2x

]

ρ(x, t|y, s). (8.66)

We look for a steady state solution of (8.66) by assuming that ∂∂tρ(x, t|y, s). This leads to the ODE

[

− d

dx(α− λx) +

1

2

d2

dx2σ2x

]

ρ(x) = 0. (8.67)

This further simplifies toσ2

2

d

dx(xρ) − (α − λx)ρ = 0. (8.68)

We assume that ρ(x) = 0 ∀x < 0. When x ≥ 0 separation of variables gives the solution

ρ(x) = Mx2α

σ2 −1e−2λx

σ2 , x ≥ 0, (8.69)

142

where M =(

∫∞0 x

2α

σ2 −1e−2λx

σ2 dx)−1

Note that M = 0 if α ≤ 0 or λ ≤ 0. Thus, is there is no steady

state when α ≤ 0 or λ ≤ 0 and the solution x(t) = 0 becomes the most probable state. There is

bizarre behavior when we let α = 0 or λ ≤ 0. Use the program chain1.m to see this behavior.

(j.) Proof that∫ t

0WNdW (s) = W N+1

N+1 − N2

∫ t

0WN−1(s)ds and E

(

∫ t

0WNdW (s)

)

= 0.

First, recall the SDE forms

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.70)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.71)

The idea is to recognize that W (t) solves dx(t)dt = ζ(t). Multiply by dt and get dx(t) = (0)dt+ dW (t).

Here a(x, t) = 0 and b(x, t) = 1, and therefore (8.71) becomes

df(x(t), t) =

[

(0)∂f

∂x+∂f

∂t+

1

2

∂2f

∂x2

]

dt+∂f

∂xdW (t). (8.72)

Let f(x, t) = xN+1

N+1 . Substituting ∂f∂x = xN , ∂f

∂t = 0 and ∂2f∂x2 = NxN−1 into (8.72) gives

df =N

2xN−1 + xNdW (t). (8.73)

Finally, integrate from 0 to t, set x = W (t), f = W N+1(t)N+1 and get

∫ t

0

WNdW (s) =WN+1(t)

N + 1− N

2

∫ t

0

WN−1(s)ds. (8.74)

For example, if N = 1 this reduces to the basic identity

∫ t

0

WdW (s) =W 2(t)

2− 1

2

∫ t

0

ds =W 2

2− t

2. (8.75)

If N = 2 then (8.74) reduces to

∫ t

0

W 2dW (s) =W 3(t)

3−∫ t

0

W (s)ds. (8.76)

If N = 3 we get∫ t

0

W 3(s)dW (s) =W 4(t)

4− 3

2

∫ t

0

W 2(s)ds. (8.77)

To prove the second identity take E of both sides of (8.74) and get

E

(∫ t

0

WN (s)dW (s)

)

=1

N + 1E(

WN+1(t))

− N

2

∫ t

0

E(

WN−1(s))

ds. (8.78)

This becomes

E

(∫ t

0

WN (s)dW (s)

)

=1

N + 1

∫ ∞

−∞wN+1 1√

2πte−

w2

2t dw − N

2

∫ ∞

−∞wN−1 1√

2πte−

w2

2t dw. (8.79)

Using the Gamma function, evaluate the two terms on the right and see that they add up to zero.

143

(k.) Proof that∫ t

0 g(s)dW (s) = g(t)W (t) −∫ t

0 W (s)dg(s) and E(

∫ t

0 g(s)dW (s))

= 0.

First, recall the SDE

dx(t) = a(x, t)dt + b(x, t)dW (t), (8.80)

df(x(t), t) =

[

a(x, t)∂f

∂x+∂f

∂t+

1

2(b(x, t))2

∂2f

∂x2

]

dt+ b(x, t)∂f

∂xdW (t). (8.81)

As in the last example, the idea is to recognize that W (t) solves dx(t)dt = ζ(t). Multiply by dt and get

dx(t) = (0)dt+ dW (t). Here a(x, t) = 0 and b(x, t) = 1, and therefore (8.81) becomes

df(x(t), t) =

[

(0)∂f

∂x+∂f

∂t+

1

2

∂2f

∂x2

]

dt+∂f

∂xdW (t). (8.82)

Let f(x, t) = g(t)x. Substituting ∂f∂x = g(t), ∂f

∂t = g′(t)x and ∂2f∂x2 = 0 into (8.82 ) gives

df = g′(t)x(t)dt + g(t)dW (t). (8.83)

Finally, integrate from 0 to t, set x = W (t), f = g(t)W (t) and get∫ t

0

g(s)dW (s) = g(t)W (t) −∫ t

0

W (s)dg(s)ds. (8.84)

To prove the second identity take E of both sides of (8.84 ) and get

E

(∫ t

0

g(s)dW (s)

)

= g′(t)E (W (t)) −∫ t

0

E (W (s)) dg(s) = 0, (8.85)

since E(W (t)) = 0 ∀t.

(l.) How to find ρ(V, t|v, 0) numerically from the SDE

The goal is to obtain the pdf for the Fokker-Planck solution ρ(V, t|v, 0), at a given t, from the SDE

dV = f(V )dt+ σdW, V (0) = v. (8.86)

The steps to do this are the following:

Step I. Select a given t, e.g. t=10. Use a for loop to solve the SDE with Euler’s method on [0,10] a

large number of times, e.g. 100000 times, and save the solution value at t=10 in a row vector callled

solutionvalues. Initialize

solutionvalues=[]

and at the end of each run put

solutionvalues=[solutionsvalues V(t=10)];

Step II. Create 1000 bins for ρ(V, t|v, 0) For example, if we want ρ(V, t|v, 0) for V ∈ [−20, 20] then let

∆ = (20 + 20)/1000 = 40/1000. Create a vector

envalues=zeros(1,1000);

For kk=1:1000, sort through the vector solutionvalues and put the no. of solutionvalues that lie in

((kk− 1) ∗∆, kk ∗∆] into envalues(kk). After this procedure is done divide envalues by size(envalues)

and plot envalues vs 1000 V values in[-20,20]. This should give the graph of ρ(V, t|v, 0) vs V.

144

9 A power law associated with dX = µXdt + σXdW, X(0) = x0

This follows the derivation given by Reed [25].

(a.). Transform dX = µXdt+ σXdW,X(0) = x0 into dYdt = µ+ σζ(t), Y (0) = y0.

Consider the SDE

dX = µXdt+ σXdW, X(0) = x0. (9.1)

Let Y = ln(X), apply Ito’s Lemma with f(X, t) = ln(X) and get the SDE

dY = (µ− σ2

2)dt+ σdW (t), Y (0) = y0 = ln(x0). (9.2)

The term (µ− σ2

2 )dt is a ’drift’ term.

(b.) Find ρ(y, t|y0, 0) for dYdt = (µ− σ2

2 )dt+ σζ(t), Y (0) = y0.

Method I. Solve the SDE

dY = (µ− σ2

2)dt+ σdW (t), Y (0) = y0. (9.3)

Integrating both sides of (9.3) from 0 to t gives the solution

Y = (µ− σ2

2)t+ y0 + σW (t). (9.4)

The function Y (t) is a random process. To find the pdf ρ(y, t|y0, 0) = fY (t)(y) we observe that

Prob[Y ≤ y] = Prob[(µ− σ2

2 )t+ y0 + σW (t) ≤ y]

= Prob

[

W (t) ≤ (y−(µ−σ2

2 )t−y0)

σ

]

=∫ (y−(µ−σ2

2 )t−y0)/σ

−∞1√2πt

eη2

2t dη.

(9.5)

Differentiating with respect to y gives

ρ(y, t|y0, 0) = fY (t)(y) =d

dyProb[Y ≤ y] =

1√2πtσ

exp

(

(y − (µ− σ2

2 )t− y0)2

2σ2t

)

. (9.6)

Method II. Find the solution ρ(y, t|y0, 0) of the associated Fokker-Planck equation

∂ρ

∂t= − ∂

∂y

(

(µ− σ2

2)ρ

)

+1

2

∂2ρ

∂y2, (9.7)

such that

ρ(y, 0|y0.0) = δ(y − y0) and

∫ ∞

−∞ρ(y, t|y0, 0)dy = 1. (9.8)

The solution of (9.7)-(9.8) is the function defined in (9.6). More detials later as to how to derive this

solution from the Fokker-Planck equation.

145

(c.) Use moment generating functions to derive the power law.

Given that t = T, where T > 0 is a constant, the moment generating function for Y (T ) is

MY (T )(s) = E(eY (T )s|T ) =

∫ ∞

0

exp(ys)1√

2πtσexp

(

(y − (µ− σ2

2 )T − y0)2

2σ2T

)

dy (9.9)

An evaluation of the right side gives

MY (T )(s) = E(eY (T )s|T ) = exp

(

y0s+ [(µ− σ2

2)s+

σ2

2s2]T

)

. (9.10)

Reed [25] assumes that T > 0 is an exponential random variable with pdf

fT (t) = e−λt, (9.11)

for some λ > 0. This assumption, together with (9.4), leads us to define the random variable

Y = Y (T ) = (µ− σ2

2)T + y0 + σW (T ), −∞ < Y <∞. (9.12)

Associated with Y is the random variable

X = eY , 0 < X <∞. (9.13)

Our goal is to find the pdf ’s fY (y) and fX(x), and to show that fX(x) has the form of a power law.

To do this we begin by finding a formula for the moment generating function

MY (s) = E(

eY s)

. (9.14)

Since Y is a combination of two random variables, T and W (T ), the evaluation of E(

eY s)

involves

a two variable pdf. In this setting Reed shows that

MY (s) = E(

eY s)

= ET

(

E(eY (T )s|T ))

= ET

(

MY (T )(s))

=∫∞0MY (t)(s)fT (t)dt

=∫∞0 exp

(

y0s+ [(µ− σ2

2 )s+ σ2

2 s2]t)

λexp−λtdt.

(9.15)

Aside. I think that the identity

E(

eY s)

= ET

(

E(eY (T )s|T ))

(9.16)

can be derived by the same method that we use to derived a similar integral representation to prove

that z2 = T2 − T1 is an exponential random variable in our analysis of the Poisson process.

146

Next, an integration of the right side of the last line in (9.15) gives

MY (s) =ey0sλ

λ− [(µ− σ2

2 )s+ σ2

2 s2]. (9.17)

An algebraically more amenable form of (9.17) is

MY (s) =−(2/σ2)ey0sλ

s2 + 2σ2 (µ− σ2

2 )s− 2λσ2

. (9.18)

The zeros of the denominator are s1 = α > 0 and s2 = −β < 0, where

α = − 1

σ2

(

µ− σ2

2

)

+1

σ2

√

(

µ− σ2

2

)2

+ 2λσ2 and β =1

σ2

(

µ− σ2

2

)

+1

σ2

√

(

µ− σ2

2

)2

+ 2λσ2

(9.19)

Note that α > 0 and β > 0, and that

αβ =2λ

σ2. (9.20)

Combining (9.18), (9.19) and (9.20), we obtain

MY (s) =ey0sαβ

(α− s)(s+ β). (9.21)

The pdf fY (y, y0) for Y satisfies

MY (s) =

∫ ∞

−∞esyfY (y)dy =

ey0sαβ

(α− s)(s+ β), − β < s < α, (9.22)

and is given by

fY (y, y0) =

αβα+β exp (β(y − y0)) , y < y0,

αβα+β exp (−α(y − y0)) , y ≥ y0.

(9.23)

To verify this claim we substitute (9.24) into (9.22) and obtain

MY (s) = αβα+β

(

∫ y0

−∞ esyeβ(y−y0)dy +∫∞

y0esye−α(y−y0)dy

)

= αβα+β

(

e−βy0∫ y0

−∞ e(β+s)dy + eαy0∫∞

y0e(s−α)dy

)

= αβα+β

(

esy0

β+s + esy0

α−s

)

= ey0sαβ(α−s)(s+β) − β < s < α.

(9.24)

Note that we have used the identities

esy0

β + s=

∫ y0

−∞esyeβ(y−y0)dy and

esy0

α− s=

∫ ∞

y0

esye−α(y−y0)dy. (9.25)

147

These can also be written as

esy0

β + s=

∫ ∞

−∞esyeβ(y−y0)H(y0 − y)dy and

esy0

α− s=

∫ ∞

−∞esye−α(y−y0)H(y − y0)dy, (9.26)

where H is the heaviside function. An integration shows that fY (y) satisfies the requirement

∫ ∞

−∞fY (y, y0)dy = 1. (9.27)

Finally, we need to determine the pdf fX(x). For this we start with

Prob[X ≤ x] = Prob[eY ≤ x]

= Prob[Y ≤ ln(x)]

=∫ ln(x)

−∞ fY (y)dy.

(9.28)

Differentiaion with respect to x gives

fX(x, x0) =d

dxProb[X ≤ x] =

1

xfY (ln(x)). (9.29)

Combining this with (9.24), we find that the pdf for X is the power law

fX(x, x0) =

αβα+β

1x0

(

xx0

)β−1

, 0 < x < x0,

αβα+β

1x0

(

xx0

)−α−1

, x ≥ x0.(9.30)

An integration shows that fX(x, x0) satisfies the requirement

∫ ∞

0

fX(x, x0)dx = 1. (9.31)

A calculation shows that the mean of X is

µX =

∫ ∞

0

xfX(x, x0)dx =

αβ(β+1)(α−1) if α > 1,

0, otherwise.(9.32)

(d.) The generalized power law.

Suppose that T is the Gamma random variable whose pdf is

fT (t) = λeλt (λt)N−1

(N − 1)!, N ≥ 1. (9.33)

Substitution of (9.33) into the last line of (9.15), followed by an integration, gives

MY (s) = ey0s

(

λ

λ− [(µ− σ2

2 )s+ σ2

2 s2]

)N

(9.34)

148

An algebraically more amenable form of (9.34) is

MY (s) = ey0s

(

−(2/σ2)λ

s2 + 2σ2 (µ− σ2

2 )s− 2λσ2

)N

(9.35)

Then

MY (s) = ey0s

(

αβ

(α− s)(s+ β)

)N

, (9.36)

where, as we observed earlier, α > 0 and β > 0 satisfy

α = − 1

σ2

(

µ− σ2

2

)

+1

σ2

√

(

µ− σ2

2

)2

+ 2λσ2 and β =1

σ2

(

µ− σ2

2

)

+1

σ2

√

(

µ− σ2

2

)2

+ 2λσ2

(9.37)

Again we note that α > 0 and β > 0, and that

αβ =2λ

σ2. (9.38)

Our immediate goal is to find the pdf gY (y,N, y0) such that

MY (s) =

∫ ∞

−∞esygY (y)dy = ey0s

(

αβ

(α− s)(s+ β)

)N

(9.39)

For this we make use of the identity

(

αβ

(α− s)(s+ β)

)N

=

(

1

Γ(N)

)2

(αβ)N ∂2(N−1)

∂αN−1∂βN−1

(

1

(α− s)(β + s)

)

. (9.40)

It folllows from (9.22) and (9.24) that

1

(α− s)(β + s)=

∫ y0

−∞

esy

α+ βeβ(y−y0)dy +

∫ ∞

y0

esy

α+ βe−α(y−y0)dy. (9.41)

Combining (9.39), (9.40 ) and (9.41 ), we obtain

∫ ∞

−∞esygY (y,N, y0)dy = ey0s

(

αβ

(α− s)(s+ β)

)N

(9.42)

where gY (y,N, y0), the pdf of Y , is given by

gY (y,N, y0) =

(

1Γ(N)

)2

(αβ)N ∂2(N−1)


(

1α+β e

β(y−y0))

, y < y0,

(

1Γ(N)

)2

(αβ)N ∂2(N−1)


(

1α+β e

−α(y−y0))

, y ≥ y0.(9.43)

The correspomding pdf for X is the generalized power law

fX(x,N, x0) =

(

1Γ(N)

)2(αβ)N

x0

∂2(N−1)


(

1α+β

(

xx0

)β−1)

, 0 < x < x0, ,

(

1Γ(N)

)2(αβ)N

x0

∂2(N−1)


(

1α+β

(

xx0

)−α−1)

, x ≥ x0, .(9.44)

149

(e.) The generalized power law when N=2

Substitution of N=2 into (9.43) and (9.44) shows that the pdf’s for Y and X are given by

gY (y, 2, y0) =

(αβ)2(

2(α+β)3 − y−y0

(α+β)2

)

eβ(y−y0), y < y0,

(αβ)2(

2(α+β)3 + y−y0

(α+β)2

)

e−α(y−y0), y ≥ y0,(9.45)

and

gX(x, 2, x0) =

(αβ)2

x0

(

2(α+β)3 − ln(x/x0)

(α+β)2

)(

xx0

)β−1

, 0 < x < x0,

(αβ)2

x0

(

2(α+β)3 + ln(x/x0)

(α+β)2

)(

xx0

)−α−1

, x ≥ x0

(9.46)

(f.) The generalized power law when N=3

Substitution of N=3 into (9.43) and (9.44) shows that the pdf’s for Y and X are given by

gY (y, 3, y0) =

(αβ)3(

4!(α+β)5 − 2(3!)

(α+β)4 (y − y0) + 2(y−y0)2

(α+β)3

)

eβ(y−y0), y < y0,

(αβ)3(

4!(α+β)5 + 2(3!)

(α+β)4 (y − y0) + 2(y−y0)2

(α+β)3

)

e−α(y−y0), y ≥ y0,(9.47)

and

gX(x, 3, x0) =

(αβ)3

x0

(

4!(α+β)5 − 2(3!)

(α+β)4 ln(x/x0) + 2(ln(x/x0))2

(α+β)3

)(

xx0

)β−1

, 0 < x < x0,

(αβ)3

x0

(

4!(α+β)5 − 2(3!)

(α+β)4 ln(x/x0) + 2(ln(x/x0))2

(α+β)3

)(

xx0

)−α−1

, x ≥ x0

(9.48)

Remark. It appears that there is at most one relative maximum for each fX(x,N, x0).

(f.) Large time behavior properties of X(t).

Here our goal is to understand the large time behavior of the original random variable X(t). First, let

K2 > K1 > 0 be arbitratily chosen. Then

Prob[K1 < X(t) < K2] = Prob[ln(K1) < ln(X(t)) < ln(K2)]

= Prob[ln(K1) < −(σ2

2 − µ)t+ y0 + σW (t) < ln(K2)]

= Prob[ 1σ

(

ln(K1

x0) + (σ2

2 − µ)t)

< W (t) < 1σ

(

(σ2

2 − µ)t+ ln(K2

x0

)

]

(9.49)

Define

τ1 =1

σ

(

ln(K1

x0) + (

σ2

2− µ)t

)

and τ2 =1

σ

(

ln(K2

x0) + (

σ2

2− µ)t

)

(9.50)

150

Then

Prob[K1 < X(t) < K2] = Prob[τ1 < W (t) < τ2] =

∫ τ2

τ1

1√2πe−x2/2dx (9.51)

Note that

limt→∞

τ1 = limt→∞

τ2 = ∞ if σ2 > 2µ. (9.52)

This and (9.51) imply that

limt→∞

Prob[K1 < X(t) < K2] = limt→∞

∫ τ2

τ1

1√2πe−x2/2dx = 0 if σ2 > 2µ. (9.53)

Suppose that K2 = ∞. Then

limt→∞

Prob[X(t) > K1] = limt→∞

∫ ∞

τ1

1√2πe−x2/2dx = 0 if σ2 > 2µ. (9.54)

This means that, for each K1 > 0,

limt→∞

Prob[X(t) ≤ K1] = limt→∞

(

1 −∫ ∞

τ1

1√2πe−x2/2dx

)

= 1 if σ2 > 2µ. (9.55)

Since K1 > 0 is arbitrary, this gives evidence that X(t) goes to extinction as t → ∞ when σ2 > 2µ.

What seems contradictory is that simultaneously we could have that the mean of X is finite when

σ2 > 2µ. This can happen, at least for σ2 just above 2µ, if λµ > 1.

(f.) A power law associated with the Gompertz equation.

The Gompertz equation consists of the system

dNdt = K1NG,

dGdt = −K2G,

(9.56)

where K1 > 0, K2 > 0. The initial conditions are

N(0) = N0 > 0, G(0) = G0 > 0. (9.57)

The solution of the second equation in (9.56) is

G(t) = G0e−K2t. (9.58)

Substituting this into the first equation of (9.56), we get

dN

dt= K1G0e

−K2tN, N(0) = N0. (9.59)

Solving this equation gives

N(t) = N0exp

(

K1G0

K2

(

1 − e−K2t)

)

. (9.60)

To introduce noise into the Gompertz equation we assume that K2 is noisy, i.e.

K2 = K2 + σζ(t), (9.61)

151

where K2 is the mean value of K2 and ζ(t) denotes white noise. Then system (9.56) becomes

dNdt = K1NG,

dGdt = −K2G+ σGζ(t).

(9.62)

Multiplying each equation by dt puts the system into the standard SDE form

dN = K1NGdt,

dG = −K2Gdt+ σGdW (t),

(9.63)

where dW (t) denotes gausssian noise. For simplicity we drop the bar on K2 and study

dN = K1NGdt, N(0) = N0

dG = −K2Gdt+ σGdW (t), G(0) = G0

(9.64)

Following the steps in (a.)-(e.) above, we let Y = ln(G) and y0 = ln(G0), and let T be a Gamma

random variable of order N. If N=1 then T is an exponential random variable. Thus, Y = Y (T ) and

G = G(T ) are random variables The pdf for Y is

fY (y, y0) =

αβα+β exp (β(y − y0)) , y < y0,

αβα+β exp (−α(y − y0)) , y ≥ y0,

(9.65)

and the pdf for G is the power law

fG(G,G0) =

αβα+β

1G0

(

GG0

)β−1

, 0 < G < G0,

αβα+β

1G0

(

GG0

)−α−1

, G ≥ G0,(9.66)

where

α =1

σ2

(

µ+σ2

2

)

+1

σ2

√

(

µ+σ2

2

)2

+ 2λσ2 and β = − 1

σ2

(

µ+σ2

2

)

+1

σ2

√

(

µ+σ2

2

)2

+ 2λσ2

(9.67)

152

(h.) The Gompertz equation with a delay term.

The Gompertz delay equation consists of the system

dN(t)dt = K1N(t)G(t),

dG(t)dt = −K2G(t− τ),

(9.68)

where K1 > 0, K2 > 0 and τ ∈ (−∞,∞). Our goal here is to analyze solutions of

dG(t)

dt= −K2G(t− τ). (9.69)

The simplest solution of (9.69) has the form

G(t) = eλt. (9.70)

Substituting (9.70) into (9.69) gives the eigenvalue equation

λ = −K2e−λτ . (9.71)

Let λ = α+ iβ. Then (9.71) becomes

α+ iβ = −K2e−(α+iβ)τ . (9.72)

Separating real and imaginary parts, we obtain the algebraic system

f(α, β, τ) = α+K2e−ατcos(βτ) = 0,

g(α, β, τ) = β −K2e−ατsin(βτ) = 0.

(9.73)


cos(βτ) = − αK2eατ and sin(βτ) = β

K2eατ . (9.74)

Therefore 1 = sin2(βτ) + cos2(βτ) = α2+β2

K22e2ατ . Solving for τ gives the function

τ = − 1

2αln

(

α2 + β2

K22

)

. (9.75)

We use (9.75) in our investigation of (9.73) when λ is real, imaginary or strictly complex.

I. Real eigenvalues.

Let β = 0 so that λ = α+ iβ = α is real. In this case (9.73) reduces to

α+K2e−ατ = 0. (9.76)

If α ≥ 0 then (9.76) has no solution. Assuming that α < 0, we solve (9.76) for τ, and obtain

τ = − 1

αln

(−αK2

)

, −∞ < α < 0, (9.77)

153

which is equivalent to (9.75). The corresponding branch Λ0 of solutions of (9.73) is defined by

Λ0 =

(α, β, τ)|α < 0, β = 0, τ = − 1

αln

(−αK2

)

(9.78)

Remarks.

(i) The function τ(α) = − 1α ln

(

−αK2

)

has a unique relative maximum value τ∗ = 1eK2

at α = −eK2,

no relative minimum, a unique zero at α = −K2,

limα→−∞

τ(α) = 0 and limα→0−

τ(α) = −∞. (9.79)

(ii) For each τ ∈ (0, 1eK2

) there are exactly two values α1 < −eK2 < α2 < 0 such that (α1, 0, τ) ∈ Λ0

and (α1, 0, τ) ∈ Λ0. Corresponding to α1 and α2 is the two parameter family

G = c1eα1t + c2e

α2t (9.80)

of solutions of the delay equationd

dtG(t) = −K2G (t− τ) . (9.81)

The function G in (9.80) either monotonically approaches zero as t → ∞, or else G has one relative

maximum or minimum at some finite t, and subsequently approaches zero monotonically.

(iii) For each α ∈ (−∞, 0) there is a unique τ(α) = − 1α ln

(

−αK2

)

such that (α, 0, τ(α)) ∈ Λ0. It follows

by direct substitution that for each α ∈ (−∞, 0), the function

G = c1eαt (9.82)

solves the delay equaiond

dtG(t) = −K2G

(

t+1

αln

(−αK2

))

. (9.83)

II. Imaginary eigenvalues. Let α = 0 so that λ = iβ is purely imaginary, and (9.73) reduces to

K2cos(βτ) = 0,

β −K2sin(βτ) = 0.

(9.84)

It follows from (9.84) that, when α = 0, the solutions of (9.73) are

(α, β, τ) =

(

0,±K2,(4N+1)

2K2

)

, N = 0, 1, 2, ..(

0,±K2,− (3N+1)2K2

)

, N = 0, 1, 2, ..(9.85)

For each integer N ≥ 0, the point (α, β, τ) =(

0,±K2,(4N+1)

2K2

)

generates the two parameter family

G(t) = c1sin(K2t) + c2cos(K2t) (9.86)

154

of periodic solutions of the delay equation

d

dtG(t) = −K2G

(

t− (4N + 1)

2K2

)

. (9.87)

Likewise, for each N ≥ 0, the point (α, β, τ) =(

0,±K2,− (3N+1)2K2

)

generates the two parameter family

G(t) = d1sin(K2t) + d2cos(K2t) (9.88)

of periodic solutions of the delay equation

d

dtG(t) = −K2G

(

t+(3N + 1)

2K2

)

. (9.89)

III. Complex eigenvalues. When λ = α + iβ is complex our goal is to show that infintely many

branches solutions of (9.73) exist. Our main result is

Theorem 9.1 For each integer N there is a branch ΓN of solutions (αN (τ), βN (τ)) of (9.73) which

bifurcate from

(α, β) = (0,K2) (9.90)

as τ passes through the critical value

τN =(2N + 1)π

2. (9.91)

The solutions (αN (τ), βN (τ)) exist for all τ ∈(

1eK2

,∞)

when N ≥ 0, and for all τ ∈ (−∞, 0) when

N < 0. Corresponding to each (αN (τ), βN (τ)) is a two family

G = c1eαN (τ)tcos(βN (τ)t) + c2e

αN (τ)tsin(βN (τ)t) (9.92)

of solutions of the delay equationd

dtG(t) = −K2G (t− τ) . (9.93)

Proof. The proof uses the Implicit Function Theorem. It follows from (9.73) that the matrix of partial

derivatives with respect to α and β of f and g is

A =

(

1 −K2τe−ατ cos(βτ) −K2τe

−ατsin(βτ)

K2τe−ατsin(βτ) 1 −K2τe

−ατ cos(βτ)

)

. (9.94)

Note that

det(A) = 1 when (α, β, τ) =

(

0,K2,(2N + 1)π

2K2

)

. (9.95)

Thus, for each integer N the Implicit Function Theorem implies that there is an ǫN > 0 such that if

τ ∈(

(2N+1)π2K2

− ǫN ,(2N+1)π

2K2+ ǫN

)

then there are unique C2 functions αN (τ), βN (τ) which satisfy

αN (τ) +K2e−αN (τ)τcos(βN (τ)τ) = 0,

βN (τ) −K2e−αN (τ)τsin(βN(τ)τ) = 0.

(9.96)

155

Let (τmin(N), τmax(N)) denote the the maximal interval of existence of (αN (τ), βN (τ)). The branch

ΓN of solutions of (9.73) which bifurcate from (α, β) = (0,K2) as τ passes through τN is defined by

ΓN = (αN (τ), βN (τ), τ)|τ ∈ (τmin(N), τmax(N)) . (9.97)

We need to show that

(τmin(N), τmax(N)) =

(

π

eK2,∞)

∀N ≥ 0, (9.98)

and

(τmin(N), τmax(N)) = (−∞, 0) ∀N ≤ −1. (9.99)

For simplicity, we prove (9.98) for the case N = 0. The details for all other N values are the same.

When N = 0 let (α(τ), β(τ)) = (α0(τ), β0(τ)), and (9.96) becomes

α(τ) +K2e−α(τ)τcos(β(τ)τ) = 0,

β(τ) −K2e−α(τ)τsin(β(τ)τ) = 0.

(9.100)

The maximal interval of existence of (α(τ), β(τ)) is denoted by (τmin, τmax) , where

τmin < τ0 =π

2K2< τmax. (9.101)

Differentiating (9.100) with respect to τ gives the ode’s

(1 + ατ)α′ − βτβ′ = β2 − α2, α( π2K2

) = 0,

βτα′ + (1 + ατ)β′ = −2αβ, β( π2K2

) = K2.

(9.102)

An algebraic manipulation transforms (9.102) into

α′ = 1(1+ατ)2+(βτ)2

(

β2 − α2 − αβ2τ − α3τ)

, α( π2K2

) = 0,

β′ = − β(1+ατ)2+(βτ)2

(

2α+ α2τ + β2τ)

, β( π2K2

) = K2.

(9.103)

To analyze the behavior of solutions of (9.103) we will use the auxiliary variables

Q(τ) = β(τ)τ and H(τ) = α(τ)τ + 1. (9.104)

It follows from (9.104) and (9.103) that Q and H satisfy

Q′ = Qτ(H2+Q2) , Q( π

2K2) = π

2 ,

H ′ = α(τ)HH2+Q2 +Q′Q, H( π

2K2) = 1.

(9.105)

Analysis of solutions of (9.103) on the interval π2K2

≤ τ < τmin.

Our goal here is to prove that

τmax = ∞, α(τ) > 0 and β(τ) > 0 ∀τ > π

2K2, and (α(τ), β(τ)) → (0, 0) as τ → ∞. (9.106)

156

We begin by proving that

τmax = ∞. (9.107)

First, note that the right side of both equations in (9.103) are continuously differentiable with respect

to α and β. Thus, if τmin <∞ then standard theory implies that

lim supτ→τ−

min

(|α′| + β′|) = ∞. (9.108)

The first step in showing that (9.108) cannot occur is to make use of Q. It follows from (9.105) that

Q′(τ) > 0 and Q(τ) ≥ π

2∀τ ∈

[

π

2K2, τmax

)

. (9.109)

From this we obtain the upper bound

1

(1 + ατ)2 + (βτ)2≤ 4

π2∀τ ∈

[

π

2K2, τmax

)

. (9.110)


τ = − 1

2αln

(

α2(τ) + β2(τ)

K22

)

> 0 ∀τ ∈(

π

2K2, τmax

)

. (9.111)

This implies that

α2 + β2 ≤ K22 ∀τ ∈

[

π

2K2, τmax

)

. (9.112)

It follows from (9.110), (9.112) and the first equation in (9.103) that

|α′| ≤ 4

π2

(

α2 + β2)

(1 + |α|τ) ≤ 4K22

π2(1 +K2τmax) . (9.113)

Similarly, it follows from (9.112) and the second equation in (9.103) that

|β′| ≤ 4|β|π2

(

|α| (2 + |α|τ) + β2τ)

≤ 4K22

π2(2 +K2τmax) . (9.114)

Thus, α′(τ) and β′(τ) are uniformly bounded on[

π2K2

, τmax

)

, hence (9.108) cannot hold. Therefore,

it must be the case that τmax = ∞, as claimed.

Estimates on the behavior of β(τ) and α(τ) when τ ≥ π2K2

.

From (9.103) we find that

α′(π

2K2) =

4K22

π2> 0 and β′(

π

2K2) = −2K2

2

π< 0. (9.115)

Thus, α(τ) > 0 and 0 < β(τ) < K2 for small τ − π2K2

> 0. Uniqueness of the solution β = 0 of the

second equation in (9.103) implies that

β(τ) > 0 ∀τ ∈ [π

2K2,∞). (9.116)

157

To obtain a sharper estimate for β(τ) we make use of the functional Q(τ). For this, we multiply the

first equation in (9.105) by Q and get

QQ′ =Q2

τ(H2 +Q2)≤ 1

τ. (9.117)

Integrating (9.117) from π2K2

to τ gives

K2 ≤ Q(τ)) ≤√

K22 + 2

(

ln(τ) − ln(π

2K2)

)

. (9.118)

Substituting Q(τ) = β(τ)τ into (9.118) gives the following upper and lower bounds on β(τ) :

K2

τ≤ β(τ)) ≤ 1

τ

√

K22 + 2

(

ln(τ) − ln(π

2K2)

)

∀τ ≥ π

2K2. (9.119)

We now determine the behavior of α(τ) when τ ≥ π2K2

. First, we claim that α(τ) > 0 ∀τ > π2K2

. If

not, there is a value τ > π2K2

such that

α(τ) > 0 ∀τ ∈ [π

2K2, τ ) and α(τ ) = 0. (9.120)

This and the second equation in (9.103) imply that

β′(τ) < 0 ∀τ ∈ [π

2K2, τ) and β(τ ) ∈ (0,K2). (9.121)

Substituting (α(τ), β(τ), τ) = (0, β(τ ), τ ) into (9.100) reduces (9.100) to

cos(β(τ )τ ) = 0,

β(τ ) = K2sin(β(τ )τ ).

(9.122)

From (9.122) it follows that β(τ )τ = (2N+1)π2K2

for some N > 0, and

β(τ ) = K2 or β(τ ) = −K2, (9.123)

which contradicts (9.121). Thus, we conclude that

α(τ) > 0 ∀τ ∈ [π

2K2,∞). (9.124)

Next, we use the functional H(τ) to obtain a precise upper bound estimate on α(τ). Combining the

two equations inin (9.105) leads to

H ′ =1

τ− H

τ(H2 +Q2)), H(0) = 1. (9.125)

From this it easily follows that suppose that α′(τ) = 0 at some τ > 0. The first equation in (9.103)

implies that

β2(1 − ατ) − α2(1 + ατ) = 0 when α′ = 0. (9.126)

158

Since α > 0, we conclude from (9.126) that

1 − ατ > 0 when α′ = 0. (9.127)

From this and the first equation in (9.103) it follows that

α′′ = 1(1+ατ)2+(βτ)2

(

2ββ′(1 − ατ) − α3)

< 0 when α′ = 0. (9.128)

Thus, there can be at most one zero of α′ on ( π2K2

,∞).

Analysis of solutions on the interval τmin < τ ≤ π2K2

.

Our goal here is to prove that

τmin =1

eK2and (α(τ), β(τ)) → (−eK2, 0) as τ → 1

eK2

+

. (9.129)

To prove (9.129) we make use of the auxiliary variables Qand H. From (9.109) and the first equation

in (9.100 ) we conclude that

α(τ) = −K2e−α(τ)τcos(Q(τ)) < 0 ∀τ ∈ (τmin,

π

2K2). (9.130)

It follows from (9.100) and (9.109) that

− 1

α(τ)τ=tan(β(τ)τ)

β(τ)τ=tan(Q(τ))

Q(τ)> 1 ∀τ ∈

(

τmin,π

2

)

. (9.131)


H(τ) = α(τ)τ + 1 ∈ (0, 1) ∀τ ∈(

τmin,π

2

)

. (9.132)

The estimates in (9.109) and (9.132) imply that

1

H2 +Q2≥ 4

π2 + 4∀τ ∈

(

τmin,π

2

)

. (9.133)

From this and the first equation in (9.105) we obtain

Q′

Q=

1

τ(H2 +Q2)≥ 4

τ(π2 + 4)∀τ ∈

(

τmin,π

2

)

. (9.134)

Integrating (9.134) from τ to π2K2

gives

ln(Q(τ)) ≤ ln

(

Q

(

π

2K2

))

+4

π2 + 4

(

ln(τ) − ln

(

π

2K2

))

∀τ ∈(

τmin,π

2

)

. (9.135)

We use (9.135) to prove that τmin ≥ 0. Suppose that τmin < 0. Since ln(τ) → −∞ as τ → 0+, it

follows from (9.135) that

ln(Q(τ)) → −∞ as τ → 0+, (9.136)

hence Q(0) = 0. However, this contradicts (9.109). Therefore it must be the case that τmin ≥ 0. Next,

we show that τmin > 0. Suppose, on the contrary, that

τmin = 0. (9.137)

159

We need to obtain a contradiction of (9.137). First, it follows from (9.109) and (9.135) that

ln(τ) → −∞ and Q(τ) → 0+ as τ → 0+. (9.138)


−α(τ)τ =Q(τ)

tan(Q(τ))→ 1 as τ → 0+. (9.139)

Thus,

α(τ) → −∞ as τ → 0+. (9.140)

However, from (9.140) and (9.75) we obtain

−α(τ)τ =1

2ln

(

α2 + β2

K22

)

→ ∞ as τ → 0+, (9.141)

contradicting (9.139). Therefore, it must be the case that

τmin > 0. (9.142)

Recall from () that

α(τ)τ → −1 and β(τ)τ → 0 as τ → τ+min. (9.143)

From this and (9.100) it follows that

α(τ) → −eK2 and β(τ) → 0 as τ → τ+min. (9.144)

Combining these properties with (9.75) gives

τmin = − 1

2α(τmin)ln

(

α2(τmin) + β2(τmin)

K22

)

=ln(e2)

2eK2=

1

eK2. (9.145)

160

10 Ito’s lemma for systems

(a.) Stament of Ito’s Lemma for systems.

We develop a two variable version following [23], pp. 45-46. Consider the system of SDE’s

dx1 = u1dt+ v11dB1 + v12dB2,

dx2 = u2dt+ v21dB1 + v22dB2.(10.1)

Here Bi denotes the Wiener process. In matrix notation system (22.75) is written as

dx(t) = udt+ vdB(t), (10.2)

where

x =

(

x1(t)

x2(t)

)

, u =

(

u1

u2

)

, v =

(

v11 v1,2

v2,1 v2,2

)

and dB =

(

dB1

dB1

)

. (10.3)

Let g(t, x) = (g1(t, x), g2(t, x)) be a C2 function of (t, x). Then

dg =

( ∂g1

∂t∂g2

∂t

)

dt+

( ∂g1

∂x1dx1 + ∂g1

∂x2dx2

∂g2

∂x1dx1 + ∂g2

∂x2dx2

)

+1

2

(

∑

i,j∂2g1

∂xi∂xjdxidxj

∑

i,j∂2g2

∂xi∂xjdxidxj

)

, (10.4)

where the rules for evaluating dxidxj are dtdt = dtdBk = dBkdt = 0, and dBidBj = δi,jdt.

(b.) Example 1.∫ t


∫ t

0AetAB(s)ds

To prove this integration by parts formula we let g(t, x) = etAx, where A is a two by two matrix, and

x =

(

x1(t)

x2(t)

)

satisfies dx = dB =

(

dB1

dB1

)

. Thus, x is an Ito process and can use the Ito Lemma for

systems as follows: a computation of each term in (22.78) shows that

dg = AetAx(t)dt + etAdx(t) (10.5)

Integrate both sides of (10.5) with respect to t and substitute x(t) = B(t), dx(t) = dB(t) and

g = etAB(t) into the result. This gives∫ t


∫ t

0AetAB(s)ds as required.

(c.) Example 2. Solve LQ′′ + RQ′ + 1CQ = G+ αζ(t)

We follow the appoach in [23], pp. 62-63 and transform the equation by setting

x =

(

x1(t)

x2(t)

)

=

(

Q

Q′

)

, A =

(

0 1

− 1CL −R

L

)

, H =

(

01LG

)

, K =

(

0αL

)

. (10.6)

Then x(t) satisfies

dx(t) = Ax+Hdt+KdB(t). (10.7)

Thus, (10.7) is an Ito system. To solve (10.7) multiply both sides by e−tA and rewrite it in the form

e−tAdx(t) − e−tAAx = e−tAHdt+ e−tAKdB(t). (10.8)

161

Now define g = e−tAx = e−tA

(

x1

x2

)

. Substitute g into

dg =

( ∂g1

∂t∂g2

∂t

)

dt+

( ∂g1

∂x1dx1 + ∂g1

∂x2dx2

∂g2

∂x1dx1 + ∂g2

∂x2dx2

)

+

(

∑

i,j∂2g1

∂xi∂xjdxidxj

∑

i,j∂2g2

∂xi∂xjdxidxj

)

, (10.9)

and obtain

dg = AetAx(t)dt+ etAdx(t). (10.10)

Substitute (10.10) into the left side of (10.8) and get

dg = e−tAHdt+ e−tAKdB(t). (10.11)

Integrate both sides of (10.11) from 0 to t and obtain

e−tAx(t) = x0 +

∫ t

0

e−sAHds+

∫ t

0

e−sAKdB(s). (10.12)

Finally, multiply both sides of (10.12) by etA, integrate∫ t

0 e−sAKdB(s) by parts using the formula

derived above in part (a.), and obtain

x(t) = etA

[

x0 + e−tAKB(s) +

∫ t

0

e−sA (H +AKB(s)) ds

]

. (10.13)

This differs from the answer in [23], p. 63, whre a minus sign replaces the plus sign in the integral

term on the right side of (10.13).

(d.) Find the SDE satisfied by Y = (cos(x), sin(x)) where dx(t) = dB(t).

This follows [23], p. 63. Substitute Y = (Y1, Y2) = g(t, x) = (g1, g2) = (cos(x), sin(x)) into

dg =

( ∂g1

∂t∂g2

∂t

)

dt+

( ∂g1

∂x1dx1 + ∂g1

∂x2dx2

∂g2

∂x1dx1 + ∂g2

∂x2dx2

)

+1

2

(

∑

i,j∂2g1

∂xi∂xjdxidxj

∑

i,j∂2g2

∂xi∂xjdxidxj

)

, (10.14)

and get(

dY1

dY2

)

=

(−sin(x)dx

cos(x)dx

)

+1

2

(−cos(x)−sin(x)

)

(dx)2 (10.15)

Substitute Y1 = cos(x), Y2 = sin(x), dx = dB and (dx)2 = dt, and get the Ito form

(

dY1

dY2

)

= −1

2

(

Y1

Y2

)

dt+

(−Y2

Y1

)

dB (10.16)

Dividing by dt gives the SDE form

( dY1

dt

dY2

dt

)

= −1

2

(

Y1

Y2

)

+

(−Y2

Y1

)

ζ(t) (10.17)

162

11 Exit time problems following Oksendal

Goal The goal of this first section on exit time problems is to understand techniques to figure out

the average time it takes for an event to occur. We follow Oksendal [23] and uses his notation.

(a.) Use of Ito’s Lemma to derive the generator A of X(t).

Assume that Xt (i.e Xt = X(t) in the notation given above) is a time homogeneous N-dimensional

Ito diffusion. This means ([23], Ch. VII, p. 104) that Xt ∈ RN satisfies a stochastic ode initial value

problem of the form

dXt = b(Xt)dt+ σ(Xt)dW, t ≥ s, Xs = x, (11.1)

where x ∈ RN , b(x) ∈ RN and σ(x) ∈ RN depend only on x, and satisfy

|b(x) − b(y)| + |σ(x) − σ(y)| ≤ D|x− y|, for some D > 0. (11.2)

Note: when N = 1 the associated probability transition density function ρ(x′, t|x, s) satisfies

ρ(x′, t|x, s)dx′ = PrXt ∈ (x′, x′ + dx′)|X(s) = x and ρ(x′, 0|x, 0) = δ(x′ − x). (11.3)

Inequality (11.2) guarantees uniqueness of the N-dimensional solution of (11.1). Oksendal lets this

unique solution be denoted by Xt = Xx,st When s = 0 he sets Xx

t = X0,xt . In particular, when s = 0

we have Xt = Xxt , and the generator ([23], p. 111) A of Xt is defined by

Af(x) = limt→0+

Ex[f(Xt)] − f(x)

t, x ∈ R and Xt ∈ RN , (11.4)

where f ∈ C20 (RN ). If N=1 we have

Ex[f(Xt)] = E[f(Xxt )] =

∫ ∞

−∞f(x′)ρ(x′, t|x, 0)dx′. (11.5)

The set of functions for which the limit in (11.4) exists for a specific x is denoted by DA(x), and DA

denotes the set of functions for which the limit exists for all x. Our goal is to prove that ([?], p. 113)

Af(x) =

N∑

i=1

bi(x)∂f

∂xi+

1

2

N∑

i=1

(σσT )i,j∂2f

∂xi∂xj, where f ∈ C2

0

(

RN)

. (11.6)

Remarks (i) When σσT is a diagonal matrix with the same entries σ2 then

Af(x) =

N∑

i=1

bi(x)∂f

∂xi+

1

2

N∑

i=1

σ2 ∂2f

∂x2i

. (11.7)

(ii) The big trick in proving (11.6) is to develop an integral representation for Af(x) inn which the

delta function is of use when t→ 0. Below this happens in (11.13)-(11.14)-(11.15).

Eq. (11.7) is the setting for the example in the next subsection When N = 1 Eq. (11.6) reduces to

Af(x) = b(x)fx(x) +σ2(x)

2fxx(x), where f ∈ C2

0

(

RN)

. (11.8)

163

We assume that N = 1 and derive (11.8 ). The general case is similar. Ito’s Lemma implies that

df(Xt) =

(

b(Xt)fx(Xt) +σ2(Xt)

2fxx(Xt)

)

dt+ σ(Xt, t)fx(Xt)dW (t). (11.9)

Integrate both sides of (11.9) from 0 to t and get

f(Xt) = f(x) +

∫ t

0

(

b(Xη)fx(Xη) +σ2(Xη)

2fxx(Xη)

)

dη +

∫ t

0

σ(Xη)fx(Xη)dW (η). (11.10)

Substitue (11.10) into the right side of (11.4) and get

Af(x) = limt→0+

Ex[f(Xt)] − f(x)

t= lim

t→0+

1

t

∫ t

0

Ex

[


2fxx(Xη)

]

dη (11.11)

Now use L’hopital’s rule and obtain

Af(x) = limt→0+

Ex

[


2fxx(Xη)

]

. (11.12)

Thus,

Af(x) = limt→0+

∫ ∞

−∞

(

b(x′)fx(x′) +σ2(x′)

2fxx(x′)

)

ρ(x′, t|x, 0)dx′. (11.13)

Recall from (11.3) that ρ(x′, 0|x, 0) = δ(x′ − x). Then (11.13) becomes

Af(x) =

∫ ∞

−∞

(

b(x′)fx(x′) +σ2(x′)

2fxx(x′)

)

δ(x′ − x)dx′, (11.14)

and this reduces to the formula (11.8), namely


2fxx(x). (11.15)

Remark. Below he makes use of the generalization of (11.15) to

Af(Xη) = b(Xη)fx(Xη) +σ2(Xη)

2fxx(Xη). (11.16)

(b.) Dynkin’s formula.

Let f ∈ C20 (RN ). Assume that X(t) satisfies SDE (11.1) with s = 0, that τ is the first time that X(t)

leaves a set, and that Ex(τ) <∞. Then Dynkin’s formula is

Ex[f(Xτ )] = f(x) + Ex

(∫ τ

0

Af(Xη)dη

)

, (11.17)

where

Af(Xη) =

N∑

i=1

bi(Xη)∂f

∂xi+

1

2

N∑

i=1

(σσT )i,j∂2f

∂xi∂xj, (11.18)

Priniciples guiding the use of Dynkin’s formula. In every situation we need f to have compact

support. In the two examples that follow this section the function f is chosen so that (i) f has

compact support, and (ii) Af(x) ≡ constant. In the 1st problem we compute an average exit time,

164

and for this the f is chosen so that the resulting constant is non-zero. In the 2nd problem where we

compute Probthe exit time τ <∞, f is chosen so that Af ≡ 0.

To derive (11.17) when N = 1 start with the formula

f(Xτ ) = f(x) +

∫ τ

0

(


2fxx(Xη)

)

dη +

∫ t

0

σ(Xη)fx(Xη)dW (η). (11.19)

Note that

Ex

(∫ t

0

σ(Xη)fx(Xη)dW (η)

)

=

∫ t

0

Ex[σ(Xη)fx(Xη)dW (η)]dη = 0. (11.20)

Now take Ex of both sides of (11.19), use (11.20) and obtain (11.17), as required.

(c.) The exit time from a ball in RN for Brownian motion (exit1.m).

This is from [26], p. 115. Let Bt be N-dimensional Brownian motion such that

B0 = a = (a1, a2, ..., aN ) ∈ RN . (11.21)

Assume that there is a first time τK where BτK hits the boundary of the ball

K = KR = x ∈ RN | |x| < R. (11.22)

To find Ea(τK) follow the following three step principle:

I. Define the related, monotone increasing sequence

σk = mink, τK. (11.23)

II. Use Dynkin’s formula, with Af ≡ constant 6= 0, to develop a corresponding sequence of related

equations, and then obtain an upper bound for Ea(σk).

III. Use the monotonicity of σk to obtain Ea(τK).

To accomplish all of this apply Dynkin’s formula (11.17) with f chosen such that Af ≡ constant 6= 0.

For this we let

X = B, x = a, τ = σk, f ∈ C20

(

RN)

such that f(x) = |x|2 =

N∑

i=1

x2i if |x| ≤ R. (11.24)

Define ∆f(x) =∑N

i=1∂2f∂x2

i

= 2N when f(x) = |x|2. Dynkin’s formula (11.17) for our f gives

Ea[f(Bσk)] = |a|2 + Ea

(∫ σk

0

1

2∆f(Bη)dη

)

. (11.25)

Equation (11.25) is the “related” equation. To make use of (11.25) note that f(Bη) = |Bη| since Bt

is in the ball K when 0 ≤ η ≤ σk ≤ τ. Thus,

Ea

(∫ σk

0

1

2∆f(Bη)dη

)

= Ea

(∫ σk

0

Ndη

)

= NEa(σk). (11.26)

165


Ea(σk) =Ea[f(Bσk

)] − |a|2N

. (11.27)

Since |Bσk|2 ≤ R2, it follows that

Ea(σk) =Ea(

|Bσk|2)

− |a|2N

≤ Ea(

|R|2)

− |a|2N

=|R|2 − |a|2

N(11.28)

The value of (11.28) is that it implies that the monotone increasing sequence Ea(σk) has a limit as

k → ∞. Also, we hopefully can prove

limk→∞

σk = τK . (11.29)

Note that f(BτK ) = R2. Combine this, (11.27) and (11.29) (i.e replace σk with τK in (11.27)), and

get

Ea(τK) =Ea[f(BτK )] − |a|2

N=Ea(R2) − |a|2

N=

R2 − |a|2N

. (11.30)

Remark. When N = 1 and a = 0 formula (11.30) predicts that E0(τK) = R2. It is interesting to

compare (11.30) with the following integration by parts formula developed in Sect 5, part (j):

∫ t

0

WdW (s) =W 2(t)

2− t

2. (11.31)

Setting t = τK and taking E0 of both sides gives

E0

(∫ τK

0

WdW (s)

)

= E0

(

W 2(τK)

2− τK

2

)

= E0

(

W 2(τK)

2

)

− E0(τK

2

)

. (11.32)

Since the term on the left is zero we conclude that

E0

(

W 2(τK)

2

)

− E0(τK

2

)

= 0. (11.33)

Since W 2(τK) = R2 and E0(

R2

2

)

= R2

2 , this further reduces to

E0 (τK) = R2, (11.34)

as predicted by (11.30). The matlab program exit1.m verifies this formula.

(d.) If |b| > R what is ProbBt ever hits the ball |x| = R ?This is from [26], p. 115. Let Bt be N-dimensional Brownian motion, let R > 0 be fixed, and let αk

denote the exit time from the annulus

Ak = x ∈ RN : R < |x| < 2kR ∀k ≥ 1. (11.35)

There are two cases to consider, N = 2 and N > 2. The method also seems to work when N = 1.

Case (i) N = 2 : When N = 2 we define f ∈ C20 (R2) by

f(x) = ln(|x|) if x ∈ Ak. (11.36)

166

The function f has been chosen so that Af(x) = ∆f(x) ≡ 0 when x ∈ Ak, and the domain of f gets

bigger as k increases. This reduces Dynkin’s formula (11.17) to

Eb (f(Bαk)) = ln(|b|) ∀k ≥ 1. (11.37)

Note that f(Bαk) = ln(R) or f(Bαk

) = ln(2kR). Thus,

Eb (f(Bαk)) = ln(R) OR Eb (f(Bαk

)) = ln(2kR). (11.38)

Let pk be the probability that |Bαk| = R, and let qk be the probability that |Bαk

| = 2kR, where

pk + qk = 1 ∀k ≥ 1.Combining these probabilities with (11.37) with (11.38), we conclude that

ln(R)pk + ln(2kR)qk = ln(|b|) ∀k ≥ 1. (11.39)

Solving for pk and qk gives

pk = 1 +ln(R) − ln(|b|)

kln(2)and qk =

ln(|b|) − ln(R)

kln(2)∀k ≥ 1. (11.40)

Thus (pk, qk) → (1, 0) as k → ∞. Also, when k = ∞ the outer part of the annulus is at R = ∞. From

these observations we conlclude that

ProbBt ever hits the ball |x| = R = 1. (11.41)

Remark. When N = 1 the same proof, with f(x) = x2, seems to give the same result.

Case (ii) N > 2 : When N > 2 we define f ∈ C20 (R2) by

f(x) = |x|2−N if x ∈ Ak. (11.42)

Again, f has been chosen so that Af(x) = ∆f(x) = 0 when x ∈ Ak, and the domain of f gets bigger

as k increases. Dynkin’s formula (11.17) gives

Eb (f(Bαk)) = |b|2−N ∀k ≥ 1. (11.43)

Note that f(Bαk) = R2−N or f(Bαk

) =(

2kR)2−N

. Thus,

Eb (f(Bαk)) = R2−N OR Eb (f(Bαk

)) =(

2kR)2−N

. (11.44)

Let pk be the probability that |Bαk| = R, and let qk be the probability that |Bαk

| = 2kR, where

pk + qk = 1 ∀k ≥ 1.Combining these probabilities with (11.43) with (11.44), we conclude that

R2−Npk +(

2kR)2−N

qk = |b|2−N ∀k ≥ 1. (11.45)

Since qk lies in [0, 1], it follows from (11.45) that

pk →( |b|R

)2−N

< 1 as k → ∞. (11.46)

167

Also, when k = ∞ the outer part of the annulus is at R = ∞. This, and (11.46) imply that

ProbBt ever hits the ball |x| = R =

( |b|R

)2−N

< 1. (11.47)

This result is qualitatively different from the cases N = 1 and N = 2.

(e.) The Backward Kolmogorov Equation - Oksendal version.

Assume that Xt satisfies a stochastic ode initial value problem of the form

dXt = b(Xt)dt+ σ(Xt)dW, t ≥ 0, X0 = x, (11.48)

Let f ∈ C20 (RN ). Then

f(Xτ ) = f(x) +

∫ τ

0

(


2fxx(Xη)

)

dη +

∫ t

0

σ(Xη)fx(Xη)dW (η). (11.49)

Define

u(t, x) = Ex[f(Xt)] = E[f(Xt)|Xt = x] (11.50)

Our goal is to prove that u(t, x) satisfies the backward Kolmogorov equation

∂u

∂t= Au, t > 0, u(0, x) = f(x), (11.51)

where A is the generator defined by

Af(x) = limt→0+

Ex[f(Xt)] − f(x)

t, x ∈ R, (11.52)

and satisfies


2fxx(x). (11.53)

Remark. The next section shows a different version of the backward Kolmogorov equation, where a

minus sign appears on the left side of (11.51), and the variable t decreases.

We will make use of the key identity

EXr [f(Xt)] = Ex[f(Xt+r)]. (11.54)

Property (11.54) follows from a uniqueness argument. We now follow ([23], p. 124), and let

g(x) = u(x, t) = Ex[f(Xt)]. (11.55)

From (11.52) and (11.55) it follows that

Ag(x) = limr→0+

Ex[g(Xr] − g(x)

r= lim

r→0+

Ex[EXr (f(Xt)] − Ex[f(Xt)])

r. (11.56)

Substitutting (11.54) into (11.56) gives

Ag(x) = limr→0+

Ex[Ex(f(Xt+r)] − Ex[f(Xt)]

r= lim

r→0+

Ex[f(Xt+r)] − Ex[f(Xt)]

r. (11.57)

168


Au = limr→0+

u(t+ r, x) − u(t, x)

r=∂u

∂t, (11.58)

and the proof is complete. Thus, if f ∈ C20 (RN ) then u(t, x) = Ex[f(Xt)] satisfies

∂

∂tEx[f(Xt)] = a(x)

∂

∂xEx[f(Xt] +

σ2(x)

2

∂2

∂x2Ex[f(Xt], (11.59)

and initial condition

Ex[f(X0)] = f(x). (11.60)

(f.) The Backward Kolmogorov Equation - Wikipedia version.

Assume that Xt satisfies a stochastic ode initial value problem of the form

dXt = b(Xt)dt+ σ(Xt)dW. (11.61)

According to Wikipedia the Bacward Kologorov equation associated with (11.61) is

− ∂

∂tp(x, t) = b(x)

∂

∂xp(x, t) +

1

2

∂2

∂x2σ2(x)p(x, t) (11.62)

where

t ≤ s and p(x, s) = us(x). (11.63)

(g.) The Feynman-Kac formula - Wikepedia version

This is from Wikipedia. Assume that Xt satisfies a stochastic ode initial value problem of the form


The starting point for the Fenman-Kac formula is the associated pde problem

− ∂

∂tf(x, t) = b(x)

∂

∂xf(x, t) +

1

2

∂2

∂x2σ2(x)f(x, t), f(x, T ) = ψ(x). (11.65)

Here, the pde is the same as the backwards Kolmogorov equation in the previous section since the

solution f(x, t) satisfies the ”terminal condition” f(x, T ) = ψ(x) instead of an intitial condition. The

Feynman-Kac formula for the solution of (11.65) is

f(x, t) = E[ψ(XT )|XT = x]. (11.66)

Does this make sense? Yes, when we use the identity

f(x, t) =

∫ ∞

−∞f(x′, T )ρ(x′, T |x, t)dx′ = E[f(x, T )|Xt = x]E[ψ(XT )|Xt = x]. (11.67)

Note that f(x, t) can also be written as

f(x, t) =

∫ ∞

−∞f(x′, t)δ(x′ − x)dx′ =

∫ ∞

−∞f(x′, t)ρ(x′, t|x, t)dx′ = E[f(x, t)|Xt = x] (11.68)

169

(h.) The Feynman-Kac formula - Oksendal version

This follows [23], p. 128. Let Xt satisfies the stochastic ode


Assume that f ∈ C20 (RN ) and q ∈ C(RN ), and define

v(x, t) = Ex[exp

(

−∫ t

0

q(Xs)ds

)

f(Xt)]. (11.70)

The claim in [23] is that v(x, t) is the Feynman-Kac formula for the solution of

∂v

∂t= Av − q(x)v, t > 0, x ∈ RN , (11.71)

v(x, 0) = f(x), x ∈ RN . (11.72)

The first step in proving (11.70) is to note that there are two random variables,

Yt = f(Xt) and Zt = exp(−∫ t

0

q(Xs)ds), (11.73)

such that

v(x, t) = Ex[ZtYt] = Ex[Ztf(Xt)], (11.74)

where Zt satisfies the SDE

dZt = −Ztq(Xt)dt. (11.75)

Now follow the steps of the proof of part (e.) above, beginning with

Av(x, t) = limr→0+

1

r(Ex[v(Xr, t)] − v(x, t)) . (11.76)

Combining (11.70), (11.73) and (11.76), we get


1

r

(

Ex[EXr [Ztf(Xt)] − Ex[Ztf(Xt))

. (11.77)

The first big trick here is to note that

Ex[EXr [Ztf(Xt)]] = Ex[Ex[f(Xt+r)exp

(

−∫ t

0

q(Xs+r)ds

)

]] (11.78)


Ex[EXr [Ztf(Xt)]] = Ex[f(Xt+r)exp

(

−∫ t

0

q(Xs+r

)

ds)]. (11.79)

We need to write the right side of (11.79) in a useful form. For this note that

−∫ t

0

q(Xs+r)ds = −∫ t+r

r

q(Xη)dη =

∫ r

0

q(Xs)ds−∫ t+r

0

q(Xs)ds. (11.80)

From this we conclude that

exp

(

−∫ t

0

q(Xs+r)ds

)

= exp

(∫ r

0

q(Xs)ds

)

exp

(

−∫ t+r

0

q(Xs)ds

)

. (11.81)

170

Thus,(11.79) becomes

Ex[EXr [Ztf(Xt)]] = Ex[exp

(∫ r

0

q(Xs)ds

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)]. (11.82)

Combine (11.82) with (11.77) and the definition of v(x, t) to get


1

r(v(x, t + r) − v(x, t))+

1

rEx[

(

exp

(∫ r

0

q(Xs)ds

)

− 1

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)].

(11.83)

Thus,

Av(x, t) =∂v

∂t+ lim

r→0+

1

rEx[

(

exp

(∫ r

0

q(Xs)ds

)

− 1

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)]. (11.84)

To obtain (11.70) it remains to prove that

limr→0+

1

rEx[

(

exp

(∫ r

0

q(Xs)ds

)

− 1

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)] = q(x)v(x, t). (11.85)

Applying L’hopital’s rule shows that the left side of (11.85 ) reduces to

limr→0+

Ex[q(Xr)exp

(∫ r

0

q(Xs)ds

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)] (11.86)

Since X0 = x it makes sense to conclude that

limr→0+

Ex[q(Xr)exp

(∫ r

0

q(Xs)ds

)

f(Xt+r)exp

(

−∫ t+r

0

q(Xs

)

ds)] = q(x)v(x, t) (11.87)

Another way to see if this makes sense is to take the following approach. First, note that

ρ(x1, t+ r, x2, r|x, 0) = ρ(x1, t+ r|x2, r)ρ(x2, r|x, 0). (11.88)

The derivation of (11.88 ) follows from combining the two identities

ρ(x1, t+r, x2, r, x, 0) = ρ(x1, t+r|x2, r, x, 0)ρ(x2, r, x, 0) = ρ(x1, t+r|x2, r)ρ(x2, r|x, 0)ρ(x, 0) (11.89)

and

ρ(x1, t+ r, x2, r, x, 0) = ρ(x1, t+ r, x2, r|x, 0)ρ(x, 0). (11.90)

From (11.88 ) we make the general conclusion

limr→0+

Ex[q(Xr)h(Xt + r)] = limr→0+

∫ ∞

−∞

∫ ∞

−∞h(x1)q(x2)ρ(x1, t+ r|x2, r)ρ(x1, r|x, 0)dx1dx2. (11.91)

Thus,

limr→0+

Ex[q(Xr)h(Xt+r)] =

∫ ∞

−∞

∫ ∞

−∞h(x1)q(x2)ρ(x1, t|x2, 0)δ(x−x1)dx1dx2 =

∫ ∞

−∞h(x1q(x)ρ(x1, t|x, 0)dx1.

(11.92)

Finally, we get

limr→0+

Ex[q(Xr)h(Xt + r)] =

∫ ∞

−∞h(x1)q(x)ρ(x1, t|x, 0)dx1 = q(x)Ex[h(Xt)]. (11.93)

Note that (11.93 ) gives a result like (11.87 ) as desired.

171

12 First passage time - Gardner version

(a.) Basic ODE for first passage time. Exercises

This follows Gardner ([11], pp.136-139). Assume that Xt satisfies

dXt = A(Xt)dt+ σ(Xt)dW, X0 = x ∈ [a, b], 0 < t < T, (12.1)

where σ(x) > 0, a < b are finite, and T = T (x) denotes the exit time from [a, b]. We restrict our

attention only to the interval [0, T ]. Following Gardner, we assume that the particle is removed if

X(t) reaches x = a or x = b, i.e. there is an ’absorbing’ barrier at x = a and x = b. Thus, if the exit

time is reached then the problem ends. Note that T ≥ 0 is a random variable, and T (a) = T (b) = 0.

However, T (x) > 0 when a < x < b, and in this case we observe that

a < Xt < b ∀t ∈ [0, T ), XT = a or XT = b. (12.2)

Gardner assumes that absorbing boundaries exist at x = a and x = b so that the particle is removed

from the system once it reaches either of these two points. In this case the solution of the corresponding

Fokker-Planck equation satisfies the boundary condition

ρ(x′, t|x, 0) = 0 if x′ = a or x′ = b, ∀t ≥ 0. (12.3)

This means (see the remarks in Gardner [11] following eq. (5.2.67) on p. 129) that the probability of

renetering (a, b) is zero once the boundary is reached.

Goals. Our goals are the following:

(i) Develop the pdf p(x, t) for T.

(ii) Use the pdf to find the ODE satisfied by M(x) = E[T (x)], namely

σ2(x)

2

d2M

dx2+A(x)

dM

dx= −1, M(a) = M(b) = 0. (12.4)

The first step is to link the cdf for T with the cdf for Xt by observing that

ProbT > t = Proba < Xt < b = G(x, t) =

∫ b

a

ρ(x′, t|x, 0)dx′, (12.5)

where ρ(x′, t|x, 0) satisfies the Fokker-Planck equation (i.e the forward Kolmogorov equation)

∂

∂tρ = − ∂

∂x′(A(x′)ρ) +

1

2

∂2

∂x′2(σ2(x′)ρ), ρ(x′, 0|x, 0) = δ(x′ − x), (12.6)

and also the backword Kolmogorov equation (Gardner [11], Eq. (5.2.143), p. 137)

∂

∂tρ = A(x)

∂

∂xρ+

σ2(x)

2

∂2

∂x2ρ, ρ(x′, 0|x, 0) = δ(x′ − x), (12.7)

The function

G(x, t) =

∫ b

a

ρ(x′, t|x, 0)dx′ (12.8)

172

is called the survival function. Below, we will make use of the property

G(x, 0) =

∫ b

a

ρ(x′, 0|x, 0)dx′ =

∫ b

a

δ(x′ − x)dx′ = 1. (12.9)

The cdf for T is defined by

ProbT ≤ t = 1 − ProbT > t = 1 −G(x, t) = 1 −∫ b

a

ρ(x′, t|x, 0)dx′. (12.10)

Let p(x, t) be the pdf for T. It is obtained by differentiating the cdf with respect to t, giving

p(x, t) =∂

∂tProbT ≤ t = −Gt(x, t) = −

∫ b

a

ρt(x′, t|x, 0)dx′. (12.11)

Therefore, the mean value of T is given by

E(T (x)) =

∫ ∞

0

t′p(x, t′)dt′ = −∫ ∞

0

t′Gt(x, t′)dt′ =

∫ ∞

0

G(x, t′)dt′ =

∫ ∞

0

∫ b

a

ρ(x′, t|x, 0)dx′dt′.

(12.12)

To find the ODE satisfied by E(T (x)), we note from (12.8)-(12.9) that

1 =

∫ ∞

0

−Gt(x, t′)dt′ = −

∫ ∞

0

∫ b

a

ρt(x′, t|x, 0)dx′dt′. (12.13)

Substituting the right side of (12.7) into the right side of (12.14) gives

1 = −∫ ∞

0

∫ b

a

[

A(x)∂

∂xρ(x′, t′|x, 0) +

σ2(x)

2

∂2

∂x2ρ(x′, t′|x, 0)

]

dx′dt′. (12.14)


1 = −A(x)d

dx

∫ ∞

0

∫ b

a

ρ(x′, t′|x, 0)dx′dt′ +σ2(x)

2

d2

dx2

∫ ∞

0

∫ b

a

ρ(x′, t′|x, 0)dx′dt′. (12.15)

Finally, define the function

M(x) = E(T (x)) =

∫ ∞

0

∫ b

a

ρ(x′, t′|x, 0)dx′dt′. (12.16)

It follows from (12.15) that M(x) satisfies the ODE

σ2(x)

2

d2M

dx2+A(x)

dM

dx= −1 (12.17)

If we have absorption at x = a and x = b then M satisfies the boundary conditions

M(a) = M(b) = 0. (12.18)

Another possibility is absorption at x = a and reflection at x = b, i.e.

M(a) = M ′(b) = 0. (12.19)

Exercises. First passage time exercises are in www.math.pitt.edu/∼ troy/stochastic/firstpassageexercises.pdf

173

(b.) Basic ODE for Nth moment.

This follows Gardner ([11], p. 138). The Nth moment of T = T (x) is defined by

MN (x) = E(TN(x)) =

∫ ∞

0

(t′)Np(x, t′)dt′ = −∫ ∞

0

(t′)NGt(x, t′)dt′. (12.20)

Recall that G(x, t) =∫ b

aρ(x′, t|x, 0)dx′. Then

MN (x) = −∫ ∞

0

(t′)NGt(x, t′)dt′ = −

∫ ∞

0

(t′)N

(

∫ b

a

ρt(x′, t|x, 0)dx′

)

dt′. (12.21)

If the N th moment MN (x) exists then it is reasonable to assume that

limt′→∞

(t′)NG(x, t′) = limt′→∞

(t′)NGt(x, t′) = 0. (12.22)

Goal. Our goal is to show that if conditions (12.22) hold then MN (x) satisfies the ODE

σ2(x)

2

d2MN

dx2+A(x)

dMN

dx= −NMN−1(x) ∀N ≥ 0. (12.23)

The first step is to note that

M0(x) = −∫ ∞

0

Gt(x, t′)dt′ = G(x, 0) =

∫ b

a

ρ(x′, 0|x, 0)dx′ =

∫ b

a

δ(x′ − x)dx′ = 1. (12.24)

Next, let k ≥ 1. An integration by parts shows that

Mk(x) = −∫ ∞

0

(t′)kGt(x, t′)dt′ =

∫ ∞

0

k(t′)k−1G(x, t′)dt′. (12.25)

Thus, we have the general identity

∫ ∞

0

(t′)k−1G(x, t′)dt′ =Mk(x)

k. (12.26)

Proceeding with the derivation of (12.23), we let N ≥ 1 and begin with an expression for MN−1(x).

It follows from (12.21 ) and the fact that G(x, t) =∫ b

aρ(x′, t|x, 0)dx′ that

MN−1(x) = −∫ ∞

0

(t′)N−1Gt(x, t′)dt′ = −

∫ ∞

0

(t′)N−1

∫ b

a

ρt(x′, t|x, 0)dx′ (12.27)

Substitute for ρt from (12.7) and get

MN−1(x) = −∫ ∞

0

(t′)N−1

∫ b

a

A(x)∂

∂xρ(x′, t′|x, 0)dx′dt′−

∫ ∞

0

(t′)N−1

∫ b

a

σ2(x)

2

∂2

∂x2ρ(x′, t′|x, 0)dx′dt′.

(12.28)

Therefore,

MN−1(x) = −A(x)d

dx

∫ ∞

0

(t′)N−1

∫ b

a

ρ(x′, t′|x, 0)dx′dt′−σ2(x)

2

d2

dx2

∫ ∞

0

(t′)N−1

∫ b

a

ρ(x′, t′|x, 0)dx′dt′.

(12.29)

174

Substituting∫ b

aρ(x′, t′|x, 0)dx′ = G(x, t′) reduces (12.29) to

MN−1(x) = −A(x)d

dx

∫ ∞

0

(t′)N−1G(x, t′)dt′ − σ2(x)

2

d2

dx2

∫ ∞

0

(t′)N−1G(x, t′)dt′. (12.30)

From (12.26) it follows that∫ ∞

0

(t′)N−1G(x, t′)dt′ =MN (x)

N. (12.31)

Finally, substitute (12.31) into the right side of (12.30) and obtain the desired ODE

NMN−1(x) = −A(x)dMN

dx− σ2(x)

2

d2MN

dx2. (12.32)

Remark. When N = 1 we combine (12.24) with (12.32) and obtain the ODE from the previous

section, namelyσ2(x)

2

d2M1

dx2+A(x)

dM1

dx= −1. (12.33)

(c.) Example I. 1st and 2nd moments for Brownian motion.

Our goal here is to compute the first and 2nd moments for Brownian motion with either absorbing or

reflecting boundary conditions. The ODE initial value problem is

dXt = σdW, X0 = x, (12.34)

Let L > 0 be fixed, and let T = T (x) denote the first time where XT = ±L.Absorbing boundary conditions. If we impose absorbing boundary conditions then the first

moment M1(x) = E(T (x)) satisfies

−1 =σ2

2

d2M1

dx2, M1(−L) = M1(L) = 0. (12.35)


M1(x) =L2 − x2

σ2∀x ∈ [−L,L]. (12.36)

Note that

M1(0) =L2

σ2, (12.37)

which agrees with (11.34) when σ = 1. The second moment M2(x) = E(T 2(x)) satisfies

−2M1(x) =σ2

2

d2M2

dx2, M2(L) = M2(−L) = 0. (12.38)


M2(x) =1

3σ4

(

x4 + 5L4 − L2x2)

(12.39)

For comparison we see that

M1(0) =L2

σ2and M2(0) =

5L4

6σ4. (12.40)

175

Reflecting and absorbing boundary conditions. If we impose an absorbing boundary condition

at x = −L and a reflecting conditon at x = L then the first moment M1(x) = E(T (x)) satisfies

−1 =σ2

2

d2M1

dx2, M1(−L) = M ′

1(L) = 0. (12.41)


M1(x) =1

σ2

(

3L2 − x2 + 2Lx)

∀x ∈ [−L,L]. (12.42)

Note that M1(0) = 3L2

σ2 . This is three times the size of M1(0) in (12.37).

176

(d.) Estimating integrals by the method of steepest descent.

In the next subsection we figure out the first passage time over a barrier. For this we need to know

how to estimate integrals of the form

∫ d

c

eMf(x′)dx′, M >> 1. (12.43)

We folllow the method of steepest descent outlined in Wikipedia. Assume that f ∈ C0(c, d), that f(x)

attains a global maximum (possible negative) at a unique x = x ∈ (c, d), and that

f(x) ≈ f(x) +f ′′(x)

2(x− x)2 when x ≈ x. (12.44)

Note that f ′′(x) < 0. Then

∫ d

c

eMf(x′)dx′ ≈∫ d

c

eM [f(x)+ f′′(x)2 (x′−x)2)]dx′ = eMf(x)

∫ d

c

e−M|f′′(x)|

2 (x′−x)2dx′. (12.45)

Now make the substitution v2 = M|f ′′(x)|2 (x′ − x)2 and obtain

∫ d

c

eMf(x′)dx′ ≈ eMf(x)

√

2

M |f ′′(x)|

∫ ∞

−∞e−v2

dv, M >> 1. (12.46)

This reduces to the final estimate

∫ d

c

eMf(x′)dx′ ≈ eMf(x)

√

2π

M |f ′′(x)| , M >> 1. (12.47)

Stirling’s formula. As an example, we use (12.47) to estimate

Γ(N + 1) =

∫ ∞

0

tNe−tdt, N >> 1 (12.48)

To put (12.48) into the form of (12.43) we let t = Nx. Then (12.48) becomes

Γ(N + 1) = NN+1

∫ ∞

0

xNe−Nxdx = NN+1

∫ ∞

0

eN(ln(x)−x)dx. (12.49)

Let

M = N, f(x) = ln(x) − x, f ′(x) =1

x− 1, f ′′(x) = − 1

x2. (12.50)

Thus, f ′(x) = 0 at x = 1 and (12.47) gives Stirling’s formula

Γ(N + 1) ≈ NN+1e−N

√

2π

N= NNe−N

√2πN, N >> 1. (12.51)

177

(e.) Escape over a double-well potential barrier.

This follows Gardner ([11], pp. 139-142). The SDE initial value problem is

dXt = −U ′(Xt)dt+√

2DdW, X0 = x. (12.52)

Assumptions: The “potential” function U(X) has exactly two relative minima, at X = a and X = c,

where a < c, a unique relative maximum at X = b ∈ (a, c), and U(X) → ∞ as |X | → ∞.

From these assumptions it follows that the deterministic equation

dXt = −U ′(Xt)dt (12.53)

has two stable constant soulutions X = a and X = c, which are separated by the stable solution

X = b. Thus, X = b acts as a “barrier” for solutions of (12.52).

Examples. One example is

dXt = (X −X3)dt+√

2DdW, X0 = x. (12.54)

Here

U ′(X) = X3 −X and U(X) =1

4

(

X2 − 1)2. (12.55)

Thus, a = −1, b = 0, c = 1 and

U(−1) = U(1) = 0 and U(0) =1

4. (12.56)

A second example is

dXt = −((X + 1)(X +1

2)(X − 1))dt+

√2DdW, X0 = x. (12.57)

Here

U ′(X) = (X + 1)(X +1

2)(X − 1) and U(X) =

X4

4+X3

6− X2

2− X

2. (12.58)

Thus, a = −1, b = − 12 , c = 1 and

U(−1) =1

12, U(1) = −13

12and U(0) =

23

8(24). (12.59)

For this example U(X) is similar to the function in Gardner ([11], Fig. 5.3, p. 140).

Goals. Our goals are the following:

(i) Let X0 = x ∈ (−∞, x0) in the stochastic equation (12.52), where x0 ∈ (b, c), and derive a formula

for the mean exit time for the solution to reach the barrier X = x0 when D is small.

(ii) Obtain an asymptotic estimate, as D → 0+, for escape time when x = a, the value where the

deterministic equation (12.53) has a stable solution.

178

Remarks. The probability density function ρ(x′, t|x, 0) associated with the solution of (12.52) satisfies

the Fokker-Planck equation

∂

∂tρ =

∂

∂x′(U ′(x′)ρ) +D

∂2ρ

∂x′2, ρ(x′, 0|x, 0) = δ(x′ − x). (12.60)

A stationary solution of (12.60) satisfies

0 =∂

∂x′(U ′(x′)ρ) +D

∂2ρ

∂x′2. (12.61)

The relevant stationary solution of (12.61) is

ρs(x′) = Nexp

(

−U(x′)

D

)

, (12.62)

where N is the constant value such that

N

∫ ∞

−∞exp

(

−U(η)

D

)

dη = 1. (12.63)

The function ρs(x′) has relative maxima at x = a and x = c, and a relative minimum at x′ = b. This

means that there is a relatively high probability of being to the left or right of b (i.e. the barrier) but

not near b.

The mean exit time is the solution of the problem

DM ′′ − U ′(x)M ′ = −1, −∞ < x < b, and M ′(−∞) = M(x0) = 0, (12.64)


M(x0) = 0 and limx→−∞

M ′(x)exp(−U(x)/D) = 0. (12.65)

The unique solution of (12.64)-(12.65) is

M(x) =1

D

∫ x0

x

exp

(

U(y)

D

)(∫ y

−∞exp

(

−U(η)

D

)

dη

)

dy. (12.66)

When x = a it follows from (12.66) that the mean exit time

M(a) =1

D

∫ x0

a

exp

(

U(y)

D

)(∫ y

−∞exp

(

−U(η)

D

)

dη

)

dy. (12.67)

Our goal is to show that

M(a) ≈ 2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

, 0 < D << 1. (12.68)

The first step in the derivation of (12.68) is to analyze the integral

∫ y

−∞exp

(−U(η)

D

)

dη, when y ≈ x0 and 0 < D << 1. (12.69)

179

The integrand f(η) = exp(−U(η)/D) has a unique maximum at η = a. Substitute

U(η) = U(a) +U ′′(a)

2D(η − a)2 (12.70)

into (12.69) and get

∫ y

−∞exp

(−U(η)

D

)

dη ≈ exp

(

−U(a)

D

)∫ y

−∞exp

(

−U′′(a)

2D(η − a)2

)

dη (12.71)

The substitution u2 = U ′′(a)2D (η − a)2 leads us to the approximation

∫ y

−∞exp

(−U(η)

D

)

dη ≈√

2D

U ′′(a)exp

(

−U(a)

D

)∫ ∞

−∞exp(−u2)du. (12.72)

Thus,∫ y

−∞exp

(−U(η)

D

)

dη ≈√

2πD

U ′′(a)exp

(

−U(a)

D

)

when 0 < D << 1. (12.73)

The second step in the derivation of (12.68) is to analyze the integral

∫ x0

a

exp

(

U(y)

D

)

dy, where 0 < D << 1. (12.74)

The integrand h(y) = exp(U(y)/D) has a unique maximum at x = b. Substitute

U(y) = U(b) +U ′′(b)

2D(η − b)2 (12.75)

into (12.74) and get

∫ x0

a

exp

(

U(y)

D

)

dy ≈ exp

(

U(b)

D

)∫ y

−∞exp

(

−|U ′′(b)|2D

(η − b)2)

dη. (12.76)

Thus,∫ x0

a

exp

(

U(y)

D

)

dy ≈√

2D

|U ′′(b)|exp(

U(b)

D

)∫ ∞

−∞exp(−u2)du, (12.77)

and therefore

∫ x0

a

exp

(

U(y)

D

)

dy ≈√

2πD

|U ′′(b)|exp(

U(b)

D

)

when 0 < D << 1. (12.78)

Finally, we multiply (12.73) and (12.78), and conclude from (12.67 ) that

M(a) ∼ 2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

, 0 < D << 1. (12.79)

Remarks.

(i) We did not have to restrict X0 to X0 = a in order to get the approximation (12.79). Just let x < b

be arbitrarily chosen so that we have formula (12.66), i.e.

M(x) =1

D

∫ x0

x

exp

(

U(y)

D

)(∫ y

−∞exp

(

−U(η)

D

)

dη

)

dy. (12.80)

180

Since x < b all of the estimates work and we get the asymptotic result

M(x) ≈ 2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

, 0 < D << 1. (12.81)

It is this “uniformity” in the asymptotics of M(x) that is used to prove that T is an exponential

random variable in the next section

(ii) Gardner ([11], p. 140) notes that (12.79) is the Arrhenius formula from chemical reaction theory.

(iii) A lower bound on M(a) it terms of x) is found by differentialing (12.67) twice with respect to

x0. This gives∂

∂x0M(a) =

1

Dexp

(

U(x0)

D

)(∫ x0

−∞exp

(

−U(η)

D

)

dη

)

. (12.82)

and∂2

∂x20

M(a) =1

D+

1

D2exp

(

U(x0)

D

)

U ′(x0)

(∫ x0

−∞exp

(

−U(η)

D

)

dη

)

. (12.83)

Eq. (12.83) reduces to∂2

∂x20

M(a) ≥ 1

D∀x0 ∈ [a, b], (12.84)

since U ′(x0) ≥ 0 when a ≤ x0 ≤ b. Integrating (12.84) gives

∂

∂x0M(a) ≥ x0 − a

Dand M(a) ≥ (x0 − a)2

D∀x0 ∈ [a, b]. (12.85)

In particular

M(a)|x0=b ≥(b− a)2

D. (12.86)

This shows that the mean passage time from X = a to X = b is large when D > 0 is small.

(iii) When x0 > c there is a contribution to mean passage time due to the relative minimum at x = c.

In this case a repitition of the arguments given above lead to

M(a) ≈ 2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

+2π

√

U ′′(c)|U ′′(b)|exp

(

U(b) − U(c)

D

)

, 0 < D << 1.

(12.87)

(f.) Why escape time is asymptotically an exponential random variable.

Our goal is to prove that, for the double-well problem, the escape time T fromX = a toX = x0 ∈ (b, c)

is a continuous random variable which satisfes the asymptotic estimates

Prob (T ≤ t) ∼ 1 − e−λt and fT (t) ∼ λe−λt as D → 0, (12.88)

where

λ =

(

Cexp

(

∆U

D

))−1

=

(

2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

)−1

. (12.89)

Remark. The first two moments of the exit time in Benzi’s paper on stochastic resonance match up

correctly with the moments for an exponential random variable.

181

I. The first step in the derivation of (12.88)-(12.89) is to use mathematical induction to prove that,

for each N ≥ 0, the moment MN (x) = E(TN (x)) satisfies

MN (x) = E(TN(x)) ∼ N !

(

Cexp

(

∆U

D

))N

as D → 0, (12.90)

where

C =2π

√

U ′′(a)|U ′′(b)|and ∆U = U(b) − U(a). (12.91)

Previously we showed that M0(x) = E(1) = 1. Also, at the end of the previous we showed that if

X0 = x ∈ [a, b) then

M1(x) ∼ Cexp

(

∆U

D

)

as D → 0. (12.92)

Induction hypothesis. Let N ≥ 1 and assume that

MN−1(x) = E(TN−1(x)) ∼ (N − 1)!

(

Cexp

(

∆U

D

))N−1

as D → 0. (12.93)

To determine the asymptotic formula for MN(x) = E(TN(x)), we solve the BVP

−NMN−1(x) = −U ′(x)dMN

dx+D

d2MN

dx2, M ′

N (∞) = MN(x0) = 0. (12.94)

Proceeding as in part (g.) above, we integrate twice and get

MN(x) =N

D

∫ x0

x

exp

(

U(y)

D

)(∫ y

−∞MN−1(η)exp

(

−U(η)

D

)

dη

)

dy. (12.95)

Susbstitute the estimate

MN−1(η) ∼ (N − 1)!

(

Cexp

(

∆U

D

))N−1

(12.96)

into the right side of (12.95) and get

MN (x) ∼ (N)!

(

Cexp

(

∆U

D

))N−11

D

∫ x0

x

exp

(

U(y)

D

)(∫ y

−∞exp

(

−U(η)

D

)

dη

)

dy. (12.97)

In the analysis of M1(x) we previously proved that

1

D

∫ x0

x

exp

(

U(y)

D

)(∫ y

−∞exp

(

−U(η)

D

)

dη

)

dy ∼ Cexp

(

∆U

D

)

as D → 0. (12.98)

Finally, combining (12.97) and (12.98) gives the desired estimate, namely

MN (x) ∼ (N)!

(

Cexp

(

∆U

D

))N

as D → 0. (12.99)

II. The second step in the derivation of (12.88)-(12.89) is to use the generating function method to

identify the what type of random variable the exit time T is. Recall (see [26], Ch. 4 - p. 126) that if

X is a random variable then

E(

etX)

= E

( ∞∑

N=0

1

N !XN tN

)

=

∞∑

N=0

1

N !E(XN)tN (12.100)

182

Let X be an expential random variable with parameter λ > 0. Then

E(

etX)

=

∫ ∞

−∞etxλe−λxdx =

∫ ∞

−∞λe(t−λ)xdx, 0 < t < λ. (12.101)

Thus, since 0 < t < λ, we get

E(

etX)

=λ

λ− t=

1

1 − tλ

=

∞∑

N=0

tN

λN(12.102)

It follows from uniqueness of generating functions, and equating the coefficients of tN on the right

sides of (12.100) and (12.102), that if X satisfies

E(

XN)

=N !

λN∀N ≥ 0, (12.103)

then X is an exponential random variable with parameter λ > 0. We have shown that the escape

time random variable T satisfies

E(

TN)

=N !

λN∀N ≥ 0, (12.104)

where

λ =

(

Cexp

(

∆U

D

))−1

=

(

2π√

U ′′(a)|U ′′(b)|exp

(

U(b) − U(a)

D

)

)−1

. (12.105)

Thus T must be an exponential random variable with parameter λ =(

Cexp(

∆UD

))−1.

183

(g.) The probability, and mean time, of exit thru a particular end of [a, b].

This follows Gardner ([11], pp.143-144). Assume that Xt satisfies

dXt = A(Xt)dt+√

B(Xt)dW, X0 = x ∈ (a, b). (12.106)

Let a < b be finite, and let T ≥ 0 be the random variable for the exit time from (a, b). Thus,

a < Xt < b ∀t ∈ [0, T ), XT = a or XT = b. (12.107)

Note that T is a function of x ∈ (a, b), and T (a) = T (b) = 0. Our goals are the following:

I. Determine πa(x) and πb(x), the probabilities that the particle exits (a, b) thru x = a or x = b.

II. Derive the the pdf’s for Ta and Tb, the times to exit (a, b) thru x = a and x = b,

III. Derive and solve the the ODE’s satisfied by E[Ta(x)] and E[Ta(x)].

I. The functions πa(x) and πb(x) are defined by

πa(x) = ProbT > 0 and XT (x) = a = ProbXT (x) = a (12.108)

and

πb(x) = ProbT > 0 and XT (x) = b = ProbXT (x) = b. (12.109)

The first step in finding πa(x) and πb(x) is to observe that

ProbT > t = Proba < Xt < b = G(x, t) =

∫ b

a

ρ(x′, t|x, 0)dx′, (12.110)

where ρ(x′, t|x, 0) satisfies the Fokker-Planck equation (i.e the forward Kolmogorov equation)

∂

∂tρ(x′, t|x, 0) = −

(

∂

∂x′(A(x′)ρ(x′, t|x, 0)) − 1

2

∂2

∂x′2(B(x′)ρ(x′, t|x, 0))

)

, ρ(x′, 0|x, 0) = δ(x′ − x).

(12.111)

and also the backword Kolmogorov equation

∂

∂tρ(x′, t|x, 0) = A(x)

∂

∂xρ(x′, t|x, 0) +

B(x)

2

∂2

∂x2ρ(x′, t|x, 0), ρ(x′, 0|x, 0) = δ(x′ − x). (12.112)

It follows from (12.111) that ρ(x′, t|x, 0)) satisfies

∂

∂tρ(x′, t|x, 0)) = − ∂

∂x′J(x′, t|x, 0), (12.113)

where

J(x′, t|x, 0) = A(x′)ρ(x′, t|x, 0) − 1

2

∂

∂x′(B(x′)ρ(x′, t|x, 0)), (12.114)

At t = 0 this reduces to

J(x′, 0|x, 0) = A(x′)δ(x − x′) − 1

2

∂

∂x′(B(x′)δ(x − x′)) = 0 ∀x ∈ (a, b). (12.115)

In particular,

J(a, 0|x, 0) = J(b, 0|x, 0) = 0 ∀x ∈ (a, b). (12.116)

184

In part III we make use of (12.116). Next, integration of (12.113) from x′ = a to x′ = b gives

∫ b

a

∂

∂tρ(x′, t|x, 0))dx′ = −J(b, t|x, 0) + J(a, t|x, 0). (12.117)

Integration of (12.117) from t′ = t to t′ = ∞ gives

∫ b

a

ρ(x′, t|x, 0)dx′ =

∫ ∞

t

J(b, t′|x, 0)dt′ −∫ ∞

t

J(a, t′|x, 0)dt′. (12.118)

Define

ga(x, t) = −∫ ∞

t

J(a, t′|x, 0)dt′ and gb(x, t) =

∫ ∞

t

J(b, t′|x, 0)dt′. (12.119)

Remark. The interpretation of ga(x, t), ga(x, 0), gb(x, t), gb(x, 0) are that

ga(x, t) = ProbT > t and XT = a, (12.120)

ga(x, 0) = ProbT > 0 and XT = a = ProbXT = a, (12.121)

gb(x, t) = ProbT > t and XT = b, (12.122)

gb(x, 0) = ProbT > 0 and XT = b = ProbXT = b. (12.123)

Thus,

πa(x) = ga(x, 0) = ProbXT = a and πb(x) = gb(x, 0) = ProbXT = b. (12.124)

Next, combine (12.118) and (12.119) and get

∫ b

a

ρ(x′, t|x, 0)dx′ = ga(x, t) + gb(x, t), x ∈ (a, b). (12.125)

Next, integrate the backward pde (12.112) from x′ = a to x′ = b and get

∫ b

a

∂

∂tρ(x′, t|x, 0))dx′ = A(x)

∂

∂x

∫ b

a

ρ(x′, t|x, 0)dx′ +B(x)

2

∂2

∂x2

∫ b

a

ρ(x′, t|x, 0)dx′. (12.126)

Substituting (12.125) into the right side of (12.126) gives

∫ b

a

∂

∂tρ(x′, t|x, 0))dx′ = A(x)

∂

∂x(ga(x, t) + gb(x, t)) +

B(x)

2

∂2

∂x2(ga(x, t) + gb(x, t)). (12.127)

Now substitute the right side of (12.118) into the left side of (12.127) and get

A(x)∂

∂x(ga(x, t) + gb(x, t)) +

B(x)

2

∂2

∂x2(ga(x, t) + gb(x, t)) = J(a, t|x, 0) − J(b, t|x, 0). (12.128)

We break this equation into two distinct equations by setting

A(x)∂

∂xga(x, t) +

B(x)

2

∂2

∂x2ga(x, t) = J(a, t|x, 0), (12.129)

185

and

A(x)∂

∂xgb(x, t) +

B(x)

2

∂2

∂x2gb(x, t) = J(b, t|x, 0). (12.130)

It follows from (12.129), (12.130) and (12.124) that πa(x) = ga(x, 0) and πb(x) = gb(x, 0) satisfy

A(x)∂

∂xπa(x) +

B(x)

2

∂2

∂x2πa(x) = 0, πa(a) = 1, πa(b) = 0. (12.131)

and

A(x)∂

∂xπb(x) +

B(x)

2

∂2

∂x2πb(x) = 0, πa(a) = 0, πa(b) = 1. (12.132)

By solving (12.131) and (12.132) we determine the probabilities that the particle leaves (a, b) thru

x = a or x = b. This completes goal I stated above.

II. To achieve goal II we need to determine the pdf’s for Ta and Tb, the exit tmes thru x = a and

x = b. The first step is to conclude from (12.110) that

ProbT ≤ t = 1 −∫ b

a

ρ(x′, t|x, 0)dx′. (12.133)

Let p(x, t) be the pdf for T. Then

p(x, t) =∂

∂tProbT ≤ t = −

∫ b

a

ρt(x′, t|x, 0)dx′ =

∫ b

a

∂

∂x′J(x′, t|x, 0). (12.134)

Completing the integration gives

p(x, t) = J(b, t|x, 0) − J(a, t|x, 0). (12.135)


∂

∂tga(x, t) = J(a, t|x, 0) and

∂

∂tgb(x, t) = J(b, t|x, 0). (12.136)

Before proceeding further we determine the pde’s satisfied by ga(x, t) and gb(x, t). These pde’s are

used in part II to find E(Ta(x)) and E(Tb(x)). Substituting (12.136) into (12.129) and (12.130) gives

A(x)∂

∂xga(x, t) +

B(x)

2

∂2

∂x2ga(x, t) =

∂

∂tga(x, t), (12.137)

and

A(x)∂

∂xgb(x, t) +

B(x)

2

∂2

∂x2gb(x, t) =

∂

∂tgb(x, t). (12.138)

We now continue with the derivation of the pdf’s for Ta and Tb. From (12.110) it follows that

ProbT > t = −∫ ∞

t

[J(a, t′|x, 0) − J(b, t|x, 0)dt′] . (12.139)

It from this equation that

ProbT > t and XT = a = ga(x, t) = −∫ ∞

t

J(a, t′|x, 0)dt′. (12.140)

186

Gardner ([11], p. 142) notes that

ProbT > 0 and XT = a = ProbXT = a = ga(x, 0). (12.141)

Thus,

πa(x) = ProbXT = a = ga(x, 0) = −∫ ∞

0

J(a, t′|x, 0)dt′, (12.142)

Equation (12.140) can be written as

ProbT > t|XT = aProbXT = a = ga(x, t) = −∫ ∞

t

J(a, t′|x, 0)dt′. (12.143)

Finally, define

ProbTa > t = ProbT > t|XT = a. (12.144)

Then (12.140)-(12.144) imply that

ProbTa > t =ga(x, t)

ga(x, 0), (12.145)

and therefore

ProbTa ≤ t = 1 − ga(x, t)

ga(x, 0). (12.146)

Thus the pdf for Ta is

pa(x, t) = − 1

ga(x, 0)

∂

∂tga(x, t). (12.147)

Likewise the pdf for Tb is

pb(x, t) = − 1

gb(x, 0)

∂

∂tgb(x, t). (12.148)

Remark. We have now determine the pdf’s for Ta and Tb which completes goal II.

III. Our third goal is to determine the mean values of Ta and Tb are given by

E(Ta(x)) =

∫ ∞

0

t′pa(x, t′)dt′ = −∫ ∞

0

t′

ga(x, 0)

∂

∂tga(x, t′)dt′ =

1

ga(x, 0)

∫ ∞

0

ga(x,′ )dt′ (12.149)

and

E(Tb(x)) =

∫ ∞

0

t′pb(x, t′)dt′ = −

∫ ∞

0

t′

gb(x, 0)

∂

∂tgb(x, t

′)dt′ =1

gb(x, 0)

∫ ∞

0

gb(x,′ )dt′. (12.150)

Since ga(x, 0) = πa(x) and gb(x, 0) = πb(x), we conclude from (12.149) and (12.150) that

∫ ∞

0

ga(x, t′)dt′ = E(Ta(x))πa(x) and

∫ ∞

0

gb(x, t′)dt′ = E(Tb(x))πa(x). (12.151)

Integration of (12.137) and (12.137) from t = 0 to t = ∞ gives

A(x)∂

∂x

∫ ∞

0

ga(x, t′)dt′ +B(x)

2

∂2

∂x2

∫ ∞

0

ga(x, t′)dt′ = −ga(x, 0) = −πa(x), (12.152)

and

A(x)∂

∂x

∫ ∞

0

gb(x, t′)dt′ +

B(x)

2

∂2

∂x2

∫ ∞

0

gb(x, t′)dt′ = −gb(x, 0) = −πb(x). (12.153)

187

Substitution of (12.151) into (12.152) and (12.153) gives

A(x)∂

∂x(E(Ta(x))πa(x))dt′ +

B(x)

2

∂2

∂x2(E(Ta(x))πa(x))dt′ = −πa(x), (12.154)

and

A(x)∂

∂x(E(Tb(x))πb(x))dt

′ +B(x)

2

∂2

∂x2(E(Tb(x))πb(x))dt

′ = −πb(x). (12.155)

These are the ODE’s to find E(Ta(x))πa(x) and E(Tb(x))πb(x). Thus, once we solve (12.131-(12.132)

and find πa(x) and πb(x), we can then find E(Ta(x)) and E(Tb(x)).

Example. Consider the Wiener process ODE

dXt = σdW, X0 = x ∈ (a, b). (12.156)

Thus A(x) = 0 and B(x) = σ2, and the function πa(x) satisfies

σ2

2

d2

dx2πa(x) = 0, πa(b) = 0, πa(a) = 1. (12.157)

The solution of this problem is

πa(x) =x− b

a− b∀x ∈ [a, b]. (12.158)

Note that πa(a+b2 ) = 1

2 as we would expect. To find E(Ta(x) we need to solve

σ2

2

d2

dx2(E(Ta(x))πa(x)) = −πa(x), E(Ta(a))πa(a) = E(Ta(b))πa(b) = 0. (12.159)

Substituting (12.158) into the right side of (12.159) reduces (12.159) to

σ2

2

d2

dx2(E(Ta(x))πa(x)) = −x− b

a− b, E(Ta(a))πa(a) = E(Ta(b))πa(b) = 0. (12.160)


E(Ta(x))πa(x) =(x− b)(a− b)2 − (x− b)3

3σ2(a− b), a ≤ x ≤ b. (12.161)

Finally, we conclude that

E(Ta(x)) =(a− b)2 − (x− b)2

3σ2, a ≤ x ≤ b. (12.162)

Note that

E(Ta(a+ b

2)) =

(a− b)2

4σ2. (12.163)

If a = L and b = −L then we would have

E(TL(0)) =L2

σ2. (12.164)

as we would expect.

(h.) Example: exit thru x = 0 or x = 1 in dU = γU(1 − U)dt+ σ√

U(1 − UdW.

188

We consider the SDE (see [5], p.246) initial value problem

dU = γU(1 − U)dt+ σ√

U(1 − UdW U0 = x ∈ (a, b). (12.165)

Recall from the last section that for an SDE of the form

dXt = A(Xt)dt+√

B(Xt)dW, X0 = x ∈ (a, b), (12.166)

the probabilities πa(x) and πb(x) of exiting thru x = a or x = b, respectively, satisfy

A(x)∂

∂xπa(x) +

B(x)

2

∂2

∂x2πa(x) = 0, πa(a) = 1, πa(b) = 0. (12.167)

and

A(x)∂

∂xπb(x) +

B(x)

2

∂2

∂x2πb(x) = 0, πa(a) = 0, πa(b) = 1. (12.168)

For equation (12.165) we observe that

A(x) = γx(1 − x) and B(x) = σ2x(1 − x), x ∈ (a, b) = (0, 1). (12.169)

Thus, the probability π0(x) pf passing thru x = 0 satisfies

γx(1 − x)∂

∂xπ0(x) +

σ2x(1 − x)

2

∂2

∂x2π0(x) = 0, π0(0) = 1, π0(1) = 0. (12.170)

In agreement with Doering et al [5], we find that the solution of this BVP is

π0(x) =e

2γ

σ2 (1−x) − 1

e2γ

σ2 − 1. (12.171)

Remarks. (i) The probability π1(x) of exit thru x = 1 is π1(x) = 1 − π0(x).

(ii) Thus, a solution must pass thru x = 0 or x = 1 no matter what the value of the ratio 2γσ2 is,

whereas in the extinction problem in part (f.) above this is not the case. That is, solutions of

dx(t)

dt= µx− x2 + σxpζ(t), X0 = x > 0. (12.172)

satisfy Xt = 0 for some finite time t > 0 if and only if 0 < p ≤ 12 .

(i.) Example: finite time extinction in dx(t)dt = µx− x2 + σxpζ(t).

We let p > 0 and consider the SDE

dx(t)

dt= µx− x2 + σxpζ(t), X0 = x > 0. (12.173)

The solution of (12.173) becomes extinct at a finite time T > 0 if XT = 0. Note that T is a random

variable. We develop analytical methods which are useful for the following:

I. Show that the mean exit time (i.e. mean extinction time) is infinite if 1 ≥ p > 12 and σ2 > 2µ.

II. Develop a formula for the mean exit time E(T (x)) when 0 < p ≤ 12 .

189

Part I. There are two cases: (i) p = 1, and (ii) 0 < p < 1.

Case (i.) p=1. Our method is to obtain the expression for mean exit time M(x) = E(T (x)), and

show that M(x) must be infinite when σ2 > 2µ. The function M(x) solves

(

µx− x2)

M ′(x) +σ2

2x2M ′′(x) = −1, M(0) = M ′(∞) = 0. (12.174)

An algebraic manipulation shows that (12.174) is equivalent to

M ′′(x) +2

σ2(µx−1 − 1)M ′(x) = − 2

σ2x−2, M(0) = M ′(∞) = 0. (12.175)

This equation can be written in the form

(

M ′(x)x2µ

σ2 e−2x

σ2

)′= − 2

σ2x

2µ

σ2 −2e−2x

σ2 , M(0) = M ′(∞) = 0. (12.176)

An integration from x to ∞ gives

M ′(x) =2

σ2x−

2µ

σ2 e2xσ2

∫ ∞

x

η2µ

σ2 −2e−2η

σ2 dη. (12.177)

Finally, we integrate (12.177) from 0 to x and get

M(x) =2

σ2

∫ x

0

t−2µ

σ2 e2tσ2

(∫ ∞

t

η2µ

σ2 −2e−2η

σ2 dη

)

dt. (12.178)

We need to show that the right side of (12.178) is infinite. First, note that 0 < t < x, hence

M(x) ≥ 2

σ2

∫ x

0

t−2µ

σ2 e2tσ2

(∫ x

t

η2µ

σ2 −2e−2η

σ2 dη

)

dt ≥ 2K

σ2

∫ x

0

t−2µ

σ2 e2tσ2

(∫ x

t

η2µ

σ2 −2dη

)

dt, (12.179)

where K = e−2x

σ2 . When σ2 > 2µ this reduces to

M(x) ≥ 2K

2µ− σ2

∫ x

0

t−2µ

σ2 e2t

σ2

(

x2µ

σ2 −1 − t2µ

σ2 −1)

dt = ∞ when σ2 > 2µ. (12.180)

When σ2 = 2µ inequality (12.179) reduces to

M(x) ≥ 2

σ2

∫ x

0

t−1e2t

σ2

(∫ x

t

η−1e−2η

σ2 dη

)

dt ≥ 2K

σ2

∫ x

0

t−1e2t

σ2

(∫ x

t

η−1dη

)

dt = ∞. (12.181)

We conclude that M(x) = E (T (x)) = ∞ ∀x > 0, hence finite time extinction does not happen.

Case (i) p 6= 1. The function M(x) solves

(

µx− x2)

M ′(x) +σ2

2x2pM ′′(x) = −1, M(0) = M ′(∞) = 0, (12.182)

or equivalently

M ′′(x) +2

σ2(µx1−2p − x2−2p)M ′(x) = − 2

σ2x−2p, M(0) = M ′(∞) = 0. (12.183)

Define the integrating factor

ψ(x) = exp

(

µ

σ2(1 − p)x2−2p − 2

σ2(3 − 2p)x3−2p

)

. (12.184)

190

Then (12.183) becomes

(ψ(x)M ′(x))′= − 2

σ2x−2pψ(x), M(0) = M ′(∞) = 0. (12.185)

Two integrations of this equation give

M(x) =2

σ2

∫ x

0

1

ψ(t)

(∫ ∞

t

η−2pψ(η)dη

)

dt. (12.186)

It follows from (12.186) that M(x) <∞ when 0 < p ≤ 12 .

(k.) Stochastic Resonance - Benzi paper ideas (benzipaper1.pdf).

This follows the stochastic resonance ideas in [1]. Can all this be extended to give period doubling,or

even more complicated behavior of the Acos(Ωt) is replaced with A1cos(Ω1t) +A2cos(Ω2t)?

Benzi considers the SDE

dXt = −(

X3t − aXt −Acos(Ωt)

)

dt+ ǫdWt X0 = x > 0. (12.187)

where a > 0 is fixed. When A = ǫ = 0 there are two stable fixed points

x1 = −√a and x2 =

√a (stable fixed points). (12.188)

These are separated by the unstable fixed point

x0 = 0 (unstable fixed point). (12.189)

Goal. The goal is to show that if 0 < A << a3/2 there is a range (ǫ1, ǫ2) of positive ǫ values such

that if ǫ1 < ǫ < ǫ2 then the system has a large peak in the power spectrum corresponding to a nearly

periodic behavior of Xt, with period nearly 2π/Ω, and amplitude 2√a. To show how this stochastic

resonance phenomenon occurs, Benzi begins by analyzing the special case

Case (i): A = 0. When A = 0 he lets τ1 and τ2 denote the exit times from the basin of attraction of

x1 = −√a and x2 =

√a, i.e.

τ1 = τ1(x) = inft > 0 : Xt = 0 and X0 = x ∈ (−∞, 0), (12.190)

τ2 = τ2(x) = inft > 0 : Xt = 0 and X0 = x ∈ (0,∞). (12.191)

Associated with these exit times are the mean values of their respective moments, i.e.

M iN = E

(

τNi

)

, i = 1, 2 and M i0 = 1. (12.192)

Below we show that the mean values of τ1 and τ2 satisfy

M11 (x) = M1

1 (−√a) ∼ π

a√

2exp

(

a2

2ǫ2

)

as ǫ→ 0+, (12.193)

191

M21 (x) = M2

1 (√a) ∼ π

a√

2exp

(

a2

2ǫ2

)

as ǫ→ 0+. (12.194)

Note that each of these estimates is uniform with respect to x. By analyzing the exit time ODE

ǫ2

2

d2MN

dx2+ (ax− x3)

dMN

dx= −NMN−1(x) (with appropriate B.C.′s), (12.195)

it is easy to prove that the second moments M i2 satisfy

M12 (x) = M1

2 (−√a) = 2

(

M11 (−

√a))2 ∼ π2

a2exp

(

a2

ǫ2

)

as ǫ→ 0+, (12.196)

M22 (x) = M2

2 (√a) = 2

(

M21 (√a))2 ∼ π2

a2exp

(

a2

ǫ2

)

as ǫ→ 0+. (12.197)

Each of these estimates is uniform with respect to x. This completes the analysis of the case A = 0.

Case (ii): 0 < A << a3/2. Benzi begins the analysis of this case by setting t = 0 and t = π/Ω

in (12.187), and studying the the behavior of solutions of the associated sub-systems

(t = 0) dXt = −(

X3t − aXt −A

)

dt+ ǫdWt, X0 = x, (12.198)

and

(t = π/Ω) dXt = −(

X3t − aXt +A

)

dt+ ǫdWt, X0 = x. (12.199)

Sub-system (12.198). Newton’s method shows that the sub-system

dXt = −(

X3t − aXt −A

)

dt+ ǫdWt, A > 0, (12.200)

has three constant solutions

x′1 ∼ −√a+

A

2a, x′0 ∼ −A

a, x′2 ∼

√a+

A

2aas A→ 0+. (12.201)

The associated potential function V (x) satisfies

V (x) =x4

4− a

2x2 −Ax, V ′(x) = x3 − ax−A. (12.202)

Then

V (x′1) ∼1

4

(

−√a+

A

2a

)4

− a

2

(

−√a+

A

2a

)2

−A

(

−√a+

A

2a

)

as A→ 0+. (12.203)

To first order in A this reduces to

V (x′1) ∼a2

4

(

1 − 2A

a3/2

)

− a2

2

(

1 − A

a3/2

)

+A√a as, (12.204)

hence

V (x′1) ∼ −a2

4

(

1 − 4A

a3/2

)

as A→ 0+. (12.205)

192

Also,

V (x′0) ∼A2

aas A→ 0+. (12.206)

For x′2 we find that

V (x′2) ∼1

4

(√a+

A

2a

)4

− a

2

(√a+

A

2a

)2

−A

(√a+

A

2a

)

as A→ 0+, (12.207)

V (x′2) ∼a2

4

(

1 +2A

a3/2

)

− a2

2

(

1 +A

a3/2

)

−A√a as A→ 0+, (12.208)

hence

V (x′2) ∼ −a2

4

(

1 +4A

a3/2

)

as A→ 0+. (12.209)

Therefore,

V (x′0) − V (x′1) ∼a2

4

(

1 − 4A

a3/2

)

and V (x′0) − V (x′2) ∼a2

4

(

1 +4A

a3/2

)

as A→ 0+. (12.210)

Next, observe that

V ′′(x′1) ∼ 2a, V ′′(x′0) ∼ −a and V ′′(x′2) ∼ +2a as A→ 0+. (12.211)

This implies that

√

|V ′′(x′0)V′′(x′1)| ∼ 2

√a and

√

|V ′′(x′0)V′′(x′2)| ∼ 2

√a as A→ 0+. (12.212)

The escape time from the left well is given by

µ(x′1) ∼2π

√

|V ′′(x0)V ′′(x1)|exp

(

2

ǫ2(V (x′0) − V (x′1))

)

as (ǫ, A) → (0, 0), (12.213)

and the escape time from the right well is given by

µ(x′2) ∼2π

√

|V ′′(x0)V ′′(x2)|exp

(

2

ǫ2(V (x′0) − V (x′2))

)

as (ǫ, A) → (0, 0). (12.214)

Our estimates show that escape time from the left well of

dXt = −(

X3t − aXt −A

)

dt+ ǫdWt, A > 0, (12.215)

is given by

µ(x′1) ∼π

a√

2exp

(

a2

2ǫ2

(

1 − 4A

a3/2

))

as (ǫ, A) → (0, 0), (12.216)

and the escape time from the right well is given by

µ(x′2) ∼π

a√

2exp

(

a2

2ǫ2

(

1 +4A

a3/2

))

as (ǫ, A) → (0, 0), (12.217)

When A = 0 the exit time from the left well, i.e. the basin of attraction of x1 = −√a, is

E(τ1) ∼π

a√

2exp

(

a2

2ǫ2

)

as ǫ→ 0+. (12.218)

193

Sub-system (12.199). Newton’s method shows that the sub-system

dXt = −(

X3t − aXt +A

)

dt+ ǫdWt, A > 0, (12.219)

has three constant solutions

x′′1 ∼ −√a− A

2a, x′′0 ∼ A

a, x′′2 ∼

√a− A

2aas A→ 0+. (12.220)

The associated potential function V (x) satisfies

V (x) =x4

4− a

2x2 +Ax, V ′(x) = x3 − ax+A. (12.221)

Then

V (x′′1 ) ∼ 1

4

(

−√a− A

2a

)4

− a

2

(

−√a− A

2a

)2

+A

(

−√a− A

2a

)

as A→ 0+. (12.222)

To first order in A this reduces to

V (x′′1 ) ∼ a2

4

(

1 +2A

a3/2

)

− a2

2

(

1 +A

a3/2

)

−A√a as, (12.223)

hence

V (x′′1 ) ∼ −a2

4

(

1 +4A

a3/2

)

as A→ 0+. (12.224)

Also,

V (x′′0 ) ∼ +A2

aas A→ 0+. (12.225)

For x′′2 we find that

V (x′′2 ) ∼ 1

4

(√a− A

2a

)4

− a

2

(√a− A

2a

)2

+A

(√a− A

2a

)

as A→ 0+, (12.226)

V (x′′2 ) ∼ a2

4

(

1 − 2A

a3/2

)

− a2

2

(

1 − A

a3/2

)

+A√a as A→ 0+, (12.227)

hence

V (x′′2 ) ∼ −a2

4

(

1 − 4A

a3/2

)

as A→ 0+. (12.228)

Therefore,

V (x′′0 ) − V (x′′1 ) ∼ a2

4

(

1 +4A

a3/2

)

and V (x′′0 ) − V (x′′2 ) ∼ a2

4

(

1 − 4A

a3/2

)

as A→ 0+. (12.229)

Next, observe that

V ′′(x′′1 ) ∼ 2a, V ′′(x′′0 ) ∼ −a and V ′′(x′′2 ) ∼ +2a as A→ 0+. (12.230)

This implies that

√

|V ′′(x′′0 )V ′′(x′′1 )| ∼ 2√a and

√

|V ′′(x′′0 )V ′′(x′′2 )| ∼ 2√a as A→ 0+. (12.231)

194

13 Large deviation theory - Freidlin-Wentzell version

(a.) Maier’s notes on exit time asymptotics

This part follows Maier’s notes [17, 18] (see santafewentzellnotes.pdf amd maier2.pdf). Consider the

system (see [17], pp. 6-7)

dxi = bi(x(t)) +∑

j

σi,j(x(t))√N

dwj(t), where x(0) = x0 = an attractor. (13.1)

Remark. Be sure that the attractor that the solution starts on at t = 0 does not satisfy σ(attractor) =

b(attractor) = 0, or else uniqueness prevents the solution from leaving the attractor even when noise

is present.

Let D = σσT and define (see [18], p.12, and also part (b.) below) the Lagrangian

L(x, x) =1

2[x− b(x)]TD−1(x)[x − b(x)]. (13.2)

For example, consider the systemdx1

dt = b1 + σ1√N

dW1

dt

dx2

dt = b2dt+σ1√N

dW2

dt .

(13.3)

Then D−1 is a square, diagonal matrix with diagonal elements 1σ21

and 1σ22, and

L(x, x) =1

2

(

1

σ21

(x1 − b1)2 +

1

σ21

(x2 − b1)2

)

. (13.4)

The mean value E(T ) of the time to exit a region U containing x(0) = x0 is ([17], pp. 6-7) is given by

E(T ) ∼ exp(NS0) as N → ∞, (13.5)

where

S0 = inf

∫

L(x(t), x(t))dt. (13.6)

The infinum is taken over all trajectories x(t) which begin at x0 and terminate on U. The ”action” S0

is defined in (13.13) in part (b.) below. The computation of S0 is motivated by Hamilton’s principle

which says that “the system undergoes the trajectory between t0 and t1 whose action is stationary.

Remarks. (I) First, one possible interpretaion of the minimization procedure described above is that

the mean time is the exponential of the minimal kinetic energy gernerated by white noise being fed

into the system, i.e.

Kinetic Energy ≡ 1

2

(

W 21 + W 2

2

)

= N1

2

(

1

σ21

(x1 − b1)2 +

1

σ21

(x2 − b1)2

)

. (13.7)

(II) Maier([17], p. 8) notes that in general one expects subdominant large-N asymptotics

E(T ) ∼ CNαexp(NS0) as N → ∞, (13.8)

195

where C and α are constants [19, 21, 22]. In the Section (c.) below we obtain similar formulas for a

one-dimensional problem where we can figure out the numbers exactly.

(b.) Links between the Hamiltonian, Lagrangian and Action integral.

The link between the Lagrangian L(x, x) and and Hamiltonian H(x, p) (i.e. type Lagrangian Mechan-

ics in google) is

H(qj , pj , t) =∑

i

qipi − L(qj , qj , t), (13.9)

where∂H

∂qj= −pj,

∂H

∂pj= qj ,

∂H

∂t= −∂L

∂t. (13.10)

For example, if we consider the equation

d2x

dt2+ x = 0, (13.11)

then p = x, and therefore

H =1

2

(

x2 + x2)

and L(x, x) =1

2

(

x2 − x2)

. (13.12)

The action is defined to be the integral

S(x(t)) =

∫ t1

t0

L(x, x)dη (13.13)

The computation of S0 in (13.8) in the previous section is motivated by Hamilton’s principle which

says that “the system undergoes the trajectory between t0 and t1 whose action is stationary.

(c.) Example: dXt = 2λXt +√

2N b(X)dWt

Our goal is to determine the large time behavior of the solution of the SDE initial value problem

dXt = 2λXt +

√

2

NdWt, X0 = 0. (13.14)

The 2’s in the equation are for only convenience. We assume that b(0) 6= 0 so that the solution will

not stay at X = 0 for all t > 0. For simplicity we let b(X) = 1 and consider the cases λ < 0 and λ = 0

separately.

Case (i) λ < 0.

In this case observe that Xt ≡ 0 is a globally stable solution of the deterministic equation

dXt = 2λXt. (13.15)

Our goal is to determine, when N >> 1, how long it will take the solution of the stochastic prob-

lem (13.14) to overcome the deterministic stability properties of the attractor X = 0, and leave the

interval (−∞, 1) thru X = 1. When N >> 1 the effect of the noise term is small, and therefore we ex-

pect that the exit time should be large. This is the type of problem that falls within Freidlin-Wentzell

196

theory of large deviations [10]. Fortunately, we can apply our one-dimensional method (see Section

10, part (a.)) and obtain an estimate for the exit time thru X = 1. In particular we show below that

the mean time, i.e. the exit time, for a solution to start at X = 0 and leave (−∞, 1) across X = 1 is

given by

Mean Exit Time ∼√

π

4|λ|3(

N−1/2e|λ|N)

, as N → ∞. (13.16)

Remark. This is the type of ”subdominant large-N asymptotic growh estimate” given above in (13.8)

by Maier [17] (see formula (9), p. 8 of santafewentzell.pdf), i.e.

Mean Exit time ∼ CNαeS0N , as N → ∞, (13.17)

where α = 12 , and S0 = 0.

The first step in the derivation of (13.16) is to solve

1

NM ′′(x) + 2λxM ′(x) = −1,−∞ < x < 1, (13.18)


M ′(−∞) = M(1) = 0. (13.19)

Write (13.18) as(

M ′(x)eλNx2)′

= −NeλNx2

. (13.20)

Integrating (13.20) from −∞ to x, and using the condition M ′(−∞) = 0, gives

M ′(x)eλNx2

= −N∫ x

−∞eλNt2dt, (13.21)

or equivalently,

M ′(x) = −Ne−λNx2

∫ x

−∞eλNt2dt. (13.22)

Next, integrate (13.22) from x to 1, using the condition M(1) = 0, and obtain

M(x) = N

∫ 1

x

e−λNs2

(∫ s

−∞eλNt2dt

)

ds. (13.23)

Thus, the exit time for the solution to start at X = 0 and leave the interval (−∞, 1) thru X = 1 is

given by

M(0) = N

∫ 1

0

e−λNs2

(∫ s

−∞eλNt2dt

)

ds. (13.24)

We now obtain an aymptotic estimate for the value of M(0) when N >> 1. First, estimate the inner

integral∫ s

−∞ eλNt2dt. The substitution u2 = |λ|Nt2 leads to

∫ s

−∞eλNt2dt =

√

1

|λ|N

∫

√|λ|Ns

−∞e−u2

du ∼√

π

|λ|N , N >> 1. (13.25)

197

Substituting (13.25) into (13.24) leads to

M(0) ∼√

πN

|λ|

∫ 1

0

e−λNs2

ds, N >> 1. (13.26)

Note that −λ = |λ|. The substitution v2 = |λ|Ns2 transforms (13.26) into

M(0) ∼√π

|λ|

∫

√|λ|N

0

ev2

dv, N >> 1. (13.27)

L’Hopital’s rule gives the desired asymptotic estimate

M(0) ∼√

π

4|λ|3(

N−1/2e|λ|N)

as N → ∞. (13.28)

Remarks. The associated action (see (13.8) and (13.13)) along a path from X = 0 to X = 1 is

S(x(t)) =

∫ T

0

L(x, x)dη =1

4

∫ T

0

(

Xt − 2λXt

)2

dη. (13.29)

We conclude from the above analysis that

S0 = inf

∫ T

0

L(x, x)dη = inf1

4

∫ T

0

(

Xt − 2λXt

)2

dη = |λ|. (13.30)

In general, is it possible that S0 = the value of the largest, or maybe the smallest, of the absolute

values of the eigenvalues? Try working it out for two systems that are independent so that the exit

time analysis is doable.

Case (ii) λ = 0.

In this case the deterministic equation becomes

dXt = 0. (13.31)

Note that X = 0 is no longer an attractor for the deterministic equation. Our goal is to determine,

when N >> 1, how long it will take the solution of the related stochastic problem

dXt =

√

2

NdWt, X0 = 0. (13.32)

to leave the interval (−L,L) thru X = ±L. When N >> 1 the effect of the noise term is small, and

therefore we expect that the exit time should be large. Again, this is the type of problem that falls

within Freidlin-Wentzell theory of large deviations [10], and we can apply our one-dimensional method

and obtain an estimate for the exit time thru X = ±L. In particular we show below that the mean

time, i.e. the exit time, for a solution to start at X = 0 and leave (−L,L) is given by

Mean Exit Time ∼ NL2

2(13.33)

198

Remark. This is the type of ”subdominant large-N asymptotic growh estimate” given above in (13.8),

with α = 1 and S0 = 0. Note that there is no attarctor to start from in (13.32). The papers always

seem to want to start the solution of the SDE on an attractor.

The first step in the derivation of (13.33) is to solve

1

NM ′′(x) = −1,−∞ < x < 1, (13.34)


M(−L) = M(L) = 0. (13.35)

Write the solution of (13.34) as

M(x) = −N2x2 + c1x+ c2. (13.36)

Conditions (13.35) imply that

M(x) = −N2x2 +N

L2

2. (13.37)

In particular, when x = 0 this gives

Mean Exit Time = M(0) = NL2

2. (13.38)

(d.) Muratov - 2 emails, and paper (Deville.pdf

This follows Deville, Vanden-Eijinden and Muratov [3] (see Deville.pdf). Consider the system

ǫdxdt = x− 1

3x3 − y,

dydt = x+ a+ δ2dW2.

(13.39)

They assume that

a >√

1 + 2ǫ. (13.40)

This causes the constant solution (x, y) = (−a,−a+3 /3) to be a stable node, and therefore solutions

won’t oscillate when they are close to the node. This gives solutions a greater chance to jump away

from the node and enter large amplitude oscillation mode when noise is present. To understand how

long it takes a solution to leave the vicinity of the stable node they rescale by setting

ζ = b−1(x+ a), η = b−2(y + a− a3/3), t = bs. (13.41)

Then (13.39) becomes, to “leading order,”

dζ = b2(−2ζ − η + ζ2)ds

dη = ζds+ δ2b−3/2dWs.

(13.42)

199

The constant solution (x, y) = (−a,−a+3 /3) rescales to

(ζ, η) = (0, 0). (13.43)

There is a unique trajectory of solutions of (13.42) which pass thru (ζ, η) = (2, 0). Let Γ denote the

component of this trajectory which lies in the region η < 0, ζ < 2. Note that if there is no noise then

a solution which lies to the right of Γ must satisfy ζ → ∞ in finite time. When noise is present we

need to estimate the meant time to reach Γ if a solution satisfies the initial condition

(ζ(0), η(0)) = (0, 0). (13.44)

To obtain the mean time estimate they use Freidlin-Wentzell theory of large deviations [10]. Suppose

that δ2b−3/2 satisfies

δ2b−3/2 → 0. (13.45)

Then they claim that τ, the mean value of the time it takes to reach Γ, satisfies

τ ∼ becb3δ−22 → ∞ as b→ 0, (13.46)

where c > 0 is a constant. The factor b indicates that τ is in terms of the original time t. According

to Muratov, ” τ = bexp(Smin/delta22) >> b(time is measured in the original units, hence a factor of

b). This does not seem quite accurate. I think they should have said that the mean time to reach Γ

satisfies

Mean time ∼ ecb3δ−22 → ∞ as δ2b

−3/2 → 0. (13.47)

First, he defines

Smin =1

2minpaths

(

b3∫ T

0

(ηs − ξ)2ds |ξ=b2ǫ−1(−2ξ − η + ξ2)

)

. (13.48)

Note that the integrand is identically zero whenever ηs = ξ (i.e. when η follows a deterministic

trajectory), so one essentially needs to find a minimizer going from the fixed point to the separatrix

of the deterministic system.

Is there a guiding principle to conclude that (ηs − ξ)2 is the integrand? His answer: Yes, it’s from

the Wiener measure of the noise. The easiest way to see how this comes about is by writing the path

integral formulation for the transition probability:

P (x(T ) ∈ A|x(0) = x0) =

∫

Dx(t)x(0)=x0,x(T )∈A)exp(−1/(2δ2)

∫ T

0

(ηs − ξ)2ds), (13.49)

which you can get by discretizing the SDE and taking into account that Delta W’s are independent

Gaussian random variables. Note that the large deviation principle exists also for more general

processes, even jump processes (see our recent JCP paper).

200

14 Continuum limits of Master equations.

(a.) Derivation of Ct = DCxx from C(x, t + τ) = λrC(x− ∆x, t) + λlC(x + ∆x, t).

This follows the derivation on p. 405 of [8]. Assume that particles move randomly with average step

length ∆t over unit time period τ. Let C(x, t)∆x = the number of particles in [x, x + ∆x] at time t.

Let λr =the probability that a particle moves to the right ∆x distance over the time period τ, and

λl =the probability that a particle moves to the left ∆x distance over the time period τ. The random

walk model describing the movement of the particles is

C(x, t+ τ) = λrC(x− ∆x, t) + λlC(x+ ∆x, t). (14.1)

To derive the Fokker-Planck equation Ct = DCxx from (14.1) we use the expansions

C(x, t + τ) = C(x, t) + Ct(x, t)τ, (14.2)

C(x + ∆x, t) = C(x, t) + Cx(x, t)∆x +1

2Cxx(x, t) (∆x)

2, (14.3)

C(x − ∆x, t) = C(x, t) − Cx(x, t)∆x +1

2Cxx(x, t) (∆x)2 . (14.4)

Substitute (14.2)-(14.4) into (14.1), assume that λl = λr = 12 , and obtain

Ct = DCxx where D =(∆x)

2

2τ= constant as τ → 0 and ∆x→ 0. (14.5)

That D is a constant implies that the average distance a particle moves during time τ is ∆x =√

2τD.

The SDE associated with Ct(x, t) = DCxx is

dX

dt=

√2Dζ(t). (14.6)

(b.) Derivation of the Fokker-Planck Eq. for dU = γU(1 − U)dt+ σ√

U(1 − UdW.

This follows [5]. We analyze (see p. 245 in the pdf file doeringphysica.pdf) the master equation

dpn

dt = −k1

Ω n(N − n)pn + k1

Ω (n− 1)(N − n+ 1)pn−1

− k2

Ω n(N − n)pn + k2

Ω (n+ 1)(N − n− 1)pn+1, n = 0, 1, . . . , N,

(14.7)

with boundary conditions given by

p−1 = pN+1 = 0. (14.8)

Define

u =n

Nand f(u, t) = pn(t), n = 0, 1, . . . , N. (14.9)

Our goal is to show that the Fokker-Planck equation associated with (14.7) is

∂f(u, t)

∂t=

∂

∂u

[

− (k1 − k2)N

Ωu(1 − u) +

1

2

(k1 + k2)

Ω

∂

∂uu(1 − u)

]

f(u, t), (14.10)

201


f(0, t) = f(1, t) = 0 ∀t ≥ 0. (14.11)

Remark (I). The function f(u, t) is the probability density function for the stochastic ode

dU = γU(1 − U)dt+ σ√

U(1 − UdW, (14.12)

where

γ =(k1 − k2)N

Ωand σ2 =

(k1 + k2)

Ω. (14.13)

The first step in deriving (14.10) is to define ∆u = 1N , substitute (14.9) into (14.7) and get

∂f(u,t)∂t = − k1

Ω(∆u)2u(1 − u)f(u, t) + k1

Ω(∆u)2 (u − ∆u)(1 − u+ ∆u)f(u− ∆u, t)

− k2

Ω(∆u)2 u(1 − u)f(u, t) + k2

Ω(∆u)2 (u+ ∆u)(1 − u− ∆u)f(u+ ∆u, t).

(14.14)

Next, define

g(u) = u(1 − u). (14.15)

Substitution of (14.15) into (14.14) gives

∂f(u,t)∂t = − k1

Ω(∆u)2 g(u)f(u, t) + k1

Ω(∆u)2 g(u− ∆u)f(u− ∆u, t)

− k2

Ω(∆u)2 g(u)f(u, t) + k2

Ω(∆u)2 g(u+ ∆u)f(u+ ∆u, t).

(14.16)

Remark (I). A random walk equation which is analagous to (14.1) in part (a.) above, and which

which approximates (14.16) is

f(u, t+ ∆t) = (1 − g(u))f(u, t) + λRg(u− ∆u)f(u− ∆u, t) + λLg(u+ ∆u)f(u+ ∆u, t), (14.17)

where

λR =k1∆t

Ω(∆u)2, λL =

k2∆t

Ω(∆u)2and

(∆u)2

∆t=k1 + k2

Ω. (14.18)

To see this we substitute (14.18) into (14.17) and obtain the approximation of (14.16) given by

f(u,t+∆t)−f(u,t)∆t = − k1

Ω(∆u)2 g(u)f(u, t) + k1

Ω(∆u)2 g(u− ∆u)f(u− ∆u, t)

− k2

Ω(∆u)2 g(u)f(u, t) + k2

Ω(∆u)2 g(u+ ∆u)f(u+ ∆u, t).

(14.19)

Continuing with the derivation of (14.10), we expand g(u−∆u)f(u−∆u, t) and g(u+∆u)f(u+∆u, t)

in Taylor series centered at (u, t) and keep terms up to order (∆u)2, and obtain

g(u− ∆u)f(u− ∆u, t) = (g(u) − g′(u)∆u+ g′′(u)2 )(∆u)2(f(u, t) − fu(u, t)∆u+ fuu(u,t)

2 (∆u)2

g(u+ ∆u)f(u+ ∆u, t) = (g(u) + g′(u)∆u+ g′′(u)2 )(∆u)2(f(u, t) + fu(u, t)∆u+ fuu(u,t)

2 (∆u)2.

(14.20)

Now substitue (14.20) into (14.16) and only keep terms up to order (∆u)2. This gives

∂f(u, t)

∂t=

∂

∂u

[

− (k1 − k2)

Ω∆ug(u) +

1

2

(k1 + k2)

Ω

∂

∂ug(u)

]

f(u, t). (14.21)

Setting g(u) = u(1 − u) and N = 1∆u transforms (14.21) into (14.10) and the derivation is complete.

202

15 Waiting time (i.e. time of first passage) problems

Goal The goal of this section is to understand techniques to figure out the average time it takes for

an event to occur, i.e. the waiting time. We’ll examine waiting times for problems involving

(i) Random walk ([26], Ch. 5): two difference equations

(ii) Poisson counting processes (e.g. the neuron firing problem in poissonneuron.pdf)

(iii) Example: the A→ B → C protein degradation process (see hnd1stPassage.pdf).

(iv) birth-death processes ([26], Ch. 9).

(v) The SDE Benzi et al stochastic resonance problem (see benzipaper.pdf).

(i.) Random walk. ([26], Ch. 5, p.173). For each N ≥ 1 let the random variable ZN satisfy

P (ZN = −1) = q and P (ZN = 1) = p where p > 0, q > 0 and p+ q = 1, (15.1)

The mean for ZN is

E(ZN ) = (1)P (ZN = 1) + (−1)P (ZN = −1) = (1)p+ (−1)q = p− q. (15.2)

Note that

E(Z2N ) = (1)2P (ZN = 1) + (−1)2P (ZN + −1) = p+ q = 1. (15.3)

Therefore the variance for ZN is

V ar(ZN ) = E(Z2N ) − (E(ZN ))

2= 1 − (p− q)2 = (p+ q)2 − (p− q)2 = 4pq. (15.4)

The random walk XN is defined by XN =∑N

k=1 Zk ∀N ≥ 1. The probability distriution function for

XN is (see [26], Ch. 5, p.176) given by

P (XN = m) =N !

(N − .5(N +m))!(.5(N +m)!p

(N+m)2 q

(N−m)2 ∀m ∈ [−N,N ]. (15.5)

To derive (15.5) we let N+ be the number of +1′s in the first N steps, and let N− denote the number

of −1′s. Then we have the two equations

N+ +N− = N and N+ −N− = XN . (15.6)

These eqations give

XN = m ⇐⇒ N+ =N +m

2. (15.7)

Since the number of +1’s is binomially distributed we conclude that

P (XN = k) = P (N+ =N + k

2) =

N !

(N − .5(N +m))!(.5(N +m)!p

(N+m)2 q

(N−m)2 . (15.8)

The mean and variance for XN are (see [26], p. 177)

E (XN ) =N∑

k=1

E (Zk) = N(p− q) ∀N ≥ 1, (15.9)

203

V ar (XN ) = V ar

(

N∑

k=1

Zk

)

=

N∑

k=1

V ar (Zk) = 4Npq ∀N ≥ 1, (15.10)

From (15.2)-(15.3) it follows([26], p. 178) that the autocorrelation is

E (XxXm) = min(n,m) + (nm−min(n,m))(p− q)2. (15.11)

Example 1. Let X0 = k ∈ 0, .., N. Find p(k) = P (Xm = 0 for some m ≥ 1|X0 = k). Assume that

absorption takes place at k = 0, and that Xi = 0 before Xi = N. Then p(k) satisies

p(k) = p(k + 1)p+ p(k − 1)q, p(0) = 1 and p(N) = 0. (15.12)

Let p(k) = λk. If p 6= 12 then λ1 = 1, λ2 = r = q

p and the solution is

p(k) =rk − rN

1 − rN. (15.13)

If p = q = 12 then λ1 = λ2 = 1 and

p(k) = 1 − k

N. (15.14)

Remark. We have only figured out the probability that absorption takes place, not how many steps

it takes to do so. To We figure out how many steps it takes in the next example.

Example 2: the waiting time problem. Suppose that absorption takes place at k = 0 or k = N

and let Tk denote the minimum number of steps m such that Xm = N or Xm = 0. Then T0 = TN = 0.

For each k ∈ 1, .., N − 1 find the average value of Tk. First, let

p(k,m) = Prob(Xm = N or Xm = 0, Xi 6= 0, Xi 6= N if i ≤ m− 1|X0 = k). (15.15)

Then p(k,m) satisfies the 2nd order difference equation

p(k,m) = p(k + 1,m− 1)p+ p(k − 1,m− 1)q (15.16)

p(k,m) = P (Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m|X0 = k)

= P (X1 = k + 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)P (X1 = k + 1|X0 = k)

+ P (X1 = k − 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)P (X1 = k − 1|X0 = k)

= P (X1 = k + 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)p

+ P (X1 = k − 1, Xm = N or Xm = 0, Xi 6= 0, Xi 6= N, 2 ≤ i ≤ m|X0 = k)q

= P (Xm−1 = N or Xm−1 = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m− 1|X0 = k + 1)p

+ P (Xm−1 = N or Xm−1 = 0, Xi 6= 0, Xi 6= N, 1 ≤ i ≤ m− 1|X0 = k − 1)q

= p(k + 1,m− 1)p+ p(k − 1,m− 1)q.

(15.17)

204

Observe that p(k,m) = P (Tk = m) Let Y (k) denote the average value of Tk. Then

Y (k) = E(Tk) =

∞∑

m=1

mp(k,m). (15.18)

Note that after one step we have the two-dimensional difference equation

p(k,m) = p(k + 1,m− 1)p+ p(k − 1,m− 1)q. (15.19)

Substitute (15.19) into (15.20) and get

Y (k) =

∞∑

m=1

m (p(k + 1,m− 1)p+ p(k − 1,m− 1)q) . (15.20)

Let i = m− 1. After some manipulation this equation reduces to ([26], pp. 193-194)

Y (k) = Y (k + 1)p+ Y (k − 1)q + 1, Y (0) = Y (1) = 0. (15.21)

The boundary conditions Y (0) = Y (N) = 0 means absorption is immedate. Proceeding as in Example

1, we let r = qp and get

Y (k) =1

q − p

(

k −N

[

1 − rk

1 − rN

])

if p 6= 1

2. (15.22)

and

Y (k) = k(N − k) if p = q =1

2. (15.23)

(ii.) Poisson processes

A random process X(t)|t ≥ 0 is a counting process if

(a) X(0) = 0, X(t) ≥ 0, X(t) is integer valued, and

(b) if s < t then X(s) ≤ X(t) and X(t) −X(s) = the number of events that have occured on (s, t).

A counting process X(t) is a Poisson process with rate λ > 0 if

(c) X(t) has stationary and independent increments (see Sec. I, part (g.) above), and

(c) the number of events in an interval of length t is Poisson distributed with mean λt, i.e. ∀s, t > 0

P X(t+ s) −X(s) = k = e−λt (λt)k

k!∀k ≥ 0. (15.24)

These properties imply that ([26], pp. 200-201) E (X(t)) = λt, V ar(X(t)) = λt, and

P X(t+ ∆t) −X(t) = 1 = λ∆t+ o(∆t),

P X(t+ ∆t) −X(t) ≥ 2] = o(δt).

A first passage time process.

The variable X(t) gives the number of events that occur on (0, t). Let T1 be the random variable that

denotes the first time that X = 1. It follows from this definiton and (20.1) that

P T1 > t = P X(t) = 0 = e−λt. (15.25)

205

Thus, the cumulative distribution function for T1 is

FT1(t) = P 0 < T1 < t = 1 − P T1 > t = 1 − e−λt. (15.26)

From this it follows that the probability density function for T1 is

fT1(t) =d

dtFT1(t) = λe−λt. (15.27)

Neuron example. Assume that a nerve impulse arrives at a synapse, and subsequently a corre-

sponding signal crosses over the synapse to contribute to the excitation of a neuron. Assume that

the arrival rate of the impulses is Posson distributed with rate λ = 1/10. The units of λ are 1/msec,

hence on average one impulse arrives at the synapse every 10 msec. Let X(t) denote the number of

impulses that have arrived by time t. Thus,

P (X(t+ τ) −X(t) = N) = exp(−λτ) (λτ)N

N !∀τ > 0 and N ≥ 0.

.

Problem 1. What is the average value of the arrival time of the N th impulse?

Solution: We need to compute E(TN), where TN denotes the time of arrival of the N th impluse.

The first method is to use the fact that the probability distribution for TN is ([26], p. 201)

fTN (t) =

λe−λt (λt)N−1

(N−1)! ∀t ≥ 0.

0 t < 0.(15.28)

Here is an alternative derivation of (15.28). Note that

P (TN > t) = P (X(t) ≤ N − 1) =

N−1∑

k=0

P (X(t) = k) =

N−1∑

k=0

e−λt (λt)k

k!, (15.29)

Thus, the cdf is

P (TN ≤ t) = 1 − P (X(t) ≤ N − 1) = 1 −N−1∑

k=0

e−λt (λt)k

k!. (15.30)

From this we get the pdf

fTN (t) =d

dt

(

1 −N−1∑

k=0

e−λt (λt)k

k!

)

= e−λt (λt)N−1

(N − 1)!. (15.31)

The average value of TN is

E(TN) =

∫ ∞

0

tλe−λt (λt)N−1

(N − 1)!dt =

N

λ. (15.32)

Problem 2. Assume that the neuron cannot fire until at least three impulses have arrived at the

synapse. What is the probability that the neuron fires in 60 msec.?

206

Solution: we need to compute the probability that at most two impulses arrive by 60 msec. since

the neuron cannot fire until three or more arrive. Thus, we compute

P (X(60) −X(0) ≤ 2)

Using the fact that X(0) = 0, we obtain

P (X(60) −X(0) ≤ 2) =

2∑

i=0

(P (X(60) −X(0) = i) =

2∑

i=0

exp(−λ60)(λ60)i

i!

Since λ = 110 , this reduces to

P (X(60)−X(0) ≤ 2) =

2∑

i=0

exp(−6)(6)i

i!= exp(−6)(1 + 6 + 18) = .0619

Problem 3. This is the “delay’ problem. How long does it take so that the probability the neuron

does not fire is equal to 12? Note that if t is small then the probability that the neuron does not fire

is pretty high. The amount of time needed is the dalay and is denoted by τ.

Solution: we need to compute the time t = τ > 0 such that probability that at most two impulses

arrive by τ msec is equal to 12 , i.e.

P (X(τ) −X(0) ≤ 2) =1

2.

Setting λ = 110 gives

P (X(τ) −X(0) ≤ 2) =

2∑

i=0

exp(−λτ) (λτ)i

i!= exp(− τ

10)(1 +

τ

10+

τ2

200) =

1

2

Using the matlab program delay.m, we find that τ ≈ 26.66

(c.) The cdf and pdf for the random variable Zi = Ti − Ti−1

Let X(t) be a Poisson process ∀t ≥ 0. Then Zi = Ti − Ti−1 represents the length of the ”inter-spike

interval” (Ti−1, Ti). Our first goal is to derive an integral representation for the cdf of the non-negative

Zi. After that we find the pdf. We will make use of the expansion

P (A) =

∞∑

i=1

P (A ∩Ai) =

∞∑

i=1

P (A|Ai)P (Ai), (15.33)

Let ∆τ > 0 be small and fixed, set τ0 = 0 and define the partition points τk = τk−1 + ∆τ and Ik =

[τk−1, τk−1 + ∆τ ] ∀k ≥ 1. Then [0,∞) = ∪k≥1Ik and we conclude that

P (Zi > z) ≈∑∞k=1 P (Zi > z and Ti−1 ∈ Ik)

=∑∞

k=1 P (Zi > z|Ti−1 ∈ Ik)P (Ti−1 ∈ Ik)

≈∑∞k=1 P (Zi > z|Ti−1 = τk)fTi−1(τk)∆τ.

(15.34)

207

Letting ∆τ → 0, we conclude from (15.34) that

P (Zi > z) =

∫ ∞

0

P (Zi > z|Ti−1 = τ)fTi−1 (τ)dτ. (15.35)

To evaluate the right side of (15.35) note that

P (Zi > z|Ti = τ) = P (Ti > Ti−1 + z = τ + z ≥ τ = Ti−1)

= P (X(τ + z) −X(τ) = 0)

= P (X(z) = 0) = e−λz.

(15.36)


P (Zi > z) =

∫ ∞

0

e−λzfTi−1(τ)dτ = e−λz

∫ ∞

0

fZ1(τ)dτ = e−λz, (15.37)

since∫∞0 fZ1(τ)dτ = 1. It follows from (15.37) that

P (Zi ≤ z) = 1 − e−λz and fZi(z) =d

dzP (Zi ≤ z) = λe−λz. (15.38)

Remark. The pdf fZi(z) = λe−λz decreases. The pdf in the next example is not strictly decreasing.

(d.) The first passage time for the reaction A→ B → C.

This follows hnd1stPassage.pdf. Assume that A, B, and C are proteins, that A decays at poisson

rate α into B, and that B decays at poisson rate β into C. This means that the lifetimes τA and τB

are exponentially distributed, e.g.

FA(t) = P0 < τA < t = 1 − e−αt and pA(t) = F ′A(t) = αe−αt.

The waiting time, or ”first passage time” for C is the time it takes C to form. The goal is to find the

average waiting time for C. For this we first need to compute pC(t)dt using

pC(t)dt = Pwaiting time for C falls in (t, t+ dt)

= Pcombined lifetime of A and B falls in (t, t+ dt)

= PτA + τB ∈ (t, t+ dt).

(15.39)

Note that

(1.) A disappears at an unknown intermediate time τ. We have to take into account every possiblity.

(2.) if B disppears (i.e. C appears) in (t, t+ dt) then the lifetime of B is t− τ.

These two events are independent since the protein has no memory. Therefore, to take into account

all possible values of τ we construct the intermediate probability

P (t, τ) = PτA ∈ (τ, τ + dτ) AND τB ∈ (t− τ, t− τ + dt)

= PτA ∈ (τ, τ + dτ)PτB ∈ (t− τ, t− τ + dt)

= (pA(τ)dτ) (pB(t− τ)dt)

= αe−ατdτβe−β(t−τ)dt.

(15.40)

208

Now add up the intermediate contribution to pC(t)dt from each τ and get

pC(t)dt =

(∫ t

0

αe−ατβe−β(t−τ)dτ

)

dt (15.41)

After division by dt we get

pC(t) =

∫ t

0

αe−ατβe−β(t−τ)dτ =αβ

α− β

(

e−βt − e−αt)

if α 6= β, (15.42)

pC(t)dt = α2te−αt if α = β. (15.43)

Here pc(t) is the distrubiton of first passage times. The average first passage time is

E(τC) =

∫ ∞

0

tpC(t)dt =1

α+

1

βif α 6= β, (15.44)

E(τC) =

∫ ∞

0

tpC(t)dt =2

αif α = β. (15.45)

We now use a second method, following part (c) above, to develop the equation (15.41) for pC(t).

Here we use the finite expansion

P (A) =

N∑

i=1

P (A ∩Ai) =

N∑

i=1

P (A|Ai)P (Ai), (15.46)

Let ∆τ = tN > 0, set τ0 = 0 and define the partition points τk = τk−1 + h and Ik = [τk−1, τk−1 +

h] ∀k ≥ 1. Then [0, t) = ∪k≥1Ik and we conclude that

PC(t)dt ≈∑Nk=1 P (τA ∈ [τk−1, τk] and τB ∈ [t− τA, t− τA + dt])

=∑N

k=1 P (τA ∈ [τk−1, τk])P (τB ∈ [t− τA, t− τA + dt])

≈∑Nk=1 pA(τk−1)∆τpB(t− τk−1)dt.

(15.47)

Let N → ∞ so that ∆τ → 0 and, as required, the approximation (15.47) converges to

PC(t)dt =

∫ t

0

pA(τ)pB(t− τ)dτdt, (15.48)

Remark. Suppose that the reaction is A→ B → G→ C. Then pC(t) satisfies

pC(t) =

∫ t

τ=0

∫ t−τ

ζ=0

αe−ατβe−βζγe−γ(t−τ−ζ)dζdτ. (15.49)

(d.) Birth-death processes This follows [26] (Ch. 9). Let X(t) be a queuing system which is

a Markov process. Define pN (t) = P (X(t) = N) . Assume the birth rate and death rate probabilities

P (X (t+ ∆t = N |X(t) = N − 1) =

∫ t+∆t

t

aN−1dt, P (X(t+ ∆t) = N |X(t) = N+1) =

∫ t+∆t

t

dN+1dt

(15.50)

209

The difference equations satisfied by pN are

pN (t+ ∆t) = aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 − aN−1∆t)(1 − dN+1)pN (t) ∀N ≥ 1, (15.51)

p0(t+ ∆t) = dN+1∆tp1(t) + (1 − d0)p0(t). (15.52)

A brief derivation of (15.51) is

pN(t+ ∆t) = P (X(t+ ∆t) = N)

= P (X(t+ ∆t) = N |X(t) = N + 1)P (X(t) = N + 1)

+ P (X(t+ ∆t) = N |X(t) = N − 1)P (X(t) = N + 1)

+ P (X(t+ ∆t) = N |X(t) = N)P (X(t) = N)

=(

∫ t+∆t

t aN−1dt)

pN−1(t) +(

∫ t+∆t

t dN+1dt)

pN+1(t) + P (X(t+ ∆t) = N |X(t) = N − 1)pN (t)

= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + P (X(t+ ∆t) = N |X(t) = N − 1)pN (t)

= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 −∫ t+∆t

taN−1dt)(1 −

∫ t+∆t

tdN+1dt)pN (t)

= aN−1∆tpN−1(t) + dN+1∆tpN+1(t) + (1 − aN−1∆t)(1 − dN+1∆t)pN (t)

(15.53)

Subtratc pN(t)∆t from both sides of (15.51), divide both sides by ∆t, let ∆t → 0, and get the ODE

p′N(t) = aN−1pN−1(t) + dN+1pN+1(t) − (aN−1 + dN+1)pN (t) ∀N ≥ 1. (15.54)

Likewise,

p′0(t) = d1p1(t) − (a0 + d0)p0(t). (15.55)

16 Time of first passage for discrete problems

(a.) Random walk: how to find probability that XN = 0 for some N > 0.

The first step in understanding random walks in one diemsnion is to follow [26], Ch. 5, Problem 5.2, p.

73. Define the set Zi, i = 1, ..., N of independent and identically distributed random variables which

have only two values, ±1, as follows: P (Zi = 1) = p and P (Zi = −1) = q = 1 − p, where 0 < p < 1.

17 White noise

(a.) Definition of white noise. A continuous time WSS random variable X(t) is called white

noise if

µX(t) = 0 and RX(τ) = σ2δ(τ) ∀τ ∈ R. (17.1)

In this case the frequency power spectrum SX(ω) satisfies

SX(ω) =

∫ ∞

−∞σ2δ(τ)dτ = σ2 ∀ω ∈ R. (17.2)

210

Because SX(ω) ≡ σ2, all frequencies contribute equally to power, hence X(t) is white noise.

(b.) Example: uniform white noise. Let X be a uniform random variable on [a, b]. The

probability density function and cumulative distribution are (see p. 45 of Schaum’s)

fX(x) =

0 for x < a,1

b−a for a ≤ x ≤ b,

0 for x > b

(17.3)

FX(x) =

0 for x < a,x−ab−a for a ≤ x ≤ b,

1 for x > b

(17.4)

The mean µX and variance σ2 of X satisfy

µX = E(X) =

∫ ∞

−∞xfX(x)dx

∫ b

a

x

b− adx =

1

2(b+ a), (17.5)

To define uniform white noise we extend all of this to the bivariate case. Let X and Y be independent

uniform random variables on [−1, 1] with joint probability distribution

fXY (x, y) =

14 for − 1 ≤ x, y ≤ 1,

0 otherwise.(17.6)

We claim that X and Y are independent. This is true if (definition on p. 80 of Schaum’s) FXY (x, y) =

FX(x)FY (y). To see this we first compute

FXY (x, y) =

∫ x

−∞

∫ y

−∞fXY (x, y)dxdy =

0 if x < −1 or y < −1,14 (x+ 1)(y + 1) if (x, y) ∈ [−1, 1]x[−1, 1],12 (x+ 1) if − 1 < x < 1 and y > 1,12 (y + 1) if − x > 1 and − 1 < y < 1,

1 if x > 1 and y > 1,

(17.7)

From (17.4) it follows that FX(x)FY (y) satisfy

FX(x)FY (y) =

∫ x

−∞fX(x)dx

∫ y

−∞fY (y)dy =

0 if x < −1 or y < −1,14 (x+ 1)(y + 1) if (x, y) ∈ [−1, 1]x[−1, 1],12 (x+ 1) if − 1 < x < 1 and y > 1,12 (y + 1) if − x > 1 and − 1 < y < 1,

1 if x > 1 and y > 1,

(17.8)

Alternatively, it is easily verified that

fXY (x, y) = fX(x)fY (y) ∀(x, y) ∈ R2, (17.9)

211

also implies that X and Y are independent.

Next, from (??) we conclude that E(X) = E(Y ) = 0. From (17.6) it follows that

E(XY ) =

∫ ∞

−∞

∫ ∞

−∞fXY (x, y)dxdy =

∫ 1

−1

∫ 1

−1

xy

4dxdy = 0 (17.10)

Thus, it must be the case that

E(XY ) = E(X)E(Y ) = 0. (17.11)

To define uniform white noise, let X(t) denote a uniform random variable on [−1, 1] such that X(t)

and X(s) are independent when t 6= s. It follows from (??) that

E(X(t)) = µX(t) = 0 and E(X2(t)) =1

3= σ2 ∀t. (17.12)

Since X(t) and X(s) are independent when t 6= s, then the autocorellation function RX(t, s) satisfies

RX(t, t+ τ) =

E(X(t)X(t+ τ)) = E(X(t))E(X(t+ τ)) = 0 if τ 6= 0,

E(X2(t)) = σ2 if τ = 0.(17.13)

That is

RX(t, t+ τ) = σ2δ(τ) ∀τ ∈ R. (17.14)

From (5.5), (17.12) and (17.14) follows that X(t) satisfies the definition of white noise.

(c.) Computing RX(τ) and SX(ω) for uniform white noise.

To compute RX(τ) and SX(ω) for uniform white noise use the matlab programs white.m and Rxest.m

18 Research ideas and papers to get

(a.) Feynman extension Could we make use of Feynman’s bm probability method to see self

organization evolve in a system of single-cells as the number of cells increases? His ideas are shownn

in ([9], see the diagram on p. 120). Would his cancelling out idea somehow work? How to set it up

so an anlaogy to his cancelling “angles” occurs? Wouyld some kind of multiplicative noise help?

(b.) Oscillations in predator-prey systems. Start with a dead system and give it multiplicative

noise to bring it to life. The key is to add on an ode for the σ term. Same for a system of genes with

a combination of multiplicative/additive noise? How to get multiplicative noise to produce a steady

state that was not there before - speciation example?

(c.) Neuro papers: Brunel-Hakim, Amit-Brunel, Brunel

Brunel & Hakim Neuro comp 1999

Amit & Brunel Cerebral cortex 1997

Brunel J. Comp. Neuroscience 1997

212

(d.) alternans. Start with a benzi system and hook it to a second system where the ǫ obeys an

equation putting it at one level for the regulat oscillations, and at a second level it gives the period

doubling oscillation, e.g.

ǫ′ = ǫ(1 − ǫ)(1 + ǫ) + σζ

19 Use of the steady state pde

The following principles lead to information about X(t) by analyzing the pde:

(1) find the time dependent pde

(2) solve the time independent steady state pde

(3) Evaluate ”average value of X” by figuring out E(X) for the steady state pde

(4) Evaluate ”average value of X2” by figuring out E(X2) for the steadystate pde.

(5) Evaluate “ average displacement” as Langevin did for Einstein’s result did by computing√

E(X2).

213

20 Population growth in a randomly varying environement.

(a.) The problem. This follows [14]. Consider the discrete population growth model

Nt+1 = ltNt, t = 1, 2, ... (20.1)

where the li are identically distributed, independent random variables (this will allow the central

limit theorem to apply), whose pdf is denoted by f(l). The solution of (20.1) is

Nt = N0

t∏

i=1

li. (20.2)

The mean and variance of l are given by

λ = µl = E(l) =

∫ ∞

−∞lf(l)dl and σl

2 = E(l2) − (E(l))2 (20.3)

Then

E (Nt) = N0

t∏

i=1

E(li) = N0λt, (20.4)

and from this we conclude that

E (Nt) → ∞ as t→ ∞ if λ > 1, (20.5)

E (Nt) → 0 as t→ ∞ if 0 < λ < 1. (20.6)

Extinction occurs if, for each K1 > 0,

Prob Nt > K1 → 0 as t→ ∞. (20.7)

When λ > 1 we know from (20.5) that the mean of Nt satisfies E (Nt) → ∞ as t → ∞. The problem

is to determine whether extinction can occur when λ > 1.

(b.) Put the extinction problem into the Central Limit Theorem setting.

To determine whether extinction occurs, derive a formula for the probability that Nt lies between two

given numbers K1 > 0 and K2 > 0. First, because ln(x) is an increasing function it is easy to prove

the following important connection between Nt and ln(Nt) :

Prob K1 ≤ Nt ≤ K2 = Probln(K1) ≤ ln(Nt) ≤ ln(K3) . (20.8)

Combining this equation with the fact that

ln(Nt) = ln(N0) +

t∑

i=1

ln(li), (20.9)

we obtain

ProbK1 < Nt < K2 = Prob

ln

(

K1

N0

)

≤t∑

i=1

ln(li) ≤ ln

(

K2

N0

)

. (20.10)

214

Division by t gives


1

tln

(

K1

N0

)

≤ 1

t

t∑

i=1

ln(li) ≤1

tln

(

K2

N0

)

. (20.11)

Subtract µln(l) from each quantity, divide by σln(l)/√t. This leads to


τ1(t) <1t

∑ti=1 ln(li) − µln(l)

σln(l)/√t

< τ2(t)

, (20.12)

where

τ1(t) =

1t ln(

K1

N0

)

− µln(l)

σln(l)/√t

and τ2(t) =

1t ln(

K2

N0

)

− µln(l)

σln(l)/√t

. (20.13)

The right side of (20.12) is now set up to apply the Cental Limit Theorem, which gives

Prob K1 < Nt < K2 ≈ Prob τ1(t) < τ < τ2(t) when t >> 1. (20.14)

where τ = N(0, 1).

(c.) Conditions which imply that E(Xt) → ∞, yet extinction occurs.

Let K1 > 0 and K2 = ∞, and assume that the positive random variable l satisfies

λ = µl = E(l) > 1 and µln(l) = E(ln(l)) < 0. (20.15)

Then τ2 = ∞ and (20.14) becomes

Prob K1 < Nt <∞ ≈ Prob τ1(t) < τ <∞ when t >> 1. (20.16)

It follows from (20.13) and (20.15) that

τ1(t) → ∞ as t→ ∞. (20.17)

This and (20.16) imply that

E(Xt) = X0λt as as t→ ∞, (20.18)

and

Prob K1 < Nt <∞ ≈ Probτ1(t) < τ <∞ → 0 as t→ ∞. (20.19)

Thus, the mean of Xt tends to +∞, yet extinction occurs when conditions (20.15) hold. It remains to

determine conditions under which µln(l) = E(ln(l)) < 0. For this we make use of the approximation

ln(l) ≈ ln(λ) +(l − λ)

λ− 1

2λ2(l − λ)2 when l ≈ λ. (20.20)

Note that

E(l − λ) = E(l) − λ = 0 and E(l − λ)2 = E(l2) − λ2 = σ2l . (20.21)

215

Combining (20.20) and (20.21), we obtain

E(ln(l)) ≈ ln(λ) − σ2l

2λ2< 0 when σl > λ

√

2ln(λ) (20.22)

Important observation. Note that CV, the coefficient of variation, is given by

CV =σ

λ. (20.23)

Thus, when λ1 > is small, we have

E(ln(l)) ≈ ln(λ) − (CV)2

2< 0 when CV =

σ

λ>√

2ln(λ) (20.24)

Thus, when λ − 1 > 0 is small and CV >√

2ln(λ) then X(t) goes extinct and E(X(t)) → ∞ as

t→ ∞.

(d.) Proof that ρ =(

∏ti=1 li

)1/t

≤ λ = 1t

∑ti=1 li when li ≥ 0.

Our goal is to prove that the geometric mean over a cycle of l′is of length t is is less than or equal to

their arithmetic mean. These are defined by

ρ =

(

t∏

i=1

li

)1/t

and λ =1

t

t∑

i=1

li. (20.25)

Note that1

λ

t∑

i=1

li = t. (20.26)

Thus,

1 = exp

(

1

λ

t∑

i=1

li − t

)

=t∏

i=1

exp

(

liλ− 1

)

. (20.27)

Use the inequaltiy exp(x) ≥ x+ 1 and obtain

1 =t∏

i=1

exp

(

liλ− 1

)

≥t∏

i=1

liλ

=1

λt

t∏

i=1

li. (20.28)


λt ≥t∏

i=1

li, (20.29)

hence

λ ≥ ρ =

(

t∏

i=1

li

)1/t

. (20.30)

This completes the proof.

(e.) A periodic sequence li such that ρ < 1 < λ and (Xt, Xt) → (0, 0) and as t→ ∞.

Define a periodic sequence li, of period 10, by

l1 = l1 = .. = l9 = 1.1, l10 = .3 (20.31)

216

Over a cycle of length 10 we have

ρ =

(

10∏

i=1

li

)1/10

= .71/10 = .96 and λ =1

10

10∑

i=1

li = 1.02 (20.32)

At t = 10N we have

X10N = .710NX0 → 0 as N → ∞, (20.33)

and also

Xt → ∞ as t→ ∞. (20.34)

Furthermore, we find that

σ2l = E(l2) − E2(l) =

1

10(l21 + ..+ l210) − λ2 = 1.098 − 1.022 = .0576 (20.35)

and therefore,

E(ln(λ)) ≈ ln(λ) − σ2l

2λ= .0198− .0576

2.02= −.0087 (20.36)

According to the theory developed above, this implies that the solution goes to extinction in spite of

the fact that the mean value of the l′is is greater than 1.

Figure 27: Left panel: X vs. t Right panel: Arithmetic mean of X vs. t. ρ = .96 and λ = 1.02

(f.) A periodic sequence li such that ρ = 1 < λ, Xt is periodic, Xt → constant as t→ ∞.

Define a periodic sequence li, of period 10, by

l1 = l1 = .. = l9 = 1.1, l10 = 1.1−9 (20.37)

Over a cycle of length 10 we have

ρ =

(

10∏

i=1

li

)1/10

= 1 and λ =1

10

10∑

i=1

li = 1.032 (20.38)

The solution is periodic (see Figure 28), and its mean approaches a constant at t→ ∞, again in spite

of the fact that the mean of the l′is is greater than 1.

217

Figure 28: Left panel: X vs. t Right panel: Arithmetic mean of X vs. t. Here ρ = 1 and λ = 1.032

21 A noisy genetic swith model

This follows [28]. The model is

x′ = −ax+ y + σ1ζ1(t)x, x(0) > 0,

y′ = x2

1+x2 − by, y(0) = 0

b′ = (b− b1)(b − b2)(b3 − b) + σ2ζ2(t), b(0) = b3.

(21.1)

where x is protein concentration, y is messenger RNA concentration, and a, b are parameters. Here

I’ve assumed that b is variable and subject to noise, 0 < b1 < b2 < b3, and b3 is chosen so that if

b ≡ b3 then (0, 0) is the only constant solution of the (x, y) system. Also, b1 > 0 is small so that if

b ≡ b1 then there are three constant solutions, and the (x, y) system can be switched on by the noise

in the x eq. Strogatz ([28], p. 245) gives further details of the (x, y) system with no noise present. To

turn the system on the idea is to let b2 < b3 and close to b3, and letσ2 > be “just large enough“ so

that b quickly jumps all the way down to b)1 and stays there with low probability of jumpimng back

up and swithing off the system

218

22 Coherence resonance in SIR (Kuske et al [13])

(a.) The problem This follows [13]. Assume that a population has three types of individuals,

S (susceptibles), I (infectives) and R (recovered), and that there is an integer N with

S1 + It + Rt = N ∀t ≥ 0. (22.1)

It is also assumed that S1, It, Rt Markov processes, nd that over a small interval [t, t + ∆t] the

increments ∆S = St+∆t − St and ∆I = It+∆t − It satisfy

∆S =(

µ(N − S) − β SIN

)

∆t+ ∆Z1 − ∆Z2,

∆I =(

β SIN − (γ + µ)I

)

∆t+ ∆Z2 − ∆Z3.(22.2)

They approximate the discrete model (22.2) with the continuous SDE system

dS =(

µ(N − S) − β SIN

)

dt+G1dW1 −G2dW2,

dI =(

β SIN − (γ + µ)I

)

dt+G2dW2 −G3dW3,(22.3)

where

G1 =√

µ(N + S), G2 =

√

β

NSI, G3 =

√

(γ + µ)I. (22.4)

Figure 29: Solution of (22.3) for parameters R0 = 15, µ = 1/55, γ = 20, N = 20000000, and initial

conditions (S(0), I(0)) = (133330, 2695). The solid curve is the stochastic solution. The dashed curve

is the deterministic solution.

(b.) What are ∆Z1,∆Z2,∆Z3, and how is (22.3) derived from (22.2)?

219

To understand what the variables ∆Z1,∆Z2 and ∆Z3 represent we consider the interval [t, t+∆t] where

increments ∆S and ∆I take place. Kuske et al assume that ∆S can be expressed as a combination

of independent Poisson processes ∆S1, ∆S2 and ∆S3, which denote the contributions to ∆S due to

birth rate, death rate and the rate of removal of susceptibles to I, the infected population. Thus,

∆S = ∆S1 − ∆S2 − ∆S3. (22.5)

Over [t, t+ ∆t] it is also assumed that the means of ∆S1, ∆S2 and ∆S3 satisfy

E(∆S1) = µN∆t, E(∆S2) = µSt∆t, and E(∆S3) = βStItN

. (22.6)

Since ∆S1, ∆S2 and ∆S3 are Poisson processes, then

V ar(∆S1) = µN∆t, V ar(∆S2) = µSt∆t, and V ar(∆S3) = βStItN

∆t. (22.7)

The difference ∆S1 − ∆S2 is also a Poisson process, with mean

E(∆S1 − ∆S2) = E(∆S1) − E(∆S2) = µN∆t− µSt∆t. (22.8)

Because ∆S1 and ∆S2 are independent, then

V ar(∆S1 − ∆S2) = V ar(∆S1) + V ar(∆S2) = µN∆t+ µSt∆t. (22.9)

To apply the Central Limit Theorem we define the variables ∆Z1 and ∆Z2 by

∆Z1 = ∆S1 − ∆S2 − [E(∆S1) − E(∆S2)] and ∆Z2 = ∆S3 − E(∆S3). (22.10)

Note that

E [∆Z1] = E [∆Z2] = 0. (22.11)

From (22.10), (22.7) and the independence of ∆S1 and ∆S2 it follows that

V ar [∆Z1] = V ar[∆S1] + V ar[∆S2] = µN∆t+ µSt∆t. (22.12)

By the Central Limit Theorem there exists c1 such that

∆Z1 = ∆S1 − ∆S2 − [E(∆S1) − E(∆S2)] ≈ c1∆X1, (22.13)

whereX1 denotes a normal random variable with E(X1) = 0 andE(X21 ) = V ar(X1) = 1.To determine

value of c1 square both sides of (22.13 ) and get

V ar(∆Z1) = E(

∆Z21

)

= V ar (∆S1 − ∆S2) ≈ c21E(

X21

)

= c21. (22.14)

Combining (22.14 ) and (22.9) gives

µN∆t+ µSt∆t ≈ c21 (22.15)

220

Thus, c1 =√

µ(N + St)∆t, and it follows from (22.6) and (22.13) that

∆S1 − ∆S2 ≈ µ(N − St)∆t+√

µ(N + St)∆tX1. (22.16)

Let W1 be a standard Wiener process, and recall that ∆W1 = W1(t+ ∆t) −W1(t) and√

∆tX1 have

the same pdf. This allows us to replace√

∆tX1 with ∆W1 in (22.16) and conclude that

∆S1 − ∆S2 ≈ µ(N − St)∆t+√

µ(N + St)∆W1. (22.17)

For the variable ∆Z2 = ∆S3 − E(∆S3) we have

V ar [∆Z2] = V ar[∆S3] = βStItN

∆t. (22.18)

As above, the Central Limit Theorem implies that there exists c2 and a Wiener process W2 such that

∆Z2 = ∆S3 − E(∆S3) ≈ c2∆W2. (22.19)

Note that ∆W1 and ∆W2 are independent since ∆S1,∆S2 and ∆S3 are independent. Squaring both

sides of (22.19), and using (22.18), we get

E(∆Z22 ) = E[(∆S3 − E(∆S3))

2] = V ar(∆S3) = βStItN

∆t ≈ c22V ar(∆W22 ) = c22∆t. (22.20)

Thus, c2 =√

β StIt

N and it follows from (22.6) and (22.18) that

∆S3 ≈ βStItN

∆t+

√

βStItN

∆W2. (22.21)

It follows from (22.5), (22.16) and (22.21) that

∆S ≈(

µ(N − St) − βStItN

)

∆t+√

µ(N + St)∆W1 −√

βStItN

∆W2. (22.22)

Replacing ∆S,∆t,∆W1 and ∆W2 with dS, dt, dW1 and dW2 gives the final approximation

dS =

(

µ(N − St) − βStItN

)

dt+√

µ(N + St)dW1 −√

βStItN

dW2. (22.23)

Next, we give a similar derivation for the second equation in (22.3). It is assumed that ∆I can be

expressed as the difference of independent Poisson processes ∆I1 and ∆I2, which denote the increase

in ∆I due to the rate of removal of susceptibles to I, and the decrease in ∆I due to the removal of

infected individuals to the recoverd population. Thus,

∆I = ∆I1 − ∆I2, (22.24)

and over [t, t+ ∆t] we assume that

∆I1 − E(∆I1) = ∆S3 − E(∆S3). (22.25)

221

Thus,

∆I1 ≈ βStItN

+

√

βStItN

dW2. (22.26)

Next, assume that that the mean of ∆I2 satisfies

E(∆I2) = (γ + µ)It∆t. (22.27)

As above, there is a Wiener process W3, independent of W1 and W2 such that

∆I2 ≈ (γ + µ)It∆t+√

(γ + µ)It∆W3. (22.28)

From (22.24), (22.26) and (22.28) we conclude that

∆I ≈(

βStItN

+ (γ + µ)It

)

∆t+

√

βStItN

∆W2 +√

(γ + µ)It∆W3. (22.29)

Finally, replace ∆I,∆t,∆W2 and ∆W3 with dI, dt, dW2 and dW3 and get the second SDE

dI =

(

βStItN

+ (γ + µ)It

)

dt+

√

βStItN

dW2 +√

(γ + µ)ItdW3. (22.30)

(c.) How to rescale and identify the small parameter?

Step I. Determine the steady state:

Seq =N

R0and Ieq =

Nµ

β(R0 − 1) . (22.31)

Step II. Non-dimenionalize and focus the solutions around (Seq , Ieq) by defining

u =S − Seq

Seqand v =

I − Ieq

Ieq. (22.32)

Step III. Identify the small parameter. For this, they first linearize the (u, v) system around the (u, v)

steady state (0, 0). At this point they also rescale time by setting η = Ωt where Ω is chosen so that

the eigenvalues λ of the linearized matrix are complex, with Im(λ) ≈ 1. Here,

λ = −ǫ2 ±√

ǫ4 − 1, (22.33)

where

ǫ2 =µR0

2Ω<< 1 and Ω =

√

βµ

R0(R0 − 1) (22.34)

(d.) Other key parameters.

Key parameters are

R0 =β

γ + µ> 1 and b2 =

Ieq

Seq=µR0(R0 − 1)

β(22.35)

Very important properties which they use below are

βb

R0Ω= 1 and b =

µ(R0 − 1)

Ω. (22.36)

222

Numbers used in Figure 29 (i.e. Kuske et al first diagram) are

R0 = 15, µ = 1/55, γ = 20, N = 20000000 (22.37)

hence

β ≈ 300.27273, Ω ≈ 2.2573, ǫ2 ≈ .0604, b ≈ .11269 (22.38)

(e.) How to derive the key linear system deterministic solution?

The big picture: To get stochastic resonance to work they work entirely with linear approximations.

For this they linearize the non-dimensionalized system around (0, 0) and get the resultant linear system

to produce stochastic resonance. Hopefully this causes the full nonlinear system to exhibit stochastic

resonance. The all important thing to do first is to derive the key approximation to the linear system.

Step I. Write the linearized system (8) in the matrix form

U ′ = MU +G(dW1, dW2.dW3)T , (22.39)

where U = (u, v)T , and M and G are the constant matrices

M =

[ −µR0

Ω−µ(R0−1)

Ωβ

ΩR00

]

=

[

−2ǫ2 −b1b 0

]

and G =

[

g1 −b2g2 0

0 g2 −g2

]

,

and

g1 =

√

µ

ΩS2eq

(N + Seq) =

√

µ

ΩN(R2

0 +R0) and g2 =

√

βSeq

ΩNIeq=

√

R0Ω3

µ3(R0 − 1)3N.

Here 0 ≤ g1, g2 << 1 and are non-constant due to the linearization around (0, 0).

The eigenvalues of M are complex and are given by

λ = −ǫ2 ± i√

1 − ǫ4.

Next, solve U ′ = MU of (22.39) and get the fundamental matrix solution

eMt = e−ǫ2tcos(√

1 − ǫ4t)E1 + e−ǫ2tsin(√

1 − ǫ4t)E2. (22.40)

Using the fact that ǫ2 = µR0

2Ω , we find that E1 and E2 are

E1 = I and E2 =1√

1 − ǫ4

[ −µR0

2Ω−µ(R0−1)

Ωβ

ΩR0

µR0

2Ω

]

.

Thus, if√

1 − ǫ4 ≈ 1 and D = (d1, d2)T then the general solution U = (u, v)T of the deterministic

part U ′ = MU of the linear system is

U ≈ e−ǫ2t

[

d1cos(t) − µR0

2Ω d1sin(t) −µ(R0−1)2Ω d2sin(t)

d2cos(t) + βΩR0

d1sin(t) +µR0

2Ω d2sin(t)

]

.

223

This reduces to the key approximation[

u

v

]

≈ e−ǫ2tC1

[

bcos(t)

sin(t)

]

+ e−ǫ2tC2

[

bsin(t)

−cos(t)

]

, (22.41)

(f.) How to derive the SDE’s which use the slow time scale T = ǫ2t?

Guided by (22.41), Kuske et al follow [28], set T = ǫt, and look for a solution

[

u

v

]

=

[

u0

v0

]

+O(ǫ2), (22.42)

where[

u0

v0

]

= A(T )

[

bcos(t)

sin(t)

]

+B(T )

[

bsin(t)

−cos(t)

]

. (22.43)

Note that (u0, v0)T satisfies

d

(

u0

v0

)

=

[

0 −b1/b 0

](

u0

v0

)

. (22.44)

Kuske et all assume that A(T ) and B(T ) are stochastic and satisfy

[

dA

dB

]

=

[

f1(A,B)

f2(A,B)

]

dT +

[

dN1(T )

dN2(T )

]

, (22.45)

where (dN1, dN2) has the general form

[

dN1

dN2

]

= Σ1

[

dζ11(T )

dζ12(T )

]

+ Σ2

[

dζ21(T )

dζ22(T )

]

+ Σ3

[

dζ31(T )

dζ32(T )

]

, (22.46)

where the dζij are standard Brownian motions on the slow time scale T. To figure out f1, f2, dN1 and

dN2 they use a Fredholm alternative type criterion which is described below.

First, note that if two functions u and v satisfy u = u(A,B, t) and v = v(A,B, t), then Ito’s Lemma

for systems (next section) gives

du = uAdA+ uBdB + (.5 ∗Q1uAA +Q2uAB + .5 ∗Q3uBB) dt+ utdt,

dv = vAdA+ vBdB + (.5 ∗Q4vAA +Q5vAB + .5 ∗Q6vBB) dt+ vtdt.(22.47)

where the subscripts mean partial derivatives. Set u = u0, v = v0. The second partials of u0 and v0

are zero because of the linear form of (22.43). Using (22.43) reduces the right side of (22.47) to

du0 = bcos(t)dA+ bsin(t)dB + (u0)tdt,

dv0 = sin(t)dA− cos(t)dB + (v0)tdt.(22.48)

Substitute dA and dB from (22.45) into (22.48) and get

du0 = bcos(t)(f1dT + dN1) + bsin(t)(f2dT + dN2) + (u0)tdt,

dv0 = sin(t)(f1dT + dN1) − cos(t)(f2dT + dN2) + (v0)tdt.(22.49)

224

In matrix form this system becomes

d

(

u0

v0

)

=

(

bcos(t) bsin(t)

sin(t) −cos(t)

)(

f1dT + dN1

f2dT + dN2

)

dT +

(

0 −b1/b 0

)(

u0

v0

)

dt (22.50)

Recall that (u, v)T satisfies

d

(

u

v

)

dt =

(

−2ǫ2 −b1/b 0

)(

u

v

)

dt+

(

g1dW1 − b2g2dW2

g2dW2 − g2dW3

)

. (22.51)

Subtract (22.50) from(22.51) and get

d

(

u− u0

v − v0

)

−(

0 −b1/b 0

)(

u− u0

v − v0

)

dt = −(

bcos(t) bsin(t)

sin(t) −cos(t)

)(

f1dT + dN1

f2dt+ dN2

)

+

(

−2ǫ2 0

0 0

)(

u

v

)

dt+

(

g1dW1 − b2g2dW2

g2dW2 − g2dW3

)

.

(22.52)

Replace (u, v)T on the right side with (22.42), set dT = ǫ2dt, ignore the O(ǫ2) term, and get

d

(

u− u0

v − v0

)

−(

0 −b1/b 0

)(

u− u0

v − v0

)

dt = −(

bcos(t) bsin(t)

sin(t) −cos(t)

)(

f1dT + dN1

f2dt+ dN2

)

+

(

−2 0

0 0

)(

A(T )bcos(t) +B(T )bsin(t)

A(T )sin(t) −B(T )cos(t)

)

dT +

(

g1dW1 − b2g2dW2

g2dW2 − g2dW3

)

.

(22.53)

(f.) How to use the adjoint system to find the fi + dNi(T ) terms

System (22.53) has a solution if its right side is othogonal to all solutions of the adjoint system

V ′ = −MTV, (22.54)

where

−MT =

[

0 − 1b

b 0

]

.

The eigenvalues of −MT are λ = ±i. The fundamental matrix solution of (22.54) is

e−MT t = cos(t)I − sin(t)MT (22.55)

Thus, the general solution of V ′ = −MTV is

V = d1

[

cos(t)

bsin(t)

]

+d2

b

[

−sin(t)

bcos(t)

]

. (22.56)

225

Let D1 = d1 and D2 = d2

b . Then

V = D1

[

cos(t)

bsin(t)

]

+D2

[

−sin(t)

bcos(t)

]

. (22.57)

To evaluate f1, f2, dN1 and dN2 we use the inner product criterion

∫ 2π

0 (u∗, v∗)

((

bcos(t) bsin(t)

sin(t) −cos(t)

)(

f1

f2

)

dT +

(

bcos(t) bsin(t)

sin(t) −cos(t)

)(

dN1(T )

dN2(T )

))

dt

+∫ 2π

0 (u∗, v∗)

((

2 0

0 0

)(

A(T )bcos(t) +B(T )bsin(t)

A(T )sin(t) −B(T )cos(t)

)

dT −(

g1dW1 − b2g2dW2

g2dW2 − g2dW3

))

dt =

(

0

0

)

(22.58)

where (u∗, v∗) = (cos(t), bsin(t)) and (u∗, v∗) = (−sin(t), bscos(t)).

Very important insight: we must expand each dWi in terms of sin(t) and cos(t) in order to apply the

inner product criterion (22.58).

Kuske et al claim that each gjdWk(t) can be written as

gjdWk(t) =gj

ǫ[cos(t)dwk1(T ) + sin(t)dwk2(T )] , (22.59)

where the dwki are independent standard Brownian motions. Here is an outline of a proof. Start with

standard normal random variables X , X1 and X2 and set

Y = cos(α)X1 + cos(α)X2. (22.60)

A computation shows that X and Y have the same moment generating function MX(t) = et2/2.

Therefore ∆tX and ∆tY have the same pdf so we write

∆tX = cos(α)∆tX1 + cos(α)∆tX2, (22.61)

or equivalently

dW = dtX = cos(α)dtX1 + sin(α)dtX2 = cos(α)dw1(t) + sin(α)dw2(t). (22.62)

Set T = ǫ2t and get

dW =1

ǫ[cos(α)dw1(T ) + sin(α)dw2(T )] . (22.63)

Replace α with t and get

dW =1

ǫ[cos(t)dw1(T ) + sin(t)dw2(T )] . (22.64)

Set for dW = dWk, dw1 = dwk1, dw2 = dwk2 and multiply by gj to get (22.59).

Finally, we evaluate (22.58) and get

f1dT + dN1(T ) = −AdT + 12bǫ

(

g1dw11(T ) − b2g2dw21(T ) + bg2dw22(T ) − bg2dw32(T ))

f2dT + dN2(T ) = −BdT + 12bǫ

(

−g1dw12(T ) + b2g2dw22(T ) + bg2dw21(T ) − bg2dw31(T ))

(22.65)

226

Thus, the (A,B)T SDE system is

d

(

A

B

)

= −(

1 0

0 1

)(

A

B

)

dT + ΓdW (T ), (22.66)

where

Γ =1

2bǫ

(

g1 −b2g2 bg2 −bg2 0 0

0 −bg2 −b2g2 0 g1 bg2

)

(22.67)

and

dW (T ) = (dw11, dw21, dw22, dw32, dw12, dw31)T (22.68)

Note that

ΓΓT =g21 + (1 + b2)b2g2

2

4b2ǫ2

(

1 0

0 1

)

. (22.69)

The SDE system (22.66) is an Ornstein-Uhlenbeck process with a statonary distribution. The mean

of the stationary porocess is zero. To see this take E of both sides of (22.66) and get

E(A(T ))′ = −E(A(T )) and E(B(T ))′ = −E(B(T )). (22.70)

Solving these equations gives

E(A(T )) = C1e−T → 0 and E(B(T )) = C2e

−T → 0 as T → ∞. (22.71)

Thus, the mean of the stationary process is E(A(∞)) = E(A(∞)) = 0.

Our next goal is to show that A and B are independent.

Step I

First, according to Arnold, p. 133, we have the following theorem (I think Kuske et al are referring

to on p. 9 of their paper):

Theorem. The solution of the multivariate equation

dXt = (A(t)Xt + a(t)) dt+B(t)dWt, X0 = c (22.72)

is a stationary gaussian process if A(t) ≡ A, a(t) ≡ 0, B(t) ≡ B, the eignevalues of A have negative

real parts, c is N(0,K) (i.e. c is n-variate normally distributed), where

K =

∫ ∞

0

eAtBBT eAT tdt. (22.73)

Step II. The normal n-variate random variable has joint pdf

fX(x) =1

(2π)N/2|detK|1/2exp

[

−1

2(x− µ)TK−1(x − µ)

]

, (22.74)

227

where µ is a vector representing normal mean, and the matrix K = (σij) = (cov(XiXj) is the

covariance matrix. In 2 dimensions, if X = (X1, X2) then

K =

[

σ2x ρσxσy

ρσxσy σ2y

]

and K−1 =1

1 − ρ2

1σ2

x

−ρσxσy

−ρσxσy

1σ2

y

,

If K is a diagonal matrix then X1 and X2 are independent. From (22.71) it follows that µ = (0, 0)

for the (A,B) system stationary distribution. Furthermore, we have K = ΓΓT , which is a diagonal

matrix.

(g.) Statement of Ito’s Lemma for systems.

We develop a two variable version following [23], pp. 45-46. Consider the system of SDE’s

dx1 = u1dt+ v11dB1 + v12dB2,

dx2 = u2dt+ v21dB1 + v22dB2.(22.75)

Here Bi denotes the Wiener process. In matrix notation system (22.75) is written as

dx(t) = udt+ vdB(t), (22.76)

where

x =

(

x1(t)

x2(t)

)

, u =

(

u1

u2

)

, v =

(

v11 v1,2

v2,1 v2,2

)

and dB =

(

dB1

dB1

)

. (22.77)

Let g(t, x) = (g1(t, x), g2(t, x)) be a C2 function of (t, x). Then

dg =

( ∂g1

∂t∂g2

∂t

)

dt+

( ∂g1

∂x1dx1 + ∂g1

∂x2dx2

∂g2

∂x1dx1 + ∂g2

∂x2dx2

)

+1

2

(

∑

i,j∂2g1

∂xi∂xjdxidxj

∑

i,j∂2g2

∂xi∂xjdxidxj

)

, (22.78)

where the rules for evaluating dxidxj are dtdt = dtdBk = dBkdt = 0, and dBidBj = δi,jdt.

(b.) Properties of the Bivariate normal distribution. This follows Schaum’s outline ([26],

pp. 88-89). A bivariate random variable (X,Y ) has pdf

fXY (x, y) =1

2πσxσy

√

1 − ρ2exp

(

−1

2q(x, y)

)

, (22.79)

where

q(x, y) =1

1 − ρ2

(

(

x− µx

σx

)2

− 2ρ

(

x− µx

σx

)(

y − µy

σy

)

+

(

y − µy

σy

)2)

, (22.80)

and ρ is the coefficient of correlation

ρ =Cov(X,Y )

σxσy=E(XY ) − E(X)E(Y )

σxσy. (22.81)

Kuske et al use the following properties in their analysis of the (A,B) amplitude sytem:

228

• (i) If ρ = 0 then X and Y are uncorellated.

• (ii) If ρ = 0 then X and Y are independent.

The N-variate normal distribution. Let The random variable X = (X1, .., XN )′ is an “n-variate

normal random variable” if its joint pdf is

fX(x) =1

(2π)N/2|detK|1/2exp

[

−1

2(x− µ)TK−1(x − µ)

]

, (22.82)

where µ is a vector representing normal mean. The matrix K = (σij) = (cov(XiXj) is the covariance

matrix. In 2 dimensions, if X = (x, y), then

K =

[

σ2x ρσxσy

ρσxσy σ2y

]

and K−1 =1

1 − ρ2

1σ2

x

−ρσxσy

−ρσxσy

1σ2

y

,

and (22.82) reduces to (22.79)-(22.80).

(c.) Properties of the Ornstein-Uhlenbeck process.

Type Ornstein-Uhlenbeck process in google. The wikepedia site gives the following: The Ornstein-

Uhlenbeck process is a stochastic process Xt given by the SDE

dXt = −θ(Xt − µ)dt+ σdWt, (22.83)

where θ, µ, σ are parameters and Wt is the Wiener process. According to Doering ([4], p. 23) the

function dXt

dt represents the acceleration of a particle in some medium subject to a frictional retarding

force proportional to its velocity, −θXt, and a rapidly fluctuating force due to collisions with the

’fluid” particles in the medium.

Let f(X, t) = Xte−θt and use Ito’s lemma to get

df = eθtθµdt+ σeθtdWt. (22.84)

Integration gives

Xteθt = X0 +

∫ t

0

eθtθµds+ σ

∫ t

0

eθsdWs, (22.85)

hence

Xt = X0e−θt + µ(1 − e−θt) + σ

∫ t

0

eθ(s−t)dWs, (22.86)

This implies that the first moment of Xt is

E(Xt) = X0e−θt + µ(1 − e−θt) → µ as t→ ∞. (22.87)

The covariance is (Arnold, p.134)

cov(XsXt) = E [(Xt − E(Xt))(Xs − E(Xs))] = e−θ(t+s)

(

var(X0) +σ2

2θ

(

e2θmint,s − 1)

)

,

(22.88)

229

which reduces to

cov(XsXt) = e−θ(t+s)var(X0) +σ2

2θ

(

e−θ|t−s| − e−θ(t+s))

(22.89)

Gardner ([11], p. 77) notes that Xt and Xs are maximally corellated when |t− s| = 1θ , this difference

being called the ”corellation time.”

The variance of Xt is bounded, i.e.

var(Xt) = cov(XtXt) = e−2θtvar(X0) +σ2

2θ

(

1 − e−θ(2t))

→ σ2

2θas t→ ∞. (22.90)

By way of comparison, the Wiener process Wt has unbounded variance

var(Wt) = σ2t→ ∞ as t→ ∞. (22.91)

If we assume that Xs = y and µ = 0 then Doering ([4], pp. 23-24) notes that the associated Fokker-

Planck equation is

∂ρ(x, t|y, s)∂t

=

[

θ∂ρ(x, t|y, s)

∂xx+

σ2

2

∂2ρ(x, t|y, s)∂x2

]

ρ(x, t|y, s), (22.92)

and ρ(x, t|y, s) is given by

ρ(x, t|y, s) =1

√

2πΣ(t− s)exp

(

− (x− ye−θ(t−s))2

2Σ(t− s)

)

, (22.93)

where

Σ(t− s) =σ2

2θ

(

1 − e−2θt)

. (22.94)

The “stationary state solution” ρs(x) can be obtained from the following limit:

ρstat(x) = lims→−∞

ρ(x, t|y, s) =

√

θ

πσ2exp

(

−θx2

σ2

)

. (22.95)

This solution can be obtained by setting ∂ρ(x,t|y,s)∂t = 0 in (22.93) and solving the resultant ODE.

The “covariance in the stationary state” is given by

cov(XsXt) =σ2

2θ

(

e−θ|t−s|)

(22.96)

Systems. According to Arnold, p. 133, we have the following theorem which is what I think Kuske

et al are referring to on p. 9 of their paper:

Theorem. The solution of the multivariate equation

dXt = (A(t)Xt + a(t)) dt+B(t)dWt, X0 = c (22.97)

is a stationary gaussian process if A(t) ≡ A, a(t) ≡ 0, B(t) ≡ B, the eignevalues of A have negative

real parts, c is N(0,K) (i.e. c is n-variate normally distributed), where

K =

∫ ∞

0

eAtBB′eA′tdt. (22.98)

230

Question 2. Kuske et al state that ” the one-time-point stationary laws of the Ornstein-

Uhlenbeck process can be identified as a bivariate normal distribution, N(0,K) with mean

zero and covariance matrix

K = cov(A(T ), B(T ))′

where

K =

∫ ∞

0

exp

[(

−1 0

0 −1

)

s

]

ΓΓ′

(

−1 0

0 −1

)′

s

ds

How do they know this? Are they using Ludwig Arnold’s theorem? The outcome of

this computatiuon gives a diagonal matrix for K, which I think means that A and B are

uncorlellated, and also independent because they have the special property that they are

bivariate normally distributed. I think that they are using Arnold’s theorem to get the covariant

matrix K, then they show that K is diagonal, hence the coefficient of variation ρ = 0. This means

that A and B are uncorellated. Then use the theorem (Rachel says it is on p. 110 of Gardner [11])

that says that the pdf of A and B ais the binormal distribution. Once we know this, and the fact that

ρ = 0, we can conclude that the bivariate normal pdf, becomes a product of normal pdf’s, hence A

and B are independent.

(e.) Computing Power Spectral Density (PSD). First, an example ( see rxoftau1.m) - compute

the PSD if the autocorellation function is

y = .5 ∗ exp(−|tau|), (22.99)

The PSD of y is the Fourier transform

S(f) =

∫ ∞

−∞.5 ∗ exp(−|tau|)e−2πifτdτ. (22.100)

Note that

S(0) =

∫ ∞

−∞.5 ∗ exp(−|tau|)dτ = 1. (22.101)

The key seems to be to let the nimber of t values be about the same as n, the number of frequency

points The program rxoftau1.m figures out the PSD. In matlab type rxoftau1

- see powerspectrum.m for the basic matlab example

An idea for Rachel’s paper is to use dt=.6, like is used in powerspectrum.m and integrate for 512

points past t=100 out to t=300.

231

References

[1] R. Benzi, A. Sultera & A. Vulpani The mechanism of stochastic resonance. J. Phys. A:

Math. Gen. Vol. 14 (1981), L453-L457 (see benzipaper1.pdf)

[2] J. de la Rocha, B. Doiron, E. S. Brown, K. Jpsoc & A. Reyes Correlation between

neural spike trains increrases ith firing rate systems. Nature (2007)

[3] R. Deville, E. Vanden-Eijinden & C. Muratov Two distinct mechanisms of coherence in

randomly perturbed dynamical systems. PRE Vol. 72 (2005), No. 031105 (see Deville.pdf)

[4] C. R. Doering Modelling complex systems: stochastic processes, stochastic differential equations,

and Fokker-Planck equations. 1990 Lectures in Complex Systems, SFI Studies in the Sciences of

Complexity, Leture Vol. III, L. Nadel and D. Stein eds., Addison Wesley 1991

[5] C. Doering, C. Mueller & P. Smerka Interacting particles, the stochastic Fisher-

Kolmogorov-Piscounov equation, and duality. Physica A 325 (2003), pp. 243-259

[6] B. Doiron, J. Rinzel & A. D. Reyes Stochastic synchronization in finite size spiking networks.

and duality. Phys Rev. E 74 (2006), pp. 243-259

[7] B. Doiron Private communication.

[8] L. Edelstein-Keshet Mathematical Models in Biology. Random House 1987

[9] I. Ekeland The best of all worlds. U. of Chicago Press 2006

[10] W. I. Freidlin & A. D. Wentzell Random Perturbations of Dynamical Systems Springer-

Verlag 1984

[11] G. W. Gardner handbook of Stochastic Methods. Springer 2003

[12] D. J. Higham An algorithmic approach to numerical simulation of Stochastic Differential equa-

tions. SIAM Review Vol. 43, no. 3 (2001), pp. 525-546

[13] R. Kuske, L. F. Cordilla & P. Greenwood Sustained oscillations via coherence in SIR

Jour. Theor. Biology, Vol. 245, No. 3 (2007), pp. 459-469

[14] R. C. Lewontin & D. Cohen On population growth in a randomly varying environemnt.

PNAS Vol. 62, No. 4 (1969), pp. 1056-1060

[15] R. Maier Large Fluctuations in stochastically nonlinear systems: applications in computing.

Santa fe lecture - santafeweVol. 63, no. 4 (2003), pp. 498-504

[16] P. D. Monahan A primer for SAD. Europhysics Letters Vol. 63, no. 4 (2003), pp. 498-504

232

[17] R. S. Maier Large fluctuations in stochastically perturbed nonliear systems: applications in

computing. Notes for a talk given at the Santa Fe Institute 1999 (see santafewentzellnotes.pdf)

[18] R. S. Maier LExponentially rare large fluctuatiosn: computing their frequency. Notes for a talk

given at U. texas (see mier2.pdf)

[19] R. Maier & D. Stein Transition-rate theory for non-gradient drift fields. Phys. Rev. Lett. 69

(1992), pp. 3691-3695

[20] M. Matia & P. D. Giudic Population dynamics of interacting spiking neurons. Phys. Rev. E.

66 (2002), pp. 051917-1 to 051917-19

[21] B. Matkowsky, Z. Schuss, C. Knessl, C. Tier & M. Mangel Asymptotic solution of the

Kramers-Moyal equation and first passage times for markov jump processes. Phys. Rev. A. 29

(1984), pp. 3359-3369

[22] T. Naeh, M. Klosek, B. Matkowsky & Z. Schuss A direct approach to the exit problem.

SIAM Appl. Math. 50 (1990), pp. 595-627

[23] B. Oksendal Stochastic Differential Equations. Springer, 4th edition

[24] Oswald Spasre connectivity of pyramidal cells

[25] W. J. Reed The Pareto, Zipf and other power laws Economics Letters Vol. 74 (2001), pp. 15-19

[26] P. D. Sardeshmukh, C. Penland & M. Newman Drifts induced by multiplicative red noise

with application to climnate. Europhysics Letters Vol. 63, no. 4 (2003), pp. 498-504

[27] Schaum’s Outline: Probability, Random Variables and random Processes

[28] S. Strogatz Nonlinear Dynamics and Chaos, Westview Press (1994)

233

Basic Theory For SDE’stroy/vlad/doering.pdf · Basic Theory For SDE’s W. C. Troy∗ July 11,...

Documents

Transcript of Basic Theory For SDE’stroy/vlad/doering.pdf · Basic Theory For SDE’s W. C. Troy∗ July 11,...