Math 605: Stochastic Models in Biology

Fall 2011

University of Wisconsin at Madison

Instructor: David F. Anderson

What is this course?

This course is

1. an introduction to stochastic processes, at Math 632 level,

2. with an added focus on computational techniques and

3. applications arising from biology.

The main mathematical models of study areI discrete time Markov chains,I branching processes,I point processes,I continuous time Markov chains,I and diffusion processes (those incorporating Brownian motion).

We will focus on computational methods for the different models.

The software package of choice will be Matlab.

What is this course?

This course is

1. an introduction to stochastic processes, at Math 632 level,

2. with an added focus on computational techniques and

3. applications arising from biology.

The main mathematical models of study areI discrete time Markov chains,I branching processes,I point processes,I continuous time Markov chains,I and diffusion processes (those incorporating Brownian motion).

We will focus on computational methods for the different models.

The software package of choice will be Matlab.

Our focus

The main mathematical models of study areI discrete time Markov chains,I branching processes,I point processes,I continuous time Markov chains,I and diffusion processes (those incorporating Brownian motion).

I We will pay special attention to the continuous time Markov chainmodels that arise in the study of population processes, such as cellularlevel biochemical models.

I Our perspective will differ from that of other texts on the subject in thatwe will primarily attempt to understand these processes via the randomtime change representation of Thomas Kurtz, which will be used toderive the relevant approximations (including the standard mass-actionkinetics model for deterministic dynamics) and simulation strategies.

Prerequisites

We require:

1. A solid understanding of calculus.

2. Linear algebra: eigenvalues, eigenvectors, matrix multiplications, etc.

3. A basic knowledge of differential equations.

4. A basic understanding of probability: sample spaces, random variables,expectations, conditional probability.

I will begin the class with a brief introduction to probability before discussingstochastic processes and stochastic models.

Stochastic versus deterministic models

Before proceeding too far, it is important to understand the basic terms“stochastic” and “deterministic.”

A process is deterministic if its future is completely determined by its presentand past. Examples of deterministic processes include solutions todifferential and difference equations.

ExampleThe initial value problem

x(t) = 3x(t) x(0) = 2,

has the solution x(t) = 2e3t . �

ExampleConsider the difference equation

F1 = F2 = 1

Fn = Fn−1 + Fn−2, for n > 2.

Then {Fn}∞n=1 is the well known Fibonacci sequence: {1, 1, 2, 3, 5, 8, . . . }. �

Stochastic versus deterministic models

Before proceeding too far, it is important to understand the basic terms“stochastic” and “deterministic.”

A process is deterministic if its future is completely determined by its presentand past. Examples of deterministic processes include solutions todifferential and difference equations.

ExampleThe initial value problem

x(t) = 3x(t) x(0) = 2,

has the solution x(t) = 2e3t . �

ExampleConsider the difference equation

F1 = F2 = 1

Fn = Fn−1 + Fn−2, for n > 2.

Then {Fn}∞n=1 is the well known Fibonacci sequence: {1, 1, 2, 3, 5, 8, . . . }. �

Stochastic versus deterministic models

Before proceeding too far, it is important to understand the basic terms“stochastic” and “deterministic.”

A process is deterministic if its future is completely determined by its presentand past. Examples of deterministic processes include solutions todifferential and difference equations.

ExampleThe initial value problem

x(t) = 3x(t) x(0) = 2,

has the solution x(t) = 2e3t . �

ExampleConsider the difference equation

F1 = F2 = 1

Fn = Fn−1 + Fn−2, for n > 2.

Then {Fn}∞n=1 is the well known Fibonacci sequence: {1, 1, 2, 3, 5, 8, . . . }. �

Stochastic versus deterministic modelsOn the other hand, a stochastic process is a random process evolving in time.

Informally, this means that even if you have full knowledge of the state of thesystem (and it’s entire past), you can not be sure of it’s value at future times.

More formally, a stochastic process is a collection of random variables, X (t)or Xt , indexed by time.

ExampleConsider rolling a fair, six-sided die many times, and for k ∈ {1, 2, . . . }, let Zk

be the outcome of the k th roll. Let

Xn =n∑

k=1

Zk .

• Thus, Xn is the accumulated total of the first n rolls.

• Knowing X1 = 3 only tells you that X2 ∈ {4, . . . , 9}, with equal probability.

• Note that time, indexed here by n, is discrete in that we only update thesystem after each roll of the die. �

Stochastic versus deterministic modelsOn the other hand, a stochastic process is a random process evolving in time.

Informally, this means that even if you have full knowledge of the state of thesystem (and it’s entire past), you can not be sure of it’s value at future times.

More formally, a stochastic process is a collection of random variables, X (t)or Xt , indexed by time.

ExampleConsider rolling a fair, six-sided die many times, and for k ∈ {1, 2, . . . }, let Zk

be the outcome of the k th roll. Let

Xn =n∑

k=1

Zk .

• Thus, Xn is the accumulated total of the first n rolls.

• Knowing X1 = 3 only tells you that X2 ∈ {4, . . . , 9}, with equal probability.

• Note that time, indexed here by n, is discrete in that we only update thesystem after each roll of the die. �

Stochastic versus deterministic modelsOn the other hand, a stochastic process is a random process evolving in time.

Informally, this means that even if you have full knowledge of the state of thesystem (and it’s entire past), you can not be sure of it’s value at future times.

More formally, a stochastic process is a collection of random variables, X (t)or Xt , indexed by time.

ExampleConsider rolling a fair, six-sided die many times, and for k ∈ {1, 2, . . . }, let Zk

be the outcome of the k th roll. Let

Xn =n∑

k=1

Zk .

• Thus, Xn is the accumulated total of the first n rolls.

• Knowing X1 = 3 only tells you that X2 ∈ {4, . . . , 9}, with equal probability.

• Note that time, indexed here by n, is discrete in that we only update thesystem after each roll of the die. �

Stochastic versus deterministic models

ExampleConsider a frog sitting in a pond with k lily pads, labeled 1 through k .

1. The frog starts the day by sitting on a randomly chosen pad (forexample, they could be chosen with equal probability).

2. However, after a random amount of time, the frog will jump to anotherpad, also randomly chosen.

3. Letting t = 0 denote the start of the day, we let X (t) ∈ {1, . . . , k} denotethe lily pad occupied by the frog at time t . In this example, time isnaturally continuous.

However, if we are only interested in which lily pad the frog is on after a givennumber of jumps,

1. then we may let Zn denote the lily pad occupied by the frog after the nthjump,

2. with Z0 defined to be the starting lily pad.

3. The process Zn is discrete in time.

4. The processes Zn and X (t) are clearly related, and Zn is usually calledthe embedded discrete process associated with X (t).

Stochastic versus deterministic models

ExampleConsider a frog sitting in a pond with k lily pads, labeled 1 through k .

1. The frog starts the day by sitting on a randomly chosen pad (forexample, they could be chosen with equal probability).

2. However, after a random amount of time, the frog will jump to anotherpad, also randomly chosen.

3. Letting t = 0 denote the start of the day, we let X (t) ∈ {1, . . . , k} denotethe lily pad occupied by the frog at time t . In this example, time isnaturally continuous.

However, if we are only interested in which lily pad the frog is on after a givennumber of jumps,

1. then we may let Zn denote the lily pad occupied by the frog after the nthjump,

2. with Z0 defined to be the starting lily pad.

3. The process Zn is discrete in time.

4. The processes Zn and X (t) are clearly related, and Zn is usually calledthe embedded discrete process associated with X (t).

Example: Bacterial Growth

Let’s consider two oversimplified models for bacterial growth (by growth here,I mean the growth of the size of the colony, not of an individual bacterium):

I one deterministic

I one stochastic.

We supposeI there are 10 bacteria at time zero.I each bacteria divides at an “average” rate of once per three hours.

Deterministic model: a “reasonable” model would be

ddt

x(t) =13

x(t) x(0) = 10, (1)

with solutionx(t) = 10et/3,

where the units of t are hours.

Example: Bacterial Growth

Let’s consider two oversimplified models for bacterial growth (by growth here,I mean the growth of the size of the colony, not of an individual bacterium):

I one deterministic

I one stochastic.

We supposeI there are 10 bacteria at time zero.I each bacteria divides at an “average” rate of once per three hours.

Deterministic model: a “reasonable” model would be

ddt

x(t) =13

x(t) x(0) = 10, (1)

with solutionx(t) = 10et/3,

where the units of t are hours.

Example: Bacterial Growth

Let’s consider two oversimplified models for bacterial growth (by growth here,I mean the growth of the size of the colony, not of an individual bacterium):

I one deterministic

I one stochastic.

We supposeI there are 10 bacteria at time zero.I each bacteria divides at an “average” rate of once per three hours.

Deterministic model: a “reasonable” model would be

ddt

x(t) =13

x(t) x(0) = 10, (1)

with solutionx(t) = 10et/3,

where the units of t are hours.

Example: Bacterial Growth

Stochastic Model: Without going into the finer details, assume

1. Each bacteria divides after a random (independent, exponential) amountof time with an average wait of 3 hours.

Similar to equation (1) for the deterministic model, it is possible to write downsystems of equations describing the time evolution of model

1. Evolution of individual sample paths – instance of experiment (like theODE model)

2. Evolution of the distribution (probability of being in certain states)

Example: Bacterial Growth

Stochastic Model: Without going into the finer details, assume

1. Each bacteria divides after a random (independent, exponential) amountof time with an average wait of 3 hours.

Similar to equation (1) for the deterministic model, it is possible to write downsystems of equations describing the time evolution of model

1. Evolution of individual sample paths – instance of experiment (like theODE model)

2. Evolution of the distribution (probability of being in certain states)

Example: Bacterial Growth - evolution of sample paths

I Below is a plot of the solution of the deterministic system versus threedifferent realizations of the stochastic system.

0 1 2 3 4 510

20

30

40

50

60

70

Colo

ny s

ize

Time

I Stochastic realizations/experiments appear to follow the deterministicsystem in a “noisy” way.

I It is clear that the behavior of a single realization or experiment of thestochastic system can not be predicted with absolute accuracy.

Example: population growth - evolution of distribution

10 12 14 16 18 200

0.2

0.4

0.6

0.8

1

n

p n(0)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

n

p n(1)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

n

p n(2)

10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

n

p n(3)

Example: population growth - evolution of distribution

10 12 14 16 18 200

0.2

0.4

0.6

0.8

1

n

p n(0)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

n

p n(1)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

n

p n(2)

10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

n

p n(3)

Example: population growth - evolution of distribution

10 12 14 16 18 200

0.2

0.4

0.6

0.8

1

n

p n(0)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

n

p n(1)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

n

p n(2)

10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

n

p n(3)

Example: population growth - evolution of distribution

10 12 14 16 18 200

0.2

0.4

0.6

0.8

1

n

p n(0)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

n

p n(1)

10 20 30 40 50 600

0.02

0.04

0.06

0.08

0.1

n

p n(2)

10 20 30 40 50 600

0.01

0.02

0.03

0.04

0.05

0.06

0.07

n

p n(3)

Example: Bacterial Growth and Death

Now suppose that we change the model “slightly” in that:

1. we allow bacteria to die as well as divide.

2. we suppose we begin with only two bacteria.

We suppose that they die after about five hours.

Our new deterministic model could be

x(t) =13

x(t)− 15

x(t) =2

15x(t), x(0) = 2,

with solutionx(t) = 2e2t/15.

Example: Bacterial Growth and Death

Now suppose that we change the model “slightly” in that:

1. we allow bacteria to die as well as divide.

2. we suppose we begin with only two bacteria.

We suppose that they die after about five hours.

Our new deterministic model could be

x(t) =13

x(t)− 15

x(t) =2

15x(t), x(0) = 2,

with solutionx(t) = 2e2t/15.

Example: Bacterial Growth and Death

For the stochastic model, we now model the two possible changes to the sizeof the colony separately. That is, the next event is either

1. a growth event (via a division) or

2. a decrease event (via a death).

Example: Bacterial Growth and Death

I Deterministic vs. three realizations/experiments of stochastic system.

0 2 4 6 8 100

5

10

15

20

25

Time

Colo

ny S

ize

I The models now behave qualitatively differently:

one of the realizations of the stochastic model (i.e. one of the coloniesunder observation) has been completely wiped out, something notpossible in the deterministic modeling context .

Example: Lotka-VolterraThink of A as a prey and B as a predator.

Aκ1→ 2A, A + B

κ2→ 2B, Bκ3→ ∅,

with A(0) = B(0) = 1000 and κ1 = 2, κ2 = .002, κ3 = 2.

400 600 800 1000 1200 1400 1600 1800

600

800

1000

1200

1400

1600

1800

Predator

Prey

Stochastic modelODE model

Behavior is qualitatively different:1. Deterministic always periodic

while stochastic oscillates inrandom way.

2. Predator and/or prey will end upextinct in stochastic model.

0 5 10 15 20 25 30 35 40700

800

900

1000

1100

1200

1300

1400

1500

PreyPredator

0 5 10 15 20 25 30 35 40400

600

800

1000

1200

1400

1600

1800

2000

PreyPredator

Example: Lotka-VolterraThink of A as a prey and B as a predator.

Aκ1→ 2A, A + B

κ2→ 2B, Bκ3→ ∅,

with A(0) = B(0) = 1000 and κ1 = 2, κ2 = .002, κ3 = 2.

400 600 800 1000 1200 1400 1600 1800

600

800

1000

1200

1400

1600

1800

Predator

Prey

Stochastic modelODE model

Behavior is qualitatively different:1. Deterministic always periodic

while stochastic oscillates inrandom way.

2. Predator and/or prey will end upextinct in stochastic model.

0 5 10 15 20 25 30 35 40700

800

900

1000

1100

1200

1300

1400

1500

PreyPredator

0 5 10 15 20 25 30 35 40400

600

800

1000

1200

1400

1600

1800

2000

PreyPredator

Some questions

We start to see some of the different types of questions that becomeinteresting in the stochastic context as opposed to the deterministic:

1. For a given birth and death rate in bacteria example, what is theprobability that the colony will eventually die out?

2. For models in which extinction is eventually guaranteed: what is theexpected amount of time before extinction?

3. If we know a stochastic processes Xt neither dies out, nor goes to infinity,and if a < b are real numbers, then what is the probability that the valueof the process is between a and b for very large t? That is, what is

limt→∞

Prob{a ≤ Xt ≤ b}?

4. How did I make those plots?

Some questions

We start to see some of the different types of questions that becomeinteresting in the stochastic context as opposed to the deterministic:

1. For a given birth and death rate in bacteria example, what is theprobability that the colony will eventually die out?

2. For models in which extinction is eventually guaranteed: what is theexpected amount of time before extinction?

3. If we know a stochastic processes Xt neither dies out, nor goes to infinity,and if a < b are real numbers, then what is the probability that the valueof the process is between a and b for very large t? That is, what is

limt→∞

Prob{a ≤ Xt ≤ b}?

4. How did I make those plots?

Some questions

We start to see some of the different types of questions that becomeinteresting in the stochastic context as opposed to the deterministic:

1. For a given birth and death rate in bacteria example, what is theprobability that the colony will eventually die out?

2. For models in which extinction is eventually guaranteed: what is theexpected amount of time before extinction?

3. If we know a stochastic processes Xt neither dies out, nor goes to infinity,and if a < b are real numbers, then what is the probability that the valueof the process is between a and b for very large t? That is, what is

limt→∞

Prob{a ≤ Xt ≤ b}?

4. How did I make those plots?

Some questions

We start to see some of the different types of questions that becomeinteresting in the stochastic context as opposed to the deterministic:

1. For a given birth and death rate in bacteria example, what is theprobability that the colony will eventually die out?

2. For models in which extinction is eventually guaranteed: what is theexpected amount of time before extinction?

3. If we know a stochastic processes Xt neither dies out, nor goes to infinity,and if a < b are real numbers, then what is the probability that the valueof the process is between a and b for very large t? That is, what is

limt→∞

Prob{a ≤ Xt ≤ b}?

4. How did I make those plots?

Some questions

We start to see some of the different types of questions that becomeinteresting in the stochastic context as opposed to the deterministic:

1. For a given birth and death rate in bacteria example, what is theprobability that the colony will eventually die out?

2. For models in which extinction is eventually guaranteed: what is theexpected amount of time before extinction?

3. If we know a stochastic processes Xt neither dies out, nor goes to infinity,and if a < b are real numbers, then what is the probability that the valueof the process is between a and b for very large t? That is, what is

limt→∞

Prob{a ≤ Xt ≤ b}?

4. How did I make those plots?

How to study stochastic models of intracellular processes?

The stochastic models should not be constructed by simply “adding noise” tothe deterministic system.

In fact, the opposite is true; the stochastic models should be developed viafirst principles or through an understanding of the system.

The deterministic models should actually arise as a consequence of somelimiting argument of the stochastic system.

Later in the course, I will show how such limiting arguments can be carriedout.

How to study stochastic models of intracellular processes?

The stochastic models should not be constructed by simply “adding noise” tothe deterministic system.

In fact, the opposite is true; the stochastic models should be developed viafirst principles or through an understanding of the system.

The deterministic models should actually arise as a consequence of somelimiting argument of the stochastic system.

Later in the course, I will show how such limiting arguments can be carriedout.

How to study stochastic models of intracellular processes?

The stochastic models should not be constructed by simply “adding noise” tothe deterministic system.

In fact, the opposite is true; the stochastic models should be developed viafirst principles or through an understanding of the system.

The deterministic models should actually arise as a consequence of somelimiting argument of the stochastic system.

Later in the course, I will show how such limiting arguments can be carriedout.

A quick review of linear differential equations

Consider the first order linear differential equation

y ′1(t) = a11y1(t) + a12y2(t) + · · ·+ a1nyn(t)

y ′2(t) = a21y1(t) + a22y2(t) + · · ·+ a2nyn(t)

...

y ′n(t) = an1y1(t) + an2y2(t) + · · ·+ annyn(t),

(2)

where the aij are constants and y1, . . . , yn are the unknown functions.

The system (2) can be written succinctly via the vector equation

y ′(t) = Ay(t),

where

y(t) =

y1(t)y2(t)

...yn(t)

, and A =

a11 a12 · · · a1n

a21 a22 · · · a2n...

. . .an1 an2 · · · ann

.

A quick review of linear differential equations

Consider the first order linear differential equation

y ′1(t) = a11y1(t) + a12y2(t) + · · ·+ a1nyn(t)

y ′2(t) = a21y1(t) + a22y2(t) + · · ·+ a2nyn(t)

...

y ′n(t) = an1y1(t) + an2y2(t) + · · ·+ annyn(t),

(2)

where the aij are constants and y1, . . . , yn are the unknown functions.

The system (2) can be written succinctly via the vector equation

y ′(t) = Ay(t),

where

y(t) =

y1(t)y2(t)

...yn(t)

, and A =

a11 a12 · · · a1n

a21 a22 · · · a2n...

. . .an1 an2 · · · ann

.

A quick review of linear differential equations

If y(0) = v ∈ Rn, the system

y ′(t) = Ay(t)

has solutiony(t) = eAtv ,

where

eAt =∞∑

k=0

(At)k

k !. (3)

Equation (3) is almost never used to compute eAt . Instead, we usually try todiagonalize the matrix A.

A quick review of linear differential equations

If y(0) = v ∈ Rn, the system

y ′(t) = Ay(t)

has solutiony(t) = eAtv ,

where

eAt =∞∑

k=0

(At)k

k !. (3)

Equation (3) is almost never used to compute eAt . Instead, we usually try todiagonalize the matrix A.

A quick review of linear differential equationsHow do we find eAt?

Suppose that1. A has n distinct eigenvalues, λ1, . . . , λn,

2. with corresponding eigenvectors v1, . . . , vn.

Avi = λivi .

3. Let Q be the matrix whose k th column is vk =⇒ Q−1 exists.

4. Let ei ∈ Rn be the canonical basis

(Q−1AQ)ei = Q−1Avi = λiQ−1vi = λiei .

Therefore, D = Q−1AQ is the diagonal matrix

D =

λ1 0 · · · 00 λ2 · · · 0...

. . .0 0 · · · λn

.

A quick review of linear differential equationsHow do we find eAt?

Suppose that1. A has n distinct eigenvalues, λ1, . . . , λn,

2. with corresponding eigenvectors v1, . . . , vn.

Avi = λivi .

3. Let Q be the matrix whose k th column is vk =⇒ Q−1 exists.

4. Let ei ∈ Rn be the canonical basis

(Q−1AQ)ei = Q−1Avi = λiQ−1vi = λiei .

Therefore, D = Q−1AQ is the diagonal matrix

D =

λ1 0 · · · 00 λ2 · · · 0...

. . .0 0 · · · λn

.

A quick review of linear differential equationsHow do we find eAt?

Suppose that1. A has n distinct eigenvalues, λ1, . . . , λn,

2. with corresponding eigenvectors v1, . . . , vn.

Avi = λivi .

3. Let Q be the matrix whose k th column is vk =⇒ Q−1 exists.

4. Let ei ∈ Rn be the canonical basis

(Q−1AQ)ei = Q−1Avi = λiQ−1vi = λiei .

Therefore, D = Q−1AQ is the diagonal matrix

D =

λ1 0 · · · 00 λ2 · · · 0...

. . .0 0 · · · λn

.

A quick review of linear differential equationsHow do we find eAt?

Suppose that1. A has n distinct eigenvalues, λ1, . . . , λn,

2. with corresponding eigenvectors v1, . . . , vn.

Avi = λivi .

3. Let Q be the matrix whose k th column is vk =⇒ Q−1 exists.

4. Let ei ∈ Rn be the canonical basis

(Q−1AQ)ei = Q−1Avi = λiQ−1vi = λiei .

Therefore, D = Q−1AQ is the diagonal matrix

D =

λ1 0 · · · 00 λ2 · · · 0...

. . .0 0 · · · λn

.

A quick review of linear differential equations

How do we find eAt?

We have:A = QDQ−1,

and so

eAt =∞∑

k=0

Ak tk

k !=∞∑

k=0

(QDQ−1)k

k !tk = Q

(∞∑

k=0

Dk tk

k !

)Q−1 = QeDtQ−1.

eAt = QeDtQ−1 = Q

eλ1t 0 · · · 0

0 eλ2t · · · 0...

. . .0 0 · · · eλn t

Q−1. (4)

Important to note: Large t behavior is dominated by largest eigenvalue of A.

A quick review of linear differential equations

How do we find eAt?

We have:A = QDQ−1,

and so

eAt =∞∑

k=0

Ak tk

k !=∞∑

k=0

(QDQ−1)k

k !tk = Q

(∞∑

k=0

Dk tk

k !

)Q−1 = QeDtQ−1.

eAt = QeDtQ−1 = Q

eλ1t 0 · · · 0

0 eλ2t · · · 0...

. . .0 0 · · · eλn t

Q−1. (4)

Important to note: Large t behavior is dominated by largest eigenvalue of A.

A quick review of linear differential equations

How do we find eAt?

We have:A = QDQ−1,

and so

eAt =∞∑

k=0

Ak tk

k !=∞∑

k=0

(QDQ−1)k

k !tk = Q

(∞∑

k=0

Dk tk

k !

)Q−1 = QeDtQ−1.

eAt = QeDtQ−1 = Q

eλ1t 0 · · · 0

0 eλ2t · · · 0...

. . .0 0 · · · eλn t

Q−1. (4)

Important to note: Large t behavior is dominated by largest eigenvalue of A.