Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c:...
Transcript of Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c:...
![Page 1: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/1.jpg)
Economics 2010c: Lecture 8Brownian Motion and Continuous Time
Dynamic Programming
David Laibson
9/25/2014
![Page 2: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/2.jpg)
Outline: Continuous Time Dynamic Programming
1. Continuous time random walks: Wiener Process
2. Ito’s Lemma
3. Continuous time Bellman Equation
![Page 3: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/3.jpg)
1 Brownian Motion
Consider a continuous time world ∈ [0∞)
Imagine that every ∆ intervals, a process () either goes up or down:
∆ ≡ (+∆)− () =
(+ with prob − with prob ≡ 1−
)
Here ∆ is a fixed interval of time. Later we will let ∆→ 0
(∆) = + (−) = (− )
[(∆)2] = 2 + 2 = 2
(∆) = [∆−∆]2 = [(∆)2]− [∆]2 = 42
![Page 4: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/4.jpg)
• Time span of length implies = ∆ steps in ∆ So ()− (0) isa binomial random variable.
• For example, suppose = 5 and = = 12
Pr [()− (0) = −5] =³50
´05 = 003
Pr [()− (0) = −3] =³51
´14 = 016
Pr [()− (0) = −1] =³52
´23 = 031
Pr [()− (0) = +1] =³53
´32 = 031
Pr [()− (0) = +3] =³54
´41 = 016
Pr [()− (0) = +5] =³55
´50 = 003
![Page 5: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/5.jpg)
• Generally, the probability that
()− (0) = ()() + (− )(−)
is³
´−
• ()− (0) is a binomial random variable with
[()− (0)] = (− ) = (− )∆
[()− (0)] = 42 = 42∆
![Page 6: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/6.jpg)
We will now vary ∆ Begin by letting
= √∆
=1
2
∙1 +
√∆¸
The following five implications follow:
= 1− =1
2
∙1−
√∆¸
(− ) =
√∆
[()− (0)] =
√∆
√∆∆ =
[()− (0)] = 4µ1
4
¶Ã1−
µ
¶2∆
!2∆
∆−→ 2 (as ∆→ 0)
![Page 7: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/7.jpg)
0 10 20 30 40 50 60 70 80 90 100-10
-8
-6
-4
-2
0
2
4
6
8
10Brownian Motion (∆t = 1, σ = 1, α = 0)
Time (t)
x(t)
![Page 8: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/8.jpg)
0 10 20 30 40 50 60 70 80 90 100-10
-8
-6
-4
-2
0
2
4
6
8
10Brownian Motion (∆t = .1, σ = 1, α = 0)
Time (t)
x(t)
![Page 9: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/9.jpg)
0 10 20 30 40 50 60 70 80 90 100-10
-8
-6
-4
-2
0
2
4
6
8
10Brownian Motion (∆t = .01, σ = 1, α = 0)
Time (t)
x(t)
![Page 10: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/10.jpg)
0 10 20 30 40 50 60 70 80 90 100-10
-8
-6
-4
-2
0
2
4
6
8
10Brownian Motion (∆t = .00001, σ = 1, α = 0)
Time (t)
x(t)
![Page 11: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/11.jpg)
1. Vertical movements proportional to√∆ (not ∆)
2. [()− (0)]→ ( 2) since Binomial →Normal
3. “Length of curve during 1 time period” = 1∆
√∆ = 1√
∆→∞
4. ∆∆ =
±√∆
∆ = ±√∆→ ±∞. Time derivative,
³
´, doesn’t exist.
5. (∆)∆ =
22
∆ = . So we write () = .
6. (∆)∆ =
4³14
´³1−()
2∆´2∆
∆ → 2. So we write () = 2.
![Page 12: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/12.jpg)
When we let ∆ converge to zero, the limiting process is called a continuoustime random walk with (instantaneous) drift and (instantaneous) variance2 We generated this continuous-time stochastic process by building it up asa limit case. We could have also just defined the process directly.
Definition 1.1 If a continuous time stochastic process, () is a WienerProcess, then (0)− () satisfies the following conditions:1. (0)− () ∼ (0 0 − )
2. If ≤ 0 ≤ 00 ≤ 000,
h((0)− ())((000)− (00))
i= 0
You can also think of the two condition as:1. (0)− () =
√0 − where ∼ (0 1)
2. non-overlapping increments of are independent
![Page 13: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/13.jpg)
Summary:
• A Wiener process is a continuous time random walk with zero drift andunit variance.
• () has the Markov property: the current value of the process is a suffi-cient statistic for the distribution of future values.
• () ∼ ((0) ) so the variance of () rises linearly with
![Page 14: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/14.jpg)
Generalization: Let () be a Wiener Process. Let () be another continuoustime stochastic process such that,
lim∆→0
∆
∆= ( ) i.e. () = ( )
lim∆→0
∆
∆= ( )2 i.e. () = ( )2
We summarize these properties by writing:
= ( )+ ( )
This is called an Ito Process. Important examples:
• = + (random walk with drift and variance 2)
• = + (geometric random walk with proportional drift and proportional variance 2)
![Page 15: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/15.jpg)
2 Ito’s Lemma
Our goal: work with functions that take an Ito Process as an argument.
• Suppose that the price of oil follows an Ito Process:
= ( )+ ( )
• The value of an oil well will depend on the price of oil and time: ( )
• We would like to be able to write the stochastic process that describes theevolution of :
= ̂( )+ ̂( )
which we will call the total differential of
![Page 16: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/16.jpg)
Theorem 2.1 (Ito’s Lemma) Let () be a Wiener Process. Let () be anIto Process with = ( )+ ( ) Let = ( ) then
=
+
+
1
2
2
2( )2
=
"
+
( ) +
1
2
2
2( )2
#+
( )
![Page 17: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/17.jpg)
Proof: Using a Taylor expansion:
=
+
1
2
2
2()2 +
+
1
2
2
2()2 +
2
+
Any deterministic term of order ()32 or higher is small relative to terms oforder Any stochastic term of order or higher is small relative to terms oforder
√ So,
()2 =
= ( )()2 + ( ) =
()2 = ( )2()2 + = ( )2+
Combining these results, we have our key result:
=
+
+
1
2
2
2( )2
¥
![Page 18: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/18.jpg)
2.1 Intuition:
• Assume ( ) = 0 (no drift in the Ito Process (): () = 0).
• Assume that () = 0 (holding fixed, doesn’t depend on ).
• But, ( ) = 1222
( )2 6= 0
• If is concave (convex), is expected to fall (rise) due to variation in
• For Ito Processes, ()2 behaves like ( )2 so the effect of concavity(convexity) is of order and can not be ignored when calculating thedifferential of
![Page 19: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/19.jpg)
• () = log(), so 0 = 1 and
00 = − 12
• = +
=
"
+
( ) +
1
2
2
2( )2
#+
( )
=∙0 +
1
+
1
2
µ− 1
2
¶22
¸+
1
=∙− 1
22¸+
• falls below due to concavity of
![Page 20: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/20.jpg)
3 Continuous time Bellman Equation
Let ( ) = instantaneous payoff function, where is state variable, iscontrol variable and is time. Let 0 = +∆ and 0 = +∆ So
( ) = max
n( )∆+ (1 + ∆)−1 (0 0)
o(1 + ∆) ( ) = max
n(1 + ∆)( )∆+ (0 0)
o ( )∆ = max
n(1 + ∆)( )∆+ (0 0)− ( )
oMultiply out and let ∆→ 0 Terms of order ()2 = 0
( ) = max{( )+( )} (*)
![Page 21: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/21.jpg)
Now substitute in for ( ) using Ito’s Lemma:
=
"
+
+
1
2
2
22#+
where = ( ), = ( ) and = ( ) + ( )
Since, = 0 we have,
( ) =
"
+
+
1
2
2
22#
Substituting this expression into equation (*), we get
( ) = max
(( )+
"
+
+
1
2
2
22#
)which is a partial differential equation (in and ).
![Page 22: Economics 2010c: Lecture 8 Brownian Motion and Continuous ...Sep 25, 2014 · Economics 2010c: Lecture 8 Brownian Motion and Continuous Time Dynamic Programming David Laibson 9/25/2014](https://reader035.fdocuments.in/reader035/viewer/2022071502/612162b190eaec38dd11cff4/html5/thumbnails/22.jpg)
Outline: Continuous Time Dynamic Programming
1. Continuous time random walks: Wiener Process
2. Ito’s Lemma
3. Continuous time Bellman Equation