Stochastic Dynamic Programming: The One Sector Growth Modelerossi/Macro/SDP.pdf · 2012. 3. 26. ·...

Stochastic Dynamic Programming:The One Sector Growth Model

Esteban Rossi-HansbergPrinceton University

March 26, 2012

Esteban Rossi-Hansberg () Stochastic Dynamic Programming March 26, 2012 1 / 31

References

We will study stochastic dynamic programing using the application inSection 13.3 in SLP.

It requires previous knowledge of Chapters 8 to 12.


Sequential Problem

The problem we will study is given in sequential form by

max{yt}∞

t=0

E

{∞

∑t=0

βtU (xt − yt )}

s.t. xt+1 = zt f (yt ) t = 0, 1, ...

0 ≤ yt ≤ xt t = 0, 1, ...

x0 ≥ 0 given


Recursive Problem

The functional equation of this problem is given by

v (x) = max0≤y≤x

[U (x − y) + β

∫v [f (y) z ] µ (dz)

](1)

where

x is the quantity of output,

y is the quantity carried over as capital for for next period’sproduction,

and x − y is the quantity consumed.Next period, a technology shock z is realized which is drawn from thedistribution µ, and so f (y) z units of output are produced tomorrow.


Utility

We impose the following assumptions on utility:

U1: 0 < β < 1;

U2: U is continuous;

U3: U is strictly increasing;

U4: U is strictly concave;

U5: U is continuously differentiable.


Technology

We also impose the following assumptions on technology:

T1: f is continuous;

T2: f (0) > 0, and for some x > 0 : x ≤ f (x) ≤ x̄ , all 0 ≤ x ≤ x̄ ,and f (x) < x all x > x̄ .

I Used to bound the state space. In particular define the state space tobe given by X = [0, x̄ ] with Borel sets X

T3: f is strictly increasing;

T4: f is (weakly) concave;

T5: f is continuously differentiable and βf ′ (0) > 1.


Technology

We will impose the following assumptions on technology shocks:

Z1: Z = [1, z̄ ] , where 1 < z̄ < +∞, with Borel sets Z ;I Used for the integral of the functional equation to be well defined.Remember that a Borel Algebra is the smallest σ−algebra in R

containing all open sets. Any set in this σ-algebra is called a Borel set.

Z2: {zt} is an i.i.d. sequence of shocks, each drawn according to theprobability measure µ on (Z ,Z)

I Needed in order for z not to be a state variable.

Z3: For some α > 0, µ ((a, b]) ≥ α (b− a), all (a, b] ⊂ Z .I So that µ assigns probability to all nondegenerate subintervals in Z .


Existence and Uniqueness of Value Function

LemmaThere is a unique bounded continuous function v : X → R satisfying

v (x) = max0≤y≤x

[U (x − y) + β

∫v [f (y) z ] µ (dz)

]Furthermore v is strictly increasing and strictly concave.


Proof: Blackwell

To prove this we want to use Theorem 9.6 in SLP which in turn usesTheorem 4.6 which is an application of the Theorem of the Maximumand the Contraction Mapping Theorem.

In particular, we want to prove that the Blackwell suffi cientconditions for a contraction hold. A contraction (with modeulusβ ∈ (0, 1)) is defined as an operator T : S → S , where (S , ρ) is ametric space, such that

ρ (Tx ,Ty) ≤ βρ (x , y) all x , y ∈ S .


Proof: Blackwell

The Blackwell suffi cient conditions are given by:

1 Monotonicity: If B (X ) is the space of bounded functions on X withthe sup norm, and T : B (X )→ B (X ) is a functional operator. Iff , g ∈ B (X ) and f (x) ≤ g (x), all x ∈ X ⇒ (Tf ) (x) ≤ (Tg) (x),all x ∈ X .

2 Discounting: There exists some β ∈ (0, 1) such that

[T (f + a)] (x) ≤ (Tf ) (x) + βa,

all f ∈ B (X ), a ≥ 0, x ∈ X .If this conditions are satisfied, Theorem 3.3. in SLP guarantees that theoperator T is a contraction.


Proof: CMT

If T is a contraction, then the Contraction Mapping Theorem (3.2in SLP) says that if (B (X ) , ρ) is a complete metric space

I Remember, a space with a metric (positive distance, symmetry, andtriangle inequality)

and every Cauchy sequence convergesI for all ε > 0, ∃N (ε) s.t. ρ (xn , xm) < ε all n,m > N (ε))

and T is a contraction then T has a unique fixed point v and for anyv0,

ρ (T nv0, v) ≤ βnρ (v0, v) .

Theorem 4.6 is an application of this result for bounded continuousfunctions.


Proof: Theorem of the Maximum

We can use the Theorem of the Maximum to say that the policycorrespondence is compact valued and u.h.c.

The Theorem of the Maximum says that if we have two functionsdefined by

h (x) = maxy∈Γ(x )

f (x , y)

G (x) = {y ∈ Γ (x) : f (x , y) = h (x)}

where f is continuous and Γ is compact valued and continuouscorrespondence.

Then h is continuous and G is non-empty compact-valued and u.h.cI for every sequence xn → x and every sequence{yn} such thatyn ∈ Γ (xn) , all n, there exists a convergent subsequence of yn whoselimit point y is in Γ (x) (no open boundaries)


Proof: CMT and Theorem of the Maximum

For our purposes we need to show none of these arguments failsbecause of the integral in (1).

In particular we need to show that∫v [f (y) z ] µ (dz) is bounded and

continuous if v and f are bounded and continuous.I This is trivially true in this case since the conditional probability of z ′

given z is the same as the unconditional probability. That is, thetransition function for the state is such that Q (z , dz ′) = µ (dz ′) .

If this was not the case we would need to guarantee that Q satisfiesthe Feller property, namely that if f is bounded and continuous,∫

f(z ′)Q(z , dz ′

)is also bounded and continuous, which is equivalent to Q beingcontinuous in z .


Proof: Increasing

To show that v is strictly increasing, we will use a corollary of theContraction Mapping Theorem.

First note that the space of bounded, continuous and weaklyincreasing and weakly concave functions is a complete metric space.

I Given this, if the operator T maps this set into the set of strictlyincreasing and concave functions, we know that the fixed point has tobe strictly increasing and strictly concave.

Because of U3 the operator maps increasing functions into strictlyincreasing ones. And because of U4 the same is true for concavity inx and y . Note that Γ (x) = [0, x ] is also increasing in x . Hence v isstrictly increasing. For all θ ∈ [0, 1] if y ∈ Γ (x) and y ′ ∈ Γ (x ′)

θy + (1− θ) y ′ ∈ Γ(θx + (1− θ) x ′

).

Hence v strictly increasing and concave.


Characterization of the Policy Function

LemmaThe policy function g : X → X is single valued, continuous, and strictlyincreasing, with g (0) = 0 and 0 < g (x) < x̄ , all 0 < x ≤ x̄ . Furthermorethe consumption function c : X → X defined by c (x) = x − g (x) is alsostrictly increasing.


Proof

The first part comes from the fact that we are maximizing acontinuous and strictly concave function in a convex set, so there is aunique maximum. Since a single valued u.h.c. correspondence is acontinuous function g is continuous.

To show that g is increasing, suppose not. Then for x > x ′,

U ′ (x − g (x)) ≤ U ′(x ′ − g

(x ′))

which implies by the first order condition,

U ′ (x − g (x)) = β∫v ′ [f (g (x)) z ] f ′ (g (x)) zµ (dz) ,

that∫v ′ [f (g (x)) z ] f ′(g (x))zµ (dz) ≤

∫v ′[f(g(x ′))z]f ′(g(x ′))zµ (dz)

a contradiction with v strictly concave. Hence g is strictly increasing.


Proof

To show that g (0) = 0 notice that Γ (0) = {0}, and for some x ,such that 0 < g (x) < x̄ suppose g (x) = x . Tthis implies, that sinceg (x) is strictly increasing that g (g (x)) = g (x) . But next period weneed g (f (g (x)) z) = g (x) all z . Butg (f (g (x)) z) > g (g (x)) = g (x), for some z . A contradiction.

To show that c (x) is increasing, notice that since v is strictlyconcave, x > x ′ implies v ′ (x) < v ′ (x) and so by T3, T4, and gincreasing∫v ′ [f (g (x)) z ] f ′(g (x) zµ (dz) <

∫v ′[f(g(x ′))z]f ′(g(x ′))zµ (dz) ,

which implies that

U ′ (c (x)) < U ′(c(x ′)).

So by U4 c (x) is increasing.


Differentiability

Lemmav is continuously differentiable with v ′ (x) = U ′ [x − g (x)] .


Proof: Benaviste and Scheinkman

With assumptions U4 and U5 this is just an application of theBenaviste and Scheinkman Theorem (SLP 4.10): v is differentiable atx∗ if there exists a concave, differentiable function W (x) defined ona neighborhood of x∗ with the property that W (x) ≤ v (x) with = ifx = x∗, on that neighborhood.

Define

W (x) = U (x − g (x∗)) + β∫v [f (g (x∗)) z ] µ (dz) ,

then since g (x∗) maximizes the RHS for x = x∗ we know thatW (x) < v (x) . Clearly W ′ (x) = U ′ [x − g (x)] .


Euler Equation

The Euler equation for this problem is therefore given by

U ′ (c (x)) = β∫U ′ [c (f (g (x)) z)] f ′(g (x))zµ (dz) , (2)

and the policy function defines a Markov process on X correspondingto the stochastic equation

xt+1 = f (g (xt )) zt+1.


Transition Function

We can define the transition function P using f , g and µ. For this,define the transition function P by

P (x ,A) = µ (z ∈ Z : f (g (x)) z ∈ A) , all A ∈ X .

In principle need to show that P (x , ·) is a probability measure (seeTheorem 8.9). Notice that since f and g are continuous we know that∫

Xh(x ′)P(x , dx ′

)=∫Zh (f (g (x)) z) µ (dz)

and so if h is a continuous function, the Lebesgue DominatedConvergence Theorem implies that if xn → x

limn→∞

∫Zh (f (g (xn)) z) µ (dz) =

∫Zh (f (g (x)) z) µ (dz) .

Hence P has the Feller property. Since X is compact, we canconclude that:


Invariant Measure

LemmaP has at least one invariant measure.

See Theorem 12.10 in SLP. The proof is an application of Helly’sTheorem that states that if {µn} is a sequence of probabilitymeasures on

(Rl ,B

)with the property that (i) for some a, b ∈ Rl ,

µn ((−∞, a]) = 0 and µn ((−∞, b]) = 1, n = 1, 2, ...

Then there exists a probability measure µ with µ ((−∞, a]) = 0 andµ ((−∞, b]) = 1 and a subsequence of {µn} that converges to it.


Weak ConvergenceWe now want to prove that this invariant measure λ∗ is unique andthat the system converges weakly to it from any initial condition.That is

T ∗nλ0 ⇒ λ∗.

The notation is defined by the operator T ∗. It is an operator thatmaps distributions today, given the transition function, intodistributions tomorrow.In particular, in our example,

(T ∗λ0) (A) =∫xP (x ,A) λ0 (dx) .

A sequence of measures is said to converge weakly if for f ∈ C (S)(the set of bounded continuous functions)

limn→∞

∫fdλn =

∫fdλ or

limn→∞

λn (A) = λ (A) all A ∈ X .


Unique Invariant MeasureTo prove this we want to apply Theorem 12.12 in SLP.The key to prove this result is to show that P is monotone (∀fnondecreasing Tf (x) =

∫f (x ′)P (x , dx ′)), has the Feller property,

and satisfies the Mixing condition.The Mixing conditions says that: There exists c ∈ X , ε > 0, andN ≥ 1 such that

PN (0, [c , x̄ ]) ≥ ε and PN (x̄ , [0, c ]) ≥ ε.

Monotone: Notice that in this case the fact that g is strictlyincreasing implies that the LHS of∫

Xh(x ′)P(x , dx ′

)=∫Zh (f (g (x)) z) µ (dz)

is strictly increasing and so P is monotone (the conditionalexpectation operator associated maps monotone into monotonefunctions).Feller: Shown above.


Mixing

Mixing: This is in general the hardest part of these problems.

Let z∗ be the mean of µ, that is,

z∗ =∫zµ (dz)

and let x∗ be the unique value satisfying

βf ′ [g (x∗)] z∗ = 1. (3)

We want to show that the mixing condition holds for point x∗.


Mixing

For any x ∈ X and z ∈ Z define the sequence {φn (x , z)} by

φ0 (x , z) = x ,

φn+1 (x , z) = f [g (φn (x , z))] z , n = 0, 1, 2, ...

Note that φn (x , z) is the quantity of goods available at the beginningof period n if x is the quantity at the beginning of period 0 and theshock is constant at z .

Note also that since f and g are continuous and strictly increasing,φn is also continuous and strictly increasing in both arguments.

This implies by Z3 that

Pn (x , (φn (x , z̄ − δ) , φn (x , z̄)]) = (µ (δ))n ≥ αnδn and

Pn (x , (φn (x , 1) , φn (x , 1+ δ)]) = (µ (δ))n ≥ αnδn.


Mixing

We want to use these two inequalities to show the result.

Consider the case of x = 0, and therefore the sequence {φn (0, z)}.Since

φ0 (0, z̄) = 0 < f (0) z̄ = f [g (0)] z̄ = φ1 (0, z̄) ,

it follows by induction, since f and g are monotone, that thesequence is nondecreasing.

Note that here assumption T5 is crucial, if it is not satisfied then {0}is an ergodic set.


Mixing

Since the sequence is bounded above by x̄ , it converges to some valueξ ∈ X , where ξ = f [g (ξ)] z̄ . Hence, from the first order condition in(2)

U ′ [c (ξ)] = β∫U ′ [c (f [g (ξ)] z)] f ′ [g (ξ)] zµ (dz)

= βf ′ [g (ξ)]∫U ′[c(

ξzz̄

)]zµ (dz)

> βf ′ [g (ξ)]U ′ [c (ξ)]∫zµ (dz)

= βf ′ [g (ξ)]U ′ [c (ξ)] z∗,

where the inequality follows from z/z̄ < 1 and U4.


Mixing

So1 > βf ′ [g (ξ)] z∗

which implies by (3) and T4 that ξ > x∗. Hence there exists anN ≥ 1 such that φN (0, z̄) > x

∗, and a δ > 0 such thatφN (0, z̄ − δ) > x∗. Hence

x∗ < φN (0, z̄ − δ) < φN (0, z̄) < x̄ ,

and so

PN (0, (x∗, x̄ ]) ≥ PN (0, (φN (0, z̄ − δ) , φN (0, z̄)]) ≥ αnδn.

This shows the first part of the result.


Mixing

To show the second, consider the sequence {φn (x̄ , 1)} which is suchthat

φ0 (x̄ , z̄) = x̄ > f (x̄) ≥ f [g (x̄)] = φ1 (x̄ , 1) .

Thus by induction and f and g nondecreasing, the sequence isnonincreasing. Since it is bounded from below by 0, it converges to avalue υ = f [g (υ)] . And so

U ′ [c (υ)] = β∫U ′ [c (f [g (υ)] z)] f ′ [g (υ)] zµ (dz)

= βf ′ [g (υ)]∫U ′ [c (υz)] zµ (dz)

< βf ′ [g (υ)]U ′ [c (υ)]∫zµ (dz)

= βf ′ [g (υ)]U ′ [c (υ)] z∗,

where the inequality follows from z ∈ Z .


Mixing

Hence1 < βf ′ [g (υ)] z∗

and by T4 υ < x∗. As before, choose N ≥ 1 such thatφN (x̄ , 1) < x

∗, and a δ > 0 such that φN (x̄ , 1+ δ) < x∗. Hence

x∗ > φN (x̄ , 1+ δ) > φN (x̄ , 1) > 0,

and so

PN (x̄ , (0, x∗]) ≥ PN (x̄ , (φN (x̄ , 1) , φN (x̄ , 1+ δ)]) ≥ αnδn.


Stochastic Dynamic Programming: The One Sector Growth Modelerossi/Macro/SDP.pdf · 2012. 3. 26. ·...

Documents

Transcript of Stochastic Dynamic Programming: The One Sector Growth Modelerossi/Macro/SDP.pdf · 2012. 3. 26. ·...