Discrete State Space Methods for Dynamic...

Discrete State Space Methods for Dynamic EconomiesA Brief Introduction

Craig Burnside

Duke University

September 2006

Craig Burnside (Duke University) Discrete State Space Methods September 2006 1 / 42

What are Discrete State Space Methods?

Discrete state space methods are computational techniques used toobtain the solutions of dynamic economic models

If the model in question assumes that the state variables are de�nedover �nite sets of points, the solutions are exact.

Often the model�s state-space is continuous, in which case thediscrete state-space is an approximation to the continuous state-space


Why use Discrete State Space Methods?

Discrete state space methods preserve nonlinearities in situationswhere log-linear approximations may not be accurate, or theiraccuracy is unknown

Properties are backed up by theoretical results on convergence.

Intuitive, usually reliable, and applicable to a wide variety of problems

One signi�cant drawback: the curse of dimensionality.

Computational burden (memory and speed) rises quickly with thenumber of state variables


Outline of this Lecture

Introduction to Gaussian quadrature

A method used in numerical integrationUseful here for developing a discrete state-space approximation to acontinuous state-space

Explain discrete state space methods using two illustrative examples

One risky asset version of Lucas�(1978) consumption-based assetpricing modelSimple stochastic neoclassical growth model

Discuss extensions, drawbacks & limitations


What is Quadrature?

Numerical methods used to approximate integrals by sums areformally referred to as quadrature methods.

Euler equations relate functions of variables today to expectations offunctions of variables at future dates (tomorrow)

e.g. u0(ct ) = βEt (1+ r)u0(ct+1)

When the state-space is discrete, this conditional expectation iscomputed as a sum over the conditional probability distribution.When the state-space is continuous, it involves evaluating an integralover the conditional density function.

We use quadrature methods to approximate conditional expectationsby sums, when they would be exactly represented by integrals


Gaussian QuadratureIntroduction

Suppose we want to evaluate the integralRY ψ(y)f (y)dy

ψ is the kernel functionf is some density function de�ned over the set Y .

Quadrature approximates the integral using a sum:

N

∑i=1

ψ(yi ,N )wi ,N �ZY

ψ(y)f (y)dy ,

with the points (abscissas), fyi ,NgNi=1, and weights, fwi ,NgNi=1, being

chosen according to some rule.

Gaussian quadrature uses a set of rules related to the orthogonalpolynomials corresponding to the density function f .


Gaussian QuadratureOrthogonal Polynomials

Gaussian quadrature with N (the number of points) arbitrarily largerequires that all non-negative integer moments of y exist.

This assumption guarantees the existence of a set of orthogonalpolynomials, fφN (y)g∞

N=0 for the density f (y) determined accordingto the following rules

φN (y) = λN0 + λN1y + λN2y2 + � � � λNNyN , λNN > 0 (1)Z

YφN (y)φM (y)f (y)dy =

�1 if N = M0 otherwise

. (2)

Easy to see that the 0-polynomial is φ0(y) = 1, so (1) and (2) implythat the others are mean zero and have unit variance. I.e. they forman orthonormal basis with respect to f .


Gaussian QuadratureOrthogonal Polynomials for the Standard Normal Density

For N = 1:E (λ10 + λ11y) = 0 =) λ10 = 0E (λ10 + λ11y)2 = 1 =) λ211 = 1

so φ1(y) = y

For N = 2:E (λ20 + λ21y + λ22y2) = 0 =) λ20 = �λ22E (λ20 + λ21y + λ22y2)y = 0 =) λ21 = 0E (λ20 + λ21y + λ22y2)2 = 1 =) 2λ222 = 1

so φ2(y) = (y2 � 1)/

p2

Then φ3(y) =�y2 � 3

�y/p6 . . .

. . . φN (y) =p1/NφN�1(y)y �

p(N � 1)/NφN�2(y)


Gaussian QuadratureAbscissas and Weights

The abscissas for an N-point quadrature rule are the roots of the Nthorthogonal polynomial.

The weights for an N-point Gaussian quadrature rule are chosen sothat if the kernel is a 2N � 1th-or-lower ordered polynomial thequadrature approximation is exact.

Although this represents 2N restrictions in N unknowns, only N of therestrictions are unique.

For a standard normal example

Abscissas Weights1 f0g f1g2 f�1, 1g f 12 ,

12g

3 f�p3, 0,

p3g f 16 ,

23 ,16g


Gaussian QuadratureGaussian quadrature has several nice properties

1 Leads naturally to a discrete state-space interpretation of theapproximation

2 An alternative interpretation connects the method to a whole familyof methods that approximate the kernel function, ψ, by an alternativefunction whose integral is easy to compute

3 A wealth of convergence results showing that the approximationimproves as N gets large.


Gaussian QuadratureGaussian Quadrature Rules Lead to a Natural Discrete State Space Interpretation

A discrete state-space interpretation is natural because it can beshown that the weights sum to 1 for any N, and all admissiblefunctions f .

Implies that the approximation ∑Ni=1 ψ(yi ,N )wi ,N can be interpreted

as E [ψ (y)] when y has a discrete distribution over the set fyi ,NgNi=1with associated probabilities fwi ,NgNi=1.


Gaussian QuadratureGaussian Quadrature has a Weighted Residual Method Interpretation

Gaussian quadrature is equivalent to replacing ψ with a step functionapproximation, ψN , whose integral is computed exactly

There exist points fzi ,NgNi=0 which satisfy zi�1,N < yi ,N < zi ,N suchthat

wi ,N =Z zi ,N

zi�1,Nf (y)dy

implying that

N

∑i=1

ψ (yi ,N )wi ,N =N

∑i=1

ψ (yi ,N )Z zi ,N

zi�1,Nf (y)dy

=ZY

N

∑i=1

ψ (yi ,N ) 1(zi�1,N ,zi ,N )(y)| {z }ψN (y )

f (y)dy .


Gaussian QuadratureConvergence

In cases where Y is a compact set [a, b], f (y) is an arbitrary densityon Y and ψ (y) is any function for which the Riemann-Stieltjesintegral

RY ψ(y)f (y)dy exists, it follows that

limN!∞

N

∑i=1

ψ (yi ,N )wi ,N =ZY

ψ (y) f (y) dy .

In fact, this convergence is uniform.

The CDF corresponding to the discrete distribution, fyi ,NgNi=1,fwi ,NgNi=1 converges pointwise to the CDF corresponding to f (y).


The Lucas ModelSetting up the Model

Economy has N identical agents with instantaneous utility function

U(Ct ) =C 1�γt � 11� γ

,

Ct is consumption at time tγ is the coe¢ cient of relative risk aversion.

Output is obtained from K assets which produce stochasticendowments of a single perishable consumption good.Agent�s budget constraint is

Ct +K

∑k=1

PktSkt+1 �K

∑k=1

(Pkt +Dkt )Skt .

Skt units of asset k at the beginning of time tDkt exogenous stochastic endowment per unit of the asset.Pkt price of the kth asset at time t in units of consumption.


The Lucas ModelOptimization

At time 0 the agent maximizes

E0∞

∑t=0

βtU(Ct ) 0 < β < 1.

by choosing contingency plans fornCt , fSkt+1gKk=1

o∞

t=0subject to

his budget constraint.

First order conditions for this problem are

PktU0(Ct ) = βEtU 0(Ct+1)(Pkt+1 +Dkt+1), k = 1, . . . ,K .

Since the agents are identical if we assume that the total supply ofeach asset is N, then Skt = 1, 8k, t, for every agent, in equilibrium.Then the budget constraint implies Ct = ∑K

k=1 Dkt for all t.


The Lucas ModelEquilibrium

We have a trivial solution for the equilibrium level of consumption interms of the exogenous endowments: Ct = ∑K

k=1 Dkt .

But we do not know how the prices of the assets, which areendogenous, depend on the values of the endowments.

Our goal: characterize the function relating the endowments to theprices of the assets.


Solving the Lucas ModelThe Case of a Single Shock

When K = 1, Ct = Dt and

PtU 0(Dt ) = βEtU 0(Dt+1)(Pt+1 +Dt+1).

Using the fact that U(Ct ) =�C 1�γt � 1

�/(1� γ) we get

PtD�γt = βEtD

�γt+1(Pt+1 +Dt+1).

Express this in terms of the price-dividend ratio Vt = Pt/Dt :

VtD1�γt = βEtD

1�γt+1 (Vt+1 + 1),

orVt = βEtX

1�γt+1 (Vt+1 + 1),

where Xt+1 = Dt+1/Dt .


Solving the Lucas ModelSingle shock is normal and iid (1)

Let xt = ln(Xt ) be distributed Niid(µ, σ2) and de�ne α = 1� γ.Altug and Labadie (1994, p.83):

Vt =θ

1� θif θ = β exp

�αµ+

12

α2σ2�< 1

This (constant) function of xt is the solution to the functionalequation

V (xt ) =Z

β exp(αxt+1) [V (xt+1) + 1] f (xt+1) dxt+1.

Drop the extra N in the notation from now on!Approximate the integral using an N-point Gaussian quadrature rule:

V (xt ) =N

∑i=1

β exp (αyi ) [V (yi ) + 1]wi (3)


Solving the Lucas ModelSingle shock is normal and iid (2)

Use an N-point Gaussian quadrature rule for f � N(µ, σ2):Abscissas Weights

Standard normal yi wiNormal(µ, σ2) yi = µ+ σyi wi = wi

Solve (3) for each xt 2 fyigNi=1 implying N linear equations in Nunknowns

V (yj ) =N

∑i=1

β exp (αyi ) [V (yi ) + 1]wi , j = 1, . . . ,N.

and a trivial solution

V =∑Ni=1 β exp (αyi )wi

1�∑Ni=1 β exp (αyi )wi

.

This is the exact solution for the discrete state space model where xthas a discrete distribution with Pr(xt = yi ) = wi .


Solving the Lucas ModelSingle shock is normal, but persistent (1)

Assumext = µ(1� ρ) + ρxt�1 + εt ,

where jρj < 1 and εt � Niid(0, σ2).Exact solution in Burnside (1998) given by

Vt =∞

∑i=1

βi exp�

α

�µ

�i � ρ(1� ρi )

1� ρ

�+

ρ(1� ρi )

1� ρxt

�+

12

α2

(1� ρ)2

�i � 2ρ(1� ρi )

1� ρ+

ρ2(1� ρ2i )

1� ρ2

�σ2�,

if

β exp�

αµ+12

α2

(1� ρ)2σ2�< 1.

Not a useful solution because it�s an in�nite series of expressions anddoes not converge quickly.


Solving the Lucas ModelSingle shock is normal, but persistent (2)

Write the functional equation as

V (xt ) =Z

β exp(αxt+1) [V (xt+1) + 1] f (xt+1jxt ) dxt+1.

Tauchen and Hussey (1991) suggest the transformation

V (xt ) =Z

β exp(αxt+1) [V (xt+1) + 1]f (xt+1jxt )f (xt+1jµ)

f (xt+1jµ) dxt+1.

Base the N-point rule on the function f (xt+1jµ):

V (xt ) �N

∑i=1

β exp(αyi ) [V (yi ) + 1]f (yi jxt )f (yi jµ)

wi

Solve for xt 2 fyigNi=1 to get N linear equations in N unknowns:

V (yj ) =N

∑i=1

β exp(αyi ) [V (yi ) + 1]f (yi jyj )f (yi jµ)

wi . (4)


Solving the Lucas ModelThe solution for a persistent discrete state space process

Let xt be a simple �rst-order Markov process, with a discretestate-space fyigNi=1.Let Pr(xt+1 = yi jxt = yj ) = πji .

Exact representation of the Euler equation for this model is

V (yj ) =N

∑i=1

β exp(αyi ) [V (yi ) + 1]πji .

In this equation πji replaces the term [f (yi jyj ) /f (yi jµ)]wi thatappeared in (4)

Notice that ∑i πji = 1 but ∑i [f (yi jyj ) /f (yi jµ)]wi 6= 1.


Solving the Lucas ModelSingle shock is normal, but persistent / Discrete state space interpretation

Tauchen and Hussey (1991) suggest solving a modi�ed version of (4):

V (yj ) =N

∑i=1

β exp(αyi ) [V (yi ) + 1]πji (5)

where

πji =f (yi jyj )f (yi jµ)

wisjand sj =

N

∑i=1

f (yi jyj )f (yi jµ)

wi .

By construction ∑i πji = 1.

Also, because limN!∞ sj = 1 for all j so that the solutions to (4) and(5) converge to the same limit.

The solution can be interpreted as the exact solution to a model witha discrete state space.


Models with Endogenous State Variables

In the asset pricing examples I�ve given, the endowments are the onlystate variables and are exogenous

In many macroeconomic models, at least one of the state variables isendogenous.

In this case we don�t know the law of motion of one of the variableswhose state-space we wish to discretize.


Models with Endogenous State VariablesStep 1: Generate a Discrete State Space for the Exogenous Variables

Assume you have an exogenous state variable

xt = µ(1� ρ) + ρxt�1 + εt ,

where jρj < 1 and εt � Niid(0, σ2).Get the N point rule for a N(µ, σ2).

Approximate the law of motion of xt with a Markov process where

Pr(xt+1 = yi jxt = yj ) = πji

and

πji =f (yi jyj )f (yi jµ)

wisjand sj =

N

∑i=1

f (yi jyj )f (yi jµ)

wi .

Easily extended to the case where xt is a vector of exogenousvariables.


Models with Endogenous State VariablesStep 2: Set up a State-Space for the Endogenous Variables

Here you set up a �nite grid of points for the relevant endogenousstate variables

You cannot specify the law of motion of the endogenous statevariables, since it is unknown.

The grid of points must be set up without full knowledge regardingthe nature of the exact solution.


Models with Endogenous State VariablesStep 3: Solve for the Optimal Policy Rule for the Endogenous Variables

For each value of the endogenous state variable, and each value ofthe exogenous variables, solve for the optimal value (tomorrow) of theendogenous state variable

Method 1: Numerical approaches to dynamic programming.

Value function iterationPolicy function iteration

Method 2: Euler-equation based methods

Guess a policy rule and use the Euler equation errors generated by it tore�ne the guess iteratively


The Stochastic Neoclassical Growth ModelModel Setup

Representative agent with lifetime expected utility

U = E0∞

∑t=0

βt ln(Ct ) 0 < β < 1,

Resource constraint is

Ct +Kt+1 � (1� δ)Kt � AtK θt , 0 < δ < 1

Assume that at = ln(At ) has the law of motion

at = ρat�1 + εt , εt � Niid(0, σ2)

A social planner maximizes U by choosing contingency plans for Ctand Kt+1 subject to the resource constraint and K0.


The Stochastic Neoclassical Growth ModelOptimality Conditions

Substitute in the resource constraint and maximize

E0∞

∑t=0

βt lnhAtK θ

t + (1� δ)Kt �Kt+1i.

The Euler equation for this problem is

1AtK θ

t + (1� δ)Kt �Kt+1= βEt

θAt+1K θ�1t+1 + (1� δ)

At+1K θt+1 + (1� δ)Kt+1 �Kt+2

.

(6)

We are looking for a solution Kt+1 = h(Kt ,At ) that solves (6).

The nonlinearity prevents solving for h in closed form unless δ = 1 inwhich case Kt+1 = βθAtK θ

t .


Solving the Growth Model by Value Function Iteration

My discussion follows Tauchen (1990)

Letr(K ,K 0; a) = ln

heaK θ + (1� δ)K �K 0

iStart with the Bellman equation for the original model:

V (K , a) = maxK 02Γ(K ,a)�KC

r(K ,K 0; a) + βZV (K 0, a0)f

�a0ja

�da0

Γ(K , a) =nK 0j0 � K 0 � eaK θ + (1� δ)K

oThe set KC is the continuous state space for K .

Under the assumption of normality, KC = [0,∞).

The next step is to solve an approximating problem.


Solving an Approximation to the Growth ModelDiscrete State Space for Technology

Approximate the law of motion of a using a discrete state-spaceprocess as in the asset pricing model.

Let a 2 A = faigNi=1 where ai = σyi and fyigNi=1 is the set ofquadrature points corresponding to an N-point rule for a standardnormal.

Let

Pr(a0 = ai ja = aj ) = πji =f (ai jaj )f (ai j0)

wisj

where

sj =N

∑i=1

f (ai jaj )f (ai j0)

wi

and fwigNi=1 are the quadrature weights for an N-point rule for thestandard normal.


Solving an Approximation to the Growth ModelDiscrete State Space for the Capital Stock

We will approximate the state space for capital with the discrete setKD � KC with KD = fKmgMm=1.Notice that the maximally sustainable capital stock is nowK = (eaN /δ)1/(1�θ), but there are many ways to set up KD .A number of alternative rules for constructing KD are available. Oneis to locate a grid of evenly spaced points, Km in the set [0, K ].

Tauchen�s (1990) rule: assume linear utility, compute the impliedmean and standard deviation of Kt implied, and then use an equalspaced grid in a �4σK band around E (K ).

My programs set up a grid in the logarithm of the capital stock basedon the E (K ) and σK implied by the log-linear approximation to themodel.


Solving an Approximation to the Growth ModelBellman�s Equation

You can imagine the following two approximations to the originalmodel:

V (K , aj ) = maxK 02Γ(K ,aj )

r(K ,K 0; aj ) + βN

∑i=1V (K 0, ai )πji , (7)

V (K , aj ) = maxK 02ΓD (K ,aj )

r(K ,K 0; aj ) + βN

∑i=1V (K 0, ai )πji , (8)

for j = 1, . . . ,N, and with ΓD (K , a) = KD \ Γ(K , a)

Since, in both problems, there is an upper bound on K , results inBertsekas (1976) imply unique bounded solutions to (7) and (8).

The solution to (8) can be made arbitrarily close to the solution to(7), as long as in the limit, as M ! ∞, with K1 = 0, KM = K ,Km > Km�1, limM!∞ supm(Km �Km�1) = 0.


Solving an Approximation to the Growth ModelComputation

Start with an initial guess at the value function, V0.

Iteratively compute, for m = 1, 2, . . . , M and j = 1, 2, . . . , N:

VS (Km , aj ) = maxK 02ΓD (Km ,aj )

r(Km ,K 0; aj ) + βN

∑i=1VS�1(K

0, ai )πji ,

Continue iterating until

supm,j

��VS (Km , aj )� VS�1(Km , aj )�� < toleranceTauchen suggests relating the tolerance to the minimum value of thevalue function, infm,j

��VS (Km , aj )��.Craig Burnside (Duke University) Discrete State Space Methods September 2006 34 / 42

Solving the Growth Model using the Euler EquationThe Euler Equation

Baxter, Crucini and Rouwenhorst (1990) use a discrete state-spacemethod which directly approximates the decision rules.

De�ne g(K , a) = eaK θ + (1� δ)K andgK (K , a) = θeaK θ�1 + 1� δ.

Looking for a decision rule, h : KC �R ! KC that satis�es the Eulerequation:

1g(K , a)� h(K , a) = β

Z gK [h(K , a), a0]g [h(K , a), a0]� h [h(K , a), a0] f

�a0ja

�da0.

(9)

Start by setting up discrete state-spaces for K and a as above.


Solving the Growth Model using the Euler EquationThe Algorithm

Start with a candidate decision rule hS : KD �A ! KDThe next decision rule is obtained by �nding the functionhS+1 : KD �A ! KD that best �ts the Euler equation.For each m, j , equivalent to �nding K 0 2 ΓD (K , aj ) to minimize�� 1

g(Km , aj )�K 0� β

N

∑i=1

gK (K 0, ai )g(K 0, ai )� hS (K 0, ai )

πji

�� .The algorithm can be stopped if

hS+1(K , a) = hS (K , a) (often occurs in practice)maximum change in the decision rule eventually satis�es some tolerance

Little known about convergence properties.


Extending a Solution to the Real LineA Step Function

In the asset pricing example, the discrete state-space method gave usan approximate solution for any x 2 fyigNi=1.

Denote the solution for x = yi as vi = V (yi ).

What do we do if we want a solution for any x 2 Y , the original statespace?

One solution is to de�ne a function

V (x) =N

∑j=1vj1(zj�1,zj )(x).

A disadvantage of this extension to any x 2 Y is that it is notcontinuous.


Extending a Solution to the Real LineAn Interpolated Spline Function

In the asset pricing example we could de�ne

V (x) =

8><>:v1 if x � y1vj�1 +

(x�yj�1)(yj�yj�1) (vj � vj�1) if yj�1 < x � yj

vN if x > yN .

A disadvantage of this extension to any x 2 Y is that it is notdi¤erentiable in x .


Extending a Solution to the Real LineThe Nystrom Extension

Substitute the solution to the discrete state-space model, fvigNi=1 intothe approximate Euler equation for any x ,

V (x) =N

∑i=1

β exp(αyi ) (vi + 1)

f (yi jx )f (yi jµ)wi

∑Ni=1

f (yi jx )f (yi jµ)wi

.

This function is continuous and di¤erentiable in x , and coincides withthe step function when x is one of the yi s.


Computational Burden

In a univariate example we had to solve N linear equations in Nunknowns.

But suppose we had a multivariate example with M state variables,and suppose the state space for each of these was approximated usingN points

Implies solving NM equations.

Computation time for solving linear equations is approximatelyproportional to the cube of the number of equations, or in our caseN3M .

So computation rapidly becomes more di¢ cult as the number of statevariables or the number of points rises.This is the so-called curse of dimensionality.


Concluding Remarks

Three main advantages of discrete state-space methods.

They are easy to implement and very intuitive.Numerous theoretical results regarding convergence.Reliable.

Main disadvantage of discrete state-space methods.

Computationally ine¢ cient for relatively complicated problems with alarge number of state variables.


Software

Simple software for my chapter in the Marimon and Scott book isavailable on the research page of my website,http://www.duke.edu/~acb8/research.htm


Discrete State Space Methods for Dynamic...

Documents

Transcript of Discrete State Space Methods for Dynamic...