Value Function Methods Quantitative Macroeconomics

Value Function MethodsQuantitative Macroeconomics

Raul Santaeulalia-Llopis

MOVE-UAB and Barcelona GSE

Fall 2018

Raul Santaeulalia-Llopis (MOVE-UAB,BGSE) QM: Value Function Methods Fall 2018 1 / 68

1 Introduction

2 Workhorse: The Neoclassical ModelRecursive Competitive Equilibrium (RCE)Pareto Optimal Equilibrium

3 VFI with Global Approximation of the Value FunctionDiscrete Methods

Deterministic ModelSpeeding-up the AlgorithmAn Extension: Adding Labor SupplyAn Extension: Stochastic ModelDiscretization of exogenous processes

Continuous Methods

4 More tricks: EGM and Precomputation of Integrals


Value Function Methods

The value function iteration algorithm (VFI) described in our previous set of slides[Dynamic Programming] is used here to solve for the value function in the neoclassicalgrowth model. We will discuss first the deterministic model, then add a productivityshock to discuss the stochastic version.1

• We focus on the application of global methods to approximate for the valuefunction using both discretization and continuous methods. For a detaileddiscussion of these methods see the set of slides [Function Approximation.pdf]

• We briefly note the existence of local methods to solve for the value function usinglinear-quadratic approximations.

1These slides build on notes by Carlos Urrutia ’Metodos Numericos para Resolver Modelos Macroeconomicos Dinamicos’,

and the courses by Jose-Vıctor Rıos-Rull and Makoto Nakajima (check their respective websites). Read further textbookreferences in the syllabus.


Workhorse: The Neoclassical Model

Sequential formulation. Consider a decentralized economy populated by a largenumber, Nt , of identical infinitely lived households. Each of them maximizes thediscounted future stream of utility,

max{ct ,kt+1}∞t=0

E0

∞∑t=0

βtu(ct , 1),

subject to a budget constraint,

ct + kt+1 = wt + (1 + rt − δ)kt ,

given k0 and z0.An aggregate firm maximizes profits (an static problem),

Yt − wtNt − rtKt ,

subject to aggregate technology

Yt = F (Kt ,Nt).

Remark. We will work with the recursive formulation of this problem (next). For anequivalence theorem between the sequential and recursive formulations see SLP.


Recursive Competitive Equilibrium (RCE)

A RCE is a set of functions v(k,K), c(k,K), k ′(k,K); prices w(K) and r(K); and anaggregate law of motion Γ(K) such that,

1 Given w , r and Γ, the value function v(k,K) solves the Bellman equation,

v(k,K) = maxc≥0,k′

u(c) + βv(k ′,K ′) (1)

such that

c + i = w(K) + r(K)k

k ′ = i + (1− δ)k

K ′ = Γ(K).

Associated to the value function we have the set of optimal decision rules c(k,K)and k ′(k,K).

2 Factor prices equate marginal products, r(K) = f ′(K) andw(K) = f (K)− f ′(K)K .

3 Market clearing, f (K) = c(k,K) + i(k,K).

4 Aggregate consistency, Γ(K) = k ′(K ,K).


Pareto Optimal Equilibrium

Remark. Welfare theorems hold, that is, the RCE is Pareto Optimal. We define therecursive PO equilibrium as a set of functions v(k), c(k), k ′(k) that solve

v(k) = maxc≥0,k′

u(c) + βv(k ′) (2)

such that

c + i = f (k)

k ′ = i + (1− δ)k

That is, the PO problem is

v(k) = maxk′∈[0,f (k)+(1−δ)k]

u(f (k) + (1− δ)k − k ′) + βv(k ′) (3)


Remark. Remember that the Bellman principal of optimality ensures that we canrecover the sequential solution using the recursive solution. To see this, note thatstarting with an initial k0 we can use the optimal decision rules of the recursive problemto, recursively, get the sequence

k1 = k ′(k0)

k2 = k ′(k1) = k ′(k ′(k0))

and so on for all t ≥ 3.


VFI with Global Approximation of the Value Function

Remark. Remember that the value function algorithm (VFI) holds independently ofwhat method we use to approximate V .

Here, we will start by discretezicing V to approximate the value function. Later we willapproximate V with finite element methods, as well as weighted residual methods thatwe have seen in our previous sessions.


Discretization

• STEP 1: The continuous variable k is discretized by defining a grid for the valuesof k, that is, K = (k1, k2, ..., ki , ..., kp) with k1 = kmin, and kp = kmax . For example,an evenly spaced grid is

ki = kmin + (i − 1) η

with η = kmax−kminp−1

. Obviously, with higher p, and smaller η, the approximation willbe more precise, but more costly to compute.

For the kmin we should choose a value slightly higher than zero to avoid potentialproblems of having zero output, and perhaps zero consumption. For kmax there isno rule, but its choice may depend on what we are interested in. For instance, ifwe know the steady state k∗, and we are interested only in dynamics below thesteady state, we can safely choose kmax to be slightly above the steady state.

Here we choose the discretatization to uniformily distributed over the state space.However, if we knew where a potential kink is, or the regions of higher curvature,we might be interested in clustering points in those regions. That is, we would notneed much points in regions where the value function is known to be linear (orclose to).


• STEP 2: Guess a solution V s=0 ∈ Rp, say the null vector, that is, V s=0(ki ) = 0for all i = {1, ..., p}. s keeps track of the number of iterations.

An alternative is to set V s=0i = u(f (ki )+(1−δ)ki−ki )

1−β . That is, kj = ki .


• STEP 3: Define the return matrix M as follows,

M =

m(k1, k1) m(k1, k2) ... m(k1, kp)m(k2, k1) m(k2, k2) ... m(k2, kp)

... ... ... ...m(kp, k1) m(kp, k2) ... m(kp, kp)

.The generic element in matrix M contains the return funcion

Mi,j = m(ki , kj) = u (f (ki ) + (1− δ)ki − kj)

evaluated at all possible combinations ki ∈ K and kj ∈ K .


• STEP 4: However, we know that not all values for c, hence k ′, are feasible. Thenonnegativity restriction on consumption cij = f (ki ) + (1− δ)ki − kj ≥ 0, impliesthat kj ≤ f (ki ) + (1− δ)ki .

Therefore, we replace Mij = m(ki , kj) = ω if kj > f (ki ) + (1− δ)ki , where ω is avery negative number.


• STEP 5: Given the vector V s = (V s1 ,V

s2 , ...,V

si , ...,V

sp ) and the matrix M:

• STEP 5.1: Compute the matrix, χ,

χ =

M11 + βV s

1 M12 + βV s2 ... M1p + βV s

p

M21 + βV s1 M22 + βV s

2 ... M2p + βV sp

... ... ... ...Mp1 + βV s

1 Mp2 + βV s2 ... Mpp + βV s

p

.where the generic element of χ is

χi,j = Mi,j + βV sj .

• STEP 5.2: Compute the updated value function V s+1 as the maximumelement in each row of χ,

V s+1 =

max{χ11, χ12, ..., χ1p}max{χ21, χ22, ..., χ2p}

.........max{χp1, χp2, ..., χpp}

.


• STEP 6: If ||V s+1 − V s || < ε, stop and report success. Otherwise, go back to theprevious step, STEP 5, replacing s = s + 1.


• STEP 7: Being thorough.

• Check the bounds of the state space are not binding. Otherwise, relax them.

• Make sure ε is small enough. Reduce ε, redo the VFI, and check your answerdoes not change. Otherwise, keep reducing ε.

• Make sure the size of the grid, p, is large enough. Increase p, redo the VFI,and check your answer does not change. Otherwise, keep increasing p.


• Remark. Once we know V s , the decision rule is obtained as

g s =

arg max{χ11, χ12, ..., χ1p}arg max{χ21, χ22, ..., χ2p}

.........arg max{χp1, χp2, ..., χpp}

.That is, g s is a column vector attaining values g s

i ∈ {1, ..., p} that state thecolumn number j that maximizes row i .

• Remark. It is going to be useful to store the decision rule at each iteration s forthe purposes of implementing some of the speeding-up algorithms that we will seenext.

• Remark. We can then find the optimal sequence of capital for a given initialcapital.


Speeding-up the VFI Algorithm

• We explore four ways of speeding up the VFI algorithm,

• Using the monotonicity of the optimal decision rule.• Using the concavity of the value function.• Local search.• Howard’s policy iteration step.


Speeding-up the VFI Algorithm I: Monotonicity of the optimal decision rule

• Here we pursue to reduce the number of points to be searched on the grid by theVFI algorithm.

• To do so, we use the known property that the optimal decision rule increases in K .Therefore, if kj > ki , then g(kj) ≥ g(ki ).2

• Hence, if we want to find g(kj) when kj > ki , we can rule out searching for all gridvalues of capital that are smaller than g(ki ).

• This means we can modify STEP 5.1 in the VFI.

2When strictness applies the equality arises if grids are too loose.


• Replace STEP 5.1

• For each ki , set an optimal lower bound for g s(ki ) as

g s(ki ) = g s(ki−1).

• Compute the matrix χ only for the pairs (ki , kj) that satisfy the lowerbound, kj ≥ g s(ki ).

• That is, not all elements in χ need to be computed.

• Note that you will have to store the decision rule at the end of eachiteration to implement this improvement.


Speeding-up the VFI Algorithm II: Concavity of the value function

• We use the known property that the maximand in the Bellman equation,Mi,j + βVj , is strictly concave in k ′.

• Therefore, if Mi,j + βVj > Mi,j+1 + βVj+1, then Mi,j + βVj > Mi,j+2 + βVj+2.

• This means we can modify STEP 5.1 in the VFI to improve the efficiency of thealgorithm.


• Replace STEP 5.1

• When computing the elements of the matrix χ apply the following loopto each row i ,

do j = {1, p}χi,j = Mi,j + βVj

if χi,j−1 > χi,j, stop & report the optimal value is χi,j−1.

end do

• That is, not all elements in χ need to be computed.


Speeding-up the VFI Algorithm III: Local search

• We exploit the property that the optimal decision rule is continuous.

• Hence, if we know that kj = g(ki ), and we we have reasonable fine grids, then wehope that g(ki+1) is in a small neigborhood of kj . This restricts the elements of χthat need to be computed.

• Local search is followed by practioners as it has been proven useful. But there isno theorem behind this.

• We can make sure that our solution is the one that is not limited to search locallyby checking that the bounds that define the neigborhood are not binding at eachiteration.

• Note that you will have to store the decision rule at the end of each iteration toimplement this improvement.


Speeding-up the VFI Algorithm IV: Howard’s policy iteration

• This method does not rely on the properties of the optimal value function, ordecision rule.

• We can apply a guessed decision rule many times to update the value functionmany times, without solving for the optimal rule at each iteration.

• We will need to decide how many times, nh, to update the value function withoutupdating the decision rule.

• If we have a decision rule that is not far from the optimal one, higher nh impliesfaster convergence, but iterating on the value function while keeping a decisionrule obtained in early stages of the VFI might not be that useful.

• Note that you will have to store the decision rule at the end of each iteration toimplement this improvement.


Remark. Note that the speeding-up algorithms are not exclusive. We can use anycombination of them to speed up the VFI.


An Extension: Adding Labor Supply

[Your homework]


An Extension: The Stochastic Model

• Productivity is subject to a stochastic process. At each period, output is

Yt = zt f (Kt)

and decisions are taken before the realization of zt is observed.

• To apply the VFI we assume that zt takes a finite number of q values,Z ∈ (z1, z2, ..., zq), and follows a first order Markov process that has a squaretransition matrix Πq×q with generic element

πl,l′ = Prob[zt+1 = zl′ |zt = zl ],

with∑q

l′=1 πl,l′ = 1.


A stochastic RCE is a set of functions v(k,K , z), c(k,K , z), k ′(k,K , z); prices w(K , z)and r(K , z); and an aggregate law of motion Γ(K , z) such that,

1 Given w , r and Γ, the value function v(k,K ,Z) solves the Bellman equation,

v(k,K , z) = maxc≥0,k′

u(c) + βEZv(k ′,K ′, z ′) = maxc≥0,k′

u(c) + β

q∑l′=1

πl,l′v(k ′,K ′, z ′)

such that

c + i = w(K , z) + r(K , z)k

k ′ = i + (1− δ)k

K ′ = Γ(K , z)

Associated to the value function we have the set of optimal decision rulesc(k,K , z) and k ′(k,K , z).

2 Factor prices equate marginal products, r(K , z) = zf ′(K) andw(K , z) = zf (K)− zf ′(K)K .

3 Market clearing, zf (K) = c(k,K , z) + i(k,K , z).

4 Aggregate consistency, Γ(K , z) = k ′(K ,K , z).


• Remark. To obtain the optimal time paths for we need to simulate a path for theproductivity shocks.

• Remark. Welfare theorems hold. The stochastic recursive PO equilibrium as a setof functions v(k, z), c(k, z), k ′(k, z) that solve

v(k, z) = maxc≥0,k′

u(c) + βEZv(k ′, z ′) = maxc≥0,k′

u(c) + β

q∑l′=1

πl,l′v(k ′, z ′) (4)

such that

c + i = zf (k)

k ′ = i + (1− δ)k

That is, the PO problem is

v(k) = maxk′∈[0,zf (k)+(1−δ)k]

u(zf (k) + (1− δ)k − k ′) + β

q∑l′=1

πl,l′v(k ′, z ′) (5)


Discretization

• STEP 1: Define a grid for the values of capital ki , that is,K = (k1, k2, ..., ki , ..., kp) and a grid for the values of the productivity shock zi ,Z = (z1, z2, ..., zq).


• STEP 2: Define the matrix M as follows,

Mpq×p =

m(k1, k1, z1) m(k1, k2, z1) ... m(k1, kp, z1)m(k2, k1, z1) m(k2, k2, z1) ... m(k2, kp, z1)

... ... ... ...m(kp, k1, z1) m(kp, k2, z1) ... m(kp, kp, z1)m(k1, k1, z2) m(k1, k2, z2) ... m(k1, kp, z2)m(k2, k1, z2) m(k2, k2, z2) ... m(k2, kp, z2)

... ... ... ...m(kp, k1, z2) m(kp, k2, z2) ... m(kp, kp, z2)

... ... ... ...m(k1, k1, zq) m(k1, k2, zq) ... m(k1, kp, zq)m(k2, k1, zq) m(k2, k2, zq) ... m(k2, kp, zq)

... ... ... ...m(kp, k1, zq) m(kp, k2, zq) ... m(kp, kp, zq)

.

The generic element in matrix M contains the return function

Mp(l−1)+i,j = m(ki , kj , zl) = u (zl f (ki ) + (1− δ)ki − kj)

evaluated at all possible combinations ki ∈ K , kj ∈ K , and zl ∈ Z .


• STEP 3: However, we know that not all values for c, hence k ′, are feasible. Thenonnegativity restriction on consumption implies,

Mp(l−1)+i,j = m(ki , kj , zl) = ω

if kj > zl f (ki ) + (1− δ)ki , where ω is a very negative number.


• STEP 4: Guess a solution V s=0 ∈ Rpq, say the null vector, that is, V s=0(ki , zl) = 0for all i = {1, ..., p} and l = {1, ..., q}. s keeps track of the number of iterations.


• STEP 5: Given the vector V s = (V s1 ,V

s2 , ...,V

spq), with generic element

V sp(l′−1)+j = V s(kj , zl′),

compute the following in-steps:

• Step 5.0: Compute the matrix W sq×p,

∑ql′=1 π1,l′V

sp(l′−1)+1

∑ql′=1 π1,l′V

sp(l′−1)+2 ...

∑ql′=1 π1,l′V

sp(l′−1)+p∑q

l′=1 π2,l′Vsp(l′−1)+1

∑ql′=1 π2,l′V

sp(l′−1)+2 ...

∑ql′=1 π2,l′V

sp(l′−1)+p

... ... ... ...∑ql′=1 πq,l′V

sp(l′−1)+1

∑ql′=1 πq,l′V

sp(l′−1)+2 ...

∑ql′=1 πq,l′V

sp(l′−1)+p

,with generic element

W sl,j =

q∑l′=1

πl,l′Vsp(l′−1)+j


• ... (continued)

• Step 5.1: Compute the matrix χpq×p,

M11 + βW s11, M12 + βW s

12, ..., M1p + βW s1p

M21 + βW s11, M22 + βW s

12, ..., M2p + βW s1p

... ... ... ...Mp1 + βW s

11, Mp2 + βW s12, ..., Mpp + βW s

1p

Mp+1,1 + βW s21, Mp+1,2 + βW s

22, ..., Mp+1,p + βW s2p

Mp+2,1 + βW s21, Mp+s,2 + βW s

22, ..., Mp+2,p + βW s2p

... ... ... ...Mp+p,1 + βW s

21, Mp+p,2 + βW s22, ..., Mp+p,p + βW s

2p

... ... ... ...Mp(q−1)+1,1 + βW s

q1, Mp(q−1)+1,2 + βW sq2, ..., Mp(q−1)+1,p + βW s

qp

Mp(q−1)+2,1 + βW sq1, Mp(q−1)+2,2 + βW s

q2, ..., Mp(q−1)+2,p + βW sqp

... ... ... ...Mp(q−1)+p,1 + βW s

q1, Mp(q−1)+p,2 + βW sq2, ..., Mp(q−1)+p,p + βW s

qp

,

with the generic element

χp(l−1)+i,j = Mp(l−1)+i,j + βW sl,j


• ... (continued)

• Step 5.2: Compute the updated value function V s+1 as the maximumelement in each row of χ,

V s+1 =

max{χ1,1, χ1,2, ..., χ1,p}max{χ2,1, χ2,2, ..., χ2,p}..., ..., ..., ...max{χp,1, χp,2, ..., χp,p}max{χp+1,1, χp+1,2, ..., χp+1,p}max{χp+2,1, χp+2,2, ..., χp+2,p}..., ..., ..., ...max{χp+p,1, χp+p,2, ..., χp+p,p}..., ..., ..., ......, ..., ..., ...max{χp(q−1)+1,1, χp(q−1)+1,2, ..., χp(q−1)+1,p}max{χp(q−1)+2,1, χp(q−1)+2,2, ..., χp(q−1)+2,p}..., ..., ..., ...max{χp(q−1)+p,1, χp(q−1)+p,2, ..., χp(q−1)+p,p}

.


• STEP 6: If ||V s+1 − V s || < ε, stop and report success. Otherwise, go back to theprevious step, STEP 5, replacing s = s + 1.


• Remark. We can store the decision rule as

g s =

arg max{χ1,1, χ1,2, ..., χ1,p}arg max{χ2,1, χ2,2, ..., χ2,p}..., ..., ..., ...arg max{χp,1, χp,2, ..., χp,p}arg max{χp+1,1, χp+1,2, ..., χp+1,p}arg max{χp+2,1, χp+2,2, ..., χp+2,p}..., ..., ..., ...arg max{χp+p,1, χp+p,2, ..., χp+p,p}..., ..., ..., ......, ..., ..., ...arg max{χp(q−1)+1,1, χp(q−1)+1,2, ..., χp(q−1)+1,p}arg max{χp(q−1)+2,1, χp(q−1)+2,2, ..., χp(q−1)+2,p}..., ..., ..., ...arg max{χp(q−1)+p,1, χp(q−1)+p,2, ..., χp(q−1)+p,p}

.


• Explicitly, the optimal value function V (k, z) and the decision rule g(k, z) take theform of vectors,

V =

v(k1, z1)v(k2, z1)

...v(kp, z1)v(k1, z2)v(k2, z2)

...v(kp, z2)

...v(k1, zq)v(k2, zq)

...v(kp, zq)

g =

g(k1, z1)g(k2, z1)

...g(kp, z1)g(k1, z2)g(k2, z2)

...g(kp, z2)

...g(k1, zq)g(k2, zq)

...g(kp, zq)


• To find the optimal sequence k0, k1, ..., kT first we need to simulate a history ofrealizations of productivity shocks z0, z1, ..., zT . To simulate these productivityshocks we use the transition matrix Π.

• Using the simulated series z0, z1, ..., zT , and an initial k0, we can find the optimalsequence k0, k1, ..., kT using the decision rule g .

• Given this time-series for the productivity shock and capital, we can recover the alltime-series in the model, consumption, output and factor prices.

• Therefore, we can compute statistics on variance, correlations, and joint dynamics.Before doing so, we usually filter them using some of the techniques in ourprevious slides [Data: Some Tools and Sources.pdf].


Solving the competitive equilibrium directly using VFI

• VFI is fairly standard as a solution method for pareto optimal economies, but notso commonly used for decentralized economies.

• The fact that we have two states, k and K , is not the reason. We have seen howto deal with two states earlier in the economy with stochastic productivity shocks.

• The problem of using VFI in decentralized economies is that agents need to know(at least, in expected terms) the aggregate law of motion (i.e., future prices)before optimizing, while out of the VFI we only know this object after solving forthe equilibrium.

• One way to circumvent that problem is to add one STEP to the VFI algorithm.What this STEP does is provide a guess a for the aggregate law of motion (i.e.,this is again a function approximation problem). Given that guess for theaggregate law of motion, solve for the decentralized economy using VFI. Once theoptimal decision rule is obtained, we compare it to the aggregate law of motion. Ifthey are close, we stop. Otherwise we update the aggregate law of motion to bethe last optimal decision rule and start again the VFI.

• There is no guarantee of convergence.


Some limitations

• While highly applied to economies in which the welfare theorem holds, it is not thateasy to solve directly for the competitive equilibrium using value function iteration.

• Increasing the state variables (or the number of points in the grid) implies thedimension of the matrix χ will grow exponentially making the algorithm verytime-consuming.

• We can only introduce shocks of a particular structure, primarily, Markov processesof order one.


Discretization of exogenous processes

1 Tauchen (1986) procedure to discretize an AR(1) stochastic process.It has a multivariate version, i.e., a VAR (Tauchen and Hussey, 1991).

2 Rouwenhorst (1995) for the case where the persistence parameter isclose to 1. See Kopecky and Suen (2010).


Discretization of exogenous processesTauchen Method

Solving the growth model with a continuous shock can be done, but it is verycostly, computationally.

So it’s useful to learn how to approximate a continuous process through afinite-state Markov chain that will mimic closely the underlying process.


Goal: Discretize an AR(1) stochastic process

zt = ρzt−1 + εt

with stationary distribution N(0, σ2z ), where σz = σε√

1−ρ2

This is a typical example for income processes that are assumed Yt = exp(yt)where yt follows the AR(1) of the class posed above.

To approximate the AR(1) process above we need two ingredients: the points onthe state space and the transition probabilities.


Example: If the Markov Chain we want to use is a simple 2-state chain then a simplemethod of moments can be used. Symmetry in the original process implies that theMarkov chain for log-income can be written as the vector [−z ,+z] and that thetransition probability matrix has the form,[

π 1-π1-π π

]so we simply need to find the value for parameters z and π in the MC.

These can be found by equating the variance and the serial correlation in the AR(1) andin the Markov chain (MC), i.e.,

σ2AR(1) ≡

σ2ε

1− ρ2= z2 ≡ σ2

MC

EAR(1)(ytyt−1) ≡ ρ σ2ε

1− ρ2= (2π − 1)z2 ≡ EMC (ytyt − 1)

which implies

z =

√σ2ε

1− ρ2and π =

1 + ρ

2

The Tauchen method is more general.


Tauchen Method (1986)

Step 1 Set n, the number of potential realizations of the process z (usually 5 or 6 works well).

Step 2 Set the upper zmax and lower zmin bounds for the process. An intuitive wa, apealing tothe symmetry of the normal distribution, is to pick m and set:

zmax = mσz

zmin = −mσz ,

where usual values of m are 2 and 3.

Step 3 Set {zi}ni=1 such that: zi = zmin + zmax−zminn−1

(i − 1) and construct the midpoints {z}n−1i=1 ,

which are given by zi =zi+1+zi

2.

Step 4 The transition probabilities πij ∈ Πz,z′ , i.e.,the probability of going to state zj conditionalon being on state zi , is computed according to:

πij = Φ

(zj − ρzi

σ

)− Φ

(zj−1 − ρzi

σ

)j = 2, 3, ..., n − 1

πi1 = Φ

(z1 − ρzi

σ

)πin = 1− Φ

(zn−1 − ρzi

σ

)where Φ(.) denotes a CDF of a N(0, 1).


For example, assume that we estimated a stochastic process to be:

zt = 0.95zt−1 + εt with εt ∼ N(0, 0.00072)

and we want to approximate it using a 5-point Markov change and m = 3.

Tauchen’s procedure implies the following grid for zt ,

zt ∈ {−0.0673,−0.0336, 0, 0.0336, 0.0673}

and the following transition matrix:

Πz,z′ =

0.9727 0.0273 0 0 00.0041 0.9806 0.0153 0 0

0 0.0082 0.9837 0.0082 00 0 0.0153 0.9806 0.00410 0 0 0.0273 0.9727

Note how the entries in the diagonal are close to 1 (the persistence of the continuousstochastic process is high).

[Credit for this example goes to the HdBook chapter by Fernandez-Villaverde, Rubio-Ramirez and Schorfheide]


[Homework:] Assume that zt is a stochastic process for household income that followsthe Markov chain that we have computed in the previous example:

• How many periods does it take to go from the lowest quintile to the highest?

• Does the monotone mixing condition hold in this case?

• What is a stationary distribution? Does it exist in this case? If yes, compute it?

• What are useful statistics for income mobility?


Discretization of exogenous processesTauchen for Multivariate Processes

Tauchen describes how to approximate also a multivariate process of the form

yt = Ayt−1 + εt

where yt is now a n × 1 vector, A is a n × n matrix and εt is a n × 1 vector of iidrandom variables with mean 0 and variance-covariance matrix Σ.


The key insight in representing such a multivariate process is to recognize thatthe variance covariance matrix (which is symmetric) can be represented as

Σ = QΛQ ′

where Λ is a diagonal matrix and Q is the matrix of eigenvectors of Σ as columns(remember that since Σ is symmentric Q ′ = Q1) so we can write the process as,

yt = Ayt−1 + εt

Q ′yt = Q ′AQQ ′yt−1 + Q ′εt

yt = Ayt−1 + εt

where yt = Q ′yt and A = Q ′AQ and εt has a diagonal variance-covariance matrixQ ′ΣQ = Λ.


We can then approximate yt with a Markov chain (since the εt are uncorrelateddoing so is a straightforward extension of the procedure described for theunivariate case) and let Y be the n ×m matrix containing the states of theMarkov chain which approximate yt (here we assume that the Markov chain hasthe same number of states (m) for each of the n variables). Then, the states of

the Markov chain which approximate yt are simply given by the matrix Y = QY .


As an example consider a vairance of a bivariate iid process yt = εt where εt hasa variance-covariance matrix given by:

Σ =

[1.0 0.80.8 1.0

]=

[-0.7071 0.70710.7071 0.7071

] [0.2 0.00.0 0.8

] [-0.7071 0.70710.7071 0.7071

]

[Credit for this example goes to Perri’s lecture notes]


Using this transformation the figure plots the grids for yt and for yt :


Note that this this strategy that can be used to approximate higher-orderautoregressive processes as well. For example, consider the AR(2) process:

yt = ρ1yt−1 + ρ2yt−2 + εt

Define a column vector Zt = [yt yt−1]′. Then one can write the AR(2) process asa multivariate process as:

Zt =

[ρ1 ρ2

0 1

]Zt−1 + [εt 0]′


Discretization of exogenous processesRouwenhorst (1995)

Kopecky and Suen (2010) show that an alternative procedure proposed byRouwenhorst (1995) is superior to Tauchens method when , thepersistence of the stochastic process, is close to 1.


Rouwenhorst Method (1995)

Step 1 Set n, the number of potential realizations of the process z (usually 5 or 6 works well).

Step 2 Set the upper zmax and lower zmin bounds for the process. Let zmin = −λ and zmax = λ,where λ can be set equal to

√n − 1σz .

Step 3 Set {zi}ni=1 such that: zi = zmin + zmax−zminn−1

(i − 1).

Step 4 When n = 2, let

Π2 =

[p 1− p

1− q q

]

where p and q can be set to be p = q = 1+ρ2

.

Step 5 When n ≥ 3, we recursively construct the transition matrix:

Πn = p

[Pn−1 00’ 0

]+ (1− p)

[0 Pn−1

0 0’

]+ (1− q)

[0’ 0

Pn−1 0

]+ q

[0 0’0 Pn−1

]where 0 is an (n − 1)× 1 column vector of zeros. Divide all but the top and bottom rowsby 2 so that the sum of the elements of each row is equal to one. The final outcome isΠz,z′ .


Continuous Methods

• Here we use a continuous method to solve the neoclassical growht model using,again, the value function algorithm (VFI).

• We approximate the value function using a linear combination of a set of m knownlinearly independent basis functions ψj(k), j = 0, ...,m

V (k) =m∑j=0

θjψj(k) (6)

with m-basis coefficients θj to be determined.

• That is, solving for V (k) reduces to find the θj coefficients.


• In our working example, the continuous method that we use to approximate forV (k) is finite elements. That is, we are going to choose the basis functions to benonzero over some subintervals of the state space, but not all. 3

• Precisely, we use the B-spline formulation to approximate V (k). This formulationsubscribes to (6). 4

• What follows applies to B-splines of any order.

3Alternatively, we could have chosen the basis functions ψj (k) to defined over the whole state space (i.e., spectral

approximation methods). For example, ψj (k) could be monomial basis, or Chebyshev basis.4

Recall from our previous session [Function Approximation], Theorem: every spline function of a given degree, smoothness,and domain paritition, can be represented as a linear combination of B-splines of that same degree and smoothness, and overthat partition (de Boor 1978).


• For instance, an order-2 spline implements tent function interpolation (piecewiselinear), and are spanned by the B1-splines.

That is, the typical B1-spline is defined by (6) with the following basis functions

ψj(k) =

k−kj−1

kj−kj−1, if kj−1 ≤ k ≤ kj

kj+1−k

kj+1−kj, if kj < k ≤ kj+1

0, if kj−1 < k, or k > kj+1

y

• Note that ψj(k) is the tent function with peak at kj , and value zero below kj−1

and above kj+1.


• STEP 1. Divide the state space into a finite number of n disjoint intervals. Thepoints at which the spline pieces are connected are called knots, and there aren + 1 of these,

kmin = k0 < k1 < k2... < ki < ... < kn = kmax


• STEP 2. Guess a solution V s(k), that is, guess {θsj }mj=1, where

V s(k) =m∑j=0

θsj ψj(k)

and s keeps track of the number of iterations. The first iteration is s = 0.

Note that, given {θsj }mj=0, we can evaluate V s(ki ) at any value of capital in thestate space, k ∈ (kmin, kmax), including the capital knots i = {0, 1, ..., n}.


• STEP 3. For each capital knot i = {0, 1, ..., n}, find

g s(ki ) = arg maxk′∈[kmin,kmax ]

u(f (ki ) + (1− δ)ki − k ′) + βV s(k ′) (7)

and store {g s(ki )}ni=0.

To solve for k ′ (i.e., find g s(ki )) in (7) one can:

• implement a numerical optimization algorithm to (7),• or one can compute the first order condition of (7)—note that we know the

value function V s(k ′), and hence we can take numerical derivatives—andsolve the for k ′ in a univariate nonlinear equation.


• STEP 4. Given {g s(ki )}ni=0, update the value function at each i = 0, 1, ..., n,

V s+1(ki ) = u(f (ki ) + (1− δ)ki − g s(ki )) + βV s(g s(ki ))

and store {V s+1(ki )}ni=0.


• STEP 5. Given {V s+1(ki )}ni=0, solve for the m-basis coefficients {θs+1j }

mj=0 in

V s+1(ki ) =m∑j=0

θs+1j ψj(ki ) ∀ i = {0, ..., n} (8)

That is, (8) is a nonlinear system with n + 1 equations and m unknowns.

Two plausible options can arise:

• If m = n, solving for {θs+1j }

mj=0 in (8) implies that, in practice, we are

implementing a weighted residuals method under collocation, and thesolution to the nonlinear system (8) exits and it is hopefully unique.

• We can choose to have more knots than basis functions, that is, n > m,hence more equations, {ki}ni=0, than unknowns, {θs+1

j }mj=0. In this case we

can solve for {θs+1j }

mj=0 in (8) by regression, that is, implementing the

weighted residuals method under least squares.


• STEP 6: If ||θs+1 − θs || < ε, stop and report success. Otherwise, go back to theprevious step, STEP 3, replacing s = s + 1.


• STEP 7: Being thorough.

• Check the bounds of the state space are not binding. Otherwise, relax them.

• Make sure ε is small enough. Reduce ε, redo the VFI, and check your answerdoes not change. Otherwise, keep reducing ε.

• Make sure the number of knots for capital, n, is large enough. Increase n,redo the VFI, and check your answer does not change. Otherwise, keepincreasing n.

• Make sure the number of basis functions, m, is large enough. Increase m,redo the VFI, and check your answer does not change. Otherwise, keepincreasing m.


Another nice example that uses continuous methods to solve the neoclassicalgrowth model is in Section 5.6 of the Chapter by Fernandez-Villaverde,Rubio-Ramırez and Schorfheide in the Handbook of Macroeconomics 2016.

In that case the authors approximate for both the value function and (inelastic)labor supply. With inelastic labor supply the Bellman equation is not enough tosolve the problem, we need to use the intratemporal FOC to pin down the optimallabor function as well.


More tricks: EGM and Precomputation of Integrals

• (1) We can implement the endogenous grid method (EGM) to value functioniteration (Carroll 2006, Barillas and Fernandez-Villaverde, 2007, and Fella, 2014)

• (2) Precomputation of integrals (Judd, Maliar, Maliar and Tsener, 2016):Construct conditional expectations functions in the initial stage of a solutionprocedure.


Value Function Methods Quantitative Macroeconomics

Documents

Transcript of Value Function Methods Quantitative Macroeconomics