Post on 17-Mar-2020
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamic OptimizationAn Introduction
M. C. Sunny Wong
University of San Francisco
University of Houston, June 20, 2014
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
BackgroundWhat is Optimization?
In general, optimization is a technique which either maximizesor minimizes the value of an objective function bysystematically choosing values of inputs (or choice variables)from a feasible range.Optimization is one of the key ideas in the literature ofEconomics.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
BackgroundWhat is Optimization?
Adam Smith (1776): “It is not from the benevolence of thebutcher, the brewer, or the baker that we expect our dinner,but from their regard to their own self-interest. We addressourselves not to their humanity but to their self-love, andnever talk to them of our own necessities, but of theiradvantages.” (The Wealth of Nations, Book I, Chapter II)
Key assumptions: (1) Rationality, and (2) Efficiency.If rationality is assumed, firms will maximize their profits(supply) and households will maximize their utility (demand).When their results are achieved, the market is in equilibrium
(efficiency).
This idea is first discovered by Wilfredo Pareto (ParetoOptimality).
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
BackgroundTypes of Optimization
There are two general methods of optimization:Analytical optimization
Solving the optimal solution(s) mathematically.
Numerical (or computational) optimizationSearching for the optimal solution(s) according to differentalgorithms (using computers).For example, simulations, calibrations, and maximumlikelihood estimations.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
BackgroundAnalytical Optimization
There are three general types of analytical optimization:Optimization without Constraints
First-order conditions (FOCs)
Optimization with ConstraintsThe method of Lagrangian multiplier
Dynamic Optimization (with/without Constraints)Discrete time: The Bellman EquationContinuous time: The method of Hamiltonian multiplier
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
EITM: The Importance of OptimizationCausality, Assumptions and Models
Social scientists are interested in causal effects:
How do x ’s affect y?
Empirical studies (e.g., regression analysis) can show us, atmost, correlations among variables (not causality!) If thecoefficient on x is significant, it could imply that:
x causes y ; ory causes x ; orthere is another unobservable variable, called z , whichcontributes x and y to move simultaneously.
But, how do we know if x ’s really cause y?
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
EITM: The Importance of OptimizationCausality, Assumptions and Models
But, how do we know if x ’s really cause y?NOBODY TRULY KNOWS!!
We need to use our logical thinking and reasoning to describewhy x 0s can cause y .
But, the world is just too complex!An easy way to do so is to build a theoretical model whichdescribes some aspect of the market (or the society) thatincludes only those features that are needed for the propose athand.It is necessary to impose assumptions to make a model simpler.
But how do we impose “appropriate” assumptions in a model?
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
EITM: The Importance of OptimizationCausality, Assumptions, and Models
A Famous Quote from Robert Solow (1956, page 65):“All theory depends on assumptions which are not quite true.That is what makes it theory. The art of successful theorizingis to make the inevitable simplifying assumptions in such a waythat the final results are not very sensitive.”“A "crucial" assumption is one on which the conclusions dodepend sensitively, and it is important that crucial assumptionsbe reasonably realistic. When the results of a theory seem toflow specifically from a special crucial assumption, then if theassumption is dubious, the results are suspect.”
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
EITM: The Importance of OptimizationCausality, Assumptions, and Models
As NOBODY TRULY KNOWS how the world works, thetheoretical model we build could be “wrong”. In other words,the predicted results in the model can be inconsistent withwhat we observed in the real world.If this is the case, probably the assumptions we make are toosensitive (too strong) to the final results.Removing those assumptions / imposing some more realisticassumptions would be necessary.Therefore, both theoretical modeling (TM) and empiricaltesting (EI) enhance our understanding of the relationshipbetween x and y .
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
EITM: The Importance of OptimizationMicrofoundation of Macroeconomics
In the literature of economics, we assume that people (oreconomic agents) are rational.This assumption helps us formulate human behavior in orderto predict outcomes in aggregate markets. This is called themicrofoundation of macroeconomics.Microfoundations refers to the microeconomic analysis of thebehavior of individual agents such as households or firms thatunderpins a macroeconomic theory. (Barro, 1993)In this lecture, we study how agents face a dynamicoptimization problem where actions taken in one period canaffect the optimization decisions faced in future periods.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
What is Optimization?EITM: The Importance of Optimization
This Presentation
In this lecture, we study two methods of dynamic optimization:(1) Discrete-time Optimization - the Bellman equations; and(2) Continuous-time Optimization - the method of Hamiltonian multiplier.
Examples:Discrete-time case:
1 Cake-eating problem2 Profit maximization
Continuous-time case:1 Cake-eating problem2 Ramsey Growth Model (see lecture notes!)
EITM: Dynamics in a Money-in-the-Utility (MIU) Model(Chari, Kehoe, and McGrattan, Econometrica 2000; Christiano,Eichenbaum, and Evans, JPE 2005)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
Two periods in the modelPeriod 1: The present; and Period 2: The future
The two-period utility function can be written as:
U = u (c1
)+1
1+ru (c
2
) .
We call 1/(1+r) as the discount factor, where r is called thediscount rate (or the degree of impatience).If an agent is more impatient (r ") 1/(1+r) #), then shewould put less weight on the utility of future consumption.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
Two periods in the modelPeriod 1: The present; and Period 2: The future
The two-period utility function can be written as:
U = u (c1
)+1
1+ru (c
2
) .
Assuming that the agent has a first-period budget constraint:
Y1
+(1+ r)A0
= c1
+A1
,
where Yt
= exogenous income at time t, At
= assets / debtsthat the individual accumulates at time t, and r = anexogenous interest rate.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
The complete model is:
The two-period utility function:
U = u (c1
)+1
1+ru (c
2
) .
The first-period and second-period budget constraints:
Y1
= c1
+A1
(1st-period BC), andY
2
+(1+ r)A1
= c2
(2nd-period BC).
We assume that the individual does not have any inheritance/debtin period 1 (i.e., A
0
= 0) and does not leave any bequest/debt afterperiod 2 (i.e., A
2
= 0).EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
Lagrangian Multiplier
The system:
U = u (c1
)+1
1+ru (c
2
) .
andY
1
= c1
+A1
, and Y2
+(1+ r)A1
= c2
.
To maximize the system of equations, we can apply the method ofLagrangian multiplier to solve the model:
L= u (c1
)+1
1+ru (c
2
)+l1
(Y1
� c1
�A1
)+l2
(Y2
+(1+ r)A1
� c2
) ,
where l1
and l2
are the Lagrangian multipliers.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
We have 5 choice variables: c1
, c2
, A1
, l1
, and l2
. We can solvefor those variables based on the 5 first-order conditions:
∂L∂c
1
= 0 ) u0 (c1
)�l1
= 0 (1)
∂L∂c
2
= 0 ) 11+r
u0 (c2
)�l2
= 0 (2)
∂L∂A
1
= 0 )�l1
+(1+ r)l2
= 0 (3)
∂L∂l
1
= 0 ) Y1
= c1
+A1
(4)
∂L∂l
2
= 0 ) Y2
+(1+ r)A1
= c2
. (5)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
From equations (1)–(3), we have:
l1
= u0 (c1
) , (6)
l2
=1
1+ru0 (c
2
) , and (7)
�l1
+(1+ r)l2
= 0. (8)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
Now we can plug (6) and (7) into (8), we have the followingequation:
u0 (c1
) =1+ r
1+ru0 (c
2
) . (9)
Equation (9) is called the Euler equation. By combining equations(4) and (5), we have the lifetime budget constraint:
Y1
+Y
2
1+ r= c
1
+c2
1+ r. (10)
Finally, given a certain functional form of u (·), we can useequations (9) and (10) to obtain the optimal levels of c
1
and c2,
(i.e., c⇤1
and c⇤2
).EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model
What does the Euler equation: u0 (c1
) = 1+r
1+r u0 (c
2
) tell us?
Suppose that an agent gives up $1 consumption today (thepresent), the utility cost to her will be �u0 (c
1
) . In return, shewill get $(1+ r) additional consumption tomorrow (the future)so that her utility gain for the tomorrow will be (1+ r)u0 (c
2
) .
However, since we assume that agents are impatient, thetotally gain from giving up today’s consumption for tomorrowwould be 1
1+r ⇥ (1+ r)u0 (c2
).
In equilibrium, the agent will not give up more or less today’sconsumption for tomorrow at the optimal level only ifu0 (c
1
) = [(1+ r)/(1+r)]u0 (c2
) .
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption Model: An Example
Let the utility function be the power function, u (c) = 1
a ca , where
a 2 (0,1), and a = 1/2, we have:
u0 (c) = c�1/2. (11)
We can plug equation (11) into equation (9), we have:
c�1/21
=1+ r
1+rc�1/22
) c2
=
✓1+ r
1+r
◆2
c1
. (12)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption ModelAn Example
Plug eq.(12) into the lifetime budget constraint (eq.(10)), we have:
Y1
+Y
2
1+ r= c
1
+1+ r
(1+r)2c1
Y1
+Y
2
1+ r= c
1
1+
1+ r
(1+r)2
!
) c⇤1
=
1+
1+ r
(1+r)2
!�1✓Y
1
+Y
2
1+ r
◆. (13)
Now we plug equation (13) into equation (12), we have:
c⇤2
=
✓1+ r
1+r
◆ 1+
1+ r
(1+r)2
!�1✓Y
1
+Y
2
1+ r
◆. (14)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Two-period Consumption ModelAn Example
The optimized consumption levels in period 1 and period 2 are:
c⇤1
=⇣1+ 1+r
(1+r)2
⌘�1
⇣Y
1
+ Y21+r
⌘
c⇤2
=⇣
1+r
1+r
⌘⇣1+ 1+r
(1+r)2
⌘�1
⇣Y
1
+ Y21+r
⌘
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
The Bellman Equation
In the previous section, we can use the method of Lagrangianmultiplier for solving a simple dynamic optimization problem.However, such the method can be sometime tedious andinefficient.This alternative technique is based on a recursiverepresentation of a maximization problem, which is called theBellman equation.The Bellman equation represents a maximization decisionbased on the forward (or backward) solution procedure withthe property of time consistency.This time consistency property of the optimal solution is alsoknown as Bellman’s optimality principle.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Consumption Dynamics
Let’s consider the following maximization problem:
maxc
t
"U =
•
Ât=1
✓1
1+r
◆t�1
u (ct
)
#, (15)
subject to the following budget constraint:
At
= (1+ r)At�1
+Yt
� ct
. (16)
At
is the state variable in each period t, which represents thetotal amount of resources available to the consumer;ct
is the control variable, where the consumer is choosing tomaximize her utility. Note that c
t
affects the amount ofresources available for the next period, that is, A
t
.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Consumption Dynamics
For this technique of dynamic optimization, the maximumvalue of utility not only depends on the level of consumptionat time t, but also the resource left for future consumption(i.e., A
t
).In other word, given the existing level of asset A
t�1
, the levelof consumption chosen at time t (that is , c
t
) will affect thelevel of assets available at time t+1, (that is, A
t
).
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Consumption Dynamics
Since the consumer would like to maximize her utility from time tonwards, we define the following value function V
1
(A0
) whichrepresents the maximized value of the objective function from timet = 1 to the last period of t = T :
V1
(A0
) = maxc1
T
Ât=1
✓1
1+r
◆t�1
u (ct
) (17)
V1
(A0
) = maxc1
u (c
1
)+
✓1
1+r
◆u (c
2
)+ · · ·+✓
11+r
◆T�1
u (cT
)
!
V1
(A0
) = maxc1
(u (c
1
)+
✓1
1+r
◆"T
Ât=2
✓1
1+r
◆t�2
u (ct
)
#)(18)
subject toAt
= (1+ r)At�1
+Yt
� ct
. (19)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Consumption Dynamics
From equation (17), we see that V1
(A0
) is the maximized value ofthe objective function at time t = 1 given an initial stock of assetsA
0
.After the maximization in the first period (t = 1) , the consumerrepeats the same procedure of utility maximization according to theobjective function in period t = 2, given an initial stock of assets inperiod 1:
V2
(A1
) = maxc2
T
Ât=2
✓1
1+r
◆t�2
u (ct
) , (20)
subject to equation (19).By plugging equation (20) into equation (18), we have:
V1
(A0
) = maxc1
u (c
1
)+
✓1
1+r
◆V
2
(A1
)
�.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Consumption DynamicsThe Bellman Equation
Assuming the consumer is maximizing her utility every period, werewrite the maximization problem recursively. Therefore, we canpresent the well-known Bellman equation as follows:
Vt
(At�1
) = maxc
t
u (c
t
)+
✓1
1+r
◆Vt+1
(At
)
�,
where At
= (1+ r)At�1
+Yt
� ct
.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Cake Eating Problem
Question
Suppose the size of a cake at time t is ⇧t
; and the utility functionis presented as u (c
t
) = 2c1/2t
. Given that ⇧0
= 1, ⇧T
= 0, and thediscount rate is r , what is the optimal path of consumption?
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Cake Eating Problem
Answer
This optimization problem can be written as:
maxc1,c2,...,c
T
T
Ât=1
✓1
1+r
◆t�1
u (ct
) , (21)
subject to ⇧t
= ⇧t�1
� ct
, for t = 1,2, . . . ,T , and ⇧0
= 1 and⇧T
= 1.In this case, we see that the choice variable is c
t
and the statevariable ⇧
t
.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Cake Eating ProblemThe Bellman Equation
Now we can formulate the Bellman equation:
V (⇧t�1
) = maxc
t
u (c
t
)+1
1+rV (⇧
t
)
�, (22)
subject to⇧t
= ⇧t�1
� ct
. (23)
Since u (ct
) = 2c1/2t
, we can rewrite the Bellman equation as:
V (⇧t�1
) = maxc
t
2c1/2
t
+1
1+rV (⇧
t
)
�. (24)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Cake Eating ProblemThe Optimal Path of Cake Consumption
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Profit Maximization
Assuming that a representative firm maximizes the present value ofall future profits by choosing the levels of investment I
t
and laborLt
for t = 1,2, . . . ,T . Therefore, we have the followingmaximization problem:
maxI1,I2,...I
T
,L1,L2,...LT
ÂT
t=1
⇣1
1+r
⌘t
pt
maxI1,I2,...I
T
,L1,L2,...LT
ÂT
t=1
⇣1
1+r
⌘t
(F (Kt
,Lt
)�wt
Lt
� I ) ,
subject toKt+1
= Kt
�dKt
+ It
for t = 1,2, . . . ,T , and K1
and KT
are given, pt
is the level of profitat time t, K
t
is the stock of capital at time t, F (Kt
,Lt
) is aproduction function, w
t
is the wage rate, r and d is the interestrate and depreciate rate in the market, respectively.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Profit Maximization
In this case,the choice variable is: I
t
and Lt
, andthe state variable is K
t
.
According to the above system, we can formulate the followingBellman equation:
Vt
(Kt
) = maxI
t
,Lt
F (K
t
,Lt
)�wt
Lt
� It
+1
1+ rVt+1
(Kt+1
)
�, (25)
whereKt+1
= (1�d )Kt
+ It
. (26)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
A Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
Profit Maximization
In this optimization problem, we have two important conditions:1 ∂F (K
t
,Lt
)∂L
t
= wt
.
This result suggests that the optimal amount of labor satisfiesthe condition where marginal product of labor (MPL) equalsthe real wage rate in each period, that is MPL
t
= wt
.
2 ∂F∂K
t
= r +d .This result suggests that the firm must choose a level ofinvestment such that the marginal product of capital (MPK)equals the sum of interest rate and depreciation rate in eachperiod, that is MPK
t
= r +d .
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Dynamic Optimization in Continuous Time
In the previous sections, we study the dynamic optimizationbased the discrete-time dynamic model, where the change intime �t is positive and finite (for example, we assume that�t = 1 for all t � 0, such that t follows the sequence of{0,1,2,3, . . .} .In the section, we consider the method of dynamicoptimization in continuous time, where �t ! 0. Therefore, weassume that agents make optimizing choices at every instantin continuous time.The method is called the technique of Hamiltonianmultiplier.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Dynamic Optimization in Continuous Time
A general continuous-time maximization problem can be written asfollows:
maxx
t
ZT
0
e�rt f (xt
,At
)dt, (27)
subject to the constraint:
At
= g (xt
,At
) , (28)
where At
is a time derivative of At
defined as dAt
/dt, and r is thediscount rate in the model, which is equivalent to the r we use in thediscrete time models.
If r = 0, then e�rt = 1. It implies that the agent does notdiscount the activities in the future. On the other hand, if r ",then e�rt # . It implies that the agent becomes more impatientand discounts the value (or utility) of future activities more.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
The Method of Hamiltonian Multiplier
We set up the following Hamiltonian function:
Ht
= e�rthf (x
t
,At
)+lt
At
i, (29)
where lt
= µt
e�rt is called the Hamiltonian multiplier.The three conditions for a solution:
1 The FOC with respect to the control variable (xt
):∂H
t
∂xt
= 0; (30)
2 The negative derivative of the Hamiltonian function w.r.t. At
:
�∂Ht
∂At
=d (l
t
e�rt)
dt; (31)
3 The transversality condition:
limt!•
lt
e�rtAt
= 0. (32)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Cake Eating Problem
Let us consider the same cake eating problem in continuous time:
maxc
t
ZT
0
e�rtu (ct
)dt, (33)
subject to⇧t
=�ct
, (34)
and ⇧0
and ⇧1
are given. Again, the choice variable is ct
, and thestate variable is ⇧
t
.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Cake Eating Problem
We set up the Hamiltonian as follows:
Ht
= e�rt⇣u (c
t
)+lt
⇧t
⌘, (35)
where lt
is the co-state variable. Now, we can plug equation (34)into equation (35), we have the final Hamiltonian equation:
Ht
= e�rt (u (ct
)�lt
ct
) (36)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
The Method of Hamiltonian MultiplierCake Eating Problem Revisited
Cake Eating Problem
We obtain the following conditions:1 The FOC w.r.t. (with respect to) c
t
:
∂Ht
∂ct
= e�rt �u0 (ct
)�lt
�= 0
) u0 (ct
) = lt
; (37)
2 The negative derivative w.r.t. ⇧t
equals the time derivative oflt
e�rt :
�∂Ht
∂⇧t
=d (l
t
e�rt)
dt; (38)
and3 The transversality condition:
limt!•
lt
e�rt⇧t
= 0. (39)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model: An Introduction
Why do people want to hold money?Holding cash does not generate any returns!
Two possible arguments:1 People feel good and safe if they hold some cash in their
pocket (Sidrauski, 1967)2 Money can provide transaction services (transaction purpose!)
(Baumol, 1952, Tobin, 1956)
Here we study a basic neoclassical model where agents’ utilitydepends directly on their consumption of goods and theirholdings of money (money demand).
Assumption: Money yields direct utility.
This is called the Money-in-the-Utility model(Chari, Kehoe, and McGrattan, Econometrica 2000; Christiano,Eichenbaum, and Evans, JPE 2005)
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
Consider the following Money-in-the-Utility Model:
W =•
Ât=0
✓1
1+r
◆t
u (ct
,mt
) ,
where mt
=Mt
/(Pt
Nt
) = real money holding per capita in aneconomy. The budget constraint in the whole economy is:
GDPt
=Consumptiont
+ Investmentt
+GovtSpendingt
+
NewNationalDebtt
+NewMoneyt
.
We can translate as:
Yt
=Ct
+[Kt
� (1�d )Kt�1
]� tt
Nt
+
B
Pt
� (1+ it�1
)Bt�1
Pt
�+
M
t
Pt
�Mt�1
Pt
�.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
Yt
=Ct
+[Kt
� (1�d )Kt�1
]� tt
Nt
+
B
Pt
� (1+ it�1
)Bt�1
Pt
�+
M
t
Pt
�Mt�1
Pt
�.
where Yt
= aggregate output (GDP), tt
Nt
= total lump-sumtransfers (positive) or taxes (negative), i =interest rate, and N
t
=population.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
Finally, the complete system is:
W =•
Ât=0
✓1
1+r
◆t
u (ct
,mt
) ,
and the budget constraint can be presented as follows:
Yt
+tt
Nt
+(1�d )Kt�1
+(1+ it�1
)Bt�1
Pt
+M
t�1
Pt
=Ct
+Kt
+M
t
Pt
+Bt
Pt
.
We also define the production function as Yt
= F (Kt�1
,Nt
).The per capita income is:
yt
=1Nt
F (Kt�1
,Nt
) = F
✓Kt�1
Nt
,Nt
Nt
◆= F
✓Kt�1
(1+n)Nt�1
,1◆
= f
✓kt�1
1+n
◆, where n = population growth rate.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
Finally, the complete system is:
W =•
Ât=0
✓1
1+r
◆t
u (ct
,mt
) ,
and the budget constraint (per capita) is:
f
✓kt�1
1+n
◆+t
t
+1�d1+n
kt�1
+(1+ i
t�1
)bt�1
+mt
(1+pt
)(1+n)= c
t
+kt
+mt
+bt
,
where pt
is the inflation rate, such that, Pt
= (1+pt
)Pt�1
,bt
= Bt
/(Pt
Nt
) , and mt
=Mt
/(Pt
Nt
) .
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
We can now formulate the model as the Bellman equation:
V (Wt
) = maxu (c
t
,mt
)+1
1+rV (W
t+1
)
�,
subject to
Wt+1
= f
✓kt
1+n
◆+ t
t+1
+
✓1�d1+n
◆kt
+(1+ i
t
)bt
+mt
(1+pt+1
)(1+n)
= kt+1
+ ct+1
+mt+1
+bt+1
.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
An MIU Model
Finally we have the following equilibrium:um
(ct
,mt
)
uc
(ct
,mt
)=
i
1+ i.
Now assume that the utility function is of the constant elasticity ofsubstitution (CES) form:
u (ct
,mt
) =hac1�b
t
+(1�a)m1�b
t
i1/(1�b)
.
The final solution based on the CES utility function is:
mt
=
✓a
1�a
◆�1/b✓ i
1+ i
◆�1/b
ct
, or
lnmt
=1b
ln✓
1�a
a
◆+ lnc
t
� 1b
lng,
where g = i/(1+ i) = the opportunity cost of holding money.EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Outline1 Background
What is Optimization?EITM: The Importance of Optimization
2 Dynamic Optimization in Discrete TimeA Simple Two-period Consumption ModelThe Bellman EquationCake Eating ProblemProfit Maximization
3 Dynamic Optimization in Continuous TimeThe Method of Hamiltonian MultiplierCake Eating Problem Revisited
4 An EITM ExampleDynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Empirical Estimation of Money Demand
According to the theoretical solution:
lnmt
=1b
ln✓
1�a
a
◆+ lnc
t
� 1b
lng, (40)
where g = i/(1+ i) which can be called the opportunity cost ofholding money.Therefore, we empirical model can be written as:
lnmt
= a0
+a1
lnct
+a2
lng + et
. (41)
According to equation (41), we expect that he coefficient onlnc
t
is a1
= 1. It implies that consumption (income) elasticityof money demand is equal to 1.The coefficient on ln i
1+i
is a2
=�1/b. For simplicity, we callit as the interest elasticity of money demand.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Empirical Estimation of Money Demand
The empirical result of the money demand function for the UnitedStates based on quarterly data from the period of 1984:1 – 2007:2.
Source: Walsh (2010: 51)
lnm = the log of real money balance (M1); ct
= real peopleconsumption expenditures; and lng = ln i
1+i
= opportunity cost ofholding money = return on M1 minus the 3-month T-Bill rate.
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction
BackgroundDynamic Optimization in Discrete Time
Dynamic Optimization in Continuous TimeAn EITM Example
Dynamics in a Money-in-the-Utility ModelTM: Theoretical ModelEI: Empirical Implications
Thank you!
Questions?
EITM SUMMER INSTITUTE 2014 Dynamic Optimization: An Introduction