Misra GamesWorkshopSlides 2013
-
Upload
lucas-piroulus -
Category
Documents
-
view
218 -
download
0
Transcript of Misra GamesWorkshopSlides 2013
-
8/12/2019 Misra GamesWorkshopSlides 2013
1/69
Estimating Dynamic Games
Sanjog MisraAnderson UCLA
Structural Workshop 2013
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
2/69
Estimating Dynamic Discrete Games
In this workshop we have gone through a lot.
The main topics include...1 Foundations: Structural Models (Reiss), Causality and
Identication (Goldfarb), Instruments (Rossi), Data (Mela)2 Methods: Static Demand Models (Sudhir), Single Agent
Dynamics: Theory and Econometrics (Hitsch), Static Games
(Ellickson)
What you should have gleaned from their talks is that theestimation of structural models requires
Data+ Theory+ Econometrics
The estimation of dynamic games combines elements from allof the above talks and requires considerable expertise inhandling data, game theory, econometrics and computational
methods.Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
3/69
Estimating Dynamic Discrete Games
Weve already learned how to estimate single-agent (SA)
dynamic discrete choice (DDC) modelsTwo main approaches
1 Full solution: NXFP (Rust, 1987)
2 Two-Step: CCP (Hotz and Miller, 1993)
In both cases, the underlying SA optimization probleminvolved agents solving a dynamic programming (DP)problem
With games, agents must solve an inter-related system of DPproblems
Their actions must be optimal given their beliefs & theirbeliefs must be correct on average (or at least self-conrming)
As you can imagine, computing equilibria can be prettycomplicated...
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
4/69
Estimating Dynamic Discrete Games
Ericson & Pakes (ReStud, 95) and Pakes & McGuire (Rand,94) provide a framework for computing equilibria to dynamicgames.
More recently Goettler and Gordon (JPE 2012) provide analternative approach
However, using the original PM algorithm, solving a
reasonably complex EP-style dynamic game even once iscomputationally demanding (if not impossible)
Estimating the model using NFXP is essentially intractable
Theres also the issue of multiple equilibria
The incompleteness this introduces can make it dicult toconstruct a proper likelihood/objective function, furthercomplicating a NFXP approach
Two-step CCP estimation provides a work-around thatcircumvents the iterative xed point calculation, solving
both problemsMisra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
5/69
Estimating Dynamic Discrete Games
CCP estimators were rst developed by Hotz & Miller (HM,1993) & Hotz, Miller, Sanders & Smith (HMS2, 1994) forDDC models
Four sets of authors (contemporaneously) suggested adaptingthese methods to games:
1 Aguirregabiria and Mira (AM) (Ema, 2007),
2 Bajari, Benkard, and Levin (BBL) (Ema, 2007),3 Pakes, Ostrovsky, and Berry (POB) (Rand, 2007), &
4 Pesendorfer and Schmidt-Dengler (PSD) (ReStud, 2008),
Well talk about AM & BBL
AM extends HM to gamesBBL extends HMS2 to games
Both are based on the framework suggested in Rust (94)
More recent contributions are Blevins et al. (2012) andArcidiacono and Miller (Ema, 2011)
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
6/69
Aguirregabiria and Mira (Ema, 2007)Sequential Estimation of Dynamic Discrete Games
Model
A dynamic discrete game of incomplete information.
Motivated by stylized model of retail chain competition.
Let dtbe a vector of demand shifters in period t.Nrms operate in the market, indexed by i2 f1, 2, ..., Ng.In each period t, rms decide simultaneously how manyoutlets to operate - choosing from the discrete setA=
f0, 1, ..., J
gThe decision of rm i in period t is ait2 AThe vector of all rms actions is at (a1t, a2t, ..., aNt)
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
7/69
Model
Firms are characterized by two vectors of state variables thataect protability: xit and it
xt (dt, x1t, x2t, ..., xNt) is common knowledge, butt (1t, 2t, ..., Nt) is privately observed by rm iei(at, xt, it) is rm is per-period prot function.
Assumefxt, tg follows a controlled Markov process withtransition probability p(xt+1, t+1jat, xt, t), which iscommon knowledge
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
8/69
Model
Each rm chooses its number of outlets to maximize expecteddiscounted intertemporal prots,
E
s=tstei(at, xt, it)j xt, it
where 2(0, 1) is the (known) discount factor.The primitives of the model are the prot functions
ei(),
the transition probability p(j
), and
AM make the following set of assumptions...
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
9/69
Assumptions
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
10/69
Strategies and Bellman Equations
AM also assume that rms play stationary Markov strategiesLet =fi(x, i)g be a set of strategy functions (decisionrules), one for each rm, with
i : X
RJ+1
!A
Associated with a set of strategy functions , dene a set ofconditional choice probabilities P =fPi (aijx)g such that
Pi (aijx) Pr(i(x, i)= aijx)= Z Ifi(x, i)= aig gi(i) di
which represent the expected behavior of rm ifrom the pointof view of the rest of the rms (& us!), when rm i follows .
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
11/69
Expected prots
Let i (ai, x) be rm is current expected prot fromchoosing alternative aiwhile the other rms follow .
By the independence of private information,
i (ai, x)= ai2AN1
j6=i
Pj (ai[ j] jx)!i(ai, ai, x)where ai[ j] is the jth rms element in the vector of actionsplayers other than i
LeteVi (x, i) be the value of rm iif it behaves optimallynow and in the future given that the other rms follow .
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
12/69
Value Functions
By Bellmans principle of optimality, we can write
eVi (x, i)= maxai2A
(i (ai, x)+i(ai)+
x02X
ZeVi x0, 0i gi 0i d0i fi x0 jx, ai)
(1)
where fi (x0jx, ai) is the transition probability ofx conditional
on rm i
choosinga
iand the other rms following :
fi
x0 jx, ai
= ai2AN1
j6=iPj (ai[j] jx)
!f
x0 jx, ai, ai
AM prefer to work with the value functions integrated over
the private information variables.
Let Vi (x) be the integrated value functionR Vi (x, i) gi(di)
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
13/69
Choice-Specic Value Functions
Based on this denition
Vi (x)= Z Vi (x, i) gi(di)and the Bellman equation from above
eVi (x, i)= max
ai2A (i (ai, x)+i(ai)+
x02X ZeVi x
0, 0i gi
0i d
0i f
i x
0
jx, ai)
(1)
we can obtain the integrated Bellman equation
Vi (x)=Z
maxai2A
fvi (ai, x)+i(ai)g gi(di)
wherevi (ai, x)i (ai, x)+
x02XVi
x0
fi
x0jx, ai
which are often referred to as choice-specic value functions.
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
14/69
Markov Perfect Equilibria
Now its time to enforce the fact that describes equilibriumbehavior (i.e. its a best response)
DEFINITION: A stationary Markov perfect equilibrium (MPE)in this game is a set of strategy functions such that for anyrm iand any (x, i)2 X RJ+1
i (x, i)=arg maxai2A
nv
i (ai, x)+i(ai)
oUltimately, an equation (somewhat) like this will be the basisof estimation
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
15/69
Markov Perfect Equilibria
Note that i , V
i , & fi only depend onthrough P, so AM
switch notation to P
i , VP
i , & fP
i so they can represent theMPE in probability space.
Let be a set of MPE strategies and let P be theprobabilities associated with these strategies
P(aijx)= Z Ifai =i (x, i)g gi(i) diEquilibrium probabilities are a xed point (P = (P)),where, for any vector of probabilitiesP, (P)=
fi(ai
jx; P
i)
gand
i(aijx; Pi)=Z
I
ai =arg max
ai2A
nvP
i (a, x)+i(a)
ogi(i) di
The functions i arebest response probability functions
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
16/69
Markov Perfect Equilibria
The equilibrium probabilities solve the coupled xed-point
problems dened by
Vi (x)=Z
maxai2A
fvi (ai, x)+i(ai)g gi(di) (2)
and
i(aijx; Pi)=Z
I
ai =arg max
ai2A
nvP
i (a, x)+i(a)
ogi(i) di
(3)
Given a set of probabilities P:
The value functions VPi are solutions of the NBellmanequations in (2), and
Given these value functions, the best response probabilities aredened by the RHS of (3).
Misra Estimating Dynamic Games
A Al i B R M i
-
8/12/2019 Misra GamesWorkshopSlides 2013
17/69
An Alternative Best Response Mapping
Its easier to work with an alternative best response mapping(in probability space) that avoids the solution of the Ndynamic programming problems in (2).
The evaluation of this mapping is computationally muchsimpler than the evaluation of the mapping (P), and it willprove more convenient for the estimation of the model.
Let P be an equilibrium and let VP1 , VP
2 , ..., VP
N be rmsvalue functions associated with this equilibrium.
Misra Estimating Dynamic Games
Al i B R M i
-
8/12/2019 Misra GamesWorkshopSlides 2013
18/69
Alternative Best Response Mapping
Because equilibrium probabilities are best responses, we canrewrite the Bellman equation
Vi (x)=Z
maxai2A
fvi (ai, x)+i(ai)g gi(di) (2)
as
VP
i (x)= ai2A
Pi (aijx)hP
i (ai, x)+ e
Pi (ai, x)
i+
x02XVP
i
x0
fP
x0 jx(4)
where fP
(x0jx) is the transition probability ofx induced byP
eP
i (ai, x) is the expectation ofi(ai) conditional on x and
on alternative aibeing optimal for player i
Misra Estimating Dynamic Games
Al i B R M i
-
8/12/2019 Misra GamesWorkshopSlides 2013
19/69
Alternative Best Response Mapping
Note that eP
i (ai, x) is a function ofgi and Pi (x) only
The functional form depends on the probability distribution giFor example, if the i(ai) are iidT1EV, then
eP
i (ai, x) E(i(a) jx, i (x, i)=ai)=ln (Pi(aijx))where is Eulers constant & is the logit dispersionparameter.Taking equilibrium probabilities as given, expression (4)
VP
i (x)= ai2A
Pi (aijx)hP
i (ai, x)+ e
Pi (ai, x)
i+
x02XVP
i
x0
fP
x0 jx
describes the vector of values VP
i as the solution of a systemof linear equations, which can be written in vector form as
IFP
VP
i = ai2A
Pi (ai) hP
i (ai)+ e
Pi (ai)
i
Misra Estimating Dynamic Games
Alt ti B t R M i
-
8/12/2019 Misra GamesWorkshopSlides 2013
20/69
Alternative Best Response Mapping
Let i(P)fi(x; P) :x2 Xg be the solution to thissystem of linear equations, such that VP
i
(x)= i(x; P
).
For arbitrary probabilities P, the mapping
i(P)=
IFP1 (
ai2A
Pi (ai) hP
i (ai)+ e
Pi (ai)
i) (5)
can be interpreted as a valuation operator.An MPE is then a xed point (P)fi(aijx; P)g where
i(aijx; P)=Z
I
ai =arg max
ai2A
(Pi (a, x)+i(a)+
x02Xi
x0; P
fPi
x0 jx, a)! gi(i) di
(6)
By denition, an equilibrium vector P is a xed point of.AMs Representation Lemma establishes that the reverse isalso true.
Equation (6) will be the basis of estimation
Misra Estimating Dynamic Games
E ti ti
-
8/12/2019 Misra GamesWorkshopSlides 2013
21/69
Estimation
Assume the researcher observes M geographically separate
markets over T periods, where M is large and T is small.The primitivesfi, gi, f, :i2 Ig are known to theresearcher up to a nite vector of structural parameters2 Rjj.
is assumed known (its verydicult to estimate)
We now incorporate as an explicit argument in theequilibrium mapping .
Let 02 be the true value of in the populationUnder Assumption 2 (i.e., conditional independence), thetransition probability function fcan be estimated fromtransition data using a standard maximum likelihood methodand without solving the model.
Misra Estimating Dynamic Games
Assumptions on DGP
-
8/12/2019 Misra GamesWorkshopSlides 2013
22/69
Assumptions on DGP
Misra Estimating Dynamic Games
Maximum Likelihood Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
23/69
Maximum Likelihood Estimation
Dene the pseudo likelihood function
QM(, P)= 1
M
M
m=1
T
t=1
N
i=1
lni(aimtjxmt; P, )
where Pis an arbitrary vector of players choice probabilities.Consider rst the hypothetical case of a model with a uniqueequilibrium for each possible value of2 .Then the maximum likelihood estimator (MLE) of0 can bedened from the constrained multinomial likelihood
MLE =arg max2
QM(, P) subject to P= (, P)
Misra Estimating Dynamic Games
MLE
-
8/12/2019 Misra GamesWorkshopSlides 2013
24/69
MLE
However, with multiple equilibria the restriction P= (, P)
does not dene a unique vector P, but a set of vectors.In this case, the MLE can be dened as
MLE =arg max
2
8
-
8/12/2019 Misra GamesWorkshopSlides 2013
25/69
Pseudo Maximum Likelihood (PML) Estimation
The PML estimators try to minimize the number of
evaluations of for dierent vectors of players probabilitiesP.
Suppose that we know the population probabilities P0 andconsider the (infeasible) PML estimator
=arg max2 QM , P0
Under standard regularity conditions, this estimator ispM-CAN, but infeasible since P0 is unknown.
However, if we can obtain a pM-consistent nonparametricestimator ofP0, then we can dene the feasible two-step PMLestimator
2S =arg max2
QM
, P0
Misra Estimating Dynamic Games
PML
-
8/12/2019 Misra GamesWorkshopSlides 2013
26/69
PML
pM-CAN ofP0 (+ regularity conditions) are sucient to
guarantee that the PML estimator isp
M-CAN.
There are two good reasons to care about this estimator
1 It deals with the indeterminacy problem associated withmultiple equilibria (its robust)
2 Furthermore, repeated solutions of the dynamic game areavoided, which can result in signicant computational gains(its simple)
Misra Estimating Dynamic Games
PML: Drawbacks
-
8/12/2019 Misra GamesWorkshopSlides 2013
27/69
PML: Drawbacks
However, the two-step PML has some drawbacks
1 Its asymptotic variance depends on the variance of thenonparametric estimator P0.
Therefore, it can be very inecient when is large.
2 For the sample sizes available in actual applications, the
nonparametric estimator ofP0 can be very imprecise (smallsample bias).
Theres a curse of dimensionality here...
3 For some models, its impossible to obtain consistentnonparametric estimates ofP0.
e.g. models with unobserved market characteristics.
To address these issues, AM introduce the Nested PseudoLikelihood estimator.
Misra Estimating Dynamic Games
Nested Pseudo Likelihood (NPL) Method
-
8/12/2019 Misra GamesWorkshopSlides 2013
28/69
Nested Pseudo Likelihood (NPL) Method
NPL is a recursive extension of the two-step PML estimator.Let P0 be a (possibly inconsistent) initial guess of the vectorof players choice probabilities.
Given P0, the NPL algorithm generates a sequence of
estimators K :K 1, where the K-stage estimator isdened asK =arg max
2QM
, PK1
and the probabilities PK :K 1 are obtained recursively as
PK = K, PK1
Misra Estimating Dynamic Games
NPL
-
8/12/2019 Misra GamesWorkshopSlides 2013
29/69
NPL
If the initial guess P0 is a consistent estimator, all elements ofthe sequence of estimators K :K 1 are consistentHowever, AM are interested in the properties of the estimatorin the limit (if it converges)
If the sequence K, PK converges, its limit , P is suchthat maximizes QM
, P
and P=
, P
and any pair that does so is a NPL xed point.
AM show that a NPL xed point always exists and, if there is
more than one, the one with the highest value of the pseudolikelihood is a consistent estimator
Of course, it may be verydicult to nd multiple roots
Misra Estimating Dynamic Games
NPL
-
8/12/2019 Misra GamesWorkshopSlides 2013
30/69
NPL
NPL preserves the two main advantages of PML
1 Its feasible in models with multiple equilibria, and2 It minimizes the number of evaluations of the mapping for
dierent values ofP
Furthermore
1 Its more ecient than either infeasible or two-step PML(because it imposes the MPE condition in sample)
2 It reduces the nite sample bias generated by impreciseestimates ofP0
3 It doesnt require initially consistent P0s (so it can
accommodate unobserved heterogeneity)
AM show how to extend NPL to settings with permanentunobserved heterogeneity, but lets skip that & look at someMonte Carlos
Misra Estimating Dynamic Games
Monte Carlos
-
8/12/2019 Misra GamesWorkshopSlides 2013
31/69
Monte Carlos
Consider a simple entry/exit example where 5 rms can
operate at most 1 store, so ait2f0,
1gVariable prot is given byRSln (Smt) RNln
1+j6=iajmt
where Smtis the size of market m in period t, and RS & RNare parameters to be estimated
The prot function of an active rm is
imt(1)= RSln (Smt) RNln
1+j6=iajmt
FC,i EC(1 aim,t1)+
where FC,i is xed cost, EC is entry cost, and imt T1EVThe prot function of an inactive rm is simply
imt(0)= imt
Misra Estimating Dynamic Games
Monte Carlos
-
8/12/2019 Misra GamesWorkshopSlides 2013
32/69
ln (Smt) follows a discrete rst order Markov process, with
known transition matrix and nite supportf1,
2,
3,
4,
5gFixed operating costs areFC,1 =1.9, FC,2 =1.8, FC,3 =1.7, FC,4 =1.6, FC,5 =1.5
so that rm 5 is most ecient and rm 1 is least ecient.RS =1 and = .95 throughout, but AM will vary RN andEC
Misra Estimating Dynamic Games
Monte Carlos
-
8/12/2019 Misra GamesWorkshopSlides 2013
33/69
The space of common knowledge state variables (Smt, at
1)has 25 5=160 cells.Theres a dierent vector of CCPs for each rm, so thedimension of the CCP vector for all rms is 5 160=800.For each experiment, they compute a MPE.
The equilibrium is obtained by iterating the best responseprobability mapping starting with a 800 1 vector of choiceprobabilities (guesses) - (e.g. Pi(ai=1jx)= .58x, i)They can then calculate the steady state distribution and
generate fake data.Table II presents some descriptive statistics associated withthe MPE of each experiment.
Misra Estimating Dynamic Games
Markov Perfect Equilibria for each experiment
-
8/12/2019 Misra GamesWorkshopSlides 2013
34/69
q p
Misra Estimating Dynamic Games
Monte Carlos
-
8/12/2019 Misra GamesWorkshopSlides 2013
35/69
They then calculate the two-step PML and NPL estimatorsusing the following choices for the initial vector of probabilities
1 The true vector of equilibrium probabilitiesP0
2 Nonparametric frequency estimates
3 Logits (for each rm) with the log of market size andindicators of incumbency status for all rms as explanatoryvariables, and
4 Independent random draws from a U(0, 1) r.v.
Tables IV and V summarize the results
Misra Estimating Dynamic Games
Monte Carlo Results
-
8/12/2019 Misra GamesWorkshopSlides 2013
36/69
Misra Estimating Dynamic Games
Monte Carlo Results
-
8/12/2019 Misra GamesWorkshopSlides 2013
37/69
Misra Estimating Dynamic Games
Discussion of Results
-
8/12/2019 Misra GamesWorkshopSlides 2013
38/69
The NPL algorithm always converged to the same estimates
(regardless of the value ofP0)
The algorithm converged faster when initialized with logitestimates
The 2S-Freq estimator is highly biased in all experiments,
although its variance is sometimes smaller than the NPL and2S-True estimators.
Its main drawback is small sample bias...
The NPL estimator performs very well relative to the 2S-True
both in terms of variance and bias.In all the experiments, the most important gains associatedwith the NPL estimator occur for the entry cost parameter.
Misra Estimating Dynamic Games
Motivation for BBL
-
8/12/2019 Misra GamesWorkshopSlides 2013
39/69
A big drawback of the AM approach (and PSD & POB as
well) is that its designed for discrete controls (and discretestates)
A big selling point of BBL is that it can
However,
1 Its not completely clear that AM cant be extended tocontinuous controls and states (see, e.g., Arcidiacono andMiller (2011))
2 The way BBL handles continuous controls is probablyinfeasible (unless there are no structural shocks to investment)
3 BBLs objective function can be dicult to optimize, so youmight want to mix & match a bit.
Lets look now at how BBL works...
Misra Estimating Dynamic Games
Background for BBL (HMSS)
-
8/12/2019 Misra GamesWorkshopSlides 2013
40/69
The original idea for the methods proposed in BBL came fromHotz, Miller, Sanders and Smith (1994)
They proposed a two-step (really 3) approach
1 Estimate the transition kernelsf(s0js,
a) and CCPs P(ajs)from the data2 Approximate V
sjP using Monte-Carlo approaches
3 Construct and maximize an objective function to obtainparameters
Lets look how this might work...
Misra Estimating Dynamic Games
Forward Simulation (HMSS)
-
8/12/2019 Misra GamesWorkshopSlides 2013
41/69
For each state (s2 S) draw a sequence ofRfuture paths asfollows
1 Draw iid shocks 0(a)
2 Compute optimal policies
P(ajs0) and pick action a0(0, s0)3 Compute payos Ur0 =u(s0, a0; )+0(a0)4 Draw next period states1~f(s
0js0, a0) and repeat for Titerations.
We can use these to construct Value functions...
Misra Estimating Dynamic Games
Value Function (HMSS)
-
8/12/2019 Misra GamesWorkshopSlides 2013
42/69
At some Twe stop the forward simulation and use the fact
thatVr (s0)=
T
t=0
tUrt
To construct
V s0jP= 1RR
r=1Vr
(s
0)We can then obtain an estimator usingA Pseudo Likelihood, GMM or Least Squares
max
t
i
aitln aitjsit,V
min
i
t
ait
aitjsit, V
Zit
1
t
ait
aitjsit, V
min P(ajs) ajs,
V0W1P(ajs) ajs,
VMisra Estimating Dynamic Games
Bajari, Benkard, and Levin (Ema, 2007)E i i D i M d l f I f C i i
-
8/12/2019 Misra GamesWorkshopSlides 2013
43/69
Estimating Dynamic Models of Imperfect Competition
So what does BBL do?
Extends HMSS to gamesGeneralizes the idea to continuous actions
Proposes an inequality conditions for estimation (Boundsestimator)
The key element of BBL is that it (like AM 2007) allows theresearch to be agnostic about equilibrium selection
and side-step the multiple equilibria problem.
Misra Estimating Dynamic Games
Bajari, Benkard, and Levin (Ema, 2007)E i i D i M d l f I f C i i
-
8/12/2019 Misra GamesWorkshopSlides 2013
44/69
Estimating Dynamic Models of Imperfect Competition
Notation
The game is in discrete time with an innite horizon
There are Nrms, denoted i=1, ..., Nmaking decisions at
times t=1,
2, ...,
Conditions at time tare summarized by discrete statesst2 S RLGiven st, rms choose actions simultaneously
Leta
it2A
i denote rm i
s action at timet
, andat= (a1t, ..., aNt) the vector of time t actions
Misra Estimating Dynamic Games
-
8/12/2019 Misra GamesWorkshopSlides 2013
45/69
State Transitions & Equilibrium
-
8/12/2019 Misra GamesWorkshopSlides 2013
46/69
st+1 is drawn from a probability distribution P(st+1jat, st)They focus on pure strategy MPE
A Markov strategy is a function i :S Vi! AiA prole of Markov strategies is a vector, =(1, ..., N),where :S V1 ... VN! AGiven , rm is expected prot can then be writtenrecursively
Vi(s; )=E
i((s, ), s, i) +
Z V
s0;
dP
s0j(s, ), s j
The prole is a MPE if, given opponent prole i, eachrm i prefers strategy i to all other alternatives
0i
Vi(s; ) Vi
s; 0i, i
Misra Estimating Dynamic Games
Structural Parameters
-
8/12/2019 Misra GamesWorkshopSlides 2013
47/69
The structural parameters of the model are the discount factor, the prot functions 1, ..., N, the transition probabilitiesP, and the distributions of private shocks G1, ..., GN.
Like AM, they treat as known and estimate Pdirectly from
the observed state transitions.They assume the prots and shock distributions are knownfunctions of a parameter vector :i(a, s, i; ) andGi(ijs, ).The goal is to recover the true under the assumption thatthe data are generated by a MPE.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
48/69
Their main (novel) example is based on the EP framework.
Incumbent rms are heterogeneous, each described by itsstate zit2f1, 2, ..., zg ; potential entrants have zit=0Incumbents can make an investment Iit 0 to improve theirstate
An incumbent rm i in period t earns
qit(pit mc(qit, st; )) C(Iit, it; )
where pit is rm is price, qit=qi(st,pt; ) is quantity,
mc() is marginal cost, and it is a shock to the cost ofinvestment.
C(Iit, it; ) is the cost of investment.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
49/69
Competition is assumed to be static Nash in prices.Firms can also enter and exit.
Exitors receive and entrants pay et, an iiddraw from Ge
In equilibrium, incumbents make investment and exit decisions
to maximize expected prots.Each incumbent iuses an investment strategy Ii(s, i) andexit strategy i(s, i) chosen to maximize expected prots.
Entrants follow a strategy e(s, e) that calls for them to
enter if the expected prot from doing so exceeds its entrycost.
Misra Estimating Dynamic Games
First Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
50/69
The goal of the rst stage is to estimate the state transitionprobabilities P(s0ja, s) and equilibrium policy functions (s, )The second stage will use the equilibrium conditions fromabove to estimate the structural parameters
In order to obtain consistent rst stage estimates, they mustassume that the data are generated by a single MPE prole
This assumption has a lot of bite if the data come frommultiple markets
Its quite weak if the data come from only a single market
Misra Estimating Dynamic Games
First Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
51/69
Stage 0:
The static payo function will typically be estimated o linein a 0th stage (e.g. BLP, Olley-Pakes)
Stage 1:
Its usually fairly straightforward to run the rst stage: just
regress actions on states in a exible manner.Since these are not structural objects, you should be as exibleas possible. Why?Of course, ifs is big, you may have to be very parametric here(i.e. OLS regressions and probits).
In this case, your second stage estimates will be inconsistent...
Continuous actions are especially tricky (hard to benonparametric here)
Misra Estimating Dynamic Games
Estimating the Value Functions
-
8/12/2019 Misra GamesWorkshopSlides 2013
52/69
After estimating policy functions, rms value functions areestimated by forward simulation.
Let Vi(s, ; ) denote the value function of rm iat state sassuming rm ifollows the Markov strategy iand rival rmsfollow iThen
Vi(s, ; )=E" 0=t
ti((st, t), st, it; )js0 = s; #
where the expectation is over current and future values ofst
and t
Given a rst-stage estimatebPof the transition probabilities,we can simulate the value function Vi(s, ; ) for any strategyprole and parameter vector .
Misra Estimating Dynamic Games
Estimating the Value Functions
-
8/12/2019 Misra GamesWorkshopSlides 2013
53/69
A single simulated path of play can be obtained as follows:
1 Starting at states0 = s, draw private shocks i0 fromGi(js0, ) for each rm i.
2 Calculate the specied action ai0 =i(s0, i0) for each rm i,and the resulting prots i(a0, s0, i0; )
3 Draw a new state s1 using the estimated transition
probabilitiesbP(ja0, s0)4 Repeat steps 1-3 for Tperiods or until each rm reaches a
terminal state with known payo (e.g. exits from the market)
Averaging rm is discounted sum of prots over many paths
yields an estimatebVi(s, ; ), which can be obtained for any(, ) pair, including both the true prole (which youestimated in the rst stage) and any alternative you care toconstruct.
Misra Estimating Dynamic Games
Special Case of Linearity
-
8/12/2019 Misra GamesWorkshopSlides 2013
54/69
Forward simulation yields a low cost estimate of the Vs fordierent s given , but the procedure must be repeated for
each candidate .One case is simpler.
If the prot function is linear in the parameters so that
i(a, s,
i;)=i(
a, s,i)
we can then write the value function as
Vi(s, ; )= E"
t=0
ti((st, t), st, it)js0 =s# =Wi(s; ) In this case, for any strategy prole , the forward simulationprocedure only needs to be used once to construct each Wi.
You can then obtain Vieasily for any value of.
Misra Estimating Dynamic Games
Second Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
55/69
The rst stage yields estimates of the policy functions, statetransitions, and value functions.
The second stage uses the models equilibrium conditions
Vi(s; i,
i; )
Vi s; 0i, i;
to recover the parameters that rationalize the strategyprole observed in the data.
They show how to do so for both set and point identied
modelsWe will focus on the point identied case here.
Misra Estimating Dynamic Games
Second Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
56/69
To see how the second stage works, dene
g(x; , )=Vi(s; i, i; , ) Vis; 0i, i; ,
where x2 X indexes the equilibrium conditions and represents the rst-stage parameter vector.
The inequality dened by xis satised at , ifg(x; , ) 0Dene the function
Q(, )
Z (min fg(x; , ), 0g)2
dH(x)
where H is a distribution over the setXof inequalities.
Misra Estimating Dynamic Games
Second Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
57/69
The true parameter vector 0 satises
Q(0, 0)=0=min2
Q(, 0)
so we can estimate by minimizing the sample analog ofQ(, 0)
The most straightforward way to do this is to draw rms andstates at random and consider alternative policies 0i that areslight perturbations of the estimated policies.
Misra Estimating Dynamic Games
Second Stage Estimation
-
8/12/2019 Misra GamesWorkshopSlides 2013
58/69
We can then use the above forward simulation procedure toconstruct analogues of each of the Vi terms and construct
Q(, ) 1nI
nI
k=1
(min fbg(Xk; , ), 0g)2How? By drawing nIdierent alternative policies, computingtheir values, nding the dierence versus the optimal policypayo, and using an MD procedure to estimate theparameters that minimize these protable deviations.
Their estimator minimizes the objective function at =bn=arg min2
Qn(,bn )See the paper for the technical details.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
59/69
Lets see how they estimate the EP model.
First, they have to choose some parameterizations.They assume a logit demand system for the product market.
There are Mconsumers with consumer r deriving utility Urifrom good i
Uri =0ln(zi) +1ln (yr pi)+riwhere zi is the quality of rm i, pi is rm is price, yr isincome, and ri is an iid logit error
All rms have identical constant marginal costs of production
mc(qi; ) =
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
60/69
Each period, rms choose investment levels Iit2 R+ toincrease their quality in the next period.
Firm is investment is successful with probability
Iit(1+Iit)
in which case quality increases by one, otherwise it doesntchange.
There is also an outside good, whose quality moves up by onewith probability each period.
Firm is cost of investment is
C(Ii) = Iiso there is no shock to investment (its deterministic)
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
61/69
The scrap value is constant and equal for all rms.
Each period, the potential entrant draws a private entry costet from a uniform distribution on
L, H
The state variable st=(Nt, z1t, ..., zNt, zout,t) includes thenumber of incumbent rms and current product qualities.
The model parameters are 0,1
,
,
,
,
L ,
H,
,
,
,&
y
They assume that & yare known, & are transitionparameters estimated in a rst stage, 0, 1, & are demandparameters (also estimated in a rst stage), so the main(dynamic) parameters are simply = , ,
L, HDue to the computational burden of the PM algorithm, theyconsider a setting in which only 3 rms can be active.They generated datasets of length 100-400 periods using PM.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
62/69
Here are the parameters they use.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
63/69
The rst stage requires estimation of the state transitions andpolicy functions (as well as the demand and mcparameters).
For the state transitions, they used the observed investmentlevels and qualities to estimate and by MLE.
They estimated the demand parameters by MLE as well, usingquantity, price, and quality data.
They recover from the static mark-up formula.
They used local linear regressions with a normal kernel to
estimate the investment, entry, and exit policies.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
64/69
Given strategy prole =(I,,e), the incumbent valuefunction is
Vi(s; ) = W1 (s; )+ W2 (s; ) + W3 (s; )
= E
"
t=0
t
ei(st)js=s0
# E
"
t=0
tIi(st)js=s0#
+E" t=0
ti(st)js=s0#
where the rst term
ei(st) is the static prot of incumbent i
given state stThe 2nd term is the expected PV of investment
The 3rd term is the expected PV of the scrap value earnedupon exit.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
65/69
To apply the MD estimator, they constructed alternativeinvestment and exit policies by drawing a mean zero normalerror and adding it to the estimated rst stage investment andexit policies.
They used ns=2000 simulation paths, each having length at
most 80, to compute the PV W1, W2, W3 terms for thesealternative policies.
They can then estimate & using their MD procedure.
Its also straightforward to estimate the entry cost distribution
(parametrically or non-parametrically) - see the paper fordetails.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
66/69
Truth: =1 & =6For small sample sizes, there is a slight bias in the estimatesof the exit value.
Investment cost parameters are spot on.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
67/69
Truth: =1, =6, l
=7, & h
=11The subsampled standard errors are on average slightlysmaller than the true SEs.
This is likely due to small sample sizes.
Misra Estimating Dynamic Games
Example: Dynamic Oligopoly
-
8/12/2019 Misra GamesWorkshopSlides 2013
68/69
The entry cost distribution is recovered quite well, despitesmall sample size (and few entry events).
Misra Estimating Dynamic Games
Conclusions
-
8/12/2019 Misra GamesWorkshopSlides 2013
69/69
Both AM & BBL are based on the same underlying idea (CCP
estimation)As such, its quite possible to mix and match from the twoapproaches
e.g. forward simulate the CV terms and use a MNL likelihood
We have found AM-style approaches easier to implement, butthat might be idiosyncratic.
Applications in marketing are growing: Goettler and Gordon(2012), Ellickson, Misra, Nair (2012), Misra and Nair (2011),
Chung et al. (2012) ...If you are interested in applying this stu, you should readeverythingyou can get your hands on!
Misra Estimating Dynamic Games