What Makes A Good Model in Economics and Finance

Ioanid Rosu

University of Chicago

ASE, December 21, 2006

Introduction What is a good theoretical model in economics?

Why is it that most famous models in economics are relatively simple?

We will not discuss here what makes a model important (i.e. useful). Typically, such a model Explains an important fact (e.g. the Phillips curve:

inflation vs. unemployment) Gives a framework to think about important issues (e.g.

Arrow’s social choice theory) Provides an important methodology or economic intuition

(e.g. Akerlof’s lemons model – asymmetric information)

Good = True = Credible Instead, we want to know what a true model is

In economics true is usually replaced by credible or appropriate We do not test an economic model, but rather we validate it, i.e.

we decide whether it is credible or not In general, a model:

Assumptions ) Conclusions (via causal chains) Naively, one may propose two criteria for a credible

economic model: Internal consistency (mathematically true) Verifiable conclusions (empirically true)

This works for physics Assumptions and causal chains are very reliable (physical laws

and mathematical derivations) Conclusions can be verified by controlled experiments

Economics vs. Physics Can economics imitate physics? Some say no:

“Economics suffers from physics envy. […] We would love to have 3 laws that explain 99 percent of economic behavior; instead, we have about 99 laws that explain maybe 3 percent of economic behavior!” (A.W. Lo)

Nevertheless, are economic assumptions reliable? For example in many models agents are assumed to be rational,

even though there is ample evidence to the contrary Milton Friedman (Nobel prize 1976): it doesn’t matter as long as

the conclusions are verified E.g. agents are assumed to behave as if they were rational, even if

individually they can be irrational

And how about the structure of the model? Can we have as many assumptions and causal chains as we want? Mathematical economists seem to believe so

Why Economics Is Hard In fact things are not so simple with economic models

Assumptions and causal chains are noisy, and controlled experiments are rare (i.e. one cannot eliminate the outside noise)

Each causal chain amplifies (compounds) the noise, so one must focus only on a few causal chains

As a consequence, a typical economic model is not a complete system, but only a window through which we study reality

Moreover, economic models themselves may affect human behavior, which creates two sides of economics:

Positive (describe reality as it is) Normative (describe reality as it “should” be)

The economic truth is a partial, moving target Often when a model fails empirically, economists still use

the model as a guide They rely on the normative aspect: if only the economics agents

were rational and listened to us…

How to Think of Economic Models Because of the noise, an economic model is not a

complete system (e.g. Newtonian mechanics), but a window which connects a few economic variables

So we should think of a model as A few causal chains between a small set of variables A plausible explanation for a set of conclusions

Now, when we think of a good model, we want to know whether the window through which we observe reality does a credible job

Let us call an economic model credible (good) if The assumptions are plausible, although possibly imperfect (noisy) The explanation itself is plausible, i.e. the results are strong enough

to be distinguished from noise The conclusions are not too sensitive to the assumptions

The 3R of Economic Research & the ABC of Model Verification

The 3R of economic research: a good (credible) economic model must be

1. Realistic2. Relevant3. Robust

Moreover, a (positive) economic model should be 4. Verifiable The ABC of model verification: a good model has

A. AlternativeB. ByproductsC. Calibration

Example: CAPM(Capital Asset Pricing Model) Assumptions

Agents only care about risk and return Their utility depends only on the mean of the portfolio ( E )

and its volatility ( ) Agents have the same beliefs about asset returns

E(R) and (R) are the same for all agents The market is in equilibrium

Conclusions Agents invest the same relative percentages of their

wealth in the risky assets (stocks) Everybody invests only in the riskless asset (= the “bond”)

and the collection of all stocks (= “the market”) Main conclusion: the expected return of a stock

satisfies the CAPM formula E(Ri) – Rf = i ( E(RM) – Rf )

The 3R of Economic Research A model approximates reality, and tries to reliably explain an

economic phenomenon Since we start with noisy assumptions and we draw noisy

inferences (causal chains), we need to make sure that conclusions are not too noisy In some sense, the noise of the conclusions equals the noise of

assumptions, compounded by the noise of the causal chain The model therefore must be

Realistic: the assumptions are plausible, i.e. they are a good approximation of reality

The assumption noise is small Relevant: The results are strong, i.e. first order

The causal chain noise outside the model is relatively small Robust: The conclusions are not too sensitive to the assumptions

The causal chain noise inside the model is small

Then the conclusion noise is also relatively small

1. Realistic Assumptions must be realistic, i.e., they should represent a

reasonable approximation of reality Contradiction with Friedman’s thesis?

No, because “plausible” is not the same as “true”. Assumptions need not be true per se.

Realistic assumptions are the foundation of good research: Suppose you find a conclusion is false Then one of the assumptions is false If the assumptions are not realistic (plausible), it is hard to learn from the

failure of the conclusion Example: When testing CAPM, we find that agents are far from

holding the market portfolio (e.g. home bias) Then one of the assumptions is false, but which one? Maybe that E(R) and (R) are the same for all agents Since the assumption is plausible, it is clear how to modify it: e.g. agents

have different information about asset returns and volatilities The Black–Litterman model modifies exactly this assumption and

suggests how agents with different beliefs should invest

2. Relevant The results must be relevant, i.e. the assumptions provide

a plausible explanation of the conclusions Since a model is only a window to reality, one has to make sure the

causal chains are strong enough compared with the outside noise For example, many models can predict that in the summer it’s hot,

but do they provide a plausible explanation? Relevance is the key feature of a good model. Typically a

relevant model satisfies the 3S rule: Has a story, i.e. a clear intuition behind it Is simple: too many assumptions weaken the value of a conclusion

(each assumption comes with an inherent error) The sizes implied by the model are plausible (e.g. from a

calibration) Example: CAPM is relevant for explaining expected returns

by betas (Fama & MacBeth 1973) Even if the CAPM formula today is empirically less true

(APT), the model remains relevant: the normative aspect!

3. Robust A model is robust when small modifications of the

assumptions do not change conclusions in an essential way It is very rare that assumptions are perfectly plausible, so we must

make sure that the inherent error in choosing them does not affect the results significantly

Example: In one version of CAPM it is assumed that agents have quadratic utility This is a strong assumption: it implies a saturation point Do the conclusions of CAPM depend essentially on this assumption? No, the conclusions also hold if stock returns are normally distributed

In reality, the distribution of stock returns is not exactly normal, but has “fat tails” (it is leptokurtic)

If we modify this assumption, it turns out that CAPM is still approximately true

4. Verifiable Karl Popper’s criterion: a (positive) scientific model must be

verifiable, i.e. falsifiable Verification of a single conclusion can be done by

Intuition or anecdotal evidence: Is the conclusion reasonable? Do we have arguments or examples that support it?

E.g. in corporate finance (moral hazard, risk shifting) Statistics: Test the conclusion by bringing it to the data

E.g. to test CAPM, one can check whether a = 0 in the linear regression E(Ri) – Rf = a + b £ i + ei

One can also check whether b = E(RM) – Rf It is not enough to verify just one conclusion of the model.

One also needs to verify the model itself, since in economics conclusions are model dependent The ABC of model verification: a good model has

A. AlternativeB. ByproductsC. Calibration

A. Alternative One must have an alternative (H1) to the main

conclusion (H0) of a model In many cases, the alternative is obtained by

negating H0 Example: for CAPM, H0: a = 0 , H1: a 0

Ideally, one would like to have an alternative model, to provide an alternative explanation for the main conclusion Example: One can compare CAPM to the alternative

model, APT (Arbitrage Pricing Theory) in its Fama–French version:

E(Ri) – Rf = i ( E(RM) – Rf ) + hi E(RHML) + si E(RSMB)

where HML is a factor that represents “value – growth” firms, and SMB represents “small – big” firms

B. Byproducts Besides an economic model’s main conclusion,

there are also secondary conclusions (byproducts) Since other alternative models may have the same main

prediction, one can check that the given model provides the correct explanation by also testing the byproducts

Example: The main conclusion of CAPM is that expected stock returns only depend on betas

It also has the byproduct that agents invest in the same way in stocks – the market portfolio This may explain the popularity of index funds (low cost

copies of the market portfolio) But there are empirical problems (e.g. home bias)

Again, one can refer to the normative side of the model

C. Calibration One should be able to calibrate the model (both

the assumptions and the conclusions) Many models can predict that in the summer it is hot The question is: exactly how hot is it in the summer? Otherwise, we cannot tell if our explanation is correct Example: CAPM not only predicts that the higher the

risk – the higher the expected return, but also gives a precise estimate of the expected return ( E(R) )

The parameters of the calibration should be plausible For this, it is important that the model be realistic

(otherwise, what does “plausible parameter” mean?)

In Conclusion A good economic model should be (the 3R)

Realistic Plausible assumptions

Relevant Results should be of first order (plausible explanations)

Robust Small changes in assumptions do not change conclusions in an

essential way Verifiable (the ABC of verifying a model)

Alternative Compare the given explanation with an alternative one

Byproducts The other conclusions of the model should also hold

Calibration One should calibrate the model (with plausible parameters) and

compare it with the data