Overquantification

9
CFA Institute Overquantification Author(s): Jack Gray Source: Financial Analysts Journal, Vol. 53, No. 6 (Nov. - Dec., 1997), pp. 5-12 Published by: CFA Institute Stable URL: http://www.jstor.org/stable/4480035 . Accessed: 11/06/2014 07:22 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. . CFA Institute is collaborating with JSTOR to digitize, preserve and extend access to Financial Analysts Journal. http://www.jstor.org This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AM All use subject to JSTOR Terms and Conditions

Transcript of Overquantification

Page 1: Overquantification

CFA Institute

OverquantificationAuthor(s): Jack GraySource: Financial Analysts Journal, Vol. 53, No. 6 (Nov. - Dec., 1997), pp. 5-12Published by: CFA InstituteStable URL: http://www.jstor.org/stable/4480035 .

Accessed: 11/06/2014 07:22

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

CFA Institute is collaborating with JSTOR to digitize, preserve and extend access to Financial AnalystsJournal.

http://www.jstor.org

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 2: Overquantification

GUEST SPEAKER

Overquantification

Jack Gray

Data are our industry's raw material. Our challenge is to add value by transforming them into meaningful and actionable information. Too narrow afocus on a "scientific" approach exposes us to the risk of overquantification-when ever- more precise measurement and ever-more complex models and calculations become substitutes for human judgment. This risk is increasing as the information age envelops us. Effective hedging requires equal emphasis on a "humanist" approach that demands creative, qualitative, hermeneutic skills, such as the use of analogies and metaphors; an awareness of human and social psychology; and a contrarian style thatforever seeks tofalsify.

I recently saw a movie devoted to the warm theme that everyone is connected to everyone

else through, at most, six acquaintances, through six degrees of separation. While watching, I did some calculations. My ex-brother-in-law works in Russia, where he has business dealings with the governor of Siberia, a political ally of Boris Yeltsin's. So, I am only three degrees of separation from Yeltsin, which I interpreted as meaning that I am extremely well connected. Subliminally, I extended the meaning to absurd levels and fanta- sized about Yeltsin dying on the operating table and my receiving the nod from Moscow to become president of all the Russias. My overinflated sense of self-importance was dashed when it dawned on me that if I am three degrees away from Yeltsin, my dog is only four degrees away.

The moral is that the precise measurement or calculation of a thing is profoundly different from the interpretation, significance, and meaning of that thing. Meaning is important, not measurement per se. We confuse the two because measurement appears to be precise, objective, and simple (it is not any of those) whereas meaning appears to be vague (or at least flexible), subjective, and complex (it is all of those.) By overemphasizing the first at the expense of the second, we are vulnerable to the bean-counter's paradigm: If it cannot be quantified or

measured, it has no significance (an extreme form of which is that there is no meaning, only measure- ment).1 This paradigm is often applied against cor- porate governance on the grounds that its effects on performance may well be unmeasurable because of the impossibility of separating cause and effect. When this suggestion was put to the general counsel of the State of Wisconsin Invest- ment Board, he replied that what is measurable is that corporate governance does give Wisconsin board executives assurances that a company is being run well (Barr 1996)-a very different sense of measurability, a response that emphasizes meaning.

Everything around us is quantified. Even films and music, being digitized, are merely coded sequences of zeros and ones. At times, overquanti- fication and our obsession with measuring and calculating is almost laughable-witness attempts to measure risk tolerance by taking physiological readings of investors when confronted with pic- tures of stock market crashes. At times, overquan- tification is dangerous, as with the inappropriate and excess measurement of human psychological attributes (see Gould 1996 and Hudson 1966). At times, it is obscene, as with Robert MacNamara's assessment of the progress of the Vietnam war by counting body bags, a futile attempt to measure the complex by the simple.

Over- and Misquantification Examples of over- and misquantification

abound. One, analogous to investment perfor- mance tables, concerns the Olympics. The United States "won" because it got the most medals. But is

Jack Gray is head of Investment Solutions at AMP Investments Australia Limited. This article is a modifi- cation of an invited address to the annual conference of the Association of Superannuation Funds of Australia, Brisbane, Australia, October 1996.

Financial Analysts Journal * November/December 1997 5

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 3: Overquantification

that what "winning" should mean? Shouldn't we discount for size and wealth? Aren't medals per capita or per GDP more-appropriate measures of a nation's athletic prowess? If so, the winner is Tonga, with but a solitary medal. In this instance, the quant's task would be to create the most appro- priate and meaningful measures of winning and to communicate them effectively. The challenge is that meaning tends to be complex, and we are often more comfortable with simple explanations, even if they are inappropriate and misleading. Examples of this problem are the many, largely unsuccessful attempts to squeeze the pair of numbers (with which we are uncomfortable) representing risk and return into a single number (with which we are comfortable), such as the Jensen measure or the Sharpe ratio.

Over- and misquantification are acute in our industry, which is dominated by data, numerical information, measurement, and calculation, a dom- inance reinforced by the personality preference, training, and narrow technical approach of many of the industry's players. Yet, quant reasoning is crucial for extracting meaning. It provides tools specifically designed to deal with variability and uncertainty, the very hallmarks of our industry. But these tools can be misapplied. A possibly apocry- phal story cites a motion passed at a U.K. union conference that in the future, all workers must have above-median wages. Amusing as this is, a consult- ant friend pointed out that 70 percent of Australian investment managers have as an objective for their balanced funds to be above median in the league tables. Not quite the same but close.

A more serious misapplication I have seen is using an asset/liability model to recommend a change of 1 percent to a strategic long-term asset- allocation weighting. No model can have that much precision; the markets are too noisy to justify that. A different example of overquantification arises with investment objectives, many of which have the form "to produce a negative return no more than 1 year in 10." If a manager produced negative returns in two consecutive years, most investors would interpret this record as prima facie evidence of fail- ure to meet the objective. But it is not. Simple statis- tical arguments show that the investor would need to wait about 100 years to be 90 percent sure the manager had, in fact, failed to meet the objective. Thus, although the numnbers and probabilities make the objectives appear measurable, in practice they are not. Again, measurement dominates meaning.

When reviewing managers, investors are reminded that past performance is a poor guide to future performance, in no small part because of the dominant role noise plays in the markets.2 Signal-

to-noise ratios are almost always so small that per- formance numbers rarely have any statistical sig- nificance; they are just as likely to have occurred by chance. For almost all managers in almost all sec- tors, this noise means investors need to wait more than 50 years to have a statistically acceptable degree of confidence that the manager's outperfor- mance was attributable to skill rather than to luck. Not only can they not wait that long, but the degree of outperformance needed to ensure significance probably would not persist for that long. The com- bined effects of weight-of-money, dependence on individual skills, and imitative competitors would erode any statistical significance.

Is there a meaningful way of using data to make rational decisions when those data have no statistical significance?3 This problem is exacer- bated at the extremes. At one extreme are relatively new asset sectors such as private investments, emerging markets, and TIPS (a.k.a. inflation-linked bonds) in the United States, where almost no past data are available. An investor who waits for the data to become available is actually waiting for that asset sector to become more informationally effi- cient, by which time arbitrage will have extracted most of the potential for enhanced returns. Inves- tors have to establish trade-offs between the oppor- tunity cost of waiting too long and the immediacy cost of entering too early, neither of which is observable or measurable, but both of which have meaning. A similar problem occurs with market impact, which is also not measurable because it requires assessing what would have been the effect had I done something, which in fact I did not do. The price at which one can trade is unknowable because the very act of trading disturbs the price. Assigning a spread to price "solves" the problem in the same sense that assigning a probability dis- tribution to velocity "solves" the uncertainty prob- lem in quantum mechanics.

At the other extreme is an oversupply of data and measurement. The United States has about 5,000 mutual funds with a further 4,000 separate accounts that encompass a variety of asset sectors. Some funds are open ended, some are closed ended; some are loaded, some have no load; they have differing fee structures and are tax advan- taged in different ways. They have different bench- marks and performance objectives; they have different investment styles ranging from, say, indexed (of which there are multifarious species) to actively managed through an artificial intelligence process. The question facing investors is how to make rational decisions in the face of this "crisis of complexity" (coined by Robert Amnott in Fabozzi 1989), particularly when there is either a paucity or

6 ?Association for Investment Management and Research

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 4: Overquantification

an oversupply of data. Some see quantification as a palliative and

argue that the solution lies in improved and ever- finer measurement and calculation (Riley 1996). In the context of U.S. mutual funds, their view is that the next generation of more elaborate attribution analysis products, with more numbers and more analysis, will provide investors with adequate deci- sion-making tools. My view is that this solution is only a partial one because it also increases complex- ity without necessarily providing further meaning. It ignores Keynes's wise counsel that "it's better to be vaguely right than precisely wrong," the thrust of which is that quantification is a limited tool that aids, but does not replace, human judgment. The chair of Nestle seems to understand the limitations of measurement. In the context of corporate plan- ning, he said, "Dealing with the figures takes no more than five minutes; we know they are wrong." He understands that even past performance figures are neither precise nor objective.4 He is aware that measurement can mesmerize and precise measure- ment can mesmerize precisely.5

Overquantification and complexity will only increase as we charge into the information age. We will all have to learn new ways of developing and managing processes for transforming complex data into meaning and action, ways of turning the raw input of data into the value-added output of mean- ing. The key is to exercise greater human judgment, aided by intelligent and critical use of measure- ment, calculation, and quantitative models.

There are some inklings as to how this end might be achieved. For defined-benefit plans, actu- aries measure the ratio of assets to notional accrued benefits to arrive at the accrued-benefits index (ABI), which they interpret as an indicator of long- term solvency. The ABI is not required to be above any prescribed target, nor does its level necessarily trigger any action by the actuary. Instead, it is used to monitor the plan's ongoing, long-term funding, to assess whether the plan is being over- or under- funded. The ABI is a measure actuaries use as a guide to decision making rather than as a prescrip- tion for it. There is a clear analogy here with the monitoring of short-term investment performance as a guide to long-term investing.

We all need to learn qualitative "soft" ways of interpreting quantitative "hard" data. Interest- ingly, some insights from the philosophy of science may help us better appreciate the qualitative and be more critical of the quantitative.

From Physics to Finance We got to this state of overquantification

through the rise of science. The scientific revolution

of the 17th and 18th centuries, driven substantially by commercial needs, led Newton and others such as Edmond Halley (who produced the first actuar- ial tables of morbidity and mortality) to realize that the world is not completely capricious, that we are not subject to the whims of the gods. The Newto- nian view is exactly the opposite: In principle, the physical zvorld is objective, nmeasurable, and totally pre- dictable.6 Mathematics and its models became the one true road to absolute truth.7 Science asserted total dominance over earlier patterns of thought. It became the role model to which all disciplines aspired. Only by attracting the appellation scientific could they attain legitimacy.8 The dismal science of economics rushed to legitimacy through Adam Smith and Thomas Malthus, both of whom pro- claimed fealty to the Newtonian paradigm. This paradigm-objectivity, measurability, and predict- ability-dominates the theory and the practice of investments.

But this paradigm may well be inappropriate for economics and finance. Even if, as Galileo claimed, "the book of nature is written in the lan- guage of mathematics," the books of economics and investments may not be because of three profound differences between physics and economics.9

A C '

IS A . 1 1 1 .

Tw Nort Cacae Suit 450 Coord Spins CO800

<a' ~~~~~~~~~~~~~~~~''W

Two North Cascade, Suite 450, Colorado Springs, CO 80903 For more information call

Columbine foundcr and president John S. Brush 800.835.075 1

Financial Analysts Journal . November/December 1997 7

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 5: Overquantification

First, physics focuses on characteristics, such as mass and velocity, that can be clearly defined and precisely measured and that, through well- specified and stable models, allow for precise pre- dictions. The theoretically valuable but practically flawed economic constructs of utility and risk tol- erance are attempts to emulate that process in investments. They have limited applicability because we do not carry around stable, objectively defined, measurable features such as utility or risk tolerance in the way we carry around physical characteristics such as weight and height. Even such fundamental notions as savings and growth are under challenge on this score. The United Nations' 1996 Human Development Report chal- lenges GDP as an adequate measure of a country's wealth on the grounds that it ignores such factors of human capital as education, health care, and overall quality of life. More worrying still is a serious suggestion that the economy may be becoming "intrinsically unmeasurable," largely because productivity in the services sector, and particularly the productivity of knowledge, is unmeasurable (see "A Survey of the World Econ- omy," the Economist, September 28, 1996, and Brock 1996).

The second difference between physics and economics is that the physicist can test hypotheses through controlled experiments, a methodology that is nigh impossible in economics.10 For exam- ple, there is an observed positive correlation between national savings and growth, which sup- ports a causal link that increased savings lead to increased growth. To the "new-growth" econo- mists, the causality may be the other way around: Increased growth (resulting from technological innovations, for instance) may lead to increased capital surplus and hence to increased savings. The policy implications are enormous, but there is no way of testing this hypothesis experimentally.

The third difference lies in the human capacity to influence the course of economic affairs in fun- damental ways that cannot be replicated in physics, for instance through market arbitrage at the micro level and through monetary policy at the macro level.1

Ross Perot's running mate, economist Pat Cho- ate, believes the optimal tax system for the United States can be determined by a quant model: "The computer will have the same role in creating a new tax system that the wind tunnel has in creating a new airplane." To me, this proposition is a serious philosophical and practical error at the heart of the problem of overquantification.12 It assumes the paradigm of physics applies to economics and finance. What is missing is the human and social

dimension of taxation policymaking. Can equity and social justice, not to mention the plethora of overlapping special interest groups, ever be ade- quately quantified?13

New Visions for Quant Partly because of our obsession with the New-

tonian paradigm and its obsession with quantifica- tion, different views of science have emerged that are worth exploring in economics and investments.

The French mathematician and philosopher Rene Thom argues that understanding does not require elaborate, quantified models and precise measurement. Rather, understanding means iden- tifying analogies between different areas and elabo- rating on them in qualitative ways. Analogies are also far more effective ways of communicating even quantitative ideas than are tight, analytic, quantitative arguments. The example of how to define Olympic winners could be elaborated into an effective way of communicating the difficulties with performance tables.

Thom is best known for catastrophe theory, a mathematical theory of sudden, dramatic changes. Yet, he mounted vitriolic attacks (as only French philosophers can) on those who sought to apply it to actual predictions and measurements. To Thom, insight, understanding, and meaning only develop through the elaboration of analogies, such as those between the suddenness of a stock market crash, the sudden changing of an embryo into human form, and the suddenness of heartbeats. Analysis of one phenomenon deepens our understanding of and provides meaning to the others. For instance, the behavior of the weather bears strong analogy to the behavior of stock markets; both exhibit ele- ments of short-term randomness, long-term stabil- ity, and extreme sensitivity to initial conditions. As a more unusual example, Australian funds may be about to make a sudden and substantial increase in their international exposures, which may be analo- gous to the known phenomenon of sudden switches in public opinion that haunted Clinton's advisors. Understanding switches in public opin- ion may help us understand the mechanism of switching to international assets and qualitatively predict the effect and timing of such switches.

Metaphors are exceptionally powerful vehicles for identifying, developing, and communicating analogies and hence for generating understanding and action. Effective examples are game theory as a metaphor for trading in the markets and, more generally, for conducting business and fractals as a metaphor for organizations (see Brandenburger and Nalebuff 1996, Bernstein 1996, and Leonard- Barton 1995).14 In that spirit, a well-developed and

8 ?Association for Investment Management and Research

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 6: Overquantification

insightful metaphor for long-term investing should help take our focus off short-term investment results more effectively than an analytic or quanti- fied argument. My best shot is the family home, which, like superannuation, is a long-term invest- ment with substantial emotional and other non- quantifiable benefits.15 Most of us keep an eye on our home's short-term price, but few make deci- sions on this basis. The family home is a working metaphor for that easily stated, but difficult to quantify, slogan: Monitor the short-terni, act on the long-term. I asked an actuary how he would inter- pret that slogan in the context of performance. His answer was as precise, simple, and definite as it was inappropriate: "I would fire a manager after six quarters of underperformance." The "right" answer must be more complex and less objective. In assessing managers, "softer," nonquantifiable criteria such as investment process, team stability, and recruitment policies are now widely used in decision making. Roger Urwin of Watson Wyatt Worldwide in the United Kingdom argues that those are essentially the only valuable criteria. The "right" answer to the question should transcend mere measurement and focus on qualitative aspects of the manager's performance. A good met- aphor would help.

A second challenge to Newton comes from the U.S. philosopher Thomas Kuhn, who died recently. He saw science as an intrinsically social activity, indulged in by humans with all their angst, egos, and foibles and complete with norms and taboos shared by all social groups. Significantly, because these attributes play such a dominant role in the acceptance or rejection of ideas and techniques, the scientific endeavor is far from rational and objective. O'Barr and Conley (1992) produced a fascinating study of decision making in an investment manage- ment organization from the point of view of social anthropology that confirmed Kuhn's position.

Kuhn's views challenge the supposed objectiv- ity of measurement and of quant models. In the face of this subjectivity, some retreat to history as the only objective and reliable basis for decisions. But the past always has a future. By relying only on historical returns as a basis for decision making, investors are implicitly forecasting that the future will be like the past. Subjectivity is unavoidable. For instance, Brown, Goetzmann, and Ross (1995) showed that in drawing inferences to other markets, one is exposed to survivorship bias, the bias induced by looking only at stock markets that have survived, such as those of the United States, the United Kingdom, and Australia. Survivorship bias ignores the many stock markets around the world that have experienced breaks or simply disappeared.

In the context of physics and chemistry, Kuhn showed that even the collection and measurement of data involve significant subjective judgments. Nearly 200 years of return data are available for the U.S. stock market, but many come from unreliable sources, some have been tampered with, and dif- ferent parts of the data set have been constructed through different calculations. The data are any- thing but pure and objective, although some sub- jectivity can be removed through a cleansing process (see Siegel 1994).16 History too is complex and subjective, replete with human choices.

Kuhn's emphasis on the human and social aspects of science links with a new and growing area of finance, behavioral economics, which tries to under- stand and legitimize the way our decisions are intrinsically influenced by. psychological factors and (nonrational) behavior. Even if you are per- fectly rational, not all players in the market are, and because you are playing a game against them, it pays to understand their nonrational behavior.17 I believe the next great leap forward for investments will evolve from a blending of behavioral finance and quant (see Kahneman, Slovic, and Tversky 1982 and Thaler 1995). The World Bank's US$6.5 billion pension fund was recently restructured around pre- cisely this coupling (Bensman 1996).

An instance of behavioral factors is that our limited human vision and tendency to herd means that forecasts are smoother than actual data and tend not to predict unfavorable outcomes. This observation is borne out by William M. Mercer's survey of Australian experts' return forecasts. Almost never do they forecast negative returns from the share market in spite of the well-known evidence that since 1900, shares have produced negative returns, on average, approximately one year in four, rising to approximately one year in two since 1980. Forecasters are also notoriously poor at predicting unexpected extreme results, such as the share market crash of 1987, the bond market crash of 1994, or German hyperinflation of the 1920s. Yet, these are precisely the areas of great- est risk.18 This view is not an attack on forecasting but a plea to recognize the very human biases and prejudices quants and forecasters have. We are all exposed to Galbraith's pithy dictum: "There are two types of forecasters: those who don't know and those who don't know they don't know."

A further example of the influence of psychol- ogy on decision making is experts' tendency to see trends where none exist. When shown random patterns, they often see, explain, and rationalize order in the patterns even after they know patterns are random. This practice is an instance of what psychologists call cognitizve illusions (similar to

Financial Analysts Journal * November/December 1997 9

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 7: Overquantification

optical illusions), a deep tendency to apply incor- rect reasoning and for it to persist even after it is known to be in error.19 One common instance occurs in retailing (and may well apply to portfolio management), in which goods priced at $1.99 sell far better than those priced at $2.00, even after buyers are cognizant of the irrelevancy of the 1 cent difference.20

I often hear the following question (usually stated in terms of trending returns): If an unbiased coin is tossed 100 times and each time it comes down heads, what is the rational bet on the 101st toss? The most common, and definitely the wrong answer, is to bet on a tail, usually supported by a flourishing appeal to the supposed "law of proba- bility." 21 Only there ain't no such law. Yet, this argument persists long after the error has been explained and understood and has the potential to result in inappropriate investment decisions. It is one of many cognitive illusions about probability.

A final challenge to the Newtonian paradigm comes from the Austrian philosopher Karl Popper. He disagreed fundamentally with the Newtonian view that science is part of an eternal search for objective truth. To Popper, there are no objective truths because we can never be certain that some- thing is true. We can be certain that something is false, so the aim of science is to find the limits to truth through an eternal search for falsification. Accordingly, scientists should not (and often do not) aim at refining and improving their models to make them more accurate and truthful. Rather, they and quants should seek ways in which their models are false.

Popper's view is immediately applicable to quant investment models, whether concerned with attribution analysis, manager selection, asset alloca- tion, or stock picking. The first question one should ask of any quant model is, under what conditions will it fail? This style of question has to be posed with some sensitivity lest the questioner cross the gray area between criticism and negativism.

Often, and with some justification, these models are derided as "black boxes," the implication being that unknowable mysteries are lurking within. Sometimes they are. When Galileo showed the tele- scope to church elders to see the mountains of the moon, their healthy assumption was that he had put something inside. In the 19th century, an entrepre- neur made a pile of money exhibiting a chess-play- ing machine. It transpired that the mechanism in the machine was but a chess-playing midget.

When building and testing models, most quants are "Popperian." We are most comfortable when the output is obviously rubbish, because then we are certain something is wrong. We live in trep-

idation when the models look reasonable, because then we are uncertain as to whether they are right or wrong. But no matter how Popperian quants are, everyone who uses quant and measurements is vulnerable to a dangerous illusion. Once the num- bers, analysis, and output appear on a screen or in hard copy, they often develop an unjustified legit- imacy. Once they have the authority of the printed word, we tend to believe in them too much. (I blame Moses for this.) Kuhn would not be surprised that having created, lived with, and owned the model, our attachment to it might transcend rational objec- tivity. It is a risk we forever try to puncture. Because of this attachment, those who concern me most are those who say "it is mathematical, therefore it must be right." I am far more comfortable with those who articulate the contrarian view that "it is mathemat- ical, therefore it must be wrong," because that reminds me of the model's and my own limitations and forces me to justify myself. The chair of Nestle should (and presumably does) spend more than five minutes on the figures precisely because he knows they are wrong.

Conclusion In one sense, I am supporting better, more-

critical, and more-appropriate measurement as a partial solution to overquantification and complex- ity. The recent performance standards are taking this tack by cleaning up performance measurement and by standardizing reporting periods to make performance more objective.

But even clean data, numbers, and measure- ment are of minimal value by themselves. They may contain embedded meaning, but extracting it requires creative skills of interpretation. They are commodities, easily and cheaply produced. Our task is to transform them into meaningful and actionable information through the value-adding process of structuring, analyzing, and interpreting. To transform them effectively requires new qualita- tive approaches, such as the use of analogies and metaphors, an awareness of the importance of human and social psychology, and a contrarian style that forever tries to falsify. These measures may protect us from the trap of overquantification: the belief that more-precise measurement and more-complex calculations are substitutes for human judgment.

Ambachtsheer, Boice, Ezra, and McLaughlin (1995) advocated new and different ways of assess- ing "best practice" for pension funds. They aimed to measure a fund's "quality shortfall": the basis points of extra performance the fund could extract if all its components operated optimally. I suspect that the actual measure of quality shortfall is an

10 ?Association for Investment Management and Research

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 8: Overquantification

instance of overquantification. The measure itself is of far less significance than the meaning behind quality shortfall and the process of trying to mea-

sure it. As with the search for the holy grail, the truth lies in the search, not in the cup.

Notes

1. The extreme form is known as logical positivism. 2. Gruber (1996) provides some evidence of persistence,

predictability, and reward for past performance, at least for open-ended U.S. equity mutual funds.

3. In physics and even in some fields of social science, statistically insignificant results are simply rejected out of hand. Information theory, an area on the border between statistics and computer science, may offer some help in answering this question.

4. The supposed objectivity of measurement was sorely tested in 1990 when the two major U.K. measuring firms differed by 60 basis points in their measurements of that year's return from U.K. shares.

5. A recent survey of the Retail Investment Funds Association of Australia exposed deep skepticism of quantitative (and qualitative) standards of research, particularly on perfor- mance.

6. Chaos theory establishes a distinction between predictabil- ity and determinism.

7. The complex of ideas and results known collectively as Godel's theorem reveals intrinsic limitations to the scope of mathematics. One reaction has been to overemphasize syntax (i.e., logic and measurement) and to underempha- size semantics (i.e., meaning).

8. There are academic books and journals devoted to mathematical psychology, mathematical sociology, mathe- matical history (cliometrics), the mathematics of law, and even the mathematics of literary criticism. Almost all are arrant nonsense.

9. Even in physics, there is serious discussion about the role and limitations of mathematics. The Nobel prize-winning physicist Richard Feynman, known (and reviled by pure mathematicians) for his singular developments and applications of quite abstract mathematics, claimed that "If mathematics had not been invented, physics would have been delayed by one week," to which Mark Kac, a great applied mathematician, replied "Yes, but that was the week the universe was created." On balance, it is hard to disagree with Eugene Wigner's (another Nobel prize-winning physicist) view that mathematics is "unreasonably effective" in physics. A priori, it is surely unreasonable to expect that the stochastic calculus developed in 1948 would, a quarter of a century later, contribute directly to the

creation of new investment instruments and entirely new markets.

10. Cosmology is the obvious exception and the reason some physicists dismiss it as unkosher physics. Financial computer simulations are an inadequate way of side- stepping the issue.

11. In aggregate, for investment markets, this arbitrage argument leads to the efficient market hypothesis, the very order and simplicity of which is one of its attractions.

12. The Boskin Commission's conclusion that the U.S. Consumer Price Index has been overestimated by 50 percent underlines the practical difficulties.

13. Perhaps all is not lost. Artificial intelligence and fuzzy logic offer some hope (and much hype) in dealing with these less quantifiable factors.

14. Analogies, metaphors, and graphical images do play a role in physics. Maxwell's demon in thermodynamics and the butterfly effect in turbulence are two well-known examples.

15. The difficulty with this metaphor is that the family home is often seen as an alternative to superannuation.

16. Data in Ibbotson and Sinquefield (1989) no doubt took many years of creative energy to clean to the point where they won the imprimatur of the finance community.

17. Rational behavior is often defined as that which maximizes expected utility. This definition, however, allows all behavior to be rational: Whatever the behavior, simply construct a utility function that is maximized by that behavior. This approach allows suicide, martyrdom, and choosing to live in abject poverty to be "rational." In the Austrian philosopher Karl Popper's terms, this set of ideas is "nonfalsifiable."

18. The relatively new branch of statistics known as extreme value theory addresses this issue.

19. There may be evolutionary advantages in seeing patterns and order where there are none. See Piatelli-Palmarini (1994).

20. This view can be justified as rational if the buyer's utility function has a discontinuity at $2.00. It is more difficult to justify in Australia where purchases decrease at $2.00 even though there is no 1 cent coin!

21. The "rational" answer is to bet on a head because the coin is almost surely biased.

References

Ambachtsheer, Keith, Craig Boice, Don Ezra, and John McLaughlin. 1995. Excellence Shortfall in Pension Fund Management: Anatoniy of a Probletmi. Toronto: Cost Effectiveness Management, Inc.

Arnott, Robert. 1989. "The Investment World of the 1990s: Reflections on the Future." In Portfolio & In vestment Managemtient, edited by Frank Fabozzi. Chicago: Probus:17-32.

Barr, Paul G. 1996. "Study: Activism Has No Impact." Pensions & Investments (September 30):61.

Bensman, Miriam. 1996. "Brave New World Bank." Instituitionial Investor International Edition, vol. 21, no. 9 (September):95-98.

Bernstein, Peter L. 1996. "Pascal's Wager and the Efficient

Market Hypothesis." Journal of Portfolio Manzagemiienit, vol. 22, no. 1 (Fall): 1.

Brandenburger, Adam, and Barry Nalebuff. 1996. Co-opetitioni. London: Harper Collins Business.

Brock, Woody. 1996. Strategic Economiic Decisionis. (Summer) Menlo Park, CA.

Brown, Stephen, William Goetzmann, and Stephen Ross. 1995. "Survival." Jolurnial of Fin1an1ce, vol. 50, no. 3 (July):853-72.

Gould, Stephen J. 1996. The Mismeasure of Man. New York: Norton.

Gruber, Martin. 1996. "Another Puzzle: The Growth in Actively

Financial Analysts Journal * November/December 1997 11

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions

Page 9: Overquantification

Managed Mutual Funds." Journal of Finance, vol. 51, no. 3 (July):783-810.

Hudson, Liam. 1966. Contrary Imaginations: A Psychological Study of the English Schoolboy. London: Metheun.

Ibbotson, Roger, and Rex Sinquefield. 1989. Stocks, Bonds, Bills & Inflation: Historical Returns (1926-1987). Chicago: Dow-Jones Irwin.

Kahneman, Daniel, Paul Slovic, and Amos Tversky (eds). 1982. Judgement under Uncertainty: Heuristics & Biases. London: Cambridge University Press.

Leonard-Barton, Dorothy. 1995. Wellsprings of Knowledge. Boston: Harvard Business School Press:260.

O'Barr, William, and John Conley. 1992. "Managing Relationships: The Culture of Institutional Investing." Financial Analysts Journal, vol. 48, no. 5 (September/October):21-27.

Piatelli-Palmarini, Massimo. 1994. Inevitable Illusions: How Mistakes of Reason Rule Our Minds. New York: John Wiley & Sons.

Riley, Barry. 1996. "Measurement." Professional Investor, vol. 6, no. 8 (October):32.

Siegel, Jeremy. 1994. Stocks for the Long Run. Burr Ridge, IL: Irwin.

Thaler, Richard (ed). 1995. Advances in Behavioral Economics. New York: Russell Sage Foundation.

12 ?Association for Investment Management and Research

This content downloaded from 195.34.79.103 on Wed, 11 Jun 2014 07:22:24 AMAll use subject to JSTOR Terms and Conditions