Biased Reg

8/8/2019 Biased Reg

http://slidepdf.com/reader/full/biased-reg 1/45Electronic copy available at: http://ssrn.com/abstract=1681609

Prudence with Biased Experts:

Ratings Agencies & Regulators

David M. LevyCenter for Study of Public Choice

George Mason [email protected]

Sandra J. Peart Jepson School of Leadership Studies

University of [email protected]

23 September 2010

We have benefited from comments from Hugh Rockoff, Roger Koppl, Garett Jones, Roger Congleton,Eric Schliesser and Michel Heijdra. We have been helped considerably by Leah Donnelly of the SpecialCollections Division of George Mason University Library. Tab Lewis checked the holdings of theNational Archives for us. Thanks are due to Jane Perry for editorial help. David Ortiz helped removemistakes. The mistakes which remain are our responsibility.

JEL Codes A11, B23, B31, G01, G24

Abstract.

Why were the rating agencies trusted? When they became required for Federal deposit insurance theirincentives for upward bias was common knowledge. The requirement was attacked by a Chicagoeconomist, Melchior Palyi, on philosophical grounds (the expertise is excessively secret) and technical(Moody‘s forecasts were inaccurate). The Federal government financed a massive study of bond ratingswhich developed a technical response to remove the bias. The study required a trade with the ratingagencies so the authors wrote prudently to avoid offending the agencies. They disguised the meaning of their procedures and did not discuss the full dimensions of Palyi‘s challenge. When the technical methodsfailed, the loss of memory did not allow us to recover Palyi‘s warning about non-transparency.

mailto:[email protected]






8/8/2019 Biased Reg

http://slidepdf.com/reader/full/biased-reg 2/45Electronic copy available at: http://ssrn.com/abstract=1681609

1

The difficulty is that the article is very critical of the bank supervisory

and rating agencies. I cannot find it possible to say much about his

article without appearing to put the memorandum of the Corporate

Bond Study in the position of criticizing the cooperating agencies

which were responsible for it.

Harold Fraine, technical director of the Corporate Bond Project,to Donald Thompson of the FDIC

March 13, 1943

8/8/2019 Biased Reg

http://slidepdf.com/reader/full/biased-reg 3/45

2

Introduction

The damage caused by the belief that AAA-rated investments were as secure as those offered by the

US government is still being tallied. The rating shopping which put a devastating upward bias in the

ratings is no longer a matter of speculation and hearsay (Kolchinsky 2010). The most interesting question

remaining is why did markets and the regulators ever trust the rating agencies? The answer we offer is that

at the moment at which the rating agencies became part of the regulatory establishment, the judgments of

individual rating agencies were not trusted. Their incentive to provide upwardly biased ratings was

common knowledge and published claims of their inability to outperform the market were uncontested.

What was trusted was a statistical procedure which bounded the influence of an individual rating agency

and attenuated the consequences of the bias. The term of art was a ―composite‖ of the ratings. The

wisdom which underlay this prudential estimation procedure was lost between the time of establishment

and the disasters of recent memory.

The ―composite‖ provided its worth in the massive study was undertaken by the Federal

government, operating through the National Bureau of Economic Research The trades with the rating

agencies to obtain their co-operation, imposed both a confidentiality agreement as well as some tacit

understanding that seems to have precluded plain speaking in public about what the ―composite‖ was

designed to do. Prudential writing requires unusual care in reading.

What was evidently private knowledge amongst the study‘s participants did not become public

knowledge and so the justification for, and the requirement of, a composite of agency ratings did not

become known outside the research group. The procedures adopted seemed to work sufficiently well so

that the insight embodied in them was forgotten. When the insight faded, we worry that the prudential

procedures were abandoned since the incentives to provide biased ratings were not changed.

8/8/2019 Biased Reg


3

Every history of rating agencies discusses the Corporate Bond Project [CBP] and the sequence of

National Bureau of Economic Research [NBER] volumes published by Braddock Hickman in the 1950s

(Hickman 1952, 1953, 1958; Hickman and Simpson 1960). It is useful to stress that the CBP came in

two phases, the first from 1938-44 which obtained massive federal funding; the second which resumed

after the war with private funding, picking up where the federally-funded phase ended.1

Less frequently discussed is the earlier, unsponsored Columbia dissertation by Gilbert Harold

published in 1938. Unlike the NBER study, which had the resources provided by the Federal Deposit

Insurance Corporation [FDIC] operating through the Works Progress Administration [WPA] to study

the universe of interesting large bonds plus 10% of the uninteresting small ones, Harold worked with a

sample of 363 bonds.2 It is easy to explain why such a tiny study would be overshadowed by one for

which resources are no constraint that utilizes (almost) all the information in the market.

Remarkable enough in modern scholarship, the enormous NBER research effort is treated as

manna from heaven as if economic research were outside economics itself. 3 Treating expertise as exogenous

to the economy may be indeed the root of the difficulty. If we think of economic analysis as itself part of

the economy, we are encouraged to ask what sorts of trades need to be made in order for the study to be

completed. If a scholar can finance the study without external assistance save from those whom he or she is

sympathetically connected then perhaps fewer trades need to be made. Indeed, it is evident simply by

1 The relationship between the two phases of the study are briefly sketched in the short papers collected as ―Corporate BondExperience Re-Examined‖ in an unidentified periodical of December 1952 found in the Arthur Burns Papers, Box 146,―Corporate Bond Project — NBER.‖ The immediate thought – this is an NBER periodical – has not been verified. Theonline NBER collection turns up nothing when searched. We will, of course, be happy to make copies available.

2 The agency which began as the ―Works Progress Administration‖ became the ―Work Projects Administration‖ in 1939.

3 ―We are fortunate the research projects of the National Bureau of Economic Research studied U.S. corporate bond quality,including the performance of bond rating agencies, during a long period ….‖ (Sylla 2002, p. 25). There is nothing in hishelpful three-page discussion of the NBER project that suggests how it was financed. For whom would it be of interest topay wages for sometimes as many as 200 full-time workers for a multi-year project? We are fortunate that themimeographed history of the first phase of the project was produced and distributed. WorldCat reports that there areseveral libraries with holdings, in addition to the Library of Congress and that of the Federal Reserve in Washington, of what we list as NBER 1941. The title page answers the question for whom is the answer important: The Corporate Bond Project, A History and Description, A Work Projects Administration Study Sponsored by the Federal Deposit Insurance Corporation, Supervised by the National Bureau of Economic Research.

8/8/2019 Biased Reg


4

looking at the Harold book and at the NBER documentation, that Harold‘s tiny study faced fewer

constraints in one critical dimension than the massive project which involved so many Manhattan-centered

financial institutions.

Harold, without the constraints imposed by the sponsorship of the Federal government and the

active participation of the ratings agencies, could report a sample of the distribution of ratings by agency

and call attention to a disturbing finding. The NBER Corporate Bond Project was precluded from

revealing the ratings given by individual agencies even in the illustrative documentation. Here‘s what the

censored documentation looks like from the Corporate Bond Project.

It will not escape notice that there are four rating agencies, and the ―composite rating‖ is said to be a

median. The median of four observations is a very subtle concept. Stephen Stigler began a famous article

asking what is the median number of pages of a four-volume book.4 This ambiguity and the non-

disclosure of individual ratings might be taken as a warning to read with extraordinary care.

4 ―The last numbered pages of the four principal volumes of Laplace ‘s Mécanique céleste are, in increasing order, 303, 347, 368,and 382. What is the median number of pages in the volumes of the Mécanique céleste? ‖ Stigler (1977, p. 544).

8/8/2019 Biased Reg


5

The potential problem of expert guidance that can flow from the self-interest of experts was

completely obvious to the economists who questioned the use of ratings and to those working at the

NBER and the FDIC who employed them. The steps taken to control the self-interested bias of experts

seem to have worked so well that the dangers were forgotten. The story we wish to tell is, we hope, more

than an intellectual ―corn-hog cycle‖ in which present success breeds future failure. The information

necessary to warn that observed trustworthy behavior depended upon a concern for the potential of

untrustworthiness did not become public because the knowledgeable economists did not find such candor

to be prudent. Their project depended upon the co-operation of rating agencies whose self-interested bias

was a central problem with which they needed to deal. A great deal less was said in public than was said in

private.

The first thing, therefore, which we need do is to demonstrate the care with which the public

record must be read. One might suspect that a desire to deal with the self-interested bias of the rating

agencies was a motive from the way the public argument developed and how a sequence of estimates were

carried out. A conjecture is all the public record allows. The public record does not reveal intentions and

without intentionality it is hard to impute meaning.5

We offer no reason to believe that anything known to be false was published. But what was

published was prudently considered to be both true and to not offend the sensibilities of the rating

agencies with whom the authorities, and those whom they were funding, were co-operating. We believe

this because the private correspondence which we reprint discusses writing truthfully but prudently to

avoid offense. We will break chronological order, and start first with a fundamental challenge by Melchior

Palyi to the use of non-transparent expert opinion for public policy and then jump deep in our history

5 In the first version of the paper, we read some of the public debates and the sequence of estimates and so conjectured aboutwhat it meant. We were immediately challenged by Hugh Rockoff to look into the possibility that the NBER estimationmethod was adopted to deal with agency bias and to mimic regulatory practice. We offer evidence to support this claim.

8/8/2019 Biased Reg


6

when the final report of the first phase of the CBP is being written on how to respond to his criticism. We

will learn how to read by reading the concerns about how to write.

Since the importance of Palyi‘s Weberian attack on the regulatory establishment of rating agencies

with their secret methods – priestcraft with a new name – was concealed, scholars who write on Chicago

economics have been directed away from evidence of Weber‘s influence on the old Chicago school.6 Palyi

was one of the two scholars who saved Max Weber‘s lectures on economic history from the oblivion of

manuscript and memory and so allowed Frank Knight‘s translation, General Economic History (Weber

[1922] 1927).

Government by Expert Raters

Following the ghastly experience with the banking collapse of 1929-33, two options were on the

table. One, often referred to as the Chicago banking plan, would make banks run-proof by turning them

into monetary warehouses. This option had the merit, from a liberal point of view, of minimizing the

uncertainty induced by political discretion. This, of course, was not the approach selected. The winning

alternative, which became law in the Glass-Steagall Act, separated the banking system into two parts, a

commercial banking sector which offered deposits insured by the Federal government, and an investment

banking sector. For many years, the country‘s experience with deposit insurance was offered as evidence

that the liberal economists were overly pessimistic about the role of discretion.7 Perhaps in light of recent

experience this confidence in expert guidance will be seriously reconsidered.8

6 There is only a little work on Weber and Chicago economics. Emmett (2006) discusses the Weber-Knight connection.Schliesser (2011) discusses the role Weberian ideas played in Milton Friedman‘s methodological thinking. Weber‘s ideal

type of a ―bureaucratic administrative staff ‖ supposes experts behaving transparently. They are ―selected upon the basis of objective qualification for his job, which, in the case of full rationality, is ascertained by an examination and proven by anofficial diploma‖ and their choice is directed by ―objectively defined official duties‖ (Rheinstein, 1954, pp. xli-xlii).

7 The importance of the integration of liberal norms into economic theory by Simons (1934) and Friedman (1962) forunderstanding the Chicago monetary contribution is stressed in Laidler (1993). An expression of confidence from theprevailing side is found in Viner (1962). Viner, in an advisory position, took upon himself the role of the opposition to theChicago Banking Plan, then advocated most effectively by Irving Fisher (Fiorito and Nerozzi 2009, p. 117).

8 Off the books government debt in the form of insured bank deposits is a theme in Reinhart and Rogoff (2009). Can we have

8/8/2019 Biased Reg


7

The problem with Federal insurance was explained by Franklin Roosevelt at his first Presidential

news conference. How does the government propose to tell the difference between good banks and bad

banks:

The general underlying thought behind the use of the word ―guarantee‖ with respect tobank deposits is that you guarantee bad banks as well as good banks. The minute the Governmentstarts to do that the Government runs into a probable loss. … We do not wish to make theUnited States Government liable for the mistakes and errors of individual banks, and put apremium on unsound banking in the future. (Quoted by Phillips, 1995, p. 38)

The solution adopted shortly thereafter appealed to an expertise found in the ratings agencies which could

tell the difference between speculative and prudent investments. The role of the ratings agencies in the

banking system was the subject of a blistering attack by Melchior Palyi in the University of Chicago‘s

Journal of Business in 1938. His attack serves as a central challenge. Palyi‘s stature, a public intellectual

who had been the chief economist of a great German bank in the days before the Hitler regime, who taught

the monetary theory course at Chicago in the days of Frank Knight, Jacob Viner and Henry Simons, was

such that his attack would demand a response. 9

Here is how Palyi opens his article:

The Banking Act of 1935 provided that the comptroller of the currency may prescribe, athis discretion, ―limitations and restrictions,‖ under which a national bank can purchase―investment securities‖ for its own account — a delegation of legislative power, the legality of whichhas not been questioned. On February 15, 1936, the comptroller promulgated the followingruling, effective as of the same date:

only one type of insured money? The experience in 2008 with ―run proof‖ money market funds occasions this query.

9 A consideration of Palyi‘s economic writings must begin with Warren Samuels (2005) from whom the importance of theattack on establishing the ratings agencies has been hidden and so does not catch the Weberian connection. Without thisinformation, the fact that Rose Director was both a Palyi student and an NBER staff member on the Corporate Bond

Project, does not leap out for comment. The Palyi connection, but not the Corporate Bond Project, is mentioned inFriedman and Friedman (1998). Palyi does not fit neatly into any of the Weber-less modern retellings of the Chicago story.Not surprisingly he is very sharp on assumptions of perfect knowledge. His published attack on Ernst John‘s Hayekiantheory of the business cycle tells us that Hayek is working in the tradition of Jevons‘s ―perfect market‖ in which allinformation is common, (Peart 1996, p. 98). Palyi (1935, p. 846): ―In one respect Dr. John‘s analysis is superior to thatof his masters. He realizes (p. 51) that his theory assumes complete knowledge of all changes of economic data by allpersons involved.‖ As one might expect from Palyi‘s missing the statistical point about how to think about the ratings‘experience, his response to Milton Friedman and Anna Schwartz‘s empirical work is rather less decisive. Nevertheless, hedoes raise the possibility that near-monies were more important than they suggested (Palyi 1972, xxiii, 324-25).

8/8/2019 Biased Reg


8

The purchase of investment securities in which the investment characteristics aredistinctly or predominantly speculative, or investment securities of a lower designatedstandard than those which are distinctly or predominantly speculative, is prohibited. Thepurchase of securities which are in default, either as to principal or interest, is alsoprohibited.

The terms employed herein may be found in recognized rating manuals, and wherethere is doubt as to the eligibility of a security for purchase, such eligibility must besupported by not less than two rating manuals.

Although power to determine eligibility was not explicitly delegated to the rating agencies,nevertheless this ruling of the controller does, in effect, place that responsibility upon them. (1938,p. 70)

Palyi‘s central challenge is to the non-transparency of the ratings which he believes inconsistent with public

authority. ―Extremely secretive‖ foundations of policy, which might in another context be called

―priestcraft,‖ is not a word of approbation in the Weberian vocabulary. 10 Students of Knight of that

period remember his doctrine – scientific knowledge is public knowledge – which is pure Weber.11

The article raises so many interesting and important issues that one is tempted to dally over the

elegance. Perhaps, in the fullness of time, it will be seen as being of comparable importance as Henry

Simons‘s attack on expert monetary discretion (Simons 1934). For example, Palyi makes the nice public

choice point that since Federal, state and municipal obligations have a rating at or above all private ratings,

this rule will guarantee an increased demand for their debt (1938, p. 72). One of Palyi‘s criticisms might

10 “Contrary to the practice of professional forecasting agencies, the rating manuals are extremely secretive about the techniqueby which they arrive at their judgment. The grades are published in annual manuals and occasionally changed in the courseof the year; more than one change of an issue‘s rating within the same year seldom occurs. Reasons for a change are rarely if ever given; the brevity and conciseness of the announcements create the impression of purely factual statements. In reality,each rating is, of course, an interpretation of an issue‘s intrinsic value … ― (Palyi 1938, p. 81). “It is therefore intelligible

that the rating manuals emphatically refuse to guarantee their statements; but it is incongruous that ratings which cannot beguaranteed as to accuracy of the underlying material are officially made the guide for bank investments — the basis for thesecurity of deposits.‖ (Palyi 1938, p. 85). This only serves to underline the importance of Laidler‘s argument that the greatcontribution of classical Chicago monetary theory was the seamless integration with liberal norms (Laidler 1993). Knight‘stranslation of Weber makes clear the importance of the city as encouraging non-secret knowledge in the progress of civilization. Weber ([1922] 1927, p. 317): ―the city alone produced theological thought, and on the other hand again, italone harbored thought untrammeled by priestcraft.‖

11 Appropriately enough, we learned of this through W. W. Cooper, a student of Knight somewhat distant from the newerChicago school.

8/8/2019 Biased Reg


9

not have been appreciated until our recent unhappy experience in the world in which highly-rated securities

are protected from mark to market discipline:

This unique phenomenon of a central banking policy which is guided by rating manuals,

and which in turn enforces their validity, requires further elucidation. This policy uses a furtherand highly questionable incentive consisting of the permission — admitted by the Reserve and stateauthorities — for the banks to carry their so-called high-grade holdings at par, or at amortized valueabove par, even if their market prices should fall below par, so long as their ratings do not declineunder the minimum requirements. Nor does any downward revision of ratings cause criticism of the bank by the examiner. As we shall see, the customer of the rating agencies can count to someextent on the latter‘s generosity; they will not let him down (for a while, at least). Consequently,the rule means a temporary guaranty for the banker against depreciation — i.e., against the logicalnecessity of providing proper reserves — and against forced sales, except in the case of a defaultinghigh-grade bond which has to be liquidated. And the writing-off as well as the liquidation whichhave to take place can be carried out in a gentle manner — as contrasted with the criticism andpressure to which the bank is exposed if its noneligible bonds are ―caught‖ in decline by theexaminer. (1938, p. 76)

But all of that is preliminary to what seems to have been viewed as his main attack; indeed, these

theoretical considerations could be dismissed as Weberian liberal philosophy. Here‘s the devasting

empirical point. Palyi checked to see how well Moody‘s did:

What, then, has been the past experience with the rating standards of the current manuals?No field study of this kind has been published, and the volume of material prohibits acomprehensive survey. The tables in the Appendix are based on random samples taken from major

railroad and public utility securities issued before 1930 and rated in 1929 as in one of the fourhigh grades. Three of the forty selected issues have maintained their grades for five years or longerwithout interruption; two succeeded in rising on the quality scale of the eight-year period. Allothers fluctuated in rating with the net outcome of a substantial decline. Not less than twenty-twoissues became ineligible, among them four which were rated Aaa in 1929 and three of which havedefaulted. There appear seven more subsequent defaults of eligible bonds of lower than Aaa ratingin 1929. (1938, p. 78)

Perhaps the most astonishing single result is that 70 per cent of the volume of railroad bondswhich defaulted in 1924 were rated Baa or better in the same year. Of course, 1924 was a recessionyear, hard on railway finance; but the interesting fact is that a minor recession is sufficient to cause

an enormous percentage of error in the rating agency‘s foresight. (1938, p. 79)

Moreover, it wasn‘t just Moody‘s. All the agency ratings performed just as miserably (Palyi 1938, p. 78

note 10).

8/8/2019 Biased Reg


10

The Composite

Palyi‘s statistical attack is not decisive. The regulation Palyi quoted called for the opinions of at

―least two‖ of the ratings manuals‘ concurrence. Palyi‘s ―pick one‖ method of testing, albeit offering a

chilling foretaste of the experience of the last decade, was inappropriate for that more suspicious era. This

is the step which the NBER statisticians filled in. They did not use a single agency rating as metric, they

used ―the median‖ as composite. That the ―composite‖ offers a response to Palyi‘s challenge is clear from

the private correspondence.12

We reproduce on the next pages the correspondence from Harold Fraine of the Securities and

Exchange Commission (SEC) who served as the Associate [Technical] Director of the CBP (NBER 1941

p. iv; Fraine and Brethouwer 1944, p. v) and Donald Thompson, Director of Research and Statistics of the

FDIC, in which Fraine worries about how to respond to Palyi‘s criticism in a way which is both honest and

prudent. 13 We quote a small part here:

The difficulty is that the article is very critical of the bank supervisory and rating agencies. I cannotfind it possible to say much about his article without appearing to put the memorandum of theCorporate Bond Study in the position of criticizing the cooperating agencies which wereresponsible for it.

Fraine thinks he has a way out

I have finally hit on the attached as a sufficiently innocuous recognition of hiscontribution. I am also attaching a copy of the latest draft …

The draft attached is enormously helpful as it shows both Fraine‘s original thoughts and what he

concluded was prudent to publish. The reference to the ―composite‖ in the preliminary draft was crossed

out. Here‘s how we read the first version of critical passage:14

12 Located in the Clark Warburton Papers, Box 108, folder 9.

13 Thompson‘s role is detailed in the letter from Leo Crowley to the NBER of May 28, 1938 commissioning the study. Thatletter is reproduced below on p. 12.

14 The original name for what became the ―Corporate Bond Project‖ was the ―Corporate Bond Study.‖ Thus, the critical folder9 of Box 108 in the Warburton Papers is labeled ―Corporate Bond Study‖ in the GMU finding aid.

8/8/2019 Biased Reg


11

The Corporate Bond Study has provided ―composite‖ agency ratings which for practical purposescan be used to indicate the eligibility for commercial bank investment of the bonds under the rulesin effect.

This was changed, after some strikeouts, to a sentence which omits any reference to the composite:

The Corporate Bond Study has provided comprehensive data to test the effectiveness of ratings asa guide to the eligibility for commercial bank investment of the bonds under the rules now ineffect.

In both versions the importance of the ―rules now in effect‖ is made obvious. This critical

omission of ―composite‖ for prudential reason Fraine explains, means that the reader of the published

Hickman study may well miss how the composite estimator is tailored to examine the properties of existing

regulations – ―the rules now in effect.‖ Here‘s how the published version reads:

The Corporate Bond Project provides comprehensive data designed to test the effectiveness of current supervisory rules which employ ratings as a guide to the eligibility of bonds for commercialbank investment. (Fraine and Brethouwer 1944, p. 5)

A reader unaware that the ―current supervisory rules‖ required a composite could not guess at that aspect

of the study.

How does the composite avoid Palyi‘s published criticism of individual ratings? Moreover, the

reader who does not find Melchior Palyi‘s name in the index in the published NBER volume on bond

quality (Hickman 1958) will not understand his role in the study.15 Nor, perhaps more importantly, will

the reader understand the composite responses to Palyi‘s public challenge.

Next we reproduce copies of Fraine‘s letter to Thompson with the corrected version of his draft

response.16

15 A search on the NBER web site for Palyi turns up several references to his work on the international gold standard butnothing on ratings.

16 It is perhaps useful to note that such correspondence does not get saved by accident.

8/8/2019 Biased Reg


12

8/8/2019 Biased Reg


8/8/2019 Biased Reg


14

We noted above that ―the median‖ of an even number of observations is a subtle concept. For

additional insight we consult the archival records of the FDIC response to the questionnaire from Senator

Wagner‘s Senate Banking Committee in 1939 (―National Monetary and Banking Policy‖) which asked

6. What use is made of the securities ratings of the recognized rating agencies in yourexamination and supervision of banks? By what means do you ascertain that the ratings are made ina manner appropriate to the use to which you put them?

A worksheet of the response is reproduced on the next page.17 We quote the critical case in which

all four agency ratings are available:

(d) A security rated by all four services will be designated of banking quality if rated within thefirst four grades by three of those services.

The banking regulations were therefore not guided by a conventional median in which one averages the

inner two observations; rather, it is the lower co-median. This is important to remember. The convention

by which the inner two observations are averaged supposes, implicitly, that there is no more reason to

worry about an error high as there is an error low. When we observe convention violated by such

knowledgeable statistical workers, our eyebrows ought to leap.

The manuscript is reproduced next.

17 Clark Warburton Archive Box 90, folder 9b.

8/8/2019 Biased Reg


15

8/8/2019 Biased Reg


16

In the large discussion of trust and the rating agencies, it seems to have been forgotten that for a

very long time the expertise employed reflected a ―composite‖ of expert judgment. A composite, save for a

trivial case, is not simply a judgment of a single agency. That changed in the 1970s when single experts

themselves became trusted. Expertise offered by rating agencies was not taken on trust by the authorities

responsible for the regulations issued by the Federal government in 1936.18

Expertise and Regulation

We return to the published record in 1936, reading the story through Hickman‘s guidance offered

in his 1958 Corporate Bond Quality , knowing that we need to read with more than customary care.

Hickman (1958, pp. 144-5) discusses the regulatory episode in detail. We simply expand his references.

We start with the announcement of a banking regulation which Palyi denounced. With the separation of

investment and commercial banks mandated by the Glass-Steagall Act of 1932, and the creation of the

FDIC to insure the deposits in commercial banks, the question of what assets are permissible for

commercial banks with insured deposits came to the fore.19

In this 1936 regulation, the Comptroller of the Currency, who also served as a director of the

FDIC, defined an ―investment security,‖ which a commercial bank could purchase for its own account.

Here‘s the passage from the Federal Reserve Bulletin with a controlling footnote emphasized.

Although the bank is permitted to purchase ―investment securities‖ for its own account forpurposes of investment under the provisions of R. S. 5136 and this regulation, the bank is notpermitted otherwise to participate as a principal in the marketing of securities.

1 The statutory limitation on the amount of the investment securities of any one obligor ormaker which may be held by the bank is to be determined on the basis of the par or face valueof the securities, and not on their market value.

18 White (2002, p. 44) emphasizes ―the presence of a growing regulatory demand for rating services …‖ without emphasizingthe critical role of the Comptroller‘s regulation of 1936, which takes center stage in White (2010, p. 213).

19 The controversy over Federal deposit insurance is studied in Phillips (1995). Speculation is discussed in Phillips without anyconsideration of how the reforms proposed to distinguish imprudent speculation from prudent investment.

8/8/2019 Biased Reg


17

2 The purchase of ―investment securities‖ in which the investment characteristics are distinctlyor predominantly speculative, or ―investment securities‖ of a lower designated standard than

those which are distinctly or predominantly speculative, is prohibited .1

The purchase of

securities which are in default, either as to principal or interest, is also prohibited. (March1936, p. 195)

We quote the footnote in which ―recognized rating manuals‖ appears:

1The terms employed herein may be found in recognized rating manuals, and where there is doubtas to the eligibility of a security for purchase, such eligibility must be supported by notless than two rating manuals. (March 1936, p. 195)

Thus, the minimum requirement for ―investment security‖ for a commercial bank holding is that in the

case of all four agencies offering an opinion, the upper co-median of the agency rankings is ―investment

grade.‖ If there are only three agencies, the median rating needs to be investment grade. With two the

minimum is investment grade.20

As Hickman informs the reader, the footnote vanishes in the 1938 regulations published in the

Federal Reserve System. He points the reader to an article in an issue of the 1938 American Banker,

subtitled ―Ratings Services only Advisory‖ that explains why. The banks evidently found the constraints

onerous:

―Inquiry has been made as to whether member banks are confined to the purchase of securitieswhich have a rating classification in one of the four groups according to rating services. The re-sponsibility for proper investment of bank funds, now, as in the past, rests with the directors of theinstitution, and there has been and is no intention on the part of this office to delegate thisresponsibility to the rating- services, or in any-way to intimate that this responsibility may beconsidered as having been fully performed by the mere ascertaining that a particular security fallswithin a particular rating classification.‖ (Volume CI, Number 183, p. 2)

The traditional moral terminology, which speaks of ―strict‖ and ―loose,‖ is in evidence in the American

Banker ‘ s report:

The Comptroller‘s office has had very few complaints or requests for change: from thebanks themselves, it is learned On the other hand, broker interests have now and then, sought foran easement of the regulation. It has occasionally happened that broker-minded banker; have

20 Even White (2010, p. 213) misses the critical footnote requiring ―not less than two rating manuals,‖ the attendantcontroversy and Hickman‘s report of the ongoing practice.

8/8/2019 Biased Reg


18

desired a loosening of the strictness of the regulation. It is pointed out; however, that it is thedesire of the Comptroller that the purpose and spirit of the law be maintained; that banksthemselves be protected from possible unwise purchase and that they be not subject to pressurefrom brokerage interests or speculative trends. (Volume CI, Number 183, p. 2)

Hickman reports from conversations with bank examiners that although the ―minimum of two‖ rule had

been formally removed as a matter of regulation, it was still enforced as a matter of practice. Indeed, the

1940 FDIC response to the Wagner Committee, which we reproduced above, insisted upon the lower co-

median, a somewhat stiffer requirement.21

Harold‘s Prudence

If we believe that the rating agencies offered ―useful advice,‖ as we do, we need to make clear that

―useful advice‖ has at least two distinct meanings which often diverge. First, useful advice can take the form

of the traditional moral concept of prudence. One is rightly advised that a ―junk bond‖ – a security which

is in fact defined in terms of its rating – is not a prudent investment. Second, useful advice can take the

form of the economic concept of maximizing well-being. One is rightly advised that a collection of ―junk

bonds‖ can be constructed in such a way that risk-adjusted income is maximized, one can buy the bonds

and insurance against their default with money left over.22 We wish to stress at the outset that prudent

behavior and maximizing behavior may part company when we are dealing with events not covered by

insurance markets. The maximizing account supposes that an act can be insured but that‘s not part of the

prudential account. Prudential behavior can be sensible in non-transparent settings. Old moral concepts

have an interesting way of coming back into the technical discussions. 23

21Hickman (1958, p. 146): ―Unofficial discussions with bank examiners indicate, however, that bonds rated in the first four

grades by two or more agencies are still generally conceded to meet the Comptroller‘s requirements.‖

22 Sylla (2002, p. 30) points out that Michael Milken‘s ―junk bond‖ venture began with a reading of Hickman‘s research report.

23 Here‘s a high point in the traditional account of prudence. ―Skill in the choice of means to one‘s own highest well-being canbe called prudence in the narrow sense. Thus the imperative which refers to the choice of means to one‘s own happiness(i.e., the precept of prudence) is still only hypothetical, and the action is not commanded absolutely but commanded onlyas means to another end in view.‖ (Kant [1785] 1997, p. 32). The uncertainty comes because it isn‘t obvious what makesus happy. ―If it were only easy to give a definite concept of happiness, the imperatives of prudence would perfectlycorrespond to those of skill and would likewise be analytical.‖ (Kant [1785], 1997, p. 34). When one deals with neo-

8/8/2019 Biased Reg


19

The rating agencies offer judgments in the form of estimates. From the history of discussion of

the properties of these estimates we can observe how the rating agencies were understood by impartial

observers. A central question to ask is whether the rating agencies were presumed to be offering unbiased

estimates? If not, what might be done to correct for the bias? Even if the experts are not to be trusted

separately, is there a way to remove their bias?

If we think of risk versus Knightian uncertainty in terms known versus unknown distributions,

when expert statistical advice (an estimate) is traded in some non-transparent fashion for things of value,

we create another sort of uncertainty. The sampling distribution of the estimates is now an unknown.24

Merely because we deal with Knightian uncertainty does not mean that we need proceed unaware. If we

know the source of the uncertainty and the direction of the bias, we can still be prudent.

Gilbert Harold‘s Bond Ratings. We start with Harold‘s Columbia dissertation published in 1938

as Bond Ratings as an Investment Guide: An Appraisal of their Effectiveness. Although Harold‘s book

continues to be cited as the first major academic study of bond ratings, a point which was made by

contemporary reviews, it has obviously not been read carefully in recent times.25 Had its message about

rating bias been absorbed, we should not have been as surprised by the events of the last decade as we were.

Harold‘s ―Preface‖ begins with two paragraphs of motivation in which he asks the good question:

Kantians such as Weber, it is perhaps useful to ask about necessary truth. Lemmon (1959) interprets ―analytical‖ as anecessary truth as satisfying S5 of the Lewis family with the distinctive thesis that what is possible is necessarily possible.Prudential behavior in the modern understanding might be connected to modal logic by way of probability withoutknowledge of the distribution. ―Wald shows under weak conditions that a minimax solution is a Bayes solution relative to aleast favorable initial distribution. Minimax solutions seem to assume that we are living in the worst of all possible worlds.Mr. R. B. Braithwaite suggests calling them ‗prudent‘ rather than ‗rational.‘‖ Good ([1952] 1983, p. 13). S5 has theinteresting implication that is a bad thing was possible, it is always possible. This might be why we find advice offered by

means of proverbs (Peart and Levy 2005).

24 We explore this expert-induced uncertainty in Levy and Peart 2007, 2008 and 2010.

25 Harold‘s study is cited by Partnoy (2002) reproducing the distribution of agency ratings (69) and noting ―a certain amount of rating ‗inflation‘ evident in each of the agency‘s scales.‖ Harold‘s finding that the skew disappeared when the minimum of the ratings was selected is not mentioned. Sylla (2002, pp. 25-30) carefully discusses the NBER Corporate Bond Projectbut does not consider how the NBER use of the downward-rounded median might adjust for the skew in the individualratings which Harold reported.

8/8/2019 Biased Reg


20

bond buyers might reasonably ask whether bond ratings have proven to be worth the bother. Dobond ratings, in other words, forecast the market? (1938, p. v).

The third paragraph notes that bond ratings have become part of the financial regulations, a point that

helps explain why the NBER Corporate Bond Project was funded:

Bankers especially are interested in the matter, for the Comptroller of the Currency on February15, 1936, issued a ruling (as further defined and interpreted in his rule of October 27, 1936)having a direct bearing on the use of bond ratings. It is common knowledge in bond circles sincethe issuance of the Comptroller‘s ruling, a bond rated below that of a ―business man‘s investment‖ … can almost never be sold to a bank. (1938, p. v).

The question of investigator impartiality is raised by Harold, we find this on the first page of his

―Preface‖:

Since the author has worked entirely as an individual, subsidized by no person, group of persons, corporations, or organization of any kind whatever, he alone is responsible for anystatement that may appear to be critical (1938, p. v-vi).

One critical distinction between Harold‘s work and that of the more famous Corporate Bond Project is

that Harold, without the involvement of the ratings agencies, was freed from any need to disguise the

properties of individual agency‘s estimates. We‘ll come back to that below.

There is another remarkable difference between Harold‘s work and the later NBER study. Harold

considers very early in his book the temptation which confronts the rating agencies to bias their ratings

upwards. This is not very subtle stuff. The private correspondence about not publishing material critical of

the rating agencies, gives us insight into how to read the published NBER volumes. Merely because there is

no discussion of rating bias does not tell us that the authors believed that was no rating bias to be found.

The published writing will be prudent. 26

26 This is how Fraine and Brethouwer summarize the literature. ―Melchior Palyi had expressed concern early in 1938 that therewere no adequate field studies of past experience with the rating standards of the current investment manuals. He raised anumber of questions regarding the use of ratings in determining the eligibility of bonds for investment by commercialbanks. Gilbert Harold‘s research on bond ratings reached the public in the same year. His tests of the accuracy of bondratings, however, consisted mainly of comparisons of number of defaults and changes in market values.‖ (Fraine andBrethouwer 1944, p. 3). Palyi is not mentioned in Hickman (1958). Harold‘s work is cited twice (1958, 144 & 190) incontexts in which one can recover neither Harold‘s concern about bias nor his worst-case estimation

8/8/2019 Biased Reg


21

Rating bias, temptation from rating shopping and negotiation is now at the center of modern

discussion.27 In Harold‘s account it was always there. Nonetheless, it will have a very subtle statistical

consequence that Harold will develop. The temptation the ratings agencies encounter is not symmetrical.28

Bribes are not offered to lower ratings:

The development of ratings in the securities field, especially in bonds, was not greatlydissimilar. Corporations were opposed to it, and many of them still are, especially among thosewhose issues are assigned low or mediocre ratings. Some indeed exert their forces to the point of protesting, through the person of an officer or in writing, to the rating agency. A typical statementby corporate officers is summed up in the challenge, ―Send your man around, and we ‘ll show him afew things that will cause you to raise your rating.‖ This attitude is typical especially of somecorporations which, directly or indirectly, maintain their own securities distributing organizations.Some corporations indeed have gone so far as to assure the rating agency that if a given rating wereraised the corporation or its sponsors ―could do something‖ for the rating agency. On the whole,however, corporations in general have now come to accept security ratings as an inevitable andpermanent part of our financial system. (1938, p. 16)

The center of the temptation to deviate high is provided by investment bankers. We quote the two critical

paragraphs:

Attitude of Investment Bankers.—In no circles has the attitude toward bond ratings beenmore hostile than among the investment bankers. Such hostility is, of course, easily understood.For not only does the existence of ratings tend to narrow the price spread between trading points,but it also affects the resale of bonds that have come back to their original sponsors, as well as the

original sale of new issues of the same corporation. If the rating is high, sales are easily effected; if low, sales are made only with great difficulty. When the rating is high it is often mentioned in listsof bonds for sale, issued to customers by the bond house. When the rating is low, it is rarely somentioned.

No complaint is ever made by a bond house that an issue which it is sponsoring is ratedtoo high. Other bond houses, however, owing to the pressure of competition, have been known tomake such complaints. So vital are the ratings to investment bankers that extreme steps have beentaken in some cases to influence those ratings. It is rumored in Wall Street that one agencyunknowingly had on its staff a man who was also on the payroll of one of the investment bankinghouses, but that to the credit of the agency concerned the man was dismissed as soon as the agencydiscovered the connection. In many cases, investment brokers call personally on the rating agencies

to complain of ratings which they feel to be too low. One firm with offices throughout the countryis reported to have threatened cancellation of all its subscriptions to Standard‘s service because the

27 White (2010, pp. 220-21) gives a very useful sketch of the dimension of the problem. Hill (2010) gives an excited accountwhich supposes that bond ratings were trusted in the good old days.

28 The problem of an asymmetric temptation to deviate from a norm is rather more general than the text suggests, e.g., Levy,Padgitt, Peart, Houser,& Xaio, forthcoming, consider the temptation to deviate from the advice of a human leader. Thissuggests that the usual assumption of symmetrical error is perilous in cases of temptation.

8/8/2019 Biased Reg


22

firm‘s salesmen had great difficulty in selling certain bonds owing to Standard‘s rating. Anotherfirm is reported to have offered options on stock to Moody in return for a more favorable rating.It is not to be inferred that such overtures are typical of the investment banking group; rather theyare the exceptions, but they indicate the importance attached to the rating system by some bondhouses. (1938, pp. 16-17)

Pausing for context. Harold, as we noted is working without formal co-operation from the rating

agencies; thus, he found it possible to discuss the agencies on an individual basis. One thing he discovered

is that the distribution of ranks does not look the same across the agencies. Moreover, the distributions

appear far from symmetric in the chart he prints (1938, p. 90):

Harold doesn‘t conduct a formal test of the hypothesis as to whether the distributions are the same or not

but the suggestion conveyed by the table is that there is no reason to believe in a common distribution.

Fitch certainly appears to give more A+s than do the others.

Harold‘s two-fold conclusion is that, on average, the ratings are a good prediction but there is a

terrible problem that the average hides:

8/8/2019 Biased Reg


23

Recapitulation — Considering averages only, there can be little doubt that both the recordof market action … and the yield record of bonds by ratings indicate that the system of ratings isan effective guide to investment values. Although an average is an important summary measure of the performance or standing of any group of economic data, it may represent the movement of individual items comprising the average rather inadequately. It is small comfort to the investor who

has been led to believe that any bond of a specific rating is even the approximate equivalent of anyother bond of the same rating when averages are considered, if he happens to own a security themarket behavior of which departs radically from the average. Taking account of such variationsfrom averages, the findings of this chapter cast considerable doubt upon the validity of ratingagencies‘ claims that the ratings have been ―perfected,‖ that they are ―scientific,‖ that they are―conservative,‖ that they are ―accurate,‖ or that they ―do not rest on opinion.‖ (1938, pp. 126-27)

What to do? Here Harold does something marvelous. This was noted at the time in the

professional reviews but it seems to have passed out of working memory.29 A decade before Abraham

Wald‘s minimax (Good [1952] 1983) and two decades before John Tukey‘s robust revolution (Tukey

1962), Harold considers the possibility of an estimator which attempts to avoid disaster. We quote at

length:

The Pessimistic Theory. — One of these would be to adopt as the proper rating of a bondthat rating which is the lowest assigned by any agency. In other words, one would accept the most―pessimistic‖ rating. Assuming that the investor harbors the ―calamity point of view‖— and mostbond rating users do — this would seem to be a reasonable procedure. It rests upon the assumptionthat four ―minds‖ are better than one, and on this basis the feasibility of using the most con-

servative appraisal of all the rating agencies to greater advantage than could be done for any oneagency appears both reasonable and attractive. Indeed, it appears entirely possible that the greatestprotection would be obtained by accepting only the lowest rating assigned. On this plan, then, theinvestor would accept as the proper rating for a bond that rating, of all four organizations, whichwas the lowest assigned. Thus a bond rated Α-, A-, A, and A by the respective agencies would beregarded by the investor as an A- bond. (1938, p. 150)

Although this rule is not linked in the texts to the temptation of the agencies to bias their ratings upwards,

we imagine that it would have been a particularly inattentive reader not to make that connection.

The term ―composite-rating‖ being popular among specialists, Harold considers an averaging

procedure but concludes in a heuristic fashion that it doesn‘t offer protection against disaster so he forms

29 Burtchett (1939, p. 223): ―By using a ‗pessimistic‖ rating, i.e.. the lowest rating assigned by the several rating agencies, a moredependable guide to investment can be obtained than by using any individual agency rating.‖

8/8/2019 Biased Reg


24

his composite on the basis of the minimum of the four ratings. He reports his resulting table.

The distribution of the minimum of the four ratings is a good deal more symmetric than the ratings

themselves.

One of the points of the pessimistic rating is to avoid default. Harold gives summaries but this

will suffice for the flavor of his results:

In 1932, defaults appeared as high as A- for three agencies, but only as high as B+ on thepessimistic-rating basis. In 1933, on the pessimistic rating basis, only 1% of A bonds were indefault, while the default ratios for the several agencies were: Fitch, 11% ; Moody, 3% ; Poor, 3%;Standard, 9%. (1938, p. 159).

Of course, disasters of the sort of 1933 don‘t happen frequently. The point though is that they do happen,

which is one of the reasons for working with uncertainty instead of risk.

The Harold Estimator and Motivated Estimation. Harold‘s procedure allowed him to report the

distribution of ratings by agency, something seemingly precluded by the NBER agreements since, as we

8/8/2019 Biased Reg


25

have seen, even the worksheets printed as depository documentation have the individual ratings suppressed!

If we take expert analysis as simply exogenous to the economy, we fail to appreciate that even the great

research of the NBER Corporate Bond Project was conducted as a part of trade that precluded the

publication of some information.

This simple point suggests that there is a great deal which might be learned had we been able to

ask the investigators whether the Harold minimum or the lower co-median had been computed and

compared with their downward rounded median.30 Save for discovering the worksheets themselves, which

would reveal the distributional characteristics of the individual rating agencies, how might we come to

understand these distributions?31 Harold tells us that bribery is not a problem. Attempts were made and

repulsed. He gives no hint of collusion among agencies.

We suppose, reading more into his words than he may have intended, if there is a bias, it comes

from something subtle, a sympathetic connection between experts and clients. The problem is not that the

experts are crooks, but rather they are sympathetic agents whose estimates can be bent by affection for

client. If we suppose that whatever bias is produced, the rating is subject to an audit constraint, then we

would not be looking at simple falsification or fabrication, rather something akin to the sort of estimation

shopping which we have explored elsewhere.32 Rating shopping is not a criminal act; thus, we do not

suppose that our raters do anything which could impose great personal costs upon themselves.

30 Although Rose Director Friedman‘s death has ended the last hope of asking the principals directly (she was perhaps theyoungest NBER staff member on the CBP in the early 1940s and surely the only one who had both studied with andworked for Palyi), perhaps the memory has been preserved. Was it her comments that emphasized the importance of acknowledging Palyi‘s criticism? We know from archival records, how deeply she impressed Clark Warburton when sheinterviewed for a job at the FDIC. ―She is unusually keen …‖ Memo from Warburton to Thompson, December 14, 1936.

Box 12 of the Clark Warburton Papers.

31 One of the great steps forward in the robust revolution was the realization that a mixture of normal distributions could serveas paradigm for an unknown heavy-tailed distribution, Andrews, Hampel, Huber, Rogers & Tukey (1972). This realizationtended to attenuate the controversial use of distributions without moments as exemplars, distributions at which limittheorems and laws of large numbers fail.

32 Models of biased estimation resulting from sympathetic agents are offered in Levy and Peart 2007, 2008, 2010; a model of biased estimation by crooks and jerks is put forward in Feigenbaum and Levy 1996.

8/8/2019 Biased Reg


26

As a simple model of estimation shopping, consider four experts who draw samples from a

standard symmetric heavy-tailed distribution and compute a sensible estimate of location. We could, of

course, allow them each to propose their own favorite estimator, e.g., one from the Princeton Robustness

Study, but for simplicity, let each of them compute the sample median and then wait for the complaints.

So, all other things being equal, the estimates of the four will be the same. Now, we introduce trade of a

sort. If the client asks nicely the agency expert will also compute the wildly inappropriate sample mean and

make the reported estimate the higher of the sample median or mean. This, of course, does not guarantee

that the agency judgment will be changed. Sympathy is not bribery.

Suppose the agencies differ how sympathetic they are, how frequently each will pick the maximum

of the two estimates. Rater 1 will pick the maximum 50% of the time, Rater 2 will do so 30% of the time

while Raters 3 and 4 only 15% of the time.33 This percentage is not the frequency with which the sample

mean is reported, but rather the frequency with which the larger of the sample median or mean is allowed.

How does the Harold estimator perform in such a context?34 As a very modest contribution to a

full-dress model of the economy in which a banking sector‘s stability depended upon the trustworthiness of

the security ratings, we consider how one might make reasonable adjustments if one were aware of the

incentives to bias the ratings upward. We create four rating distributions with sample size 100k and then

compute Harold‘s minimum. This is of interest for its own sake, especially in contexts in which it is

plausible to assume the ratings exogenous – they were drawn from pre-existing manuals – and will serve as

a benchmark when there are endogeneity issues. We‘ll have another estimator to report below.

33 Having Raters 3 and 4 the same allows a check on the computations. Is the simulation big enough? 100K suffices.

34 We used Shazam 10 (SP1) (White 2009) for the computations. The underlying distribution is a mixture of two normals, thecelebrated Tukey distribution. The Tukey is a standard normal with probability 90% and a normal with mean 0 andstandard deviation 10 with probability 10%. Each sample size is 100. We did 100K replications which gives sufficientprecision that the values of R3 and R4 are the same to reporting precision. The few lines of code needed are in theappendix. We exploit Shazam‘s delightful ability to mix syntax and semantics to create and operate on the variables so R%when the loop counter % increments from 1 to 4 creates R1 to R4. The OLS commands are simply to generate additionalsummary statistics which characterize higher order moments. The graph seem to us to suffice, but the large output isavailable upon request.

8/8/2019 Biased Reg


27

What is interesting, we believe, is the skew of the four agencies that Harold reports and the

deskewing that the minimum produces. Using his procedure with our generating process we find the bias

in the individual ratings removed in the use of the minimum. Since the experiment supposes that the true

underlying parameter is 0, the bias is simply the sample mean of the 100k.

Rating Bias Standard Deviation

R1 0.059 0.19

R2 0. 035 0.18

R3 0.018 0.16

R4 0.018 0.16

RMin 0.001 0.15

In lieu of reporting more detailed statistics to capture violations of symmetry we simply sorted the most

biased rating (R1) and the Harold minimum (RMin) from smallest to largest and plot the resulting

graphs. We can see at a glance how Harold‘s RMin makes the shape symmetrical.

8/8/2019 Biased Reg


28

Harold‘s minimum is theoretically unmotivated. Perhaps this ought not to be a surprise. Practical

workers have to take the problems as they find them. The robust revolution grew out of Tukey‘s two-fold

realization that statistical practice in dealing with ―outliers‖ was far ahead of theory and simply assuming

them away was dangerous.35 If the underlying data sheets of the NBER Corporate Bond Project survived

then we would have an occasion to examine the distribution of ratings by agencies. We might test the

Harold minimum against the NBER downward-rounded median, the lower co-median and other

estimators that would come to mind.

35 ―The treatment of ‗spotty data‘ is an ancient problem, and one about which there has been much discussion, but thedevelopment of organized techniques of dealing with ‗outliers,‘ ‗wild shots,‘ ‗blunders,‘ or ‗large deviations‘ has lagged farbehind both the needs, and the possibilities presently open to us for meeting these needs.‖ Tukey (1962, p. 394) ―When Ifirst met Charles P. Winsor in 1941 he had already developed a clear and individual philosophy about the proper treatmentof apparent ‗wild shots.‘ When he found an ‗outlier‘ in a sample he did not simply reject it. Rather he changed its value,replacing its original value by the nearest value of an observation not seriously suspect. His philosophy for doing this, whichapplied to ‗wild shots,‘ can be supplement by a philosophy appropriate to long-tailed distributions which leads to the sameactions.‖ Tukey (1962, p. 413)

8/8/2019 Biased Reg


29

The Corporate Bond Project of the NBER

Palyi‘s and Harold‘s concerns about the use of individual rating agencies‘ judgment were published

in 1938. Even before the public challenges, the topic of rating experience was being discussed by the

technical staff of the regulating agencies. Clark Warburton himself passed on one such discussion to the

chief of research at the FDIC in October 1937.36

In addition to a 1941 History and Description of the Corporate Bond Project, there is also a

history provided in the first part of the 1944 Organization and Administration of the Corporate Bond

Project by Fraine and Brethouwer. We quote in full confidence that these words will come as a surprise to

those who believe that the Federal authorities established the rating agencies as a Church of the United

States of Wall Street without due care. We quote at length from the ―Forward‖ to give some feel for the

magnitude of the study. A name will be mentioned to which we need to pay attention:

Early in the 1930‘s the problem of bank investment standards and procedures became amatter of concern to government supervisory agencies who were in need of information, not thenavailable, on the development and behavior of the investment market over a long period of time.Several of these agencies therefore projected plans to obtain such information. Informaldiscussions took place among representatives of public agencies, private investors‘ services, and theCentral Research Staff of the National Bureau ‘s Financial Research Program, and led to the

suggestion that there should be a broad investigation of the market behavior of corporate bonds.

It soon became evident, however, that such an investigation could not be carried on orfinanced by a single agency. In May 1938, Chairman Crowley of the Federal Deposit InsuranceCorporation suggested to the National Bureau that since this study would deal with subjects of broad implications and develop materials of great importance to the study of the economic andfinancial system generally, it could best be conducted as an independent scientific inquiry under thesupervisory auspices of the National Bureau‘s Program of Research in Finance. Chairman Crowleyalso stated that the Federal Deposit Insurance Corporation was prepared to sponsor a WorkProjects Administration project in New York City to provide the necessary clerical help, andwould cooperate in assembling the data, if the National Bureau would provide technical

36 Memo to Mr. Thompson from Clark Warburton, October 26, 1937 in Box 12 of the Clark Warburton Papers. ―Yesterday Isaw Mr. Harold Roelse, and inquired whether the Federal Reserve Bank might have work which could be done as a WPAproject after our work is done here. He said that was a possibility, and that he would talk with other persons, particularlythe people in the examining division. The problem in which he is interested that of bank holding of securities, particularlythe behavior of unlisted securities. He says that Moody‘s has recently undertaken a study of all of the bonds which theyrated in 1929, to see what has happened to them, whether paid on maturity, etc., as a sort of test of the goodness of theirratings. Mr. Roelse thinks a somewhat similar type of study might be made of unrated securities held by banks. Thisproblem is of so much interest to the Corporation that I wonder if some plan of cooperation on the project might beworked out.‖

8/8/2019 Biased Reg


30

supervision and work out arrangements for other government agencies and private financialadvisory services to participate in the study. (Fraine and Brethouwer 1944, p. i)

We pause for a moment to note that Leo Crowley was in his era a political figure of considerable

stature. In our time he has been the subject of a full-length biography (Weiss 1996) which sheds much

light on his ―colorful‖ career but none whatsoever of the mechanics of the FDIC which under his direction

put into practice prudent methods.37 We are told the staff of unnamed economists at the FDIC was

particularly able. They were excited by the ability to interact with economists of the stature of Jacob

Viner.38

A copy of Crowley‘s commissioning letter survives in the Warburton Papers.39 This sheds light on

both why the project was funded and why the NBER was selected. There is nothing in the public record

which reports Crowley‘s concern about conducting the research inside the FDIC proper ―For reasons

which the National Bureau can appreciate ….‖ By funding it via the NBER a nominal independence was

established. The project is funded to check out FDIC policy and, putting our interpretation on the silence,

it would be unseemly for the FDIC to make the requisite trades with the rating agencies for their co-

operation.

The letter is reproduced next:

37 Surely, the fact that the Crowley correspondence is at George Mason University Library Special Collections in the ClarkWarburton Papers and not in the National Archives has something to do with that gap. This archival history suggests howpoorly understood the rating establishment has been. As a result of our discovery, the George Mason finding aid nowreports the Crowley correspondence so other scholars can find it by the standard search methods.

38 Weiss (1996, p. 74). Viner tells us this about Crowley in 1953: ―Charges were made against Leo Crowley on the ground of the record of his family, himself, and his brothers, with respect to a bank that under the pressure of the depression was introuble. The charge was made by another high officer of the government. The matter was brought to the President. ThePresident set up a little ad hoc committee to investigate it, of which Morgenthau was a member, I think. They found that

instead of Leo Crowley‘s record being doubtful in any way, he was the man who had cleaned up the trouble. His brotherswere involved. To clean up the family name, he had thrown in everything he had, had worked and rescued the bank.Crowley was a very devout Catholic. The Crowley family in Wisconsin were very prominent in Catholic circles. Leothought he had a religious obligation to clear a well-known, notorious if you like. Catholic family from this blot on them.Apparently, no debtor lost a dollar in the bank. That‘s how I recall the story. I had part of it from Morgenthau and part of it from Crowley. I haven‘t mentioned the name of the man who made the attack, because, while it might be interesting, Idon‘t see any point in it.‖ Weiss‘s chapters ―Cover-Up in the Capital I‖ and ―Cover-Up in the Capital II‖ (Weiss 1996,pp. 33-64) are less charitable.

39 Warburton Papers, Box 109 folder 10.

8/8/2019 Biased Reg


31

8/8/2019 Biased Reg


32

8/8/2019 Biased Reg


33

We return to the public record and continue the quotation:

The National Bureau then requested and obtained the cooperation of the Board of Governors of the Federal Reserve System, the Comptroller of the Currency, the Federal ReserveBank of New York, the Securities and Exchange Commission, and the four leading privatefinancial advisory services. Over ensuing months numerous consultations took place to lay out

general plans, and in November 1938 an application was submitted by the Federal DepositInsurance Corporation to the Work Projects Administration for funds to finance the compilationof investment data from financial publications. The application was approved on December 30,1938 and the Corporate Bond Project commenced active operations on January 17, 1939.Activities extended to November 30, 1941 when the major objectives pertaining to thecompilation and organization of data had been accomplished (Fraine and Brethouwer 1944, i.)40

The FDIC, the Fed, the Federal Reserve Bank of New York, the Comptroller of the Currency, the

four ratings agencies (Moody‘s, Standard Statistics, Poor‘s and Fitch) and a WPA project to finance the

employment of clerical help with the NBER directing the project. One stares at the names in financial

regulation and asks: who is missing from this party? Someone more knowledgeable than us about the

40 The published volumes from the Corporate Bond Project are available on the NBER web site in PDF form for downloading.Searching ―Corporate Bond Project‖ will turn them all up and a good deal more. The volumes containing mimeographeddocumentation and preliminary results are available at the Library of Congress. A search on ―Corporate Bond Project‖ willturn them up.

8/8/2019 Biased Reg


34

regulatory community of the 1930s will have to answer that question. In view of Crowley‘s letter this will

be of little surprise. This is an FDIC project with an NBER veneer which speaks to the center of the

monetary order of the US. Resources, basically, are of no object. Had not the term been preempted, one

might well describe this as the Federal government‘s Manhattan Project.

In 1941 History gives a list of questions to address. One notes the questions about the average

rating performance

The second type of study which can be developed from the Record of Issue andExtinguishment Characteristics will be concerned with testing the various criteria of bond worthwhich have been developed over the past four decades. Among the studies falling in this category,that of the effect of legal restrictions clearly comes first. How effective have the legal lists reallybeen? Have they acted as a protection against loss, and how great is the protection they haveafforded? What about the average rating performance of bond rating agencies? Has this averagerating differed considerably from the ratings established by the market itself in the yields at whichit has valued bonds? When they have differed, have they been better on the average than themarket? Have the ratings of the rating agencies been more stable than the ranking given to bondsby the market, or have they also been affected by the optimism and depression that characterizesthe market? Finally, how good has the market been as a rating device … (NBER 1941, p. 14)

By 1944 the questions have sharpened considerably. Here‘s the first seven of the questions to which the

study was addressed. Of these questions, four ask explicitly about the rating agencies ranking. Expert

opinion of rating agencies versus market rating is the precise focus of question six.

1. Has the cost rate of financing, as reflected by the prospective yield of bonds, varied fordifferent industries?

2. To what extent have large companies been able to borrow at lower cost than smallcompanies? Has this differential been associated with better earnings status of thelarge companies?

3. To what extent has the increased income offered by the higher yield bonds beensufficient to offset the greater losses?

4. What has been the relative performance of bonds in different agency rating groups, asmeasured by the number of bonds defaulting in each group, the amount of loss(difference between expected and realized yield), and the realized yield to the investor?

5. How effective have legal lists been as indicators of bond quality? Have legal bonds hada lower realized yield than nonlegal bonds of the same agency and market ratinggroups?

8/8/2019 Biased Reg


35

6. Has market rating at time of offering (i.e., the difference between the prospective yieldon a given bond and that on the highest grade corporate bonds outstanding at thattime) been a good indication of future performance?

7.

To what extent have the rating agencies differed with the market appraising relativebond quality at the time of offering? Where [they] have differed, have the agenciesrated more accurately than the market? (Hitchens 1944, p. 30).

Harold‘s dissertation reported on a sample of 363 bonds. The NBER CBP with the resources of the

Federal government at its disposal collected everything in the universe of interest and 10% of the

uninteresting. When we read Hickman‘s widely cited report of the CBP results, we should therefore reflect

deeply upon how the NBER composite of a downward rounded median mimicked the Comptroller‘s rule

of a ―minimum of two‖ investment grade ratings.

The field of investigation is the universe of straight corporate bonds offered during1900 — 1943, including those outstanding on January 1, 1900. Straight corporate bonds aredefined as fixed income, single-maturity bonds offered by domestic business corporations and heldby the domestic investing public. The study covers all large straight issues (those with totalofferings of $5 million or more) of railroad, public utility, and industrial corporations, and arepresentative 10 percent sample of small straight issues (under $5 million). Excluded are realestate mortgage bonds (principally issues secured by office buildings and residential property) andbonds of financial corporations. (Hickman 1958, p. 5).

Here‘s the experience of the period, the Great Depression and all as Hickman would report:

8/8/2019 Biased Reg


36

Two things should leap out. First, high-rated bonds default a great deal less frequently than low

rated so they are indeed prudent investments which indeed deliver more than they promise.41 Second, the

realized return from low-rated bonds is very high although not as high as promised. Perhaps the finding

which attracted the most attention was the performance of ―junk bonds.‖

What perhaps will not leap out is the question of how the composite ―Agency rating‖ is

computed. This is carefully explained later in Hickman‘s text.

When only one rating could be obtained for an issue, the coded value of that rating was used as thecomposite rating. If two ratings were available, the composite is the arithmetic mean of the codedvalues of the two, rounded downward in the event of a fractional value to the next lower rating (i.e.grade ii is the composite rating assigned an issue rated Aaa by Moody‘s and Al by Standard). Forthree ratings, the composite is the middle value of the array of coded ratings; for four values, it isthe arithmetic mean of the middle two (rounded downward in the event of a split rating).‖ Hickman (1958, p. 143) [emphasis added]

We‘ve seen this before. This is a subtle variation on how the FDIC responded to the Wagner

Committee‘s questionnaire. As Harold‘s use of the minimum as a prudent estimator has fallen out of

memory, the fact that the NBER use of a median, which is rounded down when there are an even number

of observations, has not raised the appropriate statistical eyebrows.42

As part of the simulation reported above, we also computed what we called the ―NBER‖ estimator

which is the lower co-median, the second smallest value of the four. Our simulation does not take into

account the subtle integer characterization of the ratings. In comparison to Harold‘s procedure we find:

Rating Composite Bias Standard Error

RMin 0.001 0.14

NBER 0.007 0.14

41 ―Even the highest grades of corporate bonds, it is shown, were not entirely free of the risk of default; but virtually all of theprospective measures of quality provided reliable rankings in regard to such risk. In other words, the retrospective quality of bond offerings as measured by default rates declines as we move down the scale of each of the major prospective measuresof quality.‖ Hickman (1958, p. 12).

42When Sylla reports the results of Hickman‘ study, he writes of ―a composite average‖ (Sylla 2002, p. 26) and so misses the useof a downward rounded median as a correction for bias. As we noted above, Harold rejected the average as composite.

8/8/2019 Biased Reg


37

The graphs of the two composites show how they differ. The difference suggests how interesting

it would be to have the real data to work with.

Conclusion

The rating agencies, now at the center of such scandal, were a critical piece of the winning

alternative to the Chicago Banking Plan. The rating agencies were called upon to certify what bonds were

prudent for the federally-insured commercial banks to hold. The temptation that comes to bias expert

certification was very well understood at the time and sophisticated steps were taken to attenuate the bias.

Nonetheless, over time the constraints on expert pursuit of self-interest went slack.

We did not get into the problem with rating agencies through carelessness on the part of the

authorities at the time of their establishment. What needs to be emphasized is that the first two great

statistical studies of the rating agencies employed estimation procedures that reduced the influence of any

8/8/2019 Biased Reg


38

single optimistic rating. Harold‘s minimum cheerfully traded accuracy for protection from calamity.

Harold‘s minimum would be unattractive for regulatory purpose since it offers to a single agency an

unchecked blackmail ability.43 The FDIC-financed NBER downward-adjusted median also attenuated the

influence of any single optimistic rating without creating the blackmail temptation. Our variation, the

lower co-median, suggests how close it comes to the Harold minimum.

This history suggests a problem with expert guidance. It is not sufficient to start with a deep

understanding of the temptations that expertise offers, one must create an institution in which such

temptations are constrained. The institution which failed was, in our opinion, professional memory. We

forgot that the trades which financed the research precluded public speaking about the need to avoid bias

which follows from non-transparency.

We do not write in judgment to find fault with the working statisticians and economists who

found a way to deal with expert bias. The monetary system of the country was in peril. They created

documentation which told us, if we are interpreting it correctly, that we need to read the study very careful.

The correspondence in which they discuss how their constraints prevented candor was preserved for an

archival record. But we didn‘t read carefully. And so their work and their caution fell out of memory.

The fault we find is with the modern conceit that we can do economic theory without memory.

This suggests a need to rethink the foundations of theory of markets, to reconsider why we did not follow

the path that W. S. Jevons suggested, to identify perfect markets with perfect knowledge (Peart 1996).

What happens to markets when memory fails? Without memory we might see what was done but we‘ve

lost the ability to understand why this was done. We need intentionality to give us the meaning of the

estimates were observed. This is a reason why we might think again about reforms that do not suppose

that policy makers are able to foresee and pre-empt the ways which experts might exploit their privileged

position in the rule of discretion.

43 Hugh Rockoff pointed this out in correspondence.

8/8/2019 Biased Reg


39

Appendix: Simulation Code

dim r1(100000),r2(100000),r3(100000),r4(100000),rmin(100000),nber(100000),rbond(4)set nodoechoset maxcolon=100000do #=1,100000

smpl 1 100genr x=nor(1)genr p=uni(1)if (p.lt.0.10) x=nor(10)?stat x/mean=smean median=smediangen1 smax=max(smean,smedian)gen1 bribe1=.5gen1 bribe2=.30gen1 bribe3=.15gen1 bribe4=.15?do %=1,4gen1 r%=smediangen1 temp%=uni(1)

if (temp%.lt.bribe%) r%=smaxgen1 rbond:%=r%?endo %gen1 rmin=min(r1,r2)gen1 rmin=min(rmin,r3)gen1 rmin=min(rmin,r4)gen1 r1:#=r1gen1 r2:#=r2gen1 r3:#=r3gen1 r4:#=r4gen1 rmin:#=rminsmpl 1 4?sort rbond

gen1 nber:#=rbond:2endo # smpl 1 100000sort r1sort r2sort r3sort r4sort rminsort nbergenr c=time(0)stat r1 r2 r3 r4 rmin nber/maxgraph r1 rmin c /lineonlygraph nber rmin c /lineonlyols r1/maxols r2/maxols r3/maxols r4/maxols rmin/maxols nber /max

8/8/2019 Biased Reg


40

Bibliography

Archives

Clark Warburton Papers, 1920-1979. Special Collections and Archives, George Mason University

Libraries. http://sca.gmu.edu/finding_aids/warburton.html

Arthur Burns Papers. Dwight Eisenhower Presidential Library

http://www.eisenhower.archives.gov/Research/Finding_Aids/PDFs/Burns_Arthur_Papers.pdf

Periodicals

American Banker

Federal Reserve Bulletin

Secondary Sources

Andrews, D. F., F. R. Hampel, P. J. Huber, W. H. Rogers and J. W. Tukey. 1972. Robust Estimates of

Location. Princeton: Princeton University Press.

Burtchett, Floyd F. 1939. ―Review of Gilbert Harold Bond Ratings as an Investment Guide. ―Annals of

the American Academy of Political and Social Science 202: 222-224.

Emmett, Ross. 2006. ―Max Weber, Frank Knight, Chicago Economics and Institutionalism.‖ Max Weber

Studies Bhft I: 110-119.

Feigenbaum, Susan and David M. Levy. 1996. ―The Technological Obsolescence of Scientific Fraud.‖

Rationality and Society 8: 261-276.

Fiorito, Luca and Sebastiano Nerozzi. 2009. ― Jacob Viner‘s Reminiscences from the New Deal.‖

(February 11, 1953) Research in the History of Economic Thought and Methodology 27-A: 75-

136.

Fraine, Harold G. and Melvin W. Brethouwer. 1944. The Corporate Bond Project: Organization and

Methods. Part I of Organization and Administration of the Corporate Bond Project. [New

York]: National Bureau of Economic Research.

http://sca.gmu.edu/finding_aids/warburton.html






8/8/2019 Biased Reg


41

Friedman, Milton. 1962. ―Should There be an Independent Monetary Authority?‖ In In Search of a

Monetary Constitution . Edited by Leland Yeager. Cambridge: Harvard University Press, 219-243.

Friedman, Milton and Rose Director Friedman. 1998. Two Lucky People: Memoirs . Chicago: University

of Chicago Press.

Good, I. J. [1952] 1983. ―Rational Decisions.‖ Good Thinking: The Foundations of Probability and Its

Applications . Minneapolis: University of Minnesota Press, 3-14.

Harold, Gilbert. 1938. Bond Ratings as an Investment Guide. New York: Ronald Press.

Hickman, W. Braddock. 1952. Trends and Cycles in Corporate Bond Financing. Princeton: Princeton

University Press.

Hickman, W. Braddock. 1953. The Volume of Corporate Bond Financing Since 1900. Princeton:

Princeton University Press.

Hickman, W. Braddock. 1958. Corporate Bond Quality and Investor Experience . Princeton: Princeton

University Press.

Hickman, W. Braddock and Elizabeth T. Simpson. 1960. Statistical Measures of Corporate Bond

Financing since 1900. Princeton: Princeton University Press.

Hill, Claire A. 2010. ―Why Did Rating Agencies Do Such a Bad Job Rating Subprime Securities?‖

University of Pittsburgh Law Review 71: 1-24.

Hitchens, George P. 1944. The Record of Issue and Extinguishment Characteristics of Corporate Bonds,

1900-38. Part III of The Corporate Bond Project: Organization and Methods . New York:

National Bureau of Economic Research.

Kant, Immanuel. [1785] 1997. Foundations of the Metaphysics of Morals and What is Enlightenment?.

Translated by Lewis White Beck. Second edition, revised. Upper Saddle River, NJ: Prentice-Hall.

Kolchinsky, David. 2010. ―Statement and Testimony before the Financial Crisis Inquiry Commission.‖

8/8/2019 Biased Reg


42

http://www.fcic.gov/hearings/pdfs/2010-0602-Kolchinsky.pdf

Laidler, David. 1993. ―Hawtrey, Harvard, and the Origins of the Chicago Tradition.‖ Journal of Political

Economy 101: 1068-1103.

Lemmon, E. J. 1959. ―Is There Only One Correct System of Modal Logic?‖ Proceedings of the

Aristotelian Society 33: 23-40.

Levy, David M. and Sandra J. Peart. 2007. ―Sympathetic Bias.‖ Statistical Methods in Medical Research

17: 265-277.

Levy, David M. and Sandra J. Peart. 2008. ―Inducing Greater Transparency: Towards an Econometric

Code of Ethics.‖ Eastern Economic Journal 34: 103-14.

Levy, David M. and Sandra J. Peart. 2010. ―Tullock on Motivated Inquiry: Expert-Induced Uncertainty

Disguised as Risk.‖ Public Choice. Forthcoming

Levy, David M., Kail Padgitt, Sandra J. Peart, Daniel Houser, Erte Xiao. 2010. ―Leadership, Cheap Talk

and Really Cheap Talk.‖ Journal of Economic Behavior and Organization . Forthcoming.

National Bureau of Economic Reseach. 1941. The Corporate Bond Project, A History and Description, A

Work Projects Administration Study Sponsored by the Federal Deposit Insurance Corporation,

Supervised by the National Bureau of Economic Research . [New York]: NBER.

Palyi, Melchior. 1935. Review of Goldinflation und Wirtschaftsentwicklung: Gibt es ‗ lange Wellen ‘ der

Konjunktur? by Ernst John. Journal of Political Economy 43: 846.

Palyi, Melchior. 1938. ―Bank Portfolios and the Control of the Capital Market.‖ Journal of Business 11:

70-111.

Palyi, Melchior. 1972. The Twilight of Gold, 1924-36: Myths and Realities. Chicago: Regnery.

Partnoy, Frank. 2002. ―The Paradox of the Credit Ratings.‖In Ratings, Rating Agencies and the Global

Financial System . Edited by Richard M. Levich, Giovanni Majnoni and Carmen M. Reinhart.

Boston: Kluwer, 65-84.




8/8/2019 Biased Reg


43

Peart, Sandra J. 1996. The Economics of W. S. Jevons. London: Routledge.

Peart, Sandra J. and David M. Levy 2005. The ―Vanity of the Philosopher‖: From Equality to Hierarchy

in Post-Classical Economics. Ann Arbor: University of Michigan Press.

Phillips, J. Ronnie. 1995. The Chicago Plan and New Deal Banking Reform. Armonk, New York: Sharpe.

Reinhart, Catherine M. and Kenneth Rogoff. 2009. This Time It ‘ s Different. Princeton: Princeton

University Press.

Rheinstein, Max. 1954. ―Introduction.‖ In Max Weber on Law in Economy and Society. [1920].

Translated by Edward Shils. Cambridge, MA: Harvard University Press, xxv-lxxii.

Samuels, Warren J. 2005. ―Melchior Palyi: An Introduction and Bibliography.‖ Research in the History

and Methodology of Economics. 23: 305-306.

Schliesser, Eric S. 2011. ― The Surprising Weberian Roots of Milton Friedman‘s Methodology.‖ In

Explanation, Prediction and Confirmation. New Trends and Old Ones Reconsidered. Edited by

Dennis Dieks et al., Dordrecht: Springer.

Simons, Henry C. 1934. A Positive Program for Laissez Faire . Chicago: University of Chicago Press.

Stigler, Stephen M. 1977. ―Fractional Order Statistics, with Applications.‖ Journal of the American

Statistical Association 72: 544-550.

Sylla, Richard. 2002. ―An Historical Primer on the Business of Credit Rating.‖ Ratings, Rating Agencies

and the Global Financial System . Edited by Richard M. Levich, Giovanni Majnoni and Carmen

M. Reinhart. Boston: Kluwer, 19-40.

Tukey, John W. 1962. ―The Future of Data Analysis.‖ Philosophy and Principles of Data Analysis: 1949-

1964 . Volume 3 of The Collected Works of John W. Tukey . Edited by Lyle V. Jones. Monterey:

Wadsworth & Brooks, 1986, 391-484.

8/8/2019 Biased Reg


Viner, Jacob. 1962. ―The Necessary and the Desirable Range of Discretion to be Allowed to a Monetary

Authority.‖In In Search of a Monetary Constitution . Edited by Leland Yeager. Cambridge:

Harvard University Press, 244-74.

Weber, Max. [1922] 1927. General Economic History. Translated by Frank Knight from the German

edition compiled by S. Hellman and M. Palyi. New York: Greenberg.

Weiss, Stuart L. 1996. The President ‘ s Man: Leo Crowley and Franklin Roosevelt in Peace and War.

Carbondale: Southern Illinois University.

White, Kenneth. 2009. Shazam: Version 10 (SP 1). Vancouver: Shazam Econometrics.

White, Lawrence J. 2002. ―The Credit Rating Industry: An Industrial Organization Analysis.‖ Ratings,

Rating Agencies and the Global Financial System. Edited by Richard M. Levich, Giovanni

Majnoni and Carmen M. Reinhart. Boston: Kluwer, 19-40

White, Lawrence J. 2010. ―The Credit Rating Agencies.‖ Journal of Economic Perspectives 24: 211-26.

Biased Reg

Documents

Transcript of Biased Reg