1

Indicators of Knowledge Value

Conference on Estimating the Benefits of Government Sponsored Energy R&D

Department of Energy

At Hilton Crystal City - March 4-5, 2002

Diana HicksCHI Research, Inc.

10 White Horse PikeHaddon Heights, NJ 08035

Tel: 856 546 0600 Fax: 856 546 9633E-mail: dhicks@chiresearch.com

2

Outline

Who is CHI?

Knowledge value and bibliometrics

Examples: paper-patent citations

Examples: paper-paper citations

Patent citations to papers

Examples: patent-paper citations

People and knowledge value

3

Who is CHI Research?

We are a 30 year old consulting firm specializing in development and analysis of science indicators and patent-based technology metrics.

Francis Narin, founder, developed techniques for analyzing national scientific performance in the 1970’s; then developed techniques to assess corporate technological intellectual property using patent-based metrics in the 1980’s; then extended techniques to relate technology portfolios to stock market valuation in the 1990’s.

We have a staff of 21 (10 analysts and 11 support people) with diverse technical backgrounds.

Among the senior staff we have 6 PhD’s and 9 Masters degrees in areas such as Physics, Chemistry, Mathematics, Computer Science, Science Policy, Bibliometrics, and Public Policy.

4

CHI’s Databases

US Patent Database– Since 1975– 3 million US Patents, 1.1 million EPO patents– 24 million citations

Analytically ready– Filters

• CHI has developed the expertise needed to cleanly pull sets of patents on any technology

– Unification• Company names unified to link parent/subsidiary/joint ventures

– Restated for mergers, divestitures, and reassignments– Indicators

• More than 20 technology indicators – 2.4 million references to non-patent literature classified

• 1.4 million of these are references to journal articles and have been put in standard form

5

Technology

Patents citing Patents

Technology

Patents citing Patents

Knowledge Value and Bibliometric Techniques

Science

Papers citing Papers

Outcomes

Patents & Stock Marketciting Papers Performance

6

United States Patent 4,713,814

[Inventors] Andrusch et al. (Germany) Dec. 15, 1987

[Assignee] IBM (Armonk, NY)

STABILITY TESTING OF SEMICONDUCTOR MEMORIES

References Cited: U.S. PATENT DOCUMENTS

Firms Inventors

3,995,215 11/1976 IBM Chu et al. ....324/158

4,004,222 1/1977 Semi Corp. Gebhard .......324/158

4,418,403 11/1983 Mostek Corp. O'Toole et al. ....365/201

4,430,735 2/1984 Burroughs Corp. Catiller ...........371/25

4,502,140 2/1985 Mostek Corp. Prochsting .........371/21

4,503,538 3/1985 Robert Bosch GmbH Fritz ..............371/21

OTHER REFERENCES CITED

Wiedmann, IEEE Journal of Solid-State Circuits,

Vol. SC-19, no. 3, pp. 282-290, Jun. 1984.

These Citations Link This Patent With Earlier U.S.

Patents

These Citations

Link This Patent to Science

Extract from the Front Page of a U.S. Patent

7

DOE government interest patents CHI indicators

Indicator 92-96 97-01 % ChangePatents 2,160 2,920 35%Current Impact Index 0.88 0.85 -3%Science links per patent 2.6 5.1 96%TCT (years) 8.7 8.7 0%

Patents that indicate government interest and contain the word “energy” in the government interest field. Patents identified by David Eike

Indicators defined at: http://www.chiresearch.com/about/data/tech/indicator.php3

8

DOE government interest patents, leading technology areas

Patents

0 100 200 300 400 500

Office Equipment

Industrial Machinery

Power

Manufacturing

Biotechnology

Industrial Process Eq.

Electrical

Measuring & Control

Semiconductors & Electronics

Chemicals

ScienceLinkage

0 10 20

CII

0 0.5 1 1.5

9

IBM’s patents are valuable

Company in original set Indicates company with most cites

Company not in original set Indicates company with 2nd most cites

IBM

Hitachi

Compaq

Intel

AMD

Toshiba

MitsubishiElectric

TexasInstruments

NEC Motorola

Lucent

1577

2028

776

1322

19111885

1716

1088

2937

1299

2352

40361991

4587

2526

657

1535

1694

2058

2174

10

Does an agency resource support those creating high value

technology?

Incidence of

Subscriber and Non-

Subscriber Top-Decile

CFC Replacement Patents

11

Conceptual Diagram

9 U.S. Patents9 U.S. Patents

5 Foreign Patents

5 Foreign Patents

IBM Patent No.

5, 278,955

Issued 1994

IBM Patent No.

5, 278,955

Issued 1994

6 U.S. Patents6 U.S. Patents

6 Other References, Including 3 Science

References

6 Other References, Including 3 Science

References

Backward Citations (References)

Forward Citations

Time

1985-92 1994 1995-98

A Starting Patent references prior art, and is

cited by later patents

12

Science Linkage is Increasing in Many Countries

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

Israel

US

UK

France

Germany

Japan

Australia

Canada

3 year moving averagesU.S. excludes individual inventors

Science References per U.S. Patent

13

Excluding Biotechnology - Science Linkage is Increasing in Many

Countries

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

Science References per Patent with Biotechnology ExcludedU.S.

Israel

U.K.

Canada

AustraliaFrance

Germany

Japan

3 year moving averages

U.S. excludes individual inventorsPatents in pharmaceuticals, biotechnology and agriculture are excluded

14

Public Sector Science is Valuable for U.S.-Invented Technology

% of references on U.S. patents to U.S. scientific literature

1998

University

Non-Profit

Government

Industry

University Industry Government Non-Profit

54

19

13

13 7480

0

20

40

60

80

100

1988 1998

Public Sector

Source: NSF Science & Engineering Indicators and CHI Research, Inc.

15

Technological and scientific value are often aligned

Share of 1993-95 US papers cited in US invented, USPTO patents issued in 1997

0% 5% 10%

Bottom 50%

11-50%

2-10%

Top 1%S

cien

ce c

itat

ion

per

centile

Share of papers cited in patents

Overall, 1% of 1993-95 US papers were cited in 1997 US invented patents

16

U.S. Companies Preferentially Cite In-state Public Sector Science in Their

Patents

Expected # of citations = "patent state's" # of citations to all states multiplied by "paper state's" share of cites received from all states

Citations from Industry Patents to Public Sector Papers

AL VA CO FL IN NC MA MD CA

AL

CO

IN

MA

CA

0

2

4

6

8

10

12

14

Patenting companies' states

Cited university (or other public institution) state

Citation direction:

Alabama in-state: 39

Actual / Expected number of citations

papers

In-state citation ratios are on the diagonal. In-state ratios exceed out of state ratios in 23 out of 25 cases.

patents

17

Does an agency’s research support technological innovation?

Includes only papers with explicit support acknowledgements

Funding agencies acknowledged on biomedical papers cited by patents

1993-94 patents citing US authored biomedical papers published 1981-1991

18

DOE science supports US innovators

US patents 1985-2000 citing DOE papers.

Inventor addresses used to identify nationalities.

A 10 year, 2 year lagged citation window is used.

Citations fractionally counted

United States80%

Rest of World

6%

United Kingdom

2%

Germany2%

Canada2%

J apan8%

Share of patents citing DOE papers

19

DOE science has value for a range of technologies

Nu

mb

er

of

cita

tion

s fr

om

pate

nts

to t

ech

nolo

gy a

rea.

Pate

nts

19

85

-20

00

cit

ing

pap

ers

in

10

year,

2 y

ear

lag

ged

win

dow

Tech

nolo

gy-s

cien

ce c

om

bin

ati

on

s w

ith

>2

00

cit

ati

on

s are

sh

ow

n

328

233

206

3324

986

653

1465

1263

562

1626

489

391

349

255

244

201

0 1000 2000 3000 4000

Industrial Process Equipment

Plastics, Polymers And Rubber

Biotechnology

Biotechnology

Pharmaceuticals

Agriculture

Chemicals

Chemicals

Chemicals

Semiconductors And Electronics

Measuring And Control Equipmen

Miscellaneous Manufacturing

Computers And Peripherals

Fabricated Metals

Power Generation And Distribut

Office Equipment And Cameras

Chemicals

Biotechnology

Physical Sciences

Life Sciences

Chemistry

20

L. Bito and the Technology of Prostaglandins in the Treatment of

Glaucoma

21

Quadrant Model of Research Value

Research has value for:

Technology (patents)

No Yes

Yes Nuclear &

ParticlePhysics

Can be found in: Science, Nature,

Cell, PNAS

No Ultrasonic ImagingAT&T Technology

ReviewNon-science linked

patents

Science

(Scientific Literature)

After Stokes

22

Knowledge value is very unevenly distributed

0

2

4

6

8

10

12

14

16

18

16 119

Each stick represents 1 Xerox Inventor: the height is the number of his/her patents between 1981-87.

People are key, especially for prospective studies

23

Summary

Bibliometrics, properly constructed, are useful in assessing knowledge value.

The value of research for future scientific research and technological innovation can be examined.

People are crucial.