Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix...

45
Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk F. Moed SCIMago Research Group, Univ Granada, Spain CWTS, Leiden Univ, the Netherlands

Transcript of Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix...

Page 1: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Towards a bibliometric database for the social sciences and

humanities

Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk F. Moed

SCIMago Research Group, Univ Granada, SpainCWTS, Leiden Univ, the Netherlands

Page 2: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

This presentation …..

• Highlights wider context and its dynamics

• Makes analytical distinctions

• Describes actual developments

• Discusses options

Page 3: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Structure of the presentation

• Introduction; background

• Analysis of existing SSH databases

• Actual use of bibliometric indicators in research assessment

• Actual practices in dealing with non-publication outputs

• Options for creating a comprehensive database of SSH outputs

Page 4: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

General scheme

Research AssessmentObjectives; criteria

Assessment methodologies;indicators

Databases

Page 5: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

1.

Introduction; background

About SSH

General trends

Indicators

Page 6: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Science vs. Social Sci & Humanities (SSH)

Science Soc Sci + Humanities

Source/ document types

Journal research article is main

type

Great diversity; books important

(e.g. monographs, edited works)

Journal system Concentration in limited number of international

journals

Less concentration;

national journals also important

Bibliographical system

Concentration in limited number

of large internat. databases

Great diversity of bibliographies and catalogs (national,

specialized)

Page 7: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

SSH outputs and impacts

Impacts Publication/text Non-publication / non-text

Scientific-scholarly

Research paper; monograph; book

chapter

Research data file; video of experiment

Educational Teaching course book;

PhDs

Economic Patent Product; process; device; design; image

Cultural Newspaper article; TV interviews; Performances;

exhibits; events

Page 8: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Trends-1 : Increasing importance of research assessment

• Performance-based funding across scientific institutions

• Institutions operate in a global market

• Public rankings of institutions

• European Research Area and Research Council

• Management tools within institutions

• Need for adequate tools for research assessment in SSH fields

Page 9: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Trends - 2 : Electronic publishing; more databases

• Publishers make their content electronically available on-line

• More and more documents freely available via the Web (Open access, self archiving)

• Standardisation of meta-data and meta-data infastructures

• More repositories (institutional/disciplinary)

• More institutional research management systems

• Web of Science (WoS), Scopus and Google Scholar are genuine competitors

Page 10: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Trends – 3 : Better coverage of SSH; more indicators

• WoS and Scopus expand their source coverage of SSH fields

• Bibliographical databases implement bibliometric features

• Calculation of bibliometric indicators not merely done by bibliometric experts

Page 11: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Two types of bibliometric activity

Type of activity Description Type of database used

Desk-top or poor man’s bibliometrics

Collects simple indicator data directly from

database

Bibliographic database;

indicator sets

Advanced bibliometrics

Data collection protocols;

verification; sophisticated

indicators

Bibliometric database

Page 12: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Bibliometric database: examples

• Cited references parsed and linked to corresponding targets (citation index)

• De-duplicated institutional affiliations

• Author names linked to unique researchers

• Abstracts parsed into noun phrases

• Dates expressed as numbers

• Policy relevant subfield delimitations

• Acknowledgements parsed; funders de-duplicated

Page 13: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Principal bibliometric indicators and database requirements – 1

Concept Definition Minimal database requirements

Production Nr. written documents published

Publ. meta-data; publ. types

Importance of publ. source

Impact factor; expert ratings

Source categorizations

Impact Citations Citation index

Collaboration (Institutional) co-authorship

Multiple authors/ addresses

Page 14: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Principal bibliometric indicators and database requirements – 2

Concept Definition Minimal database requirements

Cognitive structures

e.g. co-word maps Titles, abstracts

Qualitative citation analysis

Citation context analysis

Full texts

Semantics-based detection of links

e.g., scientific instruments

mentioned in full texts

Full texts

Page 15: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Bibliometric impact indicators

Indicator Comment

Citations What do citations in SSH fields measure?

Full text downloads Limited availability; What do downloads measure?

Hyperlinks Webometric assessment; combine with other indicators

Book purchases Normally not publicly available (but see amazon.com)

Library holdings Exploratory studies published recently

Library loans Not used in res. perf. studies

Page 16: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

2. Analysis of existing SSH

databases

Page 17: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Sources of publication meta dataNational bibliographies

Library catalogues (OPAC) US Library of Congress; Acad libr; OCLC Worldcat

Short title catalogues ESTC; national bibliogr. for older books

Publisher or vendor catalogues

Amazon.com; Springer ebook catalogue

Special bibliographies and abstracts

FRANCIS, Sociol Abstr, PsychInfo, ECONLIT, …

Citation indexes Web of Science, Scopus, Google Scholar

Repositories In principle open access

Institutional research management systems

Output registration systems annual research reports

Google Book Search See later slide

Page 18: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Sources of citation data

Citation Indexes Web of Science (SSCI, A&HCI), Scopus

Special bibliographies PsychInfo, Sociol Abstr, World Polit Abstr,… ; most do not have citations

Repositories CiteseerX; most repositories do not have citations

Google Scholar Harzing’s publish-or-perish software allows citation counting

CrossRef Cross-publisher linking system provided by publishers

Page 19: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

ERIH Journal classification for 11 Humanities research fields (6,000 journals)

Class Description

A High ranking international journals with very strong reputation

B Standard international journals with good reputation

C Important domestic research journals

Page 20: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Analysis of databases

Producers

Special bibliographies

20 mainly European bibliographies

Big citation indexes

Thomson, Elsevier, Google

Publishers University Presses

Scientific institutions

Institutional repositories

Scientific institutions

Research management systems

Page 21: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

European special bibliographies:Aspects considered – 1

• Database producer• Dates of coverage• (Sub)disciplines covered• Number of records• Selection criteria of sources• Countries or languages covered• Type of sources covered • Nr authors of a source publication included• Nr institutional affiliations included• Standardisation of author names/affiliations

Page 22: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

European special bibliographies:Aspects considered – 2

• Categorization of documents• Type of content classification system• Does it contain cited references in source

publications?• Has the database ever been used in

bibliometric studies?

Page 23: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Database Author affiliations Cited references

Francis Soc Sci + Hum Y (All, as from ‘97) N

CSA Ling & Lang Y N

Econlit Y N

IBSS Social Sci Y N

LISA Libr & Inf Sci Y (as from 2006) N

Psychinfo Y (all authors) Y

Sociolog Abstr Y (1st author only) Y (as from 2002)

World Polit Abstr Y (1st author only) Y (as from 2001)

Historical Abstr N Y

America: Hist & Life N Y

ERIC: Educat Res Y (1st author only) N (Y in full texts)

SOLIS Social Sci Y (in Author field) N

Page 24: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Special bibliographies: Conclusions

• Some interesting databases for field-specific studies

• In most databases no author affiliations and no cited references

• Mainly journal articles covered

Page 25: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Citation Indexes: Major recent developments

Database Journals Books

Web of Science

Adds in 2009 1,500 “regional” journals (many SSH, from

ERIH lists)

?

Scopus Adds 3,500 titles (1,500 in April 2009); 2,250 from ERIH lists (all A, 1,000 B, 250 C)

Adds meta-data on highly cited book

titles

Google Further enhancements of Google Scholar

Expands Google Book Search; Link

with Google Scholar?

Page 26: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Google Book Search – 1

Aims: • Enhance the user’s ability to access and read

books• Opportunity for authors and publishers to

make their books available.

Two sources: • Partner program: Publishers and authors • Library project: partner libraries

Page 27: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Google Book Search – 2

• For books protected by copyright, search results are limited to meta-data and selected (random) text passages

• Books out-of-copyright may be read online in full length or downloaded

• Cited references not a part of meta-data• GBS books as targets in Google Scholar

Page 28: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

OAPEN - Open Access Publishing in European Networks

• OAPEN: European scholarly university presses active in SSH and in book publishing

• EU funded project aims to improve quantity, visibility and usability of high-quality OA content.

• Uses Driver infrastructure

Page 29: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

European Institutional repositories

• EU funded project DRIVER aims to establish a cohesive, pan-European infrastructure of Digital Repositories

• Driver Inventory study (2006): 230 European institutions have a repository (=10-20%?)

• These cover 37 % of institutions’ recent publication output (based on questionnaire)

• 18 % of materials is books /chapters

• 30 % of materials is SSH

Page 30: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Driver Inventory study (2006)

Type %

Textual research materials

Meta-data only 61 %

Full texts 29 %

Non textual materials

Images, videos, music, primary data sets

5 %

Other Learning materials, student papers

5 %

Page 31: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Institutions need research management systems for ….

• National research assessment exercises (e.g., UK REF)

• Funding parameters at a national level

• International rankings and benchmarking

• Positioning in European Research Area

• Information for general public and clients

• Internal research management at central and departmental level

Page 32: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

3.Actual use of bibliometric

indicators in research assessment

In progress

Page 33: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Spain:

ANEP, ANECA

Norway:

Norwegian modelObjective Evaluation /

promotion of individual scholars

Distribution of funds across institutions

Unit of assessment

Individuals Institutions

Indicators Nr papers weighted with impact factor

of JCR/WoS covered journals

Nr publications weighted with peer ratings of sources

and their publishers

Page 34: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Flanders: Distribution of funds across institutions

• Current funding parameters based on Web of Science

• Creation database of scientific-scholarly SSH publications per institution

• Institutions establish selection criteria for sources and document types

• Operational in 2011

Page 35: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

4.Actual practices in dealing with

non-publication outputs

Page 36: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

The UK Research RAE 2008

• Research: original investigation undertaken in order to gain knowledge and understanding

• Scholarship: creation, development and maintenance of the intellectual infrastructure of subjects and disciplines

Page 37: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

The UK Research RAE 2008Panel O

Research outputs must be verifiable and relate to research, and may include:

• New materials• Devices• Images• Products and buildings• Intellectual property (patents or other forms)• Performances, exhibits or events• Work published in non-print media

Page 38: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

RAE 2008 submissions: criteria

• Submitted research outputs must be verifiable and relate to research

• Non-text output: evidence of their dissemination in the public domain and of their research contents are required

Page 39: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Australia: ERA non-publication outputs

Applied research• Registered designs (if clear link with

research)

Esteem:• Editorial roles and contribution to

prestigious works of reference• Curatorial role of a prestigious event

Page 40: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Yardsticks

• Formal yardsticks for scholarly non-publication output are missing in RAE and ERA

• Prestigious events, performance outlets , media might be ranked according to prestige

Page 41: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

5.Options for creating a

comprehensive database of SSH outputs

Page 42: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Options

1. Combine European special SSH bibliographies

2. Further enhance SSH coverage of Web of Science and / or Scopus (books!)

3. Stimulate creation and standardization of institutional research management systems

4. Further stimulate institutional repositories

Page 43: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Four options: discussion

Feasibility Problems

Combine special European SSH bibliographies

Costly; current databases to be

restructured

Expand SSH coverage of WoS and Scopus

Feasible as long as there is a market for it

Sufficiently comprehensive?

(books!)

Stimulate institut. res. management systems EU initiative fits

into concept of Europ Res Area

Pre-selection of public. types necessary?Stimulate

institutional repositories

Page 44: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

Further research

• Exploration of Google Scholar / Book Search

• Further information-scientific and sociological studies of SSH fields

• How to discriminate between scholarly and non-scholarly sources (pre-selection criteria)

Page 45: Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk.

END