Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix...
-
Upload
jesus-smith -
Category
Documents
-
view
215 -
download
1
Transcript of Towards a bibliometric database for the social sciences and humanities Carmen Lopez Illescas, Felix...
Towards a bibliometric database for the social sciences and
humanities
Carmen Lopez Illescas, Felix de Moya Anegon, Janus Linmans, Anton Nederhof and Henk F. Moed
SCIMago Research Group, Univ Granada, SpainCWTS, Leiden Univ, the Netherlands
This presentation …..
• Highlights wider context and its dynamics
• Makes analytical distinctions
• Describes actual developments
• Discusses options
Structure of the presentation
• Introduction; background
• Analysis of existing SSH databases
• Actual use of bibliometric indicators in research assessment
• Actual practices in dealing with non-publication outputs
• Options for creating a comprehensive database of SSH outputs
General scheme
Research AssessmentObjectives; criteria
Assessment methodologies;indicators
Databases
1.
Introduction; background
About SSH
General trends
Indicators
Science vs. Social Sci & Humanities (SSH)
Science Soc Sci + Humanities
Source/ document types
Journal research article is main
type
Great diversity; books important
(e.g. monographs, edited works)
Journal system Concentration in limited number of international
journals
Less concentration;
national journals also important
Bibliographical system
Concentration in limited number
of large internat. databases
Great diversity of bibliographies and catalogs (national,
specialized)
SSH outputs and impacts
Impacts Publication/text Non-publication / non-text
Scientific-scholarly
Research paper; monograph; book
chapter
Research data file; video of experiment
Educational Teaching course book;
PhDs
Economic Patent Product; process; device; design; image
Cultural Newspaper article; TV interviews; Performances;
exhibits; events
Trends-1 : Increasing importance of research assessment
• Performance-based funding across scientific institutions
• Institutions operate in a global market
• Public rankings of institutions
• European Research Area and Research Council
• Management tools within institutions
• Need for adequate tools for research assessment in SSH fields
Trends - 2 : Electronic publishing; more databases
• Publishers make their content electronically available on-line
• More and more documents freely available via the Web (Open access, self archiving)
• Standardisation of meta-data and meta-data infastructures
• More repositories (institutional/disciplinary)
• More institutional research management systems
• Web of Science (WoS), Scopus and Google Scholar are genuine competitors
Trends – 3 : Better coverage of SSH; more indicators
• WoS and Scopus expand their source coverage of SSH fields
• Bibliographical databases implement bibliometric features
• Calculation of bibliometric indicators not merely done by bibliometric experts
Two types of bibliometric activity
Type of activity Description Type of database used
Desk-top or poor man’s bibliometrics
Collects simple indicator data directly from
database
Bibliographic database;
indicator sets
Advanced bibliometrics
Data collection protocols;
verification; sophisticated
indicators
Bibliometric database
Bibliometric database: examples
• Cited references parsed and linked to corresponding targets (citation index)
• De-duplicated institutional affiliations
• Author names linked to unique researchers
• Abstracts parsed into noun phrases
• Dates expressed as numbers
• Policy relevant subfield delimitations
• Acknowledgements parsed; funders de-duplicated
Principal bibliometric indicators and database requirements – 1
Concept Definition Minimal database requirements
Production Nr. written documents published
Publ. meta-data; publ. types
Importance of publ. source
Impact factor; expert ratings
Source categorizations
Impact Citations Citation index
Collaboration (Institutional) co-authorship
Multiple authors/ addresses
Principal bibliometric indicators and database requirements – 2
Concept Definition Minimal database requirements
Cognitive structures
e.g. co-word maps Titles, abstracts
Qualitative citation analysis
Citation context analysis
Full texts
Semantics-based detection of links
e.g., scientific instruments
mentioned in full texts
Full texts
Bibliometric impact indicators
Indicator Comment
Citations What do citations in SSH fields measure?
Full text downloads Limited availability; What do downloads measure?
Hyperlinks Webometric assessment; combine with other indicators
Book purchases Normally not publicly available (but see amazon.com)
Library holdings Exploratory studies published recently
Library loans Not used in res. perf. studies
2. Analysis of existing SSH
databases
Sources of publication meta dataNational bibliographies
Library catalogues (OPAC) US Library of Congress; Acad libr; OCLC Worldcat
Short title catalogues ESTC; national bibliogr. for older books
Publisher or vendor catalogues
Amazon.com; Springer ebook catalogue
Special bibliographies and abstracts
FRANCIS, Sociol Abstr, PsychInfo, ECONLIT, …
Citation indexes Web of Science, Scopus, Google Scholar
Repositories In principle open access
Institutional research management systems
Output registration systems annual research reports
Google Book Search See later slide
Sources of citation data
Citation Indexes Web of Science (SSCI, A&HCI), Scopus
Special bibliographies PsychInfo, Sociol Abstr, World Polit Abstr,… ; most do not have citations
Repositories CiteseerX; most repositories do not have citations
Google Scholar Harzing’s publish-or-perish software allows citation counting
CrossRef Cross-publisher linking system provided by publishers
ERIH Journal classification for 11 Humanities research fields (6,000 journals)
Class Description
A High ranking international journals with very strong reputation
B Standard international journals with good reputation
C Important domestic research journals
Analysis of databases
Producers
Special bibliographies
20 mainly European bibliographies
Big citation indexes
Thomson, Elsevier, Google
Publishers University Presses
Scientific institutions
Institutional repositories
Scientific institutions
Research management systems
European special bibliographies:Aspects considered – 1
• Database producer• Dates of coverage• (Sub)disciplines covered• Number of records• Selection criteria of sources• Countries or languages covered• Type of sources covered • Nr authors of a source publication included• Nr institutional affiliations included• Standardisation of author names/affiliations
European special bibliographies:Aspects considered – 2
• Categorization of documents• Type of content classification system• Does it contain cited references in source
publications?• Has the database ever been used in
bibliometric studies?
Database Author affiliations Cited references
Francis Soc Sci + Hum Y (All, as from ‘97) N
CSA Ling & Lang Y N
Econlit Y N
IBSS Social Sci Y N
LISA Libr & Inf Sci Y (as from 2006) N
Psychinfo Y (all authors) Y
Sociolog Abstr Y (1st author only) Y (as from 2002)
World Polit Abstr Y (1st author only) Y (as from 2001)
Historical Abstr N Y
America: Hist & Life N Y
ERIC: Educat Res Y (1st author only) N (Y in full texts)
SOLIS Social Sci Y (in Author field) N
Special bibliographies: Conclusions
• Some interesting databases for field-specific studies
• In most databases no author affiliations and no cited references
• Mainly journal articles covered
Citation Indexes: Major recent developments
Database Journals Books
Web of Science
Adds in 2009 1,500 “regional” journals (many SSH, from
ERIH lists)
?
Scopus Adds 3,500 titles (1,500 in April 2009); 2,250 from ERIH lists (all A, 1,000 B, 250 C)
Adds meta-data on highly cited book
titles
Google Further enhancements of Google Scholar
Expands Google Book Search; Link
with Google Scholar?
Google Book Search – 1
Aims: • Enhance the user’s ability to access and read
books• Opportunity for authors and publishers to
make their books available.
Two sources: • Partner program: Publishers and authors • Library project: partner libraries
Google Book Search – 2
• For books protected by copyright, search results are limited to meta-data and selected (random) text passages
• Books out-of-copyright may be read online in full length or downloaded
• Cited references not a part of meta-data• GBS books as targets in Google Scholar
OAPEN - Open Access Publishing in European Networks
• OAPEN: European scholarly university presses active in SSH and in book publishing
• EU funded project aims to improve quantity, visibility and usability of high-quality OA content.
• Uses Driver infrastructure
European Institutional repositories
• EU funded project DRIVER aims to establish a cohesive, pan-European infrastructure of Digital Repositories
• Driver Inventory study (2006): 230 European institutions have a repository (=10-20%?)
• These cover 37 % of institutions’ recent publication output (based on questionnaire)
• 18 % of materials is books /chapters
• 30 % of materials is SSH
Driver Inventory study (2006)
Type %
Textual research materials
Meta-data only 61 %
Full texts 29 %
Non textual materials
Images, videos, music, primary data sets
5 %
Other Learning materials, student papers
5 %
Institutions need research management systems for ….
• National research assessment exercises (e.g., UK REF)
• Funding parameters at a national level
• International rankings and benchmarking
• Positioning in European Research Area
• Information for general public and clients
• Internal research management at central and departmental level
3.Actual use of bibliometric
indicators in research assessment
In progress
Spain:
ANEP, ANECA
Norway:
Norwegian modelObjective Evaluation /
promotion of individual scholars
Distribution of funds across institutions
Unit of assessment
Individuals Institutions
Indicators Nr papers weighted with impact factor
of JCR/WoS covered journals
Nr publications weighted with peer ratings of sources
and their publishers
Flanders: Distribution of funds across institutions
• Current funding parameters based on Web of Science
• Creation database of scientific-scholarly SSH publications per institution
• Institutions establish selection criteria for sources and document types
• Operational in 2011
4.Actual practices in dealing with
non-publication outputs
The UK Research RAE 2008
• Research: original investigation undertaken in order to gain knowledge and understanding
• Scholarship: creation, development and maintenance of the intellectual infrastructure of subjects and disciplines
The UK Research RAE 2008Panel O
Research outputs must be verifiable and relate to research, and may include:
• New materials• Devices• Images• Products and buildings• Intellectual property (patents or other forms)• Performances, exhibits or events• Work published in non-print media
RAE 2008 submissions: criteria
• Submitted research outputs must be verifiable and relate to research
• Non-text output: evidence of their dissemination in the public domain and of their research contents are required
Australia: ERA non-publication outputs
Applied research• Registered designs (if clear link with
research)
Esteem:• Editorial roles and contribution to
prestigious works of reference• Curatorial role of a prestigious event
Yardsticks
• Formal yardsticks for scholarly non-publication output are missing in RAE and ERA
• Prestigious events, performance outlets , media might be ranked according to prestige
5.Options for creating a
comprehensive database of SSH outputs
Options
1. Combine European special SSH bibliographies
2. Further enhance SSH coverage of Web of Science and / or Scopus (books!)
3. Stimulate creation and standardization of institutional research management systems
4. Further stimulate institutional repositories
Four options: discussion
Feasibility Problems
Combine special European SSH bibliographies
Costly; current databases to be
restructured
Expand SSH coverage of WoS and Scopus
Feasible as long as there is a market for it
Sufficiently comprehensive?
(books!)
Stimulate institut. res. management systems EU initiative fits
into concept of Europ Res Area
Pre-selection of public. types necessary?Stimulate
institutional repositories
Further research
• Exploration of Google Scholar / Book Search
• Further information-scientific and sociological studies of SSH fields
• How to discriminate between scholarly and non-scholarly sources (pre-selection criteria)
END