www.uis.unesco.org

Measuring R&D in Developing Countries: Annex to the Frascati Manual

TRAINING WORKSHOP ON SCIENCE, TECHNOLOGY AND INNOVATION INDICATORSCairo, Egypt

28-30 September 2009

www.uis.unesco.org

Outline

• The problem

• The process

• Contents of the Paper

• Thinking ahead

www.uis.unesco.org

R&D statistics in developing countries (1)

• Recognition, meeting targets, evidence-based S&T policy, but:

• lack of interest at the level of policy makers (low policy-relevance?)

• lack of resources devoted to statistics in S&T

• lack of technical knowledge for the production of cross-nationally comparable R&D statistics

• difficulties in applying FM concepts and methods

• weak statistical institutions

www.uis.unesco.org

R&D statistics in developing countries (2)

• Particular characteristics of R&D activities to be taken into account:

• different structures in terms of government, innovation system, higher education system, statistical system

• particular ‘culture of information’

• Users of R&D stat: Gov, analysts. + international donor agencies

• S&T indicators need to be adapted to particular policy needs, and need to provide answers to actual policy questions.

• However, international comparability is foremost.

www.uis.unesco.org

The process (1)

• Experience acquired through the UIS work, in particular through direct contact with S&T statisticians in numerous workshops and other meetings around the developing world.

• Advisory Meeting to the UIS S&T Statistics Programme held in Montreal, Canada, December 2007.

• Papers commissioned by UIS to Jacques Gaillard (IRD, Paris), Michael Kahn et al (HSRC, South Africa), and Gustavo Arber et al (RICYT, Argentina).

• Proposal for an annex to the Frascati Manual on measuring R&D in developing countries was presented at the OECD 2008 and 2009 NESTI meeting.

www.uis.unesco.org

The process (2)

• Expert Meeting on Measuring R&D in Developing Countries in Windhoek, Namibia, 14 to 16 September 2009

• Consultant to draft:• Working paper on Measuring R&D in Developing

Countries

• Proposed Annex to the Frascati Manual

• Both to be released in 2010

• Some of the issues might also present measurement challenges for a future revision of the Frascati Manual

www.uis.unesco.org

Main outcomes of the Namibia meeting

Developing countries a very heterogeneous concept

Problems not unique to developing countries

Stay within boundaries of FM

Most recommendations stood up

Much additional work needed

www.uis.unesco.org

Contents of the Working Paper

1. Introduction

2. Characteristics of R&D in Developing CountriesWill be merged with

3. Special Concepts and Definitions to be Applied in Developing Countries

4. Strategies for setting up S&T statistics systems

5. Thinking Ahead

www.uis.unesco.org

Chapters 2 and 3: Characteristics of R&D in Developing Countries + measurement implications

• Heterogeneity and concentration

• Special types of R&D

• Traditional knowledge

• Clinical trials

• Industrial activities

• Other activities

• Foreign institutions

• Counting researchers

www.uis.unesco.org

Special types of R&D - Traditional knowledge

Traditional knowledge

A cumulative body of knowledge, know-how, practices and representations maintained and developed by peoples with extended histories of interaction with the natural environment.

These sophisticated sets of understandings, interpretations and meanings are part and parcel of a cultural complex that encompasses language, naming and classification systems, resource use practices, ritual, spirituality and worldview.

www.uis.unesco.org

Special types of R&D - Traditional knowledge

Dichotomy between traditional and scientific knowledge systems

• substantive grounds – because of differences in the subject matter and characteristics of traditional and scientific knowledge

• methodological and epistemological grounds – because the two forms of knowledge employ different methods to investigate reality

• contextual grounds – because traditional knowledge is more deeply rooted in its environment

www.uis.unesco.org

Special types of R&D - Traditional knowledge

Links between traditional and scientific knowledge systems

• Traditional knowledge (in general) as an object of scientific study.

• The application of scientific methods to traditional knowledge, converting it into a source of scientific information

• Interaction between scientists and communities in participatory technology development

www.uis.unesco.org

Special types of R&D - Clinical trials

Clinical trials

• (Can) involve a significant amount of R&D

• Need to be conducted on a wide population

• Growth area for developing countries

www.uis.unesco.org

Special types of R&D - Clinical trials

Measurement of clinical trials

• Registers of clinical trials available, e.g. WHO but also national

• Funding often from abroad

• Performance various possibilities• a local branch of the foreign main sponsor

• universities and university hospitals

• individual researchers

• local medical clinics

• locally registered PNPs

• international PNPs

www.uis.unesco.org

Special types of R&D - Clinical trials

Measurement issues and recommendations

• Occupation category of local staff

• Medical doctors and other professionals with at least ISCED 5A degrees should be considered as researchers

• Nurses and other staff with qualifications below ISCED 5A should be accounted for as technicians

• FTE calculation is important (often part-time)

• Attribution of sector of performance must be done with care to avoid double counting

www.uis.unesco.org

Special types of R&D - Industrial activities

• Reverse engineering: understanding the structure and functioning of an object (in order to make a new device or program creates a similar object in a different way), copying it, or improving it.

• Recommendation: If reverse engineering is carried out in the framework of an R&D project to develop a new (and different) product, it should be considered as R&D.

www.uis.unesco.org

Special types of R&D - Other activities

• Community development and other social projects should be considered R&D only as long as they are in a development and testing phase, in which case they should be counted as experimental development, most probably in the field of social sciences

• In some developing countries, religious research has a particular importance. In principle, religious research is a part of humanities, and institutions performing it should be included in R&D surveys. Such countries might consider for example compiling the R&D activity of religious institutions as a separate sector.

This (religious research) will not be a recommendation

www.uis.unesco.org

The foreign institutions sector

What is included?

• Foreign antennas

• International organizations operating in the country

Remains in the business sector:

• Foreign company’s R&D labs

Remains in the HE sector:

• Foreign universities based and conducting R&D in campuses set up in the country

www.uis.unesco.org

The foreign institutions sector

Recommendation

• Create a “foreign institutions” (FI) sector as a separate sector of performance

• Funding flowing from this sector to other sectors should be considered from “Abroad” as stated in the main body of the Frascati Manual

www.uis.unesco.org

The foreign institutions sector

The principal sector sub-classification

• Business enterprises

• Government

• Higher Education

• Private non-profit

• International organizations

www.uis.unesco.org

Counting researchers

Underestimation of researchers

• Unpaid research

• Informal research

• Research outside of the normal work setting with external funding

• Multiple part time positions not taken into account or undercounted

• Master’s research

www.uis.unesco.org

Counting researchers

Overestimation of researchers

• Counting the contract instead of the real effort

• Multiple full-time research positions

www.uis.unesco.org

Counting researchers

Special cases

• FTE calculation >1 and FTE>HC

• R&D in times of crisis

• Visiting researchers

• Brain circulation

www.uis.unesco.org

Counting researchers

Recommendations

• Peer interviews of researchers

• Include a module on barriers

• Use secondary sources• Publication databases, both national and international

• STMIS and other databases of researchers

• Databases and registers of clinical trials

• Databases and registers of the main foreign donors involved in funding R&D in the countries

• University accreditation databases

www.uis.unesco.org

Other issues

• Informal R&D:• Occasional R&D

• R&D in the informal sector

• Budget data

www.uis.unesco.org

Chapter 4: Strategies for setting up S&T statistics systems in developing countries

• Institutionalizing S&T statistics

• Establishing registers

• Structural issues in the private sector and the private not-for-profit sector

• User-producer networks

• Science & Technology Management Information Systems and other secondary sources

• Survey procedures and estimation

www.uis.unesco.org

Institutionalization of S&T statistics

Political support

Infrastructure and sustained staff training/capacity building

Involvement of NSOs: “Official statistics” status for R&D surveys.

Adequate legal framework

www.uis.unesco.org

Establishing registers

• R&D in developing countries tends to be very much the purview of public bodies

Recommendations:

• Establishing a database of public sector R&D projects

• include human and financial resources; align with national policies.

• design could reflect the R&D statistical reporting/definitions.

• source for evaluation of such projects.

• Establishing STMIS

• provide overview of research system.

• framework for establishing complete registers as sample frames for R&D surveys.

www.uis.unesco.org

Establishing registers

• Other sources

• associations (trade, academic).

• learned societies.

• registers or databases of scientists and engineers.

• database of research grants.

• databases of scientific publications.

• patents and other IP documents.

• business registers.

www.uis.unesco.org

Structural issues in the private sector and the PNP sector

• Publicly-owned businesses play a major role in R&D in some developing countries

Recommendations:

• should consider issuing data for ‘publicly-owned businesses’ separately from the ‘fully private enterprise sector’.

• private enterprises could also be disaggregated by ownership, in particular the various degrees of foreign ownership.

www.uis.unesco.org

Structural issues in the private sector and the PNP sector

• Business enterprise R&D is presumed to be generally weak in developing countries when compared to industrial countries.

Recommendations:

• take into account when conducting sample surveys, perhaps by over-sampling, especially amongst larger companies.

• big companies should not be missed out as it might imply significant error.

• invest time in interviewing key firms to understand their R&D function and obtain a clear picture of their activity.

• Private-non-profit sector: make a significant contribution to R&D in developing countries, but the sector tends to be very volatile.

www.uis.unesco.org

User-producer networks

Recommendations: • user-producer networks and other forms of stakeholder

consultation should be instituted.

• establishing national S&T statistics groups.

• involve multiple actors.

• coordinating/networking among institutions/databases.

• partnering with business associations.

• conducting face-to-face visits by statisticians and project leaders.

• exploit pre-existing personnel ties.

• get NSO involved; to deal with privacy of information.

• training of interviewers/primary data producers.

www.uis.unesco.org

Science and Technology Management Information System and other secondary sources

• STMIS (e.g. database of scientists, research grants, etc): frequent source for the production of R&D statistics.

Recommendations: • need close integration between the statistical system and the

STMIS.

• need adjustments to produce comparable statistics, taking into account issues of definitions and coverage.

• need a balanced approach using both STMIS and surveys.

• need different approach to Private sector organizations as they are frequently not covered by these systems.

• Combined R&D and innovation surveys

Recommendations: • the relative rarity of occurrence of R&D in businesses needs to be

taken into account.

www.uis.unesco.org

Survey procedure and estimation

Recommendations:• attention needs to be paid to questionnaire design.

• frequency of survey.

• prioritize area of work; accompanied by step-by-step approach.

• use of survey questionnaires of other countries for inspiration: need adaptations to local situation.

• get expertise from the NSO, in conducting survey, in sampling ….

• different questionnaires might be designed for different sectors based on stakeholder consultations. “one size does not fit all”.

• procedures need to be developed for estimating missing data.

www.uis.unesco.org

Thinking ahead: Other products – beyond R&D

Redefine the concepts of scientific and technological education and training at broadly the third level (STET), Scientific and technological services (STS) and S&T activities (STA)

Better integrate education statistics with R&D statistics

Hands on guidance

Metadata

Model questionnaire

www.uis.unesco.org

Thank you!

http://www.uis.unesco.org

m.schaaper@uis.unesco.org