General reflections on the

evaluation procedure –

comparisons with

Sweden

Leos Müller,

Stockholm University

www.metodika.reformy-msmt.cz

Necessity of efficient evaluation

system in humanities

• 1 Limited resources require priorities

• 2 Agreement in humanities on what is high quality research

• a broad agreement on minimum requirements

• more difficult to reach agreement on “excellency”, “global leadership and

especially what is “relevant” on national and international level

• this has to be taken in account when considering ways how to evaluate high

quality research (the experience or our evaluation panel)

1

General observations…

• Awareness of the necessity to employ a different evaluation system in

humanities (e g history) than in natural sciences, medicine etc.

• Our panel’s fear that the evaluation system will provide legitimacy for

reduced funding in humanities

• Evaluation procedure adjustments for conditions in humanities:

• 1/ difficulty to employ bibliometric data, to relay too much on quantitative

datasets

• 2/ difficulty to relay on tools based on priority of English publications. In

humanities publications in national language are more important than

publications in English. This is especially important for small languages

such as Czech or Swedish

2

Our panel’s experience with the

inputs

• Self-Assessment Report – the key document supplied to panel

• BUT! Current template adopted for hard sciences, many questions not

relevant or difficult to answer. Consequently, some problems with

providing relevant and comparable information.

3

• Bibliometric Report – of limited value in humanties BUT Meaningful

• Use of journal ranking, publishers’ ranking is possible and meaningful

on national level (Sweden, Norway examples)

• Necessity to support the report with book reviews (in relevant journals

AND anonymous reviews for panel’s needs)

• general observation: edited volumes less significant than monographs

and journal articles

4

• Excellent outputs according to RU – reflect the priorities of RU’s

leadership

• RUs have varying perceptions of “research quality”

• Representation ( broad selection of publications to illustrate activity) or

excellence (the best publications according to RUs)

5

Comments on the organization

and costs of evaluation procedure

• In general, preparation, provided material, project’s website, and

meetings worked well.

• Site-visits are valuable -- perhaps they could be replaced by a group

interviews in Prague

• Monograph publications should be available in physical form (pdf

format not functional for books)

• Use of Skype, or comparable internet platforms recommended: it saves

time and money.

6

Sweden and Czech Republic –

Funding and Evaluation

• Similarities: relatively small countries (c. 10 mil) small languages make

it difficult to reach outside.

• Culturally homogenous: joined heritage, history, literature. Humanities

play an important role in shaping national identity: “Who are we?”

• Research organization and funding in Sweden decentralised:

• Swedish Royal Academy has very limited funding role in comparison

with Czech Academy of Sciences

7

Funding and evaluation of

humanities in Sweden

• University level: less competitive, today fairly limited (reduced in last

two decades)

• National foundations:

• Swedish Research Council (Vetenskapsrådet)

1-2% of applications get funding

• Swedish Foundation for Humanities and Social Sciences (Riksbankens

Jubileumsfond)

• Strong private Foundations:

• Wallenberg Family, Handelsbanken etc. , better rates

• Fundations connected to specific universities/colleges:

• Växjö. Linneus University (IKEA/ Kamprad donation).

8

• Funding based on evaluation of individual project applications BUT!

Weak follow-up

• To write good application is more important than to report results in time

• Increasing importance of university funding related to research output

(similar to Metodika) yet only on university level

• Example of Fac. of Humanities in Stockholm University

• Increasing share of funding related to research output (publications, bibliometric

report) disciplines with strong international impact ( philosophy, archaeology) get

better funding than disciplines active in Sweden and publishing in Swedish

(history, literature, arts)

9

Thank you for your attention!

www.metodika.reformy-msmt.cz