Evaluating funding modes towards a concept of funding ecologies Ulf Sandström KTH Stockholm Peter...
-
Upload
keyla-ingalls -
Category
Documents
-
view
212 -
download
0
Transcript of Evaluating funding modes towards a concept of funding ecologies Ulf Sandström KTH Stockholm Peter...
Evaluating funding modestowards a concept of funding ecologies
Ulf Sandström
KTH Stockholm
Peter van den Besselaar
VU University AmsterdamOrg Science & Network Institute
van den Besselaar FTEval 2013 2
Proliferation of funding modes
• New policy aims -> new funding modes– Strengthening competition (councils) – Career grant schemes– Excellence programs– Strategic knowledge: large scale programs– Useful knowledge: thematic programs
– International funding in various ways
14/11/2013
van den Besselaar FTEval 2013 3
From evaluating funding institutions and programs …
• Evaluating of funding institutions and individual funding instruments– Does the evaluated funding
institution/instruments work well?
• But what is “well”? Frame of reference? – Comparison? But how and with what?– Different funders/instruments fulfill different and
complementary roles
14/11/2013
van den Besselaar FTEval 2013 4
… to evaluating the funding ecology
14/11/2013
5
Funding ecology
• Complex landscape of funders and funding instruments with partly different and partly overlapping agenda’s– Overlapping: this leads to competition between
funders, opening up the system for variety– Different: specializing, niche seeking, searching for
the promising fields, application domains, procedures and instruments -> variety
14/11/2013 van den Besselaar FTEval 2013
van den Besselaar FTEval 2013 6
Competition & complementarity
– What is the optimal funding ecology?
– Avoiding duplication of instruments?
– Variety versus size?
14/11/2013
van den Besselaar FTEval 2013 7
Three tasks in analyzing the funding ecology
• Describe– Distinguish modes and instruments– Measure the output and impact by funding mode– At field level
• Compare the funding institutions/instruments• Understand the dynamics of funding ecologies– Optimal variety and size and under what
conditions
14/11/2013
van den Besselaar FTEval 2013 8
Ongoing research
• Impact of large thematic programs– Knowledge for Climate program
• Searching for excellence and the effects of funding– Bank of Sweden Tercentenary Foundation – With Ulf Sandstrom and Agnes Wold
• Ongoing work -> scattered results up to now
14/11/2013
van den Besselaar FTEval 2013 9
Description: comparison and complementarity of funders
• Do different funding modes differ in terms of:
– Organizational forms, procedures, criteria
– Topical focus
– Productivity, impact, innovativeness, societal relevance
14/11/2013
van den Besselaar FTEval 2013 10
Methods and data• WoS data; delineation of fields• Topical structure• Field normalized impact • Publ: 2009/2010; Cit. window 2010-12• Clean/disambiguate funders• Classify funders: funder type
14/11/2013
van den Besselaar FTEval 2013 11
Empirical examples
• Two dimensions– Impact by funding mode– Topical coverage by funding mode
• Two fields– climate change research– immunology
• Two countries – Sweden – the Netherlands
14/11/2013
van den Besselaar FTEval 2013 12
Funding modes / organizations• Universities• Public research Institutes– Fundamental– Applied
• Councils– Basic– Applied
• Organizations – Private foundations– NGO’s– Societies
• Thematic
– Large national programs– Thematic councils
• EC: – FW program – ERC, Marie Curie
• Foreign funders
• Companies
• No funders mentioned
14/11/2013
van den Besselaar FTEval 2013 13
Delineation
• Climate change research: search terms (TS)• Immunology: journals (WoS research area)
• More fields/countries in next phase
14/11/2013
van den Besselaar FTEval 2013 14
New funders: climate change research 1988-2012
1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 20110
100
200
300
400
500
world nl sweden germany swiss
14/11/2013
van den Besselaar FTEval 2013 15
Impact by funder (NL-climate)
14/11/2013
van den Besselaar FTEval 2013 16
Impact by funder (NL-climate)
1 3 4 5 6 7 8 90%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
no funder
appl agencies
council
univ
foreign
big progs
EC
14/11/2013
van den Besselaar FTEval 2013
Topical diifferences (NL-climate)
14/11/2013 17
van den Besselaar FTEval 2013 18
Impact by funder (Sw-climate)
14/11/2013
van den Besselaar FTEval 2013 19
Int. co-auth. papers (Sw-climate)
14/11/2013
van den Besselaar FTEval 2013 20
Nat auth papers (Sw-climate)
14/11/2013
van den Besselaar FTEval 2013 21
Summary of findings up to now
• Impact:– International funding > national funding– Partly but not fully international collaboration effect – Funding conditions are important
• Topics: differences & overlap
14/11/2013
van den Besselaar FTEval 2013 22
Current work
• Extending the analysis to other fields and countries:
– Patterns (1): relation between output/impact variables and number of funders / funder types
– Patterns (2): relation between output/impact variables and size of funders / instruments
14/11/2013
van den Besselaar FTEval 2013 23
Extending the evidence
• Similar results for Immunology– International funders > national funders– Effect remains (but smaller) when controlling for
international co-authorship– Funding conditions matter
• Size of instrument matters:– Size/selection trade off
14/11/2013
Peter van den Besselaar / predictive validity 24
Predictive validity by competition level
9/05/13
van den Besselaar FTEval 2013 25
Conclusions and further work
• Comparing funding modes and instruments seems promising
• Technical improvements (classification funding modes, organizational differences, statistics)
• More fields, more countries• More performance indicators (e.g., societal impact)
• Empirical regularities? And then explanations.
• Lessons about individual funding modes / instruments• Lessons about funding ecologies14/11/2013