External Panel meeting Evaluation of SRC Medicine Panel Reorganization January 8th, 2009.
-
Upload
brook-lambert -
Category
Documents
-
view
215 -
download
0
Transcript of External Panel meeting Evaluation of SRC Medicine Panel Reorganization January 8th, 2009.
External Panelmeeting
Evaluation of SRC Medicine Panel Reorganization
January 8th, 2009
Program9.00 Welcome and introduction
Mission of the external panel Objectives of the meetingDesignation of a panel chair
9.15 Information about the Swedish Research Council Medicine and background to the evaluation
10.15 Coffee Break
10.30 Project planParameters to analyzeHow has the new organization been received by the scientific coummunity?Who should be approached - and how?Peer review evaluation and development in other organizationsThe Research Bill and peer review
12.30 Lunch
13.30 Practical and logistical details 13.45 Drafting of an interim report14.30 Action plan and conclusion
Milestones/time linesImmediate action plan
15.00 Any other business and remaining questions15.15 Meeting concludes
External panel mission
• Appointed by SRC Medicine on September 9th, 2008Dr. ir. Janna O. de Boer, ZonMw : Netherlands Organisation for Health Research and DevelopmentProfessor Rolf Reed, University of Bergen, NorgeProfessor Ulf Pettersson, Uppsala University
• To evaluate the new review panel organization• To present a final report in May 2009 to the SRC medicine• The report should contain the conclusions of the expert
panel and recommendations to the SRC on it’s review panel organization
• The appointment includes an evaluation of how the new review panels are organized, composed and functioning. The evaluation also includes, as far as possible, an evaluation on the effects of the reorganization for the peer reveiw at SRC Medicine.
• The mission does not include an evaluation of the SRC budget process
Evaluation, continued
The evaluation will be based both on information from the SRC data base and information gained through questionnaires, interviews and an open web consultation
The project is coordinated by the Deputy Secretary General
The panel assisted by a project group from the SRCDr Tove Andersson, Research Officer
Ms Maria Karlberg, Administrative Assistant
Ms Marlene Truedsson Information Officer for Medicine
There will be an interim report to the SRC Medicine for their meeting on February 3rd.
The content of the interim report will be decided at today’s meeting
Objectives of today’s meeting
• Background/information• Decided content of evaluation/parameters to
analyze and how• Drafting of interim report• Set mile stones/time lines
Information about Swedish Medical research, the Swedish Research Council and background to the evaluation
SWEDISH RESEARCH COUNCIL
The Swedish Research Council is a government agency
funding basic research of the highest scientific quality in
all disciplines. The Swedish Research Council has a
national responsibility to support and develop basic
research and promote research innovation and research
communication. The goal is for Sweden to be a leading
nation in scientific research.
RESEARCH FUNDING
• allocates support for Swedish basic research, with an emphasis on attaining the highest quality and bringing about development and renewal. The Swedish Research Council is the largest state body providing funds for basic research in Sweden.
RESEARCH POLICY
• advises the Government in matters relating to research policy. The Swedish Research Council is engaged in strategic issues concerning research and research funding in a national and international perspective.
RESEARCH INFORMATION
• to increase the contacts and dialogue between researchers and the general public in close cooperation with universities and university colleges, organisations, and other players.
• Board• Director General and Deputy Director General• Scientific Councils and Committees in five fields of operations• Secretary Generals • Office for management • Advisory bodies in special fields• Review organisation
ORGANISATION
SWEDISH RESEARCH COUNCIL
The Swedish Research Council is a government agency
funding basic research of the highest scientific quality in
all disciplines. The Swedish Research Council has a
national responsibility to support and develop basic
research and promote research innovation and research
communication. The goal is for Sweden to be a leading
nation in scientific research.
R&D within the higher education sector in Sweden 2007
Medical sciences31%
Engineering23%
Natural sciences19%
Social sciences14%
Humanities7%
Agricultural sciences5%
Not categorised1%
Funding of medical research
• The direct government R&D funding differs between the scientific areas
• For medicine it is 46 percent and includes funding that goes directly to the university hospitals (”ALF-funds”)
• The rest of the funding comes from several other sources, making the funding picture complex for the medical area
Share of direct government R&D funding and external funding per subject area
Data from Statistics Sweden
0
10
20
30
40
50
60
70
80
90
100
Human
ities
Social
scien
ces
Med
icine
Natur
al sc
ience
s
Engine
ering
Agricu
ltura
l scie
nces
%
EU
Foreign non-profit organisations
Foreign companies
Swedish companies
Swedish non-profit organisations
University funds
Research foundations
County councils and municipalities
Research councils and governmentagencies
Direct government funding
Bibliometric analyses of the productivity and citation levels
• Overview of productivitiy – trends• Indicates weak and strong fields as well as areas• Citation statistics quality indicator for aggreated
data based on large numer of publications• Journals classified according to subject fields• 255 subject fields defined, 63 medical; 2 macro
fields: biomedicine and clinical medicine• Covering 6200 journals in medicine• Defined by Thompson Reuters in the Science
Citation Index database• 91% of publications from medical university
institutions including univ hospitals appear in medical journals
Citations of Swedish publications according to subject (as determined by journals/Thompson data base)
1980 1990 2000
0,6
0,8
1,0
1,2
1,4
1,6
1,8
2,0
2,2
Fäl
tnor
mal
iser
at m
edel
värd
e
Medicin
Biomedicin
Klin. medicin
1980 1990 2000
Naturvetenskap
Biologi
Kemi
Geovet.
Fysik
Teknik
1980 1990 2000
Ingenjörsvet.
ICT
Materialvetenskap
Citations, Nordic countries and the Netherlands – competitiveness
Source: Vetenskapsrådets rapportserie 13:2006
0,8
1,0
1,2
Medicin
0,8
1,2
1,6 Naturvetenskap
Fäl
tnor
mal
iser
ad m
edel
cite
ring
0,4
0,8
1,2
0,4
0,8
1,2 Samhällsvetenskap
1985 1990 1995 2000 2005
0,6
1,0
1,4
1,8Skogs- och jordbruksvet, veterinärmed.
0,8
1,2
1,6
2,0
1985 1990 1995 2000 2005
Teknik
DanmarkNorgeSverigeFinlandHolland
0,8
1,0
1,2
Medicin
0,8
1,2
1,6 Naturvetenskap
Fäl
tnor
mal
iser
ad m
edel
cite
ring
0,4
0,8
1,2
0,4
0,8
1,2 Samhällsvetenskap
1985 1990 1995 2000 2005
0,6
1,0
1,4
1,8Skogs- och jordbruksvet, veterinärmed.
0,8
1,2
1,6
2,0
1985 1990 1995 2000 2005
Teknik
DanmarkNorgeSverigeFinlandHolland
1985 1990 1995 2000 2005
Netherlands
Fäl
tnor
mal
iser
ad m
edel
cite
ring
Medicin
1990 1995 2000 2005
0,7
0,8
0,9
1,0
1,1
1,2
1,3
1,4
Sverige
DanmarkNederländerna
Schweiz
USAGreat Britain. 1,09Canada 1,06Finland 1,04
Swedish publications in medical journals – an international comparison
Australia 0,95Belgium 0,95Norway 0,95
Number of publications per year for the 15 largest countries, Switzerland, Denmark, Finland, Norway and Sweden.
Biomedicine
0
10 000
20 000
30 000
40 000
50 000
1980 1985 1990 1995 2000 2005
USA
EU15
J apanUK
0
3 000
6 000
9 000
1980 1985 1990 1995 2000 2005
Germany
France
Italy
Canada
China
Spain
South Korea
0
1000
2000
3000
1980 1985 1990 1995 2000 2005
Netherlands
N
umbe
r of
pub
licat
ions
per
yea
r
Australia
Turkey
Switzerland
Denmark
Norway
Sweden
Finland
Number of publications per year for the 15 largest countries, Switzerland, Denmark, Finland, Norway and Sweden cont’d
Clinical medicine
0
20 000
40 000
60 000
80 000
100 000
1980 1985 1990 1995 2000 2005
USA
EU15
UK
JapanGermany
0
5 000
10 000
1980 1985 1990 1995 2000 2005
France
Italy
Canada
Netherlands
Australia
Spain
0
1000
2000
3000
4000
5000
6000
1980 1985 1990 1995 2000 2005
China
N
um
ber
of
public
ati
ons
per
year
SwedenSouth Korea
Denmark
Turkey
Switzerland
NorwayFinland
Biomedicine
1985 1990 1995 2000 2005
0.8
0.9
1.0
1.1
1.2
1.3
1.4
1.5
1.6
USA
UK
Switzerland
NetherlandsGermanySingapore
1985 1990 1995 2000 2005
0.6
0.7
0.8
0.9
1.0
1.1Denmark
NorwaySweden
Finland
Clinical medicine
1985 1990 1995 2000 2005
0.8
0.9
1.0
1.1
1.2
1.3
1.4
1.5
USA
UK
SwitzerlandNetherlands
Belgium
1985 1990 1995 2000 2005
0.8
0.9
1.0
1.1
1.2
DenmarkSweden
NorwayFinland
Field normalised citation rate 1982 to 2006 for a selection of countries (3-year moving averages). Note the varying scale on the y-axis
Research policy
•Inquiries•Research Bills – the 4 year cycle
SCM: In preparation for the Research Bill 2008
• A long-term research strategy and increased state funding are needed if Sweden is to remain a leading research nation in the future.
• Long-term support of free, research-initiated research is important and should be provided on a broad front.
• All external state resources for health research in Sweden should be handled by a Scientific Council for Health Research in the Swedish Research Council.
• a clear, predictable career structure to attract the most suitable young people to academic research
• Prompt measures are needed to improve the scope for conducting clinical research and non-commercial clinical trials in Sweden.
• strengthening infrastructure for medical research to make it accessible for the very best research.
• Researchers of both sexes must be given the same opportunities of bring their full intellectual capacity forward in their work and to enable Sweden to attain more competitive, productive research.
• Sweden must, more unequivocally than to date, assume its international responsibility for research on major global issues, especially global health.
Clinical research – a great challengeClinical research – the link between the lab and the patient.
Quality of Swedish clinical research risks to fall back (currently #8 in the world), ”good breadth, but lack of cutting edge”
The need to train a new generation of clinical investigators and to boost translational research
The Government has commissioned an inquiry (”World Class”) regarding clinical research led by Prof. O. Stendahl. An action plan was presented in February 2008. A final report will be presented in March 2009.
Action plan clinical researchIncreased health focus and better implementation of research results
• Replace the current SCM with a Scientific council for health research
• Create a national program for implementation and innovation
• The resources should be allocated to the best quality research. The ”ALF-funds” must be followed up and evaluated.
• Develop leadership at the university hospitals• Create national research schools• Create new junior and senior positions for research• Increase support for clinical infrastructure
Recent governmental inquiries with impact on the Research Bill
Forskningsfinansiering – kvalitet och relevans
SOU 2008:30
Karriär för kvalitetSOU 2007:98
Resurser för kvalitetSOU 2007:81
More resources to the Universities needed: Allocation of resources based on activity and quality
Suggested new research council organization: Priority to applied research – present needs of society and industry
Career system: tenure track
Priority not given to basic research. Increased political demands of research outcome and what research should be performed.
”Ett lyft för forskning och innovation”
Regeringens forskningspolitiska proposition 2008/09:50
Research Bill 2008A boost to research and innovation
New resources for research and develpment 2009-2012
2009-2012
Direkta anslag till lärosäten 1 550 Strategiska satsningar: 1 800 varav medicin och hälsa 585 teknik 610 klimat och miljö 535 humsam 30 övrigt 40 Forskningsfinansiärer 675 Infrastruktur 150 Industriforskningsinstitut 200 Innovationspaket 150 Forskningsmoms 300 ESS 150 Regeringens disposition 25 Totalt 5 000
Källa: Faktablad Utbildningsdepartementet
Miljoner kronor
Direct funds to universitiesStrategic areas
Medicin and healthTechnologiesClimate and environm.Humanities and social sci.other
MSEK
Funding organizationsInfrastructure
Industrial research institutesInnovation package
Resarch taxesEuropean Spallation Source
Governement’s own disposalTotal
Vetenskapsrådets organisationOrganization of Swedish Research Council
• Utökat ansvarsområde Increased responsibility– Strategiska prioriteringar av grundforskning Strategic prioritization of basic research– Samordna satsningar mellan ämnesråden coordination of calls between councils
• Styrelsen The Board– Ny roll, utökat mandat och inflytande, ökade krav New role, increased mandate and
influence, increased demands– Ändrad sammansättning Change of composition (inte företräda olika forskningsområden, bred erfarenhet från
samhälle/industri, forskningsbakgrund) not represent research areas, broad experience
• Nya medel kommer till styrelsen ej fördelat på ämnesråd New funds to the board, not to the councils
• Ämnesråd – betoning av forskarrepresentation The councils ,emphasis on scientist representation(en ny roll, ämnesstrategisk) new role, strategic within medical research
• ÄR-M ombildas till Ämnesrådet för medicin och hälsa SRC medicine to become Scientific Council for Medicine and Health
• KFI blir ämnesråd Infrastructure committee to become a council• U kvar som kommitté Educational science remains, as a committee• Expertgrupp för oredlighet inrättas vid CEPN Expert panel for misconduct in
research will be established
Ämnesråd för medicin och hälsaScientific council for medicine and health• Utökat uppdrag – hela medicinska området och
vårdområdet Increased assignment for the whole medical and care area
• Koppla forskning till hälso och sjukvårdens och industrins behov Link research to the needs of the health system and industry
• Ämnesrådets sammansättning ändras, elektorsförfarande (företrädare för industri och sjukvård) The composition should change, include representatives from industry and health care. NB this is already the case
• Ingen tidsangivelse Not communicated when this should start
Ämnesrådet för medicin Swedish Research Council Medicine
• Allmänt om forskningsråden: ”beviljningsgraden bör ökas och projektstöden bör vara tillräckligt stora för att täcka alla kostnader i ett projekt”
• Research councils, in general: the approval rate should increase and the project support be large enough to cover all expenses of a project
• Ansvar för utvärdering av ALF-medel tillsammans med SBU• Evaluate resources allocated by County Councils, toghether with the
Swedish Council on Technology Assessment in Health Care • Utlysa, bedöma, rekommendera strategiska områden 2010-2012• Announce, evaluate and recommend funding for strategic areas 2010-2012
• Psykiatri (25 milj) och vårdvetenskap (20 milj) 2009 egen utlysning• Psychiatry (25 MESK) and Care sciences (20 MSEK) 2009 in separate call• ”VR uppmanas särskilt att uppmärksamma behovet av infrastruktur inom
området” SRC should especially consider needs in infrastructure in the areas
• Kliniska forskarskolor Clinical Doctoral Programmes• Strategiska områden Strategic areas
Universitet och högskolorUniversities and Higher Education Institutions• Ett anslag till forskning och utbildning på forskarnivå• One grant for research and education at doctoral level• Kraftigt ökade basresurser – 2,865 miljarder • Increased resources - 2,865 billion SEK• Tilldelas efter konkurrens• Awarded in competition
– Dels via strategiska forskningsområden – 1,315 Strategic areas– Dels mha indikatorer – 1,55 Using indicators/metrics
• Externa medel• Bibliometri
• Ska bli strategiska och göra egna prioriteringar Universities will be strategic and make their own priorities,more independent
• Ökad ansvar för lokal infrastruktur (exv motsv VR dyr och medeldyr utrustning)
• Increased responsibility for local infrastructure• Postdok-anställningar: 2 år (+ 1år) Post doc positions 2 yr• Full kostnadstäckning Full coverage of costs• Lärarundantaget kvar Teacher´s/scientist’s right to patent • ”genom ökade anslag till UoH respektive strategiska forskningssatsningar kommer
trycket på forskningsråd… att minska när det gäller reguljära ansökningar”
Research and Innovation Bill, 2009–2012 Strategic investments
SEK 1.8 billion (of the SEK 5 billion) will be allocated to what is planned to be a permanent, annual increase in appropriations to research in a number of strategically important areas.
Guiding criteria: 1. Research that can help to find solutions to important global
problems and challenges, 2. Areas in which Sweden is already conducting world-class
research. 3. Areas where companies in Sweden are already conducting
their own research and development,
Research and Innovation Bill, 2009–2012 Strategic investments
Medicine gets about 1/3 of the investments into strategic areas, i.e. SEK 585 million
• Molecular bioscience 190• Stem cells and regenerative medicine 65• Diabetes 70• Neuroscience 70• Epidemiology 25• Cancer 70• Psychiatry 25• Health care research 70
Research and Innovation Bill, 2009–2012
Other important proposals for medical research:
• The Scientific Council for Medicine will be transformed to a Scientific Council for Medicine and Health. This includes larger responsibilities for the whole health area.
• Follow up and evaluation of the research performed with ALF-funds
• Funding of research schools in clinical medicine
Scientific Council for Medicine
Scientific Council for Medicine - Strategy 2009-2012
Vision
The Scientific Council for Medicine within the Swedish Research Council is an internationally highly respected financing organization for medical basic research. The research supported is of highest scientific quality and of great impact for human health both at a national and an international level.
Scientific Council for Medicine - Goals
1. Increased support to the research of highest scientific quality2. Recruit and provide for the coming generation of researchers 3. Collaborate and coordinate resources
4. Promote a major increase in governmental funding of Swedish medical research
5. Promote the same possibilities and rights in medical research regardless of gender or ethnic origin
6. Strengthen public awareness about medical research and its results.
Scientific council for medicineGrants
•Responsive mode•Bottom-up-funding•Few programs
• Project grants
• Positions (junior and senior scientist)
Initiatives
Researcher-initated
Funding principles
•Responsive mode•Bottom-up-funding•Few programs
Grants for research projects by research area
REVIEW ORGANISATION
• Members of the review groups are highly qualified Swedish and foreign researchers.
• The scientific councils’ evaluation panels review applications, rank them on the basis of scientific quality and the competence of the applicants, and propose funding.
• Decisions are taken by the scientific council.
Scientific Council for Medicine
Council AdvisoryPanel (Rådsberedn)
Scientific Council for Medicine Chair: Ann-Marie Begler Secretary General: Håkan BilligDeputy Secretary General : Karin Forsberg NilssonCoordinator: Teresa Karlsson
Evaluation panels
Committee for Research Infrastructure
Number of project grant applications (including junior researcher postitions) (new and
continuation)
0
250
500
750
1000
1250
1500
k200
0
k200
1
k200
2
k200
3
k200
4
k200
5
k200
6
k200
7
k200
8
k200
9
Nu
mb
er
(1127)
(936)(1029)
(922)
Applications k2000-k2006 and running projects (granterade )
0
200
400
600
800
1000
1200
1400
k200
0
k200
1
k200
2
k200
3
k200
4
k200
5
k200
6
k200
7
k200
8
k200
9
An
tal
Nya
Omprövning
garanterade
New
Competetive
renewals
Granted
Project grants
0
5
10
15
20
25
30
35
40
45
2001 2002 2003 2004 2005 2006 2007 2008 2009
r
%
Samtliga
Nya
Omprvning
0
200
400
600
800
1000
1200
1400
0-5 6-10 10-15 16-20 21-25 26-30 31-35 >35
r e fte r disputation
An
tal a
ns
knin
gar
Kvinnor
Mn
Ratio of female scientists applying for project grants
Amount of male and female applicants by career age
0
5
10
15
20
25
30
35
40
45
50
2001 2002 2003 2004 2005 2006 2007 2008 2009
r
%
Kv innor
Mn
Sucess rate in approvals
0
5
10
15
20
25
30
35
40
2001 2002 2003 2004 2005 2006 2007 2008 2009
r
%
Kvinnor
Mn
0
10
20
30
40
50
60
70
80
90
2001 2002 2003 2004 2005 2006 2007 2008 2009
r
%
Kvinnor (nya)
Mn (nya)
Kvinnor (omprvning)
Mn (omprvning)
New and competitive renewals; applications from men and women
Approval rate the first five years after doctoral degree awarded
Women, new appl.Men, new applWomen, competitive renewalsMen, competitive renewals
WomenMen
Medelprojektbidrag (tkr)
583
663
200
250
300
350
400
450
500
550
600
650
700
k199
7
k199
8
k199
9
k200
0
k200
1
k200
2
k200
3
k200
4
k200
5
k200
6
k200
7
k200
8
k200
9
Med
elan
slag
(tk
r)
Project grants average grant size tKr
Average grant size
0
100000
200000
300000
400000
500000
600000
700000
800000
2001 2002 2003 2004 2005 2006 2007 2008 2009
r
Kro
no
r
Kvinnor
Mn
SEKWomenMen
The Swedish Research Council’s gender-equality strategy (p61)
General objectives and guidelines• Attaining an even gender distribution within the Council’s
own review organisation.• Attaining an even gender distribution in the allocation of
research support.• Active efforts to promote gender equality in the research
community.• Analyze outcome after evaluation panel meeting, motivate
priority list
Women in the evaluation panels(>40% since 2000)
Junior researcher position applicatons
0
50
100
150
200
250
300
350
400
k200
0
k200
1
k200
2
k200
3
k200
4
k200
5
k200
6
k200
7
k200
8
k200
9
Budgetår
An
tal
Ansökningar
Beviljade, prelApplied
Approved
Junior researcher positions - awarded
New junior researcher positions - Medicine 2000-2009
0
10
20
30
40
50
60
k2000 k2001 k2002 k2003 k2004 k2005 k2006 k2007 k2008
Ant
al
tot beviljat
kv beviljat
man beviljat
total
women
men
Development of the peer review process
SCM continuously works to develop the peer review system
• Transparency: – web based information– dialogue with the scientific community
• biannual road-shows)• regular meetings with the Deans• Information meetings for junior scientists
– SCM Magazine– SCM newsletter– web based nomination of evaluators since 2008 open to all (now open)
• Budget process– Working group of the SCM– SCM strategy meeting focused around budget process
• Rating system– Developed feedback to applicants
• Mean values of individual criteria scores (3 or 4)• Extended information on wpv mean/max/median score values of the panel
– New criteria since 2008 (result of the SCM working group for quality assessment)
– Scoring system presently investigated• Instructions to reviewers
– Improve written instructions– One day information meeting– Half day panel chair information
SCM Evaluation panels(before 2001 MRC)
• major changes 1992 and 2005• 1992: After 30 years of unchanged evaluation
panels – due to increased application load and change of
research profiles
• 2004-2005: SCM working group to design a model to adopt to disease-oriented review panel organization
New organization of evaluation panels from 2006
• To assure continued high quality assessment in a flexible organization where the number of applications directs number of panels and the allocation of grants to the disease studied is more visible
• Meet with stricter requirements for handling conflicts of interest
• Possibility of a new budget process
• Evaluation after three years, which is this year!
Evaluation panels
• 19 panels (before 13)• 5 reviewers per application out of which 1 is the
evaluator (same)• Every application is reviewed in an evaluation
panel
Recruitment of panel members from Sweden and the other Nordic countries • Panel members appointed by the Scientific
Council for medicine (same)• 2008: 39% new reviewers in the 17 panels (38%
2007)• NEW 2008: Web nomination of panel members• Appointment on 3 year-basis with a maximum 6
year term (Swedish scientists “step out” if they apply for project grants and can then come back the next year)
• Appoint Nordic experts as regular members of Swedish evaluation committees.
• Nordic experts may serve a 2x3 year term• Reviewers from Denmark, Finland, Iceland and
Norway constitute 20%
Evaluation panels 2008A1 Musculo-skeletal diseases, Oral health and Maxillofacial diseasesA2 Musculo-skeletal diseases, Anaesthesiology and Radiology
B1 Endocrinology/Metabolic diseases and Gynaecology, Reproduction/perinatalB2 Endocrinology/Metabolic diseases including Gastrointestinal diseases
C1 Infections and Global healthC2 Infections, Respiratory tract diseases and Allergy including DermatologyC3 Infections
D1 Nervous system diseases and PsychiatryD2 Nervous system diseases and PsychiatryD3 Nervous system diseases including Sensory Organs
E1 Cardiovascular and Urogenital diseases, Transplantation and diseases of haematogenous organs
E2 Cardiovascular and Urogenital diseases, Transplantation
F1 Basic disease mechanisms: Molecular, cellular and biochemical aspectsF2 Basic disease mechanisms: Molecular, cellular and biochemical aspectsF3 Basic disease mechanisms: Molecular, cellular and biochemical aspects
G1 Public Health and Care-Sciences/NursingG2 Public Health and Care-Sciences/Nursing
Collaboration grantsInfrastructure2 Post doc panels (spring and fall call)
How are the applications distributed over the panels 2008?
A Rörelseorganens sjukdomar, Oral hälsa, Käkens sjukdomar, Anestesiologi ,Radiologi
B Endokrinologi, Metabola sjukdomar, Gynekologi, Reproduktion/perinatal
C Infektion, luftvägarnas sjd, Allergi, Global hälsa, Hudsjukdomar
D Nervsystemets sjd, psykiatri, Sinnesorganen
E Hjärt-kärlsjukdomar, Blodsjukdomar, Urogenitala sjukdomar, Transplantation
F Grundläggande sjukdomsmekanismer; molekylära, cellulära och biokemiska aspekter
G Folkhälso- och vårdvetenskap
Disease studied: 61%Basic mechanisms of disease: 24%Care sciences and public health : 15%
Applications 2008
A9%
C13%
D16%
E11%
F24%
G14%
B13%
A
B
C
D
E
F
G
Number of project grant applications per panel 2006-2008
Antal ansökningar om projektbidrag per beredningsgrupp 2006-2008
0
10
20
30
40
50
60
70
80
A1 A2 B1 B2 C1 C2 C3 D1 D2 D3 E1 E2 F1 F2 F3 G1 G2
beredningsgrupp
an
tal
an
sökn
ing
ar
Projekt 2006Projekt 2007Projekt 2008
SCM decision for Evaluation panels 2009
• No major change awaiting the expert panel evaluation
• Project grant applications: A, B, C1-2, D1-2, E1-2, F1-3, G1-2
• Junior researcher position applications separate panel (as post doc applications)
• Contiune triage procedure, next year also for junior researcher positions
• Increase number of experts in Collaboration grant panel
• Add one expert to the infrastrucure panel (large database expert)
The review process
Council Advisory Paneland
Scientific Council
April June-Aug Aug-Sept Nov
Applicationssubmitted
Distributionof applications
Reviewing Evaluation panels
Nov 3Discuss and weigh the preliminary proposals of the evaluation panels
Nov 4Final decision
Dec
Communicationof decisions
May-June
”Triage” process of applications 2008• All applications are assessed by five panel
members• All applications will recieve the same quality
assessment by individual panel members.• Only applications with a reasonable chane to be
considered for funding will be discussed further at the panel meetings
• The margin to ”sort out” applications will be considerable
• These applicants will receive wpv and information about wpv in the panel, but not an individual written assessment
• The “triage” will allow more time for discussions • The “triage” will be evaluated after the application
process this year
Triage, how will it be done?1. All assessments, including preliminary evaluations, are reported for all
applications belonging to the evaluations panel in VR-Review no later than two week sbefore the panel meeting.
2. The administrstion provides a list with the joint rating of the applications to the panel chair who is responsible for conveying the information to the panel members (avilable in VR Review).
3. During a joint telephone meeting, the panel members agree on which applications should not be subject to further discussions at the regular meeting. The margin for sorting out applications shouldbe considerable The meeting should take place no later than a week before the regular panel meeting. All panel members must be present, otherwise the meeting is not valid.
4. At the panel’s regular meeting, the triaged applications are normally not subject for further discussions. It is however possible for a panel member to, at any time and without specific grounds, bring a triaged application back into the panel for discussion.
5. After the panel meeting, the preliminary evaluations for the triaged applications will not be finalized as evaluations. An applicant of a triaged project grant application will only receive the weighted point value and other potential information that is distributed to the applicants, but will not receive an individual written evaluation. A standard cover letter explaing the triage principle will be enclosed.
Project plan
Parameters to analyzeQualitative data• How has the new organization been received by the scientific
coummunity? • Experiences gained from questionnaires 2007 and 2008• Who should be approached - and how?
applicants, reviwers, panel chairs, the SCM, the secretariatquestionnaires, interview, web consultation
Quantitative data• Numbers of applications per reviewer, per panel• Approval rates with regard to scientfic area, gender, career age, etc
Peer review evaluation and development in other organizations
The Research Bill 2008 and peer review• Peer review as an instrument for measuring quality of scientific proposals• Demands on a Scientific Council for Medicine and Health. Can this be met
by the current organization? If not, what changes are needed?
45. Did you participate in an Evaluation Panel at the Swedish Reseach CouncilMedicine in 2005 or earlier, i.e. before the reorganization of Evaluation Panels?1. Yes 2. No
2007 2008
no
yes
46 Has your workload changed due to the reorganization of the grant review process? (2007)
1 Increased a lot (1) 2.0 % 2 Increased a little (2) 4.0 %3 Unchanged (4) 8.0 %4 Decreased a little (3) 6.0 %5 Decreased a lot (3) 6.0% 0 I did not participate in an Evaluation Panel before the reorganization (37) 74.0 % Total: (50)No. of missing replies: 186
2007 2008
47 How well do you think the new grant review process is working out in terms of categorizing the applications?1 Very well (7) 13.5 %2 (32) 61.5 %3 (7) 13.5 %4 (6) 11.5 %5 Very badly (0) 0.0 % Total: (52) No. of missing replies: 184
1 Very well (16) 23.5 %2 (28) 41.2%3 Neither well nor badly (12) 17.6 %4 (3) 4.4 %5 Very badly (1) 1.5 %0 No opinion (8) 11.8 % Total: (68) No. of missing replies: 178
2007 2008
42 This year (2008), a ”triage” process was tested. Only applications with a reasonable chance to be considered for funding were discussed further at the evaluation panel meeting. Did this change your workload? 1 Increased a lot (2) 2.9 % 2 (4) 5.9 % 3 Unchanged (20) 29.4 %4 (26) 38.2 % 5 Decreased a lot (4) 5.9 %0 No opinion (12) 17.6 % Total: (68)
43. What is your overall opinion of the triage procedure? (2008)1 Very good (32) 47.1 %2 (17) 25.0 %3 Neither good nor bad (6) 8.8 %4 (2) 2.9 %5 Very bad (3) 4.4 % 0 No opinion (8) 11.8 % Total: (68) No. of missing replies: 178
44 What is your overall opinion of this year’s peer-review process? (2008) NB Replies for reviewers at all councils and committees 1 Very good (86) 35.0 %2 (125) 50.8 % 3 Neither good nor bad (32) 13.0 %4 (3) 1.2 %5 Very poor (0) 0.0 %0 No opinion (0) 0.0 % Total: (246) No. of missing replies: 0
A. Approval rate comparison for women and men applicants
41,3
44,843,1
37,3
33,9
30,4
34,8 34,5
31
34,6
40,4
35,7
28,5 29,4
25,7 26,5
30,7
25,3
43,846,8 47
41,2
36,233,1
38,936,9
34,7
0
5
10
15
20
25
30
35
40
45
50
2001 2002 2003 2004 2005 2006 2007 2008 2009
year
%
Approval rate (%) Approval rate for women (%) Approval rate for men (%)
Figure 1a. Approval rate for women and men including all project grant applications, both competitive renewal and new applications (%).
Figure 1b. Approval rate for women and men for applicationsfor competitive renewal (%).
65,7
78,375,8 76,4 74,7 74,3
82,1 80,577,8
81,3 80,983,3
79,576
81,584 82,7
74,6
0
10
20
30
40
50
60
70
80
90
2001 2002 2003 2004 2005 2006 2007 2008 2009
year
%Approval rate for competitive renewal for women (%)
Approval rate for competitive renewal for men (%)
Figure 1c. Approval rate for women and men for new applications (%).
20,1
26,324,3
16,3
11,1
14,9 15,2 16,1
12,4
15,9
24,927,3
20,5
13,614,8
17,7 17,215
0
5
10
15
20
25
30
2001 2002 2003 2004 2005 2006 2007 2008 2009
Approval rate for new applications for women (%)
Approval rate for new applications for men (%)
B. Application statistics per evaluation panel over time
Figure 2a. Number of applications reviewed per evaluation panel before panel reorganisation (2001-2006) and after panel reorganisation (2007-2009).
Number of project grant applications per evaluation panel
0
20
40
60
80
100
120
140
160
180
2001 2002 2003 2004 2005 2006 2007 2008 2009
Cellbiology1
Cellbiology2
Cellbiology3
Public health and nursing
Clinical science 1
Clinical science 2
Clinical science 3
Clinical science 4
Medical chemistry
Microbiology and immunology 1
Microbiology and immunology 2
Odontology
Phychiatrics
System physiology and pharmacology
M-A1
M-A2
M-B1
M-B2
M-C1
M-C2
M-C3
M-D1
M-D2
M-D3
M-E1
M-E2
M-F1
M-F2
M-F3
M-G1
M-G2
Figure 2b. Approval rate per evaluation panel before panel reorganisation (2001-2006) and after panel reorganisation (2007-2009) (%).
Approval rate per evaluation panel
0
10
20
30
40
50
60
70
2001 2002 2003 2004 2005 2006 2007 2008 2009
Cellbiology1
Cellbiology2
Cellbiology3
Public health and nursing
Clinical science 1
Clinical science 2
Clinical science 3
Clinical science 4
Medical chemistry
Microbiology and immunology 1
Microbiology and immunology 2
Odontology
Phychiatrics
System physiology and pharmacology
M-A1
M-A2
M-B1
M-B2
M-C1
M-C2
M-C3
M-D1
M-D2
M-D3
M-E1
M-E2
M-F1
M-F2
M-F3
M-G1
M-G2
C. Statistics for subject areas (defined by the applicants)
Table 3a. Number of applications within specific subject areas (3-year intervals).
Number of applications within specific subject areas
0
50
100
150
200
250
300
350
400
450
Anasth
esiol
ogy,
inte
nsive
car
e an
d tra
uma
Bioch
emica
l stru
cture
and
met
abolis
m
Cance
r
Cell a
nd m
olecu
lar b
iolo
gy
Coagu
latio
n an
d th
rom
bosis
Derm
atol
ogy
Develo
pmen
tal b
iolog
y
Diabe
tes
Diges
tive
syst
em a
nd kid
ney
Drugs
and
dru
g ad
dicti
on
Endoc
rinolog
y
Enviro
nmen
tal m
edici
ne a
nd to
xicolo
gy
Gen
etics
Heart
and
blood
vess
els
Gen
eral
medic
in
Micr
obio
logy,
imm
unol
ogy a
nd in
fecti
ous
Mus
culo
skel
etal
syst
em
Nervo
us sys
tem
Nursin
g
Odo
ntol
ogy
Pharm
acy
Prena
tal a
nd p
erin
atal
dise
ase
Phsyc
iatri
c dise
ase
Public
hea
lth
Radio
logy
and
imag
ing
tech
nique
s
Repro
duct
ive sy
stem
Respir
ator
y sys
tem
Senso
ry o
rgans
Subject area (defined by applicant)
Nu
mb
er
of
ap
plic
ati
on
s
2001-20032004-20062007-2009
Figure 3b. Distribution of applications in specific subject areas (3-year intervals)
Distribution of applications in subject areas (%)
0,00
2,00
4,00
6,00
8,00
10,00
12,00
14,00
16,00
18,00
Anasth
esiol
ogy,
inte
nsive
car
e an
d tra
uma
Bioch
emica
l stru
cture
and
met
abolis
m
Cance
r
Cell a
nd m
olecu
lar b
iolo
gy
Coagu
latio
n an
d th
rom
bosis
Derm
atol
ogy
Develo
pmen
tal b
iolog
y
Diabe
tes
Diges
tive
syst
em a
nd kid
ney
Drugs
and
dru
g ad
dicti
on
Endoc
rinolog
y
Enviro
nmen
tal m
edici
ne a
nd to
xicolo
gy
Gen
etics
Heart
and
blood
vess
els
Gen
eral
medic
in
Micr
obio
logy,
imm
unol
ogy a
nd in
fecti
ous
Mus
culo
skel
etal
syst
em
Nervo
us sys
tem
Nursin
g
Odo
ntol
ogy
Pharm
acy
Prena
tal a
nd p
erin
atal
dise
ase
Phsyc
iatri
c dise
ase
Public
hea
lth
Radio
logy
and
imag
ing
tech
nique
s
Repro
duct
ive sy
stem
Respir
ator
y sys
tem
Senso
ry o
rgans
%
2001-20032004-20062007-2009
Figure 3c. The approval rate of applications within each subject area (%).
Approval rate within subject areas
0
10
20
30
40
50
60
70
Anasth
esiol
ogy,
inte
nsive
car
e an
d tra
uma
Bioch
emica
l stru
cture
and
met
abolis
m
Cance
r
Cell a
nd m
olecu
lar b
iolo
gy
Coagu
latio
n an
d th
rom
bosis
Derm
atol
ogy
Develo
pmen
tal b
iolog
y
Diabe
tes
Diges
tive
syst
em a
nd kid
ney
Drugs
and
dru
g ad
dicti
on
Endoc
rinolog
y
Enviro
nmen
tal m
edici
ne a
nd to
xicolo
gy
Gen
etics
Heart
and
blood
vess
els
Gen
eral
medic
in
Micr
obio
logy,
imm
unol
ogy a
nd in
fecti
ous
Mus
culo
skel
etal
syst
em
Nervo
us sys
tem
Nursin
g
Odo
ntol
ogy
Pharm
acy
Prena
tal a
nd p
erin
atal
dise
ase
Phsyc
iatri
c dise
ase
Public
hea
lth
Radio
logy
and
imag
ing
tech
nique
s
Repro
duct
ive sy
stem
Respir
ator
y sys
tem
Senso
ry o
rgans
% 2001-20032004-20062007-2009
D. Statistics based on applicant age
Figure 4a. Number of applications received from different age categories.
Applicant age distribution (years after PhD)
0
100
200
300
400
500
600
700
800
0-5 6-10 11-15 16-20 21-25 26-30 31-35 >35
Years after PhD
Nu
mb
er o
f ap
plic
ants
2001-2003
2004-2006
2007-2009
Figure 4b. Approval rate for different age categories.
Approval rate for age categories
0
10
20
30
40
50
60
70
80
0-5 6-10 11-15 16-20 21-25 26-30 31-35 >35
Years after PhD
Ap
pro
va
l ra
te %
2001-2003
2004-2006
2007-2009
Milestones/time lines
January Start questionnaires/interviews
Web based consultation opens
February 3rd SCM meeting Interim report (presentation of project outline)
February (mid) Panel phone conference
February (end) Questionnaries concluded, web consultation closes
March Meeting with representatives of panels etc
April Analysis of data, drafting of report
May (early) Panel conference to finalize report
May 18-19th SCM strategy meeting presentation of final report
Immediate action plan:
Reviewing and scoring (p42)
Project research grant• Project 1-7• Feasiblity 1-7• Project management 1-7• Results (only for competitive renewals) 1-7
Each criterion shall be evaluated and scored independently.
ProjectThis dimension relates to the overall design of the project, its
research questions and hypotheses, and any influence it may exert on scientific knowledge. Is the project original or groundbreaking? Does it challenge prevalent opinions or practice? Does the project include an innovative hypothesis or seek to remove key barriers to further progress in the research field concerned? Does the project entail new ideas, approaches or interpretations? If the aims of the project are attained, will it substantially enrich knowledge in the field concerned or our understanding of health and ill-health?
Score: 1–7
7 = Outstanding, innovative project of highest scientific importance
1=Project with no scientific relevance and/or of poor scientific quality
Feasibility• This aspect includes the materials, methods, resources, finances and
time schedule of the project. Is the proposed procedure optimal in terms of scope for implementing the project, when it comes to general design, choice of methods, equipment, work inputs and time? Are the total resources realistic in relation to the scale of the project described? Does the methodological know-how exist to implement the project in the manner describes? Are the materials, patients and methods well adapted to the hypothesis or research question concerned? Are the numbers of measurements, tests, animals, patients or experimental subjects optimal for tackling the research issue addressed? Whether the materials and methods are appropriate for the purpose should be regarded as more important than the uniqueness of the method as such or the material. If unique material is not required or if well-established methods are fully adequate, this is not a disadvantage. The key aspect is whether the research question is well addressed in terms of methodology, so that the project yields reliable answers with minimal resource consumption. What are the strengths and weaknesses of the route chosen? Are there any alternative routes to the destination in question if the one selected does not lead to success? Are the separate subprojects well founded and properly integrated with the overall objectives of the project?
7 = Materials/patients and methods, resources, infrastructure and scientific environment are all optimal and aims are well integrated
1= The project is unrealistic
Project management• This aspect includes the skills and qualifications of the applicant and other people
involved, and also the research environment and collaboration. The project manager's skills weigh very heavily in the assessment. Does the applicant have the independence, experience and contact network that are required for the project to be implemented in the manner, and during the period, specified in the application? Do the applicant's previous publications show a distinct line of research? Independence is most clearly evident when applicants have publications to show in which they are the senior authors; when they have acted as supervisors (especially principal supervisors) to PhD students up to their disputations (public thesis defences); and when they have supervised doctoral students. For researchers still in the early stages of their careers, signs of independence also include, for example, publications as first author, preferably without participation of their own supervisors, and the existence of PhD or postdoctoral students of their own. Having been awarded grants of their own as principal applicants following peer review may also indicate independence. In applications from researchers who belong to teams or groups, it should be clearly apparent how the project in question is distinguished from the other projects under way in the team or group. It is common for young researchers to liberate themselves and develop research lines of their own by hiving off from larger projects. For the assessment of independence, it is essential for both junior and senior researchers to describe the boundaries between these projects clearly in their applications.Do the team members working on the project complement one another in their skills in an optimal way? Are the roles and responsibilities of all the project staff clearly described? Is there a critical mass of researchers in the environment concerned? An account of these aspects under a separate heading, ‘Project management', in the research plan facilitates a favourable assessment
Score: 1–77 = Established/Independent PI with expertise in the research
area, performing world-leading research1= PI lacking independence, or independent PI without
knowledge in the research area
Results (only for competitive renewals)
• The concern here is how far the project has, during the grant period, significantly contributed to an increase in knowledge in the field of medical research. Scientific publications in international journals are the key measure of the findings and importance of a project, and the assessment emphasises quality rather than quantity. Assessment focuses primarily on publications relevant to the project concerned. Other indicators of results and their importance may, for example, be amendments to national or international guidelines for medical diagnostics or treatment.
Score: 1–77= Top results including key authorship of several published
articles of highest scientific quality1= No published articles or other results of scientific relevance
Types of grants in Medicine 2008 (p5)
• Project Research GrantsThis form of grant is intended to cover the performance of a defined research task. The maximumgrant period is five years, but up to three years is the norm. The grant may comprise funding fortechnical staff, salaries for other staff, equipment up to a maximum cost of SEK 170 000, materials,publication costs and travel. Funding for his/her own salary cannot be included.In the subject area ofMedicine, a person can be the principal applicant for only one approved Project Research Grant in anygiven period.
• Grant for Junior Research Position in general fieldsJunior Research Positions are obtainable in all the fields supported by the Scientific Council forMedicine. The grant period is four years. A person applying for a Junior Research Position Grantautomatically applies for a Project Research Grant and a Starting/Establishment Grant. The amount ofthe Project Research Grant for successful applicants for Junior Research Positions in 2008 is SEK 325000 a year. The Starting/Establishment Grant, which is paid during the first year, totals SEK 500 000.
• Grant for Junior Research Position in Care SciencesThis Junior Research Position Grant relates to research in the priority field of Care Sciences. This fieldcomprises theory, methods and techniques for the study of problems and measures in healthcare andmedical contexts related to human health, quality of life and capacity for activity, and also thehealthcare and social care of people in various situations and settings. A Project Research Grant and aStarting/Establishment Grant is included as above.
• Grant for Junior Research Position in a Clinical Environment, 50%This Junior Research Position Grant relates to research carried out by people who have obtained PhDsand are working in clinical settings, such as doctors, nurses, physiotherapists, occupational therapistsand dentists. The position must be combined with half-time employment in a clinical environment. AProject Research Grant and a Starting/Establishment Grant is included as above.
• Collaboration Grant for translational and interdisciplinary researchThe purpose of this grant is to expand and strengthen mutual collaboration among researchers invarious fields, including clinical research. Three to five researchers (one coordinator and two to fourparticipating researchers) apply for the grant. At least one of the applicants must be a ‘youngresearcher’ (who obtained his or her PhD not more than six years ago). The majority of the applicantsshould come from the same HEI. One or more of the participants may come from an HEI abroad. Themajority of the applicants must have current research funding obtained in national or internationalcompetition.
Effekt 2008-2012 av periodisering av anslag 2008 (olika bidragsformer)
0,00
50,00
100,00
150,00
200,00
250,00
300,00
2008 2009 2010 2011 2012
Mil
j kr
Projekt - en huvudman
Samverkandbidrag
startbidrag (forskare + foass)
Forskartjänster (hel +halv)
postdoc + repatriering
Foasstjänster
Internationalisering (gäst+postdoc+konf)
Dyrbarutr
Blå toner = projekt,
Gröna toner = tjänster,
gul = international-isering,
röd =medeldyr-utrustning
Evaluation Policy 2008, Medicine
• The number of projects financed by the Scientific Council’s own budget should not increase
• The average funding level should increase
• The grant period should be 3 (to 5) years
• Funds should be allocated to individuals using the principle of one project per principal investigator