PaentCentered+Outcomes+Research+Ins#tute+(PCORI)+ … Phase II Scientific Reviewer... ·...
Transcript of PaentCentered+Outcomes+Research+Ins#tute+(PCORI)+ … Phase II Scientific Reviewer... ·...
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
October 2012
Pa#ent-‐Centered Outcomes Research Ins#tute (PCORI) Phase II Scien4fic Reviewer Training
2
Housekeeping – Presenta4on Mode
Ø A9endee phone lines are muted Ø Ques#ons may be submi9ed via Chat in
the lower right hand side of your screen à
Ø Please send ques#ons as they occur to
you. They will be answered at the end of the session, as #me permits
Ø Press “0” on the phone for a private help session with the operator
1. Type your question here.
2. Click Send
3
Agenda
1. Introduc4on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
4
Our SROs
Parag Aggarwal, Ph.D.
Howard Underwood, MD, MBA, MS Jessica Nadler, Ph.D.
Assessment of Preven4on, Diagnosis, and Treatment Op4ons
Lev Nevo, MD Sabina I. Robinson, Ph.D.
Improving Healthcare Systems
Kimberly A. Marschhauser, Ph.D.
Marianne H. Alcia#, Ph.D.
Communica4on and Dissemina4on Research
Addressing Dispari4es
5
Announcements
Open session to any per4nent announcements
Phase II Assignments Released – Oct 12th Preliminary Scores Due – November 2nd at 5:00 pm Op#onal Dinner – Nov 14th in Washington, DC from 6:00 pm – 9:30 pm Phase II Panels – Nov 15th in Washington, DC from 7:00 am – 3:00 pm
Key Dates
6
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
7
PCORI Mission, Vision and PCOR
PCORI Mission Statement
PCORI helps people make informed healthcare decisions and improves healthcare delivery and
outcomes by producing and promo#ng high integrity, evidence-‐based informa#on that comes from
research guided by pa#ents, caregivers, and the broader health care community.
Ø PCORI is a non-‐governmental, non-‐profit organiza#on founded by the Pa#ent Protec#on and Affordable Healthcare Act of 2010 (h9p://www.pcori.org/assets/PCORI_EstablishingLeg.pdf)
Ø PCORI aims to fund pa#ent-‐centered
research that will improve healthcare outcomes for pa#ents, their caregivers, and other stakeholders
Ø Pa4ent-‐centered outcomes research
(PCOR) helps people and their caregivers communicate and make informed healthcare decisions, allowing their voices to be heard in assessing the value of healthcare op#ons
For more informa#on on PCOR, please reference the PCORI Methodology report at: h9p://pcori.org/assets/MethodologyReport-‐Comment.pdf
Vision
Pa#ents and the public have the informa#on they need to make decisions that reflect their desired
health outcomes.
8
Provide informa#on to PCORI that informs future itera#ons of na#onal research priori#es for pa#ent-‐centered outcomes research.
The Na#onal Priori#es are:
1. Compara#ve Assessments of Preven#on, Diagnosis, and Treatment Op#ons
2. Improving Healthcare Systems
3. Communica#on and Dissemina#on
4. Addressing Dispari#es 5. Accelera#ng Pa#ent-‐
Centered and Methodological Research
Purpose
Support the collec#on of preliminary data to advance the field of pa#ent-‐centered outcomes research, providing the plagorm for an evolving PCORI research agenda. PCORI’s ini#al Research Agenda:
1. Comparisons of Preven#on, Diagnosis, and Treatment Op#ons
2. Improving Healthcare Systems 3. Communica#on & Dissemina#on 4. Addressing Dispari#es 5. Accelera#ng Pa#ent-‐Centered and
Methodological Research
Research Agenda
Support iden#fica#on of research methodologies that advance pa#ent-‐centered outcomes research
Methodologies
PCORI’s Na4onal Priori4es
9
Why PCORI is Unique
PCORI is unique because: (a) it requires stakeholders included as part of the research team, and (b) research must be focused on pa4ent-‐centered outcomes
• Projects must include stakeholders as partners with significant involvement at all appropriate stages of the research project
• Tangible, meaningful outcomes are the ul#mate goal of all funded research
Who are Stakeholders?
• Pa#ents and caregivers
• Pa#ent and caregiver organiza#ons
• Clinician and clinician organiza#ons
• Organiza#onal Providers
• Purchasers
• Payers
• Industry
• Researchers
• Policymakers
• Training ins#tu#ons
• Others who can bring insight
10
Stakeholder Engagement
PCORI is seizing the opportunity to engage stakeholders in unprecedented ways:
To create more relevant decision-‐making tools to assure
be9er pa#ent outcomes
Why Engage Stakeholders?
• Partners in the research project enterprise
• Inclusion as equal partners in research review
• Leverage their value, including wisdom and unique exper#se
• Increase the relevance and impact of research by integra#ng mul#ple stakeholders into the process
• Foster environments that facilitate cross-‐fer#liza#on and novel collabora#ons
11
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
12
PCORI Funding Announcements
The current funding cycle has four issued PFAs:
PCORI Funding Announcements (PFAs) are the mechanisms
Accelera'ng Pa'ent-‐Centered Outcomes
Research and Methodological
Research
And coming this fall 2012:
by which PCORI gives out research funding
Assessment of Preven4on, Diagnosis, and Treatment Op4ons
Improving Healthcare Systems
Communica4on and Dissemina4on Research
Addressing Dispari4es
13
PFAs
Assessment of Preven4on, Diagnosis, and Treatment Op4ons
Projects that address cri4cal decisions that pa4ents, their caregivers, and clinicians face with too lible informa4on
In this PFA we seek to fund projects that: • Address cri'cal decisions that face pa'ents, their
caregivers, and clinicians every day and with too li?le informa'on
• Address consequen'al decisions now occurring without key evidence about the compara've effec'veness of two or more op'ons
• Benefit pa'ents/caregivers with new knowledge in ways that are clear and important
Available funds: $48 Million
Expected awards: 54 awards
Maximum project period: 3 years
Ini#al funding period: December 2012 – January 2013
14
PFAs
Improving Healthcare Systems
Projects that address cri4cal decisions that face healthcare systems, the pa4ents and caregivers who rely on them, and the clinicians who work within them
In this PFA we seek to fund projects that: • Address cri'cal decisions that face healthcare
system leaders and policymakers, clinicians, and the pa'ents and caregivers who rely on them
• Offer substan'al poten'al that pa'ents/caregivers will benefit from the new knowledge in ways that are important
Available funds: $24 Million
Expected awards: 27 awards
Maximum project period: 3 years
Ini#al funding period: December 2012 – January 2013
15
PFAs
Communica4on and Dissemina4on Research
Projects that address cri4cal elements in the communica4on and dissemina4on process among pa4ents, their caregivers and clinicians
In this PFA we seek to fund projects that: • Address cri'cal knowledge gaps in the
communica'on and dissemina'on process
• Gaps to consider: Ø The communica'on and dissemina'on of
research results to pa'ents, their caregivers, and clinicians
Ø The communica'on between pa'ents, caregivers, and clinicians in the service of enabling pa'ents and caregivers to make the best possible decisions in choosing among available op'ons for care and treatment
Available funds: $12 Million
Expected awards: 14 awards
Maximum project period: 3 years
Ini#al funding period: December 2012 – January 2013
16
PFAs
Addressing Dispari4es
Projects that will inform the choice of strategies to eliminate dispari4es
In this PFA we seek to fund projects that: • Will inform the choice of strategies to eliminate
dispari'es Ø We are not interested in studies that
describe dispari'es; instead, we want studies that will iden'fy best op'ons for elimina'ng dispari'es
• Focus on areas of importance to pa'ents and their caregivers, where there are cri'cal dispari'es that disadvantage members of a par'cular group and limit their ability to achieve op'mal, pa'ent-‐centered outcomes
Available funds: $12 Million
Expected awards: 14 awards
Maximum project period: 3 years
Ini#al funding period: December 2012 – January 2013
17
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica4on and Review Process
8. Phase II: PCORI Online – Process and Procedures
18
Applica4on and Review Process Summary
Applica4on Merit Review Approval
• LOI submission via PCORI Online
• Applica#on submission via PCORI Online
• Internal quality control
• Phase I: Scien#fic/ Technical Review
• Phase II: In-‐person panel – Impact Review
• PCORI Business Review and Balance Analysis
• Board of Governors Approval
19
The Merit Review Process
The process by which applica4ons for research funding are evaluated –
Phase I • Each applica#on is assigned to a pre-‐
determined, set number of reviewers
• Phase I Reviewers have scien#fic exper#se, and assess the applica#on for scien#fic rigor and research approach
• Reviewers assign an ini#al priority score of 1 to 9 based on all eight PCORI merit review criteria
• Scores are compiled • Top scoring applica#ons proceed to
Phase II
Phase II • Scien#st and Pa#ents/Stakeholders
assess Phase I cri#que and assign one (1) preliminary score and cri#que
• Panel of chairs plus two scien#fic, one stakeholder, and one pa#ent reviewer convene in-‐person for discussion and re-‐score
20
Merit Review Phase II: Overview
As a Phase II Reviewer, you are responsible for assessing and appropriately scoring your assigned applica4ons
Access Assigned Cri4ques & Scores
Assign Preliminary Score
& Cri4que
Key Tasks
Raise issues, risks, and request support as needed
1. Access Phase II scores and cri#ques in PCORI Online 2. Conflict of Interest (COI): Ensure no conflict exits 3. Assign preliminary numerical preliminary impact score (1-‐9) and provide cri#que 4. Panels convene and discuss 5. Assign final impact scores
In-‐Person Review Panels
Final Impact Scoring
COI
21
Conflicts of Interest
What is a conflict of interest? As defined by PCORI’s establishing legisla#on, a conflict of interest is any “associa#on, including a financial or personal associa#on; that has the poten#al to bias or have the appearance of biasing an individual’s decisions in ma9ers related to the Ins#tute or the conduct of ac#vi#es”. Conflicts of interest will be considered and prohibited throughout every step of the review and selec#on process, including but not limited to: the technical and programma#c reviews, the selec#on and assignment of scien#fic and stakeholder reviewers, Board of Governors delibera#ons, and post-‐award nego#a#ons and monitoring.
More informa4on is included in the PCORI Online confiden4ality and conflict of interest document that you must agree to in order to access your applica4ons.
22
Your Role
Some addi4onal guidance about your role and ac4vi4es as a Phase II Scien4fic Reviewer:
Before the in-‐person review panels on November 15th: • Access your assigned applica#ons in PCORI Online • Score and provide wri9en comments • Be prepared to substan#vely qualify and discuss your score and comments during
the in-‐person review panel
During the in-‐person review panel: • Reviewers assigned to each applica#on will briefly discuss their preliminary score
and provide feedback • Open to panel-‐wide discussion • All reviewers assign a final score in real-‐#me, on personal laptops via PCORI
Online • Scores are compiled and averaged by PCORI
• Top scoring applica#ons will be forwarded and receive recommenda#on for funding
23
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien4fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
24
Merit Review Criteria
• Impact of the Condi4on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Criterion 1: Impact of the Condi?on on the Health of Individuals and Popula?ons
§ Does the applica#on specify the burden of the disease or area under considera#on, with a preference for the U.S. popula#on, including:
Ø The frequency of the disease/condi#on, Ø Expected mortality and burden of suffering from symptoms,
Ø Complica#ons or other consequences of the disease/condi#on, Ø The frequency with which the interven#on or treatment would
apply, Ø Costs to the US popula#on (healthcare services u#liza#on), and
to individual pa#ents (out-‐of-‐pocket and intangible costs).
§ Primary emphasis is on chronic condi#ons, as well as preven#on and treatment of common acute events that may have long-‐term consequences.
§ Studies that are relevant to pa#ents with two or more condi#ons are also of interest. Also of interest are rare diseases.
25
Criterion 2: Innova?on and Poten?al for Improvement Through Research
§ How will the research influence current prac#ce and lead to meaningful improvement in pa#ent health, well-‐being, or quality of care?
§ Does the research involve a novel interven#on or employ an innova#ve approach in terms of analy#cs, study popula#on, or research team that makes it more likely to change prac#ce?
§ Does preliminary data suggest that the comparison will show large differences in effec#veness?
§ Does the research ques#on address a cri#cal gap in current knowledge? Has it been iden#fied as important by pa#ent, caregiver, or clinician groups? Have other agencies iden#fied this topic as a priority?
§ How quickly could posi#ve findings be disseminated to affect changes in current prac#ce? How will the research findings support improved decision-‐making for pa#ents?
• Impact of the Condi#on
• Innova4on/ Poten4al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
26
Criterion 3: Impact on Healthcare Performance
§ What is the impact of the proposed research on the efficiency of pa#ent care, for individual pa#ents or for pa#ent popula#ons?
§ For example, do the findings lead to be9er outcomes for a given investment of #me, personnel, or other resources? Or does the research promise poten#al improvements in convenience or elimina#on of wasted resources, while maintaining or improving pa#ent outcomes?
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
27
Criterion 4: Pa?ent-‐Centeredness
§ Is the proposed research focused on ques#ons and outcomes of specific interest to pa#ents and their caregivers? Pa4ent-‐centeredness is a perspec4ve on health that is derived from and directly relevant to the pa4ent’s experience of illness and of care.
§ Does the research address one or more of the key ques#ons men#oned in PCORI’s defini#on of pa#ent-‐centered outcomes research?
§ Are the outcomes proposed of importance to pa#ents? Is the absence of any par#cularly important outcomes discussed?
§ Pa#ent engagement in the research team is dis#nct and discussed in Criterion 7, Team and Environment.
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa4ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
28
Criterion 5: Rigorous Research Methods
§ Does the research use appropriate and rigorous research methods to generate pa#ent-‐centered evidence?
Ø Applicants are encouraged to refer to the contents of the first dras of the PCORI Methodology Report, at h9p://www.pcori.org/what-‐we-‐do/methodology, in developing their research plan. Because the dras report will not have been finalized with the benefit of public comment before the July 31st, 2012 applica#on deadline, adherence to the Report’s standards will not be a required element of applica#ons for this funding cycle.
Ø How likely is it that the proposed study popula#on, study design, and available sample size will yield generalizable informa#on with sufficient precision to be useful and reliable for pa#ents, their caregivers, and clinicians?
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
29
Criterion 6: Inclusiveness of Different Popula?ons
§ Does the research include diverse popula#ons with respect to age, gender, race, ethnicity, geography, or previously understudied popula#ons for whom effec#veness informa#on is par#cularly needed? Is the study popula#on representa#ve of the full popula#on of interest?
§ How does the proposed research enable a more personalized approach to decision-‐making based on a pa#ent’s unique biological, clinical, or socio-‐demographic characteris#cs?
§ Does the study provide sample size calcula#ons that will describe the power available to evaluate possible differences in effec#veness in different groups, or the precision available for es#ma#ng effec#veness in a specific previously understudied popula#on?
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula4ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
30
Criterion 7: Team and Environment
§ Are the inves#gators appropriately trained and experienced to carry out the planned studies? Is the work proposed appropriate to the experience level of the principal inves#gator?
§ Does the study team have complementary and integrated exper#se; is their leadership approach, governance, and organiza#onal structure appropriate for the project?
§ Are relevant pa#ents and other key stakeholders of the study informa#on appropriately included on the team?
§ Do the experiments proposed take advantage of unique features of the scien#fic environment or employ useful collabora#ve arrangements?
§ Is there evidence of ins#tu#onal or other support?
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
31
Criterion 8: Efficient Use of Research Resources
§ Does the budget appear to be reasonable in rela#on to the poten#al contribu#on of the research?
§ Does the jus#fica#on address the efficiency with which PCORI resources would be used? Are there opportuni#es to make the study more efficient?
§ Are there addi#onal benefits to a PCORI investment in this study through the crea#on of common data or infrastructure that could support future research?
• Impact of the Condi#on
• Innova#on/ Poten#al for Improvement
• Impact on Healthcare Performance
• Pa#ent-‐Centeredness
• Rigorous Research Methods
• Inclusiveness of Different Popula#ons
• Team and Environment
• Efficient Use of Resources
Merit Review Criteria
32
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
33
The Focus on Impact
PCORI Defini4on
Reviewers will provide an overall impact score that considers the following: • Does the project have poten4al to
change clinical prac4ce or pa4ent behavior in ways that will create and sustain improvement in outcomes and the health of pa4ents?
• How quickly can the results of the project be disseminated and applied (from the assessment of dissemina4on and implementa4on poten4al)?
The assessment of impact is par4cularly informed by three of the eight PCORI Merit Review Criteria. Use criteria 2, 4, and 7 to evaluate an applica4on’s answer to these ques4ons
34
The 8 Merit Review Criteria: Phase II Focus
Phase II is centered around impact, focusing on the following of the Merit Review Criteria:
The 8 Merit Review Criteria:
1. Impact of the Condi#on
2. Innova4on/Poten4al for Improvement
3. Impact on Healthcare Performance
4. Pa4ent-‐Centeredness
5. Rigorous Research Methods
6. Inclusiveness of Different Popula#ons
7. Team and Environment
8. Efficient Use of Resources
• Innova#on – in
ways that are likely to change prac#ce?
• Poten#al for improvement (will findings improve pa#ent well-‐being or quality of care?)
Innova4on/ Poten4al for Improvement
Pa4ent Centeredness
Team and Environment
• Inves#gators
trained • Study team
exper#se • Plan for leadership
and governance • Robust pa#ent
and stakeholder engagement plan
• Inclusiveness of different popula#ons
• Ins#tu#onal or other relevant organiza#onal support
• Focus on
ques#ons and outcomes of specific interest to pa#ents and their caregivers
35
Phase II Key Focus Areas
Criterion 2: Innova?on and Poten?al for Improvement Through Research
• “Is there uncertainty?” – Varia#on in prac#ce, systema#c reviews have iden#fied as such, or pa#ent/clinician groups have
specifically called for this informa#on
• How will the research influence current prac#ce and lead to meaningful improvement in pa#ent health, well-‐being, or quality of care?
• Does the research involve a novel interven#on or employ an innova#ve approach in terms of analy#cs, study popula#on, or research team that makes it more likely to change prac#ce?
• Does preliminary data suggest that the comparison will show large differences in effec#veness?
• Does the research ques#on address a cri#cal gap in current knowledge? Has it been iden#fied as important by pa#ent, caregiver, or clinician groups? Have other agencies iden#fied this topic as a priority?
• How quickly could posi#ve findings be disseminated to affect changes in current prac#ce? How will the research findings support improved decision-‐making for pa#ents?
– PCORI is interested in funding studies with a high likelihood that results will be disseminated and incorporated into prac#ce immediately or within a short period of #me (3-‐ 5 years). Please refer to the dissemina#on and implementa#on assessment in the applica#on for detail and clarifica#on, if necessary.
36
Phase II Key Focus Areas
Criterion 4: Pa?ent Centeredness
• Is the proposed research focused on ques#ons and comparisons that have relevance and specific interest to pa#ents and their caregivers? Pa4ent-‐centeredness is a perspec4ve on health that is derived from and directly relevant to the pa4ent’s experience of illness and of care.
• Does the research fit with one or more of the key ques#ons men#oned in PCORI’s defini#on of pa#ent-‐centered outcomes research?
• Are the outcomes proposed of importance to pa#ents? Is the absence of any par#cularly important outcomes discussed?
• Note: Pa#ent engagement in the research team is dis#nct and discussed in Criterion 7, Team and Environment.
37
Phase II Key Focus Areas
Criterion 7: Team and Environment
• Are the inves#gators appropriately trained and experienced to carry out the planned studies? Is the work proposed appropriate to the experience level of the principal inves#gator?
• Does the study team have complementary and integrated exper#se; is their leadership
approach, governance, and organiza#onal structure appropriate for the project? • Are relevant pa#ents and other key stakeholders in the study informa#on appropriately
included on the team? • Do the proposed experiments take advantage of unique features of the scien#fic environment,
or employ useful collabora#ve arrangements? • Is there evidence of ins#tu#onal support?
38
Phase II Preliminary Scoring
Assignments Released
• Assignments are released in October
• In PCORI Online
• Login to access your applica#ons – Ensure no conflicts of interest,
and your qualifica#on to review
• All applica#ons re-‐released and assigned to Phase II Reviewers were top scorers in Phase I – Have been ve9ed for basic, hard
science and programma#c review
Preliminary Scoring
• Use Phase I cri#ques to assign preliminary score of 1-‐9 – You will have access to the full
applica#on, but please use only to reference
• Provide substan#ve, produc#ve
comments as well as specific strengths and weaknesses to ul#mately help answer the ques#on: – “How, and to what extent, will the
proposed research plan impact pa'ents in the next 3 to 5 years?”
• Enter comments and numerical score in PCORI Online
39
Reviewer Guidance & Scoring Chart
For the preliminary Phase II impact score, the far right column in the scoring chart below provides a descrip4ve guide of how strengths and weaknesses are considered in a ra4ng:
Impact Score Descriptor
1 Exceptional
2 Outstanding
3 Excellent
4 Very Good
5 Good
6 Satisfactory
7 Fair
8 Marginal
9 Poor
High
Medium
Low
40
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
41
General Logis4cs for Phase II Panel Reviewers
Date/Loca4on Details
Date of Phase II Panels: Thursday, November 15, 2012
Loca4on: Hya9 Regency Washington on Capitol Hill, Washington D.C.
Time: 8:00am to 3:00pm
Number of Panels: Five
Ø Panel par#cipants will reflect the fact that each applica#on has 2 scien#sts, one stakeholder, and one pa#ent assigned to provide commentary and preliminary scores
42
Phase II Panel Process
Introduction Presentation
Overview and Triage Co-Chair Presents Application Synopsis
• Chair Introduc#on ‒ Stress confiden#ality and focus on impact
• Triage process to eliminate lowest-‐ranked applica#ons from panel discussion
• SRO captures panel discussion
• Individual review and scoring on PCORI Online on personal laptops
Review Scoring
Discussion of Application Open to Discussion
• Provides their preliminary impact score and assessment and its poten#al for significant outcomes/impact
• All panelists free to discuss, Chair moderates if needed
• If no discussion – move to final vote. Reviewers score individually in PCORI Online
43
Triage Process
During Phase II, a triage process will take place:
Ø Applica#ons are ordered according to ranking (highest to lowest scores)
Ø Lowest scoring applica#ons will be eliminated all at once from in-‐person panel
discussion
Ø If you as a Reviewer want to specifically discuss an applica#on, please come to
the panels prepared to do so
44
Roles & Responsibili4es
Scien4fic Reviewers (2) Pa4ent Reviewer Stakeholder Reviewer
Role Provide addi#onal depth for up to 10 applica#ons for all par#cipants
Provides addi#onal depth for up to 10 applica#ons for all par#cipants
Key Responsibili4es
Provides their preliminary impact score and assessment and its poten#al for significant outcomes/impact
PCORI review is different because there is no primary/secondary/ter#ary reviewer structure • Also u#lizes different, PCORI-‐unique merit review criteria
Each reviewer must provide substan#ve strengths and weaknesses for his/her assigned applica#on(s)
• Reviewers must be prepared to speak to these comments and scoring during the in-‐person review panels
Each panel will be lead by a Chair, Co-‐Chair, and SRO in tradi4onal advisory roles
45
Time Breakdown per Applica4on
NOTE: Panels will spend no more than 10-‐15 minutes per applica4on. An example:
Ø Some applica#ons may be reviewed in less than the 15 minutes allocated.
Ø It is important to understand the #me constraints and keep conversa#on focused, pointed, and succinct throughout the day to ensure fair and proper scoring of all applica#ons
Up to… Descrip4on
1 minute Co-‐chair briefly introduces applica#on
2 minutes Scien#fic Reviewer #1: overview and score
2 minutes Stakeholder reviewer: overview and score
2 minutes Scien#fic Reviewer #2: overview and score
2 minutes Pa#ent Reviewer: overview and score
4 minutes General discussion, if any
2 minutes Take vote and enter scores in PCORI Online
46
Agenda
1. Introduc#on and Announcements
2. Background
5. Merit Review: Phase I – Scien#fic Review
6. Merit Review: Phase II – Impact Review
7. Merit Review: Phase II – In-‐Person Panel
3. Program Funding Announcements
4. The Applica#on and Review Process
8. Phase II: PCORI Online – Process and Procedures
47
PCORI Online: Confiden4ality Agreement
The first thing you will have to do upon log-‐in is agree to the Confiden4ality Agreement. This applies to both preliminary scoring, and any/all scores and discussion that take place during the in-‐person review panel.
48
Accessing Your Assigned Applica4ons
Next, you will be able to see your list of assigned applica4ons by selec4ng “Review Assignments” in the side bar on the lep side of your screen. The list will appear similar to below:
49
Note any Conflicts of Interest
Use the drop-‐down box to note any COIs. If there is a COI, use the second drop-‐down box to indicate type.
50
Accessing and Scoring
Once you have confirmed there are no COIs, three new icons will appear to the right of an assigned applica4on:
Ø The first symbol (farthest to the lep), a PDF form, is the complete applica4on Ø The second symbol (in from lep) opens the applica4on abstracts Ø The third, orange symbol is an applica4on’s Phase I Reviews (scores and wriben cri4ques) Ø The last symbol on the far right opens your Cri4que Form
51
Preliminary Scoring in PCORI Online
The PCORI Online scoring screen for preliminary impact scoring:
54
Wrap-‐Up
This concludes today’s session. We hope you found this training helpful and informa#ve.
Thank you again for your commitment to PCORI.
If any ques'ons remain unanswered at this point, please email them to [email protected]