Board of Governors Meeting, South Sea Tea Washington

PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE

September 19, 2011

Patient-Centered Outcomes Research Institute

Communications, Outreach and Engagement Committee (COEC) Report


COEC Members

2

• Sharon Levine, MD (Chair)

• Debra Barksdale, PhD, RN

• Robert Jesse, MD, PhD

• Grayson Norquist, MD, MSPH

• Ellen Sigal, PhD

• Harlan Weisman, MD


COEC Charter

3

The Communication, Outreach and Engagement Committee shall advise and

assist the Board of Governors of PCORI, and provide recommendations to the

Board, regarding:

1. The Institute’s communications and branding work,

2. Strategies to engage all stakeholders in the work of PCORI, and

3. Methodologically sound approaches to disseminating and implementing the

research results and ensuring their utility to patients

and clinicians


COEC Report to the Board

4

I. Recent Opportunities for Public Input (Update)

II. Promoting PCORI RFPs (Update)

III. Redesigned PCORI Website (Update)

IV. Outreach Plan for Pilot Project Funding Announcement (Recommendation for Decision)

V. Stakeholder Outreach (Update)

VI. Speakers Bureau (Update)

VII.PCORI Communications Staff (Update)


Recent Opportunities for Public Input

5

• Working definition of “patient-centered outcomes research”

– Nearly 600 responses were received (July 20-September 2, 2011)

– All responses received through the website or emailed PDFs will be posted on pcori.org

– PCORI issued an RFP to solicit proposals for the analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences


Analysis and Summary of Input on Working Definition of PCOR

6

Issue RFP (and reach out to potential applicants) August 24, 2011

Deadline for applications September 9, 2011

Complete review of applications September 16, 2011

Complete contract with chosen contractor September 19, 2011

Submit Phase I findings and draft questions for focus groups

September 30, 2011

PCORI completes revision of PCOR Definition October 26, 2011

Revise questions for focus groups November 9, 2011

Focus Groups (or other way of seeking patient input) November 18, 2011

Report due to PCORI December 16, 2011 (with preliminary report in time for December 8 meeting)

PCORI completes further revisions of PCOR Definition January 15, 2012

PCORI Board considers revised PCOR Definition January 18, 2012


Working Definition of PCOR: A Foundation for PCORI’s Work

7

• At the time of PCORI’s creation, “patient-centered outcomes research” (PCOR) was not a commonly used term, nor had it been formally defined

• Defining PCOR helps clarify PCORI’s focus and scope of work

• Through iterative and transparent processes that elicit feedback from all stakeholders, including patients, we will continue to revise the definition, ensuring it:

– Emphasizes the “patient-centered” focus of PCORI’s mission

– Is consistent with the intent of the statute that established PCORI

– Is broad enough to support the range of research PCORI should fund


Recent Opportunities for Public Input

8

• Initial topics for PCORI pilot projects.

– More than 150 responses were received (August 1-30, 2011)

– Responses provided to the PDC Working Group to review and update topics as appropriate.

• 89.26% Self Identified

• 10.74% Anonymous

• 59.76% Responded as Individual

• 40.27% On behalf of an Organization

Respondents

• 40.94% Academia

• 23.49% Provider

• 17.45% Patient

• 3.36% Industry

• 0.67% Government

• 11.41% Non-Identified

Organizational Respondents


Promoting PCORI RFPs

9

• To date PCORI has issued 4 Requests for Proposals (RFP)

– Analysis of Input Received on Working Definition of “Patient-Centered Outcomes Research”

– Review and Synthesis of Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Literature Review)

– Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Interviews)

– Methods for Setting Priorities in Research (White Papers)

• For each RFP, PCORI performed outreach to: email list subscribers (620+), supplemental lists of clinical and translational research centers (60+), academic research institutions (30+), and provider and advocacy organizations (30+)

• PCORI is an independent, non-profit, non-governmental organization. All PCORI funding announcements will be posted on pcori.org. PCORI funding announcements will not be published in the Federal Register

http://www.pcori.org/


Redesigned PCORI Website

10


Redesigned PCORI Website

11

• Includes visual changes based on logo and color palette approved at July Board Meeting

• Designed to be more user friendly

• Will support the expansion of PCORI content and opportunities for interaction as work progresses

• Expanded features include:

– Easier mechanisms for joining PCORI mailing list and providing general input

– An “Executive Director’s Corner” that provides regular updates on PCORI’s latest developments


Outreach Plan for Pilot Project Funding Announcement

12

• Post announcement on pcori.org

• Direct email to stakeholders and potential applicants

• Conduct proactive media relations

• Promote the opportunity though grassroots outreach

• Integrate the announcement with speakers bureau presentations

• Host a Q&A teleconference for applicants


Stakeholder Outreach

13

PCORI’s engagement with stakeholders around Board meetings

• Stakeholder discussion forum March

St. Louis, MO

• Stakeholder discussion forum May

New York, NY

• Two small group meetings with patients and caregivers

• Eight small group meetings with 43 stakeholder organizations

July

Washington, DC

• Invited presentations from Northwest Pacific stakeholders September

Seattle, WA

• Site visits to clinical care facilities November

New Orleans, LA


Seattle Stakeholder Engagement Forum

14

• PCORI has invited individuals and organizations from the Pacific Northwest to make brief presentations to the Board about their work or perspective and how it can inform PCORI’s work

• Native American/Alaska Native perspectives

• Complementary and alternative medicine researchers and providers

• Outcomes and comparative effectiveness researchers

• Monday, September 19, 7:00-9:00 p.m. PT

• Open to the public


Speakers Bureau

15

PCORI has presented at 24 meetings since March

Recent Speaking Engagements

• 7/25 – IOM CER Innovation Collaborative Meeting

• 7/25 – Center for Medical Technology Policy CER Institute

• 8/12 – Consumers United for Evidence Based Health Care Annual Meeting

• 8/25 – Florida Chiropractic Association National Convention & Expo

• 9/9 – University of Michigan School of Nursing

• 9/16 – Oregon Institute for Patient-Centered Comparative Effectiveness Annual Research Intensive

Upcoming Opportunities

• 9/27 -- AdvaMed 2011 MedTech Conference

• 9/28 – Partnership to Improve Patient Care Annual Membership Forum

• 10/11 – NIH National Center for Research Resources Clinical and Translational Science Awards Program

• 10/12 – Comparative Effectiveness Research Summit

• 10/12 – American Academy of Nursing, Council of Advancement of Nursing Science 2011 Special Topics Conference

• 10/19 – Health Industry Forum

• 10/20 – Pfizer Health Advocates Leadership Breakfast


PCORI Communications Staff

16

• Seeking applicants for three positions:

– Director of Communications

– Director of Patient Engagement

– Director of Stakeholder Engagement

• Staff will develop and implement strategic communications plan

• Distinct engagement officers for patients and other stakeholders, recognizing their need for different engagement mechanisms


September 19-20, 2011 Seattle, WA

Executive Director Update Board of Governors



Objectives for today

Highlight PCORI Activities since July 2011 BoG Meeting

Discuss Growing the PCORI Staff

Consider Critical Timelines/Milestones for Next 9 Months

2


PCORI Activities – (Staff, Board, MC together)

Establishing Policies and Practice

HR Policies

Purchasing/RFP policies

COI policies

Research Policies

Re-designing Website

Outreach

Developing IT Infrastructure

Growing Staff

Finding Long-term Space


4

• $48 per sq. foot • Attractive, not • extravagant • 13,000 sq feet • Close to Metro • Green building • Move-in: February

PCORI’s Long-Term Home 1828 L Street, DC


Urgent Issues Coming Out of July BoG Meeting:

Address the need for greater clarity in decision-making and take steps to find more time for substantive discussions among board, with MC

Patient Engagement! - Officer, Advisory Panel

Get started on the National Priorities and Research Agenda!

Support the PDC in preparing PCORI Pilot Projects RFP and review

Support the MC in getting the RFPs out for the Methodology Report


Patient Engagement

6

Patient-Engagement Working Group formed; Job Description for Director of Patient Engagement drafted Directors of Stakeholder Engagement and Director of Communications Added All three positions are posted and a search is underway

Patient Engagement

Communi-cations

Stakeholder Engagement

External Engagement


Program Development Committee

National Priorities and Research Agenda

• C Clancy and A Epstein, co-chairs

• Environmental scan on past CER analysis and developed an initial framework for the National Priorities which will serve as the foundation for developing specific priorities

• Recognized the close link between the National Priorities and Research Agenda (H Krumholz and L Hole-Curry, co-chairs)

• Developed a timeline and notional process for engaging stakeholders as part of the National Priorities development timeline

7


Developing PCORI’s National Priorities and Research Agenda

8

Collect Stakeholder

Input

Develop Candidate Priorities

Public Comment

Finalize Priorities

Develop Candidate Priorities

Underway Sept - Nov Dec-Feb March

Collect Stakeholder

Input

Public Comment

Finalize Research Agenda

April Feb - Mar Sept - Jan

PRIORITIES:

RESEARCH AGENDA:


Program Development Committee PCORI Pilot Projects

C Goertz and G Hunt co-chairing this process.

Public Input has been reviewed and incorporated into revised “areas of interest” in the PCORI Funding Announcement (PFA)

An application form and instructions have been developed and revised

Working closely with NIH staff on defining each step in the application and review process

8


Supporting Methodology Committee

Interim researcher process established; at least one already working

RFPs (2) for Patient-centeredness Working Group posted

RFP (1) for Research Prioritization Working Group posted

Review Process in Development, including COI policies for review

9


Definition of PCOR

Input on the working definition of “patient-centered outcomes research” gathered (n=600); to be posted shortly after anonymization

RFP posted for qualitative analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences posted on PCORI website

12 responses received and reviewed

First PCORI research contract issued on 9/16 to National Opinion Research Center

II MC I

9/16 9/30 11/15 12/31

Synthesis & Question Generation

Revision of Definition

Testing of Revised Definition

Preparation of MS for Publication Led by MC


Three Engagement

Directors Applications

Arrive LOI’s

Arrive

Growing PCORI Staff:

09/18/11 –

09/20/11

We are here

11/1/11 2/1/12 05/1/12 04/1/12

11

1/1/12

COO

Begins

Mark

In!

Director

Finance

Begins

First

Scientists

Additional

Scientists

Additional

Admin

Ass’ts

Director of

Staff

12/1/11 3/1/12

Funding

Decision

Made

Funding

Issued

Program

Staff Hiring

Begins

Finance/Grant

Staff


Growing PCORI Staff

13

Research Scientist Patient engagement

Research

Research Scientist Stakeholder Engagement

Research Scientist Communicaiton/Disseminat

ion Research

Research Scientist(s) Comparative Clinical

Effectiveness Research

Director Patient Engagement

Director* Stakeholder Eng.

Director Communications

Chief Financial Officer (CFO)

Director HR

Director IT

Director Grants Management

Research Scientist(s) Evidence Synthesis

External Engagement

Executive Director

Chief Operating Officer Chief Science Officer


Critical Issues: September 2011 – June 2012

14

Gathering Stakeholder Input on National Priorities and Research Agenda

Thorough, balanced, incorporated

September – November:

November- December:

Finalizing First Draft of Priorities (Prior to Public Comment)

How are Priorities expressed, what’s in, is anything out?

January – February:

Finalizing First Draft of Research Agenda (Prior to Public Comment)

What (specifically) is in, out – what will the first RFAs fund?

Data Infrastructure Grants?

Translation/Implementation Research?

Systems research?

Pharmacogenetics?

Large clinical trials?


Finance & Administration Committee September 20, 2011, Seattle, WA



Bylaws Revisions

• GAO currently reviewing

• Committee Name Change: Finance, Administration & Audit

• Adoption Proposed at November BOG Meeting


Audit Update

• Released this week

• Bidders selected by annual revenues

• Proposals by end of October

• Recommendation to BOG

• Field work in November/December

• Completed audit by March

• Provided to GAO for report to Congress due April 1st


Financial Statements – August 31, 2011

• Statement of Financial Position: Total Assets of $46.3M; $5M Cash on Hand

• Expenditures:

– Budget: $7.5M

– Actual: $3.7M

– Positive variances in Board compensation, management fees, professional services, travel

– Negative variance in conferences


Conflicts of Interest Policy

• Working group created

Members: Larry Becker, Chair

Debra Barksdale

Arnie Epstein

Sherine Gabriel

Bob Jesse

• Group is gathering key documents


1

September 19, 2011

PCORI Methodology Committee Progress Report

Sherine E. Gabriel, MD, MSc Sharon-Lise T. Normand, PhD

PCORI Methodology Committee


2

• Establishment

• Overview

• Timeline

• Structure

• Mission

• Scope of Work

• Progress to Date

• Next Steps

• Questions

1. Methodology Committee

2. Methodology Committee Workgroups

3. Wrap Up

• Next Steps

Agenda


3

Methodology Committee Charge

Establishes priorities to address gaps in

research methods and their application

The Methodology Committee shall make

recommendations regarding methods for

patient-centered outcomes research

Provides guidance about the appropriate

use of methods


4

Methodology Committee Structure

Methodology Committee Co Chairs: Sherine Gabriel, MD, Mayo Clinic & Sharon-Lise Normand, PHD, Harvard Medical School

Patient-Centeredness

Research Prioritization

Methods to incorporate the patient perspective into all phases of PCOR

Methods to inform prioritization of new research studies

Methods for using data, design, and statistical analyses to conduct PCOR

Research Methods

Re

po

rt A

ssim

ilati

on


5

Methodology Committee Activity Timeline

Rese

arc

h

Me

tho

ds

Aug „11 Oct „11 Nov „11 Dec „11

Pa

tie

nt-

Cen

tere

dn

ess

Jan „12 Feb „12 Mar „12 Apr „12 May „12

Workshop Final

Report

Workshop

Identify

Gaps/

Issues

Translation

Table/Tool

Workshop Final

Papers

Due

Final

Draft Final

Report

Final

Report

Compile

First

Draft

Legend

Re

se

arc

h

Prio

ritiza

tio

n

Rep

ort

Assim

ilatio

n

Feb – Jul „11

Key Workgroup Activities MC Activities

Revise Report Outline & Edit Chapters

Achievements to Date

Final

Report

Final

Papers

Due

Electronic Data Systems Landscape Review

Identify & Implement Voting

Standards Process

Solicitations: White Papers/Identify Experts

Solicitations: Literature Review & Interviews

MC

Chart

er

MC

Work

Pla

n

Inte

rim

Researc

hers

Sta

ffed

PC

OR

Definitio

n S

olic

itation

Commission White Papers

Sep „11


6

Methodology Committee Discussion Points

• Importance of methods generated by Patient Centeredness Work Group for incorporating patient engagement

• Importance of methods generated by Research Prioritization Work Group vis-à-vis prioritization framework set by the board

• Generation of tools/standards for Methods Report is a multiyear process

• Importance of continuing to interact with Board to ensure coordination of efforts

• Board/Methodology Committee teleconference calls were a productive forum to receive input from the Board regarding their needs and priorities


7

• Establishment

• Overview

• Structure

• Timeline

• Mission

• Scope of Work


• Next Steps

• Questions



3. Wrap Up

• Next Steps


8

Patient-Centeredness Workgroup

Ethan Basch, MD

Memorial Sloan-Kettering

Cancer Center

Workgroup Chair

Mary Tinetti, MD

Yale University School of

Medicine

Workgroup Co-Chair

Naomi Aronson, PhD

Blue Cross and Blue Shield

Workgroup Member

Brian Mittmann, MD

VA Center for Implementation

Practice & Research Support

Workgroup Member

David Flum, MD, MPH

University of Washington

Report Assimilation Group Liaison


9

-

Patient Centeredness Mission/Scope of Work

Development and prioritization of research

questions

Identify methodological standards for incorporating

the patient perspective into three key areas:

Design of study components, including

selection of interventions, comparators, and outcomes (including patient-reported

outcomes)

Processes of clinical decision-making/care

delivery


10

Patient Centeredness Progress to Date

Released RFP

for Literature

Review

Released RFP

for Interviews

“Review and Synthesis of

Evidence for Eliciting the

Patient‟s Perspective in

Patient-Centered

Outcomes Research”

“Expert Stakeholder

Interviews to Identify

Evidence for Eliciting the

Patient‟s Perspective in

Patient-Centered Outcomes

Research”


11

Proposals Due

Awardee Announced

Final Report Due

Oct 6, 2011

Oct 13, 2011

March 1, 2012

Issue Open RFI Plan Workshop

Develop Sections

for Methodology

Report

Patient Centeredness Next Steps


12

Patient Centeredness Question

How does this work complement ongoing and planned activities of the

Board?


13

Research Prioritization Workgroup

David Meltzer, MD, PHD

University of Chicago

Department of Medicine

Workgroup Chair

John Ioannidis, MD, DSc

Stanford Prevention

Research Center

Workgroup Member

Jean Slutsky, PA, MSPH

Agency for Healthcare

Research and Quality

Workgroup Member

Clyde Yancy, MD

Northwestern University

Feinberg School of Medicine

Workgroup Member

Alfred Berg, MD, MPH


Department of Family Medicine


http://news.uchicago.edu/sites/newsmachine.uchicago.edu/files/imagecache/image_landingpage_zoom/images/image/20110429/lunjfympqe.7833.20110429.jpg


14

Research Prioritization Mission

Mission

To provide guidance concerning the use of methods to inform the establishment of research prioritization approaches that best fulfill PCORI’s mission.

• To the PCORI Board to aid in development and future refinement of Research Priorities]

• To the broader multi-stakeholder communities to enhance understanding of PCORI’s approach to research prioritization and to encourage their engagement in PCORI’s research prioritization process


15

-

Commission White

Papers

To prepare a section for the May 2012 Methodology Committee Report that addresses methods to inform the establishment of research

prioritization

Synthesize White Papers during January 2012

Workshop

Research Prioritization Scope of Work

Topic Generation

Gap Analysis in

Systematic Reviews

Value of Information

Analysis Peer Review

Suggest strategies to identify possible future

priority areas

Suggest standards for how PCORI systematic reviews should be performed and used to generate research

topics

Provide PCORI Board and/or grant applicants with tools

to quantify expected benefits of research to

inform priorities

Provide data to inform how PCORI might design,

evaluate and continually improve its peer review

process


16

Research Prioritization Progress to Date

Identified

Methodological

Areas of Interest

in Research

Prioritization

Identified

Interim

Researcher to

Assist

Workgroup

Released RFA

for White

Papers on

September 9,

2011


17

White Paper Proposals Due

Review and select among White Paper proposals

Awardee Announced

Sept 30, 2011

Oct 1- 13, 2011

Oct. 14 2011

Explore the role of research prioritization methods vis-à-vis the broader

PCORI research prioritization process

Engage researchers as needed to fill in expected gaps from White Papers

Research Prioritization Next Steps


18

Research Prioritization Question

How can we best work with you to understand your evolving plans for

research prioritization so the methods we are studying can be of

most use to you?


19

Research Methods Workgroup

Steven Goodman, MD, MHS, PHD

The John Hopkins University

School of Medicine and Public Health

Workgroup Chair

Mike Lauer, MD

National Institute of Health

National Heart, Lung, and

Blood Institute

Workgroup Member

Robin Newhouse, PHD RN

University of Maryland

School of Nursing

Workgroup Member

Sebastian Schneeweiss,

MD, ScD

Harvard Medical School

Workgroup Member

Mark Helfand, MD MPH MS

Oregon Health & Science University

School of Medicine


Sharon-Lise Normand, PHD


Methodology Committee Chair

http://www.hsph.harvard.edu/faculty/sebastian-schneeweiss/index.html


20

-

Develop Translation

Table/Tool (from research question to design and

analysis)

Identify Standards for the use of data, design, and statistical analyses to

conduct patient centered outcomes research through:

Review current methodological “state of

the art” statements issued by expert bodies to guide

PCORI research

Research Methods Mission/Scope of Work

Review existing data systems designed to

permit exploration of causal questions from data gathered in the course of clinical practice. Identify PCORI role in this arena.


21

Research Methods Progress to Date

Established initial

specifications for

dimensions to be used

in translation

instrument, with

options for

implementation

Developing

prototype for

standards

documents that

combines expert

statements,

published examples

and PCORI MC input

Interviewing leading

researchers of

electronic data, from

which research and

methods

recommendations

will be developed


22

Research Methods Progress to Date (Detailed)

Framework for

Methods Standards /

Recommendations

1. Method 2. Key sources 3. Major recommendations 4. PCORI MC commentary 5. Published examples 6. Tools for researchers

Translation Table

Dimensions

Intrinsic Factors • Internal validity (aka bias) • External validity (aka

generalizability, transportability) • Precision • Heterogeneity in risk or benefit

(aka “personalized” evidence) • Ethical dimensions

Extrinsic Factors • Timeliness (Rapidly changing

technology, policy urgency) • Logistical burden (e.g. study

size, complexity, cost) • Constraints (Data availability,

randomization possible?)

Question: condition, population, treatment & comparator, outcomes, setting


23

Finalize translation table dimensions and

determine categories of development

Finalize “standards” document format and determine categories of development

Finish and summarize electronic data

systems environmental scan

Research Methods Next Steps

(e.g. diagnostic tests, missing data, systematic reviews, etc.)

(e.g. drug safety, therapeutic

efficacy, etc.)

Jan. 2012 Workshop


24

Research Methods Question

Can the Board provide feedback on the preliminary formats developed (i.e., translation table dimensions and methods standards format)?


25

Report Assimilation Workgroup

Mark Helfand, MD


School of Medicine

Workgroup Chair

Alfred Berg, MD, MPH


Workgroup Member

David Flum, MD, MPH


Workgroup Member

Howard Balshem, MS


Interim Researcher

Sherine Gabriel, MD

Mayo Clinic

Workgroup Member

Sharon-Lise Normand, PHD


Workgroup Member


26

• Establishment

• Overview

• Structure

• Timeline

• Mission

• Scope of Work


• Next Steps

• Questions



3. Wrap Up

• Next Steps


27

Solicitation Responses and Questions Received to Date

Methodology Committee Next Steps

Letters of Intent Received**

Questions

Patient Centeredness WG -- Stakeholder Interviews

25 8*

Patient Centeredness WG - Literature Review

26 4*

Research Prioritization WG - White Papers

11 0

**as of 9/16 at 8:30 a.m. *This number of questions includes a document containing 13 individual sub-questions.


28

Administrative Review

Content Review

Programmatic Review

Executive Decision

1. Clear, transparent and reproducible review process based on best

practices (eg. NIH)

2. Aligned with PCORI Vision, Mission, Principles and Policies (eg CoI)

3. Timely (~Applicants notified of award two weeks after submission)

Will be Established prior to receipt of first response (September 30, 2011)

Solicitation Review Process and Principles



29


Rese

arc

h

Me

tho

ds

Aug „11 Sep „11 Oct „11 Nov „11 Dec „11

Pa

tie

nt-

Cen

tere

dn

ess

Jan „12 Feb „12 Mar „12 Apr „12 May „12

Workshop Final

Report

Workshop

Identify

Gaps/

Issues

Translation

Table/Tool

Workshop Final

Papers

Due

Final

Draft Final

Report

Final

Report

Compile

First

Draft

Legend

Re

se

arc

h

Prio

ritiza

tio

n

Rep

ort

Assim

ilatio

n

Feb – Jul „11

Key Workgroup Activities MC Activities

Revise Report Outline & Edit Chapters

Achievements to Date

Final

Report

Final

Papers

Due

Electronic Data Systems Landscape Review

Identify & Implement Voting

Standards Process

Solicitations: White Papers/Identify Experts

Solicitations: Literature Review & Interviews

MC

Chart

er

MC

Work

Pla

n

Inte

rim

Researc

hers

Sta

ffed

PC

OR

Definitio

n S

olic

itation

Commission White Papers


30

Thank You!

1

Presentation to PCORI Board of Governors September 20, 2011

PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE:


2


Committee Members

• Carolyn Clancy

• Francis Collins

• Arnold Epstein

• Christine Goertz

• Leah Hole-Curry

• Gail Hunt

• Harlan Krumholz

• Richard Kuntz

Methodology Committee Representatives

• Sherine Gabriel

• Sharon-Lise Normand

3

Report Outline

PCORI Pilot Project Grant Program

National Priorities and Research Agenda

Landscape Review Program


4

Presentation to PCORI Board of Governors September 20, 2011

Christine Goertz, DC, PhD

PCORI Pilot Projects

5

Report Outline:

• Purpose of PCORI Pilot Projects

• PCORI Funding Announcement - Key Highlights

• Areas of Interest

• Overview of the Four-Stage Review Process

• Application Review Criteria

• Grant Application & Instructions

• Webinar & Process to Handle Inquiries

• Timeline, Key Dates, and Activities

6

Purpose of PCORI Pilot Projects

The purpose of the PCORI Pilot Projects Grant Program is threefold: 1. National Priorities. Pilot projects will provide information to

PCORI that informs future iterations of national research priorities for patient centered outcomes research.

2. Research Agenda. The program will support the collection of preliminary data that can be used to advance the field of patient-centered outcomes research, providing the platform for an evolving PCORI research agenda.

3. Methodologies. The program will support identification of research methodologies that advance patient-centered outcomes research

7

PCORI Funding Announcement: Key Highlights

Applicants must address at least one of the eight areas of interest.

Stakeholder involvement is required unless the application can sufficiently explain why it is not feasible.

Expect to commit approximately $13 million in FY 2012 to support approximately 40 projects.

Applicants can propose project periods of up to two years. Second year funding is subject to noncompetitive review by PCORI staff.

Direct costs are limited to $250,000 per year with an additional 40% of salary and fringe allowable as indirect costs.

Letters of intent are due on November 1 with applications due on December 1.

NIH will conduct the merit review process with all other aspects of grant awards and grants management handled by PCORI.

8

Areas of Interest: Development

Areas of Interest

Board Members

Methodology

Committee

Stakeholder / Public Input

Projects focused on developing, testing, and/or evaluating methods or approaches that: Inform the evolving

National Priorities Bring together different

stakeholders Translate research into

practice Identify gaps in relation to

issues for disadvantaged populations

Identify predictors of patient outcomes

9

Areas of Interest

Developing, testing, refining, and/or evaluating new or existing methods (qualitative and quantitative) and approaches that can inform the process of establishing and updating national priorities for the conduct of patient-centered outcomes research (PCOR). This may include research prioritization approaches (such as Value of Information (VOI), burden of illness, peer review/expert opinion/Delphi approaches) or methods for incorporating the perspectives of patients or other stakeholders into the development of national priorities.

Developing, testing, and/or refining existing methods for bringing together patients, caregivers, clinicians including non-traditional partners, and other stakeholders in all stages of a multi-stakeholder research process, from the generation and prioritization of research questions to the conduct and analysis of a study to dissemination of study results – including methods for training participants in participatory research and the potential use of new technologies to facilitate engagement.

Developing, refining, testing, and/or evaluating patient-centered approaches, including decision-support tools, for translating evidence-based care into health care practice in ways that account for individual patient preferences for various outcomes. This may include developing or comparing conceptual models of translation or dissemination of CER research findings from the patient perspective.

Developing, refining, testing, and/or evaluating methods to identify gaps in CE knowledge such as tools for the ongoing collection and assessment of gaps as perceived by patients and providers. Of special interest are gaps that are particularly relevant to vulnerable populations, including but not limited to, low-income populations; minorities; children; elderly; women; people with disabilities, chronic, rare and/or multiple medical conditions.

10

Areas of Interest

Identifying, testing, and/or evaluating patient-centered outcomes instruments. This may include predictive tools (eg: instruments that measure or predict outcomes of interest to patients) or identifying standards for measurement properties of patient-reported outcomes for use in comparative effectiveness research, across a variety of interventions and patient populations.

Identifying, testing, and evaluating methods that can be used to assess the patient perspective when researching behaviors, lifestyles, and choices within the patient’s control that may influence their outcomes.

Identifying, testing, refining and/or evaluating methods for studying the patient care team interaction in situations where multiple options for wellness, prevention, diagnosis or treatment exist. Of special interest are strategies that respect patient autonomy and promote informed decision-making, incorporating the best health care knowledge into the application of care.

Advancing analytical methods for CER. Examples include but are not limited to the incorporation of mixed methods research designs (qualitative/quantitative), identifying existing methodology to statistically accommodate irregularly spaced multivariate longitudinal data, the use of instrumental variables; and potential solutions for assessing treatment heterogeneity in observational and randomized CER studies.

11

Overview of the 4-Step Process

Preliminary

Check

Merit Review

Deliberation

Business

Determination that the application meets the minimum technical requirements enumerated in the PFA and fits within programmatic priorities.

PCORI Responsive/eligible applications will be forwarded for review.

Remaining applications are assigned to 3 reviewers with appropriate expertise who will critique independently and then meet to discuss and designate a final score.

NIH A rank-ordered list of applications and summary sheets

The Pilot Projects Review Committee considers the merit review scores and programmatic balance across several categories to develop a recommended award slate for consideration by the BOG.

PCORI Funding decisions are made.

A final review for suitability to manage an award, adherence to human subjects requirements, receipt of IRB approval, budget review, etc.

PCORI Negotiations, where needed

Description Responsible Result

12

Preliminary Check

Technical Requirements

• Receipt of Letter of Intent

• Eligible applicant

• Stakeholder requirements

• Page limit and formatting requirements

• Deadline met and application properly submitted

Programmatic Requirements

• Does the application fit within PCORI program requirements listed in the PFA?

Transfer to NIH

• Applications meeting both technical and programmatic requirements will be forwarded to NIH for the Merit Review.

Conducted by PCORI staff

13

Merit Review

Prior to the Review Meeting

• Each application is assigned to 3 reviewers

• Reviewers include scientists and other stakeholders

• Stakeholders without previous review experience will be trained

• Reviewers assign an initial priority score of 1 to 9 based on PCORI-provided review criteria

• Provides basis for discussion at in person meeting

In-Person Review

• In-person meeting of reviewers to discuss the most promising applications

• Review and discuss applications based on impact, stakeholder involvement, innovation, and significance of the science

• Provide a final priority score of 1 to 9

• Scientific Review Officer compiles a summary statement with reviewer critiques for each application

Summary & Rankings

• The review process results in a ranked, scored summary of applications for PCORI consideration

Conducted by NIH

14

PCORI Deliberations

PCORI Staff

• Analyzes the applications using priority score and classification categories.

• Creates funding scenario options based on analytics.

• Supports the PCORI Review Committee and Board of Governors by providing revised options, as needed.

PCORI Review Committee

• Meets to review materials

• Considers the balance of priority-scored applications across classification categories, requesting additional analysis and options from staff, as needed

• Prepares a recommended slate of selected projects for funding consideration

Board of Governors

• Meets to consider the recommended slate

• Reviews the slate based on priorities and balance to ensure appropriate distribution

• Requests additional options, if needed

• Approves a final slate of selected projects for funding

Classification categories may include: Applicability of the 8 areas of interest, geographic distribution, research methodology, and identification of target populations.

15

Recommended Funding Slate Analysis

Following the Review Committee meeting, analysts will prepare the recommended slate along with accompanying analysis for consideration by the Board of Governors.

16

Merit Review Criteria

Significance. Does the project address an important problem or a critical barrier to patient-centered outcomes research? Does the project address one of the key questions outlined in the PCOR definition described above? Is the project focused on one of the areas of interest identified in this PFA? Will the results produce new knowledge that can advance PCOR methods or infrastructure? Does the investigator demonstrate thorough knowledge of previous and ongoing work related to their proposed topic?

Patient/Stakeholder Engagement. Will the research make a unique contribution to learning about engagement of patients and non-traditional stakeholders in PCOR research efforts? Does the research team demonstrate authentic, feasible, sustainable, novel partnerships with patients, families and care givers, providers, and other appropriate non-traditional stakeholders? Is there evidence that non-traditional stakeholders were involved in the preparation of the research proposal?

Investigator(s). Is the research team well suited to the project? Is it multi-disciplinary? Is there appropriate scientific expertise? Does at least one member of the study team have experience in patient and other stakeholder engagement? Is there a high level of confidence that the Principle Investigator and rest of the study team will be able to achieve the study aims as described? Does the study team have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?

17

Merit Review Criteria (continued)

Innovation. Does the project either address a new method or approach or apply a proven method or approach in a novel way to the field of PCOR? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? Will the strategy establish feasibility and will particularly risky aspects be managed? Is the proposed budget and timeframe appropriate for the research plan? Are there appropriate plans for dissemination among key PCOR stakeholders in education, practice and policy?

Environment. Will the environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the research environment, community involvement, patient populations, or non-traditional stakeholder collaborative arrangements?

18

Grant Application & Instructions

• Application is based on the NIH PHS 398 form with changes to ease responses and to meet the needs of PCORI.

• Applicants will be bound by the Human Subjects policies of NIH and will include appropriate plans within the Research Strategy section.

• Applicants will provide information related to classification categories to enable the Review Committee and Board to consider this information when developing a balanced slate for funding.

Self-Reported Classification Categories:

• Applicability of the “4

questions” to the research • 8 areas of interest • Identification of specific

populations including:

― Underserved or disadvantaged populations

― Specific ethic or cultural populations

― Rural or urban populations

― Disabled populations

19

Webinar & Inquiry Process

• Webinar for Applicants: October 12, 2011 at 2PM EST.

• Policy for responding to inquiries:

– A set of FAQs will be developed and issued along with the PFA with frequent updates posted on the PCORI website. Questions and answers will be framed in terms of general applicability.

– Inquiries should be directed to the PCORI website at pcori.org. PCORI staff and appropriate consultants will receive, triage, route, and ensure answers are provided and shared generally through the website.

20

Timeline, Key Dates, and Activities

2011 09 /11

PFA Posted

11/01/11 Letters of

Intent Due

12/1/11 Applications

Due

11-12/11 Identification of Merit Reviewers

2012 12/11

Completeness/ Compliance/

Eligibility Check

2/12 Merit Review

3/12 Slate of Projects

to Board & Selection Made

05/12/12 Award

Notification & Funding

PCORI Applicant NIH/PCORI Applicant

PCORI NIH PCOR PCORI / Applicant

21

Questions?

www.pcori.org

22

Request Approval

23

Arnold Epstein, MD Presentation to PCORI Board of Governors September 20, 2011

PCORI National Priorities Development

24

National Priorities are the critical beginning and will be dynamic

Incre

asing Sp

ecificity

National Priorities

Research Agenda

Individual PFAs

25

Develop

Candidate Framework

Engage

Stakeholders

Public

Comment

Finalize National Priorities

Currently Underway

September-November

December-February

March

Proposed Timeline for National Priorities

26

Environmental scan of existing priorities and

criteria

Candidate priorities and

criteria identified

Framework to inter-relate

Priorities and Criteria

Reviewed initial stakeholder input advising us to not “reinvent the wheel.”

Reviewed prior CER frameworks (e.g., IOM, FCCCER, National Priorities Partnership, NQF)

Identified broad priorities and criteria that were used often in prior frameworks and fit PCOR.

Framework to be used for refining priorities, and determining Research Agenda and PFAs.

Initial Stakeholder

feedback

Development of a National Priorities Framework

27

Previous priorities have varied in granularity

Granularity

IOM 2009 (e. g., Compare the effectiveness of upper endoscopy utilization and frequency for patients with gastroesophageal reflux disease on morbidity, quality of life, and diagnosis of esophageal adenocarcinoma.)

National Priorities Partnership (e. g., Engage patients and families in managing their health and making decisions about their care.)

National Prevention Council (e. g., Elimination of health disparities)

FCCCER (e.g., Expanding high-impact patient registries)

National Quality Forum (e. g., Infrastructure – Information Technology)

Least Most

National Quality Strategy (e. g., Promoting effective communication and coordination of care)

28

Ten candidate priorities consistent with prior efforts

Source P

reve

nti

on

Acu

te C

are

Ch

ron

ic

Dis

eas

e C

are

Pal

liati

ve C

are

Car

e

Co

ord

inat

ion

Pat

ien

t En

gage

me

nt

Safe

ty

Ap

pro

pri

ate

U

se

HIT

to

Imp

rove

P

atie

nt

Exp

eri

en

ce

Imp

act

of

New

Te

chn

olo

gy

IOM 2009: Priorities for CER

√ √ √ √ √ √ √ √ √

Federal Coordinating Committee for CER

√ √ √ √ √

AHRQ National Quality Strategy

√ √ √ √

AHRQ Effective Health Care Program

√ √ √ √ √ √

National Quality Forum √ √ √ √ √ √ √ √

National Prevention Council

√ √

National Priorities Partnership

√ √ √ √ √ √

29

Proposed criteria include those in statute and additional criteria

Used in Prior Efforts

Criterion Required in

statute

√ Impact on Health √

√ Improvability via Research √

√ Inclusiveness of Subpopulations √

√ Impact on Health System Performance √

√ Gaps in Knowledge √

√ Variation or Disparities in Delivery or Outcomes √

√ Potential to Inform Decision-Making at Point of Care √

√ Responsiveness to Expressed Needs √

Advances CER Methods

Fits the Definition

30

•Impact on Health of Individuals and Populations

•Probability of Improvability via Research

•Inclusiveness of Different Sub-populations

•Current Gaps in Knowledge / Variation in Care

•Impact on Health System Performance

•Current Health Disparities

•Potential to Influence Decision-Making at Point of Care

•Responsiveness to Expressed Needs

•Advances CER Methods

•Fits the Definition of PCOR

Framework to Inspire Drafting of PCOR-Specific National Priorities

Health Information

to Improve Patient

Experience

Prevention &

Screening

Acute Care

Impact of New

Technology

PCORI National Priorities (Illustrative Examples)

Prevention & Screening

Develop information that will guide

patients and providers to improve

outcomes in chronic conditions.

Develop information to help patients

coordinate their care across multiple

providers and care settings.

Develop evidence to help patients

improve the safety of their health

care. Appropriate Use

PCORI Criteria

Proposed PCORI Criteria

Existing Priority Areas

In S

tatu

te

Chronic Disease

Care

Palliative Care &

Pain Management

Care Coordination

Patient

Engagement

Safety

31


Guiding Principles

Balanced Representation

Transparency

Facilitate Participation

Enable an open engagement process that makes clear how

participants can get involved and how their input will be

incorporated

Obtain feedback from a diverse and representative range of

patients, health care stakeholders and the general public

Provide easy, accessible forums for participants to provide

feedback equitably

32

Stakeholder Engagement Overview


Plan

Identify

Engage

Track

Assess

Report

33

PATIENTS PROFESSIONAL ORGANIZATIONS

Individuals with Disease Families

Caregivers Patient Advocacy Groups

General Public

Providers & Practitioners Employers

Payers Research Societies

Academia Health Information Exchanges

Federal/State/Local Government

Congress Life Sciences Industry

Who are PCORI’s stakeholders?

Key Stakeholder Groups

34

Proposed Stakeholder Engagement Timeline

Create

Engagement

Plan

and Identify

Stakeholders

Decide on

Content,

Methods

of Dialogue

Conduct Dialogue with Stakeholders

Review, incorporate and Report

on Feedback

September October November December Week

1 Week

2 Week

3 Week

4 Week

5 Week

6 Week

7 Week

8 Week

9 Week

10 Week

11 Week

12 Week

13 Week

14 Week

15 Week

16 * *

* Board Meetings *

35

Questions?

www.pcori.org

36

Richard Kuntz, MD

Presentation to PCORI Board of Governors

September 20, 2011

Landscape Review Program

37

Goals of Landscape Reviews

• Identify existing knowledge and gaps in existing knowledge related to PCOR and CER

• Avoid funding patient-centered outcomes research that duplicates ongoing or existing research

• Provide background information to spur innovation in research and methodology, and contribute to the infrastructure for conducting patient-centered outcomes research

• Create shared understanding about the state of the art of dissemination and patient engagement strategies and contribute to best practices

• Provide input into National Priorities and Research Agenda

38

Chronology of Landscape Review Development

Initial Phase: Environmental Scan of

Comparative Effectiveness Research

Discovery Phase: Learning about

Lewin Group Inventory of CER

Refining the Scope of Landscape Reviews &

Methodology Committee

Requirements

Coordination of Landscape

Reviews across PCORI: Landscape Review Program

Landscape Review Program:

• Process to initiate and manage landscape reviews

• Templates to develop contract specifications and budget requirements

• Resource library to share findings

39

Landscape Review Process Overview

Confirm questions to be answered

Document known information & gaps

Assess relevance / usefulness of information

Engage with NIH, AHRQ, others to leverage work of interest to PCORI

Identify additional information

needed

Prepare RFI as needed to acquire

information

PCORI RESOURCE LIBRARY

40

Landscape Reviews/Summaries Completed or Underway

Report Status

“Existing and Ongoing NIH Work Related to Dissemination of Research: Overview of NIH Briefing Book”

Completed 06/04/2011

“Existing and Ongoing Work Related to Patient Involvement in Peer Review: Overview of NIH Briefing Book”

Completed 06/04/2011

“Overview of Lewin CER Inventory Project” Completed 07/07/2011

“Organizations that Conduct CER” Completed 08/09/2011

Review and Synthesis of Evidence for Eliciting the Patient’s Perspective in Patient-Centered Outcome Research (Literature Review)

RFP issued 8/31/2011

Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient’s Perspective in Patient-Centered Outcome Research (Interviews)

RFP issued 8/31/2011

Landscape review of existing methodological standards Under development by MC

Landscape review of patient engagement Under consideration by COEC

41

Questions?

www.pcori.org

Board of Governors Meeting, South Sea Tea Washington

Health & Medicine

Transcript of Board of Governors Meeting, South Sea Tea Washington