Managing the Business of Inefficient Oversight: How Heavy ...

Proprietary & Confidential

1

Managing the Business of Inefficient Oversight:

How Heavy or Light is Your Touch When it Comes to Overseeing Vendors?

Clinical Trial Risk and Performance Management Summit

September 4th, 2019


22

SURVEY BACKGROUND

• 32 Questions over 4 sections:1. Company Overview Information

2. Vendor Oversight Practices

3. Internal Oversight Resources

4. Free text (challenges, tools, technologies, future implications)

• Sent to Halloran targeted contact list (not posted for general input)

• Targeted interviews

• Collected responses over 4 weeks

• 40 responses (14.1%); median completion time = 14 minutes

• Survey still open and analysis ongoing


33VENDOR OVERSIGHT: YOUR ROLE

(POLLING QUESTION 1)

Which of the following best describes your role?

A. Study team member

B. Other Clinical Operations role

C. Other non-clinical study team member

D. Quality Assurance

E. Vendor Management/Oversight

F. Other


4

SECTION I: COMPANY OVERVIEW INFORMATION


55

VENDOR OVERSIGHT: SURVEY PARTICIPANTS


66

VENDOR OVERSIGHT: COMPANY OVERVIEW


7

SECTION II: VENDOR OVERSIGHT PRACTICES


88VENDOR OVERSIGHT: CURRENT STATE


What is the current state of your organization’s vendor oversight practices?

A. We have an Oversight Plan Template AND implement it well across all of our clinical trials

B. We develop Oversight Plans but don’t follow them.

C. We do oversight and document it, but we do not have a standard Oversight Plan Template.

D. We perform vendor oversight but it is not always documented nor does it follow a standard process

E. We know that we need to do something, but do not know where to start.

F. We do not know what we need to do.


99

VENDOR OVERSIGHT: CURRENT STATE

Q: What is the current state of your organization’s vendor oversight practices?


1010

VENDOR OVERSIGHT: TOOLS

<100 Employees 100+ Employees

14 16

7 15

11 11

8 12

6 8

4 5

Q: Which of these tools does your organization have to help with Vendor Oversight (check all that apply)?


1111VENDOR OVERSIGHT: BARRIERS


10 10

8 8

6 8

4 5

3 2

Q: What barriers/challenges do you see with implementing the ICH E6(R2) guidelines around Vendor Oversight (check all that apply)?


1212VENDOR OVERSIGHT: RESPONSIBLE FUNCTION


What function within your organization is responsible for Vendor Oversight at the trial and/or vendor level (check all that apply)?

A. Centralized Vendor Oversight Group

B. Responsibility resides within each function

C. Clinical Operations

D. Procurement

E. Outsourcing / Vendor Management

F. Other


1313

VENDOR OVERSIGHT: RESPONSIBLE FUNCTION


17 15

4 8

1 4

0 3

2 0

Q: What function within your organization is responsible for Vendor Oversight at the trial and/or vendor level (check all that apply)?

1 2


14

SECTION III: OVERSIGHT RESOURCES


1515VENDOR OVERSIGHT: TIME COMMITMENT


Vendor Oversight (per ICH E6(2)) will require more of my time to manage in the future.

A. Strongly agree

B. Agree

C. Neither agree nor disagree

D. Disagree

E. Strongly disagree


1616

VENDOR OVERSIGHT: TIME ALLOTMENT

70.2% Strongly Agree/Agree

65% Strongly Agree/Agree

76.5% Strongly Agree/Agree

Q: Vendor Oversight (per ICH E6(R2)) will require more of my time to manage in the future.


1717

VENDOR OVERSIGHT: METHODS USED


19 17

17 16

18 14

16 11

17 7

11 13

12 7

12 8

2 0

Q: What types of vendor oversight methods do you employ (check all that apply)?


1818

VENDOR OVERSIGHT: METRICS USED

• The following metrics were each listed once: Audit Performance, Change Orders, Co-monitoring Visits, Data Quality, Inactive Sites, LPLV-DBL, Monitor Turnover, Monitoring Plan Compliance, Relationship, Risk Management, Site Activation, Site Compliance, Vendor Meetings

• Non-specific entries not included (i.e. “KPIs”, “SLAs”, etc.)

Q: What do you consider the top 3 most important vendor oversight metrics?


1919

VENDOR OVERSIGHT: DATA ACCESS

Q: How easy is it to readily access your patient/operational data?

<100 Employees: 5% (1)100+ Employees: 44% (7)

73% of companies responded that they receive their metric information from the vendor


2020VENDOR OVERSIGHT: STUDY MANAGER TIME


How much time do you estimate a Study/Trial Manager spends weekly on Vendor Oversight activities for an individual study?

A. < 0.5 day/week

B. 0.5 – 1 day/week

C. 1 - 1.5 days/week

D. 1.5 – 2 days/week

E. 2 – 2.5 days/week

F. 2.5 – 3 days/week

G. 3 – 4 days/week

H. 4 – 5 days/week

I. > 5 days/week


2121VENDOR OVERSIGHT: STUDY MANAGER TIME

Q: How much time do you estimate a Study/Trial Manager spends weekly on Vendor Oversight activities for an individual study?

This translates to 10%-60% of a Study

Manager’s time is spent on oversight activities.


22

SECTION IV: ADDITIONAL FEEDBACK


2323

VENDOR OVERSIGHT: OTHER CHALLENGES

We have sponsor oversight - central team, but then there is study team level - Trial Manager; the study team level is the most challenging due to scope service differences, time to commit, and appropriate experience

Turnover within Vendor organization, retraining of staff.

CROs reluctance to want to have oversight

Implementation of a standard process; Implementing a process that will work across various studies, i.e. not re-inventing the wheel every time we do a study.

Subcontracting by vendors - how to maintain oversight for third party vendors

-Consistency between trials/vendors; - Vendors do sloppy work

Education outside of Clinical Operations; understanding the level of documentation that is "adequate" to pass regulatory inspections.

Organization shifting with the new guidelines fast enough (very slow), findings are arising in QA audits and regulatory inspections, trends and needs to be addressed. Challenges between early and late phase drug trials, template but is geared towards late phase, early has complexities ie oncology. Budget for programmed reports and outputs, vendors and CROs not prepared for this level of work and oversight, too much or too little. Micromanaging vs High level oversight tough balance..

Building consistent, evenly applied tools and templates

Q: What other challenges/concerns has your organization experienced with Vendor Oversight process implementation?

Organizational

Process

Requirements

Tools

Training


2424VENDOR OVERSIGHT: EFFECTIVE TOOLS/TECHNOLOGIES

CTMS where all vendor info and metrics get feed into. , graphs, bar charts, are helpful. this is a challenge in a large organization and with multiple vendors, CROs. Preferred vendors list helps but then they are maxed out and overstretched and hard to prioritize with a sponsor gives them all their studies. You loose negotiation on timelines and items slip; CTMS, Spend data, plan templates that are good quality, standardized and risk-based governance model; Having your own CTMS Would love to have CTMS, but have not implemented; CTMS system where we could develop our own metrics rather than going to someone else. Dedicated person to follow up on issues.

Vendor oversight plan, standard reporting metrics that are practical and provide useful data, training of team on a process for effective vendor oversight; Scheduler/tracker for what to review when to generate documentation of completion of tasks (vs. documenting in TCRs and Emails); Oversight trackers used for documentation

Regular detailed meetings with vendors review of every vendor deliverable review of data; Oversight SOP for Sponsor; Joint Quarterly Clinical Quality meetings; TUVA by DDI for RBM; Creating partnerships to improve trust and visibility, plans (joint or sponsor-only), keeping eTMF in-house; monthly check in meetings for governance work well

Data lake, issue management process/tool; CTMS, eTMF, EDC, IVRS, Vendor Portal, and a Sample Management system; eTMF Dashboard with CRO APIs Will implement MediData central monitoring in future

Qlik or Spotfire Dashboards for defined KPIs/SLAs SharePoint Online (or central online file folder) to store metrics, governance materials; Regularly scheduled governance meetings for CROs. Using reverse metrics where applicable to ensure vendor knows we, the sponsor, have skin in the game too. QBRs for other important vendor relationships. Quality Agreements where applicable can be helpful. Have recently been working with AVOCA Quality Consortium and have access to their best practice tools.; Data collection and visualization; Self service analytics tools that do not rely upon IT or specialized vendors for implementation; Data visualization tools; initiate early a risk matrix with triggers and monitoring plan with key data fields to review internal study specific word document with weekly/biweekly updates of 3 parts: overview of study with key time/budget KPI; issues to be resolved (escalation); CRO and site specific KPI

Q: Since ICH E6 (R2) was published, where do you see vendor oversight heading in the future? Do you anticipate any new initiatives or structural changes to accommodate increasing vendor oversight activities?

Systems

CTMS

Documentation

Meetings

Metrics


2525

Q: Please add any additional thoughts or comments you may have on vendor oversight that you feel was not addressed but may be relevant

VENDOR OVERSIGHT: ADDITIONAL THOUGHTS

Our organization measures Third Party Supplier Risk Assessment at a corporate, regional, and country level. This is a evolving list of suppliers and helps track and improve oversight practices continuously

More risk-based approaches. More technology applications to have readily available data to mine and trend.

We are working to uplift vendor oversight across the organization and see this approach as a continued evolution to stay in front of requirements as best possible across all necessary vendor categories.

Yes I am currently working on initiatives and any new studies I have started in the last 2 years have an oversight plan, create on my own and shift with updates and priorities.

More structured documentation tools

Implementing risk based oversight, RBM and risk based IR Significant focus on documentation of activities that were previously happening but not necessarily documented, application of a risk based approach to oversight, we are adding significant robustness to our vendor oversight SOPs, plans, tools, and work instructions along with our standard SOPs and plan templates

Risk based oversight. Yes, we are currently working on new initiatives for CRO Oversight and our Risk Based Quality Management.

We are currently working on new initiatives to make this process as efficient as possible.

Time spent over-engineering process and documentation instead of managing the vendor!

More and improved access/ offerings from CROs to obtain real time access to data to view in CRO/vendor platform or internal sponsor owned data lakes/dashboards

More automation

We have always been doing vendor oversight; this is not new. What is new is perhaps a more formal process to document vendor oversight. What would be ideal would be a standardized way to ensure that vendor oversight is being conducted and documented.

Trying to build infrastructure to accommodate. Given our platform we will be looking across sites and across platform not only within study -assume this is the direction regulations will move

Centralized monitoring with monitoring reports and trend analysis. Additional CAPA activities for sites and CROs Fully electronic TMF

Still trying to implement basics

Depends on pipeline

We may choose to do more in house and outsource less.

Yes - activities planned but do not use ICH E6. For device trials, we use ISO 14155, which does not have specific vendor oversight requirements yet.

It must be risk-based. Sponsors can no longer hand-over 100% of any study conduct to a CRO.

I would see it more data driven and risk based

More involved on the Sponsor side. It requires us to know more about what our vendors are doing to meet deliverables and more "behind the scenes" stuff (ie. how they validate their systems, what is their vendor oversight practices, etc) ...things that are not just study-specific per se but business specific.

More budget needed for our central team to manage processes

Vendor oversight will become more important in the future. We are working on new initiatives and structural changes to accommodate consistent vendor oversight


2626

Q: What is the current state of your organization’s vendor oversight practices?

VENDOR OVERSIGHT: CURRENT STATE

Having a central sponsor oversight team for supplier relationships is key to help study level teams and controls in place for supplier relationship

Equally so an honest and sincere desire for collaboration between sponsor and CRO.

It's imperative to have a functional senior management oversight contact (or committee if needed) between CRO and sponsor (emphasis on "functional").

Vendor oversight for device trials, per ISO 14155.

We have tried various methods to integrate vendors into sponsor team, making they feel accountable for project but the vendor model doesn't really allow for true integration

Some dashboards and clinical programs have too much data or not emphasize data that will identify issues on compliance for pt safety and data integrity

it would be ideal to be able to quantify the impact of not doing this work earlyInteresting topic. I look forward to seeing the results of the survey.

Industry standard on what is acceptable documentation, with regulatory agency buy-in would be a big help!

It would be great for all of us to work together on how best to address this issue and share best practices, tools, and templates.

Lack of transparency related to CRO internal oversight activities, and documentation of that oversight filed to the eTMF.

Company setting is key. smaller organizations easier to streamline and implement easier than a very large global organization, different layers, roles, SOPs, processes, etc. takes a long time to implement.

Creating and maintaining a solid vendor oversight program is very challenging. It really comes down to the discipline of applying the tools and processes in a consistent manner and keeping the study teams and outsourcing team aligned on the right amount of vendor oversight and when to escalate issues.


2727THANK YOU!

Managing the Business of Inefficient Oversight: How Heavy ...

Documents

Transcript of Managing the Business of Inefficient Oversight: How Heavy ...