Staffing and Workload Management Strategies

Speakers

Phil Butera, MBAAssistant Vice President, Clinical TrialsLevine Cancer Institute, Atrium Health

Renee Rakvica, RN, BSN, OCNDirector of Site ServicesUS Oncology Research

Oncology Hematology Care, Inc.

Marge Good, RN, MPHNurse Consultant – Program Director

National Cancer Institute

IntroductionMarge Good, RN, MPHNational Cancer Institute

Topics to be Covered

ASCO Clinical Trial Workload Assessment Tool

Levine Workload/Staffing Assessment Tool

Diversified Workforce in Research

Career Ladders

Staffing and Retention Strategies

ASCO Clinical Trial Workload Assessment Tool

Marge Good, RN, MPHNational Cancer Institute

Assessing Staffing Mix and FTEs Over Time: An Example

1999‐00

2000 ‐ 01

2001 ‐ 02

2002 ‐ 03

2003 ‐ 04

2004 ‐ 05

2005 ‐ 06

2006‐07

2007‐08

2008‐09

2009‐10

Oncology Service Line Director 0.5 0.5 0.5 0.5 0.5 0.1 0.1 0.1 0.1 0.1 0.1

CCOP Manager 1 1 1 1 1 1 1 1 1 1 1

Secretary/Assistant 0 0 0 1 1 1 1 1 1 1 1

Regulatory Specialist 1 1 2 2 2 2 2.5 2.5 3 3 2

CRA ‐ Treatment Focused 1 1 1 1 1 1 1 6.25 8.25 9.2 8

CRA ‐ Cancer Control Focused 1 1 2 2 3 3 2.5 2.5 3 2.9 3

Research Nurse ‐ Treatment Focused 6 7 9 10 11 11 11 10 10 8 7

Research Nurse ‐ Cancer Control Focused

0 0 2.5 4 5 5 4.5 5.5 5 5 5

Total FTE 10.5 11.5 18 21.5 24.5 24.1 23.6 28.8 31.4 30.2 27.1

Total New Enrollments 430 1424 653

Treatment Trial Average Acuity 22.8 19.3 45.6 37.6

Cancer Control Trial Average Acuity 13.8 8.97 69.8 38.8

Off Treatment Follow‐Up Acuity 14.2 10.4 39.3 39.3

Wichita CCOP data: 1999 - 2010

Early years• Primarily supported with research nurses• Adjuvant treatment trials• Minimal biospecimen collection/submission• Cancer control primarily symptom management• Everyone cross trained

2001 - 2006• Prevention trials introduced

Later years• Acuity increased across all categories (number

of patients and trial complexity)• Fewer oncology nurses available• Non-nurse CRAs hired – work

redistributed/teams• More biospecimen (kit room required)• Insurance issues

ASCO Research Community Forum 2019 Annual Meeting

1

7

Why Assess Clinical Trial Workload?

Have actual metrics of research staff effort (not just statements “I’m overwhelmed”)

Provide validation of need for more staff

Budget justification (institution, grant applications, etc.)

Tool for staff management

Assess and ensure equal distribution of work

Change in metrics over time / Signals indicating reaching maximum effort

Staff-specific issues (accruing patients, knowledge gaps, etc.)

Monitor data submission delinquency

Interim and annual performance reviews

8

Implications for Assessing Clinical Trial-Associated Workload

Consistent Workload

Assessment

Balance between

staff

Staff Satisfaction

Quality data

More trial options

Higher Accrual Rates

Increased funding

ASCO Research Community Forum Membership Survey

Conducted in Spring 2011

Goal – Assess needs related to conduct of clinical trials

How helpful would various research-related projects be ifdeveloped by ASCO? • Ranked 4th out of 12 → Workload Assessment Tool

ASCO’s Research Community Forum convened a WorkloadAssessment Working Group

Literature Review SummaryName Pub Year Model/Focus/Metric Findings

Fowler & Thomas Acuity Rating Tool(Research Practitioner 4(2):64-71. 2003)

2003 Points assigned to protocol tasks. Time in hrs/protocol task X # points = score

500 – 750 points/coordinator3 – 7 trials per coordinator

NCI Trial Complexity Elements & Scoring Model(http://ctep.cancer.gov/protocolDevelopment/docs/trial_complexity_elements_scoring.doc )

2009 Points assigned for each of 10 elements Standard complexity = 0 ptsMod complexity = 1 ptHigh complexity = 2 pts

None reported

US Oncology Research Study Clinical Coordination Grading(Unpublished. Personal communication)

2009 Points assigned to each of 21 grading criteria. Complexity based on number of points (↑ points = ↑ score)

None reported

Ontario Protocol Assessment Level (OPAL)(Smuck, et al: JOP 7(2):80-84. 2011)

2011 Score of 1-8 assigned based on # of contact events, type of trial

None reported

University of Michigan – ResearchEffort Tracking Application (RETA)(James, et al: J of NCCN 9(11):1228-1233. 2011)

2011 Staff logged daily time spent per protocol tasks

70-75% staff time = trial-related tasks25-30% = non-trial (vacation, mtgs, etc)72% of DM effort ->opening studies25% effort ->not yet open/closed

Wichita CCOP Protocol Acuity Tool (WPAT)(Good, et al: JOP 9(4):211-215. 2013)

2013 Trials ranked 1-4 based on 6 complexity elements

Data collected over 10 years* Yrly average Acuity Score per nurse: Tx=30.6; CC=37.8;Off S=15.9* Yrly average Pts per nurse: New enrollments=69;On S=103;Off S=97

Goals:

1. Develop a tool that is simple, reproducible, and usable in the longterm Implement within community research programs

Establish clinical trial workload metrics or benchmarks

2. To help research sites assess staff workload based on: Complexity of research protocols

Number of patients assigned to each research nurse and CRA

3. Design Clinical Trial Workload Assessment Tool Developed web-based/electronic data capture tool

Goal = Test tool in multiple community-based research sites

Participating Sites

51 completed 6 months of data collection May through November 2013

5 Groups based on type and size

1. Group 1: CCOPs/MBCCOPs < 7 FTEs (13)

2. Group 2: CCOPs/MBCCOPs > 7 FTEs (10)

3. Group 3: Community hospitals/NCCCPs (8)

4. Group 4: Non-hospital-based private practice/private research networks (12)

5. Group 5: hospital-based private practice (7)

ASCO Research Community Forum 2019 Annual Meeting

2

Two Acuity Metrics

Protocol Acuity Score Scored 1 to 4 (Per Protocol Acuity Scoring Worksheet)

On Study/On active treatment

Follow-up (assumed 1)

o On Study/Off active treatment

o Off Study

Nurse/CRA Acuity Score (Patient Centered Effort) Calculation

protocol acuity score x number of patients

Individual Nurse/CRA FTE

Protocol Acuity Scoring Worksheet

Complexity of treatment,Trial specific laboratory and/or testing requirements,Treatment toxicity potential,Data forms required (consider complexity and number of forms),Degree of coordination required (involvement of ancillary

departments, outside offices/sites and/or disciplines),Number of randomizations/steps.

Conclusions

ASCO tool intuitive/simple and usable

Collecting data from across multiple sites was complicated, butachievable

Applicable to academic and community settings

CT Workload Benchmark data now available

Good, et al. Journal of Oncology Practice, 2016. 12(5):457-458

Assessing workload requires time and commitment

More work needs to be done

e.g., screening, regulatory, etc.

Tool Available on ASCO Website

Go to www.workload.asco.org.

It’s free but you need to register to use it!

ASCO Web-Based Workload Tool

As of August 15, 2019

Total Registrants: 403 unique sites

United States: 371

International: 32

Australia, Brazil, Canada, Chile, China, India, Ireland, Italy, South Korea, Spain, Saudi Arabia, Thailand, United Kingdom, Switzerland

Community-based: 253

Academic: 116

Other (e.g., government, etc.): 34

Workload Capacity Planning - Levine Cancer Institute

Phil Butera, MBALevine Cancer Institute, Atrium Health

ASCO Research Community Forum 2019 Annual Meeting

3

What we are doing?

Evaluating protocol complexity

Understanding the make-up of our teammates experience and what this resourcing available capacity consists of

Analyzing the level of actual work effort occurring

Forecasting our actual and pending trial accruals

Benefits of reviewing capacity for work life balance and organizational process improvements

Evaluation of Protocol Complexity

Evaluation is based upon protocol driven criteria as well as LCI processes Some of the key drivers for patient and protocol management are:

– # of sites– # of modalities– Are there PKs and if so, how many and what is the frequency– Are there specimens to be collected– Any fresh bio-specimens– Are there patient diaries, QOLs, self-assessments using a tablet– Imaging requirements, frequency, central reads– Screening – Visit frequency – Follow up frequency– Estimated rate of accrual based upon patient population and past history

Evaluation of Protocol Complexity

Utilizing 4-point scale for treatment trials based upon the make-up of approved protocols– Assessment of the scale is continuous as new trials may have more complex requirements not

already accounted for

– The scale can change for active patient management based upon where a patient is in theirtreatment plan

Active Complexity Values– Protocol complexity values from 1 – 4 (low to high complexity)

– Phase I trials assigned a complexity value of 4 with some exceptions

– Complexity values for specimen/registry trials 0.05, 0.25, 0.5

– Values entered in CTMS per protocol

Follow-Up Complexity Values– Follow-up complexity values of 0, 0.05, and 0.25

– Zero value used when there is no follow-up

– 0.05 used for Long Term Follow Up (LTFU)

– 0.25 used for normal follow-up once off treatment

– Values entered in CTMS per protocol

Evaluation of Protocol Complexity

Protocol No. Title PI Name Current Status

Current Status Date Phase

Protocol Type

PrincipalSponsor

Sponsor Type Oncology Group

Total Accrual

Lowerl Accrual Goal

Upper Accrual

Goal

Active Protocol

ComplexityFU Protocol Complexity

Theoretical Lower

Accrual Goal Work Unit Capacity

Theoretical Upper Accrual

Goal Work Unit Capacity

Actual Protocol Accrual

Work Unit Capacity

Theoretical Lower WU

Cap vs Actual WU Cap

Theoretical Upper WU

Cap vs Actual WU Cap

ABC 123 A Phase II trial … . White, P eter OTA 01/07/2017 II T reatment Pharma 1 Indus try Lung 7 3 10 3 0.25 9 30 21 ‐12 9

BC A456 A Phase II trial … . Adams, William OTA 07/13/2016 II T reatment Pharma 2 Indus try Lymphoma 2 5 8 3 0.25 15 24 6 9 18

L JD678 A Phase II trial … . Brown, J ordan OTA 08/09/2017 II T reatment Pharma 3 Indus try Ovary 5 5 10 3 0.25 15 30 15 0 15

YTT987 A Phase I trial … . S mith, R obert O TA 04/20/2015 I T reatment Pharma 4 Indus try Multiple Myeloma 6 3 10 4 0.25 12 40 24 ‐12 16

R FG234 A Phase I trial … . Late, C arol O TA 02/11/2016 I T reatment Pharma 5 Indus try Breas t 5 4 8 4 0.25 16 32 20 ‐4 12

PO L131 A Phase II trial … . Young, Michelle OTA 08/06/2017 II T reatment Pharma 6 Indus try GI 2 2 5 3 0.25 6 15 6 0 9

Active complexity values are used for ongoing and pending trials to provide a forecasted workload estimate

– These values are used in conjunction with our contracted lower and upper accrual goals or are set internally where not defined

The theoretical upper work unit values can be aggregated and used to determine if there is enough overall capacity based upon actual protocol accruals

This evaluation can also be used to see what protocols may be requiring more effort faster than expected

Understanding Resourcing for Research Nurses & CRAs

Developed a theoretical Work Unit (WU) capacity value for each FTE

Created ranges for expected WU’s based on the level of experience as defined below

0-8 months – 12 WU’s

9-18 months – 24 WU’s

>18 months – 36 WU’s

Assess the level of workload an FTE can undertake for the positions of research nursesand CRAs

Experience is updated monthly

Cap FTE capacity at 80%

23

The total number of months is based upon time with LCI and previous research experience

Understanding Resourcing for Research Nurses & CRAs

Theoretical WU capacity for each FTE is used to assess available WU capacity for patient management

FTE WU values based upon the management of 12 active patients from low to high complexity protocols (12 – 36)

Ability to aggregate theoretical WU capacity for each role to forecast potential shortfalls of workload

24

ASCO Research Community Forum 2019 Annual Meeting

4

Department Workload

25

All staff workload is reviewed for the department

Workload for the roles of clinical research nurse and CRA’s areaggregated separately to determine overall capacity for each

Workload can be compared for each staff member over time to look at trends based upon assignments

This helps to guide us directionally over time to determine if we have future resourcing needs based upon the type of protocols and number of patient's we are accruing

Forecasting Active and Pending Trial Accrual

26

A report is generated looking at ongoing accruals by protocol

The number of total accruals per protocol is broken down into a monthly accrual number based upon the length of time the protocol has been open to accrual (OTA)

This monthly accrual number is multiplied by the protocol complexity to calculate the forecasted number of WU’s

The forecasted number of WU’s is aggregated for all OTA protocols and compared to our available WU’s of our staff

Forecasting Active & Pending Trial Accrual

27

This helps us define if we will have any potential gap in the ability to accrue patients for existing OTA protocols

The monthly accrual estimate for pending trials is determined by the PIwhich is multiplied by the protocol complexity to determine what the expected WU will be when the protocol is OTA

Biweekly meetings are held where all pending trials are reviewed with the latest status for the start-up meeting, budget, contract, IRB submission, SIV, and target OTA date

The target OTA date is used as a placeholder to forecast WU’s and these WU’s are added to the monthly estimated OTA protocols

What’s next? What do you do with this data?

28

All the calculations and reports do not help if you do not pressure test them with your staff through ongoing workload discussions

Meetings

Weekly resourcing meetings are held where our managers review workload to determine if there are any resourcing needs within teams and across the department

One on one meetings between managers and their staff are held to review individual workload

Examine trends over time and outliers

Review any over capacity to determine if it is necessary to make any workload adjustments

What’s next? What do you do with this data?

29

Monthly Research Review meetings held with our Clinical Trial Leadership team and Medical Director for Clinical Trials Each staff members workload is reviewed in detail

Look at previous month’s capacity for trending across areas

Discuss how each staff member is progressing with their assignments

Discuss how newer staff are managing initial assignments and progressing in their role

Biweekly Pending Trial Scheduling Meetings Attended by Budget Specialist, Contract Specialist, Regulatory, Start-Up team, Clinical

managers, Director Clinical Trials

Held for Adult non-IIT, IIT, and Peds trials

Decisions are made for the pace of opening pending trials

The Other 20%

30

20% protected time accounts for the below areas Administrative Time – Defined as internal meetings, not protocol

specific, professional development, training not protocol specific, project work

Time off

Site travel time – less frequent to more frequent

Monitoring visits – include if more frequent visits are occurring

Data locks – include when occurring

Serving as PC/PCRA

Different treatment plan intervals – volume of patients in the higher acuity part of the trial at the same time

ASCO Research Community Forum 2019 Annual Meeting

5

What else does a capacity model allow you to do?

31

Use workload capacity to justify existing FTEs and for new FTE requests helping to reduce staff attrition

Review of time and effort (T&E) data to assist in validating protocol complexity

Use protocol complexity to aid the budgeting process

Include pre-screening T&E percentages in workload capacity review

Use workload capacity to make smarter assignments across the department and develop alternate work patterns for work-life balance

Utilize capacity model to create stretch assignments to support career advancement opportunities aligned with our career ladder.

Going Forward

32

Do not overcomplicate and maintain simplicity in the model

Knowing that review of protocol complexity criteria can be somewhat subjective, use past reviews to inform future evaluations

Continue 1:1 meetings with staff to understand different treatment plan intervals – # of patients in the higher acuity part of the trial at the same time

Continue to understand when staff have multiple patients on the same protocol at different timepoints in their treatment plan

Continually evaluate internal processes to determine if more available capacity exists for patient management activities

Diversified Workforce in Research Career Ladders Staffing and Retention Strategies

Renee Rakvica, RN, BSN, OCNUS Oncology Research, Oncology Hematology Care, Inc.

Monitoring VisitsTreatment Plan

SAE Reporting

Adverse Event Evaluation

RegistrationBaseline Screening ProceduresConsenting Tissue CollectionLab Work

Response Evaluation

Follow-up

Patient Identification

Symptom Management

Drug Accountability Physician Oversight

Patient Compliance

Radiological Exams

Source DocumentationTrainingImaging Transmittal

Regulatory Documents

Audits

eDC

Case Report Forms

Patient Education

Patient DiariesCentral Labs

PKs

In-services

Billing Guides

CROs Sponsors

Re-consents

Research Finances

Research Responsibilities

Job Diversification

Research Responsibilities

Clinical Job Roles Responsibilities

Clinical Research Nurse Supports management and coordination of tasks for multiple clinical research studies Screening patients for eligibility, ICF and enrollmentParticipates in the subject’s study visits and required activities per protocol.Assuring protocol compliance for all patients enrolled in the trial and perform ongoing assessments and documentation in collaboration with physicians and other providers, and participation in required training and education

Clinical Research Coordinator

As above- Non-NurseFirst focus at non-treatment trials

Research Assistant Central Labs- expert in lab manual: request, track, prep, sendTissue request, submission and trackingMay assist with patient care: vitals, EKGMay be assist with Follow-up patient schedulesMay support data collection and entries

Research Pharmtech Drug Accountability: receiving drugs, temp monitoring inventory, returning, unblinded personnel Participates in SIVs

ASCO Research Community Forum 2019 Annual Meeting

6

Other Job Roles Responsibilities

Data Coordinator Collection, coordination, processing and quality control of clinical trial data, maintaining research protocol and other research files as applicable. Works with clinical staff to meet data entry timelines and communicate any concerns about the data. SIV facilitation and Monitor contact

Financial Coordinator Contracts/ BudgetsReconciling revenueTracking paymentsWorks closely with vendors and research team

Research Regulatory Coordinator

Submission, tracking and maintenance of regulatory documentsIRB submissionsSend and tracking of protocol trainingPSSV/SIV involvementWork closely with the monitors

NEW- Data Query Nurse Involved in resolution of clinical queriesWork closely with DCs and Clinical TeamMaintain tracker to identify trendsPres-screening patient for clinical trials

CAREER LADDERS

§ Separated Research nurse vs. Research Coordinator§ Research Nurse Supervisor vs. Research Manager§ Research Data Coordinator Lead/Supervisor§ Added Levels:– Clinical Research Nurse and CRC: 1,2,3– DC: 1,2, 3-Lead– Research Assistant 1 and Senior– Finance: 1 and Senior level

RETENTION

§ Training/Mentoring Program§ Source and Communication templates§ Disease program focus§ Office coverage§ Staff meetings and one on one with staff§ MDs available and engaged§ Recognition§ Enjoyable environment

Meet CAR-TResearch New Pet

ASCO Research Community Forum 2019 Annual Meeting

7

What are ways to address staff turnover? Do you feel you have an effective research staffing model?

If yes, what makes it effective?

If not, why not?

What are your greatest challenges?

Are there staffing or workload issues that could be a priority for the RCF?

Wrap Up

Share your resources and strategies!

Join the online forum and/or email researchcommunityforum@asco.org

ASCO Research Community Forum 2019 Annual Meeting

8