How to Analyze and Implement

a Research Protocol Phyllis Carelllo, CCRCBridget Adams, MSHS, CCRASusan Peterson, RN

Oregon Health & Sciences University

NIH CA P30 069533, NCRR UL1 024120

What is a protocol?

A detailed plan that sets forth the study:– Objectives– Design– Methodology

Why do we need a Protocol?

Why do we need one?– Scientific validity – Subject safety– Replicate the science if necessary– Regulatory requirements

Federal Regulations

45 CFR 46– Applies to research involving human subjects

conducted, supported or regulated by any federal department or agency

– IRB’s are required to review and approve research– IRBs have the authority to suspend or terminate

approval of research that puts subjects at risk or is not conducted in accordance with IRB requirements

21 CFR 50, and 312 or 812 – Applies to FDA regulated studies

Who Writes the Protocol?

Sponsor– Commercial manufacturer

● Pharmaceutical or Biotech company● Medical Device company

Investigators and study team– Investigator initiated– OHSU Investigator or investigator at another

institution

Industry Sponsored Protocols

Protocol generally part of larger investigational plan to get the drug/device approved by FDA– May be large multi-site trials– Changing protocol is difficult– Investigator usually has no input in to study design

Centralized research activities– Data Analysis– FDA reporting– May use a Central Lab

Funding– Sponsor usually pays for all research activities– Contract between sponsor and OHSU

Investigator-Initiated

Independently conceived and developed by scientists - the primary means by which biomedical research is conducted

Range from a small pilot study to large multi-site clinical trials May study a disease process or an intervention Investigator responsible for all research activities

– Reporting to funding and oversight entities– Data Analysis– Publications

Funding– Company (pharmaceutical, biotech, medical device)– DHHS (NIH, DOD, DOE, VA, etc.)– Foundations– Departmental– OCTRI Awards

Who Evaluates the Protocol?

IRB : Subject Protection Funding agency (NIH, Foundation): $ Federal Oversight (FDA) if applicable:

Protect Public Health Investigator: Feasibility, Scientific Interest Study Coordinator: Operational

Implementation

IRB Reviewers

Is this scientifically valid? Is it safe? Is the risk/benefit ratio appropriate? Is this ethical? Is it compliant with the regulations?

Funding/Oversight Agencies

Is it scientific valid? Is it feasible? What it the experience of the

Investigator(s)? What are the facilities and resources

available to conduct the study?

Investigator

Industry/Multicenter Studies: PI reads protocol to find out if they are interested in participating– Academic Interest?– Is it scientifically valid?– Does the PI/prospective Co-investigators have

the patients in their clinic?– Does the PI have time to conduct the study?

Coordinator

Learn the protocol requirements– What do I need to do?

May use protocol to develop checklists and data collection forms

Will the proposed budget cover the costs? Are there unfamiliar procedures?

– Specialized expertise– Training

Time– How long is study enrollment open?– Does the PI have time?– Are study personnel over-committed?

Coordinator, cont. Provide feedback on the feasibility of the study (feasibility

checklist http://www.octri.org/octri/public/index.aspx?pageid=126&siteid=1&MenuSelectedIndex=7) – Is there adequate space?

● Clinic space● Storage space● Coordinator space

– Doe we have the subject population?● In clinic● Competing studies● Advertising

– Equipment● Is equipment available for use?● Do we have to buy equipment?● Will equipment be provided by the sponsor?

Protocol Review

High

Scientific Reviewers:NIH/ Foundation

Study CoordinatorsLaboratory StaffIRB Members

Co-InvestigatorsStatisticianAuditors

Low

Level of Scientific Information

Level of Detail

“Protocol” Documents

High

Scientific Reviewers:NIH/ Foundation

Study CoordinatorsLaboratory StaffIRB Members

Co-InvestigatorsStatisticianAuditors

Low

Level of Scientific Information

Level of Detail

Protocols Contain:

High Level Scientific ConceptsGenerally describe one study Moderate Level of detailMay be revised with IRB approval

Grants Contain:

High Level Scientific ConceptsMay describe several studies Generally have little detailGenerally not revised Revisions require funding source approval

Operations Manual:

Little or no scientific informationHigh level of detail for completing tasks May be revised frequently without IRB approval (must be consistent with the IRB approved protocol)

Study Phase Considerations

Clinical Trials

Any research project that prospectively assigns human subjects to intervention and comparison groups to study the cause-and-effect relationship between a medical intervention and a health outcome.

Intervention can be:– Drug– Device– Behavioral– Educational– Other

Control can be:– Placebo– Standard of Care– Nothing

Study Trial PhasesPhase I

Population: – If healthy volunteers – where will you recruit?– If ill population – what is standard of care?

Visit Duration: – What study personnel need to be available and for how long

at each study visit?– Where do you have the space to conduct a long visit?

Design– Open label

Aim– Safety

Study Phases cont’dPhase II Population

– Criteria is generally strict – do we have the population?– Do we have conflicting studies?

Duration– Studies generally aren’t long, so when does enrollment end?– If subjects have to go off their meds for a period of time, will you

expect a high drop out rate? Design

– Randomized - Who will randomize subjects?– Double blind studies - Who is going to be available to unblind if

necessary? Aim

– Continued Safety Evaluations– Initial Efficacy Data

Study Phases Cont.Phase III

Population– Generally large # of subjects, do we have the capacity to see

them?– Representative Population (broader inclusion/exclusion criteria), do

we have these subjects in our clinic? Duration

– Parallels expected treatment use (several days to years) – do we have the capacity to follow the subjects for a long time?

Design– Generally double bind

Aims– Establish safety and efficacy in the target population

Phase IV

Population– Population prescribed the drug – who will pay for the drug?

Duration– Long Term for chronic diseases – do we have the capacity to see the

subjects for years? Design

– Generally open label – Procedures generally same as clinical care – what is research and what is

Standard of care? Who pays? Aim

– Economic comparisons– Quality of life– Adherence and compliance

Study Design

Blinding

Single Blind – subject doesn’t know what treatment group they have been assigned

Double Blind – subject and investigator don’t know treatment assignments

Randomization

Selection of treatment/control group by chance Purpose – Minimize Bias Several methods

– Fixed Allocation Randomization ● Simple (flipping a coin)● Blocked● Stratified

– Adaptive randomization● Baseline● Response randomization

Parallel Design

Subjects are assigned to study arms/groups and stay in that group for the duration of their study participation

Group 1

Group 2

Group 3

Group 4

Randomization

Cross-over Design

One group is assigned to treatment and one to a control group, at some point in the study they switch assignments

Intervention

PlaceboGroup 2

Group 1

Randomization

Open Label Extension Study

Investigators and subjects know what intervention they receive

Group 1

Observational Study

No Intervention Draw inferences about the effect of a treatment on

subjects, where the assignment of subjects into a treated group versus a control group is outside the control of the investigator – Used when it may be unethical or impractical to conduct

a randomized trial– Used for studying public health effects

May involve clinical procedures May be long term

Break!

Back in 10

Analyze, Analyze, Analyze

If we knew what it was we were doing, it would not be called research, would it?

Albert Einstein

Analyzing a Protocol

Who? What? Where? When? How? Why?

Protocol Elements

Title Introduction, background, rational, literature review Purpose(s), Objective(s) Research Methods/ Study design

● Endpoints● Stopping Rules● Duration● Drug/ Device/ Interventions schedule, modifications, precautions● Study procedures● Subjects (#, inclusion/exclusion, withdrawal criteria)● Safety Assessments

Risks Statistical Analysis

Protocol Elements, cont.

Title page● Protocol title● Sponsor● Spaces for sign-off by investigator (industry sponsor)● Date and version● Study Identifier (e-IRB#, OCTRI#, IND#, Sponsor

protocol #), as applicable

Protocol Elements, cont.

Introduction with background, rationale, and Literature Review– Provide basis for the research– Summary/Introduction

● Brief description of what is currently known about the drug and/or disease condition to be studied

● Background information (journal articles)● Justification for dose ● Route of administration

Protocol Elements, cont.

Objectives/Specific Aims– Characterize population – Scope– Duration – Outcomes– Experimental measures

Protocol Elements cont.

Study Population– Total number of subjects (including screen failures/subject

withdrawals)– Inclusion/Exclusion criteria

● Age restrictions● Diagnostic methods (HGB > 12) to determine eligibility● Pre-existing conditions

– Disallowed concomitant medications and/or treatment– Inclusion of vulnerable populations (if applicable)– Individual subject withdrawal criteria

– Allowances for temporarily stopping drug/intervention– Dose Modifications– Non-compliance

Protocol Elements cont.

Research Design and Methods– Endpoints

● Identifiable change that shows the intervention did what it was supposed to do

– Primary Endpoint: measures the specific clinical effect the intervention is preventing/treating (i.e. survival, resolution of desease)

– Surrogate endpoints: measures changes in symptom/biological indicator for the success of the intervention (i.e. T-cell count, radiographic imaging, etc.)

Protocol Elements cont.

Research Design and Methods, cont– Endpoints, cont

● Should be Measurable● Measures should be as objective as possible to

avoid bias ● Should be feasible

Example: Endpoints

Study to determine efficacy of new antibiotic for pneumonia (n=300)– Not objective: patient report of “cure”– More objective: clinical cure evaluation by

physician– Most Objective: chest x-ray

Protocol Elements cont.

Methodology and study scheduleMethodology and study schedule– Description of recruitment and enrollment proceduresDescription of recruitment and enrollment procedures– Screening and allocation of subjects to study groupsScreening and allocation of subjects to study groups– Subject numbering system to be employedSubject numbering system to be employed– Preparation of study drug for usePreparation of study drug for use– Schedule for administration of test article to subjectSchedule for administration of test article to subject– ContraindicationsContraindications– Schedule of visits and assessmentsSchedule of visits and assessments– Measures to maintain blinding/ unblinding proceduresMeasures to maintain blinding/ unblinding procedures– Required documentationRequired documentation

Protocol Elements cont.

Study measures and assessments– Physical examinations– Measurements of vital signs– Assessment procedures– Clinical Laboratory– Handling of samples– Special safety requirements

● Detailed

Protocol Elements cont.

Research Design and Methods cont.– Description of study procedures/interventions

● Research procedures described in detail so they can be performed consistently

● Common medical procedures may have no detail to allow for appropriate variation among clinicians (if scientifically acceptable)

● Procedures for carrying out physical exams, vital signs, handling samples may be very detailed if it is essential to the data collection

Identification of Unspecified Procedures

Procedures required for safe conduct of a study aren’t always listed in the protocol

Example: the protocol states a Dexa Scan will be completed at Visit 3

What is a potential unspecified procedure for a Dexa Scan?

Identification of Labs

Example: Protocol states 20 mls of blood– Why?– What tubes to you need to draw?– What is the minimum amount of blood needed?– Does the sample need to be on ice?– Will the sample be processed at OHSU or at a

central lab?– Does it need to be shipped that day?

Identification of Standard of Care

PI or MD investigator makes determination Not always obvious in the protocol Not always obvious to the MD Standard of care varies from institution to

institution Be consistent –

– don’t bill insurance for one subject and not another without a documented reason.

Protocol Elements cont.

Drug/Device – Drug preparation, administration, storage– Dose modifications– Device Implantation Procedures– Required concomitant medications, if applicable– Drug/Device accountability, retention, final

disposition

Protocol Elements cont.

– Risks● Known Risks● Protection from risks● Risk/Benefit Analysis

– Safety Assessments● Identify procedures for safety monitoring● Adverse event recognition, documentation, and

reporting requirements– May not be specifically stated in the protocol

● Special precautions for specific side effects, if required

Protocol Elements Cont

Adverse Experiences– Recognition and management– Documentation – Criteria for grading severity– Criteria for grading relatedness– Follow-up for subjects

● After an adverse event - usually follow until event is resolved● UPs that occur within 30 days of last study intervention/subject

discontinuation must be reported to the OHSU IRB

Protocol Elements Cont

Research Design and Methods (cont.)– Stopping rules/criteria

● Based on results of interim analysis– Study will be stopped if a certain pre-determined

proportion of adverse events occur– Study will be stopped if endpoints aren’t being

met – Study may be stopped if a clear benefit is shown– May be determined by sponsor, DSMB, or PI

depending on the study

Protocol Elements Cont’d

Research Design and Methods (cont.)– Duration (individual subjects and study)

● Make sure it is long enough to collect data and accommodate enrollment

● Should be no longer than required to collect enough data to evaluate study endpoints

● Balance Scientific Need with Subject Burden

Protocol Elements Cont’d

Data analyses– Variables to be analyzed– Statistical approaches to employed– Geared toward statisticians – Selection of subjects to be analyzed

● all enrolled subjects (“intent to treat”)● all eligible subjects (e.g., 80% compliant)

Other Important Study Documents

OHSU IRB Initial Review Questionnaire (IRQ)

Investigator Drug/Device Brochure Laboratory/Operations Manual Standard Operating Procedures (SOPs) Grant/Contract

Initial Review Questionnaire (IRQ)

Approved by the IRB Some information that may not be in the

protocol– Approved research staff– Funding information– # of subjects at OHSU– Recruiting and consenting procedures

Investigator Drug Brochure/ Device Manual

Reviewed by the IRB The IB contains information on the drug Study results to date All known adverse events and risks Information for the investigator on dosing Device Manual will include

implant/compatibility/programming requirements

Lab/Operations Manual

Do not require IRB approval Maybe known by other names Include minute detail on:

– Lab processing– Case Report Form Completion– Patient flow during a visit (e.g. call ahead to see if X-ray

is backed up before you head down there with a patient)– Calling and paying for a taxi– Many, many other details

SOPs

Do not require IRB approval Unit, not protocol, specific Define the who, what, where, when and how of research

activities not outlined in your protocol– Obtaining/verifying informed consent– Drug accountability

Not required of study sites per regulations (required for sponsors) but good to have:– Assure study conduct/data quality– Looked upon favorably by sponsors and regulators

If you have SOPs you will be held to them in an audit

Grant/Contract

Grants are reviewed by RGC and the IRB Contracts are reviewed and negotiated by the

Clinical Trials Office, not reviewed by the IRB Contracts must be signed by the appropriate

OHSU institutional signatory (not the PI or Department) before the study can begin

Contract

Industry sponsored trials– # of subjects you can enroll– # of screen failures– Additional procedures– Payment terms– Invoicing (or not)– Term of study

Contracts must be amended if:– PI changes– Budget Changes– Want to enroll more subjects– The term of the contract is exceeded

Break!

Back in 10

Protocol Implementation

Ready, Set, Go!

Protocol Implementation

Not quite yet …

There are a few more hoops…

Training

Research Education Requirements– Responsible Conduct of Research for all– Responsible Conduct of Research Involving

Human Subjects– Other RCR as applicable

Training, cont.

Verify study staff have training on Core Competencies – (see OHSU Clinical Research Coordinator Required

training Checklist)– Basic Life Support (BLS – initial and every two years)– Phlebotomy (contact Steve Osgood for training)– Point of care testing (contact Jane Culbertson)

● ECG, BP, Vital signs, HCG urine, Urine dipstick EPIC access/training as required

Training

Staff protocol training– Make sure study staff read and understand the

protocol – Have study staff acknowledge their roles in the

study– Make sure study staff know how to identify and

report deviations– Document that training was completed (sign-in

sheet template - see Inservice Documentation Log )

Training, cont.

Research Pharmacy Services (RPS) study initiation visit– PI/ study contact to set up appoint with

pharmacy staff per pharmacy policy (see http://www.ohsu.edu/pharmacy/rps/policies.htm)

Training, cont.

OHSU Inservice(s) If beside nurses will be involved in the study

– Complete and submit Clinical Research Study Nursing Summary (link available in the electronic IRQ)

– The PI/designee is responsible to provide special training to some or all of bedside nursing staff. Training fees may apply.

Clinical and Translational Research Institute– Inservices required for all studies requesting CTRC nursing

services– Complete CTRC nursing services request form– PI/study staff will present protocol and clarify any study specific

procedures to be conducted at the CTRC

Other considerations

Required Health screening/immunizations– TB test – Hepatitis B immunizations– Ask Employee Health if you have questions

503.494.5271

Work-flow planning

Where are prospective subjects to report? How are subjects routed during the study visit? Where are study working materials stored?

– Template order forms (lab, radiology, RPS)– MD orders– Blank consent documents

How will you communicate changes in procedures/protocols etc to study staff?

Scheduling considerations

Reconcile protocol requirements with practical constraints– Fasting?– Duration of study visit?– Study Personnel availability?– Are you dispensing research medication(s)? – Inpatient studies (who is available to discharge, who is

going to see the subject each day of admission)– Lab considerations?– MD orders to

● CTRC -48 hours in advance ● Pharmacy

Scheduling Considerations cont

– Does the subject have/need a medical record?– You need a medical number if the subjects are going to

have any clinical services at OHSU (e.g. labs, x-rays, MRI)

– Medical Record Number Request form at http://ozone.ohsu.edu/healthsystem/HIS/medrectrx/reqmrnumhim.pdf

Scheduling Example

Lumbar puncture– Who: MD available– What: LP Tray – are separate needles required?

Does the MD have a preference?– When: Time of day? Are subjects NPO? How

long is the observation period?– Where: Private room, Bed or gurney? – Why: Endpoint data?

Common Scheduling Problems

Not scheduling enough time Not scheduling in the appropriate clinical

area Not scheduling with the appropriate

personnel (MD available) Not informing everyone that needs to know Not formally getting subjects on the

clinic/CTRC/or other schedule

EPIC and Research

Consent forms and research documentation must be in Epic if subjects are having clinical services at OHSU per the OHSU Research Documentation in Medical Records Policy (coming soon) and the Informed Consent Forms for Research in Medical Records, Clin 02.15 http://ozone.ohsu.edu/healthsystem/POLICYMANUALS/Clin02MedRec/Clin02-15.pdf

Research Flags – identify a patient as a research subject and provide study- specific information to all personnel involved in their care

Setting research flags in the Integrated Health Record http://www.ohsu.edu/research/crp/docs/researchFYI.doc

Epic and Research Records

To request the ability to set research flags complete the Research Flag Access Request form at http://www.ohsu.edu/research/crp/docs/research coordinator.xls

Scheduling Cont.

Research Admission - request form available at http://ozone.ohsu.edu/research/rates/researchadmission.pdf)

Scheduling at the CTRC instructions available at http://www.octri.org/octri/public/index.aspx?pageid=183&menuid=51&siteid=1&MenuSelectedIndex=1

Equipment and Supplies

Do you have everything you need? Who is responsible for providing supplies

Industry sponsored trials usually provide lab kits(but not always)

Who is responsible for ordering? Who is responsible for tracking?

Equipment and Supplies Cont.

Use of Research Devices and Equipment– http://ozone.ohsu.edu/HealthSystems/Adm10En

viron/Adm10-04-10.html– Must be inspected by Clinical Technology

Services if the device/equipment was not purchased by OHSU

Computers– If the sponsor provides you with a computer for

the study (not purchased through OHSU) it must be inspected by ITG

Administrative Set-up

Prior to enrollment– OGA #/ FAID– Industrial Accounts/ Research Rates (see

Research Study Rate Request, Account(s) Setup, and Billing of Clinical Services, Adm 05.17 )

– Send account #s to Research pharmacy

Administrative Set-up

Clinical Trials Registration– Guidance Regarding the International Committee of Medical

Journal Editors (ICJME) Requirement for Clinical Trial Registration– OHSU does not have an institutional account # , so make sure to

register for an individual account #– Takes up to two weeks– Need to submit IRB approval memo and keep study status up to

date (minimum updates every 6 months)– Not Required if:

● Study doesn’t meet definition of clinical trial● Industry-sponsored multicenter (Sponsor registers)● Multicenter trials where OHSU is not the coordinating center

(Coordinating center registers)

Administrative Set-up

Investigational Devices– Prior to enrolling Medicare beneficiaries you

Must submit to OHSU’s Medicare Fiscal Intermediary in advance for review and approval

– http://ozone.ohsu.edu/HealthSystems/Adm05Billing/Adm05-18.html

Administrative Set -up

Off campus activities– If OHSU employees will be working where they

are directly contacting human subjects in facilities that are not owned or leased by OHSU you need to complete the Off Campus Clinical Activities Authorization Request

Incomplete Protocol Analysis and Implementation

Leads to protocol non-compliance Leads to poor decision making

– Subject Safety– Study Conclusions

Leads to protocols that go nowhere Leads to budget deficits

– More involved in the study than you thought Regulatory Headaches

Common Protocol Deviations

Non-adherence to inclusion/exclusion criteria Failure to comply with dispensing and dosing

requirements Incorrect storage of study medications Use of prohibited concomitant medications Visits outside of study windows Protocol requirements not followed

– tests not done – incorrect tests– safety labs not done

Common Protocol Deviations cont.

Failure to follow subject termination plan Failure to report adverse events Over-enrolling study Failure to follow statistical plan Failure to amend protocol and obtain IRB

approval prior to implementing changes Failure to document and report deviations

How to Avoid Deviations cont.

Follow OHSU Research and Hospital Policies:– IRB Policies:

http://www.ohsu.edu/xd/about/services/integrity/policies/

– OHSU Research Policies: http://www.octri.org/octri/public/index.aspx?pageid=123&siteid=1&menuid=1&siteid=1&MenuSelectedIndex=6

How to Avoid Deviations cont.

PI needs to supervise the conduct of the study

Delegate study tasks to qualified individuals (e.g. physical exams delegated to MD, FNP, PA)

Conducting the Protocol for Compliance

Once the study has IRB approval the study team is expected to adhere to the protocol without deviations – unless it is for the safety of the subject

Research Protocol

Conducting the Protocol cont.

Protocol Modifications– An amendment/modification can be submitted for IRB

approval to address new information/difficulties– Should analyze new procedures prior to implementation– Strive to minimize modifications so that data remains

“poolable”● Don’t want to compare apples and oranges

– Maintain consistency within and between the protocol, consent, and procedure manuals

Conducting Protocol cont.

Modifications– Re-evaluate before submitting modification to

the IRB, when appropriate, but always before implementation

● Budget● Location/Scheduling● Training● Feasibility

Conducting the Protocol cont.

Document, Document, Document– If it wasn’t documented it wasn’t done– If the procedure documentation is

questionable so is the data– Document all special circumstances that

might affect the interpretation of the results

ResearchRecords

References

Drugs Dispensed to Research Subjects, Clin 05.11 http://ozone.ohsu.edu/healthsystem/POLICYMANUALS/Clin05Meds/Clin05-11.html