Ralph Nitkin, Ph.D. - RN21E@nih.govNational Center for Medical

Rehabilitation Research (NCMRR)

Eunice K. Shriver National Institute of Child Health and

Human Development (NICHD) National Institutes of Health (NIH)

Why do Clinical Trials?Why do Clinical Trials?Assess new therapeutic approaches Modify and optimize current therapiesAdapt existing therapies to new populationsChallenge existing treatment practice/beliefsInvestigate safety and efficacy; adverse effectsExtending treatment (e.g., chronic conditions)

Evidence-based medicine is central to health-care management and policy

Theory DevelopmentTheory DevelopmentAn empirical comparison of two treatments

simply establishes that one is better than the other;

It provides no guidance how the better treatment could be improved upon, or what components of it could be sacrificed without losing potency.

A theory-based comparison eliminates a whole family of unsuccessful treatments and provides a dimension along which the better treatment can be tuned

The Gold Standard: The Gold Standard: The Randomized Clinical TrialThe Randomized Clinical Trial

•RANDOMIZATION•Balanced treatment arms (controls?), delivered in parallel

•Double-Blinded: neither participant nor assessor knows the assignment

•But other valid clinical trials models exist

Human Subjects Research IssuesHuman Subjects Research Issues“Protection of human subjects”

Risks to subjectsAdequacy of protection against risksImportance of the knowledge to be gainedPotential benefits of proposed research to

participantsRecruitment and informed consent strategiesData and Safety Monitoring Plan

HIPAA and Patient PrivacyNIH Policies on inclusion of children, of

women, and of minorities in subject population

Statistical ExpertiseStatistical ExpertiseGet expert help and get it early, especially from

those familiar with biomedical research and clinical trialsExplain your goals and hypothesesProvide information on patient heterogeneityPrimary outcomes measures and their parametrics

Provide sufficient time for discussion and be flexible in your design; take notes

Together develop a power analysis (the number of subjects required to get a conclusive answer)

Be willing to rewrite the hypothesis

Topics for DiscussionTopics for DiscussionThe treatment packageTreatment arms and controlsRecruitment and randomizationOutcome measuresDeveloping a research application

The Treatment PackageThe Treatment Package

Treatments may be MultifacetedTreatments may be Multifaceted“Treatment” often consists of a set of tasks

and activities delivered by specific disciplines in a particular dose or schedule and according to a particular protocol

But which of these are important determinants of treatment outcome?

What should be the dosing, intensity, frequency?

How do you experimentally isolate the “active ingredient”?

Operationalizing Treatment DeliveryOperationalizing Treatment Delivery

Need to standardize the delivery of treatment

Develop a manual of operations, with contingency responses to likely subject actions

Train the treatment providers and assessorsConsider periodic re-certificationStandardize delivery across multiple sitesDetailed plans to monitor for adverse

events throughout trial

Other aspects of Treatment DeliveryOther aspects of Treatment DeliveryMonitoring alterations in the subject’s

outside activities, especially those associated resulting from treatment and/or those that could impact outcomesConsider asking subject to keep an

activity logGiven your hypotheses about the “active

ingredient” in your therapy, what are the proper contrasting treatments (control) groups?

Treatment arms and ControlsTreatment arms and Controls

What about the “Controls”?What about the “Controls”?•Problems with “historical controls”? Changes in:

• base population, disease rates, disease definition, disease treatments, and available health resources

•“No treatment” probably not the proper control • Clinical equipoise: balanced uncertainty of benefit

across all treatment arms

•Treatment arms should also be balanced for psychosocial impact & subject’s belief in efficacy • Don’t underestimate the power of the placebo effect

Other strategies for Treatment ArmsOther strategies for Treatment ArmsControls:

Standard of care Sham treatment Alternative procedure

Dose-response Predicts improved outcomes with increasing dose

Cross-over design: A -> B versus B -> AMust leave sufficient time for “washout” Subject serves as own control esp. if

heterogeneityDelayed treatment (subject establishes initial

baseline, also helps dealing with heterogeneity)

Other considerationsOther considerationsResearcher must “sell” potential benefits of

each treatment armBalance expectations, compliance (placebo potential) Avoids subjects who drop out after being randomized

to the “wrong” groupBalance quality of delivery (e.g., therapists) and

support across each treatment armIf drug study, placebo pills should be matched

for appearance, maybe even well-known side effects

Minimize potential for subjects to compare notes

Consider debriefing subjects after study on expectations as to which group they were in

Recruitment and RandomizationRecruitment and Randomization

RecruitmentRecruitment<5% of U.S. population participates in clinical trials$1.76 billion of $8 billion total annual clinical

research spending is dedicated to patient enrollment efforts

Only 1 out of 20 patients who respond to clinical trial recruitment promotions eventually enrolls in a study

85% of trials don’t finish on time due to low patient accrual 65-80% of U.S. clinical trials don’t meet their end points,

largely due to challenges in patient recruitment30% fail to enroll even a single patient

Targeting Study ParticipantsTargeting Study ParticipantsGiven the proposed mechanism of action, who

can realistically benefit?Special populations (women, minorities, elderly,

rural, and poor) often require targeted intervention plans with grassroots outreach activities

One study shows:1/3 of African American women avoided clinical

trials because they didn’t trust scientists37% expressed a preference to be treated by an

African American doctorOnly 28% felt clinical research in the U.S. is

ethical

To the Participants in an RCT we:To the Participants in an RCT we:Admit that we don’t know how best to treat

their serious illnessIntroduce them to a lot of scientific jargon, and

then ask them to sign a paper agreeing:to not take something that could help them to take something that could harm them that we will flip a coin to decide which

And this process is called “Informed Consent”Hint about improved access to health services

Study Subjects Want..Study Subjects Want..•Simple explanation of the study•Clear understanding of what is expected of

them•Clear understanding of potential benefits &

risks •Whom to contact if they have questions or

concerns•To know that they can quit at any time•To eventually find out about the results and

outcomes of the trial•To feel appreciated and valued

20

Recruitment: Other considerationsRecruitment: Other considerations•HIPAA compliance is time consuming and

costly; budget sufficient staff and time•Phone recruitment and access is more limited

due to the use of caller ID•Don’t underestimate transportation and

family issues• Costs and work-time lost by

caregiver/transporter• Likelihood to remain in the area throughout trial

•Complications due to pre-morbid interactions and secondary health complications

Recruitment NetworksRecruitment NetworksBuilding recruitment networks among

cliniciansVA centers but lack sufficient patient

diversityReferring fees: Ethics & conflicts of interest

Networking with patient support and advocacy groups; building community trust

Use of professional recruitment services

Why Randomize?Why Randomize?To prevent investigator biasTo prevent patient biasTo establish balance at baselineTo provide basis for statistical

comparisons

However, subjects do not really like it . . .

Randomization StrategiesRandomization StrategiesComplete randomization (“coin flip”)Objective algorithm to achieve balance Balance across disease severity, time since

onset, and occurrence of associated conditions

Balance for gender, race and ageKey physiological factors Socio-economic status (education,

resources)

Confounds: Concurrent TreatmentsConfounds: Concurrent TreatmentsSubjects may be taking other drugs or treatments, or

may seek such treatments once enrolled in studyEliminate through Inclusion/Exclusion criteria?Ask them to voluntarily refrain during course of study?

Is this ethical? What is appropriate washout periodHow do you ensure compliance?

Subjects may minimize outside activities to get into or remain in study

Subjects don’t want to disappoint the researchers Blood tests or other ways to confirm compliance?

If concurrent treatments are prevalent, what is validity of a trial that eliminates access to them?

Allow access to other treatments but monitor as covariate?

Outcome MeasuresOutcome Measures

Initial ConsiderationsInitial ConsiderationsLimit the number of primary outcomesDefine which outcomes are primary vs secondaryOutcome measures should be at same conceptual level

as intervention, and driven by your hypothesesUse outcomes that are widely accepted and clinically

relevant (vs merely statistically relevant)Measures that have been used on this subject populationProbably not a good time to invent new measures

Just because something can be measured does not mean that it is a necessary an outcome measure

Outcomes: Other Considerations Outcomes: Other Considerations Single measures versus the use of a test

batteryThe same treatment may be judged effective or

ineffective depending on outcome measures chosenSingle measures may miss effectBut need to minimize subject burden

Consider the use of Surrogate MarkersSaves time and money, and potential safety

benefitUse surrogate markers widely accepted/validatedMeasures that are directly related to primary

outcome and the level of interventionConsider any other potential confounds in their

interpretation

Outcomes: Other ConsiderationsOutcomes: Other ConsiderationsTiming: How soon after intervention? How often? Follow-up to test durability of treatment effectEcological validity: Measures connected to real

world goals; outcomes that impact subject’s life Generalization: do results extend to broader

environments? Make sure that assessors are blinded; Also make sure that the patients do not give away

their treatment assignment to the assessors

Outcomes: SummaryOutcomes: SummaryUse valid measures that are most sensitive

and selective for expected changesAvoid ceiling and floor effectsMinimize subject burden and fatigueMeasures sensitive to monitor change over timeMinimize variability (e.g., time of day, season)Clinical trial should not take credit for what

Nature does: change due to disease progression and/or natural recovery processes

Developing a Research Developing a Research ApplicationApplication

NIH Discussion PointsNIH Discussion PointsWhy should it be done?

Need, relevance, timelinessCurrent practice perspectivesExpected impact of the results on practice

Who is the target population?Disease, condition, subgroupsInclusion/exclusion criteria

Phase of the trial: pilot to efficacyStudy designOutcome measure(s)

Likelihood of success

Pre-ClinicalPre-Clinical

Safety and DoseSafety and Dose

Dose ConfirmationDose Confirmation

Phase III ProtocolPhase III Protocol

Manual of OperationsManual of Operations

Perform Phase III TrialPerform Phase III Trial

R01 Pilot Clinical Trials

R01 Randomized Trial

R01 Basic Research

R34 Planning Grant

Review ConsiderationsReview ConsiderationsAdequacy of preliminary data and experience

of applicant teamAppropriateness of inclusion / exclusion criteriaProtection from Type I and Type II errorsPower analysis and sample sizeRecruitment and retention strategies; evidence

of ability to recruit the projected number of patients

Randomization and blinding strategiesOutcome measures & data analysis; data

securityPrimary outcome measure(s) clearly defined;

clinical relevance; hierarchy of other outcomes

Top 10 pitfalls in Clinical Trials10) Inadequate involvement of statistical

expertise9) Too many “outcomes” proposed8) Overly restrictive inclusion/exclusion

criteria7) Insufficient resources6) Rush to propose an efficacy trial, when

more piloting required, esp. optimizing dosing

Top 10 pitfalls in Clinical Trials5) Inadequate data management plans4) Inadequate subject recruitment plans3) Hypothesizing an unrealistically large

treatment effect2) Believing that submitting a lousy

application will yield useful feedback from the review committee (First impressions count!)

1) Failing to consult with the NIH before writing the application