Clinical Trials in Clinical Trials in SurgerySurgery

Mazen S. Zenati, MD, MPH, Ph.D.Mazen S. Zenati, MD, MPH, Ph.D.

University of Pittsburgh,University of Pittsburgh,Department of Surgery and EpidemiologyDepartment of Surgery and Epidemiology

Modern Clinical Trials in Modern Clinical Trials in HistoryHistory

Clinical trials began in 1800 onward to Clinical trials began in 1800 onward to proliferate and more attention was proliferate and more attention was paid to study design. paid to study design.

Placebos were first used in 1863Placebos were first used in 1863 The idea of randomization was The idea of randomization was

introduced in 1923. introduced in 1923. The first trial using properly The first trial using properly

randomized treatment and control randomized treatment and control groups also featured double blind groups also featured double blind assessment was carried out in 1948 assessment was carried out in 1948 by the Medical Research Council, and by the Medical Research Council, and involved the use of streptomycin to involved the use of streptomycin to treat pulmonary tuberculosis.. treat pulmonary tuberculosis..

The Need for Clinical TrialsThe Need for Clinical Trials in Surgery in Surgery

In surgery many plausible clinical theory In surgery many plausible clinical theory are institutionalized into practice before are institutionalized into practice before as systematic risk-benefit, cost-as systematic risk-benefit, cost-effectiveness and long term quality of effectiveness and long term quality of life have been measured and evaluatedlife have been measured and evaluated

Many surgical procedures implemented Many surgical procedures implemented for years before we discovered the for years before we discovered the ineffectiveness to eventually discredited:ineffectiveness to eventually discredited: Halsted radical mastectomyHalsted radical mastectomy Kidney decapsulation (for hypertension)Kidney decapsulation (for hypertension) Uterine suspensionUterine suspension Legation of internal mammary arteries (for Legation of internal mammary arteries (for

coronary insufficiency)coronary insufficiency)

The Need for Clinical TrialsThe Need for Clinical Trials in Surgery.. cont. in Surgery.. cont.

New surgical procedures are not regulated New surgical procedures are not regulated in the same way new drugs or devices arein the same way new drugs or devices are

New procedures can be approved in the New procedures can be approved in the hospital based on the surgeon ability to hospital based on the surgeon ability to convince the patients and some time upper convince the patients and some time upper management of the potential benefit of the management of the potential benefit of the new idea or the technique. new idea or the technique.

No enough cases or controls, No enough cases or controls, Protocols and IRB in many cases are often Protocols and IRB in many cases are often

not requirednot required No proper documentation of the outcome in No proper documentation of the outcome in

systematic mannersystematic manner No real plan for study or follow upNo real plan for study or follow up

The Unique Nature of Surgical The Unique Nature of Surgical TrialsTrials

Blinding of patients or Surgeon is almost Blinding of patients or Surgeon is almost infeasibleinfeasible

Placebo-controlled procedures are Placebo-controlled procedures are ethically controversial specially if ethically controversial specially if invasiveinvasive

The placebo effect is powerful in surgical The placebo effect is powerful in surgical procedures and should be identifyprocedures and should be identify

The bias of the investigator due to the The bias of the investigator due to the need for that procedure or program to need for that procedure or program to be successfulbe successful

Patient selection, extra skills in Patient selection, extra skills in performance, and extra care and performance, and extra care and attention (efficacious vs. effective)attention (efficacious vs. effective)

The Unique Nature of The Unique Nature of Surgical Trials.. cont.Surgical Trials.. cont.

Difficulty in selecting centers to Difficulty in selecting centers to represent the practicing community represent the practicing community (VA sites vs. academic centers)(VA sites vs. academic centers)

Standardization across the sites is Standardization across the sites is difficultdifficult

When we compare a well-established When we compare a well-established technique with an alternative, the skills technique with an alternative, the skills and experience in the first might bias and experience in the first might bias the comparison (the need for video the comparison (the need for video tapes and operative reports for QC)tapes and operative reports for QC)

Clinical trial in Surgery Clinical trial in Surgery Necessary forNecessary for

To test the safety and efficacy of a To test the safety and efficacy of a new technology in the short and long new technology in the short and long term outcometerm outcome

To test if the new technology is Cost To test if the new technology is Cost effectiveeffective

To test if the new technology is at To test if the new technology is at least as effective and/or safe as least as effective and/or safe as existing proven techniquesexisting proven techniques

To test for the feasibility of To test for the feasibility of implementing the new technology in implementing the new technology in relation to the needed skills/training, relation to the needed skills/training, equipment, and other factors.equipment, and other factors.

Barriers to Clinical Trials in Barriers to Clinical Trials in SurgerySurgery

Underutilized due to difficulties in design Underutilized due to difficulties in design and recruitmentsand recruitments

The high cost: hospitalization, operation, The high cost: hospitalization, operation, personnel, and related special method of personnel, and related special method of data collection and analysisdata collection and analysis

Funding is difficult to obtain from public Funding is difficult to obtain from public and agenciesand agencies

Peer-reviewers, IRB, and other over Peer-reviewers, IRB, and other over criticisms (protecting patients right and criticisms (protecting patients right and safety) delay the process and takes years safety) delay the process and takes years that the timeliness of introduction may be that the timeliness of introduction may be lost lost

Barriers to Clinical Trials in Barriers to Clinical Trials in Surgery.. cont.Surgery.. cont.

Multicenter trials might starts in Multicenter trials might starts in different point with unbalance different point with unbalance recruitments)recruitments)

Funding fro the industry carries its Funding fro the industry carries its own risk of design and ownershipown risk of design and ownership

Mastering complex procedures Mastering complex procedures related to learning curverelated to learning curve

Consent documents very sensitive Consent documents very sensitive (the risk is more of an open ended (the risk is more of an open ended format)format)

Forces Opposing Clinical Forces Opposing Clinical Trials in SurgeryTrials in Surgery

The magnitude of potential benefits The magnitude of potential benefits might subside the necessity for wait might subside the necessity for wait and for the total prove through a and for the total prove through a clinical trialclinical trial

The high risk might hold recruitment The high risk might hold recruitment for a control clinical trialfor a control clinical trial

The wide range and high number of The wide range and high number of potential recipients also a factor in potential recipients also a factor in determine the need compared to a determine the need compared to a rare disease that may require special rare disease that may require special surgical procedure. surgical procedure.

ExceptionException

When we deal with disease that When we deal with disease that carry very bad prognosis as in carry very bad prognosis as in oncology, cardiology, and oncology, cardiology, and transplant where the intervention transplant where the intervention is highly morbid and invasiveis highly morbid and invasive

The evidence-based medicine The evidence-based medicine appraised by patients themselves appraised by patients themselves and third-party payers might and third-party payers might modify a total embracement modify a total embracement traditionally done by the surgical traditionally done by the surgical communitycommunity

Randomized Randomized ControlControl TrialTrial

Can we really have a controls in surgical Can we really have a controls in surgical clinical trials?clinical trials?

Controls is it:Controls is it: Not to operate on half of the patients?Not to operate on half of the patients? Mock operation not related to the indication?Mock operation not related to the indication?

Can we really deny half of the patients in Can we really deny half of the patients in need for surgical interventions just for the need for surgical interventions just for the sake of research? sake of research? Are we then doom half of them to their death? Are we then doom half of them to their death? But which half is that? But which half is that? (will be really exposed to a greater (will be really exposed to a greater

eventful hazard?)eventful hazard?) The interventions one or the controls one… do The interventions one or the controls one… do

we really know?we really know?

The Basic design of randomized The Basic design of randomized Controlled Clinical TrialControlled Clinical Trial

Population

EligibleIneligible

Agree to Participate, Informed Consent

Randomization

ControlsCurrent standard

No therapy/placebo

InterventionNew surgical

technique

Good Outcome

BadOutcome

Good Outcome

Bad Outcome

Decline to Participate

Recruitment

Eligibility Criteria

Barriers to Performing Barriers to Performing Randomized TrialsRandomized Trials

Surgeon bias for or against Surgeon bias for or against specific proceduresspecific procedures

Morbidity associated with Morbidity associated with surgerysurgery

The for granted acceptance of The for granted acceptance of lesser forms of clinical evidence lesser forms of clinical evidence by the surgical community by the surgical community

Difficulties in recruitmentsDifficulties in recruitments Ethics and legality … consentsEthics and legality … consents

Threats of the Lack of Threats of the Lack of Randomized Control GroupRandomized Control Group

Internal validity related to the Internal validity related to the temporal trendstemporal trends

Surgical learning curve effectsSurgical learning curve effects Regression to the meanRegression to the mean The frequent lack of equipoise The frequent lack of equipoise

among surgeonsamong surgeons Amplifying the fact that the Amplifying the fact that the

clinical practice outpaces clinical practice outpaces research evaluationresearch evaluation

RCT vs. Quasi-experimental RCT vs. Quasi-experimental Study DesignStudy Design

RCT minimize the measurable confounding RCT minimize the measurable confounding that potentially biasing the study's that potentially biasing the study's conclusions.conclusions.

RCT reduces selection biasRCT reduces selection bias RCT however, restricts the generalizablity of RCT however, restricts the generalizablity of

the results because of the strict inclusion the results because of the strict inclusion and exclusion criteriaand exclusion criteria

Some worthy to consider designs in case of Some worthy to consider designs in case of unlikely to achieve RCTunlikely to achieve RCT The use of properly selected nonrandomized The use of properly selected nonrandomized

control groupscontrol groups Using pre and post comparisonsUsing pre and post comparisons

Bias and ValidityBias and Validity

Bias:Bias: Systematic error within the study Systematic error within the study that results in mistaken estimate of the that results in mistaken estimate of the effect of therapy on diseaseeffect of therapy on disease

Internal validity:Internal validity: The ability of a trial The ability of a trial to come to the correct conclusion to come to the correct conclusion regarding the question being studiedregarding the question being studied

External validity:External validity: The ability of trial to The ability of trial to produce results that are generalizable produce results that are generalizable to a larger population of patients with to a larger population of patients with disease disease

Formulating Research QuestionFormulating Research Question

RCT to answer as narrow as a possible clinical research RCT to answer as narrow as a possible clinical research questionquestion

To answer an important question justifying:To answer an important question justifying: Recruitment of large number of patients,Recruitment of large number of patients, The high expenses of the trialThe high expenses of the trial The need to compare: the exist of legitimate uncertainty The need to compare: the exist of legitimate uncertainty

between the effective of at least two therapies of the same between the effective of at least two therapies of the same disease (clinical equipoise)disease (clinical equipoise)

In surgery there are many patients avoid surgeryIn surgery there are many patients avoid surgery Many surgeons are and perhaps patients reluctant to relinquish the Many surgeons are and perhaps patients reluctant to relinquish the

control of choice of treatment of deeply held beliefs of most control of choice of treatment of deeply held beliefs of most appropriate treatmentappropriate treatment

Ideal problems to study by RCT:Ideal problems to study by RCT: Similar morbidity of the procedure using for example similar Similar morbidity of the procedure using for example similar

incisionsincisions The disease has high morbidity, new procedure promise to The disease has high morbidity, new procedure promise to

improve outcomeimprove outcome Disease with current treatment result in potential long term Disease with current treatment result in potential long term

morbidities that will may justify the risk for exploring a new morbidities that will may justify the risk for exploring a new choicechoice

Issues in Formulating Issues in Formulating Research QuestionResearch Question

What are the end points What are the end points Can these end points be accurately and Can these end points be accurately and

reliably measuredreliably measured What potential surrogate end points What potential surrogate end points

are available and at what expenses to are available and at what expenses to the internal validitythe internal validity

All the above in consideration to All the above in consideration to practicality, expenses, and feasibility of practicality, expenses, and feasibility of RCTRCT

Pragmatic trials vs. explanatory trialsPragmatic trials vs. explanatory trials

Pragmatic vs. Pragmatic vs. ExplanatoryExplanatory

Pragmatic: attempt to stimulate clinical Pragmatic: attempt to stimulate clinical realities more accurately. The outcomes can realities more accurately. The outcomes can be easily generalized and accepted into clinical be easily generalized and accepted into clinical practicepractice

Explanatory: attempts to answer more specific Explanatory: attempts to answer more specific and narrow question (strict inclusion and and narrow question (strict inclusion and exclusion criteria, homogeneous population)exclusion criteria, homogeneous population)

Eligibility CriteriaEligibility Criteria

Inclusions and exclusionsInclusions and exclusions Important to determine the external Important to determine the external

validityvalidity The study population should not be The study population should not be

too narrow or to broadtoo narrow or to broad It is a balance between statistical It is a balance between statistical

advantage and the generalizability of advantage and the generalizability of the resultsthe results

Pragmatic trial tend to be broader in Pragmatic trial tend to be broader in inclusioninclusion

Explanatory trials are narrower in Explanatory trials are narrower in focusfocus

Outcome Measurement in Outcome Measurement in SurgerySurgery

Traditional outcome measures:Traditional outcome measures: MortalityMortality

MeasuresMeasures Mortality rate, crude mortality rates, specific Mortality rate, crude mortality rates, specific

mortality rates, adjusted mortality rates, mortality rates, adjusted mortality rates, standardizedstandardized

MorbidityMorbidity MeasuresMeasures

Prevalence, point prevalence, period Prevalence, point prevalence, period prevalence, incidence, cumulative incidence, prevalence, incidence, cumulative incidence, attack rate, incidence rateattack rate, incidence rate

Speed of recovery: ICU LOS, Hospital Speed of recovery: ICU LOS, Hospital LOS…LOS…

Health related quality of LifeHealth related quality of Life Measuring patients satisfactoryMeasuring patients satisfactory

Example on EligibilityExample on Eligibility

A study in vascular surgery to compare between a A study in vascular surgery to compare between a newly MIT and the conventional approach in treating newly MIT and the conventional approach in treating occlusive peripheral vascular disease of lower occlusive peripheral vascular disease of lower extremitiesextremities Inclusion criteria patients< 65 yr with clinical and Inclusion criteria patients< 65 yr with clinical and

radiographic evidenceradiographic evidence Randomly allocated for the above interventionsRandomly allocated for the above interventions End points: angiographic resolution of the occlusionEnd points: angiographic resolution of the occlusion

Typical patients may be older than 65 yr, lack of Typical patients may be older than 65 yr, lack of generalizability lower the external validitygeneralizability lower the external validity

Eligibility criteria should be more objectively definedEligibility criteria should be more objectively defined Severity of lesion, symptomsSeverity of lesion, symptoms

Endpoint might not be the best to determinant of Endpoint might not be the best to determinant of therapeutic success: therapeutic success:

Patient symptoms, activity levels, and wound healing (more Patient symptoms, activity levels, and wound healing (more appropriate and more relevant clinically) appropriate and more relevant clinically)

Example on selecting Example on selecting EndpointEndpoint

A RCT to comparing between prothrombin complex A RCT to comparing between prothrombin complex concentrate and FFP in emergency need (surgery/ concentrate and FFP in emergency need (surgery/ trauma) for reversing the effect of warfarin. The trauma) for reversing the effect of warfarin. The INR has been suggested as an endpoint but the INR has been suggested as an endpoint but the FDA did not accept the proposal and asked for FDA did not accept the proposal and asked for more clinically related outcome. Any suggestion?more clinically related outcome. Any suggestion?

A RCT to test the effect of preoperative oral A RCT to test the effect of preoperative oral Citrulline on surgical wound healing . The speed of Citrulline on surgical wound healing . The speed of wound healing, strength of the healing or LOS was wound healing, strength of the healing or LOS was though of as endpoint outcomes. Anything wrong though of as endpoint outcomes. Anything wrong with these?with these?

Sample SizeSample Size

We might need in surgery to conduct first a We might need in surgery to conduct first a pilot nonrandomized observational study on pilot nonrandomized observational study on homogenous population to determine the homogenous population to determine the differences anticipated in outcome between differences anticipated in outcome between the randomized groups in case we do not the randomized groups in case we do not have enough datahave enough data

The above will be also a good source to The above will be also a good source to estimate the variability in the outcome data: estimate the variability in the outcome data: i.e. standard deviation, or standard errori.e. standard deviation, or standard error

What is clinically relevant is more important What is clinically relevant is more important of just statistically significant in power of just statistically significant in power analysisanalysis

Statistical consultation in this regard is very Statistical consultation in this regard is very essential and should be obtained early in the essential and should be obtained early in the course of study preparationcourse of study preparation

Controls and Level of Controls and Level of EvidenceEvidence

Controls to increase the level of evidence Controls to increase the level of evidence without controls is just as case series study without controls is just as case series study which is important in surgery to obtain all the which is important in surgery to obtain all the descriptive analysis of outcome but will not descriptive analysis of outcome but will not provide evidence of superiority (biased)provide evidence of superiority (biased)

Without control we can not account for the Without control we can not account for the placebo effectplacebo effect

Without control we can not account for Without control we can not account for confounding variablesconfounding variables

Historical controls might be used frequently in Historical controls might be used frequently in surgery when a prospective controls is beyond surgery when a prospective controls is beyond the reach, specially when performed by the the reach, specially when performed by the same surgeon and in the same institution same surgeon and in the same institution Population and the time change in technology still Population and the time change in technology still

differdiffer

Controls and Level of Controls and Level of EvidenceEvidence

Variables that we usually match with in Variables that we usually match with in selecting controls in surgical trials beside selecting controls in surgical trials beside all inclusion and exclusion criteria:all inclusion and exclusion criteria: Age, sex, BMI, Age, sex, BMI, Comorbidities, preexisting illnessComorbidities, preexisting illness Disease stage, duration, severity, ISS, Disease stage, duration, severity, ISS,

mechanism of injury, sitemechanism of injury, site Surgeon, procedureSurgeon, procedure Others: smoking, alcohol use, depression, Others: smoking, alcohol use, depression,

insurance coverage, etcinsurance coverage, etc

““Still the superior level of evidence attributed to RCT Still the superior level of evidence attributed to RCT is contingent on a successful randomization”is contingent on a successful randomization”

Randomized Trial in Randomized Trial in SurgerySurgery

Random allocation of the subjects to the trial groupsRandom allocation of the subjects to the trial groups Breaks the link between any unmeasured Breaks the link between any unmeasured

confounding variables and treatment statusconfounding variables and treatment status Assume all differences in effect between treatment Assume all differences in effect between treatment

groups to be a result of the differences in treatment groups to be a result of the differences in treatment (absent confounding)(absent confounding)

Randomization is the major challenge clinical trials Randomization is the major challenge clinical trials in surgery and causes considerable weakness the in surgery and causes considerable weakness the design when not satisfactory attained design when not satisfactory attained The Challenge is convincing patients to relinquish control of The Challenge is convincing patients to relinquish control of

their care to random process by whether or not they their care to random process by whether or not they receive surgery or even which kind of surgery they actually receive surgery or even which kind of surgery they actually receivedreceived

Randomized trials in surgery can be either Randomized trials in surgery can be either Pragmatic or ExplanatoryPragmatic or Explanatory

Pragmatic vs. Pragmatic vs. ExplanatoryExplanatory

It is similar toIt is similar to effectiveness effectiveness vs. vs. efficacyefficacy in in drug trials where:drug trials where:

Effectiveness:Effectiveness: is the ability of an intervention to is the ability of an intervention to accomplish its intended outcome in population accomplish its intended outcome in population under real-life circumstancesunder real-life circumstances

Efficacy:Efficacy: Is the ability of an intervention to obtain Is the ability of an intervention to obtain its intended outcome under ideal situationits intended outcome under ideal situation

Most randomized trials in surgery tend to be Most randomized trials in surgery tend to be categorized as pragmatic “effectiveness”, categorized as pragmatic “effectiveness”,

comparing surgical technique and comparing surgical technique and outcome under usual clinical conditionsoutcome under usual clinical conditions

RandomizationRandomization

Stabilizes the internal validity of the studyStabilizes the internal validity of the study Randomization processRandomization process can be predicted by specific can be predicted by specific

logistics and practical constraints of the study. The logistics and practical constraints of the study. The strong desire to operate or not “by a resident” my strong desire to operate or not “by a resident” my tempt for the selection bias.tempt for the selection bias. The process must be truly unpredictable and concealed from The process must be truly unpredictable and concealed from

the investigator. Allocate the patient to randomization after the investigator. Allocate the patient to randomization after obtaining the consent. Using random number table/a obtaining the consent. Using random number table/a computer generated and then enclosed in opaque, serially computer generated and then enclosed in opaque, serially numbered enveloped.numbered enveloped.

Investigator selection biasInvestigator selection bias in selecting the in selecting the intervention to subjects whenever more than one intervention to subjects whenever more than one patient is presented at the same time as in a busy patient is presented at the same time as in a busy trauma center for instancetrauma center for instance Use distance randomization by phone or internetUse distance randomization by phone or internet““Randomized Trials appear to annoy human nature-if properly conducted, indeed Randomized Trials appear to annoy human nature-if properly conducted, indeed

they should”they should”

Randomization, cont..Randomization, cont..

Block Randomization:Block Randomization: to result to result equal distributed patients equal distributed patients between the arms of the studybetween the arms of the study

Stratified Randomization:Stratified Randomization: to to assure certain traits are equal assure certain traits are equal among treatment groupsamong treatment groups

Example on RandomizationExample on Randomization

In a busy trauma center, RCT took place to In a busy trauma center, RCT took place to assess whether abdominal ultrasound or assess whether abdominal ultrasound or diagnostic peritoneal lavage was more diagnostic peritoneal lavage was more accurate in diagnosing intra-abdominal injury accurate in diagnosing intra-abdominal injury after blunt trauma. In the ER a clinical indicator after blunt trauma. In the ER a clinical indicator was used to determine the need for rapid was used to determine the need for rapid abdominal assessment. If met, a resident abdominal assessment. If met, a resident would open the next opaque, serially would open the next opaque, serially numbered envelope stored in a designated numbered envelope stored in a designated folder in the ED to indicate which assessment folder in the ED to indicate which assessment would be used. In such busy trauma centers, would be used. In such busy trauma centers, two or more patients may presented two or more patients may presented simultaneously. At the end of trial, it was noted simultaneously. At the end of trial, it was noted that, more often than not, diagnostic peritoneal that, more often than not, diagnostic peritoneal lavage was performed on thinner patients!lavage was performed on thinner patients!

Example on RandomizationExample on Randomization

Two cases or more at the same time Two cases or more at the same time might offer the busy resident an might offer the busy resident an opportunity to open more than one opportunity to open more than one envelope at once and subsequently envelope at once and subsequently decide which patients receive which decide which patients receive which assessment.assessment.

The process was corrupted by a third The process was corrupted by a third part motivated by self interest. part motivated by self interest. Likely/hopefully not understanding Likely/hopefully not understanding the potential effects on the outcome the potential effects on the outcome of the trial.of the trial.

Masking…. BlindingMasking…. Blinding

To prevent or reduce observation bias and placebo effect.To prevent or reduce observation bias and placebo effect. Observation bias:Observation bias: is systematic variation between the is systematic variation between the

true outcome and the outcome observed true outcome and the outcome observed (consciously or (consciously or unconsciously)unconsciously)

Placebo effect:Placebo effect: exaggerate the effects of treatment based exaggerate the effects of treatment based on their enthusiasm and expectationson their enthusiasm and expectations

Single blinded:Single blinded: treatment assignment is concealed from subject treatment assignment is concealed from subject onlyonly

Double-blinded: Double-blinded: treatment assignment is concealed from both treatment assignment is concealed from both subject and the investigator who ascertains the study end pointsubject and the investigator who ascertains the study end point

Contrary to studies of pharmacologic therapies for a Contrary to studies of pharmacologic therapies for a disease, blinding in surgery is not that easy:disease, blinding in surgery is not that easy: Logistic Challenges: we do not have placebo surgery similar to Logistic Challenges: we do not have placebo surgery similar to

placebo tablets. We have to use sham surgeryplacebo tablets. We have to use sham surgery Ethical Challenges: many of the sham-surgery interventions are Ethical Challenges: many of the sham-surgery interventions are

considered unethical by today’s standardsconsidered unethical by today’s standards

Masking…. Blinding.. Masking…. Blinding.. Cont.Cont.

Using blinded third party (not the Using blinded third party (not the operating surgeon) might be necessaryoperating surgeon) might be necessary

We could verify the effectiveness of We could verify the effectiveness of blinding by asking all study participants blinding by asking all study participants to guess which treatment they had to guess which treatment they had received.received. If the patient’s guesses were no better than If the patient’s guesses were no better than

random chance.. indicated that the blinding random chance.. indicated that the blinding remained successfulremained successful

Oftentimes, it is not possible to perform Oftentimes, it is not possible to perform a randomized trial in blinded fashion: a randomized trial in blinded fashion: different surgical techniques, therapies, different surgical techniques, therapies, approaches, or incisionsapproaches, or incisions

Example on BlindingExample on Blinding

In 1959, Cobb et al. reported a RCT designed to In 1959, Cobb et al. reported a RCT designed to determine if the internal mammary artery ligation was determine if the internal mammary artery ligation was effective in the treatment of angina. The researchers effective in the treatment of angina. The researchers enrolled 17 patients with classic symptoms and enrolled 17 patients with classic symptoms and recorded preoperative severity of symptoms as well recorded preoperative severity of symptoms as well as stress test electrocardiogram data. The patients as stress test electrocardiogram data. The patients then underwent surgery. After the internal mammary then underwent surgery. After the internal mammary arteries were isolated, an envelope was opened and arteries were isolated, an envelope was opened and directed to either ligate or simply end the operation. directed to either ligate or simply end the operation. The patients were informed preoperatively that the The patients were informed preoperatively that the ligation procedure was experimental, but they ligation procedure was experimental, but they unaware of the randomized nature of the study. Post unaware of the randomized nature of the study. Post operatively data were collected and found that the operatively data were collected and found that the patients undergoing the sham procedure had greater patients undergoing the sham procedure had greater symptomatic relief than those undergoing ligation, symptomatic relief than those undergoing ligation, and one patient in the sham group had reversal of and one patient in the sham group had reversal of stress test electrocardiogram abnormalities. Overall, stress test electrocardiogram abnormalities. Overall, patients in both groups showed only modest patients in both groups showed only modest improvement.improvement.

Example on Blinding.. Example on Blinding.. cont.cont.

The importance of sham surgery, The importance of sham surgery, without it (as case series design) the without it (as case series design) the author could incorrectly concluded author could incorrectly concluded that intervention had a meritthat intervention had a merit

The differences might be related to:The differences might be related to: Placebo effect caused the improvement Placebo effect caused the improvement

in both groupsin both groups Something beside the trial intervention Something beside the trial intervention

caused the improvementcaused the improvement The observed benefit was part of the The observed benefit was part of the

natural history of the diseasenatural history of the disease

Patient Crossover and Patient Crossover and Intention-To-Treat AnalysisIntention-To-Treat Analysis

Intention-To-Treat Analysis: Analyze the outcome Intention-To-Treat Analysis: Analyze the outcome based on the original randomization assignments, based on the original randomization assignments, regardless of treatment actually receivedregardless of treatment actually received

The alternative: Analyze the data based on the The alternative: Analyze the data based on the treatment actually received not by random treatment actually received not by random allocation allocation (as if here we are breaking the randomization (as if here we are breaking the randomization introducing selection bias since crossover does not occur introducing selection bias since crossover does not occur by chance)by chance)

A high crossover rates questioned the validity of A high crossover rates questioned the validity of the studythe study

High crossover rates in surgical clinical trials is High crossover rates in surgical clinical trials is commoncommon

Especially in trials comparing surgical with Especially in trials comparing surgical with medical interventionsmedical interventions

Patient Crossover and Patient Crossover and Intention-To-Treat Analysis.. Intention-To-Treat Analysis..

cont.cont. There is a specific need to multiple There is a specific need to multiple

statistical methods and mixed modeling statistical methods and mixed modeling techniques to deal with crossovertechniques to deal with crossover

We stick with Intention-To-Treat We stick with Intention-To-Treat Analysis approach whenever Analysis approach whenever randomization is preserved and in order randomization is preserved and in order to keep the statistical assumptions valid to keep the statistical assumptions valid for the analysis approachesfor the analysis approaches

What is the nonrandom factor that What is the nonrandom factor that caused patients to crossovercaused patients to crossover

These nonrandom factors will biases These nonrandom factors will biases the study resultsthe study results

Example on Example on CrossoverCrossover

From 1972 through 1974, the Veterans From 1972 through 1974, the Veterans Administration Coronary Artery Bypass Surgery Administration Coronary Artery Bypass Surgery Cooperative Study Group enrolled patients into a Cooperative Study Group enrolled patients into a randomized study comparing coronary artery randomized study comparing coronary artery bypass surgery with medical therapy in patients bypass surgery with medical therapy in patients with angina and radiographic evidence of CAD. with angina and radiographic evidence of CAD. Patients were randomized to receive either Patients were randomized to receive either surgical or medical therapy. After 14 yr of follow surgical or medical therapy. After 14 yr of follow up, 55% of patients assigned to receive medical up, 55% of patients assigned to receive medical therapy had crossed over to receive surgical therapy had crossed over to receive surgical treatment, whereas 6% of the patients assigned treatment, whereas 6% of the patients assigned to receive surgical treatment decided to not to receive surgical treatment decided to not undergo the procedure and were thus managed undergo the procedure and were thus managed clinically. Should we analyze the data based on clinically. Should we analyze the data based on the intention-to-treat (original randomization) or the intention-to-treat (original randomization) or analyze the data based on the treatment actually analyze the data based on the treatment actually received (not by random allocation)?received (not by random allocation)?

Example on Example on Crossover.. Crossover.. Cont.Cont.

Should we analyze based on the Should we analyze based on the intention-to-treat with such large intention-to-treat with such large cross over. Is not that seem cross over. Is not that seem counterintuitive of the randomization counterintuitive of the randomization process?process?

Should we accept to break the Should we accept to break the randomization and allowing the randomization and allowing the introduction of selection bias by introduction of selection bias by analyzing according to the actual analyzing according to the actual treatment received?treatment received?

ConclusionsConclusions

Surgeons have fallen behind the rest of medical community partly Surgeons have fallen behind the rest of medical community partly because of the challenges required to complete a valid because of the challenges required to complete a valid randomized trial of surgical therapyrandomized trial of surgical therapy Clinical Trials in surgery is underutilizedClinical Trials in surgery is underutilized

Recent era of increase in medical evidence demand from policy Recent era of increase in medical evidence demand from policy maker and caregivers and even patients resulted in an increase in maker and caregivers and even patients resulted in an increase in the demand of randomized clinical trials for publications and the demand of randomized clinical trials for publications and quality assurance. And partly due to our acceptance and reliance quality assurance. And partly due to our acceptance and reliance on lesser forms of evidenceon lesser forms of evidence Clinical Trials in surgery is urgently neededClinical Trials in surgery is urgently needed

Randomized clinical trials in surgery are not easy but still can be Randomized clinical trials in surgery are not easy but still can be accomplished with the appropriate planning and selecting suitable accomplished with the appropriate planning and selecting suitable designdesign

Good controls, selection of end point, randomization, blinding are Good controls, selection of end point, randomization, blinding are the keys for successful CTthe keys for successful CT

We need to standardized the ascertainment of outcome, We need to standardized the ascertainment of outcome, attempting to quantify outcome as objectively as possibleattempting to quantify outcome as objectively as possible

We also should minimize the potential observation bias and We also should minimize the potential observation bias and placebo effect by selecting the most appropriate designplacebo effect by selecting the most appropriate design

Final ThoughtsFinal Thoughts

Only until clinical trails in surgery became the gold Only until clinical trails in surgery became the gold slandered to prove safety and effectiveness of the slandered to prove safety and effectiveness of the treatment, the current practice will continue to be treatment, the current practice will continue to be dominant by opinions or the enthusiasm for a dominant by opinions or the enthusiasm for a procedure.procedure.

Still however the surgeon's Judgment will never be Still however the surgeon's Judgment will never be a dispensablea dispensable

The results still need to be interpreted in light of The results still need to be interpreted in light of the study design and clinical reasonablenessthe study design and clinical reasonableness

The better commitment to evidence-based medical The better commitment to evidence-based medical practice and the persistence and the application of practice and the persistence and the application of novel approaches to overcoming difficult novel approaches to overcoming difficult methodological hurdles will be continuously methodological hurdles will be continuously necessary to advance medicinenecessary to advance medicine

References and recommended Reading:

John Gallin, Principles and Practice of Clinical Research, Academic Press, 2002

H. Troidl, Surgical Research, Springer, 1987 David Machin, Text book of Clinical Trials, Wiley, 2006 John Wei, Clinical Research Methods for Surgeons,

Humana Press, 2006 Friedman LM, Fundamentals of clinical Trials-Third Edition,

PSH Inc, 1998 Ovid MEDLINE: http://www.hsls.pitt.edu/resources/ovid/ PubMed: http://www.ncbi.nlm.nih.gov/sites/entrez?

holding=upittlib FDA: http://www.fda.gov/cder/guidance/iche6.htm Clinical Research Office for Surgery and Trauma (CROST),

University of Pittsburgh, Department of Surgery: http://www.surgery.upmc.edu/General/research.htm

Clinical Trials in SurgeryClinical Trials in Surgery