Why we misunderstand medical evidence: A view from the shop floor DR. JAMES GALLOWAY CLINICAL...
-
Upload
jodie-ball -
Category
Documents
-
view
219 -
download
2
description
Transcript of Why we misunderstand medical evidence: A view from the shop floor DR. JAMES GALLOWAY CLINICAL...
Why we misunderstand medical evidence:A view from the shop floorDR. JAMES GALLOWAYCLINICAL LECTURER AND CONSULTANT RHEUMATOLOGISTKING’S COLLEGE LONDON
Focus of the modern medical school curriculum:
Patient centered care
Emphasis on communication
Safe practice
Medical education Medical students have 5 years to gain a superficial understanding of human health and disease
Post Graduation Learning becomes more disease focused
Attention to practice guidelines◦ Areas of uncertainty (artificially) diminishing
Relevance of methodology understated◦ Clinicians encouraged to just get on and do research◦ Methodological support sparse, sometimes confrontational and often
lacking in enthusiasm
“you should have come and seen me months ago, not 24 hours before your deadline…”
“no, my p value isn’t wrong, you collected too few data. And yes, the word data is plural.”
Designing Clinical Research
Ideas, questions, hypotheses
Idea◦ I have seen quite a few patients on anti-TNF therapy admitted to hospital
with infections
Question◦ Does anti-TNF therapy a predispose people to infections?
Hypothesis◦ Patients on anti-TNF therapy are more likely to experience a serious infection
compared to patients treated with other anti-rheumatic treatments
Designing Clinical Research
A wish list for my trainees:
1) They should understand what a p value means
(and why a series of good anecdotes ≠ evidence)
P values on a pedestal
Renal carcinoma in the US
The counties with the lowest incidence of cancer are mostly rural, sparsely populated and located in traditionally Republican states in the Midwest, the South and the West.
“less pollution, no food additives, healthier lifestyle”
Renal carcinoma in the US
The counties with the highest incidence of cancer are mostly rural, sparsely populated and located in traditionally Republican states in the Midwest, the South and the West…
Training clinicians not to see patterns when they do not exist
What do Google, Facebook and Twitter have in common?
◦ They all have brand names that repeating letters (‘oo’ or ‘tt’)
Heuristics and medicine:
A 21 year old female has been admitted with chest pain. She has a rash.
◦ Emergency physician: “Panic attack”◦ Pulmonologist: “Pulmonary embolism”◦ Cardiologist: “Myocarditis”◦ Rheumatologist: “Lupus”
Designing clinical research
A wish list for my trainees:
2) They should understand the difference between statistical significance and clinical relevance
Interventional studies
Conclusion:
“This study demonstrates that CS improves hand pain and function in patients with symptomatic OA of the hand and shows a good safety profile.”
Gabay et al. Arthritis & Rheumatism, 2011
Designing clinical research
A wish list for my trainees:
3) They should understand the concept of statistical power
A negative result does not mean a drug does not work…
Sample sizes
(amount of money I have)*√(random estimate of how many people might agree to participate)
n =(cost of one patient being put through trial)
Sample sizes It is unethical to under power a study
Sample sizes It is unethical to under power a study
Methotrexate in Giant Cell Arteritis?
◦ Meta analysis of 3 RCTs◦ Hazard ratio for relapse: 0.65 (p = 0.4)
Damascene meta analysis BMJ 2014
Designing clinical research
A wish list for my trainees:
4) They should understand the difference between relative and absolute risks
50% increased cardiovascular risk
Incidence rates (BSRBR data)
Results nbDMARD All TNF
Follow-up, patient-years 9259 36,230
Number of SIs 296 1512
Rate/1000 patient-years (95% CI) 32 (28–36) 42 (40–44)
Unadjusted HR Ref. 1.5 (1.3–1.7)
adjHR* (95% CI) Ref. 1.2 (1.1–1.5)
Follow-up, months 0–6 Ref. 1.8 (1.2–2.6)
6–12 Ref. 1.4 (0.9–2.0)
12–24 Ref. 1.2 (0.8–1.6)
24–36 Ref. 0.9 (0.6–1.3)
Rheumatology (Oxford). 2011;50:124-31
Hazard ratios (BSRBR data)
Results nbDMARD All TNF
Follow-up, patient-years 9259 36,230
Number of SIs 296 1512
Rate/1000 patient-years (95% CI) 32 (28–36) 42 (40–44)
Unadjusted HR Ref. 1.5 (1.3–1.7)
adjHR* (95% CI) Ref. 1.2 (1.1–1.5)
Follow-up, months 0–6 Ref. 1.8 (1.2–2.6)
6–12 Ref. 1.4 (0.9–2.0)
12–24 Ref. 1.2 (0.8–1.6)
24–36 Ref. 0.9 (0.6–1.3)
Rheumatology (Oxford). 2011;50:124-31
Describing risk to patients
Absolute
“On anti-TNF therapy, your risk of infection is increased by 1%”
Relative
“On anti-TNF therapy, your risk of infection is increased by 50%”
Frustratingly, both of these statements are technically correct
• The infection rate on anti-TNF is 4% per year, which is 1 percentage point higher than that observed in the comparator group (3%)
• However, relatively speaking, the rate of infection in people on anti-TNF increases by 50%
Designing clinical research
A wish list for my trainees:
5) They should understand causal inference
Causal / casual assumptions
“Methotrexate is an important cause of chronic lung damage in patients with rheumatoid arthritis”
Channeling biasDisease
OutcomeIntervention
Causal inference
In search of causality
Making sense of it allYes, take a closer look.
You can’t be serious?
Well, just to be safe I’m going to
assume that cancer causes mobile
phones.
This association between mobile phone use and
cancer is fascinating isn’t it?
Well, I actually think they have it all the wrong way round.
Designing clinical research
A wish list for my trainees:
6) They should understand the sensitivity and specificity
Quick test A 50 year old woman undergoes screening mammography. Her mammogram is positive.
She is worried and asks her doctor what the chance of her having breast cancer is based upon this test result.
What is the probability that she has breast cancer?OPTIONS
a) 9 in 10
b) 8 in 10
c) 1 in 10
d) 1 in 100
BACKGROUND INFORMATION
The probability that a woman has breast cancer is 1% ("prevalence")
If a woman has breast cancer, the probability that she tests positive is 90% ("sensitivity")
If a woman does not have breast cancer, the probability that she nevertheless tests positive is 9% ("false alarm rate")
Results Out of 160 respondents (all doctors) only 21% answered correctly.
◦ Most thought the correct answer was 9 in 10.
1000 women
10 have cancer
9 test +ve
1 tests -ve
990 don’t
89 test +ve
901 test -ve
1 in 10
My methodology wish list for doctors:
1) Understand why statistics are needed
2) Appreciate significance ≠ relevance
3) Recognise the value of statistical power
4) Comprehend absolute and relative risk
5) Understand concept of causation
6) Distinguish sensitivity and specificity
Teaching the teachers Challenge trainees
Use active learning (keep it relevant)
Ensure teachers care about the subject
Teachers must be credible leaders
Feedback, feedback, feedback
◦ Move with the times… Facebook, Twitter…
In conclusion Doctors have a skill set that does not naturally align with mathematics
The emphasis of the modern curriculum has evolved to a less scientific model
◦ The result of a weak understanding of statistics is that doctors misinterpret evidence and patients are misinformed about risk and benefit
We need to make medical statistics more prominent in the curriculum
Final wish… All students should be taught to mistrust pie charts and anyone who uses them.