Sociology 6Z03 McMaster University - McMaster Faculty of ...
PPlanlan - McMaster University Evidence-Based Clinical...
Transcript of PPlanlan - McMaster University Evidence-Based Clinical...
Plan Plan • • GRADE background GRADE background
• • two steps two steps – – quality of evidence quality of evidence – – strength of recommendation strength of recommendation
• • quality and strength can differ quality and strength can differ
• • profiles and summary of findings profiles and summary of findings
• • importance of values/preferences importance of values/preferences
Plan Plan • • GRADE background GRADE background • • two steps two steps
– – quality of evidence quality of evidence – – strength of recommendation strength of recommendation
• • evidence profiles evidence profiles
• • an exercise in applying GRADE an exercise in applying GRADE
Plan Plan • • GRADE background GRADE background • • two steps two steps
– – quality of evidence quality of evidence – – strength of recommendation strength of recommendation
• • quality and strength can differ quality and strength can differ
• • profiles and summary of findings profiles and summary of findings
• • importance of values/preferences importance of values/preferences
• • an exercise in applying GRADE an exercise in applying GRADE
Plan Plan • • GRADE background GRADE background
• • two steps two steps – – quality of evidence quality of evidence – – strength of recommendation strength of recommendation
• • importance of values/preferences importance of values/preferences
• • an exercise in applying GRADE an exercise in applying GRADE
Plan Plan • • GRADE background GRADE background
• • two steps two steps – – quality of evidence quality of evidence – – strength of recommendation strength of recommendation
• • application to breast cancer screening application to breast cancer screening – – contrast with USPSTF contrast with USPSTF
Summarizing recommendations Summarizing recommendations
• • clinicians need succinct summaries clinicians need succinct summaries
• • should include should include – – quality of evidence quality of evidence – – summaries of best estimates of effect summaries of best estimates of effect
• • all patient all patient- -important outcomes important outcomes – – strength of recommendations strength of recommendations
• • GRADE working group GRADE working group – – BMJ 2004 and 2008 BMJ 2004 and 2008
• • Is grading recommendations a good Is grading recommendations a good idea? idea?
• • Why? Why?
• • experience with grading experience with grading – – systems used? systems used?
Why Grade Why Grade Recommendations? Recommendations?
• • strong recommendations strong recommendations – – strong methods strong methods – – large precise effect large precise effect – – few down sides of therapy few down sides of therapy
• • weak recommendations weak recommendations – – weak methods weak methods – – imprecise estimate imprecise estimate – – small effect small effect – – substantial down sides substantial down sides
Which grading system to use? Which grading system to use? • • many available many available
– – Australian National and MRC Australian National and MRC – – Oxford Center for Evidence Oxford Center for Evidence- -based Medicine based Medicine – – Scottish Intercollegiate Guidelines (SIGN) Scottish Intercollegiate Guidelines (SIGN) – – US Preventative Services Task Force US Preventative Services Task Force – – American professional organizations American professional organizations
• • AHA/ACC, ACCP, AAP, Endocrine society, etc.... AHA/ACC, ACCP, AAP, Endocrine society, etc....
• • cause of confusion, dismay cause of confusion, dismay
A common international A common international grading system? grading system?
• • GRADE ( GRADE (G Grades of rades of r recommendation, ecommendation, a assessment, ssessment, d development and evelopment and e evaluation) valuation)
• • international group international group – – Australian NMRC, SIGN, USPSTF, WHO, NICE, Australian NMRC, SIGN, USPSTF, WHO, NICE,
Oxford CEBM, CDC, CC Oxford CEBM, CDC, CC
• • ~ 25 meetings over last ten years ~ 25 meetings over last ten years • • (~10 (~10 – – 50 attendants) 50 attendants)
GRADE Uptake GRADE Uptake Agencia sanitaria regionale, Bologna, Italia Agency for Health Care Research and Quality (AHRQ) Allergic Rhinitis and Group - Independent Expert Panel American Association for the study of liver diseases American College of Cardiology Foundation American College of Chest Physicians American College of Emergency Physicians American College of Physicians American Endocrine Society American Society of Gastrointestinal Endoscopy American society of Interventional Pain Physicians American Thoracic Society (ATS) BMJ Clinical Evidence British Medical Journal Canadian Agency for Drugs and Technology in Health Canadian Cardiovascular Society Canadian Task Force on Preventive Health Care Centers for Disease Control Cochrane Collaboration EBM Guidelines Finland Emergency Medical Services for Children National
Resource Center European Association for the Study of the Liver European Respiratory Society European Society of Thoracic Surgeons Evidence-based Nursing Sudtirol, Alta Adiga, Italy Finnish Office of Health Technology Assessment
German Agency for Quality in Medicine Heelth Inspectorate for Scotland Infectious Disease Society of America Japanese Society of Oral and Maxillofacial Radiology Joslin Diabetes Center Journal of Infection in Developing Countries Kaiser Permanente Kidney Disease International Guidelines Organization National and Gulf Centre for Evidence-based Medicine National Institute for Clinical Excellence (NICE) National Kidney Foundation Norwegian Knowledge Centre for the Health Services Ontario MOH Medical Advisory Secretariat Panama and Costa Rica National Clinical Guidelines Program Polish Institute for EBM Scottish Intercollegiate Guideline Network (SIGN) Society of Critical Care Medicine Society of Pediatric Endocrinology Society of Vascular Surgery Spanish Society of Family Practice (SEMFYC) Stop TB Diagnostic Working Group Surviving sepsis campaign Swedish Council on Technology Assessment in Health Care Swedish National Board of Health and Welfare University of Pennsylvania Health System for EB Practice UpToDate WINFOCUS World Allergy Organization World Health Organization (WHO)
What are we grading? What are we grading?
• • two components two components
• • quality of body of evidence quality of body of evidence – – extent to which confidence in estimate of extent to which confidence in estimate of
effect adequate to support decision effect adequate to support decision • • high, moderate, low, very low high, moderate, low, very low
• • strength of recommendation strength of recommendation • • strong and weak strong and weak
What are we grading? What are we grading?
• • two components two components
• • quality of body of evidence quality of body of evidence – – confidence in estimate of effect confidence in estimate of effect
• • high, moderate, low, very low high, moderate, low, very low
• • strength of recommendation strength of recommendation • • strong and weak strong and weak
Interpretation of quality Interpretation of quality • • High quality High quality— — Further research is very unlikely to Further research is very unlikely to
change our confidence in the estimate of effect change our confidence in the estimate of effect • • Moderate quality Moderate quality— — Further research is likely to Further research is likely to
have an important impact on our confidence in the have an important impact on our confidence in the estimate of effect and may change the estimate estimate of effect and may change the estimate
• • Low quality Low quality— — Further research is very likely to Further research is very likely to have an important impact on our confidence in the have an important impact on our confidence in the estimate of effect and is likely to change the estimate of effect and is likely to change the estimate estimate
• • Very low quality Very low quality— — Any estimate of effect is very Any estimate of effect is very uncertain uncertain
Interpretation of quality Interpretation of quality • • High: We are very confident that t High: We are very confident that the true effect lies he true effect lies
close to that of the estimate of the effect. close to that of the estimate of the effect.
• • Moderate: We are moderately confident in the effect Moderate: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it estimate of the effect, but there is a possibility that it is substantially different. is substantially different.
• • Low: Our confidence in the effect estimate is limited: Low: Our confidence in the effect estimate is limited: The true effect may be substantially different from the The true effect may be substantially different from the estimate of the effect. estimate of the effect.
• • Very low: We have very little confidence in the effect Very low: We have very little confidence in the effect estimate: The true effect is likely to be substantially estimate: The true effect is likely to be substantially different from the estimate of effect. different from the estimate of effect.
Health Care Question (PICO)
Systematic reviews
Studies
Outcomes
Important outcomes
Rate the quality of evidence for each outcome, across studies RCTs start high, observational studies start low ()Study limitations Imprecision Inconsistency of results Indirectness of evidence Publication bias likely
Final rating of quality for each outcome: high, moderate, low, or very low
(+) Large magnitude of effect Dose response Plausible confounders would ↓ effect when an effect is present or ↑ effect if effect is absent
Decide on the direction (for/against) and grade strength (strong/weak*) of the recommendation considering:
Quality of the evidence Balance of desirable/undesirable outcomes
Values and preferences Decide if any revision of direction or strength is
necessary considering: Resource use *also labeled “conditional” or “discretionary”
Rate overall quality of evidence (lowest quality among critical outcomes)
S1 S2 S3 S4
OC1 OC2 OC3 OC4
OC1 OC3 Critical outcomes
OC4
Generate an estimate of effect for each outcome
OC2
S5
Structured question Structured question
• patients: lymphoma patients at risk of developing chemotherapy-induced febrile neutropenia
• granulocyte colony-stimulating (G-CSF)
• alternative not using G-CSF
Structured question Structured question • patients:
– women considering breast cancer screening – age 40-9; 50 to 74; > 75 – no ñ risk genetic mutation chest radiation
• intervention – film mammography
• alternative – no screening
Need to define all patient Need to define all patient- -important outcomes important outcomes and evaluate their importance and evaluate their importance
• desirable consequences – reduction in breast cancer mortality
• undesirable consequences – false positive screening results – invasive procedures from positive results – complications of invasive procedures – unnecessary diagnosis and treatment
Determinants of quality Determinants of quality • • RCTs RCTs start high start high • • observational studies start low observational studies start low
• • what can lower quality? what can lower quality? – – detailed design and execution detailed design and execution – – inconsistency inconsistency – – indirectness indirectness – – imprecision imprecision – – reporting bias reporting bias
Determinants of quality Determinants of quality • • RCTs RCTs start high start high • • observational studies start low observational studies start low
• • 5 limitations can lower quality 5 limitations can lower quality • • detailed design and execution detailed design and execution
– – concealment, blinding, loss to follow concealment, blinding, loss to follow- -up up
• • inconsistency inconsistency – – variability in results (heterogeneity) variability in results (heterogeneity)
• • publication bias publication bias
Determinants of quality Determinants of quality • • RCTs start high RCTs start high • • observational studies start low observational studies start low
• • 5 limitations can lower quality 5 limitations can lower quality • • Bias Bias
– – detailed design and execution detailed design and execution • • concealment, blinding, loss to follow concealment, blinding, loss to follow- -up up
– – publication bias publication bias
• • Imprecision Imprecision – – wide confidence intervals wide confidence intervals
Determinants of quality Determinants of quality • • RCTs RCTs start high start high • • observational studies start low observational studies start low
• • limitations can lower quality? limitations can lower quality? • • Bias Bias
– – detailed design and execution detailed design and execution • • concealment, blinding, loss to follow concealment, blinding, loss to follow- -up up
• • Imprecision Imprecision – – wide confidence intervals wide confidence intervals
Determinants of quality Determinants of quality
• • RCTs RCTs start high start high
• • observational studies start low observational studies start low
• • limitations can lower quality? limitations can lower quality?
Determinants of quality Determinants of quality • • 5 limitations can lower quality 5 limitations can lower quality
• • risk of bias risk of bias – – concealment, blinding, loss to follow concealment, blinding, loss to follow- -up up
• • imprecision imprecision
• • inconsistency inconsistency – – variability in results (heterogeneity) variability in results (heterogeneity)
• • Indirectness Indirectness • • publication bias publication bias
Determinants of quality Determinants of quality • • 5 limitations can lower quality 5 limitations can lower quality
• • risk of bias risk of bias – – concealment, blinding, loss to follow concealment, blinding, loss to follow- -up up
• • imprecision imprecision
• • inconsistency inconsistency – – variability in results (heterogeneity) variability in results (heterogeneity)
• • publication bias publication bias
Risk of Bias Risk of Bias
• • well established well established – – concealment concealment – – intention to treat principle observed intention to treat principle observed – – blinding blinding – – completeness of follow completeness of follow- -up up
• • more recent more recent – – early stopping for benefit early stopping for benefit – – selective outcome reporting bias selective outcome reporting bias
Breast cancer risk of bias Breast cancer risk of bias • • most trials not concealed most trials not concealed
• • blinding blinding – – ? adjudication of ? adjudication of outome outome – – no other blinding no other blinding
• • ? loss to follow ? loss to follow- -up up
• • all trials rated as all trials rated as “ “fair fair” ” by USPSTF by USPSTF
Consistency of results Consistency of results
• • if inconsistency, look for explanation if inconsistency, look for explanation – – patients, intervention, outcome, methods patients, intervention, outcome, methods
• • judgment of consistency judgment of consistency – – variation in size of effect variation in size of effect – – overlap in confidence intervals overlap in confidence intervals – – statistical significance of heterogeneity statistical significance of heterogeneity – – I I 2 2
Relative Risk with 95% CI for Vitamin D Nonvertebral Fractures
Chapuy Chapuy et al, (2002) 0.85 (0.64, 1.13) et al, (2002) 0.85 (0.64, 1.13)
Pooled Random Effect Model 0.82 (0.69 to 0.98)
p= 0.05 for heterogeneity, I 2 =53%
Chapuy Chapuy et al, (1994) 0.79 (0.69, 0.92) et al, (1994) 0.79 (0.69, 0.92)
Lips et al, (1996) 1.10 (0.87, 1.39) Lips et al, (1996) 1.10 (0.87, 1.39)
Dawson Dawson Hughes et al, (1997) 0.46 (0.24, 0.88) Hughes et al, (1997) 0.46 (0.24, 0.88)
Pfeifer et al, (2000) 0.48 (0.13, 1.78) Pfeifer et al, (2000) 0.48 (0.13, 1.78)
Meyer et al, (2002) 0.92 (0.68, 1.24) Meyer et al, (2002) 0.92 (0.68, 1.24)
Trivedi Trivedi et al, (2003) 0.67 (0.46, 0.99) et al, (2003) 0.67 (0.46, 0.99)
'
'
'
'
'
'
'
'
0.1 1 10
Favours Vitamin D Favours Control
Relative Risk 95% CI
Relative Risk with 95% CI for Vitamin D (NonVertebral Fractures, Dose >400)
Chapuy et al, (1994) 0.70 (0.69, 0.92)
DawsonHughes et al, (1997) 0.46 (0.24, 0.88)
Pfeifer et al, (2000) .48 (0.13, 1.78)
Chapuy et al, (2002) 0.85 (0.64, 1.13)
Trivedi et al, (2003) 0.67 (0.46, 0.99)
Pooled Random Effect Mode 0.75 (0.63 to 0.89)
p= 0.26 for heterogeneity, I 2 =24%
'
'
'
'
'
'
0.1 1 10
Favours Vitamin D Favours Control
Relative Risk 95% CI
Relative Risk with 95% CI for Vitamin D (NonVertebral Fractures, Dose = 400)
Lips et al (1996) 1.10 (0.87, 1.39)
Meyer et al (2002) 0.92 (0.68, 1.24)
Pooled Random Effect Mode 1.03 (0.86 to 1.24)
p = 0.35 heterogeneity, I 2 =0%
'
'
'
0.1 1 10
Favours Vitamin D Favours Control
Relative Risk 95% CI
Meta-analysis of women 39 to 49 years of age
Should Should we we believe believe sub sub- -group group analysis analysis? ?
• • within within- -study study comparison comparison? ? No No • • large large difference difference in in effect effect Borderline Borderline • • unlikely unlikely chance chance Yes Yes, p = 0.006 , p = 0.006 • • consistent consistent across across studies studies Yes Yes • • a priori a priori hypothesis hypothesis Yes Yes • • one one of of small small number number hypotheses hypotheses Yes Yes • • biologically biologically compelling compelling Yes Yes
• • shall shall we we believe believe sub sub- -group group analysis analysis? ?
Directness of Evidence Directness of Evidence
• • differences in differences in – – patients patients – – interventions interventions – – comparators comparators
• • differences in outcomes differences in outcomes – – surrogates surrogates
Quality judgments: Directness Quality judgments: Directness
• • populations populations – – older, sicker or more co older, sicker or more co- -morbidity morbidity
• • interventions interventions – – new statins versus old new statins versus old
• • outcomes outcomes – – important versus surrogate outcomes important versus surrogate outcomes – – glucose control versus CV events glucose control versus CV events
Flatulence
Figure 6: Hierarchy of outcomes according to their patient Figure 6: Hierarchy of outcomes according to their patient- -importance to assess the importance to assess the effect of phosphate lowering drugs in patients with renal failur effect of phosphate lowering drugs in patients with renal failure and hyperphophatemia e and hyperphophatemia
Importance of endpoints
Critical for decision making
Important, but not critical for decision making
Of low patient importance 2
5
Pain due to soft tissue Calcification / function 6
Fractures 7
Myocardial infarction 8
Mortality 9
3
4
1
Coronary calcification
Ca 2+ /P Product
Surrogates of declining importance
Bone density
Ca 2+ /P Product
Soft tissue calcification
Ca 2+ /P Product
Lower by one level for
indirectness
Lower by two levels for
indirectness
Alendronate Risedronate
Placebo
Directness Directness interested in A versus B interested in A versus B
available data A available data A vs vs C, B C, B vs vs C C
Issues in directness breast Issues in directness breast cancer screening cancer screening
• • estimated screening effects in over 75 estimated screening effects in over 75 – – could also use observational studies could also use observational studies
• • no direct comparisons of screening no direct comparisons of screening intervals intervals
Imprecision Imprecision
• • small sample size small sample size – – small number of events small number of events
• • wide confidence intervals wide confidence intervals – – uncertainty about magnitude of effect uncertainty about magnitude of effect
• • primary criterion: primary criterion: – – would decisions differ at ends of CI would decisions differ at ends of CI
Meta-analysis of women 39 to 49 years of age
25% RRR, reduction in breast cancer deaths 1/1,000
4% RRR, reduction in breast cancer deaths 1/6,000
Imprecision Imprecision • • small sample size small sample size
– – small number of events small number of events
• • wide confidence intervals wide confidence intervals – – uncertainty about magnitude of effect uncertainty about magnitude of effect
• • problems problems – – analogy to stopping early analogy to stopping early – – lack of prognostic balance lack of prognostic balance
• • solution: optimal information size solution: optimal information size – – # of pts from conventional sample size # of pts from conventional sample size
calculation calculation – – specify CER, effect, specify CER, effect, α α, , β β, , Δ Δ
Fluoroquinolone prophylaxis in neutropenia: infection-related mortality
Total number of events: 47
Fluoroquinolone prophylaxis in neutropenia: infection-related mortality
sample size 1,002 sample size 1,002 α α 0.05, 0.05, β β 0.20, 0.20, Δ Δ 0.25, N = 6,000 0.25, N = 6,000
Stroke Stroke – – Fixed Effects Fixed Effects
Downgrading for precision Downgrading for precision
• • if OIS not met rate down for imprecision if OIS not met rate down for imprecision – – unless very large unless very large ss ss (? > 1,000 per group) (? > 1,000 per group)
• • if OIS met and CI exclude RR = 1, don if OIS met and CI exclude RR = 1, don‛‛t t downgrade downgrade
• • if OIS met and CI includes, RR = 1, if OIS met and CI includes, RR = 1, downgrade only if RR < 0.75 or > 1.25 downgrade only if RR < 0.75 or > 1.25
Precision Precision • • atrial fib at risk of stroke atrial fib at risk of stroke
• • warfarin increases serious warfarin increases serious gi gi bleeding bleeding – – 3% per year 3% per year
• • 1,000 patients 1 less stroke 1,000 patients 1 less stroke – – 30 more bleeds for each stroke prevented 30 more bleeds for each stroke prevented
• • 1,000 patients 100 less strokes 1,000 patients 100 less strokes – – 3 strokes prevented for each bleed 3 strokes prevented for each bleed
• • where is your threshold? where is your threshold? – – how many strokes in 100 with 3% bleeding? how many strokes in 100 with 3% bleeding?
0 1.0%
0 1.0%
0 1.0%
0 1.0%
0 0.5% 1.0%
Example: Example: clopidogrel clopidogrel or ASA? or ASA? • • pts with threatened stroke pts with threatened stroke
• • RCT of RCT of clopidogrel clopidogrel vs vs ASA ASA – – 19,185 patients 19,185 patients
• ischaemic stroke, MI, or vascular death compared – 939 events (5∙32%) clopidogrel – 1021 events (5∙83%) with aspirin
• RR 0.91 (95% CI 0.83 – 0.99) (p=0∙043)
• • downgrade for precision? downgrade for precision?
0 1.0%
Clopidogrel or ASA for threatened vascular events
RCT 19,185 patients
1.7% - 0.9 – 0.1%
RR 0.91 (95% CI 0.83 – 0.99)
0
Non-inferiority
0
Non-inferiority
0
Non-inferiority
Avoiding misleading conclusions Avoiding misleading conclusions
• • analogy analogy – – SR accumulating data just as single trial SR accumulating data just as single trial
• • risk of spurious effect by stopping early risk of spurious effect by stopping early
• • how to avoid how to avoid – – insist on sufficient data insist on sufficient data – – optimal information size optimal information size
Avoiding misleading conclusions Avoiding misleading conclusions
• • optimal information size optimal information size – – # of pts from conventional sample size # of pts from conventional sample size
calculation calculation – – specify CER, effect, specify CER, effect, α α, , β β
• • alternative, number of events alternative, number of events – – ? 300 ? ? 300 ?
Criteria for NOT downgrading Criteria for NOT downgrading • • CI narrow enough to permit confident CI narrow enough to permit confident
recommendation for or against recommendation for or against
• • if positive benefit outcome, if positive benefit outcome, safeguard against false positive safeguard against false positive – – OIS OIS or or- - number threshold met (300) number threshold met (300)
Publication bias Publication bias
• • high likelihood could lower quality high likelihood could lower quality
• • when to suspect when to suspect • • number of small studies number of small studies • • industry sponsored industry sponsored
What can raise quality? What can raise quality? • • large magnitude can upgrade one level large magnitude can upgrade one level
– – very large two levels very large two levels
• • common criteria common criteria – – everyone used to do badly everyone used to do badly – – almost everyone does well almost everyone does well – – quick action quick action
• • hip replacement for hip osteoarthritis hip replacement for hip osteoarthritis
• • mechanical ventilation in respiratory failure mechanical ventilation in respiratory failure
Dose Dose- -response gradient response gradient
• • childhood lymphoblastic leukemia childhood lymphoblastic leukemia
• • risk for CNS malignancies 15 years after risk for CNS malignancies 15 years after cranial irradiation cranial irradiation
• no radiation: 1% (95% CI 0% to 2.1%) • 12 Gy: 1.6% (95% CI 0% to 3.4%) • 18 Gy: 3.3% (95% CI 0.9% to 5.6%).
Observational study starts low Observational study starts low What can move up? What can move up?
• plausible confounders strengthen association
• FP hospitals higher mortality than NFP hospitals – NFP treat sicker patients – FP treat wealthier patients
Quality assessment criteria Quality assessment criteria
Overall level of evidence Overall level of evidence • • most systems just use evidence about most systems just use evidence about
primary benefit outcome primary benefit outcome
• • but what about others (risk)? but what about others (risk)?
• • what to do? what to do?
• • options options – – ignore all but primary ignore all but primary – – weakest of any outcome weakest of any outcome – – some blended approach some blended approach – – weakest of critical outcomes weakest of critical outcomes
Quality Assessment
Summary of Findings
Quality Relative Effect (95% CI)
Absolute risk difference
Outcome Number of participants (studies)
Risk of Bias Consistency Directness Precision Reportin
g Bias
Myocardial infarction
10,125 (9)
No serious limitations
No serious imitations
No serious limitations
No serious limitations
Not detected High 0.71
(0.57 to 0.86) 1.5% fewer
(0.7% fewer to 2.1% fewer)
Mortality 10,205 (7)
No serious limitations
Possiblly inconsistent
No serious limitations Imprecise Not
detected Moderate or low
1.23 (0.98 – 1.55)
0.5% more (0.1% fewer to 1.3% more)
Stroke 10,889 (5)
No serious limitaions
No serious limitations
No serious limitations
Possible imprecision
Not detected
Moderate or High
2.21 (1.37 – 3.55)
0.5% more (0.2% more to 1.3% more0
Beta blockers in non‐cardiac surgery
Quality Assessment Summary of Findings
Quality
Relative Risk
(95% CI) pvalue
Illustrative risks
Outcome No. of patients (studies)
Risk of Bias Inconsistency Indirectness Imprecision Publication Bias
Example control rate
Associated risk with PVL
Hospital mortality
1,664 (9)
Inability to blind. 2 trials stopped early with few events and large effects; were also confounded by ‘open lung’ strategies.
p = 0.07 I 2 = 45.6% Varied populations, interventions. Not robust in sensitivity analyses
Direct Precise Undetected Moderate (due
to inconsistency)
0.82 (0.68 – 0.99) p = 0.04
40% 32.8% (27.2 – 39.6)
Barotrauma 1,497 (7) Inability to blind.
p = 0.24 I 2 = 25.3% Varied populations, interventions
Direct Imprecise Undetected Moderate (due
to imprecision)
0.90 (0.66 – 1.24) p = 0.53
NS NS
Paralysis 1,202 (5) Inability to blind.
p = 0.004 I 2 = 59% Varied populations, interventions, measurements
Direct Precise Not assessed
Moderate (due to
inconsistency)
1.37 (1.04 – 1.82) p = 0.03
30% 41.1% (31.2 – 54.6)
Dialysis 173 (2) Inability to blind.
p = 0.26 I 2 = 22.8% Varied populations, interventions
Direct Imprecise Not assessed
Moderate (due to
imprecision)
1.76 (0.79 – 3.90) p = 0.16
NS NS
Pressure limited ventilation
Quality Assessment Summary of Findings
Quality
Relative Risk
(95% CI) pvalue
Illustrative risks
Outcome No. of patients (studies)
Risk of Bias Inconsistency Indirectness Imprecision Publication Bias
control rate
vaccinated rate
Zoster episodes
38,546 (1) No serious risk only one study Direct Precise Undetected High not reported
11.12 per 1,000 patient years
5.42 (difference
5.7 per 1,000 ptyears (p< 0.001)
Post herpetic neuralgia
38,546 (1)
No serious risk only one study
Direct Precise Undetected High not reported
1.38 per 1,000
patient- years
0.46 (difference
0.92 per 1,000 pt-
years (p< 0.001)
Serious adverse events
38,546 (1) No serious risk only one study Direct Precise Undetected High Not
reported
13 per 1,000 19 (difference
6 per 1,000)
Zoster vaccine
Population No. of participants (trials) †
Higher PEEP
Lower PEEP
Adjusted Relative Risk (95% CI; P-value) ‡
Adjusted Absolute Risk Difference (95% CI)
Quality
Patients with ARDS
1892 (3) 324/951 (34.1%)
368/941 (39.1%)
0.90 (0.81 to 1.00; 0.049)
-3.9% (-7.4% to -0.04%) High
Patients without ARDS
404 (3) 50/184 (27.2%)
41/220 (18.6%)
1.37 (0.98 to 1.92; 0.065)
6.9% (-0.4% to 17.1%) Moderate (imprecision)
High versus low PEEP in ALI and ARDS
Whipples procedure pancreatic cancer with or without duodenectomy
Patients or population: Anyone taking a long flight (lasting more than 6 hours) Settings: International air travel Intervention: Compression stockings 1
Comparison: Without stockings
Outcomes Illustrative comparative risks* (RANGE OF UNCERTAINTY)
Relative effect (95% CI)
Number of participants (studies)
Quality of the evidence (GRADE)
Comments
Assumed risk Corresponding risk
Without stockings With stockings (95% CI)
Symptomatic deep vein thrombosis (DVT)
See comment See comment Not estimable 2637 (9 studies)
See comment 0 participants developed symptomatic DVT in these studies.
Symptomatic deep vein thrombosis – surrogate symptomless deep vein thrombosis
Low risk population 2 RR 0.10 (0.04 to 0.25)
2637 (9 studies)
⊕⊕⊕ Moderate 3
Estimates of control group asymptomatic thrombosis from the primary studies range from 15 per 1,000 in low risk patients to 25 per 1,000 in high risk patients
5 per 10,000 0.5 per 10,000 (0 to 1)
High risk population 2
18 per 10,000 1.8 per 10,000 (0.5 to 4)
Superficial vein thrombosis
13 per 1000 6 per 1000 (2 to 15)
RR 0.45 (0.18 to 1.13)
1804 (8 studies)
⊕⊕⊕ Moderate 4
Diagnostic tests
• same logic as for treatment
• judges quality of evidence NOT for accuracy, but for change in patient-important outcome
• ideally establish through RCT – focus on patient-important outcome – screening RCTs (breast cancer, colon cancer)
• most of time not available – new complexities, process in evolution
Study designs
Study designs II
Test accuracy is a surrogate for patient important outcomes
• When clinicians think about diagnostic tests, they focus on their accuracy
• Underlying assumption: obtaining a better idea of whether a target condition is present or absent will result in superior patient management and improved outcome.
Example of new test and reference test or strategy
Putative benefit of new test
Diagnostic accuracy Patient Outcomes and expected impact on management for the following test outcomes
Sensitivity Specificity
True positives
False positives
True negatives
False negatives
Helical CT for renal calculus compared with intravenous pyeolgram
Detection of more (but smaller) calculi
greater equal Presumed influence on patient important outcomes
Certain benefit for larger stones, for smaller stones the benefit is less clear and unnecessary treatment can result
Likely detriment from unnecessary additional invasive tests
Almost certain benefit from avoiding unnecessary tests
Likely detriment for large stones, less certain for small stones More testing
Directness of the evidence (test results) for patient‐ important outcomes
Some uncertainty
No uncertainty
No uncertainty
Major uncertainty
Balance between presumed patient outcomes, complications and cost: Less complications and downsides compared to IVP would support the new test’s usefulness, but the balance between desirable and undesirable effect not clear in view of the uncertain consequences of identifying smaller stones.
Strength of recommendations Strength of recommendations • • degree of confidence that desirable degree of confidence that desirable
effects of adhering to recommendation effects of adhering to recommendation outweigh undesirable effects. outweigh undesirable effects.
• • strong recommendation strong recommendation – – benefits clearly outweigh risks/hassle/cost benefits clearly outweigh risks/hassle/cost – – risk/hassle/cost clearly outweighs benefit risk/hassle/cost clearly outweighs benefit
Strength of recommendations Strength of recommendations
• • degree of confidence that desirable degree of confidence that desirable effects of adhering to recommendation effects of adhering to recommendation outweigh undesirable effects. outweigh undesirable effects.
• • strong recommendation strong recommendation – – benefits clearly outweigh risks/hassle/cost benefits clearly outweigh risks/hassle/cost – – risk/hassle/cost clearly outweighs benefit risk/hassle/cost clearly outweighs benefit
• • what can downgrade strength? what can downgrade strength?
Strength of Recommendation Strength of Recommendation
• • strong recommendation strong recommendation – – benefits clearly outweigh risks/hassle/cost benefits clearly outweigh risks/hassle/cost – – risk/hassle/cost clearly outweighs benefit risk/hassle/cost clearly outweighs benefit
• • what can downgrade strength? what can downgrade strength?
• • low quality evidence low quality evidence
• • close balance between up and downsides close balance between up and downsides
Grades Translations Strong recommendations • Just do it • Virtually all well informed individuals would want
the intervention and only a small proportion would not
• Most individuals should receive the intervention • Use of the intervention according to the
guideline could be used as a quality criterion or performance indicator
Grades Translations Weak recommendation • Examine the evidence yourself • The majority of well informed individuals
would want the intervention, but a substantial proportion would not
• Many but not all individuals should receive the intervention
• The intervention is not a candidate for a quality criterion or performance indicator.
Risk/Benefit tradeoff Risk/Benefit tradeoff
• • aspirin after myocardial infarction aspirin after myocardial infarction – – 25% reduction in relative risk 25% reduction in relative risk – – side effects minimal, cost minimal side effects minimal, cost minimal – – benefit obviously much greater than benefit obviously much greater than
risk/cost risk/cost
• • warfarin in low risk atrial fibrillation warfarin in low risk atrial fibrillation – – warfarin reduces stroke warfarin reduces stroke vs vs ASA by 50% ASA by 50% – – but if risk only 1% per year, ARR 0.5% but if risk only 1% per year, ARR 0.5% – – increased bleeds by 1% per year increased bleeds by 1% per year
Strength of Recommendations • Resuscitate fast in septic patient
- do it!
• Prone ventilation in failing patient with ARDS – Probably do it – Probably do not do it
Strength of Strength of Recommendations Recommendations
Aspirin after MI Aspirin after MI – – do it do it
Warfarin rather than ASA in Warfarin rather than ASA in Afib Afib -- -- probably do it probably do it -- -- probably don probably don‛ ‛t do it t do it
Significance of strong Significance of strong vs vs weak weak • • variability in patient preference variability in patient preference
– – strong, almost all same choice (> 90%) strong, almost all same choice (> 90%) – – weak, choice varies appreciably weak, choice varies appreciably
• • interaction with patient interaction with patient – – strong, just inform patient strong, just inform patient – – weak, ensure choice reflects values weak, ensure choice reflects values
• • use of decision aid use of decision aid – – strong, don strong, don‛‛t bother t bother – – weak, use the aid weak, use the aid
• • quality of care criterion quality of care criterion – – strong, consider strong, consider – – weak, don weak, don‛‛t consider t consider
USPSTF documents - clinical summary - supporting article (decision analysis) - evidence summary - recommendations
For women age 40 – 49 we suggest NOT screening. (Weak recommendation based, moderate quality evidence. 2B)
Values and Preferences: This recommendation places a relatively low value on a very small, uncertain mortality decrease and reflects concerns with false positive results, unnecessary biopsies, and unnecessary dx of breast cancer Women who place a higher value on a small reduction in breast cancer mortality and are less concerned about the undesirable consequences will choose screening
For women age 50 - 74 we suggest screening. (Weak recommendation, moderate quality evidence. 2B)
Women who do not place a high value on a small reduction in breast cancer mortality and are concerned about false positive results, unnecessary biopsies, and unnecessary diagnosis of breast cancer will decline screening
Using GRADE no insufficient - no recommendation or recommend for or against
Breast self-examination: We recommend against breast self-examination. (strong recommendation, high/moderate quality evidence)
Weak recommendation Weak recommendation
• • practice will vary practice will vary – – according to what? according to what?
• • interpretation of evidence interpretation of evidence – – clopidogrel clopidogrel in stroke in stroke
• • patients patients‛‛ values and preferences values and preferences – – atrial fibrillation atrial fibrillation
• • inclination to gamble (risk aversion) inclination to gamble (risk aversion) – – HRT HRT
When evidence is low quality When evidence is low quality
• • choice more preference dependent choice more preference dependent
• • risk aversion risk aversion
• • steroids for pulmonary fibrosis steroids for pulmonary fibrosis – – low quality evidence in support of low quality evidence in support of
benefit benefit – – high quality evidence of toxicity high quality evidence of toxicity
When evidence is low quality When evidence is low quality • • recommendation to the hopeful patient recommendation to the hopeful patient
– – I I‛‛m likely to deteriorate m likely to deteriorate – – if something might work, let if something might work, let‛‛s try it s try it – – damn the torpedoes damn the torpedoes
• • recommendation to the fearful patient recommendation to the fearful patient – – doctor, you mean you know it doctor, you mean you know it‛‛s toxic s toxic
• • diabetes, skin changes, body diabetes, skin changes, body habitus habitus, infection, , infection, osteoporosis osteoporosis
– – you don you don‛‛t know for sure it works? t know for sure it works? – – are you crazy? are you crazy?
• • weak recommendation mandated weak recommendation mandated
Strong recommendation Strong recommendation when evidence is low quality? when evidence is low quality? • • known benefit, strong recommendation for known benefit, strong recommendation for
one of two alternatives one of two alternatives – – antipyretics in children with chickenpox antipyretics in children with chickenpox – – but which one: ASA or acetaminophen but which one: ASA or acetaminophen
• • benefit: high quality evidence of benefit: high quality evidence of equivalence equivalence
• • harm: low quality evidence that harm harm: low quality evidence that harm differs appreciably differs appreciably – – Reye syndrome from ASA Reye syndrome from ASA
• • strong recommendation for acetaminophen strong recommendation for acetaminophen
Strong recommendation when Strong recommendation when evidence is low quality? evidence is low quality?
• • Blastomycosis Blastomycosis – – low quality evidence low quality evidence amphotericin amphotericin more more
effective than effective than itraconazole itraconazole – – high quality evidence more toxic high quality evidence more toxic
• • patients with life threatening patients with life threatening blasto blasto – – life and death situation life and death situation – – strong recommendation for strong recommendation for ampho ampho
Strong recommendation when Strong recommendation when evidence is low quality? evidence is low quality?
• • head to toe CT scanning head to toe CT scanning – – prevent cancer deaths prevent cancer deaths
• • very low quality evidence of benefits very low quality evidence of benefits • • moderate quality evidence re risks, moderate quality evidence re risks,
high re costs high re costs • • strong recommendation against strong recommendation against
Presentation Presentation • • strong and weak strong and weak
– – discomfort with discomfort with “ “weak weak” ” – – alternative wording: discretionary, conditional alternative wording: discretionary, conditional
• • strong strong – – “ “we recommend we recommend”… ”…
• • discretionary discretionary – – “ “we suggest we suggest…” …”
• • never never – – we recommend (or suggest) you consider we recommend (or suggest) you consider… …
• • always: quality of evidence and grade always: quality of evidence and grade
When (not to) GRADE When (not to) GRADE “ “good to remind/alert good to remind/alert” ”
• • if no systematic review undertaken if no systematic review undertaken
• • no sensible person would consider contrary no sensible person would consider contrary – – We recommend that the patient, and the clinician We recommend that the patient, and the clinician
responsible for the patient responsible for the patient‛‛s care, should be made s care, should be made aware of any change in a prescribed medication, aware of any change in a prescribed medication, including change to a generic drug including change to a generic drug
• • very general (not sufficiently specific) very general (not sufficiently specific) – – We suggest that long We suggest that long- -term maintenance term maintenance
immunosuppression immunosuppression be tailored to individual patient be tailored to individual patient‛‛s s adverse events or risk of adverse events adverse events or risk of adverse events
Explicit comparator Explicit comparator
• • we recommend hourly urine volume we recommend hourly urine volume measurement for at least 24 hours measurement for at least 24 hours – – in contrast to every 2 hours, every 3 in contrast to every 2 hours, every 3… …? ?
• • we suggest measuring serum creatinine in we suggest measuring serum creatinine in all all KTRs KTRs at least at least – – daily for 7 days daily for 7 days – – 2 to 3 X per week for weeks 2 to 4 2 to 3 X per week for weeks 2 to 4 – – every 2 weeks for months 4 to 6 every 2 weeks for months 4 to 6
Value and preference Value and preference statements statements
• • underlying values and preferences underlying values and preferences always present always present
• • sometimes crucial sometimes crucial
• • important to make explicit important to make explicit
Values and preferences Values and preferences
Stroke guideline: patients with TIA clopidogrel over aspirin (Grade 2B).
Underlying values and preferences: This recommendation to use clopidogrel over aspirin places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures.
Values and preferences Values and preferences
peripheral vascular disease: aspirin be used instead of clopidogrel (Grade 2A).
Underlying values and preferences: This recommendation places a relatively high value on avoiding large expenditures to achieve small reductions in vascular events.
Flavanoids for Hemorrhoids Flavanoids for Hemorrhoids • • venotonic venotonic agents agents
– – mechanism unclear, increase venous return mechanism unclear, increase venous return
• • popularity popularity – – 90 90 venotonics venotonics commercialized in France commercialized in France – – none in Sweden and Norway none in Sweden and Norway – – France 70% of world market France 70% of world market
• • possibilities possibilities – – French misguided French misguided – – rest of world missing out rest of world missing out
Systematic Review Systematic Review • • 14 trials, 1432 patients 14 trials, 1432 patients • • key outcome key outcome
– – risk not improving/persistent symptoms risk not improving/persistent symptoms – – 11 studies, 1002 patients, 375 events 11 studies, 1002 patients, 375 events – – RR 0.4, 95% CI 0.29 to 0.57 RR 0.4, 95% CI 0.29 to 0.57
• • minimal side effects minimal side effects
• • is France right? is France right? • • what is the quality of evidence? what is the quality of evidence?
What can lower quality? What can lower quality?
• • risk of bias risk of bias – – lack of detail re concealment lack of detail re concealment – – questionnaires not validated questionnaires not validated
• • indirectness indirectness – – no problem no problem
• • inconsistency, need to look at the inconsistency, need to look at the results results
Review : Phlebotonics for hemorrhoids Comparison: 01 Venotonics vs placebp Outcome: 08 Overall improvement: no improvement/some improvement
Study RR (random) Weight RR (random) or subcategory log[RR] (SE) 95% CI % 95% CI
01 Up to seven days Chauvenet 0.8916 (0.2376) 12.67 0.41 [0.26, 0.65] Cospite 2.2073 (0.6117) 5.51 0.11 [0.03, 0.36] Thanapongsathorn 0.4308 (0.2985) 11.18 0.65 [0.36, 1.17]
Subtotal (95% CI) 29.36 0.37 [0.18, 0.77] Test for heterogeneity: Chi² = 6.92, df = 2 (P = 0.03), I² = 71.1% Test for overall effect: Z = 2.67 (P = 0.008)
02 Up to four w eeks Annoni F 1.6094 (0.7073) 4.50 0.20 [0.05, 0.80] Clyne MB 0.9943 (0.3983) 8.94 0.37 [0.17, 0.81] Pirard J 1.1712 (0.3086) 10.94 0.31 [0.17, 0.57] Thanapongsathorn 1.1087 (1.1098) 2.18 0.33 [0.04, 2.91] Thorp 0.2624 (0.3291) 10.46 1.30 [0.68, 2.48] Titapan 0.8916 (0.3691) 9.56 0.41 [0.20, 0.85] Wijayanegara 0.5978 (0.1375) 14.97 0.55 [0.42, 0.72]
Subtotal (95% CI) 61.54 0.48 [0.32, 0.72] Test for heterogeneity: Chi² = 13.87, df = 6 (P = 0.03), I² = 56.7% Test for overall effect: Z = 3.57 (P = 0.0004)
03 Further than four w eeks Godeberg 1.7719 (0.3906) 9.10 0.17 [0.08, 0.37]
Subtotal (95% CI) 9.10 0.17 [0.08, 0.37] Test for heterogeneity: not applicable Test for overall effect: Z = 4.54 (P < 0.00001)
Total (95% CI) 100.00 0.40 [0.29, 0.57] Test for heterogeneity: Chi² = 28.66, df = 10 (P = 0.001), I² = 65.1% Test for overall effect: Z = 5.14 (P < 0.00001)
0.001 0.01 0.1 1 10 100 1000
Favours treatment Favours control
Publication bias? Publication bias?
• • size of studies size of studies – – 40 to 234 patients, most around 100 40 to 234 patients, most around 100
• • all industry sponsored all industry sponsored
Review : Phlebotonics for hemorrhoids Comparison: 01 Venotonics vs placebp Outcome: 08 Overall improvement: no improvement/some improvement
0.001 0.01 0.1 1 10 100 1000
0.0
0.4
0.8
1.2
1.6
RR (fixed)
What can lower quality? What can lower quality? • • detailed design and execution detailed design and execution
– – lack of detail re concealment lack of detail re concealment – – questionnaires not validated questionnaires not validated
• • inconsistency inconsistency – – almost all show positive effect, trend almost all show positive effect, trend – – heterogeneity p < 0.001; I heterogeneity p < 0.001; I 2 2 65.1% 65.1%
• • indirectness indirectness • • imprecision imprecision
– – RR 0.4, 95% CI 0.29 to 0.57 RR 0.4, 95% CI 0.29 to 0.57
• • reporting bias reporting bias – – 40 to 234 patients, most around 100 40 to 234 patients, most around 100
Recommendation Recommendation
• • for clinician for clinician – – offer to patient offer to patient – – don don‛‛t offer to patient? t offer to patient?
• • strength of recommendation strength of recommendation – – strong or weak strong or weak
• • for the funding body for the funding body – – publicly funded publicly funded – – not publicly funded not publicly funded – – strong or weak? strong or weak?
Is France right? Is France right?
• • recommendation recommendation – – yes yes – – no against use no against use
• • strength strength – – strong strong – – weak weak
Resource Use Resource Use
• • why not cost? why not cost? - - may lead to focus on cost of intervention may lead to focus on cost of intervention
rather than downstream resource use rather than downstream resource use - - resource use emphasizes alternative resource use emphasizes alternative
uses of resources (opportunity cost) uses of resources (opportunity cost)
Resource Use just another Resource Use just another outcome? outcome?
• • yes and no yes and no
• • who benefits? who benefits? – – other outcomes usually clear other outcomes usually clear – – costs borne by different payers costs borne by different payers
• • across societies and within (age) across societies and within (age)
• • some argue costs aren some argue costs aren‛‛t relevant to t relevant to clinicians when third party payer clinicians when third party payer
Why resource use different Why resource use different
• • costs vary much more than other outcomes costs vary much more than other outcomes – – across jurisdictions across jurisdictions – – within jurisdictions within jurisdictions – – over time over time
• • even when resource use the same, even when resource use the same, implications may differ implications may differ – – year year‛‛s supply of expensive drug s supply of expensive drug – – nurses nurses‛‛ salary in U.S., 6 in Poland, 30 in China salary in U.S., 6 in Poland, 30 in China
Why resource use different Why resource use different • • opportunity cost differs by perspective opportunity cost differs by perspective
• • hospital pharmacy, fixed budget hospital pharmacy, fixed budget – – new expensive drug, clear what give up new expensive drug, clear what give up
• • envelope public spending envelope public spending – – more on health, less on education, social more on health, less on education, social
services services – – will refraining from spending on drugs will refraining from spending on drugs
really mean more for other services? really mean more for other services? – – should envelope include military spending? should envelope include military spending?
Implications Implications • • unbearable lightness of resource use unbearable lightness of resource use • • consider balance of desirable and consider balance of desirable and
undesirable before considering undesirable before considering resource use resource use
• • may decide not to consider resource may decide not to consider resource use at all use at all – – intervention not useful intervention not useful – – desirable consequences >>>> undesirable desirable consequences >>>> undesirable – – relevant only when difference small relevant only when difference small
Similarities with other Similarities with other outcomes outcomes
• • only consider important resource use only consider important resource use
• • need estimate of difference between need estimate of difference between trt trt and control and control
• • explicit judgments about the quality of the explicit judgments about the quality of the evidence, special judgments evidence, special judgments – – perspective perspective – – how to judge quality of evidence how to judge quality of evidence – – ? use of economic model ? use of economic model
Evidence summary Evidence summary • • includes quality of evidence, summary includes quality of evidence, summary
of findings of findings – – “ “balance sheet balance sheet” ”, special form of grade , special form of grade
profile profile
• • resource use and not just costs resource use and not just costs – – can judge whether resource use can judge whether resource use
applicable to local setting applicable to local setting – – focus on focus on coss coss relevant to them relevant to them
(pharmacy) (pharmacy) – – apply unit costs to local setting apply unit costs to local setting
Example question Example question • • patients patients
– – women with pre women with pre- -eclampsia eclampsia
• • intervention intervention – – intravenous magnesium intravenous magnesium
• • RCT done in 33 countries RCT done in 33 countries – – over 9,000 patients over 9,000 patients
• • for presentation of resource use evidence for presentation of resource use evidence need to specify perspective need to specify perspective – – health system health system
Quality assessment
Studies Design Limitations Inconsistency Indirectness Imprecision No of patients
Relative effect
(95% CI)
Quality
Eclampsia
Duley 2003 RCT No one trial only No No 9,992 RR 0.41 (0.290,58)
High
Maternal death
Duley 2003 RCT No one trial only No Imprecision 9,992 RR 0.54 (0.261,10)
Moderate
Quality assessment
Design Limita tions
Inconsis tency
Indirect ness
Impre cision
Resources Costs per patient
Studies per patient (US $; year 2001)
Place bo MgSO4 Placebo MgSO4
Magnesium sulphate
High GNI 0 6 0 20
Simon 2005 Middle GNI RCT No one trial only No No 0 6 0 3 High
Low GNI 0 6 0 5
Administration of the drug
High GNI 0 1 0 66
Simon 2005 Middle GNI RCT No one trial only No No 0 1 0 14 High
Low GNI 0 1 0 8
Other hospital resources a, b
High GNI Large
variationre sources c
NA NA 12,839 12,818
Simon 2005 Middle GNI RCT No one trial only No NA NA 1,412 1,416 Modera
te
Low GNI NA NA 155 157
Outcomes
Typical control group risk
Typical absolute effect (95% CI)
Relative effect (95% CI)
Nr. of participants (studies)
Quality of the evidence
Comments
Clinical outcomes
Eclampsia Severe RR 0.41 (0.29 0.58)
11,444 ⊕⊕⊕⊕ High
27 per 1,000 16 fewer per 1,000 (11 to 19)
Not severe
15 per 1,000 9 fewer per 1,000 (6 to 11)
Maternal death Severe RR 0.54 (0.26 1.10)
10,795 ⊕⊕⊕ Moderate 2
6 per 1,000 3 fewer per 1,000 (0.6 more to 4 fewer)
Not severe
3 per 1,000 1 fewer per 1,000 (0.3 more to 2 fewer)
Side effects 46 per 1,000 3 196 more per 1,000 (165 to 231)
RR 5.26 (4.59 6.03)
9.992 ⊕⊕⊕⊕ High
Mostly flushing. Other side effects include nausea, vomiting, slurred speech, muscle weakness, dizziness, drowsiness, confusion and headache.
Magnesium sulphate ampoules
0 6 10 ml. ampoules per woman
9.996 ⊕⊕⊕⊕ High
Cost High GNI Middle GNI Low GNI
$20 more per patient $ 3 more per patient $ 5 more per patient
Administration of magnesium sulphate
0 1 per woman 9.996 ⊕⊕⊕⊕ High
Cost High GNI Middle GNI Low GNI
$66 per patient $14 per patient $ 8 per patient
Resources for administering magnesium sulphate included midwife time (main cost), intravenous cannula/needle, syringe, IV fluids, drug.
Other hospital resources Varied widely 9.996 ⊕⊕⊕ Moderate 5
There was large variation in the use of other hospital resources in both intervention and control groups.
Cost High GNI Middle GNI Low GNI
$12,839 $ 1,416 $ 157
$20 less per woman (0 to 60)
$ 4, less per woman (0 to 10)
$ 2 less per woman (1 to 3)
Other hospital costs have been adjusted based on the influence of eclampsia to control for the many other factors that influenced these costs.
Resource use from the perspective of the health system
control grop difference trt vs control
Issues in resource use Issues in resource use • • broad perspective desirable broad perspective desirable
– – narrow perspectives ignore much resource use narrow perspectives ignore much resource use – – users can pick costs relevant to them users can pick costs relevant to them – – either health care system or societal either health care system or societal
• • indirect costs controversial indirect costs controversial
• • indirect evidence of resources use indirect evidence of resources use – – costs only reported costs only reported – – RCT but doesn RCT but doesn‛‛t reflect practice t reflect practice
• • ulcer prevention everyone gets repeat endoscopy ulcer prevention everyone gets repeat endoscopy
Quality of evidence for Quality of evidence for resource use resource use
• • rules basically the same rules basically the same – – RCTs RCTs start high, observational low start high, observational low
• • may need multiple sources of evidence may need multiple sources of evidence – – RCTs RCTs may not fully report resource use may not fully report resource use
• • variation across settings variation across settings – – RCT may not reflect real world RCT may not reflect real world – – time frame may extend beyond trial time frame may extend beyond trial
• • different quality for different resources different quality for different resources – – mag mag sulphate sulphate versus hospital resources versus hospital resources
Formal economic models Formal economic models • • limitations limitations
– – supported by industry, biased supported by industry, biased – – setting specific setting specific – – reduces transparency reduces transparency – – if evidence low quality, speculative if evidence low quality, speculative – – often many assumptions often many assumptions
• • solution: develop own model solution: develop own model – – OK if you are NICE with lots of resources OK if you are NICE with lots of resources
• • even so, don even so, don‛‛t include in profile t include in profile
Costs versus affordability Costs versus affordability
• • intervention may be intervention may be “ “cost cost- -effective effective” ” – – $10,000 per $10,000 per qaly qaly gained gained
• • but if applicable to huge proportion but if applicable to huge proportion of population, may still be of population, may still be unaffordable unaffordable
healthy asymptomatic postmenopausal healthy asymptomatic postmenopausal qomwn qomwn: : HRT in 1992? HRT in 1992?
Possible benefits Possible benefits – – CHD, Hip fracture, Colorectal cancer CHD, Hip fracture, Colorectal cancer
Possible harms Possible harms – – Breast cancer Breast cancer – – Stroke Stroke – – Thrombosis Thrombosis – – Gall bladder disease Gall bladder disease
Can GRADE lead to change?
Evidence profile: Quality assessment Evidence profile: Quality assessment Oestrogen + progestin for prevention Oestrogen + progestin for prevention
in 1992 (before WHI and HERS) in 1992 (before WHI and HERS)
Oestrogen + progestin versus usual care
Oestrogen + progestin for Oestrogen + progestin for prevention after WHI and HERS prevention after WHI and HERS
Postulate
• major work in preparing guideline/HTA assessment is systematic review
• If already doing this, GRADE framework should add little
• history: Rolls-Royce and Volkswagen
VW and RR VW and RR appraoches appraoches
• • Rolls Royce (NICE) Rolls Royce (NICE) – – systematic review for every outcome systematic review for every outcome – – production of evidence profiles production of evidence profiles – – involvement of multiple constituencies involvement of multiple constituencies
• • including patients including patients – – inclusion of economic analysis inclusion of economic analysis
• • cost $1 million per guideline cost $1 million per guideline
MOPED GRADE MOPED GRADE • • UpToDate UpToDate
– – 5,000 graded recommendations 5,000 graded recommendations
• • generate PICO (informal) generate PICO (informal) – – no formal rating of outcome importance no formal rating of outcome importance
• • use of existing reviews, primary studies use of existing reviews, primary studies – – no new evidence syntheses no new evidence syntheses
• • quality for key outcomes quality for key outcomes – – 5 reasons rating down, 3 up 5 reasons rating down, 3 up – – no new evidence profiles, no new evidence profiles, SoF SoF tables tables
• • recommendations recommendations – – strong or weak, consider 3 factors strong or weak, consider 3 factors – – value and preference statements value and preference statements
ACCP • formal structured questions
• no formal rating of outcome importance – trying to change
• hit-and miss systematic reviews – largely only available ones
• hit-and-miss individual study evidence summaries
• rare evidence profiles – trying to change
VW approach VW approach
• • take systematic reviews if available take systematic reviews if available
• • if not, review key, accessible evidence if not, review key, accessible evidence
• • no meta no meta- -analysis if not done analysis if not done
• • no evidence profiles no evidence profiles
• • small group make expert small group make expert judgement judgement
Conclusion Conclusion
• • clinicians, policy makers need summaries clinicians, policy makers need summaries – – quality of evidence quality of evidence – – strength of recommendations strength of recommendations
• • explicit rules explicit rules – – transparent, informative transparent, informative
• • GRADE GRADE – – simple, transparent, systematic simple, transparent, systematic – – increasing wide adoption increasing wide adoption