Topic 03 Biomedical Decision Making Part 2...Measurement of Diagnostic Tests – Other Test Perfm....
36
Kevin Robertson, MBA Topic 03 Biomedical Decision Making – Part 2 ACS-2816 Health Information Systems Winter 2020
Transcript of Topic 03 Biomedical Decision Making Part 2...Measurement of Diagnostic Tests – Other Test Perfm....
Kevin Robertson, MBA
Topic 03Biomedical Decision Making – Part 2
ACS-2816 Health Information Systems
Winter 2020
Measurement of Diagnostic Tests –Other Test Perfm. Measurements: FNR
2
Sometime it is important to know other test performance measurement
FNR – False Negative Rate is the likelihood that a diseased patient has a negative test p [ negative test | disease ]
FNR = 1 - TPR(see p77)
Measurement of Diagnostic Tests –Other Test Perfm. Measurements: FPR
3
FPR – False Positive Rate is the likelihood that a non-diseased patient has a positive test p [ positive test | non-diseased ]
FPR = 1 – TNR
These are important to knowing what adjustments to test score is needed
(see p77)
Measurement of Diagnostic Tests –Bias in the Measurement of Tests
4
Issue: Published estimates of disease prevalence (from study population) may differ from the clinically relevant population
Similar issue may apply in sensitivity and specificity estimates
E.g. Early 1970’s blood test called the carcinoembryonic antigen (CEA) test for colon cancer
Measurement of Diagnostic Tests –Bias in the Measurement of Tests
5
Referral bias classes
Spectrum bias
Test-referral bias
Test-interpretation bias
Measurement of Diagnostic Tests –Bias in the Measurement of Tests (p79)
6
Spectrum bias – study population only includes individuals with advanced disease FNs ? FNR? (among study and clinical population) TPR = 1 – FNR ?
Test-referral bias – occurs when a positive index test is a criteria for ordering the gold standard test for study population FNs? FNR? (among study and clinical population) TPR = 1 – FNR ?
(Index test – test under discrimination)
Measurement of Diagnostic Tests –Bias in the Measurement of Tests (p81)
7
Test-interpretation bias – occurs when the interpretation of the index test affects that of the gold standard test and vice versa.
All 3 biases affect TPR and TNR when applied to new population TPR (sensitivity) higher for 3 biases, how should
we adjust? (Downward) its is artificially high! TNR (specificity)? depends on bias
Key point: if higher then we lower, and vice-versa
Measurement of Diagnostic Tests –Bias in the Measurement of Tests
8
Sensitivity (TPR) and specificity (TNR) help to characterize the performance of a test
TPR & TNR reveal the probability of a test result given the true state of the patient E.g. Joe has a positive test. The test has a TPR of 0.8 E.g. Mary has a negative test. The test has a TNR of 0.2
But, what is the probability that Joe (or Mary) has the disease?
Need to calculate the post-test probability of disease
Topic 3 Outline
9
Biomedical Decision Process
Probability assessment Pre-test probability Post-test probability
Bayes Theorem
Predictive Value
Measurement of Diagnostic Tests
Post-test Probability –Bayes’ Theorem and Predictive Value
10Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p71, Figure 3.2a
Post-test Probability - Bayes’ Theorem
11
Bayes’ theorem – quantitative method to calculate post-test probability using the pre-test probability, and the sensitivity and specificity of the test
Conditional probability? e.g. p ( A | B ) General Goal: Probability that disease is
present (event A), given that the test is known to be positive (event B)
P (Stroke | High Blood Pressure )
Post-test Probability - Bayes’ Theorem
12
Notation: D = presence of disease, -D = absence of disease, R = test result, p[D] = pretest probability of disease
Bayes’ Theorem
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 82
Post-test Probability - Bayes’ Theorem
13
Performing substitutions for a positive test (+) p[D|+] for p[D|R] p[+|D] for p[R|D] p[+|-D] for p[R|-D] 1 – p[D] for p[-D] p[+|D] = TPR p[+|-D] = FPR
Bayes’ Theorem for a positive test
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 82
Post-test Probability - Bayes’ Theorem
14
Example 8: Pretest probability of heart disease is 0.95, based on the prevalence of heart disease in men who have typical symptoms of heart disease and on the prevalence in people with a family history of heart disease. Assuming that the TPR and FPR of the exercise stress test are 0.65 and 0.20 respectively. What is the probability of heart disease given a positive test result?
Substituting values
What is the effect in p[D|+] if we reduce FPR close to 0?
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 82
Post-test Probability - Bayes’ Theorem
15
Too complex in general to do using mental math and substitution, need to simplify
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 82
Post-test Probability - Bayes’ Theorem Odds and Likelihood Ratios
16
Probability and odds are related as follows
E.g. Probability of bombers winning championship is 0.2, what are their odds?
Odds = 0.2 / (1 – 0.2) = 0.25
odds are 0.25 : 1 or 1 : 4
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 82
Post-test Probability - Bayes’ Theorem Odds and Likelihood Ratios
17
Post-test odds = pre-test odds x likelihood ratio (LR)
This equation is the ‘odds-ratio form’ of Bayes’ theorem
Formula source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 83
Post-test Probability - Bayes’ Theorem Odds and Likelihood Ratios
18
Likelihood ratio (LR)
LRs types: positive or negative
Formulas source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 83
Post-test Probability - Bayes’ Theorem Odds and Likelihood Ratios
19
Example 9: Now for Joe, we estimate the pretest probability of heart disease as 0.75, based on the prevalence of heart disease in men who have typical symptoms of heart disease and on the prevalence in people with a family history of heart disease. Assuming that the TPR and FPR of the exercise stress test are 0.65 and 0.20 respectively. What is Joe’s probability of heart disease given a positive test result using odds & likelihood ratios?
Steps:1. Pretest odds
Example source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 83
Post-test Probability - Bayes’ Theorem Odds and Likelihood Ratios
20
2. LR+ for the stress test
3. Post-test odds using Bayes’ theorem odds-ratio form
4. Convert odds to probability
Formulas source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 83
Post-test Probability –Bayes’ Theorem and Predictive Value
21Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 71, Figure 3.2a
Post-test Probability –Predictive Value of a Test
22
Predictive value is an alternative approach to calculate post-test probability
It can be calculated directly from contingency tables
Two types: Positive predictive value (PV+) Negative predictive value (PV-)
Post-test Probability –Predictive Value of a Test
23
Positive predictive value (PV+) – likelihood that a patient who has a + test also has the disease
Negative predictive value (PV-) – likelihood that a patient who has a – test does not have the disease
Formulas source: Shortliffe et al, ‘Biomedical Informatics’, 3rd Edition, Chapter 3, p 102
Post-test Probability –Predictive Value of a Test
24
Example 10: What is the PV of the EIA test from the 2x2 contingency table below
Example source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, p 84
How do we interpret these values?
Post-test Probability –Implications of Bayes’ Theorem (p85)
25Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, Figure 3.5, p 85
Note:It assumes a +ve test resultSensitivity = 0.9 Specificity = 0.9
What is the post-test probability when the pre-test probability increases?
What does the dotted line mean?(i.e. LR=1)
Post-test Probability –Implications of Bayes’ Theorem
26Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, Figure 3.5, p 85
Note:It assumes a - test resultSensitivity = 0.9 Specificity = 0.9
What is the post-test probability when the pre-test probability increases?
What does the dotted line mean?(i.e. LR=1)
Post-test Probability –Implications of Bayes’ Theorem
27Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, Figure 3.6, p 86
Note:+ve test results, top curves-ve test results, bottom curvesVariable specificity
What is the post-test probability when the pre-test probability increases and specificity (i.e. TNR) varies?
Post-test Probability –Implications of Bayes’ Theorem
28Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 3, Figure 3.6, p 86
Note:+ve test results, top curves-ve test results, bottom curvesVariable sensitivity
What is the post-test probability when the pre-test probability increases and sensitivity (i.e. TPR) varies?
Post-test Probability –Implications of Bayes’ Theorem
29
How do we use previous curves to answer the following: If we are trying to rule in a diagnosis, which kind
of test do we use based on the curves given? Tests with high specificity
If we are trying to exclude a disease, which kind of tests do we use based on the curves given? Tests with high sensitivity
Post-test Probability – Cautions in the Application of Bayes’ Theorem
30
Some common issues
Inaccurate pre-test probabilities
Faulty application of test-performance measures
Violation of the assumptions of Conditional independence
Mutual exclusivity
The Nature of Clinical Decisions –Recap of the Diagnostic Process
31Figure source: Shortliffe et al, ‘Biomedical Informatics’, 4th Edition, Chapter 2, p 60, Figure 2.11
ID – Patient IdentificationCC – Chief ComplaintHPI – History of Present IllnessPMH – Past Medical HistoryFH – Family HistoryROW – Review of SystemsPE – Physical Examination
Pre-test Probability
Post-test Probability
Biomedical Decision Making –Probabilistic Clinical Reasoning
32
Often pathophysiology (i.e. disordered physiological processes associated with disease or injury) not enough for decision-making
Other methods needed to choose among treatments
E.g. Decision trees, Markov modelsNot part of thiscourse!
Biomedical Decision Making –To Be or Not To Be
33
Clinician must choose among next actions when evaluating patient’s symptoms and suspects a disease: Do nothingObtain more info to Treat Obtain more info, go back to top Do nothing
Treat
Biomedical Decision Making –Role of Probability and Decision Analysis
34
Reality – Not all clinical decisions require decision analysis
What do clinicians do to make a decision?
Decisions can depend Physiologic principles Deductive reasoning Little certainty Lack of information -> lot of uncertainty Lack of time -> “quick and dirty” decisions
Biomedical Decision Making –Role of Probability and Decision Analysis
35
Often, there is need to estimate a range of probabilities for decision variables
Complexity of medical decision + need to control costsClinical practice guidelines
Fact: Medical decision involves uncertainty for the clinician and risk for the patient
Two Questions (Recap)
36
What is the complexity in the medical decision process?
How probability assist to improve the medical decision process?
