Diagnostic research
description
Transcript of Diagnostic research
![Page 1: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/1.jpg)
Diagnostic researchDelivered by Nia Kurniati
![Page 2: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/2.jpg)
Lecture Contents
I. Diagnostics in practice- Explained with a case
II. Scientific development of diagnostic research– Design– Data-analysis– Reporting
III. ExercisesIV. Summary
![Page 3: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/3.jpg)
Diagnostics in practice
Diagnostics always start with a patient with a complaint/symptom
Case: neck stiffness • Child, 2 years-old, comes to ER with parents • Child turns out to have a very stiff neck
What is the physician’s aim?
![Page 4: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/4.jpg)
Diagnostics in practiceAim of the physician• Quickly and efficiently determine the correct
diagnosis
Why diagnose?• Basis medical handling • Determines treatment choice• Gives information about prognosis
What are possible diagnoses for neck stiffness?
![Page 5: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/5.jpg)
Diagnostics in practice
Differential diagnosis (DD)• Bacterial meningitis • Viral meningitis• Pneumonia• ENT infection• Other (e.g. myalgia)
What is the most important diagnosis? Which one does the physician not want to miss?
![Page 6: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/6.jpg)
Diagnostics in practice
Most important diagnosis• Bacterial meningitis (BM) • If missed: often fatal
![Page 7: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/7.jpg)
Diagnostics in practice
Suppose: 20% of all children on the ER with neck stiffness has BM
20% with disease in that population = prevalence ≈ Prior-probability
What is your decision for the child in this case?
![Page 8: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/8.jpg)
Diagnostics in practice
Decision for child in case • Prior-probability too low to treat• Prior-probability too high to send home
Decision: reduce uncertainty diagnostics
What is the best test?
![Page 9: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/9.jpg)
Diagnostics in practice
Best testLumbal punction (liquor culture)
![Page 10: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/10.jpg)
Diagnostics in practiceGold standard• True disease status; ‘truth’
– Never 24 karat• Reference standard/test• Decisive test with doubt
Perform reference test for everybody (=every child on ER with neck stiffness)?
![Page 11: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/11.jpg)
Reference test for everybody?• Unethical too invasive/risky• Inefficient too expensive• Do not perform unnecessarily
How should we then determine the probability of disease presence and what would be ideal?
Diagnostics in practice
![Page 12: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/12.jpg)
How then?• Simpler diagnostics:
– Usually history taking, physical exam, simple lab tests, imaging, etc.
– Ideal: diagnosis without reference test
• Diagnostic process in practice: – Stepwise process: less more invasive– Not one diagnosis based on 1 test – Each item: separate test
Diagnostics in practice
![Page 13: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/13.jpg)
Diagnostics in practiceSuppose: after anamnesis & PE + 10% probability
of BM
• Probability of disease given test results = posterior-probability
• The bigger the difference between prior and posterior probability, the better the diagnostic value of the tests
Our decision for child in case: probability is too high to send home --> next step?
![Page 14: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/14.jpg)
Diagnostics in practice
Next step– Additional research, e.g.
blood tests (leucocytes, CRP, sedimentation, etc.)
![Page 15: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/15.jpg)
Diagnostics in practice
Suppose: + 1% posterior-probability after anamnesis, PE+ simple lab tests posterior probability low enough to send home
• Ideal diagnostic process: simple tests reduce posterior probability to 0 or 100% (without reference)• Most often physician continues testing until sufficiently sure (approximation of 0 or 100%)• Choose when sufficiently sure: depends on prognosis of disease if untreated + risks/costs treatment
![Page 16: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/16.jpg)
Diagnostics in practice
Summarizing • What does diagnosing involve in practice?
– Estimation of probability of disease presence based on test(s) results of the patient
When is the probability of disease best estimated? Why is this usually not done?
![Page 17: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/17.jpg)
Diagnostics in practice Why not all possible tests?– Invasive (for patient and budget)– Unnecessary: different test results give same info– However: In practice often more tested than
necessary!
What diagnostics truly necessary scientific diagnostic research
![Page 18: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/18.jpg)
BREAK
![Page 19: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/19.jpg)
SCIENTIFIC DEVELOPMENT OF DIAGNOSTIC RESEARCH
![Page 20: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/20.jpg)
Study designScientific diagnostic research
– What tests truly contribute to probability estimation?– Has to serve practice follow practice
![Page 21: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/21.jpg)
Study design
• Research question• Domain• Study population • Determinant(s): test(s) to study• Endpoint: presence/absence disease (outcome)• Study design: design• Data analysis, interpretation + reporting
![Page 22: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/22.jpg)
Research question
• With as few as possible simple, safe, and cheap tests estimate the probability of the presence/absence of disease.
• Determinant-outcome relation:– probability of disease as a function of test results– outcome = probability of disease = % = prevalence– test results = determinants
![Page 23: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/23.jpg)
Research question
Case• What tests contribute to probability estimation
of presence or absence of BM in children with neck stiffness at the ER?
• Or: Determinants of presence/absence disease (BM)?
• %BM = ƒ(age, gender, fever, blood leucocytes, blood CRP, etc)
![Page 24: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/24.jpg)
Research populationCase: • All children with neck stiffness in 2002
at ER Utrecht
![Page 25: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/25.jpg)
Domain• For whom domain, generalisation
= type of patient with certain symptom /complaint + setting
• Research population = 1 sample from domain
Case: All children (e.g. in Western world) suspected of disease (BM) based on neck stiffness (characteristic) in secondary care (setting)
![Page 26: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/26.jpg)
Determinants
= Tests to study
• Diagnostic determinants• All possible important tests (in domain)
CaseItems anamnesis, PE and lab (blood and urine)
tests
![Page 27: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/27.jpg)
Endpoint
‘True’ presence/absence disease = Diagnostic outcome = Results reference test
NB: reference = not infallible but always best available test in practice at that moment
Case• Positive liquor culture
![Page 28: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/28.jpg)
PICO EBM• Population/ problem• Intervention• Comparison/ control• Outcome
• Domain• Determinant• Reference test• Outcome
![Page 29: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/29.jpg)
Measure determinants/endpoint
• Determinants– Without knowledge (blinded) of the outcome – Same method in study and practice
never measure more precisely than in practice (overestimation information yield)
• Endpoint– Assessment blind for determinants– With the best possible test known in practice
![Page 30: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/30.jpg)
Study design
• Observational and descriptive – Observational = no manipulation of
determinants– Descriptive = not causal – if the determinant only predicts– no hypothesis functional mechanism
determinant-outcome
• >1 determinant
![Page 31: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/31.jpg)
Study design • Cross-sectional
= Simultaneously measure determinants and outcome
![Page 32: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/32.jpg)
Data-analysisAfter data collection, per patient
– Value determinants (test results)– Diagnostic outcome (reference test)
![Page 33: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/33.jpg)
Data-analysis• Data analysis: 3 steps
1) Estimate prior probability (before additional test results)
2) Compare each test result separately with reference = univariate3) Compare combination of test results with reference = multivariate (via model)
- Following order in practice - Determine added value test result to already collected (previous) test results
![Page 34: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/34.jpg)
Data-analysis
CaseData scientific research available:200 patients with neck stiffness at ER
Liquor culture positive (BM+) n=40Liquor culture negative (BM-) n=160
Step 1: Prior probability (prevalence) of BM?
= % BM+ = 40/ 200 patients = 20%
![Page 35: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/35.jpg)
Data-analysis reading 2 by 2 table
Disease
Presence Absence
Test Positive True positiveA
False positiveB
Negative CFalse negative
DTrue negative
• Step 2: Analysis per determinant (univariate) • Use 2 by 2 table
![Page 36: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/36.jpg)
Data-analysis reading 2 by 2 table
Horizontally• Positive predictive value (PV+)
= probability Disease (+) if test (+) PV(+) = A / A + B
• Negative predictive value (PV-)= probability disease (-) if test (-)
PV(-) = D / C + DVertically• Sensitivity (SE) = probability test (+) if disease
(+) SE = A / A + C• Specificity (SP) = probability test (-) if disease
(-) SP = D / B + D
What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)?
TP A
FN C
B FP
Gold standardDisease + Disease –
Test +
Test – D TN
![Page 37: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/37.jpg)
Data-analysis
Perfect diagnostic testFalse Positive = 0 False Negative = 0
e.g. Fever > 380C as predictor for BM
20
40 160
70 90
200
20 90 110
BM+BM- tot.
Yes (+)Fever > 380C
No (-)
![Page 38: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/38.jpg)
Data-analysis reading 2 by 2 table
Horizontally• probability BM+ if fever+ = 20/110 = 18%
PV+ = A / A + B• probability BM - if fever- = 70/90 = 78%
PV- = D / C + DVertically• probability fever+ if BM+ = 20/40 = 50%
SE = A / A + C• probability fever- if BM- = 70/160 = 44%
SP = D / B + D
What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)?
20
TP AFN C
20
90
B FP
Gold standardBM+ BM–
Fever +
Fever –D TN
70
![Page 39: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/39.jpg)
Data-analysis: combination of determinants
• In practice not one single diagnosis based on 1 test
– Tests together distinguish ill/non-ill– Method: statistical model
• Moreover: diagnostic process is hierarchical (simple to invasive/expensive) • therefore always start with anamnesis model --> see case
![Page 40: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/40.jpg)
Data-analysisCase: model with all anamnestic tests (gender + age + fever + pain)
%BM = ƒ(gender, age, fever, pain)
• Statistical model can be seen as 1 (composed) test
• Quantify diagnostic value model with area under ROC curve (Receiver Operating Characteristic =Area Under Curve (AUC))
![Page 41: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/41.jpg)
Data-analysis
ROC Curve
1 - Specificity
1,00,75,50,250,00
Sen
sitiv
ity
1,00
,75
,50
,25
0,00
![Page 42: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/42.jpg)
Data-analysisCase: AUC model = 0,71
Informal interpretation AUC = % correctly diagnosed
The larger the ROC area the better the model AUC range: 0,5 1,0
AUC = 0,5 bad (Se = 1- Sp diagonal [coin])AUC > 0,7 reasonableAUC > 0,8 goodAUC > 0,9 excellent AUC = 1,0 perfect (Se=100% & 1-Sp=0%)
![Page 43: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/43.jpg)
Data-analysis
Quantify added value additional tests to previous tests
• Extend previous model (follow order practice)• Quantify change in AUC
CaseModel 1 anamnesis model + physical exam (5 extra tests) -->
AUC = 0,72 interpretation?
Model 2 anamnesis model + 3 blood tests ---> AUC = 0,90 interpretation?
![Page 44: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/44.jpg)
Data-analysis
ROC Curve
1 - Specificity
1,00,75,50,250,00
Sen
sitiv
ity1,00
,75
,50
,25
0,00
Coin flip
Patient hisotry
Pat hist + test
![Page 45: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/45.jpg)
Data-analysis
• The AUC does not directly say anything about individual patients and is therefore not directly applicable
![Page 46: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/46.jpg)
Reporting
Research question
Study set-up• Research population, setting, determinants,
outcome, design
Results• Predictive values (new) test and/or ROC curve• ROC curve combination of tests• Added value new test --> ROC curve
![Page 47: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/47.jpg)
BREAK
![Page 48: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/48.jpg)
Exercise 1Mercury thermometer or timpanic membrane infrared meter to use for temperature measurement
![Page 49: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/49.jpg)
Exercise 1Ad question 1
Research question: Can fever be determined with the TIM?Determinant: test under study = timpanic membrane infrared meterOutcome: fever determined with rectal mercury thermometer (RMT)Domain: Children in secondary/tertiary care (ER hospital)
![Page 50: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/50.jpg)
Exercise 1Ad question 2
77
TP AFN C
19
9
B FP TIM >38°
TIM 38°D TN
108
GS RMTFever+ Fever–
Se = probability TIM+ if RMT+ = 77/96 = 80 %
SP = probability TIM- if RMT- = 108/117 = 92%
![Page 51: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/51.jpg)
Exercise 1Ad question 3
77
TP AFN C
19
9
B FP TIM >38°
TIM 38°D TN
108
GS RMTFever+ Fever–
PV+ = probability RMT+ if TIM+ = 77/86 = 90 %
PV- = probability RMT- if TIM- = 108/127 = 85%
![Page 52: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/52.jpg)
Exercise 1
Ad question 4– The prior probability of fever in the general practice is
lower, e.g. 20% (X/213=0,2 X=43)– For similar Se and SP:
(A/43=0,8 A=34)(D/170=0,92 D=156)
– PV+ becomes lower (34/48 = 70%)– PV – becomes higher (156/164 = 95%) 9
43 170
156 164
213
34 14 48 TIM+
TIM-
GS RMTFever+ Fever–
![Page 53: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/53.jpg)
Exercise 1
Ad question 5
– In the general practice an unjustly referred or treated child is less of a problem than an unjust reassurance of the parents
– Negative predictive value must therefore be sufficiently high
![Page 54: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/54.jpg)
Ad question 1- Cross-sectional study in patients suspected of a
stomach or duodenum ulcer- For all patients anamnestic data were collected
- For all patients a gastroscopy was done- Independent diagnostic value of anamnestic factors
(determinant) for the diagnosis of ulcer (outcome:
determined by gastroscopy) were calculated
Exercise 2
![Page 55: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/55.jpg)
Exercise 2
Ad question 2Adults with stomach complaints referred to a gastro-enterology policlinic in a peripheral hospital
![Page 56: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/56.jpg)
Exercise 2
Ad question 3Score is 5, risk is 57%
![Page 57: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/57.jpg)
Exercise 2
Ad question 4- Everybody above the cut-off point has the same risk
(and the same below the cut-off point) - Of course this is not true and the score loses
precision- Preferably predictive values for score-categories and
predictive values for more cut-off points
![Page 58: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/58.jpg)
Exercise 2
Ad question 5
20
TP AFN C
5
11
B FP Test +
Test -D TN
64
Peptic ulcus+ –
PV+ = 20/31 = 65%
PV- = 64/69 = 93%
![Page 59: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/59.jpg)
Exercise 2
Ad question 6- Predictive values more favourable and therefore
preferred- But it is not about the isolated predictive value but
about the added diagnostic value given the results of the anamnestic score
![Page 60: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/60.jpg)
Exercise 2Ad question 7• Perform the anamnestic score and the breath test for a
population from the domain. Subsequently perform the reference test (endoscopy) for everybody
• Compare the next determinant-outcome relations: • P(ulcus) = ƒ (age, gender, anamnesis, ...)• P(ulcus) = ƒ (age, gender, anamnesis, ..., breath test)• Then compare the Receiver Operating Characteristic
(ROC)-curve of the models
![Page 61: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/61.jpg)
Exercise 2
Ad question 8- Breath test partially contains the same
information as the score- Suppose that the breath test is more often
positive with age- Then age should also be measured and
therefore the added value is less than when the breath test would be completely independent of the score
![Page 62: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/62.jpg)
Exercise 2
Ad question 9- Preferably not, but if the assessor is informed
of data score in practice, than it should be the same in the study
![Page 63: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/63.jpg)
SUMMARY
![Page 64: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/64.jpg)
Diagnostics Summary (1)
Diagnostics in practice– Reduce uncertaincies– Determines prognosis & policy
Diagnostic research Design– Observational– Descriptive
![Page 65: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/65.jpg)
• Cross-sectional– Simultaneous measurement determinant and
outcome (reference standard) – Always study >1 determinant
Design– Assess determinants as in practice
– Assess disease status & determinant status with double blinding
Diagnostics Summary (2)
![Page 66: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/66.jpg)
Analysis– Univariate (per determinant)– Multivariate: combination of test results in relation to
outcome • Endpoint = ƒ(combination of determinants)• Determine added value; first analyse least
invasive tests (as in practice)
Reporting– Mainly added value of test
Diagnostics Summary (3)
![Page 67: Diagnostic research](https://reader034.fdocuments.in/reader034/viewer/2022051518/56816077550346895dcfa24c/html5/thumbnails/67.jpg)
THANK YOUfinish