Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case...
-
Upload
deasia-yule -
Category
Documents
-
view
221 -
download
2
Transcript of Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case...
![Page 1: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/1.jpg)
Diagnostic research
![Page 2: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/2.jpg)
Lecture Contents
I. Diagnostics in practice
- Explained with a case
II. Scientific diagnostic research– Design– Data-analysis– Reporting
III. Exercises
IV. Summary
![Page 3: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/3.jpg)
Diagnostics in practice
Diagnostics always start with a patient with a complaint/symptom
Case: neck stiffness • Child, 2 years-old, comes to ER with parents • Child turns out to have a very stiff neck
What is the physician’s aim?
![Page 4: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/4.jpg)
Diagnostics in practice
Aim of the physician• Quickly and efficiently determine the correct
diagnosis
Why diagnose?• Basis medical handling • Determines treatment choice• Gives information about prognosis
What are possible diagnoses for neck stiffness?
![Page 5: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/5.jpg)
Diagnostics in practice
Differential diagnosis (DD)• Bacterial meningitis • Viral meningitis• Pneumonia• ENT infection• Other (e.g. myalgia)
What is the most important diagnosis? Which one does the physician not want to miss?
![Page 6: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/6.jpg)
Diagnostics in practice
Most important diagnosis• Bacterial meningitis (BM) • If missed: often fatal
![Page 7: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/7.jpg)
Diagnostics in practice
Suppose: 20% of all children on the ER with neck stiffness has BM – 20% with disease in that population =
prevalence– Prior-probability
What is your decision for the child in this case?
![Page 8: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/8.jpg)
Diagnostics in practice
Decision for child in case • Prior-probability too low to treat• Prior-probability too high to send home
Decision: reduce uncertainty diagnostics
What is the best test?
![Page 9: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/9.jpg)
Diagnostics in practice
Best test
Lumbal punction (liquor culture)
![Page 10: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/10.jpg)
Diagnostics in practice
Gold standard• True disease status; ‘truth’
– Never 24 karat• Reference standard/test• Decisive test with doubt
Perform reference test for everybody (=every child on ER with neck stiffness)?
![Page 11: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/11.jpg)
Reference test for everybody?• Unethical too invasive/risky• Inefficient too expensive• Do not perform unnecessarily
How should we then determine the probability of disease presence and what would be ideal?
Diagnostics in practice
![Page 12: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/12.jpg)
How then?• Simpler diagnostics:
– Usually anamnesis, physical exam, simple lab tests, imaging, etc.
– Ideal: diagnosis without reference test
• Diagnostic process in practice: – Stepwise process: less more invasive– Not one diagnosis based on 1 test – Each item: separate test
Diagnostics in practice
![Page 13: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/13.jpg)
Diagnostics in practice
Suppose: after anamnesis & PE 10% probability of BM
• Probability of disease given test results = posterior-probability
• The bigger the difference between prior and posterior probability, the better the diagnostic value of the tests
Our decision for child in case: probability is too high to send home --> next step?
![Page 14: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/14.jpg)
Diagnostics in practice
Next step– Additional research, e.g.
blood tests (leucocytes,
CRP, sedimentation, etc.)
![Page 15: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/15.jpg)
Diagnostics in practice
Suppose: 1% posterior-probability after anamnesis, PE+ simple lab tests posterior probability low enough to send home
• Ideal diagnostic process: simple tests reduce posterior probability to 0 or 100% (without reference)• Most often physician continues testing until sufficiently sure (approximation of 0 or 100%)• Choose when sufficiently sure: depends on prognosis of disease if untreated + risks/costs treatment
![Page 16: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/16.jpg)
Diagnostics in practice
Summarizing • What does diagnosing involve in practice?
– Estimation of probability of disease presence based on test results of the patient
When is the probability of disease best estimated? Why is this usually not done?
![Page 17: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/17.jpg)
Diagnostics in practice
Why not all possible tests?– Invasive (for patient and budget)– Unnecessary: different test results give same info– However: In practice often more tested than
necessary!
What diagnostics truly necessary scientific diagnostic research
![Page 18: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/18.jpg)
BREAK
![Page 19: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/19.jpg)
Study design
Scientific diagnostic research– What tests truly contribute to probability estimation?– Has to serve practice follow practice
![Page 20: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/20.jpg)
Study design
• Research question• Domain• Study population • Determinant(s): test(s) to study• Endpoint: presence/absence disease (outcome)• Study design: design• Data analysis, interpretation + reporting
![Page 21: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/21.jpg)
Research question
• With as few as possible simple, safe, and cheap tests estimate the probability of the presence/absence of disease.
• Determinant-outcome relation:– probability of disease as a function of test results– outcome = probability of disease = % = prevalence– test results = determinants
![Page 22: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/22.jpg)
Research question
Case• What tests contribute to probability estimation of
presence or absence of BM in children with neck stiffness at the ER?
• Or: Determinants of presence/absence disease (BM)?
• %BM = ƒ(age, gender, fever, blood leucocytes, blood CRP, etc)
![Page 23: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/23.jpg)
Research population
Case: • All children with neck stiffness
in 2002 at ER Utrecht
![Page 24: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/24.jpg)
Domain
• For whom domain, generalisation = type of patient with certain symptom/complaint +
setting• Research population = 1 sample from domain
Case: All children (e.g. in Western world) suspected of disease (BM) based on neck stiffness (characteristic) in secondary care (setting)
![Page 25: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/25.jpg)
Determinants
= Tests to study
• Diagnostic determinants• All possible important tests (in domain)
CaseItems anamnesis, PE and lab (blood and urine)
tests
![Page 26: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/26.jpg)
Endpoint
‘True’ presence/absence disease = Diagnostic outcome = Results reference test
NB: reference = not infallible but always best available test in practice at that moment
Case• Positive liquor culture
![Page 27: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/27.jpg)
PICO EBM
• Population/ problem• Intervention• Comparison/ control• Outcome
• Domain• Determinant• Reference test• Outcome
![Page 28: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/28.jpg)
Measure determinants/endpoint
• Determinants– Without knowledge (blinded) of the outcome – Same method in study and practice
never measure more precisely than in practice (overestimation information yield)
• Endpoint– Assessment blind for determinants– With the best possible test known in practice
![Page 29: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/29.jpg)
Study design
• Observational and descriptive – Observational = no manipulation of determinants– Descriptive = not causal
– if the determinant only predicts– no hypothesis functional mechanism determinant-
outcome
• >1 determinant
![Page 30: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/30.jpg)
Study design
• Cross-sectional= Simultaneously measure determinants and outcome
![Page 31: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/31.jpg)
Data-analysis
After data collection, per patient– Value determinants (test results)– Diagnostic outcome (reference test)
![Page 32: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/32.jpg)
Data-analysis• Data analysis: 3 steps
1) Estimation a priori probability (without test results)
2) Compare each test result separately with reference = univariate3) Compare combination of test results with reference = multivariate (via model)
- Following order in practice - Determine added value test result to already collected (previous) test results
![Page 33: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/33.jpg)
Data-analysis
CaseData scientific research available:200 patients with neck stiffness at ER
Liquor culture positive (BM+) n=40Liquor culture negative (BM-) n=160
Step 1: A priori probability (prevalence) of BM?
= % BM+ = 40/ 200 patients = 20%
![Page 34: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/34.jpg)
Data-analysis reading 2 by 2 table
Disease
Presence Absence
Test Positive True positiveA
False positiveB
Negative CFalse negative
DTrue negative
• Step 2: Analysis per determinant (univariate) • Use 2 by 2 table
![Page 35: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/35.jpg)
Data-analysis reading 2 by 2 table
Horizontally• Positive predictive value (PV+)
= probability Disease + if test + PV+ = A / A + B
• Negative predictive value (PV-)= probability disease - if test -
PV- = D / C + DVertically• Sensitivity (SE) = probability test + if disease +
SE = A / A + C• Specificity (SP) = probability test - if disease -
SP = D / B + D
What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)?
TP A
FN C
B FP
Gold standardDisease + Disease –
Test +
Test –D TN
![Page 36: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/36.jpg)
Data-analysis
Perfect diagnostic testFalse Positive = 0 False Negative = 0
e.g. Fever > 380C as predictor for BM
20
40 160
70 90
200
20 90 110
BM+BM- tot.
Yes (+)Fever > 380C
No (-)
![Page 37: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/37.jpg)
Data-analysis reading 2 by 2 table
Horizontally• probability BM+ if fever+ = 20/110 = 18%
PV+ = A / A + B• probability BM - if fever- = 70/90 = 78%
PV- = D / C + DVertically• probability fever+ if BM+ = 20/40 = 50%
SE = A / A + C• probability fever- if BM- = 70/160 = 44%
SP = D / B + D
What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)?
20
TP A
FN C
20
90
B FP
Gold standardBM+ BM–
Fever +
Fever –D TN
70
![Page 38: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/38.jpg)
BREAK
![Page 39: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/39.jpg)
Exercise 1
Mercury thermometer or timpanic membrane infrared meter
![Page 40: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/40.jpg)
Exercise 1
Ad question 1
Research question: Can fever be determined with the TIM?
Determinant: test under study = timpanic membrane infrared meter
Outcome: fever determined with rectal mercury thermometer (RMT)
Domain: Children in secondary/tertiary care (ER hospital)
![Page 41: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/41.jpg)
Exercise 1
Ad question 2
77
TP A
FN C
19
9
B FP TIM >38°
TIM 38°D TN
108
GS RMTFever+ Fever–
Se = probability TIM+ if RMT+ = 77/96 = 80 %
SP = probability TIM- if RMT- = 108/117 = 92%
![Page 42: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/42.jpg)
Exercise 1
Ad question 3
77
TP A
FN C
19
9
B FP TIM >38°
TIM 38°D TN
108
GS RMTFever+ Fever–
PV+ = probability RMT+ if TIM+ = 77/86 = 90 %
PV- = probability RMT- if TIM- = 108/127 = 85%
![Page 43: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/43.jpg)
Exercise 1
Ad question 4– The prior probability of fever in the general practice is
lower, e.g. 20% (X/213=0,2 X=43)– For similar Se and SP:
(A/43=0,8 A=34)
(D/170=0,92 D=156)– PV+ becomes lower (34/48 = 70%)– PV – becomes higher (156/164 = 95%) 9
43 170
156 164
213
34 14 48 TIM+
TIM-
GS RMTFever+ Fever–
![Page 44: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/44.jpg)
Exercise 1
Ad question 5
– In the general practice an unjustly referred or treated child is less of a problem than an unjust reassurance of the parents
– Especially the negative predictive value must therefore be sufficiently high
![Page 45: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/45.jpg)
Data-analysis: combination of determinants
• In practice not one single diagnosis based on 1 test
– Tests together distinguish ill/non-ill– Method: statistical model
• Moreover: diagnostic process is hierarchical– (simple --> invasive/expensive) --> always start with
anamnesis model --> see case
![Page 46: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/46.jpg)
Data-analysisCase: model with all anamnestic tests gender + age + fever + pain
%BM = ƒ(gender, age, fever, pain)
• Statistical model can be seen as 1 (composed) test
• Quantify diagnostic value model with area under ROC curve (Receiver Operating Characteristic =Area Under Curve (AUC))
![Page 47: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/47.jpg)
Data-analysis
ROC Curve
1 - Specificity
1,00,75,50,250,00
Se
nsi
tivity
1,00
,75
,50
,25
0,00
![Page 48: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/48.jpg)
Data-analysis
Case: AUC anamnesis model = 0,71
Informal interpretation AUC = % correctly diagnosed
The larger the ROC area the better the model AUC range: 0,5 1,0
AUC = 0,5 bad (Se = 1- Sp diagonal [coin])AUC > 0,7 reasonableAUC > 0,8 goodAUC > 0,9 excellent AUC = 1,0 perfect (Se=100% & 1-Sp=0%)
![Page 49: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/49.jpg)
Data-analysis
Quantify added value additional tests to previous tests
• Extend previous model (follow order practice)• Quantify change in AUC
CaseModel 1 anamnesis model + physical exam (5 extra tests) -->
AUC = 0,72 interpretation?Model 2 anamnesis model + 3 blood tests ---> AUC = 0,90
interpretation?
![Page 50: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/50.jpg)
Data-analysis
ROC Curve
1 - Specificity
1,00,75,50,250,00
Se
nsi
tivity
1,00
,75
,50
,25
0,00
Coin flip
Patient hisotry
Pat hist + test
![Page 51: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/51.jpg)
Data-analysis
• The AUC does not directly say anything about individual patients and is therefore not directly applicable
![Page 52: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/52.jpg)
Reporting
Research question
Study set-up• Research population, setting, determinants, outcome,
design
Results• Predictive values (new) test and/or ROC curve• ROC curve combination of tests• Added value new test --> ROC curve
![Page 53: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/53.jpg)
Ad question 1- Cross-sectional study in patients suspected of a
stomach or duodenum ulcer
- For all patients anamnestic data were collected
- For all patients a gastroscopy was done
- Independent diagnostic value of anamnestic factors (determinant) for the diagnosis of ulcer (outcome: gastroscopy) were calculated
Exercise 2
![Page 54: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/54.jpg)
Exercise 2
Ad question 2
Adults with stomach complaints referred to a gastro-enterology policlinic in a peripheral hospital
![Page 55: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/55.jpg)
Exercise 2
Ad question 3
Score is 5, risk is 57%
![Page 56: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/56.jpg)
Exercise 2
Ad question 4- Everybody above the cut-off point has the same risk
(and the same below the cut-off point)
- Of course this is not true and the score loses precision
- Preferably predictive values for score-categories and predictive values for more cut-off points
![Page 57: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/57.jpg)
Exercise 2
Ad question 5
20
TP A
FN C
5
11
B FP Test +
Test -D TN
64
Peptic ulcus+ –
PV+ = 20/31 = 65%
PV- = 64/69 = 93%
![Page 58: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/58.jpg)
Exercise 2
Ad question 6- Predictive values more favourable and therefore
preferred
- But it is not about the isolated predictive value but about the added diagnostic value given the results of the anamnestic score
![Page 59: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/59.jpg)
Exercise 2
Ad question 7• Perform the anamnestic score and the breath test for a
population from the domain. Subsequently perform the reference test (endoscopy) for everybody
• Compare the next determinant-outcome relations: • P(ulcus) = ƒ (age, gender, anamnesis, ...)• P(ulcus) = ƒ (age, gender, anamnesis, ..., breath test)• Then compare the Receiver Operating Characteristic
(ROC)-curve of the models
![Page 60: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/60.jpg)
Exercise 2
Ad question 8
- Breath test partially contains the same information as the score
- Suppose that the breath test is more often positive with age, then the breath test also measures age and therefore the added value is less than when the breath test would be completely independent of the score
![Page 61: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/61.jpg)
Exercise 2
Ad question 9
- Preferably not, but if the assessor is informed of the data in the score in practice, than it should be the same in the study
![Page 62: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/62.jpg)
Diagnostics Summary (1)
Diagnostics in practice– Uncertainty reduction– Determines prognosis & determines policy
Diagnostic research
Design– Observational– Descriptive
![Page 63: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/63.jpg)
– Cross-sectional• Simultaneous measurement determinant
and outcome (reference standard) – Always study >1 determinant
Design– Assess determinants as in practice
– Assess disease status & determinant status with double blinding
Diagnostics Summary (2)
![Page 64: Diagnostic research. Lecture Contents I. Diagnostics in practice - Explained with a case II.Scientific diagnostic research – Design – Data-analysis –](https://reader035.fdocuments.in/reader035/viewer/2022062318/551b49b5550346d41a8b5bd1/html5/thumbnails/64.jpg)
Analysis– Univariate (per determinant)– Multivariate: combination of test results in relation to
outcome • Endpoint = ƒ(combination of determinants)• Determine added value; first analyse least
invasive tests (as in practice)
Reporting– Mainly added value of test
Diagnostics Summary (3)