Validity and Reliability
Click here to load reader
-
Upload
tauseef-javeed -
Category
Healthcare
-
view
1.170 -
download
0
Transcript of Validity and Reliability
Validity and Reliability of
Analytical TestsBY
DR MUHAMMAD TAUSEEF JAVEDCONSULTANT MINISTRY OF HEALTH
MAKKAH&
IPH LAHORE
Analytical Tests include both:• Screening Tests • Diagnostic Tests
•Two Important Objectives
To distinguish between people in the population who have the diseases and those who do not
To determine how good the test is in separating populations of people with and without the disease in question?
Epidemiological Surveillance
vs.Screening
Epidemiological Surveillance
• What is it?• Why do it?
Epidemiological Surveillance
• Definition - ongoing & systematic collection, analysis & interpretation of data related to health, disease & conditions
• Two typesPassive Surveillance – uses available data or
reporting from health care provider or regional health officer
Active Surveillance – periodic field visits to health care facilities to identify new cases
• The present approach is the survey
Epidemiological Surveillance
• Why do it?Can help discover and control the
transmission of infectious diseasesPrevention and control programs can be
planned and implemented
Screening• Definition - use of quick and simple
testing procedures to identify and separate persons: who have a disease from those that do not
ORwho are apparently (appear to be) well, but
who may be at risk of a disease, from those who probably don’t have the disease.
Terms Related to Screening Tests
• Validity - relates to accuracy (correctness)
• Reliability - repeatability
• Yield - the # of tests that can be done in a time period
Terms Related to Screening Tests (cont’d)
• Sensitivity - ability of a test to identify those who have disease
• Specificity - ability of a test to exclude those who don’t have disease
Terms Related to Screening Tests (cont’d)
• Tests with dichotomous results – tests that give either positive or negative results
• Tests of continuous variables – tests that do not yield obvious “positive” or “negative” results, but require a cutoff level to be established as criteria for distinguishing between “positive” and “negative” groups
An important public health consideration, particularly in screening free-living populations, is:
How good is the test at identifying people with the disease and without the disease?In other words:If we screen a population, what proportion of people who have the disease will be correctly identified?
POPULATION
Test Results With Disease Without Disease
PositiveTrue Positive
(TP)False Positive
(FP)
NegativeFalse Negative
(FN)True Negative
(TN)
Sensitivity = = X 100True positivesTrue positives +false negatives
True positivesAll persons with
the disease
= TP
TP + FN
Specificity = = X 100True negativesTrue negatives+false positives
True negativesAll persons without the
disease
= TN
TN + FP
Percent false negatives = % of people with the disease who were not detected by the test
FN FN + TP X 100
Percent false positives = % of people without the disease who were incorrectly labeled by the test as having the disease
FPFP + TN X 100
In the clinical setting, a more important question is:
If the test results are positive (or negative) in a given patient, what is the probability that this patient has (or does not have) the disease?
In other words:What proportion of patients who test positive (or negative) actually have (or do not have) the disease in question?
Predictive Value
Pos. PV = X 100 = % True Positives TP + FP
Neg. PV = X 100 = % True Negatives TN + FN
Biologic Variation of Human Populations &
Diagnostic Issues
Distribution of Tuberculin Reactions
Bimodal Distribution
Easy to distinguish between exposed group and those not exposed.
Distribution of Systolic Blood Pressure
Unimodal DistributionWith continuous variables, a
cutoff level must be established to separate the hypertensive
group. Could choose based on statistics, but better to base on
biologic considerations.
Effects of Choosing
Different Cutoff Levels for Diabetes
Diagnosis in Population with 50% Prevalence
Real World
Pseudo-Real World
The major issue with deciding to set a cutoff high or low
is the problem of false positives
and false negatives.
Possible Groups with Dichotomous Test
True Disease Status is Known,
as with dichotomous tests.
Grouping All Positives and All Negatives
True Disease Status is Unknown,
as with continuous variables.
Artificial Cutoff
•Use of Multiple Screening Tests
Sequential (Two-stage) Testing
Simultaneous Testing
Hypothetical Two-Stage Screening
Only Pos. Test 1 are given Test 2
Hypothetical Two-Stage Screening (cont.)
TEST 2 (Glucose Tolerance Test)Sensitivity = 90%Specificity = 90%
DIABETES
+ -TEST
RESULTS + 315 190 505
- 35 1710 1745
350 1900 2250
Net Sensitivity = 315/500 = 63%Net Specificity = 7600 + 1710 = 98%
9500
Predictive Value
Prevalence & Predictive Value
Positive
As prevalence increases, positive predictive value increases.
Prevalence & Predictive Value
Note: Test has 95% sensitivity and 95% specificity
Specificity & Predictive
Value
As specificity increases,
positive predictive value increases. As sensitivity increases, positive predictive value also increases, but to a
much lesser extent.
Specificity & Predictive Value
As specificity increases, positive predictive value increases.
Results reliable but NOT valid
Results reliable and
valid
Reliability (Repeatability) of Tests