Dr. Alaa Hassan - MB BCh, MSc (PH), MD

ROC Curve interpretation

Dr. Alaa El Dine HassanMB BCh, MSc (PH), MD

Consultant- Community and Preventive Medicine

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

How Much is Too Much?

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

• In real life situation we rarely have a test which give you Positive / Negative readings

• A test mainly gives you a numerical value and it is our role to decide:– what values should be considered positive – what values should be considered negative.

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

• A sphygmomanometer gives a reading for blood pressure.

• It is Your role do decide how much means that the patient is hypertensive.

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

• The job would have been much easier if there was no overlap between the values of diseased and disease free individuals

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Diseased

Disease free5

10

15

20

25

30

35

The job would have been much easier if

there was no overlap between the values of diseased and disease

free individuals

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

• In practice, we choose a cutoff level above which we consider the test to be Positive and below which we consider the test to be Negative.

• The position of the cut point will determine the number of :– true positive, true negatives, – false positives and false negatives.

• We may wish to use different cutoff levels for different clinical situations if we wish to minimize one of the erroneous types of test results.

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

• Receiver Operating Characteristic (ROC) curves plot the sensitivity of a test versus its false positive rate (1-Specificity) for various Cut-off points

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

If we recall the values from the worksheet:

Cut-off point Sensitivity Specificity

10 0.9 0.266

15 0.8 0.53

20 0.6 0.733

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

calculate 1- specificity

Cut-off point Sensitivity Specificity 1- Specificity

10 0.9 0.266 0.734

15 0.8 0.53 0.47

20 0.6 0.733 0.267

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Cutoff =10Sensitivity = 0.9

1-Specificity = 0.734

Cutoff =15Sensitivity = 0.8

1-Specificity = 0.47

Cutoff =20Sensitivity = 0.6

1-Specificity = 0.267

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Optimum Cut-off Point The nearer to the Upper

Left Corner

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Hisotry• The name "Receiver Operating Characteristic" came from

"Signal Detection Theory" developed during World War II for the analysis of radar images.

• Radar operators had to decide whether a blip on the screen represented an enemy target, a friendly ship, or just noise.

• Signal detection theory measures the ability of radar receiver operators to make these important distinctions.

• Their ability to do so was called the Receiver Operating Characteristics.

• It was not until the 1970's that signal detection theory was recognized as useful for interpreting medical test results.

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

The area under the ROC Curve (AUC)

• Measure of a diagnostic test's discriminatory power.• An AUC value of 1.0 indicates a (theoretically)

perfect test (i.e., 100% sensitive and 100% specific). • An AUC value of 0.5 indicates no discriminative value

(i.e., 50% sensitive and 50% specific) and is represented by a straight, diagonal line extending from the lower left corner to the upper right

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

Sen

siti

vity

1 - Specificity

Area under ROC Curve

Dr. Alaa Hassan - MB BCh, MSc (PH), MD

It is important to note that ROC performance may change when the diagnostic test is

applied to different Populations