Receiver Operator Characteristics
-
Upload
dalton-calhoun -
Category
Documents
-
view
44 -
download
1
description
Transcript of Receiver Operator Characteristics
![Page 1: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/1.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 2: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/2.jpg)
Early radar signals
Is this an enemy plane?
Signal Noise Ratio
![Page 3: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/3.jpg)
The problems of decision• Sound the alarm when the signal is very small
– Advantages• Plenty of time to get the fighters off the ground• Reduce the number of bombers reaching the target
– Disadvantages• Lots of false alarms• Waste of gasoline, wear and tear on fighter planes• Exhaust fighter pilots
• Sound the alarm when the signal is unmistakable– Advantages
• No waste, no wear and tear, no exhaustion
– Disadvantages• More bombers get through, more bombs, more destructions
![Page 4: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/4.jpg)
Solution to decisions
• Code Yellow
– Signal suggests possible incoming bomber
– Pilots get dressed, fighter planes get loaded with gasoline and ammunition
• Code Orange
– Signal suggests incoming bomber likely
– Fighter planes towed to runway, pilots goes to the planes
• Code Red
– Signal is unmistakable
– Fighter planes take off
![Page 5: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/5.jpg)
Receiver Operator Characteristics
False alarms
Sen
sitiv
ity
Useless operatorSensitivity and false alarm rate changes together
Perfect operator100% Sensitive0% false alarms
Most operators
![Page 6: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/6.jpg)
Receiver Operator Characteristics
False alarms
Sen
sitiv
ityCode Yellow
Code Orange
Code Red
Increasingsignal
strength
![Page 7: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/7.jpg)
False alarms
Sen
sitiv
ityCode Yellow
Code Orange
Code Red
Increasingsignal
strength
Receiver Operator Characteristics
![Page 8: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/8.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 9: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/9.jpg)
![Page 10: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/10.jpg)
![Page 11: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/11.jpg)
![Page 12: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/12.jpg)
![Page 13: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/13.jpg)
![Page 14: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/14.jpg)
![Page 15: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/15.jpg)
![Page 16: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/16.jpg)
Statistical ROC• A measurement is normally distributed in two groups,
those outcome negative and those outcome positive
• Using a cut off level to make a decision will create a number of TP, FN, FP, and TN. From these Sensitivity and Specificity is calculated
• If the cut off value changes
– TP,FN,FP,TN changes
– Sensitivity and Specificity changes
• The relationship between Sensitivity and Specificity over the range of the measurement defines the ROC
![Page 17: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/17.jpg)
Statistical ROC
![Page 18: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/18.jpg)
![Page 19: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/19.jpg)
![Page 20: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/20.jpg)
![Page 21: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/21.jpg)
Receiver Operator Characteristics
• What is it and where does it come from
• Statistical aspects
• Use of ROC
![Page 22: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/22.jpg)
Advantages of using ROC
• It defines the quality of a test or prediction using a measurement without specifying a cut off value for decision making
• Assuming Normal distribution– The mean and Standard Error can be estimated
– The 95% CI can be estimated
– Statistical significance can be determined
– Whether one test is better than another can be determined
![Page 23: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/23.jpg)
23
چهار ترکیب ممکن برای تصمیم و خطا
درص�د افراد غی�ر مبتال که مدل :FPمثب�ت نادرس�ت•داده تشخیص مبتال را آنه�ا اشتباه ب�ه تص�میم
است.نادرس�ت• :FNمنف�ی� مدل که مبتال افراد درص�د
تص�میم ب�ه اشتباه آنه�ا را غیرمبتال تشخیص داده .است
:TNمنف�ی درس�ت• مبتال که افرادی غی�ر در ص�د مدل تصمیم به درستی آن را تشخیص داده است.
درص�د افرادی ک�ه بیماری دارند :TPمثب�ت درس�ت•داده تشخیص را آ�ن درس�تی ب�ه تص�میم مدل و
است.
TP+FN=100%FP+TN=100%
هدف مدل تصمیم حداقل کردن
)FN وFP (خطاهاست
![Page 24: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/24.jpg)
24
نکته
و FNدرص�د FP مستقل ص�ورت ب�ه شود حداقل نمی
افزای�ش م�ی یابد و FN ک�م شود FPاگ�ر برعکسمی ROCنمودار پیدا را بهین�ه مقدار
کند.
![Page 25: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/25.jpg)
25
:مثال
است. افراد در باال خون فشار تشخی�ص هدف سه در افراد خون فشار کار ای�ن انجام برای
گیری می شود. گروه فرضی اندازه
اند. - جمعیت سالم که تحت مراقبت قرار گرفته1- جمعی�ت بیماران مشمول مراقب�ت اولی�ه ک�ه به 2
خاطر شکایات جزئی تحت بررسی هستند.بستری 3 قلب�ی بخ�ش در ک�ه بیماران�ی - جمعی�ت
هستند.
![Page 26: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/26.jpg)
26
![Page 27: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/27.jpg)
27
ROC رسم منحنی
![Page 28: Receiver Operator Characteristics](https://reader034.fdocuments.in/reader034/viewer/2022042901/5681364b550346895d9dcaf2/html5/thumbnails/28.jpg)