Structural Changes -Color Images
1
Structural Changes -Color Images Algorithms for Structural and Functional Change Analysis from Multi-Modal Retinal Fundus Images Harihar Narasimha-Iyer 1 , James M. Beach 2 , Ali Can 3 , Badrinath Roysam 1 , Charles V. Stewart 1 , Howard Tanenbaum 4 1 Rensselaer Polytechnic Institute, Troy, NY-12180 2 Institute for Technology Development, Stennis Space Center, Mississippi 39529, USA 3 Woods Hole Oceanographic Institution, Woods Hole, Massachusetts 02543, USA. 4 The Center for Sight, 349 Northern Blvd., Albany, New York 12204, USA. This work was supported in part by CenSSIS, the Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award Number EEC- 9986821) Type of Color Type of Color Change Change Significance Significance Increase in Red Appearance of bleeding or microaneurysm Decrease in Red Disappearance of bleeding or microaneurysm Increase in Yellow Appearance of exudate Decrease in Yellow Disappearance of exudate Types of Changes in Non- vascular Regions 1. M. J. Cree, J.A. Olson, K.C. McHardy, J.V. Forrester and P.F. Sharp, “Automated microaneurysm detection,” IEEE Int. Conf. on Image Processing, vol. 3, pp.699-702, Lausanne, Switzerland, 1996. 2. M.J. Cree, J.A. Olson, K.C. McHardy, P.F. Sharp, J.V. Forrester, “A fully automated comparative microaneurysm digital detection system,” Eye, vol. 11, pp. 622-628,1998. 3. K.A. Goatman, M. J. Cree, J.A. Olson, J.V. Forrester and P.F. Sharp, “Automated measurement of microaneurysm turnover,” Investigative ophthalmology and Visual Science, vol.44,5335-5341, 2003. 4. Z.B. Sbeh, L.D. Cohen, G. Mimoun and G. Coscas, “A new approach of geodesic reconstruction for drusen segmentation in eye fundus images,” IEEE Trans. Medical. 5. J.M. Beach, K.J. Schwenzer, S. Srinivas, D. Kim, and J.S. Tiedeman, “Oximetry of retinal vessels by dual-wavelength imaging: calibration and influence of pigmentation,” Journal of Appl. Physiology. vol. 86,748-758, 1999. 6. H. Narasimha-Iyer, J. M. Beach, B. Khoobehi, J. Ning, H. Kawano and B. Roysam, “Algorithms for Automated Oximetry along the Retinal Vascular Tree from Dual- Wavelength Fundus Images,” Accepted for publication, Journal of Biomedical Optics, May 2005. 7. H. Narasimha-Iyer, A. Can, B. Roysam, C.V. Stewart, H.L. Tanenbaum, A. Majerovics and H. Singh, " Robust Detection and Classification of Longitudinal Changes in Color Retinal Fundus Images for Monitoring Diabetic Retinopathy," Accepted for publication in IEEE Transactions on Biomedical Engineering, September 2005. State of the art Only a few methods have been described for quantifying the dynamic nature of diabetic retinopathy from a time series of retinal images. Cree et al. [1, 2] detect microaneurysms from a region of interest around the fovea from images at distinct time points and compare them to find changes. Studies of microaneurysm turnover were also made by Goatman et al. [3]. They detected microaneurysms from baseline and follow-up angiograms, registered the images and categorized the microaneurysms into three classes namely, static, new and regressed. Sbeh and Cohen [4] segment drusen based on a morphological method called geodesic reconstruction and study the evolution of drusen over time..All these methods have the limitation that they handle only a certain kind of lesion and also the changes are obtained from individual segmentation of the images. The present work contributes multiple advances over the above literature, overcoming many of the noted limitations. The net result is an algorithm and a systematic non- limiting framework that allows a broad range of longitudinal changes to be detected and classified with a high degree of reliability. The Framework Vascular Linking 1 2 3 (,, )/ (,, ); (,, ) (,, ); (,, ) (,, ) 2; i green i red i green j green i green j green f R xy R xy f R xy R xy f R xy R xy Bayesian Change Classifier ( | ) ( | ), m =1,2...5,k m. k m PC X PC X for Features used for the classification: Bayesian classification rule: Bayesian model selection used to associate descriptions with regions of changes in the vasculature. Information from the tracing algorithm (IUS) and the change classifier are combined to get the likelihood functions. Describing Vascular Changes 1 2 ( , ) i region i segment XOR IUS IUS 1 1 , , 2 2 ( , , /region ihasdisappeared) ( /region ihasdisappeared) ( , /region ihasdisappeared) im jn im jn pII pII p 1 2 1 2 ( , /region ihasdisappeared) ( , / ) x x x region pI I pI I red down ( , /region ihasdisappeared)= 2 im jn p confidence factorassociated w ith the IU S outputs. Changes in Nonvascular regions Changes in Nonvascular regions t i Modality m Modality k Image Understanding System IUS Assisted Image Correction Image Understanding System IUS Assisted Image Correction Registration Application Specific Pre- Application Specific Pre- Processing Processing Application Specific Pre- Application Specific Pre- Processing Processing Spectral Change Features Structural/Functional Change Analyzer ij T i IU S j IU S i C j C i D j D ij SP i IU S j IU S (Objects) (Corrected Images) (Features for change analysis) (Corrected & Processed Images) High level change descriptions t j Modality m Modality k Identification of Vessel Type Retinal Image Understanding System Functional Changes-Dual Wavelength Images Changes in Blood Oxygen Saturation Room Air Breathing Pure O 2 Breathing OD Measurement Day 1 Trace Mask –IUS 1 Day 2 Trace Mask – IUS 2 Vascular Changes Change Regions Absorption spectra of HbO 2 ( ) and Hb ( ) , 570 , ( ) log out seg im in seg I OD I 1 2 3 ; ( ) ( ); ( ) ( ). im jn im jn f ODR f ODR f ODR ODR 600 570 ( ) ( ) . ( ) im im im OD ODR OD Retinal Vessel Oximetry The image at 570nm is traced and the trace segments are linked and named according to the hierarchy Image correction involves median filtering to remove any noise Measure the optical density (OD) based on the minimum reflectance inside the vessel and the average outside reflectance
description
Absorption spectra of HbO 2 ( ) and Hb ( ). t i. t j. Modality k. Modality k. Modality m. Modality m. Image Understanding System. Image Understanding System. Registration. (Objects). IUS Assisted Image Correction. IUS Assisted Image Correction. (Corrected Images). - PowerPoint PPT Presentation
Transcript of Structural Changes -Color Images
Structural Changes -Color Images
Algorithms for Structural and Functional Change Analysis from Multi-Modal Retinal Fundus ImagesHarihar Narasimha-Iyer1, James M. Beach2 , Ali Can3, Badrinath Roysam1, Charles V. Stewart1, Howard Tanenbaum4
1Rensselaer Polytechnic Institute, Troy, NY-121802Institute for Technology Development, Stennis Space Center, Mississippi 39529, USA
3Woods Hole Oceanographic Institution, Woods Hole, Massachusetts 02543, USA. 4The Center for Sight, 349 Northern Blvd., Albany, New York 12204, USA.
This work was supported in part by CenSSIS, the Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award Number EEC-9986821)
Type of Color ChangeType of Color Change SignificanceSignificance
Increase in RedAppearance of bleeding or
microaneurysm
Decrease in RedDisappearance of bleeding or
microaneurysm
Increase in Yellow Appearance of exudate
Decrease in Yellow Disappearance of exudate
Types of Changes in Non-vascular Regions
1. M. J. Cree, J.A. Olson, K.C. McHardy, J.V. Forrester and P.F. Sharp, “Automated microaneurysm detection,” IEEE Int. Conf. on Image Processing, vol. 3, pp.699-702, Lausanne, Switzerland, 1996.
2. M.J. Cree, J.A. Olson, K.C. McHardy, P.F. Sharp, J.V. Forrester, “A fully automated comparative microaneurysm digital detection system,” Eye, vol. 11, pp. 622-628,1998.
3. K.A. Goatman, M. J. Cree, J.A. Olson, J.V. Forrester and P.F. Sharp, “Automated measurement of microaneurysm turnover,” Investigative ophthalmology and Visual Science, vol.44,5335-5341, 2003.
4. Z.B. Sbeh, L.D. Cohen, G. Mimoun and G. Coscas, “A new approach of geodesic reconstruction for drusen segmentation in eye fundus images,” IEEE Trans. Medical. Imaging, vol. 20, no.12, pp.1321-1333, 2001.
5. J.M. Beach, K.J. Schwenzer, S. Srinivas, D. Kim, and J.S. Tiedeman, “Oximetry of retinal vessels by dual-wavelength imaging: calibration and influence of pigmentation,” Journal of Appl. Physiology. vol. 86,748-758, 1999.
6. H. Narasimha-Iyer, J. M. Beach, B. Khoobehi, J. Ning, H. Kawano and B. Roysam, “Algorithms for Automated Oximetry along the Retinal Vascular Tree from Dual-Wavelength Fundus Images,” Accepted for publication, Journal of Biomedical Optics, May 2005.
7. H. Narasimha-Iyer, A. Can, B. Roysam, C.V. Stewart, H.L. Tanenbaum, A. Majerovics and H. Singh, " Robust Detection and Classification of Longitudinal Changes in Color Retinal Fundus Images for Monitoring Diabetic Retinopathy," Accepted for publication in IEEE Transactions on Biomedical Engineering, September 2005.
State of the artOnly a few methods have been described for quantifying the dynamic nature of diabetic retinopathy from a time series of retinal images. Cree et al. [1, 2] detect microaneurysms from a region of interest around the fovea from images at distinct time points and compare them to find changes. Studies of microaneurysm turnover were also made by Goatman et al. [3]. They detected microaneurysms from baseline and follow-up angiograms, registered the images and categorized the microaneurysms into three classes namely, static, new and regressed. Sbeh and Cohen [4] segment drusen based on a morphological method called geodesic reconstruction and study the evolution of drusen over time..All these methods have the limitation that they handle only a certain kind of lesion and also the changes are obtained from individual segmentation of the images. The present work contributes multiple advances over the above literature, overcoming many of the noted limitations. The net result is an algorithm and a systematic non-limiting framework that allows a broad range of longitudinal changes to be detected and classified with a high degree of reliability.
The Framework
Vascular Linking
1
2
3
( , , ) / ( , , );
( , , ) ( , , );
( , , ) ( , , ) 2;
i green i red
i green j green
i green j green
f R x y R x y
f R x y R x y
f R x y R x y
Bayesian Change Classifier
( | ) ( | ), m=1,2...5,k m.k mP C X P C X for
Features used for the classification:
Bayesian classification rule:
Bayesian model selection used to associate descriptions with regions of changes in the vasculature. Information from the tracing algorithm (IUS) and the change classifier
are combined to get the likelihood functions.
Describing Vascular Changes
1 2 ( , )iregion i segment XOR IUS IUS
1 1, ,2 2( , , / region i has disappeared) ( / region i has disappeared)
( , / region i has disappeared)
im jn
im jn
p I I p I I
p
1 2 1 2( , / region i has disappeared) ( , / )x xx region
p I I p I I red down
( , / region i has disappeared ) = 2im jnp
confidence factor associated with the IUS outputs.
Changes in Nonvascular regions
Changes in Nonvascular regions
ti
Modality m Modality k
Image Understanding System
IUS Assisted Image Correction
Image Understanding System
IUS Assisted Image Correction
Registration
Application Specific Pre-Application Specific Pre-Processing Processing
Application Specific Pre-Application Specific Pre-ProcessingProcessing
Spectral Change Features
Structural/Functional Change Analyzer
ijT
iIUS jIUS
iCjC
iD jD
ijSP
iIUS jIUS
(Objects)
(Corrected Images)
(Features for change analysis)
(Corrected & Processed Images)
High level change descriptions
tj
Modality m Modality k
Identification of Vessel Type
Retinal Image Understanding System
Functional Changes-Dual Wavelength Images
Changes in Blood Oxygen Saturation
Room Air Breathing Pure O2 BreathingOD Measurement
Day 1
Trace Mask –IUS1
Day 2
Trace Mask – IUS2
Vascular Changes
Change Regions
Absorption spectra of HbO2 ( ) and Hb ( )
,570
,
( ) log out segim
in seg
IOD
I
1
2
3
;( )
( );
( ) ( ).
im
jn
im jn
f ODR
f ODR
f ODR ODR
600
570
( )( ) .
( )im
imim
ODODR
OD
Retinal Vessel Oximetry
The image at 570nm is traced and the trace segments are linked and named according to the hierarchy
Image correction involves median filtering to remove any noise
Measure the optical density (OD) based on the minimum reflectance inside the vessel and the average
outside reflectance
