Ideal Observer Approach for Assessment of Image Quality on Stereo Displays Devices for Medical...
38
Ideal Observer Approach for Assessment of Image Quality on Stereo Displays Devices for Medical Imaging. Fahad Zafar, Dr. Yaacov Yesha, Dr. Aldo Badano IAB Meeting Research Report June 2013 12/18/12 CHMPR IAB 2013 1
-
Upload
shayna-sherard -
Category
Documents
-
view
215 -
download
0
Transcript of Ideal Observer Approach for Assessment of Image Quality on Stereo Displays Devices for Medical...
CHMPR IAB 2013 1
Ideal Observer Approach for Assessment of Image Quality on Stereo Displays Devices for Medical
Imaging.
Fahad Zafar, Dr. Yaacov Yesha, Dr. Aldo Badano
IAB Meeting Research ReportJune 2013
12/18/12
CHMPR IAB 2013 2
Introduction
• Evaluation of Stereo Displays for medical imaging diagnosis.
• Image quality assessment of stereo display devices for medical imaging applications.
• Investigate– Fundamental limitations and benefits.– Parameters crosstalk, display noise, luminance, stereo technology.
12/18/12
CHMPR IAB 2012 3
Introduction
• The goal of this research is to assess image quality on stereo display devices for medical imaging applications which have great potential since they provide depth perception to the observer when looking at medical datasets.
12/18/12
CHMPR IAB 2013 4
Objectives
12/18/12
◦Mathematical Model for Image Quality Assessment on Stereo Displays◦Propose a Stereo Model Observer.Computationally investigate parameters
Crosstalk, stereo angle, display noise.
◦Effect on performance related to◦3D display device.◦3D display technology.
CHMPR IAB 2012 5
Objectives
• Human observer studies are slow and costly, we propose to use a computational model that can conduct signal detection tasks. Using this model we can explore multiple parameters for the display with many different settings which is very hard to do with human observers.
• Crosstalk: Luminance intended for one eye, leaking into the other.
• Stereo angle: The angle between the two stereo cameras and the center of the focus point.
• Display noise: Unintentional output luminance disturbance present in every display.
12/18/12
CHMPR IAB 2013 6
Success Criteria
• Present an Ideal Linear Stereo Observer model that can be used to assess image quality for 3D displays using computational approach.
• Show applications of the stereo model to compare multiple 3D display devices with different technologies (active vs. passive)
• Present another use case for the stereo observer that does not directly relate to medical imaging.
12/18/12
CHMPR IAB 2012 7
Success Criteria
• Show that the stereo observer we have formulated clearly assess image quality for different 3D display devices using a computational approach.
• Compare different types of 3D display technologies using our model. We would also like to use the stereo observer for other assessment tasks not related to medical image.
• One example is compression of a stereo stream and understanding the tradeoffs between quality and compression ratio.
12/18/12
CHMPR IAB 2013 8
Existing Solutions
12/18/12
• Human Observer Studieso Content
o Use of entertainment 3D content and no connection to medical imaging tasks.o ScopeoDifferent scope (target the observer, not displays)o Limited in scope.
oResultsoLengthy and expensive.oResults are often subjective and inconsistent.
oToo many variations in 3D display technologies and multiple parameters to explore (MIP vs. Absorption model).
CHMPR IAB 2012 9
Existing Solutions
• Current work does not associate directly with medical imaging applications. Human observer studies conducted mostly user visual datasets from video games and the movie industry.
• The task performed by human observers in existing published research is focused on the user not the 3D display. We believe there are just too many parameters to explore and thus conducting a human observer study is infeasible.
12/18/12
CHMPR IAB 2013 10
Development Approach: Ideal Linear Observer
12/18/12
CHMPR IAB 2012 11
Development Approach: Ideal Linear Stereo Observer
• gl, gr: pixel data in the left and right image respectively, after visualization.
• Sv: Physical Display emulation operator.• Ll, Lr: Luminance reaching the left and the right
eye respectively.• g: discrete dataset. (eg. Simulated White Noise
generated through a Random Number Generator.)
•
12/18/12
CHMPR IAB 2012 12
Development Approach
• Mathematical formulation of the observer. We use Signal to Noise ratio to assess performance of the observer. Essentially, the data, which is a simulated medical imaging dataset is visualized using a rendering algorithm (Sg) and then post processing is applied to emulate the display (Sv).
• SNR: Signal to noise ratio.• KLg-1: Inverse covariance matrix of the background.
(background could be White noise or Lumpy (which is a bunch of Gaussians super imposed on each other) )
• S: signal image (3D Gaussian blob)
12/18/12
CHMPR IAB 2012 13
Development Approach
• Sg: The visualization operator. Could be Maximum Intensity Projection or Absorption Model.
• Alpha: Transparency.
12/18/12
CHMPR IAB 2013 14
Development Approach
12/18/12
2D/3D Observer
Previous Results
Sg
Sg
Sg
2D Observer
3D Observer
Stereo Observer
?
Orthographic, Voxel/Pixel = 1, Opacity = 1, Vision = Absorption or MIP
Orthographic, Voxel/Pixel = 1, Opacity = 1, Vision = MIP
Orthographic, Voxel/Pixel = 1, Opacity = X, Vision = Absorption
SgOur Results
No visualizati-on
CHMPR IAB 2012 15
Development Approach
• With our approach you can emulate all sorts of displays including 2D, stereoscopic 3D, multi-view stereoscopic 3D etc. Our computational model is a generalized model that can be tweaked to any form of observer by simply changing a few parameters.
• You can use MIP or absorption for a single trial.
12/18/12
CHMPR IAB 2012 16
Development Approach
• Vision = the vision model used. It could be either Absorption or MIP. We have only used Absorption so far in our trials.
• Voxel/Pixel ratio: is set to one, so one voxel in the 3D dataset covers 1 exact pixel on the screen when viewing it head on.
12/18/12
CHMPR IAB 2012 17
Development Approach
• 2D ideal linear is from previous work, but we have generalized it using our approach and conducted experiments using it to compare with previous work. We compare and confirm that they are both equal. But, with our model we can swap parameters to do more observers like Ideal Linear 3D and stereo. So our approach is more robust
• Opacity = X means opacity is varied while everything else is kept constant. Opacity is the opposite of transparency.
12/18/12
CHMPR IAB 2012 18
Development Approach
• We have results for 2D, 3D and stereo. 2D and 3D are computed only to compare and establish our approach as equivalent to previous work but no one has done stereo. Our model can do all these three and more (such as multi-view stereo observer represented by the ? sign)
12/18/12
CHMPR IAB 2012 19
Development Approach
• -- MIP - both the meaning of the acronym and the definition
• Maximum Intensity Projection is a rendering model where we shoot rays from each pixel into the voxelized dataset. The ray traverses through all the voxels with in line of sight and the voxel with the highest intensity (or voxel value) with in that line of sight is selected as the final color of that pixel.
12/18/12
CHMPR IAB 2012 20
Development Approach
• Absorption model is an alternative rendering model, we shoot rays from the pixels and interpolate between the voxels as the line of sight intersects with them. While interpolating between voxel values, there is a weighting parameter applied to each voxel. That parameter is called the "alpha" or transparency. So the equation looks something like this ..
• color-voxel-1 * alpha1 + (color-voxel-2 * (1- alpha1))
12/18/12
CHMPR IAB 2012 21
Development Approach
• And you do this for the first 2 voxels that the ray hits, then you traverse the ray and if it hits the third voxel, you use the previously generated color form voxel-1 and voxel-2 and interpolate with voxel-3 and so on. Sometimes you can fix the alpha for every slice of the volume dataset and every voxel has a fixed alpha. other times there are alpha maps that are generated with the dataset to accentuate some feature.. for instance one alpha map of a cube would highlight the lungs while another would highlight the heart etc.. but we dont do this. we use a constant for all voxel.
12/18/12
CHMPR IAB 2012 22
Development Approach
• Transparency is alpha
• Opacity is the opposite of alpha.. .opacity = 1- transparency
12/18/12
Comparing 2D Ideal methodologies: Our method vs. Theoretical
23
CHMPR IAB 2012 24
Comparing 2D Ideal methodologies: Our method vs. Theoretical
• So in this result, we perform the Ideal Linear 2D observer on the White Noise data. Our method outputs the exact same results as the theory suggests.
• For white noise the outputs can be theoretically calculated so that how we know our SNR outputs are correct.
12/18/12
CHMPR IAB 2012 25
Comparing 2D Ideal methodologies: Our method vs. Theoretical
• The x-axis is the signal amplitude, meaning the amplitude of the Gaussian blob which the observer was viewing with in the image.
• The y-axis is the final SNR.
• These results do not emulate any display device and no crosstalk was simulated for these results. Adding crosstalk is our next step
12/18/12
Our method quantifies stereo perception at different stereo angles
CHMPR IAB 2012 27
Our method quantifies stereo perception at different stereo angles
• Now we use our model to calculate SNR at different stereo angles. No other method can potentially do that since they don’t visualize their dataset, hence cannot generate projections at different angles.
• We quantify the different gain in observer performance when the stereo angle is changes. Beta = stereo angle in this case.
12/18/12
CHMPR IAB 2012 28
Our method quantifies stereo perception at different stereo angles
• The x-axis is the signal amplitude, meaning the amplitude of the Gaussian blob which the observer was viewing with in the image.
• The y-axis is the final SNR.
• These results do not emulate any display device and no crosstalk was simulated for these results
12/18/12
CHMPR IAB 2013 29
Evaluation
12/18/12
200 sec
900 sec
Resolution: 32x32Number of Images: 240,000Dataset size: 1 GB Covariance size: 2 MB
10 sec
Total Simulation time = ~19 minutes
•Successfully verified our model•Successfully implemented the simulation pipeline
CHMPR IAB 2012 30
Evaluation
• Currently, we have completed the implementation of the pipeline and it takes about 19 minutes for a complete trial which includes dataset generation, visualization (creating stereo pairs), calculating data statistics and then calculating the SNR
• We compared 2D observer SNR for White noise with 2D observer SNR from previous works SNR and they were equal, within a range of <1% error.
12/18/12
CHMPR IAB 2012 31
Evaluation
• The equations applied are in the previous silde. 1. generate a dataset for some background (White noise or lumpy) 2. Then visualize using Absorption model volume rendering 3. Then create stereo pairs (about 240,000 image pairs). 4. Then calculate covariance for the images.
12/18/12
CHMPR IAB 2012 32
Evaluation
• 5. Then calculate inverse of the covariance matrix and multiply 3 things to get snr (signal image transpose * inverse covariance of background * signal image)
• .6 proof of this methodology is present in the book “Foundations of Image Science” by: Harrison Barrett and Kyle Myers.
12/18/12
CHMPR IAB 2013 33
Evaluation
12/18/12
•Extend the model• Measure physical display crosstalk• Add additional stage to the pipeline and emulate the display• Conduct comparative analysis.
WorkflowExperimental setup
CHMPR IAB 2012 34
Evaluation
• Next, we are working on exploring the crosstalk parameter to see how it affects observer performance. We measure the crosstalk from actual 3D display device, experimental setup is presented in the slide. Then we use that data as a post processing function with in the model to emulate the display.
• Crosstalk: Luminance intended for one eye leaking into the other.
12/18/12
CHMPR IAB 2012 35
Evaluation
• We measure the luminance response for both the left and the right displays independently through the 3D glasses. Then we measure the crosstalk through the glasses independently and combine all this data to create a luminance response for the display. This luminance response is used as a 2D lookup table with in the code and considered to be its emulation.
• SNR calculated for the stereo observer for this 3D display device vs. SNR calculated without any display (which is the ideal since there is no display emulation). This comparision will show the quantitative decrease in perception because the device contains crosstalk and crosstalk negatively affects signal perception.
12/18/12
CHMPR IAB 2013 36
Status
• Have a working pipeline that simulates display devices.
• Have collected luminance data from one passive 3D display device.
• Next step – Emulate the device and analyze performance.– Get luminance measurements of active 3D displays
and compare.
12/18/12
CHMPR IAB 2012 37
Status
• Have a working pipeline. We are currently trying to emulate an active and passive stereo device and compare their performance. We would like to measure luminance from many displays and then add do comparative analysis using our model.
• We measure the luminance response for both the left and the right displays independently through the 3D glasses. Then we measure the crosstalk through the glasses independently and combine all this data to create a luminance response for the display. This luminance response is used as a 2D lookup table with in the code and considered to be its emulation.
12/18/12
CHMPR IAB 2012 38
Thank You!
12/18/12
