On the perception of Bandlimited Phase Distortion in Natural scenes
description
Transcript of On the perception of Bandlimited Phase Distortion in Natural scenes
1
On the perception of Bandlimited Phase Distortion in Natural scenes
Kedar Vilankar, Logesh Vasu and Damon ChandlerComputational Perception and Image Quality
LaboratorySchool of Electrical and Computer Engineering
Oklahoma State University
Importance of Phase Oppenheim and Lim (1981) demonstrated the
importance of phase in signals.
Phase spectrum contributes more to the image’s visual appearance.
Magnitude Spectrum
Phase Spectrum
2 of 51
Cells Compute Local Information Primates V1 is dominated by complex cells. V1 complex cells are insensitive to phase V1 complex cells encode the magnitude
information only. V1 complex cells are localized to receptive
fields.
3 of 51
Local Magnitude is all we need Phase information is implicitly encoded in
local magnitude.
Morrone and Burr demonstrated computation of location of lines and edges (Phase congruency) from local magnitude (Complex cell response).
Also, other researchers have shown only local magnitude is required for scene recognition and categorization.
4 of 51
Local Magnitude is all we need Morgan et. al demonstrated local phase is of
lesser importance than local magnitude.
HVS uses local magnitude information to determine global (image-wide) phase information.
Magnitude Spectrum
Phase Spectrum
5 of 51
Does HVS use Local Phase ? Signal processing perspective, local phase is
important.
If HVS uses only local magnitude, then we should not see any impact of distortion in local phase.
Original ImageLocal Phase Distorted Image
6 of 51
Does HVS use Local Phase ? Complex cells compute local magnitude by
combining the responses of two simple cells.
May also exist a visual mechanism to compute local phase using simple cells.
7 of 51
Does HVS use Local Phase ? Complex cells compute local magnitude by
combining the responses of two simple cells.
May also exist a visual mechanism to compute local phase using simple cells.
If HVS has mechanism to compute local phase, then do we infer global phase from
1. only local magnitude or 2. both local magnitude and local phase
8 of 51
Summary Global phase is most important for image
appearance. (Oppenheim & Lim)
Local magnitude can implicitly encode global phase. (Morrone et al. and Shams et al.)
Local phase is of lesser importance for image appearance
However, local phase distortion has substantial impact on image quality.
9 of 51
Outline Experiment (Local Phase Contribution)
Results (Interesting and surprising)
Discussion (Open questions. We need help)
Algorithm (Local magnitude and phase Distortion Rater)
Conclusion (Our belief)10 of 51
Experiment Measure the relative contribution of local
magnitude and local phase towards image appearance.
Stimuli were created by forming hybrid in complex wavelet subbands.
Each subbands local magnitude and local phase was taken from 2- 4 original images.
11 of 51
Experiment
Frequency (cyc/deg)Low Frequency
HighFrequency
LocalMagnitude
LocalPhase
Original Images12 of 51
Experiment
Frequency (cyc/deg)Low Frequency
HighFrequency
LocalMagnitude
LocalPhase
Original Images13 of 51
Experiment Based on permutations to make hybrid images 14
combination were created.
For each combination 12 stimuli were created.
Five subjects were asked to rate how much each original image contribute to the appearance of the stimulus.
Viewing distance : 60cm.
Five choices : 5%, 10%, 25%, 50% and 75%14 of 51
Result : Combination 1
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Original Images15 of 51
Result : Combination 1
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
16 of 51
Result : Combination 1
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
59%
27%8%
6%
17 of 51
18
Result : Combination 1
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
59%
27%8%
6%
When across frequencies, local magnitude and phase have non cooperative information, then HVS relies mostly on high frequency local magnitude information.
Result : Combination 3
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Original Images19 of 51
Result : Combination 3
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
20 of 51
Result : Combination 3
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
39%
55%
6%
21 of 51
22
Result : Combination 3
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
39%
55%
6%
When local phase across entire frequency channels cooperate, then local phase information dominates local magnitude information.
Result : Combination 12
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Original Images23 of 51
Result : Combination 12
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
24 of 51
Result : Combination 12
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
48%
52%
25 of 51
Result : Combination 12
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
48%
52%
26 of 51
Local phase and local magnitude have equal importance for the appearance of an image
Result : Combination 13
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Original Images27 of 51
Result : Combination 13
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
28 of 51
Result : Combination 13
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
24%
76%
29 of 51
Result : Combination 13
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
24%
76%
30 of 51
High frequency information is more important than low frequency for image appearance.
Discussion Do we infer global phase from local phase?
31 of 51
Result : Combination 12
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
48%
52%
32 of 51
Discussion Do we infer global phase from local phase? Is there visual summation of local phase
across frequency channels?
33 of 51
Result : Combination 3
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
39%
55%
6%
34 of 51
Discussion Do we infer global phase from local phase? Is there visual summation of local phase
across frequency channels? Why HVS needs local phase information?
35 of 51
Discussion Do we infer global phase from local phase? Is there visual summation of local phase
across frequency channels? Why HVS needs local phase information? How local phase is computed in HVS?
36 of 51
Discussion Do we infer global phase from local phase? Is there visual summation of local phase
across frequency channels? Why HVS needs local phase information? How local phase is computed in HVS? What is the neural basis for this computation?
37 of 51
Discussion What is more important low frequency or high
frequency?
38 of 51
Result : Combination 13
Original Images
Low Frequency
Frequency (cyc/deg)HighFrequency
LocalMagnitude
LocalPhase
Stimulus
24%
76%
39 of 51
Discussion What is more important low frequency or high
frequency? Previous research:
Low and high convey independent information about image structure
For categorization task1. Between class – Low frequency2. Within class – High frequency
Information content in low and high frequency. Is this task dependent?
40 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Algorithm was developed to rate the quality of
local phase distorted images using experimental results.
Algorithm computes local phase as well as local magnitude distortions in an image.
41 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Local Magnitude distortion
1. Decompose original and distorted images in five scale and ten orientation log-Gabor subbands.
2. For each scalea. Compute local energy maps of the original and
distorted images.b. Compute local MSE map between local energy maps of
the original and distorted images. Use block size of 16 × 16 for local MSE.
c. Collapse local MSE map via the L2 − norm into a single scalar value.
d. Compute correlation between local energy maps of original and distorted images.
e. Compute local magnitude distortion Si(where, i is the index for current scale) by multiplying scalar values obtained in step (c) and (d).42 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater
3. Using Equation combine the local magnitude distortion obtained for each scale in step (e) to compute local magnitude distortion rating in the distorted image.
= () + () + () + () + ()
1, 2, 3, 4, 5 are power coefficients with values of 4, 4, 2, 1.5 and 0.143 respectively
43 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Local phase distortion
1. Decompose original and distorted images in four levels and four orientation complex wavelet subbands.
2. For each level of the complex wavelet subband
a. Extract local phase information of the original and distorted image.
b. Compute local phase distortion Ei (where, i is the index for current level) by computing MSE between local phase of the original and distorted image obtained in step (a).
44 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater
3. Using Equation combine the local Phase distortion obtained for each scale in step (b) to compute local phase distortion rating in the distorted image.
= () + () + () + ()
1, 2, 3, 4are power coefficients with values of 2.1, 2.4, 2.3, and 2.2 respectively
45 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Final quality rating of distorted image
α = 0.6
46 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Database of 48 phase distorted images. Five subjects rated the distorted images. Performance of LMPD compared with other
image quality assessment algorithms.
47 of 51
Algorithm LMPDLocal Magnitude and Phase Distortion Rater Database of 48 phase distorted images. Five subjects rated the distorted images. Performance of LMPD compared with other
image quality assessment algorithms.
48 of 51
PSNR SSIM CWSSIM NQM VIF MAD LMPD
Correlation 0.387 0.531 0.458 0.424 0.580 0.281 0.708
Conclusion Local magnitude is most important
information for image appearance. Local phase also has a substantial,
sometimes dominating contribution. Local phase distortion of the images also
has a substantial effect on image quality.
49 of 51
Conclusion Local magnitude is most important
information for image appearance. Local phase also has a substantial,
sometimes dominating contribution. Local phase distortion of the images also
has a substantial effect on image quality.
We believe that an explicit mechanism does exist in visual cortex for the computation of local phase information.50 of 51
Thank You Questions?
51 of 51
For more stimuli examples please visit
http://vision.okstate.edu/localphase/