Presented By: Anthony Karloff
-
Upload
laura-myers -
Category
Documents
-
view
23 -
download
0
description
Transcript of Presented By: Anthony Karloff
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Presented By:Presented By:Anthony KarloffAnthony Karloff
Demosaicing with ImprovedDemosaicing with ImprovedEdge Direction DetectionEdge Direction Detection
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
OverviewOverview
• Demosaicing BackgroundDemosaicing Background• Basics and ChallengesBasics and Challenges• Advanced Methods (State of the Art)Advanced Methods (State of the Art)• Color Channel ReconstructionColor Channel Reconstruction• ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
• Incomplete color planes from CCD sensors.Incomplete color planes from CCD sensors.
Why Image Reconstruction?Why Image Reconstruction?
Demosaicing BackgroundDemosaicing Background
Color Filter Array (CFA) on image sensor.Color Filter Array (CFA) on image sensor.
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Basics and ChallengesBasics and Challenges
• Must Interpolate color planes to re-create Must Interpolate color planes to re-create image.image.
Red Channel InterpolationRed Channel Interpolation
Color Plane InterpolationColor Plane Interpolation
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Basics and ChallengesBasics and Challenges
• Pixel AveragingPixel Averaging- lose image resolution- lose image resolution
Color Plane Interpolation MethodsColor Plane Interpolation Methods
• Nearest NeighborNearest Neighbor- Poorest Quality- Poorest Quality
• Bilinear/SplineBilinear/Spline- Color artifacts at edges- Color artifacts at edges
G2 B1
R1 G1P1
G2 B1
R1 G1 P1
G4 B2
R1 G3
G5
R2
G1 B1 G2
P1
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Basics and ChallengesBasics and Challenges
• Problem with most simple interpolation Problem with most simple interpolation algorithms is the presence of color artifacts.algorithms is the presence of color artifacts.
Color ArtifactsColor Artifacts
Original ImageOriginal Image Bilinear Interpolation of Bayer ImageBilinear Interpolation of Bayer Image
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Basics and ChallengesBasics and Challenges
• Due to interpolation across edges.Due to interpolation across edges.
Color ArtifactsColor Artifacts
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25
BilinearInterpolation
Artifacts
Dark to Light Edge over Bayer Pattern
Resulting Edge after Interpolation
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
• Group pixels of similar objectsGroup pixels of similar objects
Advanced Techniques for Color Advanced Techniques for Color Plane InterpolationPlane Interpolation
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
• Interpolate along edges (not across)Interpolate along edges (not across)
• Interpolate green color plane firstInterpolate green color plane first
• Interpolate image more than one iteration Interpolate image more than one iteration (refinement)(refinement)
• Use color plane gradientsUse color plane gradientsChannel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
• Better estimation of color plane behavior.Better estimation of color plane behavior.
Using Gradients for Image Using Gradients for Image ReconstructionReconstruction
Bayer Pattern for GreenBayer Pattern for GreenCentered PixelCentered Pixel
P1 P2P5
P3
P7 P9P4 P6
P8
2
64)5(
PPPDx
2
82)5(
PPPDy
22
73)5(
PPPDxd
22
91)5(
PPPDyd
• Notice that the differences are always from Notice that the differences are always from the same color plane.the same color plane.
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
1. GRADIENTS
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods • Associates colors of the same object.Associates colors of the same object.
Kimmels ‘E’ Function for Pixel Kimmels ‘E’ Function for Pixel GroupingGrouping
P1 P2P5
P3
P7 P9P4 P6
P8
Ie. If P5 and Pi are part of the same Ie. If P5 and Pi are part of the same object, E will be close to unity.object, E will be close to unity.
22 )()5(1
1)5(
PiDiPDiPEi
• There are eight Ei values for each pixel. One There are eight Ei values for each pixel. One for each neighbor.for each neighbor.
Bayer Pattern for GreenBayer Pattern for GreenCentered PixelCentered Pixel
1. GRADIENTS
2. GROUPING
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
• Interpolation is best performed in the same Interpolation is best performed in the same direction as an edge.direction as an edge.
Using Edge Detection (Wide)Using Edge Detection (Wide)
Bayer Pattern for RedBayer Pattern for RedCentered PixelCentered Pixel
GGG PPPH 1412)13(
GGG PPPV 188)13(
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25RRRR PPPPH 1321511)13(
RRRR PPPPV 132233)13(
Edge detection of radius 3Edge detection of radius 3
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
• Uses narrow edge detection to improve Uses narrow edge detection to improve edges by looking between color planes.edges by looking between color planes.
Narrow Edge DetectionNarrow Edge Detection
Bayer Pattern for RedBayer Pattern for RedCentered PixelCentered Pixel
RGGGR PPPPH 1321412)13(
RGGGR PPPPV 13282)13(
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25
Edge detection of radius 2Edge detection of radius 2
BBBGBBGB PPPPPPPH 18219178297)13( 21
BBBGBBGB PPPPPPPV 142199122177)13( 21
4. EDGE DETECT 2 pxls
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
4. EDGE DETECT 2 pxls
• Compare average color differences in a 5x5 Compare average color differences in a 5x5 region to determine whether the Red or Blue region to determine whether the Red or Blue channel is more closely related to the green.channel is more closely related to the green.
Local Inter-Channel CorrelationLocal Inter-Channel Correlation
5555 xxGR RGC
5555 xxGB BGC
5. COLOR CORRELATION
Bayer Pattern for RedBayer Pattern for RedCentered PixelCentered Pixel
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Advanced MethodsAdvanced Methods
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
4. EDGE DETECT 2 pxls
5. COLOR CORRELATION
• Now we can complete the edge detectorNow we can complete the edge detector
GB
GRGR H
HHHH
otherwise
CCif GBGR
GB
GRGR V
VVVV
otherwise
CCif GBGR
Improved Edge DetectorImproved Edge Detector
34 5
6. IMPROVED EDGE DETECTION
Channel Channel ReconstructionReconstruction
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Channel ReconstructionChannel Reconstruction
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Channel Channel ReconstructionReconstruction
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
4. EDGE DETECT 2 pxls
5. COLOR CORRELATION
6. IMPROVED EDGE DETECTION
• Reconstruct the green channel using edge Reconstruct the green channel using edge detectors and the approximated red and blue.detectors and the approximated red and blue.
Channel Reconstruction OverviewChannel Reconstruction Overview
• Reconstruct the red and blue channels using Reconstruct the red and blue channels using the complete green channel.the complete green channel.
• For each pixel we now have:For each pixel we now have: )(PiEi H V
ConclusionConclusion
• Approximate the red and blue channels Approximate the red and blue channels using Bilinear Interpolation.using Bilinear Interpolation.
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Channel ReconstructionChannel Reconstruction
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Channel Channel ReconstructionReconstruction
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
4. EDGE DETECT 2 pxls
5. COLOR CORRELATION
6. IMPROVED EDGE DETECTION
• For each Green pixel on a red center…For each Green pixel on a red center…
Green Channel ReconstructionGreen Channel Reconstruction
Bayer Pattern for RedBayer Pattern for RedCentered PixelCentered Pixel
P9P7
P19P17
P8
P14P12 P13
P18
otherwiseE
PiPiE
VHifEE
PPEPPE
VHifEE
PPEPPE
PP
iPi
iRGPi
PP
RGPRGP
PP
RGPRGP
RG
18,14,12,8
18,14,12,8'
188
'18'8
1412
'14'12
)(
)1818()88(
)1414()1212(
1313
• A similar approach is taken to the finding the A similar approach is taken to the finding the green value at a blue centered pixelgreen value at a blue centered pixel
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
Channel ReconstructionChannel Reconstruction
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Channel Channel ReconstructionReconstruction
1. GRADIENTS
2. GROUPING
3. EDGE DETECT 3 pxls
4. EDGE DETECT 2 pxls
5. COLOR CORRELATION
6. IMPROVED EDGE DETECTION
• Blue and red channels are then completed Blue and red channels are then completed using the full green channel.using the full green channel.
Blue and Red Channel ReconstructionBlue and Red Channel Reconstruction
Bayer Pattern for RedBayer Pattern for RedCentered PixelCentered Pixel
P9P6
P11
P7
P19P17
P8
P14
P16
P12 P13
P18
P15
P10
P1 P4P2 P3 P5
P20
P21 P24P22 P23 P25
191797
1919171799771313
EEEE
KEKEKEKEPP GB
Where…Where… GB PiPiKi
Similar approach is taken for completingSimilar approach is taken for completingRed channel.Red channel.
ConclusionConclusion
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
ConclusionConclusion
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Channel Channel ReconstructionReconstruction
ConclusionConclusion
• Highly computational and hence slow.Highly computational and hence slow.
ConclusionConclusion
• Not suitable for real-time applications.Not suitable for real-time applications.
• Drastically reduces color artifacts.Drastically reduces color artifacts.
• Improved Edge Quality.Improved Edge Quality.
Thank YouThank You
RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR
ReferencesReferences
BackgroundBackground
BasicsBasics
Advanced Advanced MethodsMethods
Channel Channel ReconstructionReconstruction
ConclusionConclusion
[1] D. Darian Muresan, S. Luke, and T. W. Parks, “Reconstruction of Color Images From CCD Arrays,” Cornell University, Ithaca NY. 1485, [2] R. Kimmel, “Demosaicing: Image Reconstruction from Color CCD Samples” IEEE Transl. J. Image Processing, vol. 8, Sept. 1999.
[3] Xiaomeng Wang, Weisi Lin, Ping Xue, “Demosaicing with Improved Edge Direction Detection” IEEE Transl. J. Image Processing, 2005.