Transcript of School of Computer Science Simon Fraser University November 2009 Sharpening from Shadows: Sensor...
- Slide 1
- School of Computer Science Simon Fraser University November
2009 Sharpening from Shadows: Sensor Transforms for Removing
Shadows using a Single Image Mark S. DrewHamid Reza Vaezi Joze
mark@cs.sfu.cahamid_reza@cs.sfu.ca
- Slide 2
- Outline Image Formation Invariant Image Formation Finding
invariant direction by calibration Finding invariant direction by
minimizing entropy Sharpening Matrix Proposed Method Optimization
problem Result 2
- Slide 3
- Shadow Removal Method To generate shadowless images, there are
two steps: 1. Finding Illuminant Invariant image (grayscale) 2.
Creating colored shadowless images using edges in main image and
invariant image. [Finlayson et al. (ECCV2002)] 3 Main
ImageShadowless ImageInvariant Image This Paper
- Slide 4
- Image Formation 4 Surface reflection Camera sensitivity Light
spectral Camera Response:
- Slide 5
- Image Formation Simplification 5 1. Camera sensors represented
as delta functions. 2. Illumination is restricted to the Planckian
locus. 3. Wiens approximation for temperature range 2500K to
10000K. We have:
- Slide 6
- Invariant Image Formation 6 Using the simplified model we form
band-ratio chromaticities r k by dividing R and B by G and taking
the logarithm: As temperature T changes, 2d-vectors r k,k=R,B, will
follow a straight line in 2d chromaticity space. For all surfaces,
the lines will be parallel, with slope (e k e G ). Surface
Dependent Camera Dependent
- Slide 7
- Invariant Image Formation 7 The invariant image, then, is
formed by projecting 2-d colors into the direction orthogonal to
the 2-vector (e k e G ). So, the problem is reduced to finding the
direction. Why we are interested? Shadow is nothing just the
surface in different illumination condition. (they should be in a
line)
- Slide 8
- Finding Invariant Direction 8 Calibrating Camera to find the
invariant direction. [Finlayson et al. (ECCV2002)] Need many images
under different illumination. Good for camera company not images
with unknown camera. HP912 Digital Still Camera: Log-chromaticities
of 24 patches; 7 patches, imaged under 9 illuminants.
- Slide 9
- Finding Invariant Direction 9 Without calibrating the camera,
can use entropy of projection to find the invariant direction
[Finlayson et al. (2004)] : Correct direction smaller entropy Wrong
direction higher entropy
- Slide 10
- Sharpening Transform Matrix 10 Convert a given set of sensor
sensitivity functions into a new set that will improve the
performance of any color- constancy algorithm that is based on an
independent adjustment of the sensor response channels. Transform
the camera sensors to made them more narrow band, which is one of
the assumption that we made. It also could apply to the image
instead of sensors.
- Slide 11
- Proposed Method 11 1. Select shadow and non shadow pixels for
the same surface material. 2. Find the sharpening matrix which
makes the chromaticities of selected pixels as linear as possible
in log-log plane = an optimization problem. 3. Transform the main
image by sharpening matrix. 4. Create illumination invariant image
by entropy-minimization method [Finlayson et al. (2004)]. 1 1 2 2 4
4 3 3.7 0.15.15.15.70.15.15.15.70
- Slide 12
- Shadow and Non Shadow Regions 12 The user selects the shadow
and non shadow region of a surface. For future work this could be
automatic. According to invariant formation in ideal condition, the
chromaticities of these point in log-log plane should be in a line.
User Defined
- Slide 13
- Optimization Problem 13 To find best sharpening matrix M 3x3 in
order to make the chromaticity as linear as possible: m 11 m 12 m
13 m 21 m 22 m 23 m 31 m 32 m 33 sum is 1 Linear combination more
than 1- Colors dont change completely
- Slide 14
- Objective Function 14 F return the minimum entropy of log
chromaticities projected to all directions. rank is meant to
encourage a non- rank-reducing matrix M. entropy Log chromaticities
Minimum entropy For this M
- Slide 15
- Sharpening Matrix 15 Sharpening Matrix Shadow and non shadow
region chromaticity Less linear More linear
- Slide 16
- Results 16 DifferenceInvariantSharpenedOriginal
- Slide 17
- Good vs. poor sharpening matrix 17 More linear.90.30 -.14
-.04.79.16.14 -.09.98.75 -.20.02.01.86.13.24.34.84 minimum Obj.
Func. =.0942 Obj. Func. =.0487
- Slide 18
- Result 18
- Slide 19
- Result 19
- Slide 20
- Conclusion 20 We proposed a new schema for generating
illumination invariant for removing shadow. The contribution of
this paper is using sharpener matrix to get better shadow removal.
The method use single images which is more practical compared to
camera calibration methods which needs bunch of images in different
illumination condition.
- Slide 21
- References 21 Sharpening Matrix: G.D. Finlayson, M.S. Drew, and
B.V. Funt. Spectral sharpening: sensor transformations for improved
color constancy. J. Opt. Soc. Am. A, 11(5):15531563, May 1994.
Illumination invariant image: G.D. Finlayson, S.D. Hordley, and
M.H. Brill. Illuminant invariance at a single pixel. In 8th Color
Imaging Conference: Color, Science, Systems and Applications.,
pages 8590, 2000. Shadow removal method: G.D. Finlayson, S.D.
Hordley, and M.S. Drew. Removing shadows from images. In ECCV 2002:
European Conference on Computer Vision, pages 4:823836, 2002.
Lecture Notes in Computer Science Vol. 2353. Entropy minimization
method: G.D. Finlayson, M.S. Drew, and C. Lu. Intrinsic images by
entropy minimization. In ECCV 2004: European Conference on Computer
Vision, pages 582595, 2004. Lecture Notes in Computer Science Vol.
3023.
- Slide 22
- Questions? Thank you. 22 Thanks! To Natural Sciences and
Engineering Research Council of Canada