Introduction of the intrinsic image

Intrinsic Images

The method of Finlayson & Hordley （ 2001 ）

Two assumptions 1. the camera’s sensors are sufficiently narrow band. 2. the illuminant can be approximated by a black-body radiator.

The power spectrum of a black-body radiator The radiance of a black-body

radiator

temperature T （ Kelvin ） , wavelength , Planck’s constant

Boltzmann’s constant ,

the speed of light

Finlayson & Hordley ： many fluorescent light sources can be approximated by a black-body radiator.

Simplification of the equation

The temperature T ＜ 10000 K. The visual spectrum ranges ： 400 n

m ~ 700 nm.

Different intensities of the black-body radiator ： another constant k.

The intensity measured by the sensor

Assumption 1. the illuminant can be

approximated by a black-body radiator. 2. pixel colors are linearly related to

the data measured by the sensor.

Logarithm

The first term ： the power of the illuminant and the scene geometry but independent of the color of the illuminant and the reflectance.

The second term ： the wavelength to which the sensor responds and on the reflectance of the object.

The last term ： the color of illuminant.

Compute the differences

Let and .

As a 2-D vector. If the temperature T is a

parameter, the two equations define a line where

is a point on the line and other points on the line are reached by adding with varying amounts.

The constants only depend on the wavelength to which the sensor responds.

is independent of reflectance.

T ： 1000 K ~ 10000 K （ in steps of 20 ％） Several different surfaces （ red －＞ whit

e ） were illuminanted by a black-body radiator of different temperatures.

sensors ： The direction of the lines only depends on t

he type of sensors used in the camera.

Gamma correction

Pixel colors ： logarithm ： The influence of any gamma correctio

n does not exist for the line in log-chromaticity difference space.

Removal of the temperature T

Project the data points in a direction orthogonal to the line.

The equation is independent of the illuminant.

Color circle log-chromaticity difference coordinates

If we project the data points along the invariant direction, some information will be lost.

Mix of blue and red or cyan and yellow.

The direction in which to project the log-chromaticity difference is unique for each camera.

Drawback

＆ ： the response of the chosen channel may be very low, leading to noisy results.

Which channel should be chosen ？

Modification

Finlayson ＆ Drew （ 2001 ）： dividing by the geometric mean of the three channels to remove the dependence on the shading information G and the dependence on the intensity k.

the vector is orthogonal to the vector ,

all points are located on the 2-D plane defined by u.

Finlayson （ 2004 ） define the following coordinate system for the geometric mean chromaticity space.

： the two basis vectors Find the two vectors

The geometric mean 2-D chromaticity space

Color circle Geometric mean 2-D chromaticity space

Geometric mean chromaticities If we project along a line in

geometric mean 2-D chromaticity space, we’ll lose some information about the color of the objects.

Using the method, we can transform the input image of a calibrated camera to an intrinsic image.

Intrinsic image The intrinsic images only depend on the

reflectance of the object points. The 2-D coordinates have to be

projected onto a line that is orthogonal to the vector .

For a given camera, this vector can be found experimentally by imaging a calibration image under several different illuminants.

Uncalibrated image Finlayson （ 2004 ）： compute intrin

sic images for an uncalibrated camera from a single image

The projection has to be done onto a line inside the geometric mean 2-D chromaticity space.

Which line is the correct one ？

The projection of the data points for two different lines.

Entropy

Finlayson ： the correct orientation is the orientation where the resulting image has minimum entropy.

Let be the set of lines for which we compute the entropy.

Entropy

For each line, the invariant data points

A gray-scale image is formed from the projected data points that are transformed to the range[0, 1].

A histogram is computed for the image.

Compute the probability that the gray value occurs in the image.

Compute the entropy

The correct direction for the line will be the one where the entropy is minimal.

The invariant direction is located at an angle of 150.75°, which corresponds to a projection direction of 60.75°.

The intrinsic image that is based only on reflectance can be used for object recognition.

Can we move from an intrinsic image back to a full color image ？

As we have known, some color information is lost.

Color image

Drew （ 2003 ）： go back to a color image

The projected coordinates don’t have to be interpreted as a gray-scale image.

The coordinates along the projection line are 2-D.

Compute the corresponding 3-D coordinates in geometric mean chromaticity space by multiplying the projected coordinates by .

Drew ： adding a little illuminant in order to obtain a color image from the 1-D data points.

Compute the and project these points onto the invariant direction .

Compute the median values of this data.

An illuminanted image is obtained by moving all projected data points by along the direction .

1% of the brightest image pixels

＝ the median of the brightest 1% of the image pixels

Exponentiate to compute output colors.

Intrinsic RGB image

Rescale the intrinsic image to have the original lightness of the input image.

Segment the input image Compute edges of the segmented input ima

ge and the RGB chromaticity intrinsic image Take the threshold ： shadow edges 1. edge values ＞ a threshold in the segmented image 2. edge values ＜ another threshold in the RGB color space

Apply the first derivative to the logarithm of the input image

Use a morphological operator to binarize and thicken the shadow edges

Replace iteratively unknown derivative values on the boundary of the shadow edges by the median of known ones in the vicinity

（ e ）＝（ b ）－（ d ）

Marc Ebner

Graham D. Finlayson

Reference Marc Ebner. Color Constancy

John Wiley & Sons, England, 2007. Finlayson, G.D. and Drew, M.S. and Lu, C., "Intrinsic

Images by Entropy Minimisation", In 8th European Conference on Computer Vision III, pp. 582-595, 2004.

G.D. Finlayson, S.D. Hordley, and M.S. Drew. Removing shadows from images. In ECCV 2002: European Conference on Computer Vision, pages 4:823–836, 2002. Lecture Notes in Computer Science Vol. 2353, http://www.cs.sfu.ca/∼mark/ftp/Eccv02/shadowless.pdf.