Modelling, calibration and rendition of colour logarithmic CMOS image sensorsDileepan Joseph and Steve CollinsDepartment of Engineering ScienceUniversity of Oxford

IMTC 2002 2May 21-23

Outline

Logarithmic CMOS image sensors Modelling sensor response Image sensor calibration

Fixed pattern noise Sensation of colour

Rendition of images CIE Lab (perceptual error) IEC sRGB (standard display)

Summary and future work

IMTC 2002 3May 21-23

Logarithmic CMOS image sensors

CMOS displacing CCD because of integration of signal processing and economies of scale

Logarithmic sensors offer high dynamic range and high frame rate

Linear sensors offer low fixed pattern noise and good colour rendition

Example images taken from IMS Chips website

Linear CCDsensor

Logarithmicsensor

Logarithmicsensor

IMTC 2002 4May 21-23

Modelling sensor response

Since Ik = ∫ fk(λ) s(λ) dλ For photocurrent Ik, spectral response

fk(λ) and light stimulus s(λ) at a pixel, where k = R, G or B

And fk(λ) = gL(λ) gk(λ) gP(λ) For spectral responses of lens gL(λ),

colour filter gk(λ) and photodiode gP(λ) Approximating a linear combination of

three CIE XYZ basis functions Then Ik = dk • x

For mask coefficients dk and tricolour vector x, i.e. s(λ) in CIE XYZ space

Ideally, y = a + b ln (c + Ik) + ε For digital response y of pixel with

offset a, gain b, bias c and error ε Pixel-to-pixel variation of a, b or c

causes fixed pattern noise (FPN)

IMTC 2002 5May 21-23

FPN calibration

Three types of FPN of interest: Offset variation Offset and gain variation Offset, gain and bias variation

Partition pixels by colour filter to permit FPN calibration of three monochromatic sensors

Take images of uniform stimuli under different illuminances

Calibrate each pixel’s response to average response of all pixels by least squares estimation of varying model parameters

Fuga 15RGB sensor exhibits offset, gain and bias variation

IMTC 2002 6May 21-23

Colour calibration

Take and segment images of a standard chart, having patches of known CIE XYZ colour, under different illuminances

Calibrate pixel responses to colour by estimating non-varying model parameters (e.g. mask dk), using estimates of varying parameters

Ideal model fails for Fuga 15RGB because absolute relationship between y and Ik invalid (strong inversion component?)

Empirical model y = a + b ln (c + (α + dk • x)β) worked well, with no change to relative responses of pixels or FPN calibration

IMTC 2002 7May 21-23

Image rendition (CIE Lab)

Images of a Macbeth Colour Chart, taken by the Fuga 15RGB, were rendered into CIE Lab space with the calibrated empirical model

The perceptual error increases in dim lighting as the bias term c dominates the photocurrent Ik

Excluding the dimmest image (i.e. 5 lux), the error equals 12 over a 60 dB dynamic range for offset, gain and bias variation

Images in Digital Photographer show that conventional (linear) digital cameras have an error of 15 over a 30 dB dynamic range

IMTC 2002 8May 21-23

Image rendition (IEC sRGB)

A Fuga 15RGB image of the Macbeth Chart, taken in 11 lux of illuminance, was rendered into IEC sRGB space with the calibrated empirical model

Results for offset variation (top-left), offset and gain variation (top-right), offset, gain and bias variation (bottom-left) and true colours (bottom-right) are shown

Two types of residual deviation for the rendered patches are visible: Fixed pattern noise Colour desaturation

IMTC 2002 9May 21-23

Summary and future work

Logarithmic image sensors offer high dynamic range and frame rate Combine theories of colour linear sensors and monochromatic

logarithmic sensors to model colour logarithmic sensors Calibrate FPN, using images of uniform stimuli, by relative

estimation of model parameters that vary from pixel to pixel Calibrate colour, using images of a colour chart, by absolute

estimation of model parameters that do not vary Fuga 15RGB results expose limitations of ideal model in absolute

estimation but reveal empirical model that works well Macbeth Chart results show colour rendition with calibrated Fuga

15RGB competes with conventional digital cameras Seek to minimise bias variation, so simple FPN models suffice, and

bias magnitude, to improve colour rendition in dim lighting

IMTC 2002 10May 21-23

Acknowledgements

The authors gratefully acknowledge the support of the Natural Sciences and Engineering Research Council (Canada) and the Engineering and Physical Sciences Research Council (UK)