Image Formation and Capture Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P....
-
Upload
jason-shaw -
Category
Documents
-
view
213 -
download
0
Transcript of Image Formation and Capture Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P....
Image Formation and CaptureImage Formation and Capture
Acknowledgment:Acknowledgment: some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P. Horn, and A. Theuwissen some figures by B. Curless, E. Hecht, W.J. Smith, B.K.P. Horn, and A. Theuwissen
Image Formation and CaptureImage Formation and Capture
• DevicesDevices
• Sources of ErrorSources of Error
Real worldReal worldReal worldReal world OpticsOpticsOpticsOptics SensorSensorSensorSensor
OpticsOptics
• Pinhole cameraPinhole camera
• LensesLenses
• Focus, aperture, distortionFocus, aperture, distortion
PinholePinholeObjectObject
Pinhole cameraPinhole camera
Pinhole CameraPinhole Camera
• ““Camera obscura” – known since Camera obscura” – known since antiquityantiquity
ImageImage
Image planeImage plane
Pinhole CameraPinhole Camera
• ““Camera obscura” – known since antiquityCamera obscura” – known since antiquity
• First recording in 1826 onto a pewterFirst recording in 1826 onto a pewterplate (by Joseph Nicéphore Niepce)plate (by Joseph Nicéphore Niepce)
PinholePinholeObjectObject
ImageImage
Pinhole cameraPinhole camera
Image planeImage plane
Pinhole Camera LimitationsPinhole Camera Limitations
• Aperture too big: blurry imageAperture too big: blurry image
• Aperture too small: requires long Aperture too small: requires long exposure or high intensityexposure or high intensity
• Aperture much too small: diffraction Aperture much too small: diffraction through pinhole through pinhole blurry image blurry image
LensesLenses
• Focus a bundle of rays from a scene Focus a bundle of rays from a scene point onto a single point on the imagerpoint onto a single point on the imager
• Result: can make aperture biggerResult: can make aperture bigger
Camera AdjustmentsCamera Adjustments
• Iris?Iris?– Changes apertureChanges aperture
• Focus?Focus?– Changes Changes ddii
• Zoom?Zoom?– Changes Changes f f and sometimes and sometimes ddii
Zoom Lenses – VarifocalZoom Lenses – Varifocal
Zoom Lenses – ParfocalZoom Lenses – Parfocal
• For a given For a given ddii, “perfect” focus at only one , “perfect” focus at only one ddoo
• In practice, OK for some range of depthsIn practice, OK for some range of depths– Circle of confusionCircle of confusion smaller than a pixel smaller than a pixel
• Better depth of field with smaller aperturesBetter depth of field with smaller apertures– Better approximation to pinhole cameraBetter approximation to pinhole camera
• Also better depth of field with wide-angle Also better depth of field with wide-angle lenseslenses
Focus and Depth of FieldFocus and Depth of Field
Field of ViewField of View
• Q: What does field of view of camera Q: What does field of view of camera depend on?depend on?– Focal length of lensFocal length of lens
– Size of imagerSize of imager
– Object distance?Object distance?
Computing Field of ViewComputing Field of View
xoxo
didi
xixi
dodo
1/1/ddoo + 1/ + 1/ddii = 1/ = 1/ff
xxo o / / ddoo = = xxi i / / ddii
= 2 tan= 2 tan-1-1 ½ ½ xxi i
(1/(1/ff1/1/ddoo))Since typically Since typically ddoo >> >> f,f,
2 tan2 tan-1-1 ½ ½ xxi i / / ff
tan tan /2 = ½ /2 = ½ xxo o // ddoo
xxi i / / ff
ApertureAperture
• Controls amount of lightControls amount of light
• Affects depth of fieldAffects depth of field
• Affects distortion (since thin-lens Affects distortion (since thin-lens approximation is better near center of approximation is better near center of lens)lens)
ApertureAperture
• Aperture typically given as “Aperture typically given as “ff-number”-number”(also “(also “ff-stops” or just “stops”)-stops” or just “stops”)
• What is What is ff /4?/4?– Aperture is ¼ the focal lengthAperture is ¼ the focal length
SensorsSensors
• FilmFilm
• VidiconVidicon
• CCDCCD
• CMOSCMOS
VidiconVidicon
• Best-known in family ofBest-known in family of“photoconductive video cameras” “photoconductive video cameras”
• Basically television in reverseBasically television in reverse
Electron GunElectron Gun
Photoconductive PlatePhotoconductive Plate
Lens SystemLens System
+ + + ++ + + +
Scanning Electron BeamScanning Electron Beam
MOS CapacitorsMOS Capacitors
• MOS = Metal Oxide SemiconductorMOS = Metal Oxide Semiconductor
Gate (wire)Gate (wire)
SiOSiO22 (insulator) (insulator)
pp-type silicon-type silicon
MOS CapacitorsMOS Capacitors
• Voltage applied to gate repels positive Voltage applied to gate repels positive “holes” in the semiconductor“holes” in the semiconductor
+10V+10V
Depletion regionDepletion region(electron “bucket”)(electron “bucket”)
+ + + + + ++ + + + + ++ + + + + ++ + + + + +
MOS CapacitorsMOS Capacitors
• Photon striking the material createsPhoton striking the material createselectron-hole pairelectron-hole pair
+10V+10V
+ + + + + ++ + + + + +
PhotonPhoton
++
Charge TransferCharge Transfer
• Can move charge from one bucket to Can move charge from one bucket to another by manipulating voltagesanother by manipulating voltages
CMOS ImagersCMOS Imagers
• Recently, can manufacture chips that Recently, can manufacture chips that combine photosensitive elements and combine photosensitive elements and processing elementsprocessing elements
• Benefits:Benefits:– Partial readoutPartial readout
– Signal processingSignal processing
– Eliminate some supporting chips Eliminate some supporting chips low cost low cost
ColorColor
• 3-chip vs. 1-chip: quality vs. cost3-chip vs. 1-chip: quality vs. cost
Errors in Digital ImagesErrors in Digital Images
• What are some sources of error in this What are some sources of error in this image?image?
Sources of ErrorSources of Error
• Geometric (focus, distortion)Geometric (focus, distortion)
• Color (1-chip artifacts, chromatic Color (1-chip artifacts, chromatic aberration)aberration)
• Radiometric (cosine falloff, vignetting)Radiometric (cosine falloff, vignetting)
• Bright areas (flare, bloom, clamping)Bright areas (flare, bloom, clamping)
• Signal processing (gamma, Signal processing (gamma, compression)compression)
• NoiseNoise
Monochromatic AberrationsMonochromatic Aberrations
• Real lenses do not follow thin lens Real lenses do not follow thin lens approximation because surfaces are approximation because surfaces are spherical (manufacturing constraints)spherical (manufacturing constraints)
• Result: thin-lens approximation only Result: thin-lens approximation only valid iffvalid iffsin sin
Spherical AberrationSpherical Aberration
• Results in blurring of image, focus shifts Results in blurring of image, focus shifts when aperture is stopped downwhen aperture is stopped down
• Can vary with the way lenses are Can vary with the way lenses are orientedoriented
DistortionDistortion
• Pincushion or barrel Pincushion or barrel radial distortionradial distortion
• Varies with placement of apertureVaries with placement of aperture
• Varies with placement of apertureVaries with placement of aperture
DistortionDistortion
• Varies with placement of apertureVaries with placement of aperture
DistortionDistortion
• Varies with placement of apertureVaries with placement of aperture
DistortionDistortion
First-Order Radial DistortionFirst-Order Radial Distortion
• Goal: mathematical formula for distortionGoal: mathematical formula for distortion
• If small, can be approximated by “first-If small, can be approximated by “first-order” formula (like Taylor series order” formula (like Taylor series expansion):expansion):
• Higher-order models are possibleHigher-order models are possible
rr’ = ’ = rr (1 + (1 + rr22))rr = = ideal distance to center of imageideal distance to center of imagerr’ = ’ = distorted distance to center of imagedistorted distance to center of image
Chromatic AberrationChromatic Aberration
• Due to dispersion in glass (focal length Due to dispersion in glass (focal length varies with the wavelength of light)varies with the wavelength of light)
• Result: color fringesResult: color fringes
• Worst at edges of imageWorst at edges of image
• Correct by buildingCorrect by buildinglens systems withlens systems withmultiple kinds of glassmultiple kinds of glass
Correcting for AberrationsCorrecting for Aberrations
• High-qualityHigh-qualitycompound lensescompound lensesuse multipleuse multiplelens elements tolens elements to“cancel out”“cancel out”distortion anddistortion andaberrationaberration
• Often 5-10 elements, more for extreme wide Often 5-10 elements, more for extreme wide angleangle
Other Limitations of LensesOther Limitations of Lenses
• Optical vignetting: less power per unit Optical vignetting: less power per unit area transferred for light at an oblique area transferred for light at an oblique angleangle– Transferred power falls off as cosTransferred power falls off as cos44 – Result: darkening of edges of imageResult: darkening of edges of image
• Mechanical vignetting: due to aperturesMechanical vignetting: due to apertures
Other Limitations of LensesOther Limitations of Lenses
• Flare: light reflectingFlare: light reflecting(often multiple times)(often multiple times)from glass-air interfacefrom glass-air interface
– Results in ghost images or hazinessResults in ghost images or haziness
– Worse in multi-lens systemsWorse in multi-lens systems
– Ameliorated by optical coatings (thin-film Ameliorated by optical coatings (thin-film interference)interference)
BloomBloom
• Overflow of charge in CCD bucketsOverflow of charge in CCD buckets– Spills to adjacent bucketsSpills to adjacent buckets
– Streaks (usually vertical) next to bright Streaks (usually vertical) next to bright areasareas
• Some cameras have “anti-bloom” Some cameras have “anti-bloom” circuitrycircuitry
Flare and BloomFlare and Bloom
TanakaTanaka
Dynamic RangeDynamic Range
• Most common cameras have 8-bitMost common cameras have 8-bit(per color channel) dynamic range(per color channel) dynamic range– With gamma, this can translate to more With gamma, this can translate to more
than 255:1than 255:1
• Too bright: clamp to maximumToo bright: clamp to maximum
• Too dim: clamp to 0Too dim: clamp to 0
• Specialty cameras with higher dynamic Specialty cameras with higher dynamic range (usually 10-, 12-, and 16-bit)range (usually 10-, 12-, and 16-bit)
High Dynamic Range (HDR)High Dynamic Range (HDR)from Ordinary Camerasfrom Ordinary Cameras
• Take pictures of same scene with Take pictures of same scene with different shutter speedsdifferent shutter speeds
• Identify regions clamped to 0 or 255Identify regions clamped to 0 or 255
• Average other pixels, scaled by 1 / Average other pixels, scaled by 1 / shutter speedshutter speed
• Can extend dynamic range, but Can extend dynamic range, but limitations of optics and imager (noise, limitations of optics and imager (noise, flare, bloom) still applyflare, bloom) still apply
GammaGamma
• Vidicon tube naturally has signal that variesVidicon tube naturally has signal that varieswith light intensity according to a power law:with light intensity according to a power law:Signal = Signal = EE, , 1/2.5 1/2.5
• CRT (televisions) naturally obey a power law CRT (televisions) naturally obey a power law with gamma with gamma 2.5 2.5
• Result: standard for video signals hasResult: standard for video signals hasa gamma of 1/2.5a gamma of 1/2.5
• CCDs and CMOS linear, but gamma often CCDs and CMOS linear, but gamma often appliedapplied
NoiseNoise
• Thermal noise: in all electronicsThermal noise: in all electronics– Noise at all frequenciesNoise at all frequencies
– Proportional to temperatureProportional to temperature
– Special cooled cameras available for low Special cooled cameras available for low noisenoise
• Shot noise: discrete photons / electronsShot noise: discrete photons / electrons– Shows up at extremely low intensitiesShows up at extremely low intensities
– CCDs / CMOS can have high efficiency – CCDs / CMOS can have high efficiency – approaching 1 electron per photonapproaching 1 electron per photon
NoiseNoise
• 1/f noise – inversely proportional to 1/f noise – inversely proportional to frequencyfrequency– Not completely understood – shows up in Not completely understood – shows up in
semiconductorssemiconductors
– Can be dominant source of noiseCan be dominant source of noise
• All of the above apply for imager and All of the above apply for imager and amplifieramplifier
Filtering NoiseFiltering Noise
• Most common method – simple blurMost common method – simple blur– e.g., convolution with Gaussiane.g., convolution with Gaussian
• Adaptive filters to prevent bleed acrossAdaptive filters to prevent bleed across intensity edges intensity edges
• Other filters for specialized situationsOther filters for specialized situations– e.g., “despeckling” (median filters) for dead e.g., “despeckling” (median filters) for dead
pixelspixels