Slide 1

776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013 Slide 2 Last class Slide 3 Last Class World to camera coord. trans. matrix (4x4) Perspective projection matrix (3x4) Camera to pixel coord. trans. matrix (3x3) = 2D point (3x1) 3D point (4x1) Slide 4 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o aperture is not infinitely small o lens o vignetting Slide 5 No distortionPin cushionBarrel Radial Distortion o Caused by imperfect lenses o Deviations are most noticeable near the edge of the lens slide: S. Lazebnik Slide 6 Radial Distortion Browns distortion model o accounts for radial distortion o accounts for tangential distortion (distortion caused by lens placement errors) typically K 1 is used or K 1, K 2, P 1, P 2 (x u, y u ) undistorted image point as in ideal pinhole camera (x d,y d ) distorted image point of camera with radial distortion (x c,y c ) distortion center K n n-th radial distortion coefficient P n n-th tangential distortion coefficient Slide 7 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o aperture is not infinitely small o lens o vignetting, radial distortion Slide 8 Depth of Field http://www.cambridgeincolour.com/tutorials/depth-of-field.htm Slide by A. Efros Slide 9 How can we control the depth of field? Changing the aperture size affects depth of field o A smaller aperture increases the range in which the object is approximately in focus o But small aperture reduces amount of light need to increase exposure Slide by A. Efros Slide 10 F Number of the Camera f number (f-stop) ratio of focal length to aperture Slide 11 Varying the aperture Large aperture = small DOFSmall aperture = large DOF Slide by A. Efros Slide 12 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o aperture is not infinitely small o lens o vignetting, radial distortion o depth of field Slide 13 Field of View Slide by A. Efros What does FOV depend on? Slide 14 f Field of View Smaller FOV = larger Focal Length Slide by A. Efros f FOV depends on focal length and size of the aperture Slide 15 Field of View / Focal Length Large FOV, small f Camera close to car Small FOV, large f Camera far from the car Sources: A. Efros, F. Durand Slide 16 Same effect for faces standard wide-angletelephoto Source: F. Durand Slide 17 The dolly zoom Continuously adjusting the focal length while the camera moves away from (or towards) the subject http://en.wikipedia.org/wiki/Dolly_zoom slide: S. Lazebnik Slide 18 The Dolly Zoom Slide 19 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o aperture is not infinitely small o lens o vignetting, radial distortion o depth of field o field of view Slide 20 Digital camera A digital camera replaces film with a sensor array o Each cell in the array is light-sensitive diode that converts photons to electrons o Two common types Charge Coupled Device (CCD) Complementary metal oxide semiconductor (CMOS) o http://electronics.howstuffworks.com/digital-camera.htm http://electronics.howstuffworks.com/digital-camera.htm Slide by Steve Seitz Slide 21 Color sensing in camera: Color filter array Source: Steve Seitz Estimate missing components from neighboring values (demosaicing) Why more green? Bayer grid Human Luminance Sensitivity Function Slide 22 Problem with demosaicing: color moire Slide by F. Durand Slide 23 The cause of color moire detector Fine black and white detail in image misinterpreted as color information Slide by F. Durand Slide 24 Color sensing in camera: Prism Requires three chips and precise alignment More expensive CCD(B) CCD(G) CCD(R) slide: S. Lazebnik Slide 25 Color sensing in camera: Foveon X3 Source: M. Pollefeys http://en.wikipedia.org/wiki/Foveon_X3_sensorhttp://www.foveon.com/article.php?a=67 CMOS sensor Takes advantage of the fact that red, blue and green light penetrate silicon to different depths better image quality Slide 26 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o Aperture is not infinitely small o Lens o Vignetting, radial distortion o Depth of field o Field of view o Color sensing Slide 27 Rolling Shutter Cameras Many cameras use CMOS sensors (mobile, DLSR, ) To save cost these are often rolling shutter cameras o lines are progressively exposed o line by line image reading Rolling shutter artifacts image source: Wikipedia Slide 28 Rolling Shutter regular camera (global shutter) rolling shutter camera Slide 29 Facing Real Cameras There are undesired effects in real situations o perspective distortion Camera artifacts o Aperture is not infinitely small o Lens o Vignetting, radial distortion o Depth of field o Field of view o Color sensing o Rolling shutter cameras Slide 30 Digital camera artifacts Noise low light is where you most notice noisenoise light sensitivity (ISO) / noise tradeoff stuck pixels In-camera processing oversharpening can produce haloshalos Compression JPEG artifacts, blocking Blooming charge overflowing into neighboring pixelsoverflowing Smearing o columnwise overexposue Color artifacts purple fringing from microlenses, purple fringing white balance modified from Steve Seitz Slide 31 Conventional versus light field camera slide: Marc Levoy Slide 32 Conventional versus light field camera slide: Marc Levoy Slide 33 Conventional versus light field camera slide: Marc Levoy Slide 34 Prototype camera 4000 4000 pixels 292 292 lenses = 14 14 pixels per lens Contax medium format cameraKodak 16-megapixel sensor Adaptive Optics microlens array125 square-sided microlenses slide: Marc Levoy Slide 35 Slide 36 Digitally stopping-down stopping down = summing only the central portion of each microlens f / N light field camera, with P P pixels under each microlens, can produce views as sharp as an f / (N P) conventional camera slide: Marc Levoy Slide 37 Digital refocusing refocusing = summing windows extracted from several microlenses f/N light field camera can produce views with a shallow depth of field ( f / N ) focused anywhere within the depth of field of an f / (N P) camera images: Marc Levoy Slide 38 Example of digital refocusing images: Marc Levoy Slide 39 Extending the depth of field conventional photograph, main lens at f / 22 conventional photograph, main lens at f / 4 light field, main lens at f / 4, after all-focus algorithm [Agarwala 2004] images: Marc Levoy Slide 40 Digitally moving the observer moving the observer = moving the window we extract from the microlenses images: Marc Levoy Slide 41 Example of moving the observer slide: Marc Levoy Slide 42 Moving backward and forward slide: Marc Levoy Slide 43 Historic milestones Pinhole model: Mozi (470-390 BCE), Aristotle (384-322 BCE) Principles of optics (including lenses): Alhacen (965-1039 CE) Camera obscura: Leonardo da Vinci (1452-1519), Johann Zahn (1631-1707) First photo: Joseph Nicephore Niepce (1822) Daguerrotypes (1839) Photographic film (Eastman, 1889) Cinema (Lumire Brothers, 1895) Color Photography (Lumire Brothers, 1908) Television (Baird, Farnsworth, Zworykin, 1920s) First consumer camera with CCD Sony Mavica (1981) First fully digital camera: Kodak DCS100 (1990) Niepce, La Table Servie, 1822 CCD chip Alhacens notes Slide 44 Early color photography Sergey Prokudin-Gorskii (1863-1944) Photographs of the Russian empire (1909- 1916) http://www.loc.gov/exhibits/empire/ http://en.wikipedia.org/wiki/Sergei_Mikhailovich_Prokudin-Gorskii Lantern projector Slide 45 First digitally scanned photograph 1957, 176x176 pixels http://listverse.com/history/top-10-incredible-early-firsts-in-photography/