Computational Photography and Intro. to Computer...
Transcript of Computational Photography and Intro. to Computer...
![Page 1: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/1.jpg)
CSE 391/591:
Computational Photography and
Intro. to Computer Visionwww.cs.sunysb.edu/~cse591
CSE 391/591:
Computational Photography Computational Photography andand
Intro. to Computer VisionIntro. to Computer Visionwww.cs.sunysb.edu/~cse591
Instructor: Prof. M. Alex O. VasilescuEmail: [email protected]: 631 632-8457Office: 1421
![Page 2: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/2.jpg)
![Page 3: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/3.jpg)
![Page 4: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/4.jpg)
![Page 5: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/5.jpg)
![Page 6: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/6.jpg)
![Page 7: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/7.jpg)
GRAPHICS
Traditional Computer Graphics
3D geometry
physics
Simulation
projection
![Page 8: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/8.jpg)
State of the Art
•Amazingly real•But so sterile, lifeless, futuristic (why?)
![Page 9: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/9.jpg)
The richness of our everyday world
Photo by Svetlana Lazebnik
![Page 10: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/10.jpg)
People
From “Final Fantasy”
On the Tube, London
![Page 11: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/11.jpg)
Faces / Hair
Photo by Joaquin Rosales Gomez
From “Final Fantasy”
![Page 12: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/12.jpg)
Urban Scenes
Virtual LA (SGI)
Photo of l LA
![Page 13: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/13.jpg)
Nature
River Cherwell, Oxford
![Page 14: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/14.jpg)
The Realism Spectrum
• + easy to create new worlds• + easy to manipulate
objects/viewpoint• - Very hard to look
realistic
• + instantly realistic• + easy to aquire• - very hard to manipulate
objects/viewpoint
Computer Graphics PhotographyComputationalPhotography
RealismManipulationEase of capture
![Page 15: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/15.jpg)
Automatic Photo Pop-Up
![Page 16: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/16.jpg)
Structure from Motion(Tomasi and Kanade 1992)
Video Features
3D Reconstruction
![Page 17: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/17.jpg)
Photo Collections
![Page 18: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/18.jpg)
Organize Photos
![Page 19: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/19.jpg)
Panoramic imaging
Image and video registration
Spatial warping operations
![Page 20: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/20.jpg)
Detect ground plane in video andintroduce pictures on them__________
![Page 21: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/21.jpg)
Insert new objects
Video example: http://break.com/index/ufo-lands-on-guys-desk.html
![Page 22: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/22.jpg)
Video Summary
![Page 23: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/23.jpg)
Game: Decathlete
![Page 24: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/24.jpg)
Optical-flow-based Decathlete figure motion analysis
![Page 25: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/25.jpg)
Decathlete javelin throw
![Page 26: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/26.jpg)
Decathlete javelin throw
![Page 27: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/27.jpg)
Decathlete 100m hurdles
![Page 28: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/28.jpg)
http://www.ri.cmu.edu/projects/project_271.html
![Page 29: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/29.jpg)
Black or White• Face Detection• Face Localization• Segmentation• Face Tracking• Facial features localization• Facial features tracking• Morphing
www.youtube.com/watch?v=ZI9OYMRwN1Q
![Page 30: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/30.jpg)
General Comments• Prerequisites
Linear algebra!!!Some computer graphics, vision, or image processing is
useful, but not required.
• Emphasis on programming projects!Building something from scratch (Matlab!)
![Page 31: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/31.jpg)
References• There is no required text. Various course notes and papers will be
made available. Furthermore, there is an optional textbook that you might find helpful. It will be placed on reserve at the Wean Hall library:
• Computer Vision: The Modern Approach, Forsyth and Ponce
• There is a number of other fine texts that you can use for general reference:
• Photography (9th edition), London and Upton, • Vision Science: Photons to Phenomenology, Stephen
Palmer • Digital Image Processing, 2nd edition, Gonzalez and Woods • Multiple View Geometry in Computer Vision, Hartley &
Zisserman• The Computer Image, Watt and Policarpo• Linear Algebra and its Applications, Gilbert Strang
![Page 32: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/32.jpg)
Grading
30%Final Project:
–An original implementation of a new or published idea
–A detailed empirical evaluation of an existing implementation of
one or more methods
Project proposal not longer than two pages must be submitted and approved before the end of October.
10%Class Participation
30%One take-home exams. (Take-home exams may not be discussed.)
30%Problem Sets (~6) with lab exercises in Matlab.Problem sets may be discussed, but all written work and coding must be done individually.
40%
10%
0%
50%
A B
![Page 33: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/33.jpg)
Administrative Stuff• Late Policy
Seven late days total, to be spent wisely• Cheating
Let’s not embarrass ourselves• Software
MATLAB!!!
![Page 34: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/34.jpg)
Internet Resources• Matlab:
University of Colorado Matlab Tutorials• A decent collection of Matlab tutorials, including one focusing on
image processing.
Matlab Image Processing Tutorial• A short introduction to the manipulation of images in Matlab, including
an introduction to principal components analysis via eigenfaces.
• Computer Vision: Computer Vision HomepageFace Recognition HomepageFace Detection Homepage
![Page 35: Computational Photography and Intro. to Computer …alumni.media.mit.edu/~maov/classes/comp_photo_vision08f/lect/01... · CSE 391/591: Computational Photography and Intro. to Computer](https://reader031.fdocuments.in/reader031/viewer/2022020114/5b80f77b7f8b9aad638b4cfb/html5/thumbnails/35.jpg)
Introductions
• Name, year, supervisor• Why do you want to take this class?• What are you hoping to learn?