Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera...
Transcript of Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera...
![Page 1: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/1.jpg)
1
Computer Vision
Thursday, August 30
![Page 2: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/2.jpg)
2
Today
• Course overview• Requirements, logistics• Image formation
![Page 3: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/3.jpg)
3
Introductions• Instructor: Prof. Kristen Grauman
grauman @ csTAY 4.118, Thurs 2-4 pm
• TA: Sudheendra Vijayanarasimhansvnaras @ csENS 31 NQ, Mon/Wed 1-2 pm
• Class page: Check for updates to schedule, assignments, etc.
http://www.cs.utexas.edu/~grauman/courses/378/main.htm
![Page 4: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/4.jpg)
4
Introductions
![Page 5: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/5.jpg)
5
Computer vision
• Automatic understanding of images and video
• Computing properties of the 3D world from visual data
• Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities.
![Page 6: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/6.jpg)
6
Why vision?
• As image sources multiply, so do applications– Relieve humans of boring, easy tasks– Enhance human abilities– Advance human-computer interaction,
visualization– Perception for robotics / autonomous agents
• Possible insights into human vision
![Page 7: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/7.jpg)
7
Some applications
VisualizationVisualization and tracking
Factory – inspection (Cognex)
Monitoring for safety (Poseidon)
License plate reading
Surveillance
![Page 8: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/8.jpg)
8
Some applications
Visual effects (the Matrix)
Medical imaging
Assistive technologyNavigation, driver safetyAutonomous robots
![Page 9: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/9.jpg)
9
Some applications
Multi-modal interfacesSituated search
Image and video databases - CBIR
Tracking, activity recognition
![Page 10: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/10.jpg)
10
Why is vision difficult?
• Ill-posed problem: real world much more complex than what we can measure in images– 3D 2D
• Impossible to literally “invert” image formation process
![Page 11: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/11.jpg)
11
Challenges: robustness
Illumination Object pose Clutter
ViewpointIntra-class appearance
Occlusions
![Page 12: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/12.jpg)
12
Challenges: context and human experience
Context cues Function Dynamics
![Page 13: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/13.jpg)
13
Challenges: complexity• Thousands to millions of pixels in an image• 3,000-30,000 human recognizable object categories• 30+ degrees of freedom in the pose of articulated
objects (humans)• Billions of images indexed by Google Image Search• 18 billion+ prints produced from digital camera images
in 2004• 295.5 million camera phones sold in 2005• About half of the cerebral cortex in primates is
devoted to processing visual information [Fellemanand van Essen 1991]
![Page 14: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/14.jpg)
14
Why is vision difficult?
• Ill-posed problem: real world much more complex than what we can measure in images– 3D 2D
• Not possible to “invert” image formation process
• Generally requires assumptions, constraints; exploitation of domain-specific knowledge
![Page 15: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/15.jpg)
15
Related disciplines
Cognitive science
Algorithms
Image processing
Artificial intelligenceGeometry,
physicsPattern
recognitionComputer
vision
![Page 16: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/16.jpg)
16
Vision and graphics
ModelImages Vision
Graphics
Inverse problems: analysis and synthesis.
![Page 17: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/17.jpg)
17
Research problems vs. application areas
• Feature detection• Contour representation• Segmentation• Stereo vision• Shape modeling• Color vision• Motion analysis• Invariants• Uncalibrated, self-
calibrating systems• Object detection• Object recognition
• Industrial inspection and quality control
• Reverse engineering• Surveillance and security• Face, gesture recognition• Road monitoring• Autonomous vehicles• Military applications• Medical image analysis• Image databases• Virtual reality
List from [Trucco & Verri 1998]
![Page 18: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/18.jpg)
18
Goals of this course
• Introduction to primary topics• Hands-on experience with algorithms• Views of vision as a research area
![Page 19: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/19.jpg)
19
Topics overview
• Image formation, cameras• Color• Features• Grouping• Multiple views• Recognition and learning• Motion and tracking
![Page 20: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/20.jpg)
20
We will not cover (extensively)
• Image processing• Human visual system• Particular machine vision systems or
applications
![Page 21: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/21.jpg)
21
Image formation
• Inverse process of vision: how does light in 3d world project to form 2d images?
![Page 22: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/22.jpg)
22
Features and filters
Transforming and describing images; textures and colors
![Page 23: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/23.jpg)
23
Grouping
[fig from Shi et al]Clustering, segmentation, fitting; what parts belong together?
![Page 24: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/24.jpg)
24
Multiple views
Hartley and Zisserman
Lowe
Tomasi and Kanade
Multi-view geometry and matching, stereo
![Page 25: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/25.jpg)
25
Recognition and learning
Shape matching, recognizing objects and categories, learning techniques
![Page 26: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/26.jpg)
26
Motion and tracking
Tomas Izo
Tracking objects, video analysis, low level motion
![Page 27: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/27.jpg)
27
![Page 28: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/28.jpg)
28
Requirements
• Biweekly (approx) problem sets– Concept questions– Implementation problems
• Two exams, midterm and final• Current events (optional)
In addition, for graduate students:• Research paper summary and review• Implementation extension
![Page 29: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/29.jpg)
29
Grading policy
Final grade breakdown:• Problem sets (50%)• Midterm quiz (15%)• Final exam (20%)• Class participation (15%)
![Page 30: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/30.jpg)
30
Due dates
• Assignments due before class starts on due date
• Lose half of possible remaining credit each day late
• Three free late days, total
![Page 31: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/31.jpg)
31
Collaboration policy
You are welcome to discuss problem sets, but all responses and code must be written individually.
Students submitting solutions found to be identical or substantially similar (due to inappropriate collaboration) risk failing the course.
![Page 32: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/32.jpg)
32
Current events (optional)
• Any vision-related piece of news; may revolve around policy, editorial, technology, new product, …
• Brief overview to the class• Must be current• No ads
• Email relevant links or information to TA
![Page 33: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/33.jpg)
33
Paper review guidelines
• Thorough summary in your own words• Main contribution• Strengths? Weaknesses? • How convincing are the experiments?
Suggestions to improve them?• Extensions?• 4 pages max
• May require reading additional references
![Page 34: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/34.jpg)
34
Miscellaneous
• Check class website• Make sure you get on class mailing list• No laptops in class please
• Feedback welcome and useful
![Page 35: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/35.jpg)
35
![Page 36: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/36.jpg)
36
Image formation
• How are objects in the world captured in an image?
![Page 37: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/37.jpg)
37
Physical parameters of image formation
• Photometric– Type, direction, intensity of light reaching sensor– Surfaces’ reflectance properties
• Optical– Sensor’s lens type– focal length, field of view, aperture
• Geometric– Type of projection– Camera pose– Perspective distortions
![Page 38: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/38.jpg)
38
Radiometry
• Images formed depend on amount of light from light sources and surface reflectance properties (See F&P Ch 4)
![Page 39: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/39.jpg)
39
Image credit: Don Deering
Light source direction
![Page 40: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/40.jpg)
40
Surface reflectance properties
[fig from Fleming, Torralba, & Adelson, 2004]
Specular
Lambertian
![Page 41: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/41.jpg)
41
Perspective projection
• Pinhole camera: simple model to approximate imaging process
Forsyth and Ponce
If we treat pinhole as a point, only one ray from any given point can enter the camera
![Page 42: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/42.jpg)
42
Camera obscura
"Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January 24, 1544, and later he used this illustration of the event in his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle
http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html
In Latin, means ‘dark room’
![Page 43: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/43.jpg)
43
Camera obscura
Jetty at Margate England, 1898.
Adapted from R. Duraiswamihttp://brightbytes.com/cosite/collection2.html
Around 1870sAn attraction in the late 19th century
![Page 44: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/44.jpg)
44
Perspective effects
• Far away objects appear smaller
Forsyth and Ponce
![Page 45: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/45.jpg)
45
Perspective effects
• Parallel lines in the scene intersect in the image
Forsyth and Ponce
![Page 46: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/46.jpg)
46
Perspective projection equations• 3d world mapped to 2d projection
Forsyth and Ponce
Camera frame
Image plane
Optical axis
Focal length
Board
![Page 47: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/47.jpg)
47
Perspective projection equations
Forsyth and Ponce
Scene point Image coordinates
Camera frame
Image plane
Optical axis
Focal length
Non-linear
![Page 48: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/48.jpg)
48
Projection properties
• Many-to-one: any points along same ray map to same point in image
• Points points• Lines lines (collinearity preserved)• Distances and angles are not preserved• Degenerate cases:
– Line through focal point projects to a point.– Plane through focal point projects to line– Plane perpendicular to image plane projects
to part of the image.
![Page 49: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/49.jpg)
49
Perspective and art
• Use of correct perspective projection indicated in 1st century B.C. frescoes
• Skill resurfaces in Renaissance: artists develop systematic methods to determine perspective projection (around 1480-1515)
Durer, 1525Raphael
![Page 50: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/50.jpg)
50
Weak perspective
• Approximation: treat magnification as constant• Assumes scene depth << average distance to
camera• Makes perspective equations linear
World points:
Image plane
![Page 51: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/51.jpg)
51
Orthographic projection
• Given camera at constant distance from scene• World points projected along rays parallel to
optical access• Limit of perspective projection as
![Page 52: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/52.jpg)
52
Planar pinhole perspective
Orthographicprojection From M. Pollefeys
![Page 53: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/53.jpg)
53
Which projection model?
• Weak perspective:– Accurate for small, distant objects; recognition– Linear projection equations - simplifies math
• Pinhole perspective:– More accurate but more complex– Structure from motion
![Page 54: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/54.jpg)
54
![Page 55: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/55.jpg)
55
Pinhole size / aperture
Smaller
Larger
Brighter, blurrier
Dimmer, blur from defraction
Dimmer, more focus
![Page 56: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/56.jpg)
56
Pinhole vs. lens
![Page 57: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/57.jpg)
57
Cameras with lenses• Gather more light, while keeping focus; make
pinhole perspective projection practical
Thin lensRays entering parallel on one side go through focus on other, and vice versa.
In ideal case – all rays from P imaged at P’.
Left focus Right focus
Focal length f
Field of view (portion of 3d space seen by camera) depends on dand f.
Lens diameter d
![Page 58: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/58.jpg)
58
• As f gets smaller, image becomes more wide angle (more world points project onto the finite image plane)
• As f gets larger, image becomes more telescopic (smaller part of the world projects onto the finite image plane)
Field of view
from R. Duraiswami
![Page 59: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/59.jpg)
59
Focus and depth of field
• Depth of field: distance between image planes where blur is tolerable
Thin lens: scene points at distinct depths come in focus at different image planes.
(Real camera lens systems have greater depth of field.)
Shapiro and Stockman
“circles of confusion”
![Page 60: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/60.jpg)
60
Focus and depth of field
Image credit: cambridgeincolour.com
![Page 61: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/61.jpg)
61
Depth from focus
[figs from H. Jin and P. Favaro, 2002]
Images from same point of view, different camera parameters
3d shape / depth estimates
![Page 62: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/62.jpg)
62
Camera parameters
• How do points in real world relate to positions in the image?
• Perspective equations so far in terms of camera’s reference frame…
![Page 63: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/63.jpg)
63
Camera parameters
Need to estimate camera’s intrinsic and extrinsicparameters to calibrate geometry.
Camera frame
Intrinsic:Image coordinates relative to camera Pixel coordinates
Extrinsic:Camera frame World frame
World frame
![Page 64: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/64.jpg)
64
Camera calibration
• Knowing the relationship between real world and image coordinates useful for estimating 3d shape
• Extrinsic params: rotation matrix and translation vector
• Intrinsic params: focal length, pixel sizes (mm), image center point, radial distortion parameters
![Page 65: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/65.jpg)
65
Demirdjian et al.
Articulated tracking
![Page 66: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/66.jpg)
66
Brostow et al, 2004
3d skeleton extraction
![Page 67: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/67.jpg)
67
Human eye
Shapiro and Stockman
Pupil/Iris – control amount of light passing through lens
Retina - contains sensor cells, where image is formed
Fovea – highest concentration of cones
![Page 68: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/68.jpg)
68
Sensors• Often CCD camera: charge coupled device• Record amount of light reaching grid
photosensors, which convert light energy into voltage
• Read digital output row-by-row
cameraCCD array optics frame
grabber computer
![Page 69: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/69.jpg)
69
Digital images
Think of images as matrices taken from CCD array.
![Page 70: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/70.jpg)
70
im[176][201] has value 164 im[194][203] has value 37
width 520j=1
500 height
i=0Intensity : [0,255]
Digital images
![Page 71: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/71.jpg)
71
R G B
Color images, RGB color space
![Page 72: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/72.jpg)
72
Resolution• sensor: size of real world scene element a that
images to a single pixel• image: number of pixels• Influences what analysis is feasible, affects best
representation choice.
[Mori et al]
![Page 73: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/73.jpg)
73
Resolution
…though not necessarily for the human visual system with familiar faces…
[Sinha et al]
![Page 74: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/74.jpg)
74
Other sensors
• Stereo cameras• MRI scans• Xray• LIDAR devices…
[Jim Gasperini]
geospatial-online.com
![Page 75: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/75.jpg)
75
Summary
• Image formation affected by geometry, photometry, and optics.
• Projection equations express how world points mapped to 2d image.
• Lenses make pinhole model practical.
• Imaged points related to real world coordinates via calibrated cameras.
![Page 76: Computer Vision - University of Texas at Austingrauman/courses/378/slides/lecture1_full.pdf42 Camera obscura "Reinerus Gemma-Frisius, observed an eclipse of the sun at Louvain on January](https://reader033.fdocuments.in/reader033/viewer/2022050217/5f62b62e96dfaf0ef53b80f4/html5/thumbnails/76.jpg)
76
Next
Problem set 0 due Sept 6• Matlab warmup• Image formation questions• Read F&P Chapter 1
Reading for next lecture:• F&P Chapter 6