Recommendation and Information Retrieval: Two Sides of the Same Coin?
Computer Vision and Computer Graphics: Two sides of a coin
description
Transcript of Computer Vision and Computer Graphics: Two sides of a coin
![Page 1: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/1.jpg)
Computer Vision and Computer Graphics: Two sides of a coin
COS 116: Apr 22, 2008
Sanjeev Arora
![Page 2: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/2.jpg)
Brief history of image-making
Camera obscura.Camera obscura.
Known to chinese; 5th century BCKnown to chinese; 5th century BC
19th century: Replace hole withlens; sketchpaper with light-sensitive paper. “Camera”
19th century: Replace hole withlens; sketchpaper with light-sensitive paper. “Camera”
Late 20th century: Replace light-sensitive paper with electronic light sensor: “Digital camera.”Late 20th century: Replace light-sensitive paper with electronic light sensor: “Digital camera.”
![Page 3: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/3.jpg)
Theme 1: What is an image?
![Page 4: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/4.jpg)
What is an image?
Rectangular (2D) array of pixels
Continuous imageDigital image
“Pixels”“Pixels”
![Page 5: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/5.jpg)
“Pixel” is a sample; need not be square
(Many choices for “rendering” the same information)(Many choices for “rendering” the same information)
(Remember music lecture: (Remember music lecture:
![Page 6: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/6.jpg)
RGB Color Model
Plate II.3 from FvDFH
R G B Color 0.0 0.0 0.0 Black1.0 0.0 0.0 Red0.0 1.0 0.0 Green0.0 0.0 1.0 Blue1.0 1.0 0.0 Yellow1.0 0.0 1.0 Magenta0.0 1.0 1.0 Cyan1.0 1.0 1.0 White0.5 0.0 0.0 ?1.0 0.5 0.5 ?1.0 0.5 0.0 ?0.5 0.3 0.1 ?
Colors are additive
![Page 7: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/7.jpg)
Adjusting Brightness
Simply scale pixel componentsMust clamp to range (e.g., 0 to 1)
Original Brighter
![Page 8: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/8.jpg)
Adjusting Contrast Compute average luminance L for all pixels
luminance = 0.30*r + 0.59*g + 0.11*b
Scale deviation from L for each pixelMust clamp to range (e.g., 0 to 1)
Original More Contrast
L
![Page 9: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/9.jpg)
Scaling the image Resample with
fewer or more pixels(mathy theory…)
Original 1/4X resolution
4X resolution
![Page 10: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/10.jpg)
Theme 2: Computer vision vs
Computer Graphics (and why they get mathy)
Computer Vision: Understanding the “content” of an image (usually by creating a “model” of the depicted scene)
Computer graphics: Creating an image from scratchUsing a computer model.
Computer Vision: Understanding the “content” of an image (usually by creating a “model” of the depicted scene)
Computer graphics: Creating an image from scratchUsing a computer model.
![Page 11: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/11.jpg)
Math used to understand/create imagesMath used to understand/create images
1) Coordinate geometry (turns geometry into algebra)1) Coordinate geometry (turns geometry into algebra)
2) Laws of perspective Laws of perspective2) Laws of perspective Laws of perspective
![Page 12: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/12.jpg)
(Math needed..) Physics of light
Lighting parametersLight source emissionSurface reflectance
NN
eye
Surface
LightSource
![Page 13: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/13.jpg)
Math needed in the design of algorithms Example: Image Morphing
Image0
Image1
Warp0
Warp1
[Beier & Neeley]
Result
![Page 14: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/14.jpg)
Intro to computer vision.Intro to computer vision.
What is depicted in this image?What is depicted in this image?
![Page 15: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/15.jpg)
Edge detection
What is an “edge”?What is an “edge”?
Place where image“changes” suddenly.Place where image“changes” suddenly.
How to identify edges?How to identify edges?
![Page 16: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/16.jpg)
A very simple edge detection idea
A[i,j] <- 5 A[i, j] - A[i+1, j] - A[i-1,j] - A[i, j+1] -A[i, j-1] A[i,j] <- 5 A[i, j] - A[i+1, j] - A[i-1,j] - A[i, j+1] -A[i, j-1]
More sophisticated edge-detection uses smarter versions ofthis; use Gaussian filters, etc.
Human eye does some version of edge detection.
Edge info is still too “low level.”
More sophisticated edge-detection uses smarter versions ofthis; use Gaussian filters, etc.
Human eye does some version of edge detection.
Edge info is still too “low level.”
![Page 17: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/17.jpg)
Image Segmentation
What are the regions in this image?What are the regions in this image? Uses many many algorithmic
ideas; still not 100% accurateUses many many algorithmicideas; still not 100% accurate
![Page 18: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/18.jpg)
High level vision: Object recognition
What do you see in thispicture?What do you see in thispicture?
Much harder task than it may seem. Tiger needs to be recognized from any angle, and under any lighting condition and background.
Much harder task than it may seem. Tiger needs to be recognized from any angle, and under any lighting condition and background.
![Page 19: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/19.jpg)
Aside
At least 8 “levels” in human vision system.Object recognition seems to require transfer ofinformation between levels, andthe highest levels seem tiedto rest of intelligence
At least 8 “levels” in human vision system.Object recognition seems to require transfer ofinformation between levels, andthe highest levels seem tiedto rest of intelligence
![Page 20: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/20.jpg)
Next: Computer Graphics
Applications: Entertainment Computer-aided design Scientific visualization Training Education E-commerce Computer art
Boeing 777 AirplaneInside a Thunderstorm
(Bob Wilhelmson, UIUC)
![Page 21: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/21.jpg)
Step 1: Modeling How to construct and represent shapes (in 3D)
(Remo3D)
![Page 22: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/22.jpg)
Modeling in SketchUp (demo)
![Page 23: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/23.jpg)
Example of “model”: wireframe Most common: list of triangles
Three vertices in 3D(x1, y1, z1)(x2, y2, z2)(x3, y3, z3)
Usually would be augmentedwith info about texture, coloretc.
Usually would be augmentedwith info about texture, coloretc.
![Page 24: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/24.jpg)
Step 2: Rendering
Given a model, a source of light, and a point of view, how to render it on the screen?
![Page 25: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/25.jpg)
Rendering (contd)
Direct illumination One bounce from light to eye Implemented in graphics cards OpenGL, DirectX, …
Global illumination Many bounces Ray tracing
Direct Illumination(Chi Zhang, CS 426, Fall99)
Ray Tracing(Greg Larson)
![Page 26: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/26.jpg)
Ray Casting
A (slow) method for computing direct illumination For each sample:
Construct ray from eye through image plane
Find first surface intersectedby ray
Compute color of sample based on surface properties
eye
![Page 27: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/27.jpg)
Simple Reflectance Model
Simple analytic model: diffuse reflection +specular reflection +ambient lighting
SurfaceBased on modelproposed by Phong
Based on modelproposed by Phong
![Page 28: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/28.jpg)
Diffuse Reflection Assume surface reflects equally in all directions
Examples: chalk, clay
Surface
![Page 29: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/29.jpg)
Specular Reflection
Reflection is strongest near mirror angle Examples: mirrors, metals
N
LR
![Page 30: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/30.jpg)
Ambient Lighting
This is a total cheat (avoids complexity of global illumination)!
Represents reflection of all indirect illumination
![Page 31: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/31.jpg)
Path TypesL = lightD = diffuse bounceS = specular bounceE = eye
![Page 32: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/32.jpg)
Path Types?
Henrik Wann Jensen
![Page 33: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/33.jpg)
Ray Tracing
Henrik Wann Jensen
![Page 34: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/34.jpg)
Ray Tracing
RenderPark
![Page 35: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/35.jpg)
Ray Tracing
Terminator 2
![Page 36: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/36.jpg)
Step 3: Animation
Keyframe animationArticulated figures
SimulationParticle systems
Animation(Jon Beyer,
CS426, Spring04)
Simulation
![Page 37: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/37.jpg)
Articulated Figures
Rose et al. `96
Well-suited for humanoid characters
Root
LHip
LKnee
LAnkle
RHip
RKnee
RAnkle
Chest
LCollar
LShld
LElbow
LWrist
LCollar
LShld
LElbow
LWrist
Neck
Head
![Page 38: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/38.jpg)
Keyframe Animation: Luxo Jr.
Pixar
![Page 39: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/39.jpg)
Keyframe Animation
Define character poses at specific times: “keyframes”
“In between” poses found by interpolation
Lasseter `87
![Page 40: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/40.jpg)
Keyframe Animation
Inbetweening: may not be plausible
Lasseter `87
![Page 41: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/41.jpg)
Keyframe Animation
Solution: add more keyframes
Lasseter `87
![Page 42: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/42.jpg)
But, animator cannot specify motion for:o Smoke, water, cloth, hair, fire
o Soln: animation!
Cloth(Baraff & Witkin `98)
Water
Hot Gases(Foster & Metaxas `97)
![Page 43: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/43.jpg)
Particle Systems A particle is a point mass
Mass Position Velocity Acceleration Color Lifetime
Many particles to model complex phenomena Keep array of particles
p = (x,y,z)
v
![Page 44: Computer Vision and Computer Graphics: Two sides of a coin](https://reader035.fdocuments.in/reader035/viewer/2022062314/568143f3550346895db07ceb/html5/thumbnails/44.jpg)
Particle Systems Recall game of life, weather etc.… For each frame (time step):
Create new particles and assign attributes Delete any expired particles Update particles based on attributes and physics
Newton’s Law: f=ma Render particles