Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and...
-
Upload
carli-frisbey -
Category
Documents
-
view
224 -
download
2
Transcript of Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and...
![Page 1: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/1.jpg)
Computer Graphics Inf4/MSc
Computer Graphics
Lecture Notes #16
Image-Based Modelling, Rendering
and Lighting
![Page 2: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/2.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 2
Global Illumination and Image-Based Lighting
• Traditional Computer Graphics involves:– Modelling with matter: geometry with reflectance
properties.
• Image-Based Modelling & Rendering is:– Modelling and rendering with Light, often deriving
geometry and materials in the process.
• Image-Based Lighting allows:– Combination of real and synthetic graphics with
consistent illumination, using images as light sources.
![Page 3: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/3.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 3
Pursuit of Photorealism
• For complex models, tools can require lot of effort and rendering is very slow.
• Choice: do usual modelling or just take a photo (that would provide photorealism!)
• IBMR is about using these photos and transcending their limitations (eg. Light).
• Don’t need anything like a complete model, but may need a number of photographs…
![Page 4: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/4.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 4
Pursuit of Photorealism
• Need enough photographs to give coverage.
• Major result is - rendering is faster.
• So Image-Based Modelling and Rendering (IBMR) starts with the answer!
• Research inspired by film industry.
• Related to techniques in Computer Vision.
• New so no taxonomy, just techniques.
![Page 5: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/5.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 5
IBMR Techniques
• Panoramas – movement within and between panoramic cylinders, eg. Quicktime VR.– Panoramas can be real or synthetic.
• View interpolation –from a few photos, interpolate view from any position (morphing).
• * Lighting – adding real or synthetic objects to a photographed scene, illuminated with scene lights.
• Modelling – extracting models from photos.
![Page 6: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/6.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 6
“Traditional graphics”
![Page 7: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/7.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 7
Computer Vision
![Page 8: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/8.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 8
Vision meets graphics
![Page 9: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/9.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 9
However ...
• Vision falls short
![Page 10: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/10.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 10
And ...
… so does graphics
![Page 11: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/11.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 11
Image-Based Rendering (IBR)
![Page 12: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/12.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 12
What is an image?
• Collection of radiance values
• radiance along a ray
• 3D position
• 2D direction
![Page 13: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/13.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 13
Plenoptic function• Radiance value for all
possible rays = plenoptic function.
• All possible images are a subset of this plenoptic function.
• Too much stuff!• Goal of IBR is generate a
continuous representation of the plenoptic function.
![Page 14: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/14.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 14
Plenoptic function
• Radiance is constant along a ray (line)
• 2D position
• 2D direction
Once we know one ‘origin’, we know them all
![Page 15: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/15.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 15
What is an image?• Image = rays going through one point
• usually restricted to viewing frustum, but can also be panoramic
![Page 16: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/16.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 16
What is an object?• Image = rays going through one point + image
plane
• 2D function (position on image plane)
![Page 17: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/17.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 17
What is an object?• Outgoing radiance field of an object
• 2D function (position on surface)
![Page 18: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/18.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 18
What is an object?• All light leaving the object
![Page 19: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/19.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 19
What is an object?• All light leaving the object
• 4D function (2D position + 2D direction)
![Page 20: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/20.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 20
What is an object?• All possible images of an object
![Page 21: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/21.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 21
What is an object?• All possible images of an object
![Page 22: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/22.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 22
What is an object?• We don’t really need the object
![Page 23: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/23.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 23
What is an object?• We don’t really need the object
![Page 24: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/24.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 24
Lumigraph / Light Field• Object is only defined by its radiance field
stuff4D function
(Levoy - Cohen et al 96)
![Page 25: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/25.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 25
Lumigraph - capture
(Stanford - Levoy et al.)
![Page 26: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/26.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 26
Layered Depth Images
(McMillan)
Problem with one photo is “holes” when view point moved:
- this is why a number of photos are generally required
- fill the holes using intensities of neighbouring pixels
- interpolation …
![Page 27: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/27.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 27
Image Based Lighting
• Add models or objects to scenes and allow them to be manipulated in the scenes.
• Modelling with light allows added objects to be illuminated consistent with image existent lighting - photorealistic inclusions.
• We start with the answer by finding the scene illumination.
![Page 28: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/28.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 28
Image Based Lighting
• Real Scene
• Goal: place synthetic objects on table
(all pictures P. Debevec 98-99)
![Page 29: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/29.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 29
Extracting scene lighting
• Capture illumination using illumination sphere
![Page 30: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/30.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 30
Image Based Lighting
Real scene
![Page 31: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/31.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 31
Image Based Lightingcaptured illumination field
![Page 32: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/32.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 32
Image Based Lighting
Real scene
light based model
local scene
synthetic objects
![Page 33: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/33.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 33
Image Based Lighting• Use renderer - compute effects of synthetic
objects on local scene
local scene (brdf estimated)synthetic objects (brdf known)
light based model
![Page 34: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/34.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 34
Image Based Lighting
• Render into the scene
background
![Page 35: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/35.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 35
Image Based Lighting
• Render synthetic objects
![Page 36: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/36.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 36
Image Based Lighting• Effect of local scene on real scene
![Page 37: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/37.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 37
Image Based Lighting• Add differences to image
![Page 38: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/38.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 38
Reconstruction from images?
• Computer Vision related.• Constructs new views and extracts models.• Epipolar geometry
– expresses relationships between points in different images.
• Difficult to predict full impact in CG.• State of the Art
– 3D reconstruction from uncalibrated images.
![Page 39: Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.](https://reader035.fdocuments.in/reader035/viewer/2022062511/551c4823550346a5458b48c5/html5/thumbnails/39.jpg)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 39
References
• You won’t find this material in the major graphics texts since it is new and not yet mainstream but I recommend the following:– The Computer Image, Watt & Policarpo, Addison-
Wesley 1998.
– SIGGRAPH courses for the past few years, we have some on CD rom and on-line within EdVEC and not immediately available to Informatics machines – ask me if you have a strong interest and need.