Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light...
-
Upload
jasper-simon -
Category
Documents
-
view
217 -
download
2
Transcript of Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light...
![Page 1: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/1.jpg)
Light Field Rendering
Shijin Kong
Lijie Heng
![Page 2: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/2.jpg)
Overview
• Purpose
• Algorithm 1. representation of light field-4D (u, v, s, t)
2. creation
3. compression
4. display
• Discussion
• Applications
![Page 3: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/3.jpg)
Purpose
• To generate a new view from an arbitrary position
• Previous methods Environment maps - depth information
Interpolating - correspondence points
![Page 4: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/4.jpg)
Purpose-Cont.
• Limit: “fill in the gaps”, incorrect correspondences
• To avoid these problems, a new method was proposed in this paper by using light field
![Page 5: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/5.jpg)
What is Light Field
• In the real 3D world, how to represent a ray? Using 5D Plenoptic Function
![Page 6: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/6.jpg)
4D Representation
• In free space (no occluders), oriented lines could be represented in 4D
![Page 7: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/7.jpg)
Representation of Light Field
![Page 8: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/8.jpg)
Creation of Light Field
• Rendered Images
![Page 9: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/9.jpg)
Creation of Light Fields
• Aliasing Low-pass filter
![Page 10: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/10.jpg)
Display
Show video
![Page 11: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/11.jpg)
Discussion
• Advantages 1. easy, robust
2. fast to create new views -sampling
• Limitations Large amount of data
Building acquisition device
Fixed focal surface
![Page 12: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/12.jpg)
Discussion
• Limitations Large amount of data
Building acquisition device
Fixed focal surface
![Page 13: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/13.jpg)
Compression
• VQ
• Lempel-Zic Coding
![Page 14: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/14.jpg)
Compression -Cont.
![Page 15: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/15.jpg)
Discussion
• Limitations Large amount of data
Building acquisition device
Fixed focal surface
![Page 16: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/16.jpg)
Acquisition-Option1
• Using a single camera
-hard to manipulate
-need refocus
-aliasing/blurriness
![Page 17: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/17.jpg)
Signal Camera
![Page 18: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/18.jpg)
Acquisition-Option2
• Using an array of cameras
-fixed
-inflexible
![Page 19: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/19.jpg)
Camera Arrays
![Page 20: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/20.jpg)
Acquisition-Option3
• Using an microlens array to replace the camera array
-easy to use
-reduce aliasing
![Page 21: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/21.jpg)
Hand-held Plenoptic Camera
![Page 22: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/22.jpg)
Hand-held Plenoptic Camera- Cont.
![Page 23: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/23.jpg)
More on Plenoptic Camera
• Higher image resolution, increased sharpness of the refocused photographs.
• Shorter exposures and lower image noise
![Page 24: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/24.jpg)
Discussion
• Limitations Large amount of data
Building acquisition device
Fixed focal surface
![Page 25: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/25.jpg)
Dynamic Reparameterized Light Fields
• Focal Surface is fixed when create light fields• What if recover pixels that are not on the focal surface• Reparameterize to required focal surface
![Page 26: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/26.jpg)
Dynamic Reparameterized Light Fields
• Move to desired new focal surface• Create a new 4D space with new focal surface• Recove ray with Reparameterization
• (u, v, s, t) => (u, v, f, g)F
![Page 27: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/27.jpg)
Dynamic Reparameterized Light Fields
• Recover ray r• Resample from ray (s’, t’, f, g) and (s’’, t’’, f, g)• Interpolation, reconstruction with filter, … , etc
![Page 28: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/28.jpg)
Dynamic Reparameterized Light Fields
![Page 29: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/29.jpg)
Dynamic Reparameterized Light Fields
• Change the shape of focal surface • Gives focus on 3D object rather than planes
![Page 30: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/30.jpg)
Dynamic Reparameterized Light Fields
![Page 31: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/31.jpg)
Dynamic Reparameterized Light Fields
![Page 32: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/32.jpg)
Variable Apertures
• Also can generate variable aperture• Aperture
– Control amount of light– Control depth of fields
• Aperture Filter: – Control how many cameras are used to resample a required ray– Larger apertures produce images with narrow range of focus
![Page 33: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/33.jpg)
Aperture Filters
![Page 34: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/34.jpg)
Variable Apertures
![Page 35: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/35.jpg)
Variable Apertures
![Page 36: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/36.jpg)
Application of Light Fields
• Multipersepective panorama
• Lens simulation
![Page 37: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/37.jpg)
Applications of Light Field
• Multipersepective panorama
![Page 38: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/38.jpg)
Applications of Light Field
• Lens simulation– Simulate lens with large aperture
![Page 39: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/39.jpg)
Future of Light Field
– Need better ways to capture larger collection of viewpoints
– Reconstruct 3D shape with vision algorithms– More images captured allow peeking arround
occlusions
![Page 40: Light Field Rendering Shijin Kong Lijie Heng. Overview Purpose Algorithm 1. representation of light field-4D (u, v, s, t) 2. creation 3. compression 4.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d035503460f949d630f/html5/thumbnails/40.jpg)
Thank you!
Questions?