Taking advantage of virtual optics in computer holography
Kyoji Matsushima, Daichi Fujita,
Yutaka Yoshizaki, Sumio Nakahara
Kansai University, JAPAN
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Introduction
Background • Resolution of high-definition CGH are being comparable with that in
classical holography. • Computer holography is the technique for creating CGHs. • Digitized holography is the technique to digitize the whole process of
holography; real-existing object waves are recorded by an image sensor and the object waves are reconstructed by CGHs.
• In computer holography and digitized holography, object waves are handled as digital wave-fields, and thus both can take advantage of virtual optics.
Outline • Source material of 3D scene in computer holography • Some techniques used in our computer holography
– Polygon-based method and Silhouette method
• Creation of holographic stereograms through virtual optics with numerical methods
• Techniques to realize digitized holography • Digital editing of 3D scene and resizing of 3D object in digitized
holography
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Some of early works
The Venus, 2009 The first work 4 G Pix (64K×64K)
Aqua 2, 2009 Hidden-surface removal 8 G Pix (128K×64K)
The Moon, 2010 Texture-mapping 8 G Pix (128K×64K)
1K = 1024 HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Bear II, 2010 Digitized holography
Some of early works
The Metal Venus I 2010 Specular flat shading
The Metal Venus II 2011 Specular smooth shading
iShion V2, 2011 Image hologram White light reconst.
1K = 1024
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Picture image 2D object
Real-existing object
Polygon-mesh 3D object
Computer hologram
Field synthesis Field capturing
Digital editing
Computer Holography: A Concept
Multiple source materials are used to build the 3D scene.
Synthesized or captured fields are integrated in a virtual 3D scene using field-based digital editing.
Multi-viewpoint image
Source materials
Printing fringe pattern
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
3D scene in computer holography
Captured field (Real-existing object)
Multi-viewpoint image (Background)
CG-modeled 3D object (Virtual object)
Hologram
Photograph or illustration
Conceptual illustration of a typical 3D scene of computer holography.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Picture image 2D object
Real-existing object
Polygon-mesh 3D object
Computer hologram
Field synthesis Field capturing
Digital editing
Computer holography for polygon-mesh objects
Multi-viewpoint image
Printing fringe pattern
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Polygon-based method
Spherical wave
Surface source of light (polygon source)
Polygon field
Point source of light
Ray-oriented rendering
Field-oriented rendering
Sampling grid
Polygon-based method is usually faster than the point-based method in rendering surface objects, because the number of the polygons composing a surface object is much smaller than that of point sources.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Surface function
P3
P1
P 2
x1
y2
y1
x2
y3
x3
x1
^
y1
^
x2
^
y2
^
x3
^
y3
^
x1
y1
x2
y2
x3
y3
x^
y^
z^
Hologram
Quasi random phase plays
role of diffuser
( , ) ( , ) exp[ ( , )]n n n n n n n n n
h x y a x y i x y
Amplitude Phase
Amplitude Phase
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Surface function
x3
^
y3
^
x2
^
y2
^
x1
^
y1
^
x1
y1
x2
y2
x3
y3
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Texture mapping in polygon-based method
Amplitude Phase
( , ) ( , )n n n n n n
a x y I x y
In(xn, yn) : Texture image
Sphere
Texture
Surface function
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Brothers exhibited in MIT museum
3D laser scanner
The shape of live faces are measured by 3D laser scanner, Konica Minolta Vivid 910.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Optical reconstruction
Brothers, 2012 Diffuse smooth shading, Live faces Big hologram, 25 G pix (196,608×131,072)
Exhibition in MIT museum
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Exhibition of Brothers
Exhibition in MIT museum
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
LED light box
Brothers
Description
Exhibition in MIT museum
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Picture image 2D object
Real-existing object
Polygon-mesh 3D object
Computer hologram
Field synthesis Field capturing
Digital editing
Digitized holography: Computer holography for real-existing objects
Multi-viewpoint image
Printing fringe pattern
Interference
Diffraction
Digitized Holography
The object field is captured by the technique of digital holography.
The captured field is digitally edited and mixed with virtual 3D scene.
The mixed field is optically reconstructed by computer-hologram.
Digital holography
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Problems to realize digitized holography
Image sensor 3,000 × 2,200 pixels (3.5 m ×3.5 m)
1. Expansion of captured area (Increase of number of samplings) 2. Reduction of sampling intervals
The captured area of current image sensors is not sufficient for high-definition computer holograms.
The sampling interval of the captured field is also not sufficient for high-definition computer holograms.
Shion, 2010 131,072×65,536 pixels (0.8 m ×0.8 m)
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Interference fringe
Techniques for capturing large fields at high-sampling density
Image sensor
1. Lensless-Fourier setup for reducing sampling interval 2. Synthetic aperture digital holography for extending captured area
As the sensor moves, a part of field is captured.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
3D scene including real-existing fields : Penguin
z
Units : mm
Captured fields of real-existing objects
CG-modeled 3D object
Hologram
Wallpaper of 2D digital image
2D digital images
100
100
50
50
131
65
80
160
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Optical reconstruction
Penguin, 2012 Digitized holography 8 G pix
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Field-based digital editing of 3D scene
Hologram
Component 0
Component 1
Component 2
Component 3
Component 4
The 3D scene in computer holography includes many components.
The mutual occlusion among components must be reproduced in optical reconstruction.
We use silhouette method to achieve this.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
The silhouette method: light-shielding by opaque object
Silhouette mask
Object
A real opaque object shields light behind the object. In computer holography, to prevent the object being a phantom
image, the field of incident light is masked by the object' silhouette. We calculate the background field at the center of object and then
mask it by the object’ silhouette.
Real object Wave field rendering
Wave-field
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Example of Silhouette Masking
Background Object plane Hologram
+ =
Wave-field of object
Superimposed wave- field in object plane
1st propagation
2nd propagation
The background field, propagated to the object plane, is masked with the silhouette of the Venus stature.
After superimposing Venus field, the wave field is propagated to the hologram.
Wave-field of background
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
binarize and
reverse Numerical Reconstruction
Small part of captured field
Amplitude image Silhouette mask
Silhouette mask in digitized holography
In digitized holography, we have no information about the object shape. Numerical reconstruction of real fields is used for produce the silhouette mask.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Self-occluded object
Masking by silhouette of object Masking by silhouette of polygon
Problems of self-occluded objects
Masking technique by object silhouettes does not work well in self-occluded objects.
To properly shield light of self-occluded object, fields must be masked by the silhouette of every polygon.
However, masking by the polygon’s silhouettes is very time-consuming.
Self-occluded object
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Polygon P1
Procedure of switch-back method
Hologram Polygon P2
Polygon P3
Polygon P4
Polygon P5
The switch-back method is developed to speed-up masking by the polygon’s silhouettes.
In this technique the field of object is numerically propagated back and forth and masked by the polygon’s silhouette.
Many formulas are necessary for the explanation of the mechanism. HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Hologram for self-occluded object
Rose in Ring, 2012 Self-occluded object (Switch-back method) Specular smooth shading
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
USE OF VIRTUAL OPTICS
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Resizing objects in digitized holography
𝑓 𝑓
a b
W
𝑊𝑏
Numerical propagation
𝑓 𝑓
a b
a
bM
Magnification
Numerical propagation
Lens function
)(exp
22yx
fi
x
y
z
x x
y y
Object plane Image plane HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
M = 0.25 M = 0.5
M = 1.5
Original
Hamsters, 2012 Resizing objects
Background
Hologram
M = 0.5
M = 1.5
Fields of resized objects
Original object field
Computer hologram for resizing objects: Hamsters
M = 0.25
Hamsters was created for testing the technique.
The 3D scene include 3 resized object obtained from the original field.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Picture image 2D object
Real-existing object
Polygon-mesh 3D object
Computer hologram
Field synthesis Field capturing
Digital editing
Using multi-viewpoint image as background
Multi-viewpoint image
Source materials
Printing fringe pattern
2D digital illustration or pictures were used for background so far.
Multi-viewpoint image is desirable for the background of the 3D scene.
A holographic stereogram is numerically produced and arranged in the 3D scene.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
3D scene in computer holography
Multi-viewpoint image (Background)
Hologram
CG-modeled 3D object (Virtual object)
View pane
Numerical propagation
Numerical back-propagation
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Hologram using multi-viewpoint image
Parked Car 1, 2012 Multi-viewpoint image as background
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Software
• Software can be downloaded from our website:
– http://www.laser.ee.kansai-u.ac.jp/WaveFieldTools
– Or search keyword WaveFieldTools
– All contents are written in Japanese.
• The current software is provided as Library of C++. Windows software will be available in a few months.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Conclusion
• Computer holography can use many source materials and create brilliant CGHs whose reconstructions are comparable with that by traditional holography.
• Computer holography has the advantage that the 3D images are digitally archivable and transmittable.
• Real-existing objects and 3D scenes can be incorporated into CGHs by using digitized holography or as holographic stereograms through virtual optics with numerical methods.
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
References • K. Matsushima, S. Nakahara, et al.: Computer holography: 3D digital art based on high-definition
CGH, International Symposium on Display Holography 2012 (ISDH2012), MIT Media Lab, (2012). http://river-valley.tv/computer-holography-3d-digital-art-based-on-high-definition-cgh/
• K. Matsushima, H. Nishi, S. Nakahara: Simple wave-field rendering for photorealistic reconstruction in polygon-based high-definition computer holography, J. Electron. Imaging 21, 023002(2012). DOI: 10.1117/1.JEI.21.2.023002
• H. Yamashita, K. Matsushima, S. Nakahara: Image-type high-definition CGHs encoded by optimized error diffusion, SPIE Proc. #8281, 82810Z(2012.1.25). DOI: 10.1117/12.908353
• H. Nishi, K. Matsushima, S. Nakahara: Advanced rendering techniques for producing specular smooth surfaces in polygon-based high-definition computer holography, SPIE Proc. #8281, 828110(2012). DOI: 10.1117/12.908871
• K. Matsushima, Y. Arima, S. Nakahara: Digitized holography: modern holography for 3D imaging of virtual and real objects, Appl. Opt. 50, H278-H284 (2011). DOI: 10.1364/AO.50.00H278
• H. Nishi, K. Matsushima, S. Nakahara: Rendering of specular surfaces in polygon-based computer-generated holograms, Appl. Opt. 50, H245-H252 (2011). DOI: 10.1364/AO.50.00H245
• K. Matsushima, S. Nakahara: Extremely High-Definition Full-Parallax Computer-Generated Hologram Created by the Polygon-Based Method, Appl. Opt. 48, H54-H63 (2009). DOI: 10.1364/AO.48.000H54
• K. Matsushima: Computer-Generated Holograms for Three-Dimensional Surface Objects with Shade and Texture, Appl. Opt. 44, 4607-4614(2005). DOI: 10.1364/AO.44.004607
• K. Matsushima, A. Kondoh: A Wave Optical Algorithm for Hidden-Surface Removal in Digitally Synthetic Full-Parallax Holograms for Three-Dimensional Objects, SPIE Proc. #5290, 90-97(2004). DOI: 10.1117/12.526747
HODIC in Taiwan 2012: Symposium for Holographic Display Technology and Art, Tainan (Taiwan), (2012.12.11, Invited talk)
Top Related