Uniform angular resolution integral imaging display with...

Uniform angular resolution integral imaging displaywith boundary folding mirrors

Joonku Hahn, Youngmin Kim, and Byoungho Lee*School of Electrical Engineering, Seoul National University, Gwanak-Gu Sillim-Dong, Seoul 151-744, South Korea

*Corresponding author: [email protected]

Received 27 October 2008; revised 10 December 2008; accepted 10 December 2008;posted 11 December 2008 (Doc. ID 103167); published 14 January 2009

Uniform angular resolution integral imaging display is proposed. Conventionally, inefficient bias of theperspectives around boundaries is disregarded and it is impossible to display the field of view over fullspatial resolution. However, the proposed display has four boundary folding mirrors and these mirrorsfold the view volumes correctly to form uniform angular resolution within them. The distribution ofperspectives and field of view in the proposed system are analyzed and the removal of this boundaryeffect is confirmed experimentally. © 2009 Optical Society of America

OCIS codes: 100.6890, 110.2990.

1. Introduction

Integral imaging (InIm) is one of the most feasibledirectional displays that give different perspectivesaccording to view directions. This technique displaysa three-dimensional (3D) object with full parallaxwithout any viewing aids, such as glasses, and it doesnot require any diffuser screen that constrains thedirections of perspective [1–3]. Therefore, the ele-mental images are focused on the image plane follow-ing the lens law; this image plane is commonly calledthe central depth plane (CDP).In this CDP, directions of perspective are distribu-

ted not only horizontally but also vertically, resultingfrom a two-dimensional lens array. The number ofperspectives is understood as angular resolution.Figure 1 shows the spatial and angular resolutionson the CDP of an InIm display. There is a trade-offrelation between the spatial and the angular resolu-tions [4–6]. Therefore, the angular resolution is in-creased at the cost of the reduction of spatialresolution. The viewing angle of the InIm displayis defined as the extent of the perspectives at thefixed position of the CDP.

In an InIm display, many methods to widen theviewing angle have been proposed. In one method,the surface of elemental images is embossed to widenthe individual viewing angle [7]. In another method,both elemental images and lens array are curved toenhance the field of view [8]. However, these studiesare concerned only with the viewing angle in the bulkregion. In the boundaries of an InIm display, theviewing angle is not as uniform as that in the bulkregion, and the perspective views around the bound-aries are biased to the outward direction, which isinefficient for creating the field of view.

The clues to removing the effect of boundary can befound in a multiprojection display such as a holo-gramlike display. In these displays, two mirrorsare utilized to fold left- and right-side perspectives[9–11]. The folding mirrors make the additional vir-tual projections by reflection and the folded view vo-lumes efficiently form the fields of view over the fullspatial resolutions. However, these multiprojectiondisplays have only a one-dimensional projection ar-ray and every projection optic is positioned apartfrom the others. Therefore, the asymmetric diffuserthat is usually called a holographic screen is neces-sary and full parallax is impossible.

We propose an InIm display with boundary foldingmirrors. This display gives uniform angularresolution within four boundaries. Figure 2 shows

0003-6935/09/030504-08$15.00/0© 2009 Optical Society of America

504 APPLIED OPTICS / Vol. 48, No. 3 / 20 January 2009

a schematic of the proposed InIm display. These fourfolding mirrors extend to CDP whose location is de-termined by the focal length of the lens array and thegap between the elemental image and the lens array.In Section 2, the angular resolution and distribu-

tion of perspectives in the InIm display are describedand the effects of boundary folding mirrors are ex-pressed analytically. In Section 3, the field of viewof the proposed InIm display is compared with thatof the conventional InIm display and elementalimages of the proposed system are explained and ex-perimental results are presented and discussed. InSection 4, conclusions and perspective are given.

2. Angular Resolution in Proposed Integral ImagingDisplay

In directional displays, the angular resolution is gi-ven by the number of perspectives. In a view volumeby a single lens, the perspective is a function of itsposition. Therefore, the angular resolution is equalto the number of overlapping-in-view volumes at agiven position. Figure 3 shows the view volume anddirections of perspective in the InIm display. Sincethe CDP is an image plane of the elemental image,

the chief ray of an individual lens is important todescribe the directions of perspective and the viewvolume generated by this lens is represented asthe existence of the chief ray in the space. The viewvolume generated by the lens positioned at ðxnm;ynm; zLÞ is defined by

Vnmðx; y; zÞ ¼ rect�dgapðx − xnmÞwðz − zLÞ

�rect

�dgapðy − ynmÞwðz − zLÞ

�:

ð1Þ

Here, dgap is the gap between the lens array and theelemental image and w is the lens spacing of the lensarray. The function rectðÞ is a rectangular function,which is only true (i.e., gives numerical value of 1as a function output) when the absolute value ofthe input argument is less than 1=2. Otherwise, itsfunctional output is 0. Therefore, the view volume de-fined in this paper is a Boolean value. For example,this view volume includes the position ðxnm; ynm;z ≠ zLÞ. This means that the extension of a line con-necting two points, ðxnm; ynm; zLÞ and ðxnm; ynm;z ≠ zLÞ, passes through the corresponding elementalimage. Therefore, the view volume is shaped as twopyramids with the summits in contact [12]. Sinceangular resolution is the number of imbricate viewvolumes, angular resolution at a position ðx; y; zÞ isgiven by

RAðx; y; zÞ ¼XNn¼1

XMm¼1

Vnmðx; y; zÞ; ð2Þ

where N and M are the number of lenses in the hor-izontal and vertical directions, respectively, as shownin Fig. 3. The directions of perspectives are deter-mined by the positions of individual lenses withinthe corresponding view volumes. At a given positionðx; y; zÞ, the distribution of perspectives is the set ofdirections of chief rays passing through the lensarray. Then the distribution of perspectives fromthe imbricate view volumes are defined by

Fig. 1. (Color online) Spatial and angular resolutions on centraldepth plane of InIm display.

Fig. 2. (Color online) Schematics of InIm display with boundaryfolding mirrors.

Fig. 3. (Color online) View volume and directions of perspectivein InIm display.

20 January 2009 / Vol. 48, No. 3 / APPLIED OPTICS 505

DPðx; y; zÞ ¼ f~r0j~r0 ¼ ð~r −~rnmÞVnmð~rÞfor 1 ≤ n ≤ N; 1 ≤ m ≤ Mg:

ð3Þ

Here~r and~rnm mean the position ðx; y; zÞ and the po-sition of lens ðxnm; ynm; zLÞ, respectively.From the point of view in angular resolution, the

proposed InIm display is compared with the conven-tional InIm display. To make the issue clear, the ana-lysis is restricted on the xy plane but it does not losethe generality. Figure 4 shows the imbricate viewvolumes and directions of perspective view on theCDP in the conventional InIm display. In Fig. 4(a),the number of overlapping-in-view volumes de-creases around the boundaries and the angularresolution also decreases across the boundaries. Asshown in Fig. 4(b), the local viewing angle is relatedto the angular resolution and the resultant viewingangle from perspectives becomes narrow near theboundaries. Moreover, in the region over the bound-aries, only the outgoing directions of perspectivesremain.Figure 5 shows the angular resolution and the dis-

tribution of viewing angle on the CDP in the conven-tional InIm display shown in Fig. 4. In Fig. 5(a), theviewing angle has the maximum value in the bulkregion and decreases coming up to both upper andlower boundaries. As shown in Fig. 5(b), the distribu-tion of perspectives has sawtooth shape in the bulkregion since each direction of perspective changes ac-cording to the relative positions from the individuallenses. Moreover, there is only the outgoing direction

in perspectives around the boundaries and, then, thedistribution of perspectives in the conventional InImdisplay is roughly a parallelogram shape. Therefore,

the bias in perspectives results in a reduction of thefield of view, which can be displayed over the totalviewing angle. This phenomenon is discussed inmore detail in Section 3.

In the proposed InIm display with boundary fold-ing mirrors, all view volumes are folded at the mirrorplanes. Since the number of mirror planes is four,there are eight reflected positions per a single lens.The original position of lens ðxnm; ynmÞ and its re-flected positions are represented as

�xnmp ¼8<:

2x11 −w − xnm forp ¼ −1xnm forp ¼ 0

2xN1 þw − xnm forp ¼ 1; ð4aÞ

�ynmq ¼8<:

2y11 −w − ynm for q ¼ −1ynm for q ¼ 0

2yN1 þw − ynm for q ¼ 1: ð4bÞ

The resultant folded view volume is defined by

�Vnmðx; y; zÞ ¼X1p¼−1

X1q¼−1

�Vnmpqðx; y; zÞ: ð5Þ

Here, �Vnmpqðx; y; zÞ is a part of the folded view volumefrom nine positions, which are the original position ofthe lens and its eight reflected positions. Since theboundary folding mirrors have finite length,�Vnmpqðx; y; zÞ is defined by

�Vnmpqðx; y; zÞjðp;qÞ¼ð0;0Þ ¼ H

�dmirrorðx − �xnmpÞ

ð�xnmp − x11 þw=2Þðz − zLÞþ 1

�H

�dmirrorðx − �xnmpÞ

ð�xnmp − xN1 −w=2Þðz − zLÞþ 1

�

×H

�dmirrorðy − �ynmqÞ

ð�ynmq − y11 þw=2Þðz − zLÞþ 1

�H

�dmirrorðy − �ynmqÞ

ð�ynmq − y1M −w=2Þðz − zLÞþ 1

�

× rect�dgapðx − �xnmpÞ

wðz − zLÞ�rect

�dgapðy − �ynmqÞ

wðz − zLÞ�; ð6aÞ

�Vnmpqðx; y; zÞjðp;qÞ≠ð0;0Þ ¼ H

�−

dmirrorðx − �xnmpÞð�xnmp − x11 þw=2Þðz − zLÞ

− 1

�H

�−

dmirrorðx − �xnmpÞð�xnmp − xN1 þw=2Þðz − zLÞ

− 1

�

×H

�−

dmirrorðy − �ynmqÞð�ynmq − y11 þw=2Þðz − zLÞ

− 1

�H

�−

dmirrorðy − �ynmqÞð�ynmq − y1M −w=2Þðz − zLÞ

− 1

�

× rect�dgapðx − �xnmpÞ

wðz − zLÞ�rect

�dgapðy − �ynmqÞ

wðz − zLÞ�: ð6bÞ

Here, H½ � is the unit step function and dmirror is thelength of the boundary folding mirrors, which is de-signed by


dmirror ¼f dgap

f þ dgap: ð7Þ

As previously mentioned, the angular resolution isthe number of overlapping-of-view volumes and theangular resolution of the proposed InIm display is gi-ven by

�RAðx; y; zÞ ¼XNn¼1

XMm¼1

X1p¼−1

X1q¼−1

�Vnmpqðx; y; zÞ: ð8Þ

The distribution of perspectives from the folded im-bricate view volumes are defined by

�DPðx; y; zÞ ¼ f~r0j~r0 ¼ ð~r −~rnmpqÞ�Vnmpqð~rÞfor 1 ≤ n ≤ N; 1 ≤ m ≤ M;p; q ¼ −1; 0; 1g: ð9Þ

Figure 6 shows the folded imbricate view volumesand directions of perspective views on the CDP in

the proposed InIm display. In Fig. 6(a), the angularresolution is uniform since the reflected parts of theview volume complement the angular resolutionnear the boundaries. In this system, the viewing an-gle is also uniform and the directions of perspectiveare not biased, as shown in Fig. 6(b).

Figure 7 shows the angular resolution and the dis-tribution of perspectives on the CDP in the proposedInIm display shown in Fig. 6. In Fig. 7(a), the viewingresolution uniformly has the same value as the max-imum in the bulk region. This effect is understood asthe rotation of the outside part into the inside. Asshown in Fig. 7(b), the distribution of perspectivesis roughly a rectangle shape where the upper andlower sides have a sawtooth shape in the bulk region.This phenomenon is also understood as the rotationof the outside part into the inside.

3. Field of View of Proposed Integral Imaging Display

In an InIm display, the field of view means the spacewhere the 3D objects to be displayed are positioned.In the proposed InIm display, the field of view is dif-ferent from that of the conventional InIm display.Though their dimensions and shapes are equal toeach other, the locations are different. Figure 8 showsthe movement of the field of view by means of bound-ary folding mirrors. Here, the total viewing angle isthe viewing angle defining the viewing zone wherethe whole field of view can be observed. It has thesame value as the local viewing angle in the bulk re-gion, which is determined by

Ω ¼ 2 arctanðw=2dgapÞ: ð10Þ

Fig. 4. (Color online) Conventional InIm display with (a) imbri-cate view volumes and (b) directions of perspective on CDP.

Fig. 5. (Color online) (a) Angular resolution and (b) distributionof perspective on CDP in conventional InIm display.


In an InIm display, the view volume generated by anindividual lens is shaped as two pyramids with thesummits in contact, as represented by Eq. (1). Thefield of view can be defined as the region wherethe full perspective exists. The objects in this regioncan be watched by observers positioned at an infinitedistance from the display within the viewing angle.Therefore, the field of view of the InIm display is arhombus shape. Observers are positioned only infront of the display and the viewing zone, which isdefined as a perfectly overlapped region of all theview volumes located in front of the display. Withinthis viewing zone, the observers can watch the wholefield of view with full perspective.In the conventional InIm display, the largest cross

section in the field of view is located on the lens arrayplane, as shown in Fig. 8(a). On the other hand, in theproposed system, the folding mirrors move the fieldof view into the front of the observer, as shown inFig. 8(b). This phenomenon occurs because an addi-tional lens array effectively exists in the mirrorfolded region. Here, the width wfolded of this mirrorfolded region is determined by

wfolded ¼ w × ⌈ðMx − 1Þ=2⌉: ð11Þ

Here, ⌈ ⌉ is the ceiling function that converts a realnumber to the smallest integer not less than it and

Mx means the horizontal magnification by the lenslaw, which is given by

Mx ¼ f =ðdgap − f Þ: ð12Þ

In the case that Mx is not an integer, Eq. (11) is validbut the uniformity of angular resolution is broken.

In comparison with the conventional InIm display,the field of view in the proposed InIm display has alarger cross section on the CDP. In general, the closerto the CDP 3D objects are positioned, the morecorrectly they are represented. Therefore, the pro-posed system uses the angular resolutions efficientlyand displays the field of view over the full spatialresolution.

The boundary folding mirrors make the field ofview move toward observers and the proposed meth-od has the advantage only for the real display modewhen the CDP is located in front of the lens array. Onthe other hand, in the virtual display mode, the CDPis located behind the lens array. Then the field ofview in the proposed InIm display has a smaller crosssection on the CDP than that in the conventionalInIm display. Therefore, it is improper to use thismethod for the virtual mode.

In the proposed system, the reorganization isnecessary to generate the set of elemental images.Figure 9 shows this reorganization relation. The ad-ditional lenses in the mirror folded region do not ac-tually exist and the corresponding elemental imagesshould be reorganized. In Fig. 9, we assume that thehorizontal number of lenses is seven and that themagnification Mx is 5. Each number on the CDP re-presents the spatial portions with the same width as

Fig. 6. (Color online) Proposed InIm display with (a) folded im-bricate view volumes and (b) directions of perspectives on CDP.

Fig. 7. (Color online) (a) Angular resolution and (b) distributionof perspectives on CDP in proposed InIm display.


the spacing w. Since the magnification Mx is 5, twolens rows per boundary need to be reorganized.Every numbered portion is traceable from the CDPto elemental images. As expected, the elemental im-age near the boundary contains the mirror folded re-gion. In this elemental image, two separate portionsof it represent the same positions on the CDP withtwo different perspective views. Then the same num-bers appear again in one elemental image. In this pa-per, viewpoint vector rendering is applied to generateraw elemental images [13]. It is the computer gen-

eration algorithm based on directions of perspec-tives. Therefore, the location of the field of view iseasily defined and folding the elemental images isclearly calculated.

The proposed system is embodied with a Fresnellens array and the first surface mirrors. Figure 10shows the experimental setup. The lens array is com-posed of 13 × 8 lenses with a focal length of 22mmand a spacing of 10mm. The set of elemental imageshas a 1755 × 1080 pixel resolution, which is projectedby a full high-definition (full HD) projector. The mag-nificationMx is five and the resultant length dmirror ofthe boundary folding mirrors is 132mm.

Figure 11 shows a set of elemental images reorga-nized for the proposed InIm display. Here, the dashedlines separate the set of elemental images into indi-viduals and the solid lines are the borders of the mir-ror folded regions. As previously discussed, in theelemental images near the boundaries, one object ap-pears twice. For example, there are two blue stripedballs in the far left elemental images.

Figure 12 shows the perspective views of the em-bodied system. Each image is the view depending onthe position of the CCD. The dashed rectangles are

Fig. 8. (Color online) Movement of field of view by boundary fold-ing mirrors: (a) field of view in the conventional InIm display and(b) field of view in the proposed InIm display.

Fig. 9. (Color online) Reorganization relation in generating set ofelemental images.

Fig. 10. (Color online) Experimental setup.

Fig. 11. (Color online) Reorganized set of elemental images forthe proposed InIm display.


the areas of the lens array unfolded by mirrors. Thatis, the outside regions of the dashed rectangle are thereflected images. In the conventional InIm, it is im-possible to avoid the reduction of area displaying theimage for the observer positioned apart from the cen-ter. And the conventional technique cannot displaythe outsides of the dashed rectangles. In Fig. 12, themaximum width of the outside region is two timesthe lens spacing. This experimental result agreeswith the calculation that the width wfolded of the mir-ror folding region is twice as wide as the lens spacingw from Eq. (11).

4. Conclusion

A uniform angular resolution InIm display withboundary folding mirrors is proposed. By this pro-posed method, the angular resolution near theboundaries is complemented with the undesirableperspectives outside the boundaries and the bias ofthe perspectives around the boundaries is removed.Therefore, the field of view in the proposed display ismoved and its cross section on the CDP becomeslarger than that in a conventional system. It is desir-able to display the field of view over the full spatialresolution. By experiments, the proposed InIm dis-play is confirmed to form a uniform angular resolu-tion and the field of view is displayed in agreementwith the design parameters. It is expected that theproposed technique can be one of the most effectivetechniques for compensating the decrease of angularresolution in the boundaries of the InIm display.

This work was supported by the Korea Science andEngineering Foundation (KOSEF) and the Ministryof Education, Science and Engineering of Koreathrough the National Creative Research InitiativeProgram (R16-2007-030-01001-0).

References1. B. Lee, J.-H. Park, and S.-W. Min, “Three-dimensional display

and information processing based on integral imaging,” inDigital Holography and Three-Dimensional Display,T.-C. Poon, ed. (Springer, 2006), pp. 333–378.

2. B. Javidi and F. Okano, eds., Three Dimensional Television,Video, and Display Technologies (Springer, 2002).

3. H. Liao, M. Iwahara, N. Hata, and T. Dohi, “High-quality in-tegral videography using a multiprojector,” Opt. Express 12,1067–1076 (2004).

4. T. Georgiev, K. C. Zheng, B. Curless, D. Salesin, S. Nayar,and C. Intwala, “Spatio-angular resolution tradeoffs inintegral photography,” in Proceeding of the 17th Euro-graphics Workshop on Rendering (Eurographics, 2006),pp. 263–272.

5. A. Stern and B. Javidi, “3-D computational synthetic apertureintegral imaging (COMPSAII),” Opt. Express 11, 2446–2451(2003).

6. M. Hain, W. von Spiegel, M. Schmiedchen, T. Tschudi, andB. Javidi, “3D integral imaging using diffractive Fresnel lensarrays,” Opt. Express 13, 315–326 (2005).

7. S.-W. Min, J. Kim, and B. Lee, “Wide-viewing projection-typeintegral imaging system with an embossed screen,” Opt. Lett.29, 2420–2422 (2004).

8. Y. Kim, J.-H. Park, S.-W. Min, S. Jung, H. Choi, and B. Lee,“Wide-viewing-angle integral three-dimensional imagingsystem by curving a screen and a lens array,” Appl. Opt.44, 546–552 (2005).

Fig. 12. (Color online) Perspective views of the proposed InIm display: (a) upper left view, (b) upper center view, (c) upper right view,(d) middle left view, (e) middle center view, (f) middle right view, (g) lower left view, (h) lower center view, and (i) lower right view.


9. T. Balogh, T. Forgács, T. Agocs, O. Balet, E. Bouvier, F. Bettio,E. Gobbetti, and G. Zanetti, “A scalable hardware andsoftware system for the holographic display of interactive gra-phics applications,” in Eurographics Short Papers Proceedings(Eurographics, 2005), pp. 109–112.

10. T. Agocs, T. Balogh, T. Forgacs, F. Bettio, E. Gobbetti,G. Zanetti, and E. Bouvier, “A large scale interactive holo-graphic display,” in Proceedings of IEEE Conference on VirtualReality (IEEE, 2006), p. 311.

11. Y. Takaki, “High-density directional display for generatingnatural three-dimensional images,” Proc. IEEE 94, 654–663(2006).

12. J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickupmethod of both virtual and real objects for integral imaging,”Opt. Express 16, 13969–13978 (2008).

13. S.-W. Min, K.-S. Park, B. Lee, Y. Cho, andM. Hahn, “Enhancedimage mapping algorithm for computer-generated integralimaging system,” Jpn. J. Appl. Phys. 45, L744–L747 (2006).


Uniform angular resolution integral imaging display with...

Documents

Transcript of Uniform angular resolution integral imaging display with...