Exploring 3D Virtual Environments Through Op8mised Spherical Panorama

Naviga8on

The applica8on goal is to provide a lightweight environment to be deployed in a web browser for the interac(ve naviga(on and browsing of an ordered sequence of spherical images.

An example of a real world panorama: (a) View of the current panorama (zoom 1:1) aligned to the outward direc8on; (b) next panorama (zoom 1:1) aligned to the inward direc8on.

(a) (b)

The rst of the two key features of the system is to enable a smooth transi(on between the adjacent panoramas through an algorithm of sub-scene matching.

(a) Opera8ng a zoom on the current panorama results into a reduc8on of the size of the rectangular area corresponding to the part of scene that is viewed. By zooming the current panorama in the outward direc8on (in (b) the zoom level is 1:2) the content of the sub-scene P0 can beRer approximate the content of the next panorama.

(a) (b)

The iden8ca8on of the rectangular sub-region of an image Ik that best matches the content of Ik+1 in correspondence to the inward direc8on is accomplished by minimiza8on of the following cost func(on with respect to the three variables (x, y, ) (x0, y0, 0) = arg[minx,y, D(f(x, y, ), Ik+1)] where: - f(x, y, ) = Crop(Ik, x, x + wk+1, y, y + hk+1) - wk and hk are the width and height in pixels of the image Ik; - Crop(I,x,x,y,y) returns the rectangular sub-region of the image I; - D(In,Im) returns the dissimilarity between the content of two images In and Im; - (xo, yo, o) denes the coordinates of the boRom-leZ vertex of the sub-region Po = (xo , yo) and the zoom factor Zo = 1/o to be applied to the sub-region of Ik in order to match the size of Ik+1.

5

Some results: in the rst column images of the user current view are presented; in the second the best matching sub-region of the current image with the op8mal zoom factor for the transi8on is shown; nally, in the third column the next panorama is visualised.

We experimentally observed that computa8on of the dissimilarity func8on D(In,Im) through the distance of the image histograms provi- des higher eec(veness compared to solu8ons based on scale invariant local keypoint descriptors such as SIFT.

Sub-scene matching results

The second key feature of the system is an innova(ve interac(on metaphor for a beRer frui8on of mul8media materials associated to interac(ve hotspots related to Point Of Interests (POIs). These are distributed along a cultural walk-through and include 3D models, PDFs, galleries of images/videos and indoor panoramas

Visualisa8on of an interac8ve indoor panorama in the 3D space

- The interface is built using HTML5, CSS3, javascript and WebGL; - naviga8on among spherical images is obtained replacing a texture mapped into a sphere taking into account the op8mal zoom factor es8mated by the sub-matching module; - POIs infos and associated mul8media materials are shown in oa(ng panels in the 3D space overcoming the limita8on of standard interfaces that use 2D light-boxes zoom factor computed by the sub-scene matching module.

Visualisa8on of an interac8ve outdoor panorama in the 3D space

Hotspots associated interac8ve mul8media

material can be accessed by a circular menu

represented by four icons for the dierente

types: PDF, image and video gallery, 3D

object, nested panorama.

The system automa8cally provides the best

visual arrangement of mul8media materials

with respect to the user point-of-view and

allows the user to drag the hotspots in order

to customise the disposi8on of elements in

the 3D space.

Mini-maps are provided allowing contextual

awareness.

Hot-spost can be draggled in 3D space

Interac8ve mini-map showing hot-spots and user posi8on and orienta8on

The demo will show an interac(ve route based on spherical photos which starts from the current Via della Ninna near the Uzi Gallery, to the church of San Leonardo in Arcetri in Florence (IT), giving the possibility to experience an important, though liRle-known mul8media virtual i8nerary of cultural and historical heritage.

The church of San Leonardo in Arcetri

The famous romanesque pulpit inside San Leonardo

The remains of the church of San Pier Scheraggio

in Via della Ninna