MIXED FANTASY: Exhibition of Entertainment … FANTASY: Exhibition of Entertainment Research for...
2
MIXED FANTASY: Exhibition of Entertainment Research for Mixed Reality Christopher B. Stapleton IST & Digital Media Media Convergence Lab Univ. of Central Florida [email protected] Charles E. Hughes Computer Science Media Convergence Lab Univ. of Central Florida [email protected] J. Michael Moshell Digital Media Media Convergence Lab Univ. of Central Florida [email protected] Abstract This exhibition presents a Mixed Reality experience that applies basic research to the media industries of entertainment, training and informal education. Contributing areas of research include tracking, registration, rendering, real-time and distributed algorithms, and adaptations of existing as well as the creation of new forms of artistic convention. 1. Introduction: Why Entertainment? The power of entertainment stretches far beyond venues for amusement. Entertainment is a language unto itself affecting world culture, world economy and even world opinion. Entertainment is able to excite, motivate, satiate, communicate and inspire. This power is being applied to all aspects of life including learning, training, designing, communicating and collaborating for purposes ranging from social engagements to high powered teams making world decisions. The power of Mixed Reality provides us with the ability to combine the fantastical worlds of the imagination through virtual reality and seamlessly merge these with the compelling aspects of reality. MR envelops the most important aspect of reality- -other people. The participants bring the most important part of Mixed Reality—the imagination. The Media Convergence Laboratory’s exhibition of entertainment research for Mixed Reality explores the convergence of art, science and technology to create compelling venues and content prototypes to engage the audience’s imagination. The end results become catalysts for cross-industry innovation of Mixed Reality entertainment. Being exhibited is MCL’s MR immersion booth demonstrating three content pieces applied to three different MR applications of entertainment -- amusement, informal education and training. All entertainment experiences consist of three aspects (figure 1): the venue, the content, and the use of artistic convention to capture and engage the imagination through story. These three components represent the areas of multidisciplinary research being conducted in Mixed Reality to extend Milgram’s Reality-Virtuality Continuum to include the interactive imagination [1,2]. Figure 1 Mixed Fantasy Continuum 2. Experience: Content, Venue and the Interactive Imagination. Venue: The MR Expo experience (figure 2) will include guests entering and viewing the MR world from multiple entrances around the Immersion MR Booth kiosk. Guests will have three levels of interactivity to choose from including 1) Interactive Participant (Diver) who will suit up with the MR HMD and interface, 2) Interactive Spectator (Swimmer) who will manipulate virtual elements and trigger events, and 3) Spectator (wader) who will be observing, but not influencing the outcome. The technical infrastructure is laced with multi-modal special effects driven by a central show control to activate the physical environment to correspond to virtual elements (e.g., physical objects can be knocked over based on interaction with virtual objects). Content: The versatility of the venue will be demonstrated through the flexibility and interchangeability of the content. Three different content pieces will be used to serve multiple applications of training, informal education and experience-based entertainment. The first piece (figure 3), Cetacean Communication, was developed in collaboration with the MR Systems Laboratory, Canon Inc. for application to Infotainment industry (marketing, conventions and conferences). Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR ’03) 0-7695-2006-5/03 $17.00 © 2003 IEEE
Transcript of MIXED FANTASY: Exhibition of Entertainment … FANTASY: Exhibition of Entertainment Research for...
MIXED FANTASY: Exhibition of Entertainment Research for Mixed Reality
Christopher B. Stapleton IST & Digital Media
Media Convergence Lab Univ. of Central Florida
Charles E. Hughes Computer Science
Media Convergence Lab Univ. of Central Florida
J. Michael Moshell Digital Media
Media Convergence Lab Univ. of Central Florida
Abstract This exhibition presents a Mixed Reality experience
that applies basic research to the media industries of entertainment, training and informal education. Contributing areas of research include tracking, registration, rendering, real-time and distributed algorithms, and adaptations of existing as well as the creation of new forms of artistic convention.
1. Introduction: Why Entertainment?
The power of entertainment stretches far beyond venues for amusement. Entertainment is a language unto itself affecting world culture, world economy and even world opinion. Entertainment is able to excite, motivate, satiate, communicate and inspire. This power is being applied to all aspects of life including learning, training, designing, communicating and collaborating for purposes ranging from social engagements to high powered teams making world decisions. The power of Mixed Reality provides us with the ability to combine the fantastical worlds of the imagination through virtual reality and seamlessly merge these with the compelling aspects of reality. MR envelops the most important aspect of reality--other people. The participants bring the most important part of Mixed Reality—the imagination.
The Media Convergence Laboratory’s exhibition of entertainment research for Mixed Reality explores the convergence of art, science and technology to create compelling venues and content prototypes to engage the audience’s imagination. The end results become catalysts for cross-industry innovation of Mixed Reality entertainment. Being exhibited is MCL’s MR immersion booth demonstrating three content pieces applied to three different MR applications of entertainment -- amusement, informal education and training.
All entertainment experiences consist of three aspects (figure 1): the venue, the content, and the use of artistic convention to capture and engage the imagination through story. These three components represent the areas of multidisciplinary research being conducted in Mixed Reality to extend Milgram’s Reality-Virtuality Continuum to include the interactive imagination [1,2].
Figure 1 Mixed Fantasy Continuum
2. Experience: Content, Venue and the Interactive Imagination.
Venue: The MR Expo experience (figure 2) will include guests entering and viewing the MR world from multiple entrances around the Immersion MR Booth kiosk. Guests will have three levels of interactivity to choose from including 1) Interactive Participant (Diver) who will suit up with the MR HMD and interface, 2) Interactive Spectator (Swimmer) who will manipulate virtual elements and trigger events, and 3) Spectator (wader) who will be observing, but not influencing the outcome.
The technical infrastructure is laced with multi-modal special effects driven by a central show control to activate the physical environment to correspond to virtual elements (e.g., physical objects can be knocked over based on interaction with virtual objects).
Content: The versatility of the venue will be demonstrated through the flexibility and interchangeability of the content. Three different content pieces will be used to serve multiple applications of training, informal education and experience-based entertainment. The first piece (figure 3), Cetacean Communication, was developed in collaboration with the MR Systems Laboratory, Canon Inc. for application to Infotainment industry (marketing, conventions and conferences).
Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR ’03) 0-7695-2006-5/03 $17.00 © 2003 IEEE
Figure 2 MR Fantasy ISMAR 2003 floor plan
Figure 3 Cetacean Communication ©2003 MCL/UCF
The second piece (Figure 4), Warrior Games, was
developed in conjunction with the U.S. Department of Defense and major US entertainment companies. The third piece (Figure 5), MR Museum, was developed in conjunction with the UCF Biology Department and the Orlando Science Center to showcase the use of interactive visualization applied to informal education.
Story: The MR Story Engine integrates the venue and content elements to help seamlessly merge the real and physical worlds. The interactive logic uses evolving standards of interactive narrative rhetoric devised from an
array of more traditional interactive and passive media conventions.
Figure 4 Warrior Games ©2003 MCL/UCF
Figure 5 MR Museum ©2003 MCL/UCF
3. Research: Science and Technology
The primary scientific contributions of this research lie in the creation of real-time algorithms for illumination (virtual objects reacting to changing real world lighting conditions), level-of-detail control (graphics hardware assisted dynamic computation of occlusion-based LOD) and depth from stereo (also graphics hardware assisted). The technology contributions lie in the creation of an integrated system that controls and synchronizes visual, audio and special effects based on the story-driven behaviors of virtual objects, both reactive (responding to user activities) and proactive (instigating actions based on internal goals). We have also developed novel software to dynamically control audio (3d, point source and ambient) and special effects (lights turning on and off, bottles jumping in the air, etc.)
4. References
[1] P Milgram et al., “Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum,” Proceedings of SPIE: Telemanipulator and Telepresence Technologies, vol. 2351, Boston, MA, 1994, pp. 282-292. [2] C. B. Stapleton et al., “Applying Mixed Reality to Entertainment,” IEEE Computer 35(12), Dec. 2002, pp. 122-124.
-- Retro-Reflective Chroma-Key Panel-- Virtual Scenery
Spectators
--Overhead
Team 1 Team 2
Interactive Spectator Station
Interactive Participant Station
Interactive Spectator Station
Spectators
Spectators
Interactive Participant Station
Spectators
Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR ’03) 0-7695-2006-5/03 $17.00 © 2003 IEEE
