SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
-
Upload
sebastien-kuntz -
Category
Technology
-
view
111 -
download
5
description
Transcript of SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
![Page 1: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/1.jpg)
Sébastien ‘Cb’ Kuntz @Cb_VRGeek
Philosophy and architecture!SEARIS 2014
SEARIS: Software engineering and architecture in interactive systems, an IEEE VR workshop.
![Page 2: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/2.jpg)
When I was in engineering school, we had software architecture classes and projects. Without any experience, this is a very abstract topic When I was an intern at Discreet (Autodesk, 3DS Max, Flame, Inferno, movies sfx), was very impressed with the “architect”, and wanted to be like him ! Turns out it takes a lot of experience to be a good architect, so share my experience architecting MiddleVR and show that the philosophy that you choose, consciously or not, actually drives your architecture
![Page 3: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/3.jpg)
Plan
• Introduction!
• Origin of MiddleVR!
• Goals!
• Demo!
• Philosophy & Architecture!
• Conclusion
![Page 4: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/4.jpg)
(French railways)
“Creating Virtual Reality Games:!
The Fundamentals”
Founder & President
Sébastien ‘Cb’ Kuntz
Also board member of the national French VR association AFRV (http://www.afrv.fr) and founded the VR Geeks association (http://www.vrgeeks.org)
![Page 5: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/5.jpg)
And it seems I have been interested in head-mounted displays for longer that I remember.
![Page 6: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/6.jpg)
Zone hunter (Virtuality-1994)
I fell in love with VR on a field trip in London when I was a teenager. The experience was far from perfect: the HMD was bulky, one display was blurry, but being able look around and shoot where I wanted incredible. I quickly decided I wanted to make a career in VR.
![Page 7: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/7.jpg)
FRENCH RAILWAYS - SNCF
• Polhemus magnetic tracker, treadmill, 5DT Glove • C++ / OpenSceneGraph / VRPN / OpenAL
Fiacre: train the operators at maintenance of equipments on the railway tracks. I was in charge of all the VR engine and application development !(sorry I can’t put SNCF videos online)
![Page 8: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/8.jpg)
FRENCH RAILWAYS - SNCF
Simurat: check wagons before they leave on the tracks
![Page 9: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/9.jpg)
VIRTOOLS
After 4 years I was hired by Virtools as their VR lead engineer. Virtools was a great tool to create 3D applications. Simple to learn thanks to the building blocks programming scheme, and many non-programmers, like designers and even farmers, created great applications.
![Page 10: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/10.jpg)
VIRTOOLS VR PACK
I was in charge of adding VR functionality's to Virtools engine, coding, writing documentation, support customers
![Page 11: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/11.jpg)
I often say that we are at the prehistory of VR. Last week was the Big bang of VR. I could test Oculus DK2, Sony Morpheus and Valve’s HMD. The future is coming quickly.
![Page 12: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/12.jpg)
• VR tools to simplify VR
![Page 13: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/13.jpg)
Virtual reality
Presence in a virtual world
Interesting because: - natural interactions: no interface to learn. if we want to look under a table, no button to press, just move your body. - natural reactions & emotions : if I throw a ball at you, you will have the reflex to grab it. if you are afraid of heights, you will have this fear in VR.
![Page 14: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/14.jpg)
Ridiculous
If it’s real VR, you probably
look ridiculous !
![Page 15: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/15.jpg)
• Build hardware
To create and maintain presence, you need to ...
• Build software
• Build user experience
Most people forget about taking time to build the user experience. Test with users !
![Page 16: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/16.jpg)
The challenges of creating a VR software
We already have great hardware, but we don’t know how to make the most of it.
![Page 17: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/17.jpg)
“VR is mainly a software problem”!!Jaron Lanier! !
Software is like a fuel for your car. Without any fuel, you will not go very far, even if you have a Porsche.
![Page 18: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/18.jpg)
Device management
Physical device!!
Driver Application
An application needs to get the data from a physical device through a driver.
![Page 19: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/19.jpg)
Device management
Tracker 1! Driver Tracker 1
Application
Tracker 2!!
Driver Tracker 2
The problem is that each driver gives access to similar data in a different way so you need to have specific part of programs for similar drivers.
![Page 20: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/20.jpg)
code for one driver if you switch to another device, another driver:
![Page 21: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/21.jpg)
This is a completely different code which gives exactly the same information but for another device.
![Page 22: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/22.jpg)
And there are a lot of devices out there.
![Page 23: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/23.jpg)
Correct perspective
(c) Johnny Lee
https://www.youtube.com/watch?v=Jd3-eiid-Uw !3D applications assume that the user is in front of the screen. If you move in front of the screen, it looks like a 2d painting !If the application knows where the user is, the screen becomes a window on the virtual world (perceived depth without stereoscopy)
![Page 24: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/24.jpg)
Correct perspective
In general 3D engines allow you to do that but you have to do all the maths by yourself Necessary for every big VR wall, anytime stereoscopy, HMDs, Caves... !
![Page 25: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/25.jpg)
Stereoscopy
Even after you have computed your stereo images, you have different ways to display them. It can be active stereo, passive stereo, autostereo etc.
![Page 26: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/26.jpg)
Warping & Blending
Then if you have a curved screen, your perspective will have to be distorted.
![Page 27: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/27.jpg)
Warping & Blending
Warping is about getting a correct perspective for screens that are not planar (curved) Blending is about managing the overlapping regions between several projectors.
![Page 28: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/28.jpg)
Graphics cards limitations
• Number of outputs!
• Processing power
![Page 29: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/29.jpg)
multiple graphics cards
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines are not optimized for multi-pipe
![Page 30: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/30.jpg)
Cluster
• Pro!
• More outputs!
• More processing power!
• Con!
• 3D engines don’t support clustering
![Page 31: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/31.jpg)
• No seams across displays require 3 layers of sync!
• 3D objects synchro (Scenelock)!
• New images display (Swaplock)!
• Left/Right eyes (Genlock)
Cluster
![Page 32: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/32.jpg)
Interactions
• Many different 3D user interactions!
• Navigation!
• Selection!
• Manipulation!
• Draw!
• System control!
• etc.
![Page 33: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/33.jpg)
Interactions are dependent on hardware
For example navigation will be different for different VR systems
![Page 34: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/34.jpg)
Menus
Menus are also a big issue in VR.
![Page 35: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/35.jpg)
Haptics (force feedback)
If you need haptics, you will have to have a physics engine that run at 1000hz and interface with specific hardware.
![Page 36: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/36.jpg)
Deployment
• Modify application to!
• Change drivers!
• Change displays!
• Change interactions
![Page 37: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/37.jpg)
So to create a complete VR application, you have to manage all of this !
![Page 38: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/38.jpg)
• Platform for creating interactive 3d applications!
• Widely used!
• 2.5 million registered developers
![Page 39: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/39.jpg)
• High-quality rendering!
• Deferred renderer!
• Occlusion culling!
• Realtime shadows!
• Ambient occlusion!
• Light mapping
![Page 40: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/40.jpg)
![Page 41: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/41.jpg)
Unity 3D Features• Programming!
• C#, Javascript!
• Profiler!
• Physics!
• NVidia PhysX!
• Cloth!
• Soft/rigid bodies!
• Ragdolls!
• Cars
3D formats !
FBX, Collada, 3DS, OBJ…!
Audio engine!
Networking!
Terrain
![Page 42: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/42.jpg)
Editor
• Simple to use!
• Create your own tools inside Unity
![Page 43: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/43.jpg)
Asset store
• Buy / sell assets!
• 3D models!
• Textures!
• Shaders!
• Scripts!
• Editor plugins
![Page 44: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/44.jpg)
Desktop
Unity is a great tool but it doesn’t natively support VR systems easily.
![Page 45: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/45.jpg)
• Build hardware
To create and maintain presence, you, the author, need to ...
• Build software
• Build user experience
Our goal is to minimize the time you spend on building the software so that you can focus on building the user experience.
![Page 46: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/46.jpg)
In my mind, VR is not that difficult. Doesn’t have to be. A part is that the hardware is hard to use, but a bigger part is that the software is hard to use mostly by design.
![Page 47: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/47.jpg)
In 2010 sent an e-mail to Roland Blach asking for all the SEARIS papers and studied them attentively
Software Engineering and Architectures for Realtime Interactive Systems Working Group
![Page 48: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/48.jpg)
Existing VR software
• VRPN!
• OpenTracker!
• VRJuggler!
• Virtools!
• Worldviz!
• …
![Page 49: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/49.jpg)
VRPN
In VRPN, you first have to edit a configuration file, understandable by computer scientists only. Then you have to understand how it works: callbacks, terminology is also a bit obscur (analog..) VRPN is only solving the devices management, and as we have seen, there are a lot of other issues when creating a VR app.
![Page 50: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/50.jpg)
VR Juggler
VR Juggler needs to understand a lot of concept: proxy device? derive of off OpenGL::App ? My main problem is that you have to bend your application to their design
![Page 51: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/51.jpg)
VIRTOOLS VR PACK
VRPack’s configuration file is even more complex than VRPN. Even working at Virtools, it took me quite a while to understand how it works ! Switching configuration files is also complicated.
![Page 52: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/52.jpg)
Worldviz Vizard
• Based on OpenSceneGraph!
• Python
Vizard is a really nice environment but still a bit too complex to use.
![Page 53: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/53.jpg)
Worldviz Vizard
For example this is how you would configure trackers and a Cave.
![Page 54: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/54.jpg)
The need for better VR software tools
• Hardware getting cheaper!
• But engineers are not getting cheaper!
• Need to be more efficient and spend less time creating apps!
• Democratization!
• VR developers are not all engineers anymore
![Page 55: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/55.jpg)
Goals
• Simplify VR!
• Simplify creation of VR applications!
• Simplify deployment of VR applications!
• Create new VR tools!
• Adapting existing 3D applications
![Page 56: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/56.jpg)
What is ?
• Generic VR plugin (C++ SDK) !
• Functionalities ○Manage input devices
•3D trackers, keyboard, mouse, joystick ○Manage display
•Viewports, cameras, stereoscopy, cluster ○High-level building-blocks ○Interactions ○Menu
CaveLib - VRJuggler
![Page 57: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/57.jpg)
Architecture
MiddleVR (C++)
Drivers OpenGL DirectX Network
C# Wrapper
GUI UnityLua
WrapperPython
Wrapper
![Page 58: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/58.jpg)
GUI to easily configure any VR system quickly.
![Page 59: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/59.jpg)
Tutorial: https://www.youtube.com/watch?v=EEKH-1EilXs
![Page 60: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/60.jpg)
Once you have exported your application, you can simply add it to the list of Simulations in MiddleVR’s gui. Then you can pick any configuration and simply click Run and your application will be dynamically reconfigured for this configuration
![Page 61: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/61.jpg)
https://www.youtube.com/watch?v=yyR6w43hgSo
![Page 62: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/62.jpg)
MiddleVR Features
![Page 63: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/63.jpg)
Supported Devices• Via VRPN :
• A.R.T • Vicon • Optitrack • Wiimote !
• Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers,
• Ascension Flock of Bird • Intersense IS600, IS900 • 3dTech HiBall-3000 Wide Area Tracker, • Advanced Realtime Tracking Gmbh DTrack
Client, • World Viz Precision Position Tracker PPT
1.2, • Natural Point Optitrack Rigid Body Toolkit, • Phasespace optical tracking OWL, • ...
• Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint
Fusion • GameTrak • Razer Hydra • SpaceMouse • Motion Analysis (beta)
![Page 64: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/64.jpg)
Haptics (force feedback)
https://www.youtube.com/watch?v=lnoE_VclnOI Haption IPSI
![Page 65: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/65.jpg)
VR Camera• MiddleVR computes all the cameras’ parameters
automatically!
• Correct perspective!
• Scale 1
![Page 66: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/66.jpg)
Stereoscopy
• Active stereoscopy (OpenGL Quad-Buffer)!
• Which Unity can’t natively handle!
• Passive stereoscopy
![Page 67: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/67.jpg)
Viewports management
Viewports configuration
![Page 68: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/68.jpg)
Simply configure your system
Configure what is happening in the real life. Seeing things happening in real life changes a lot. Can easily check - axes swap, offsets - screens are correctly placed - if tracking is covering the area !Allows you to experiment.
![Page 69: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/69.jpg)
Navigations
We provide standard navigations: - classic joystick navigation - elastic navigation - grab the world
![Page 70: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/70.jpg)
Homography
A nice new feature is the homography. This allows you to grab corners of your viewports and have them match the physical corners of your cave.
![Page 71: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/71.jpg)
Cluster & Multi-pipe
• Scenelock!
• Swaplock!
• Genlock!
• Multi-pipe (soon)
Regarding clustering, one unity player is running on each computer. SceneLock: synchronise the state of all input devices and position/orientation of some specific objects. SwapLock: implemented our own mechanism Genlock: done in HW
![Page 72: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/72.jpg)
Supported displays
• Virtually any HMD • VR-Wall, Powerwall • Workbench, Holobench • HoloStage • CAVEs • 3D TVs • zSpace • Igloo Vision • …
![Page 73: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/73.jpg)
Goals
• Simplify creation of VR applications!
• Simplicity of adding VR into application!
• Simplicity of API!
• High-level tools!
• Interactions: Selection, manipulation, navigation, menus, GUIs
![Page 74: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/74.jpg)
Example of device management
![Page 75: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/75.jpg)
Driver 1
![Page 76: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/76.jpg)
Driver 2
![Page 77: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/77.jpg)
Device management
Replaced manual edition of cfg file by GUI Simply choose the driver from the list, don’t need to rewrite it.
![Page 78: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/78.jpg)
Looks like english ! No need to understand what a sink is (opentracker), or forced to set callbacks.
![Page 79: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/79.jpg)
MiddleVR
Here is the minimum code you have to write to use MiddleVR. You don’t have to modify the structure of your application to integrate MiddleVR !
![Page 80: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/80.jpg)
Class reference
Most VR systems out there can be described with these highlighted simple, atomic, concepts
![Page 81: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/81.jpg)
The value of Simplicity
![Page 82: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/82.jpg)
The value of simplicity
“Perfection is achieved, not when there is nothing more to add, but when there is
nothing left to take away”!
Antoine de St Exupery
Simplicity is a huge principle for me. We don’t hesitate to remove code/feature that are covering way too much. EVEN IF ALREADY WRITTEN !! We’re a Startup, we don’t have resources, so we have to be simple No resources is a strength Lots of resources can actually be a curse
![Page 83: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/83.jpg)
The value of simplicity
• For each feature you add!
• Test!
• Document!
• Support / maintain!
• Each feature costs !!
Each feature you add, you have to support it for the entire life of the product. Simplicity is harder to achieve, so cost more at first, but in the long run is beneficial
![Page 84: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/84.jpg)
The value of simplicity• Huge value in simplicity !!
• User side!
• Faster to learn, less to remember!
• Developer side!
• Faster to code!
• Easier maintain!
• Less bugs!
• Business side!
• Need less coders!
• More interesting features!
• Easy download / setup > Quickly validate on user’s hardware !!
• Easier support
![Page 85: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/85.jpg)
API in simple english• Long discussions for naming of functions, classes, variables!• Consistent api!
• SetNumButtons, SetButtonsNum, SetNbButtons > SetButtonsNb!• French not good at english!
• Analog ? Digital ? Proxy ?!• Use natural terms depending on device!
• Joystick->GetAxisValue()!• Joystick->IsButtonPressed()!• Keyboard->IsKeyPressed()!
• Why “Register” ? Why not “Add” ?!• Why so complex, want to sound very clever ?!• AddDevice, AddCamera, …!
• API in degrees, because more intuitive / simple to work with
![Page 86: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/86.jpg)
Simplified API
• Simple but complete API!• Like legos!• Rotate!
• Node3D->Rotate( Quat )!• Node3D->Rotate( Euler )!• Node3D->Rotate( Axis, Angle)!• Quat( quat ), Quat( Euler ), Quat( Axis, Angle)
Simplified API: only one way of doing things: less to learn, less to remember Everything you need is in the Quat class, so it’s reusable in any part of the code
![Page 87: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/87.jpg)
The value of simplicity
“Simplicity is the ultimate sophistication”! DaVinci
Fight hard to create / maintain simplicity. Engineers come and implemented what was in their mind, anticipate cases that would rarely happen. We do code reviews. Often rewrite often 3 times before acceptable.
![Page 88: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/88.jpg)
The value of simplicity• Design user interface for the 80%!
• Don’t ruin the experience for the 20%!
• Simple things should be simple!
• Complex things can be complex!
• Put in an “advanced menu”!
• Don’t let beauty of design ruin user experience!
• Like real life architects who want their design to be beautiful but totally unusable on a day to day!
• Example of Lego!
• Simple but complete
Tell horror network socket story :) !Current tools designed for geeks/engineers (examples vrpn/virtools vr)
![Page 89: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/89.jpg)
Optimizations
• Engineers have a strong tendency to over-optimise!
• Complexifies code!
• Code must be above all readable!
• To avoid bugs!
• To maintain maintenability!
• If you want to optimise!
• First profile your code!
• Optimize what is *really* taking time!
• not what you *think* takes time
![Page 90: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/90.jpg)
Empathy
“The ability to understand and share the feelings of another.”!
!
Test with users
• if user don’t understand, not add more doc,!
• simplify !!
![Page 91: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/91.jpg)
Hardware SDK are mostly too complicated
!Even Oculus !!!!! Less time to implement haptics !!!
![Page 92: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/92.jpg)
Drivers
• Why not just a few simple functions with sensible defaults ?!
• Init()!
• Update()!
• GetValue()!
• Destroy()!
• And add more functions for more advanced
HW manufacturers don’t know how to write an API
![Page 93: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/93.jpg)
Documentation
• User Guide!
• Class references
Of course the ideal tool would not need documentation, I honestly like to write documentation. It helps spot errors, inconsistencies. If it’s hard to document, the feature must probably must be simplified instead of writing a more sophisticated doc !!
![Page 94: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/94.jpg)
Goals
• Simplify deployment of VR applications!
• GUI to configure VR systems!
• Dynamic reconfiguration of application!
• Ship with standard configurations!
• Abstractions
![Page 95: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/95.jpg)
Deployment
• Abstractions!
• Devices!
• 3D Nodes (User)!
• Interactions (soon)
![Page 96: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/96.jpg)
Device management
Driver Tracker 1
Application
Driver Tracker 2
** The problem is that each driver gives access to similar data in a different way so you need to have specific part of programs for similar drivers.
![Page 97: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/97.jpg)
Device abstraction
Virtual Device Application
Driver Tracker 1
Driver Tracker 2
A virtual device only stores data. For tracker, position+orientation Drivers write data in the virtual device. Allow to change driver without application (PORTABILITY) --- Replay allows you to playback a recorded working session. It enables quality insurance because you can give the same input to your application and check that the output is always the same.
![Page 98: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/98.jpg)
Device management
Driver
Virtual Device Application
Simulation
Replay
Replay allows you to playback a recorded working session. It enables quality insurance because you can give the same input to your application and check that the output is always the same.
![Page 99: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/99.jpg)
Display
• Cameras & viewport created dynamically
![Page 100: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/100.jpg)
User• Human-centric rather than techno-centric
Rather than accessing devices, we could simply ask the position of the user’s head, or hand ! This allows to write scenarios like : What does the user see ? Is the hand close to ... ?
![Page 101: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/101.jpg)
Virtual Device 1
Hand (3D Node) Application
Virtual Device 2
Inverse Kinematics
User abstraction
![Page 102: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/102.jpg)
Interactions• MiddleVR includes high-level interactions!
• Navigation!
• Selection!
• Manipulation!
• … More to come!
• Future: Interactions abstractions!
• Dynamically change interactions based on hardware
![Page 103: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/103.jpg)
Interactions abstraction?
Grab-air
Navigation
Application
Point
Steering
Selection
Manipulation
Raycasting
Go-go
Hand
Homer
Hand
WIM
![Page 104: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/104.jpg)
• How to create portable interactions ?!
• Navigations can be portable!
• Except collisions!
• Generalise what is generalisable!
• + Custom part for engine dependent!
• Collisions!
• Highlights!
• Gizmos
Interactions
![Page 105: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/105.jpg)
Menus
• Ability to create menus and GUIs in HTML5 (soon)
![Page 106: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/106.jpg)
![Page 107: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/107.jpg)
Create immersive HTML5 GUI
HTML 5 gui in VR
![Page 108: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/108.jpg)
C++ Reflexion
• Also for configuration file!
• Cluster synchronisation!
• Accessing drivers specific information
Property is simple couple of getter/setter. In header write naturally Set/GetParent !One line to add property in source code.
![Page 109: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/109.jpg)
Working with multiple 3D engines
• Different coordinate systems!• vrCoordinateSystem( Front, Up, Right )!
• +/- X,Y,Z!• OpenGL:!
• Front= -Z!• Up = +Y!• Right = +X!
• DirectX:!• Front=+Z!• Up=+Y!• Right=+X
![Page 110: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/110.jpg)
Working with multiple 3D engines
• C++!
• Compiling!
• STL, Boost!
• Strings (Wide strings)!
• Link issues!
• Different compilers (VC2005/2008/2010/2012)!
• Different languages!
• C wrapper!
• Lua wrapper
![Page 111: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/111.jpg)
!
• Simplicity!• Creation!• Deployment!
•Adaptability!•Hardware!• Software
![Page 112: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/112.jpg)
Our future
• Games/apps development:
![Page 113: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/113.jpg)
Conclusion
Talked about the philosophy and architecture You see how deeply the philosophy impacts the architecture Simplicity at its core which has a huge value for everybody, but has to fight hard for it.
![Page 114: SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture](https://reader033.fdocuments.in/reader033/viewer/2022051412/54c8033d4a7959037f8b458b/html5/thumbnails/114.jpg)
http://www.imin-vr.com/download !
Thank you ! Questions ?
@Cb_VRGeek @imin_vr