VR Everywhere March 2015

VR everywhere

consumer virtual

reality for desktop,

mobile and web

tony parisi

march, 2015

about me

http://www.tonyparisi.com 3/18/2015

CONTACT

[email protected]

skype: auradeluxe

http://twitter.com/auradeluxe http://www.tonypa

risi.com/

http://www.learningwebgl.com/

GET THE BOOKS!

WebGL: Up and Running

http://www.amazon.com/dp/144932357X

Programming 3D Applications with HTML and WebGL

http://www.amazon.com/Programming-Applications-HTML5-WebGL-

Visualization/dp/1449362966

MEETUPS

http://www.meetup.com/WebGL-Developers-Meetup/

http://www.meetup.com/Web-VR/

BOOK CODE

https://github.com/tparisi/WebGLBook

https://github.com/tparisi/Programming3DApplications

GET GLAM

http://www.glamjs.org/

https://github.com/tparisi/glam/

WORK

http://www.thirdeye.gl/

CREDS

Co-creator, VRML and X3D

mailto:[email protected]

http://twitter.com/auradeluxe

http://www.tonyparisi.com/



http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966








the promised land!

consumer VR

3/18/2015http://www.tonyparisi.com

hardware explosion


headsets

input

devices

high performance laptops

AFFORDABLE STEREOSCOPIC

DISPLAYS, CRAZY-FAST GPUS,

CHEAP MOTION TRACKING

IMUS… FTW!

PC HMDs


Provide deepest, most immersive VR experiences

Desktop peripheral - acts as external monitor

Position and orientation tracking

Tethered – one or more cables to the computer; camera for position tracking

Oculus Rift – the Gold Standard of VR headsets

Developer Kit CHEAP $350

In production late 2015/early 2016

HTC/Valve VIVE

OSVR – Razer open dev kit and SDK; ecosystem initiative

mobile HMDs


Deluxe Mobile – Samsung GearVR

Optics and IMU licensed from Oculus

Custom Android build w/ VR extensions

and enhanced UI

The best VR experience out there!

(Look Ma, no wires)

BUT $200 + requires new Phone (Note

4) NOT CHEAP $1,000

Support for S6 coming, price coming

down

Orientation tracking only

So far… no competition… but EXPECT

IT!

drop-in phone viewers


Google Cardboard

Just Add SmartPhone

Orientation tracking only

Open specification for mobile VR

Standard Android OS support, using simple stereo rendering and accelerometer tracking

$2 in parts or $25 ready-to-assemble Kits WAY CHEAP

Not as nice… but great for the price!

MergeVR, Wearality, Moggles…

Some of these have different rendering thus require custom SDK

operating systems


Desktop – Oculus SDK supports Windows/Mac/Linux; Vive SDK not yet released…

GearVR - custom Android build w/Oculus mobile SDK

Cardboard

Google has SDK for native Android with simple stereo render and head tracking; also adding first-class VR support into future OS

Cardboard is just side-by side rendering and accelerometer tracking… so it works on iOS too

Mobile browsers can run Cardboard apps! Just stereo render with WebGL and track w/DeviceOrientation API

The Oculus Store for GearVR

rendering


Oculus Rift/GearVR

110 diagonal FOV

Side-by-side stereo

Barrel distortion

Cardboard

90 degree FOV

Side-by-side stereo

head tracking


Oculus Rift/GearVR

1000hz refresh rate

Cardboard

200hz refresh rate

Desktop IMUs built into HMDs track position and orientation.

Use camera or wall-mounted sensors for position tracking

Mobile IMUs track orientation only

INERTIAL MEASURE UNITS (IMUS)

the challenge:

“presence”


Camera must respond to head tracking at 75FPS+

higher latency risks motion sickness and breaks the illusion

Rendering, as always, must be at 60FPS+

“Motions-to-photons” latency goal is <20ms

Desktop and GearVR can hit this, currently mobile can’t (though it’s in the works); most phone IMU motions-to-photons are are 50ms+

What’s wrong with this picture???

Classic targeting of 60FPS makes rendering the bottleneck

If targeting 75FPS, that’s minimum 150FPS simulation (75FPS x 2 cameras), or <7ms frame time!!!

performance (1)


Software techniques

Of course push everything you can onto the GPU

New engine optimizations

Example: render each state change to each camera vs. whole scene to each camera

Mobile: create experiences with lower res than typical games... till the hardware gets faster and resolutions get higher; low latency trumps hi poly count

Time Warp

Oculus SDK technique, transforms image in Z-buffer at next refresh based on new IMU position/orientation… even if new rendered frame isn’t ready

Already supported in game engines and other runtimes

performance (2)


First attempts at a comprehensive GPU approach to VR -NVIDIA VR Direct

New features enabled in Maxwell processors (GeForce GTX 980 and GTX 970)

Overall attack on latency – whole pipeline

Asynchronous Warp (NVIDIA’s version of Time Warp)

Dynamic Super Resolution (DSR) – up to 4 megapixels per eye downsampled to HD

Multi-frame sampled anti-aliasing (MFAA)

Boasts motion-to-photons latency of 25ms

… plus “GeForce Experience” control panel – one end-user settings dialog instead of several

ACCELERATING VR USING THE POWER OF THE GPU

developing desktop and

mobile VR


Game engines

Unity3D – targets all platforms: desktop Oculus, Gear VR,

Cardboard. Free and open prefabs available on Unity store and

from HMD vendors. Expect Vive and other platforms quickly; they

move fast.

Unreal - desktop now, mobile in development

Native SDKs

Oculus SDK C++

Oculus Mobile SDK C++

Android SDK for Cardboard

WebVR:

fast, cheap and democratized


Experimental browser API

Query browser for attached HMD devices

Render fullscreen stereo in WebGL

Track HMD position/orientation by polling HMD device

Already in Firefox and Chrome nightly builds, uses same API

Build apps using open source JavaScript engines, or WebGL “exported” from Unity/Unreal via Emscripten cross-compiler

Browser refresh currently throttled at 60FPS but they are lifting this limitation

Quake 3 WebVR demo, developed by Brandon

Jones of Google

http://media.tojicode.com/q3bsp/

NO-DOWNLOAD VR APPLICATIONS.

INSTANT ACCESS IN THE BROWSER, SHARE IT WITH A URL!

WRITTEN IN JAVASCRIPT!?

http://media.tojicode.com/q3bsp/

input


We still don’t have the mouse of VR

State of the practice is blind use of keyboard or gamepad controller

Motion controllers like Leap Motion, Sixense STEM system are gaining popularity for PC-based VR

GearVR has a really innovative trackpad design, plus simple, elegant navigation UI

Drop-in phone viewers are experimenting with side-mounted click devices and Bluetooth controllers

Leap Motion

controller

Sixense

STEM

System

image:

https://www.attachmate.com/blogs/legacymodernization/legacy-data-the-elephant-in-the-room/

https://www.attachmate.com/blogs/legacymodernization/legacy-data-the-elephant-in-the-room/

augmented reality


Explosion of VR has renewed

interest in AR

Promising new hardware

Microsoft Hololens

CastAR - table-top AR display for

gaming, close to coming to

market

Magic Leap ????

A little farther out… most projects

don’t have ship dates

Another SDK babel looming…

THE NEXT NEXT BIG THING?

Hololens Sculpting Tool Demo

CastAR Multi-Player Game

coming in May 2015


keep in touch

http://www.tonyparisi.com 3/18/2015

CONTACT

[email protected]

skype: auradeluxe

http://twitter.com/auradeluxe http://www.tonypa

risi.com/

http://www.learningwebgl.com/ MEETUPS



BOOK CODE



GET GLAM



WORK


CREDS

Co-creator, VRML and X3D

GET THE BOOKS!

WebGL: Up and Running


Programming 3D Applications with HTML and WebGL

http://www.amazon.com/Programming-Applications-HTML5-WebGL-

Visualization/dp/1449362966

mailto:[email protected]

http://twitter.com/auradeluxe

http://www.tonyparisi.com/










http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/dp/1449362966

VR everywhere

consumer virtual

reality for desktop,

mobile and web

tony parisi

march, 2015

VR Everywhere March 2015

Technology

Transcript of VR Everywhere March 2015