Mobile Augmented Reality: Technology Advances and UX Challenges
UX and Interaction in Virtual Reality
-
Upload
devgamm-conference -
Category
Software
-
view
72 -
download
1
Transcript of UX and Interaction in Virtual Reality
UX and Interaction in Virtual Reality
Ciro ContinisioTechnical Evangelist@[email protected]
Before we startTakeaway: Hints and ideas on how to create good user experience in VR, through the analysis of the solutions adopted by several VR games and apps.
This talk is code-free
4
2016 - The Year of VR(be quick though, because were in November!)
Since last year, everyone has been saying that 2016 would have been the year of VR, and in part they were right.We have seen the launch of Oculus and Vive to the public, and soon PlayStation VR.
Sadly, or luckily, its also almost over.5
Takeaways of 2016No market for it... yet85k-100k Vive owners 3 monts after launch40k+ Rift on Steam (how many devkits?)1M monthly users on GearVR (May 2016), 5M Cardboard app downloadsNo standards... yetNo killer application yetSource: http://www.roadtovr.com/htc-vive-sales-figures-data-100000-steamspy-data/http://www.roadtovr.com/oculus-rift-growth-spurt-has-halted-htc-vive-market-share-growth/
So, since its almost over, what are the takeaways of this 2016? One thing is sure, we know that the market is very small, with around 100 thousands Vive sold, probably 40 thousand active Rifts, and mobile VR that sits just in the tens of millions users.Theres no standard, like we have the controllers on consoles, which makes cross-headset development practically impossible.Finally, VR hasnt got its killer application, yet.6
Get ready for 2017
The good news is, you can still be part of the VR revolution in 2017,7
VR killer application
and possibly, create the first real VR killer application8
Audience is fresh
9
No magic formula
10
Interactivity
11
InteractivityTwo-way flow of information, between the player and the game.
The interactivity happens through the User Interface (UI) which, when put to action, defines the User Experience (UX).Source: the Art of Game Design by Jesse Schell
12
InteractivityThe User Interface is made-up of 3 parts:The real-world physical input (controllers) and output (screen, audio)The simulated game worldThe in-game virtual interface (often called UI)
The relationship between them is called mapping. When it works well, the player feels in control, the UI becomes intuitive and the resulting UX is great.
13
Interactivity in VRMotion-controlled physical input (HMD and hand controllers)Now we are inside the simulated game worldNew complications to deal withMotion sicknessFatigue and session lengthLocomotion in the game world
14
Interactivity in VRis largely uncharted
15
Build a vocabulary
16
Creating Presence
17
Presence... a psychological state or subjective perception in which even though part or all of an individuals current experience is generated by and/or filtered through human-made technology, part or all of the individuals perception fails to accurate acknowledge the role of the technology in the experience.Source: ISPR - https://ispr.info/about-presence-2/about-presence/... a psychological state or subjective perception in which even though part or all of an individuals current experience is generated by and/or filtered through human-made technology, part or all of the individuals perception fails to accurate acknowledge the role of the technology in the experience.
18
Types of VR platformsMobile (Cardboard, Daydream, Gear VR, other plastic shells)Low processing power, less pixels on-screenSimplified inputTethered (Oculus, Vive, PlayStation VR)High power, but high hardware requirementsHead trackingMore refined physical interactions
When we look at the current VR platforms available, theres a clear technological divide between the mobile ones, which include []. Making content for these platforms requires less effort, but at the same time we have to optimise for phones, and try to maintain a consistent 60fps. Not easy.
On the other side, we have the VR platforms that have a tethered HMD, [...], which have much more power but at the same time require expensive hardware to run. This greatly limits the potential audience, but on the other side they offer a more refined range of physical interactions.19
Map VR platformsby type of experience
20
Types of VR experiencesOne-button/touch controlsCardboard, Gear VR, other plastic shellsControllers, hand tracking or gesturesOculus Rift, Daydream, PlayStation VRRoom-scale, full-body movementsHTC Vive
Means of interactivity in VR
22
Visuals
23
VisualsSame techniques on all platformsCubemaps, baking light, smokes and mirrors,Reduce post-processing effects, but absolutely use AADifferent fidelity per platformEven static images work wonders!
24
Motion sickness
25
How to induce motion sickness
26
Motion sicknessIn order of importance:Rotational accelerationRotational speedMovement accelerationMovement speed
Roll being particularly bad
Building the VR legs
28
Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of reference
Frame of reference
30
Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of referenceShake that instead of the camera
Screen shake
SubLevel Zero32
Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of referenceShake that instead of the cameraPrefer fast and short to long but slow
Fast movements
SubLevel Zero34
Audio
35
Audio in 3D gamesEasy to do in UnitySounds positioned in the 3D sceneSpread over 2 audio channels depending on listener positionReverberation filtersDoppler effect
36
Audio in VR - Spatial AudioSpatialisation is created mainly by 3 elements:Interaural time differenceInteraural level differenceSpectral filtering done by outer ears
Source: https://developers.google.com/vr/concepts/spatial-audio
37
Audio in VR - Spatial AudioDirect soundEarly reflectionLate reverb
Source: https://developers.google.com/vr/concepts/spatial-audio
38
Audio in VR - Proprietary SDKsThey offer a lot more control over sound:They create Spatial AudioAlter levels, and timePerform spectral filteringThey allow to define materials and size of the virtual environmentOften optimised (Google VR SDK for Daydream, which uses a diff. thread)Source: https://developers.google.com/vr/concepts/spatial-audio
39
General tips for good VR soundsUse mono files with no pre-built reverbUse recognisable soundsAnimate the sound sourceRepeat the soundUse complex sounds
Source: https://developers.google.com/vr/concepts/spatial-audio
40
Input
41
GazeMost basic form of inputVery simple to implementApplies to all types of VRSometimes the only choice
42
Reticule + target feedback
43
Gaze
44
Involuntary interactions
45
Gaze aiming
46
Off-screen events feedback
47
Off-screen events feedback
48
One-button controls
49
Games for the motion impaired
50
Motion controls
51
Hacking the physics
52
Accelerometer
53
Gyroscope
54
Tracked hands
55
New Vive controllers prototype
56
New Vive controllers prototype
57
New Vive controllers prototype
58
NewtonVR
Source:github.com/TomorrowTodayLabs/NewtonVR
59
Never use a buttonif you can go physical
60
Physical tool swap
61
Physical inventory
62
Physical inventory
63
Physical inventory
64
No real-life feedback
65
Haptic feedback
The gallery66
Perceive the Z axis
67
Z axis issues
The gallery68
Locomotion
69
Teleport and reorient
The gallery70
Walking in place
Source: Eelke Folmer - https://www.youtube.com/watch?v=lWqA_4crAEA
71
Freezing the world
Source: Bumble - https://www.youtube.com/watch?v=p0YxzgQG2-E
Cosmic Wandering72
Holosphere
Source: Frooxius - https://www.youtube.com/watch?v=oSP5sjg9TIs
73
Trick the players mind
74
Unlimited corridor
Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA
Yohei Yanase75
Unlimited corridor
Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA
76
Unlimited corridor
Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA
77
Locomotion as a game mechanic
78
Budget Cuts
79
Unseen Diplomacy
Redirected walking80
Escher
81
Clipping through walls
82
Clipping feedback
Source: REFLEKT at Vision Summit - https://www.youtube.com/watch?v=At_Zac4Xezw
83
Comfort
84
HMD comfort
85
HMD comfortNot all HMDs allow for the same degree of freedom in head movements.
86
HMD cables strangling the player
87
HMD cablesWe know the direction the player is facing, and the one he was facing before.
The developer can try to prevent the player from entangling in the cable by using (audio) cues in the reverse direction.
Not every gameplay can use this trick.
88
89
Now the magic formula is
90
Challenge everything I said
91
The future
92
The futureNear future:Larger install base on all platformsNext-level mobile VR thanks to more powerful phonesMore online VRTracked objects in room-scale VR
Tracked objects
94
The futureNear future:Larger install base on all platformsNext-level mobile VR thanks to more powerful phonesMore online VRTracked objects in room-scale VRVR as a social experience
Social VR
96
Entertain the audience
97
Entertain the audience
98
Smooth the camera!
99
Entertain the audience
100
The futureFar future:No cables on tracked HMDsReduced motion sickness thanks to higher resolutions and frameratesGeneral audience will be used to interacting in VRHololens will bring new interest to AR and Mixed Reality
Mixed RealitySource: Owlchemy Labs - http://owlchemylabs.com/owlchemyvr-mixed-reality-tech/
The gallery102
Thank [email protected]@CiroContns