COMP 4010: Lecture2 VR Technology
-
Upload
mark-billinghurst -
Category
Technology
-
view
1.525 -
download
15
Transcript of COMP 4010: Lecture2 VR Technology
![Page 1: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/1.jpg)
LECTURE 2: VR TECHNOLOGY
COMP 4010 - Virtual RealitySemester 5 - 2017
Mark Billinghurst, Bruce ThomasUniversity of South Australia
![Page 2: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/2.jpg)
Overview
•Presence in VR•Perception and VR•Human Perception•VR Technology
![Page 3: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/3.jpg)
PRESENCE
![Page 4: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/4.jpg)
Presence ..“The subjective experience of being in one place or environment even when physically situated in another”
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments, 7(3), 225-240.
![Page 5: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/5.jpg)
Immersion vs. Presence• Immersion: describes the extent to which technology is
capable of delivering a vivid illusion of reality to the senses of a human participant.
• Presence: a state of consciousness, the (psychological) sense of being in the virtual environment.
• So Immersion, defined in technical terms, is capable of producing a sensation of Presence
• Goal of VR: Create a high degree of Presence• Make people believe they are really in Virtual Environment
Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and virtual environments, 6(6), 603-616.
![Page 6: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/6.jpg)
How to Create Strong Presence?• Use Multiple Dimensions of Presence
• Create rich multi-sensory VR experiences• Include social actors/agents that interact with user• Have environment respond to user
• What Influences Presence• Vividness – ability to provide rich experience (Steuer 1992)• Using Virtual Body – user can see themselves (Slater 1993)• Internal factors – individual user differences (Sadowski 2002)• Interactivity – how much users can interact (Steuer 1992)• Sensory, Realism factors (Witmer 1998)
![Page 7: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/7.jpg)
Example: UNC Pit Room• Key Features
• Training room and pit room• Physical walking • Fast, accurate, room scale tracking• Haptic feedback – feel edge of pit, walls• Strong visual and 3D audio cues
• Task• Carry object across pit
• Walk across or walk around
• Dropping virtual balls at targets in pit
• http://wwwx.cs.unc.edu/Research/eve/walk_exp/
![Page 8: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/8.jpg)
Typical Subject Behaviour
• Note – from another pit experiment• https://www.youtube.com/watch?v=VVAO0DkoD-8
![Page 9: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/9.jpg)
Benefits of High Presence
• Leads to greater engagement, excitement and satisfaction• Increased reaction to actions in VR
• People more likely to behave like in the real world• E.g. people scared of heights in real world will be scared in VR
• More natural communication (Social Presence)• Use same cues as face to face conversation
• Note: The relationship between Presence and Performance is unclear – still an active area of research
![Page 10: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/10.jpg)
Measuring Presence• Presence is very subjective so there is a lot of debate
among researchers about how to measure it• Subjective Measures
• Self report questionnaire• University College London Questionnaire (Slater 1999)• Witmer and Singer Presence Questionnaire (Witmer 1998)• ITC Sense Of Presence Inventory (Lessiter 2000)
• Continuous measure • Person moves slider bar in VE depending on Presence felt
• Objective Measures• Behavioural
• reflex/flinch measure, startle response• Physiological measures
• change in heart rate, skin conductance, skin temperaturePresence Slider
![Page 11: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/11.jpg)
Relevant Papers• Slater, M., & Usoh, M. (1993). Representation systems, perceptual positions, and presence
in immersive virtual environments. Presence, 2:221–233.
• Slater, M. (1999). Measuring presence: A response to the Witmer and Singer Presence Questionnaire. Presence, 8:560–565.
• Steuer, J. (1992). Defining virtual reality: Dimensions determining telepresence. Journal of Communication, 42(4):72–93.
• Sadowski, W. J. and Stanney, K. M. (2002) Measuring and Managing Presence in Virtual Environments. In: Handbook of Virtual Environments: Design, implementation, and applications.http://vehand.engr.ucf.edu/handbook/
• Schuemie, M. J., Van Der Straaten, P., Krijn, M., & Van Der Mast, C. A. (2001). Research on presence in virtual reality: A survey CyberPsychology & Behavior, 4(2), 183-201.
• Lee, K. M. (2004). Presence, explicated. Communication theory, 14(1), 27-50.
• Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments, 7(3), 225-240.
• Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2000). Development of a new cross-media presence questionnaire: The ITC-Sense of presence. Paper at the Presence 2000 Workshop, March 27–28, Delft.
![Page 12: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/12.jpg)
PERCEPTION AND VR
![Page 13: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/13.jpg)
What is Reality?
![Page 14: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/14.jpg)
How do We Perceive Reality?
• We understand the world through our senses:• Sight, Hearing, Touch, Taste, Smell
(and others..)
• Two basic processes:• Sensation – Gathering information• Perception – Interpreting information
![Page 15: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/15.jpg)
Simple Sensing/Perception Model
![Page 16: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/16.jpg)
Goal of Virtual Reality
“.. to make it feel like you’re actually in a place that you are not.”
Palmer LuckeyCo-founder, Oculus
![Page 17: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/17.jpg)
Creating the Illusion of Reality• Fooling human perception by using technology to generate artificial sensations• Computer generated sights, sounds, smell, etc
![Page 18: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/18.jpg)
Reality vs. Virtual Reality
• In a VR system there are input and output devices between human perception and action
![Page 19: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/19.jpg)
Example Birdly - http://www.somniacs.co/
• Create illusion of flying like a bird• Multisensory VR experience
• Visual, audio, wind, haptic
![Page 20: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/20.jpg)
Birdly Demo
• https://www.youtube.com/watch?v=gHE6H62GHoM
![Page 21: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/21.jpg)
‘Virtual Reality is a synthetic sensory experience which may one day be indistinguishable from the real physical world.”
-Roy Kalawsky (1993)
Today
Tomorrow
![Page 22: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/22.jpg)
HUMAN PERCEPTION
![Page 23: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/23.jpg)
Motivation
• Understand: In order to create a strong sense of Presence we need to understand the Human Perception system
• Stimulate: We need to be able to use technology to provide real world sensory inputs, and create the VR illusion
VR Hardware Human Senses
![Page 24: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/24.jpg)
Senses
• How an organism obtains information for perception:• Sensation part of Somatic Division of Peripheral Nervous System • Integration and perception requires the Central Nervous System
• Five major senses:• Sight (Opthalamoception)• Hearing (Audioception)• Taste (Gustaoception)• Smell (Olfacaoception)• Touch (Tactioception)
![Page 25: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/25.jpg)
Other Lessor Known Senses..• Proprioception = sense of body position
• what is your body doing right now• Equilibrium = balance• Acceleration• Nociception = sense of pain• Temperature• Satiety (the quality or state of being fed or gratified to or beyond capacity)
• Thirst• Micturition• Amount of CO2 and Na in blood
![Page 26: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/26.jpg)
Relative Importance of Each Sense• Percentage of neurons in
brain devoted to each sense• Sight – 30%• Touch – 8%• Hearing – 2%• Smell - < 1%
• Over 60% of brain involved with vision in some way
![Page 27: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/27.jpg)
VR System Overview• Simulate output• Map output to devices• Use devices to
stimulate the senses
Visual Simulation
3D Graphics HMD Vision System
Brain
Example: Visual Simulation
Human-Machine Interface
![Page 28: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/28.jpg)
Sight
![Page 29: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/29.jpg)
The Human Visual System
• Purpose is to convert visual input to signals in the brain
![Page 30: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/30.jpg)
The Human Eye
• Light passes through cornea and lens onto retina• Photoreceptors in retina convert light into electrochemical signals
![Page 31: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/31.jpg)
Photoreceptors – Rods and Cones
• Retina photoreceptors come in two types, Rods and Cones
![Page 32: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/32.jpg)
Rods vs. Cones
• RODS• 125 million cells in retina• Concentrated on
periphery of retina• No color detection• Most sensitive to light• Scotopic (night) vision• Provide peripheral vision,
motion detection
• CONES• 4.5-6 million in retina• Responsible for color
vision• Sensitive to red, blue,
green light• Work best in more
intense light
![Page 33: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/33.jpg)
Colour Perception
• Humans only perceive small part of electromagnetic spectrum
![Page 34: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/34.jpg)
Horizontal and Vertical FOV
• Humans can see ~135o vertical (60o above, 75o below)• See up to ~ 210o horizontal FOV, ~ 115o stereo overlap• Colour/stereo in centre, Black & White/mono in periphery
![Page 35: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/35.jpg)
Types of Visible Perception Possible
• As move further from fovea, vision becomes more limited
![Page 36: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/36.jpg)
Dynamic Range
• Rods respond to low Luminance light, Cones to bright light
![Page 37: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/37.jpg)
Comparing to Displays
• Human vision has far higher dynamic range than any available display technology• 40 f-stops, cf 17 f-stops for HDR display
![Page 38: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/38.jpg)
Vergence + Accommodation
• saas
![Page 39: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/39.jpg)
Vergence/Accommodation Demo
• https://www.youtube.com/watch?v=p_xLO7yxgOk
![Page 40: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/40.jpg)
Vergence-Accommodation Conflict
• Looking at real objects, vergence and focal distance match• In Virtual Reality, vergence and accommodation can miss-match
• Focusing on HMD screen, but accommodating for virtual object behind screen
![Page 41: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/41.jpg)
Visual Acuity
Visual Acuity Test Targets
• Ability to resolve details• Several types of visual acuity
• detection, separation, etc• Normal eyesight can see a 50 cent coin at 80m
• Corresponds to 1 arc min (1/60th of a degree)• Max acuity = 0.4 arc min
![Page 42: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/42.jpg)
Resolution of the Eye
• Decreases away from the fovea • Maximum resolution of 1 arcmin – spot of 6x10-6 m size on retina
![Page 43: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/43.jpg)
Stereo Perception/Stereopsis
• Eyes separated by IPD• Inter pupillary distance• 5 – 7.5cm (average. 6.5cm)
• Each eye sees diff. image• Separated by image parallax
• Images fused to create 3D stereo view
![Page 44: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/44.jpg)
![Page 45: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/45.jpg)
Depth Perception• The visual system uses a range of different Stereoscopic
and Monocular cues for depth perception
Stereoscopic Monoculareye convergence angledisparity between leftand right imagesdiplopia
eye accommodationperspectiveatmospheric artifacts (fog)relative sizesimage blurocclusionmotion parallaxshadowstexture
Parallax can be more important for depth perception!Stereoscopy is important for size and distance evaluation
![Page 46: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/46.jpg)
More Depth Cues
![Page 47: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/47.jpg)
Example: Perspective Cues
• asdas
![Page 48: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/48.jpg)
More Examples
Occlusion
Texture GradientShadows
![Page 49: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/49.jpg)
Depth Perception Distances
• asdf
![Page 50: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/50.jpg)
Fooling Depth Perception
• https://www.youtube.com/watch?v=p-eZcHPp7Go
![Page 51: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/51.jpg)
Properties of the Human Visual System
• visual acuity: 20/20 is ~1 arc min• field of view: ~200° monocular, ~120° binocular, ~135° vertical• resolution of eye: ~576 megapixels• temporal resolution: ~60 Hz (depends on contrast, luminance)• dynamic range: instantaneous 6.5 f-stops, adapt to 46.5 f-stops• colour: everything in CIE xy diagram• depth cues in 3D displays: vergence, focus, (dis)comfort• accommodation range: ~8cm to ∞, degrades with age
![Page 52: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/52.jpg)
The Perfect Retina Display• A HMD capable of creating images indistinguishable from
reality would need to match the properties of the eye: • FOV: 200-220° x 135° needed (both eyes)
• 120° stereo overlap• Acuity: ~0.4 arc min (1 pixel/0.4 arc min)• Pixel Resolution: ~30,000 x 20,000 pixels
• 200*60°/0.4 = 30,000, 135*60°/0.4 = 20,250• Pixels/inch: > 2190 PPI @ 100mm (depends on distance to screen)• Update rate: 60 Hz
• The biggest challenge: bandwidth• compress and transmit huge amount of data• drive and operate display pixels
![Page 53: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/53.jpg)
Comparison between Eyes and HMD
Human Eyes HTC ViveFOV 200° x 135° 110° x 110°Stereo Overlap 120° 110°Resolution 30,000 x 20,000 2,160 x 1,200Pixels/inch >2190 (100mm to screen) 456Update 60 Hz 90 Hz
See http://doc-ok.org/?p=1414http://www.clarkvision.com/articles/eye-resolution.htmlhttp://wolfcrow.com/blog/notes-by-dr-optoglass-the-resolution-of-the-human-eye/
![Page 54: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/54.jpg)
Hearing
![Page 55: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/55.jpg)
Anatomy of the Ear
![Page 56: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/56.jpg)
How the Ear Works
• https://www.youtube.com/watch?v=pCCcFDoyBxM
![Page 57: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/57.jpg)
Sound Frequency and Amplitude• Frequency determines the pitch of the sound• Amplitude relates to intensity of the sound
• Loudness is a subjective measure of intensity
High frequency = short period
Low frequency = long period
![Page 58: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/58.jpg)
Distance to Listener• Relationship between sound intensity and distance to the
listenerInverse-square law
• The intensity varies inversely with the square of the distance from the source. So if the distance from the source is doubled (increased by a factor of 2), then the intensity is quartered (decreased by a factor of 4).
![Page 59: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/59.jpg)
Auditory Thresholds
• Humans hear frequencies from 20 – 22,000 Hz• Most everyday sounds from 80 – 90 dB
![Page 60: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/60.jpg)
Sound Localization• Humans have two ears
• localize sound in space• Sound can be localized using 3 coordinates• Azimuth, elevation,
distance
![Page 61: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/61.jpg)
Sound Localization• Azimuth Cues
• Difference in time of sound reaching two ears• Interaural time difference (ITD)
• Difference in sound intensity reaching two ears• Interaural level difference (ILD)
• Elevation Cues• Monaural cues derived from the pinna (ear shape)
• Head related transfer function (HRTF)• Range Cues
• Difference in sound relative to range from observer• Head movements (otherwise ITD and ILD are same)
![Page 62: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/62.jpg)
Sound Localization
• https://www.youtube.com/watch?v=FIU1bNSlbxk
![Page 63: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/63.jpg)
Sound Localization (Azimuth Cues)
Interaural Time Difference
![Page 64: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/64.jpg)
HRTF (Elevation Cue)• Pinna and head shape affect frequency intensities• Sound intensities measured with microphones in ear and
compared to intensities at sound source• Difference is HRTF, gives clue as to sound source location
![Page 65: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/65.jpg)
Accuracy of Sound Localization• People can locate sound
• Most accurately in front of them• 2-3° error in front of head
• Least accurately to sides and behind head• Up to 20° error to side of head• Largest errors occur above/below elevations and behind head
• Front/back confusion is an issue• Up to 10% of sounds presented in the front are perceived coming
from behind and vice versa (more in headphones)
BUTEAN, A., Bălan, O., NEGOI, I., Moldoveanu, F., & Moldoveanu, A. (2015). COMPARATIVE RESEARCH ON SOUND LOCALIZATION ACCURACY IN THE FREE-FIELD AND VIRTUAL AUDITORY DISPLAYS. InConference proceedings of» eLearning and Software for Education «(eLSE)(No. 01, pp. 540-548). Universitatea Nationala de Aparare Carol I.
![Page 66: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/66.jpg)
Touch
![Page 67: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/67.jpg)
Touch• Mechanical/Temp/Pain stimuli transduced into Action Potentials (AP)
• Transducing structures are specialized nerves:• Mechanoreceptors: Detect pressure, vibrations & texture• Thermoreceptors: Detect hot/cold• Nocireceptors: Detect pain• Proprioreceptors: Detect spatial awareness
• This triggers an AP which then travels to various locations in the brain via the somatosensory nerves
![Page 68: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/68.jpg)
Haptic Sensation• Somatosensory System
• complex system of nerve cells that responds to changes to the surface or internal state of the body
• Skin is the largest organ• 1.3-1.7 square m in adults
• Tactile: Surface properties• Receptors not evenly spread• Most densely populated area is the tongue
• Kinesthetic: Muscles, Tendons, etc.• Also known as proprioception
![Page 69: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/69.jpg)
Somatosensory System
• Map of somatosensory areas of the brain, showing more area for regions with more receptors
![Page 70: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/70.jpg)
Anatomy of the Skin
![Page 71: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/71.jpg)
Cutaneous System• Skin – heaviest organ in the body
• Epidermis outer layer, dead skin cells• Dermis inner layer, with four kinds of mechanoreceptors
![Page 72: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/72.jpg)
Mechanoreceptors• Cells that respond to pressure, stretching, and vibration
• Slow Acting (SA), Rapidly Acting (RA)• Type I at surface – light discriminate touch• Type II deep in dermis – heavy and continuous touch
Receptor Type
Rate of Acting
Stimulus Frequency
Receptive Field DetectionFunction
Merkel Discs
SA-I 0 – 10 Hz Small, well defined
Edges, intensity
Ruffinicorpuscles
SA-II 0 – 10 Hz Large, indistinct Static force, skin stretch
Meissnercorpuscles
RA-I 20 – 50 Hz Small, well defined
Velocity, edges
Paciniancorpuscles
RA-II 100 – 300 Hz
Large, indistinct Acceleration,vibration
![Page 73: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/73.jpg)
Spatial Resolution• Sensitivity varies greatly
• Two-point discrimination
BodySite
Threshold Distance
Finger 2-3mm
Cheek 6mm
Nose 7mm
Palm 10mm
Forehead 15mm
Foot 20mm
Belly 30mm
Forearm 35mm
Upper Arm 39mm
Back 39mm
Shoulder 41mm
Thigh 42mm
Calf 45mm
http://faculty.washington.edu/chudler/chsense.html
![Page 74: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/74.jpg)
Proprioception/Kinaesthesia• Proprioception (joint position sense)
• Awareness of movement and positions of body parts• Due to nerve endings and Pacinian and Ruffini corpuscles at joints
• Enables us to touch nose with eyes closed• Joints closer to body more accurately sensed• Users know hand position accurate to 8cm without looking at them
• Kinaesthesia (joint movement sense)• Sensing muscle contraction or stretching
• Cutaneous mechanoreceptors measuring skin stretching• Helps with force sensation
![Page 75: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/75.jpg)
Smell
![Page 76: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/76.jpg)
Olfactory System
• Human olfactory system. 1: Olfactory bulb 2: Mitral cells 3: Bone 4: Nasal epithelium 5: Glomerulus 6: Olfactory receptor neurons
![Page 77: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/77.jpg)
How the Nose Works
• https://www.youtube.com/watch?v=zaHR2MAxywg
![Page 78: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/78.jpg)
Smell• Smells are sensed by olfactory sensory neurons in the
olfactory epithelium• 10 cm2 with hundreds of different types of olfactory receptors• Human’s can detect at least 10,000 different odors
• Some researchers say trillions of odors
• Sense of smell closely related to taste• Both use chemo-receptors• Olfaction + taste contribute to flavour
• The olfactory system is the only sense that bypasses the thalamus and connects directly to the forebrain
![Page 79: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/79.jpg)
Taste
![Page 80: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/80.jpg)
Sense of Taste
• https://www.youtube.com/watch?v=FSHGucgnvLU
![Page 81: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/81.jpg)
Basics of Taste
• Sensation produced when a substance in the mouth reacts chemically with taste receptor cells
• Taste receptors mostly on taste buds on the tongue• 2,000 – 5,000 taste buds on tongues/100+ receptors each
• Five basic tastes: • sweetness, sourness, saltiness, bitterness, and umami
• Flavour influenced by other senses• smell, texture, temperature, “coolness”, “hotness”
![Page 82: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/82.jpg)
Taste Trivia
![Page 83: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/83.jpg)
VR TECHNOLOGY
![Page 84: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/84.jpg)
Using Technology to Stimulate Senses• Simulate output
• E.g. simulate real scene• Map output to devices
• Graphics to HMD• Use devices to
stimulate the senses• HMD stimulates eyes
Visual Simulation
3D Graphics HMD Vision System
Brain
Example: Visual Simulation
Human-Machine Interface
![Page 85: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/85.jpg)
Key Technologies for VR System• Visual Display
• Stimulate visual sense• Audio/Tactile Display
• Stimulate hearing/touch• Tracking
• Changing viewpoint• User input
• Input Devices• Supporting user interaction
![Page 86: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/86.jpg)
Mapping Between Input and Output
Input
Output
![Page 87: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/87.jpg)
VISUAL DISPLAY
![Page 88: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/88.jpg)
Creating an Immersive Experience
•Head Mounted Display• Immerse the eyes
•Projection/Large Screen • Immerse the head/body
•Future Technologies• Neural implants• Contact lens displays, etc
![Page 89: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/89.jpg)
HMD Basic Principles
• Use display with optics to create illusion of virtual screen
![Page 90: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/90.jpg)
Key Properties of HMDs• Lens
• Focal length, Field of View• Occularity, Interpupillary distance• Eye relief, Eye box
• Display• Resolution, contrast• Power, brightness• Refresh rate
• Ergonomics• Size, weight• Wearability
![Page 91: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/91.jpg)
Simple Magnifier HMD Design
pq
Eyepiece(one or more lenses) Display
(Image Source)
Eye fVirtual Image
1/p + 1/q = 1/f wherep = object distance (distance from image source to eyepiece)q = image distance (distance of image from the lens)f = focal length of the lens
![Page 92: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/92.jpg)
Field of View
Monocular FOV is the angular subtense(usually expressed in degrees) of the displayed image as measured from the pupil of one eye.Total FOV is the total angular size of the displayed image visible to both eyes.
Binocular(or stereoscopic) FOV refers to the part of the displayed image visible to both eyes.
FOV may be measured horizontally, vertically or diagonally.
![Page 93: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/93.jpg)
Ocularity• Monocular - HMD image goes to only one eye.
• Bioccular - Same HMD image to both eyes.
• Binocular(stereoscopic) -Different but matched images to each eye.
![Page 94: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/94.jpg)
Interpupillary Distance (IPD)
nIPD is the horizontal distance between a user's eyes.
nIPD is the distance between the two optical axes in a binocular view system.
![Page 95: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/95.jpg)
Distortion in Lens Optics
A rectangle Maps to this
![Page 96: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/96.jpg)
Example Distortion
Oculus Rift DK2 HTC Vive
![Page 97: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/97.jpg)
To Correct for Distortion
• Must pre-distort image• This is a pixel-based distortion
• Graphics rendering uses linear interpolation!
• Too slow on most systems• Use shader programming
![Page 98: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/98.jpg)
![Page 99: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/99.jpg)
HMD Design Trade-offs
• Resolution vs. field of view• As FOV increases, resolution decreases for fixed pixels
• Eye box vs. field of view• Larger eye box limits field of view
• Size, Weight and Power vs. everything else
vs.
![Page 100: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/100.jpg)
Oculus Rift• Cost: $599 USD• FOV: 110o Horizontal• Refresh rate: 90 Hz• Resolution 1080x1200/eye• 3 DOF orientation tracking• 3 axis positional tracking
![Page 101: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/101.jpg)
Inside an Oculus Rift
![Page 102: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/102.jpg)
Comparison Between HMDs
![Page 103: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/103.jpg)
Computer Based vs. Mobile VR Displays
![Page 104: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/104.jpg)
• dsfsaf
![Page 105: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/105.jpg)
Google Cardboard
• Released 2014 (Google 20% project)• >5 million shipped/given away• Easy to use developer tools
+ =
![Page 106: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/106.jpg)
Multiple Mobile VR Viewers Available
![Page 107: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/107.jpg)
Projection/Large Display Technologies• Room Scale Projection
• CAVE, multi-wall environment• Dome projection
• Hemisphere/spherical display• Head/body inside
• Vehicle Simulator• Simulated visual display in windows
![Page 108: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/108.jpg)
CAVE
• Developed in 1992, EVL University of Illinois Chicago• Multi-walled stereo projection environment
• Head tracked active stereo
Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., & Hart, J. C. (1992). The CAVE: audio visual experience automatic virtual environment. Communications of the ACM, 35(6), 64-73.
![Page 109: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/109.jpg)
Typical CAVE Setup
• 4 sides, rear projected stereo images
![Page 110: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/110.jpg)
Demo Video – Wisconsin CAVE
• https://www.youtube.com/watch?v=mBs-OGDoPDY
![Page 111: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/111.jpg)
CAVE Variations
![Page 112: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/112.jpg)
Stereo Projection• Active Stereo
• Active shutter glasses• Time synced signal• Brighter images• More expensive
• Passive Stereo• Polarized images• Two projectors (one/eye)• Cheap glasses (powerless)• Lower resolution/dimmer• Less expensive
![Page 113: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/113.jpg)
Caterpillar Demo
• https://www.youtube.com/watch?v=r9N1w8PmD1E
![Page 114: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/114.jpg)
Allosphere• Univ. California Santa Barbara
• One of a kind facility• Immersive Spherical display
• 10 m diameter• Inside 3 story anechoic cube• Passive stereoscopic projection • 26 projectors• Visual tracking system for input
• See http://www.allosphere.ucsb.edu/
Kuchera-Morin, J., Wright, M., Wakefield, G., Roberts, C., Adderton, D., Sajadi, B., ... & Majumder, A. (2014). Immersive full-surround multi-user system design. Computers & Graphics, 40, 10-21.
![Page 115: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/115.jpg)
Allosphere Demo
• https://www.youtube.com/watch?v=25Ch8eE0vJg
![Page 116: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/116.jpg)
Vehicle Simulators• Combine VR displays with vehicle
• Visual displays on windows• Motion base for haptic feedback• Audio feedback
• Physical vehicle controls• Steering wheel, flight stick, etc
• Full vehicle simulation• Emergencies, normal operation, etc• Weapon operation• Training scenarios
![Page 117: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/117.jpg)
Demo: Boeing 787 Simulator
• https://www.youtube.com/watch?v=3iah-blsw_U
![Page 118: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/118.jpg)
HAPTIC/TACTILE DISPLAYS
![Page 119: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/119.jpg)
Haptic Feedback• Greatly improves realism• When is it needed?
• Other cues occluded/obstructed• Required for task performance• High bandwidth!
• Hands and wrist are most important• High density of touch receptors
• Two kinds of feedback• Touch Feedback
• information on texture, temperature, etc. • Does not resist user contact
• Force Feedback• information on weight, and inertia. • Actively resists contact motion
![Page 120: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/120.jpg)
Haptic Devices• Pin arrays for the finger(s)• Force-feedback "arms"• "Pager" motors• Particle brakes• Passive haptics• Many devices are application specific
• Like surgical devices
![Page 121: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/121.jpg)
Active Haptics
• Actively resists contact motion• Dimensions
• Force resistance• Frequency Response• Degrees of Freedom• Latency• Intrusiveness• Safety• Comfort• Portability
![Page 122: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/122.jpg)
Example: Phantom Omni
• Combined stylus input/haptic output• 6 DOF haptic feedback
![Page 123: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/123.jpg)
Phantom Omni Demo
• https://www.youtube.com/watch?v=REA97hRX0WQ
![Page 124: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/124.jpg)
Immersion Cybergrasp
• Haptic feedback on Glove• Combined with glove input
![Page 125: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/125.jpg)
CyberMotion (MPI Tübingen)
125
https://www.youtube.com/watch?v=shdS-hynLHg
![Page 126: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/126.jpg)
CableRobot (MPI Tübingen)
126
https://www.youtube.com/watch?v=cJCsomGwdk0&t=3s
![Page 127: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/127.jpg)
Passive Haptics
• Not controlled by system• Pros
• Cheap• Large scale• Accurate
• Cons• Not dynamic• Limited use
![Page 128: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/128.jpg)
UNC Being There Project
![Page 129: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/129.jpg)
Passive Haptic Paddle
• Using physical props to provide haptic feedback• http://www.cs.wpi.edu/~gogo/hive/
![Page 130: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/130.jpg)
Tactile Feedback Interfaces• Goal: Stimulate skin tactile receptors• Using different technologies
• Air bellows• Jets• Actuators (commercial)• Micropin arrays• Electrical (research)• Neuromuscular stimulations (research)
![Page 131: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/131.jpg)
Vibrotactile Cueing Devices• Vibrotactile feedback has been incorporated into many
devices• Can we use this technology to provide scalable, wearable
touch cues?
![Page 132: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/132.jpg)
Vibrotactile Feedback Projects
Navy TSAS Project
TactaBoard andTactaVest
![Page 133: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/133.jpg)
Example: CyberTouch Glove• Immersion Corporation
• Expensive - $15000• Six Vibrotactile actuators
• Back of finger• Palm
• Off-centered actuator motor• Rotation speed=frequency of
vibration (0-125 Hz)• When tracked virtual hand intersects with virtual object, send signal to glove to vibrate
http://www.cyberglovesystems.com/cybertouch/
![Page 134: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/134.jpg)
AUDIO DISPLAYS
![Page 135: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/135.jpg)
Audio Displays• Spatialization vs. Localization• Spatialization is the processing of sound signals to make
them emanate from a point in space• This is a technical topic
• Localization is the ability of people to identify the source position of a sound• This is a human topic, i.e., some people are better at it than others.
![Page 136: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/136.jpg)
Audio Display PropertiesPresentation Properties• Number of channels• Sound stage• Localization• Masking• Amplification
Logistical Propertieso Noise pollutiono User mobilityo Interface with trackingo Environmental
requirementso Integrationo Portabilityo Throughputo Cumbero Safetyo Cost
![Page 137: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/137.jpg)
Channels & Masking• Number of channels
• Stereo vs. mono vs. quadrophonic• 2.1, 5.1, 7.1
• Two kinds of masking• Louder sounds mask softer ones
• We have too many things vying for our audio attention these days!• Physical objects mask sound signals
• Happens with speakers, but not with headphones
![Page 138: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/138.jpg)
Audio Displays: Head-worn
Ear Buds On Ear Open Back
Closed Bone Conduction
![Page 139: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/139.jpg)
Spatialized Audio Effects• Naïve approach
• Simple left/right shift for lateral position• Amplitude adjustment for distance
• Easy to produce using consumer hardware/software• Does not give us "true" realism in sound
• No up/down or front/back cues• We can use multiple speakers for this
• Surround the user with speakers• Send different sound signals to each one
![Page 140: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/140.jpg)
Head-Related Transfer Functions• A.k.a. HRTFs• A set of functions that model how sound from a source at
a known location reaches the eardrum
![Page 141: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/141.jpg)
Constructing HRTFs• Small microphones placed into ear canals• Subject sits in an anechoic chamber
• Can use a mannequin's head instead• Sounds played from a large number of known locations
around the chamber• Functions are constructed for this data• Sound signal is filtered through inverse functions to place
the sound at the desired source
![Page 142: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/142.jpg)
OSSIC 3D Audio Headphones• https://www.ossic.com/3d-audio/
![Page 143: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/143.jpg)
OSSIC Demo
• https://www.youtube.com/watch?v=WjvofhhzTik
![Page 144: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/144.jpg)
Measuring HRTFs
• Putting microphones in Manikin or human ears• Playing sound from fixed positions• Record response
![Page 145: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/145.jpg)
More About HRTFs• Functions take into account, for example,
• Individual ear shape• Slope of shoulders• Head shape
• So, each person has his/her own HRTF!• Need to have a parameterizable HRTFs
• Some sound cards/APIs allow you to specify an HRTF • Check Wikipedia or Google for more info!
![Page 146: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/146.jpg)
TRACKING
![Page 147: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/147.jpg)
Tracking in Virtual Reality
• Registration• Positioning virtual object wrt real/virtual objects
• Tracking• Continually locating the users viewpoint/input
• Position (x,y,z), Orientation (r,p,y)
![Page 148: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/148.jpg)
Tracking Technologies• Active
• Mechanical, Magnetic, Ultrasonic• GPS, Wifi, cell location
• Passive• Inertial sensors (compass, accelerometer, gyro)• Computer Vision
• Marker based, Natural feature tracking
• Hybrid Tracking• Combined sensors (eg Vision + Inertial)
![Page 149: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/149.jpg)
Mechanical Tracker
• Idea: mechanical arms with joint sensors
•++: high accuracy, haptic feedback • -- : cumbersome, expensive
Microscribe Sutherland
![Page 150: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/150.jpg)
Magnetic Tracker• Idea: difference between a magnetic transmitter and a receiver
• ++: 6DOF, robust • -- : wired, sensible to metal, noisy, expensive• -- : error increases with distance
Flock of Birds (Ascension)
![Page 151: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/151.jpg)
Example: Razer Hydra
• Developed by Sixense• Magnetic source + 2 wired controllers
• Short range (1-2 m)• Precision of 1mm and 1o
• $600 USD
![Page 152: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/152.jpg)
Razor Hydra Demo
• https://www.youtube.com/watch?v=jnqFdSa5p7w
![Page 153: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/153.jpg)
Inertial Tracker• Idea: measuring linear and angular orientation rates (accelerometer/gyroscope)
• ++: no transmitter, cheap, small, high frequency, wireless • -- : drift, hysteris only 3DOF
IS300 (Intersense) Wii Remote
![Page 154: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/154.jpg)
Optical Tracker• Idea: Image Processing and Computer Vision• Specialized
• Infrared, Retro-Reflective, Stereoscopic
• Monocular Based Vision Tracking
ART Hi-Ball
![Page 155: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/155.jpg)
Outside-In vs. Inside-Out Tracking
![Page 156: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/156.jpg)
Example: Vive Lighthouse Tracking• Outside-in tracking system• 2 base stations
• Each with 2 laser scanners, LED array • Headworn/handheld sensors
• 37 photo-sensors in HMD, 17 in hand • Additional IMU sensors (500 Hz)
• Performance• Tracking server fuses sensor samples• Sampling rate 250 Hz, 4 ms latency• 2mm RMS tracking accuracy
• See http://doc-ok.org/?p=1478
![Page 157: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/157.jpg)
Lighthouse Components
Base station- IR LED array- 2 x scanned lasers
Head Mounted Display- 37 photo sensors
- 9 axis IMU
![Page 158: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/158.jpg)
Lighthouse Setup
![Page 159: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/159.jpg)
How Lighthouse Tracking Works• Position tracking using IMU
• 500 Hz sampling• But drifts over time
• Drift correction using optical tracking• IR synchronization pulse (60 Hz)• Laser sweep between pulses• Photo-sensors recognize sync pulse, measure time to laser• Know when sensor hit and which sensor hit• Calculate position of sensor relative to base station• Use 2 base stations to calculate pose
• Use IMU sensor data between pulses (500Hz)• See http://xinreality.com/wiki/Lighthouse
![Page 160: COMP 4010: Lecture2 VR Technology](https://reader034.fdocuments.in/reader034/viewer/2022050614/5a6479217f8b9a94568b4603/html5/thumbnails/160.jpg)
Lighthouse Tracking
Base station scanning
https://www.youtube.com/watch?v=avBt_P0wg_Yhttps://www.youtube.com/watch?v=oqPaaMR4kY4
Room tracking