#Build2016

Spatial Audio - Spatial Sphere Demo ExplainedSteven WilssensSenior Program ManagerSilicon Graphics and Media

Please watch the talk first.

http://aka.ms/spatialaudiosessionbuild2016

What is Spatial Audio again?

Spatial audio gives the illusion that sounds are coming directly from an object in a 3-dimensional space. They are audible all around the user.

When you hear something as if you would in real-life… coming from an application. That’s spatial audio.

What is Spatial Audio Again?

UWP developers can convincingly place sounds in a 3-dimensional space (sphere) by using Microsoft’s HoloLens HRTF technology.

HRTF technologies work with all stereo headphones.You can now create spatial audio experiences in UWP apps and games on HoloLens, Desktop, Phone, and Xbox One.

Spatial Audio in Windows 10 @ //BUILD 2016

What are HRTF’s again?

Humans have just two ears, but can locate sounds in three dimensions:• Distance• Direction (above/below)• Front/Back, Left/Right

HRTF (Head-related transfer function)

This is possible because the brain, inner ear and the external ears (pinna) work together to make inferences about location.

We take sound localization cues from one ear (monaural cues), and compare them with cues received at both ears (difference cues or binaural cues).

HRTF (Head-related transfer function)

• Left or right cues originate from differences in the time sound

• Up and down, front and back cues originate from spectral changes produced by the outer ear shape (pinnae).

• Distance cues originate from intensity and reflection differences.

HRTF transfers a sound to how an ear would receive that sound from a point in space.

The AudioGraph 1.0 API

Hello Old Friend - AudioGraph 1.0

Easy Coding Mic In Speaker

OutPlay from

FileSave to

FileLow

Latency Effects

AudioGraph 1.0

🎤 📢

📄 💾

🔀

🎤

🔀

💠

DeviceInputNode

FileInputNode FileOutputNode

DeviceOutputNode

SubmixNode

IBasicAudioEffect

The AudioGraph 1.1 API

AudioGraph 1.1

🎤 📢

📄 💾

🔀🎤

🔀

💠

DeviceInputNode

FileInputNodeFileOutputNode

DeviceOutputNode

SubmixNode

IBasicAudioEffect

AudioNodeEmitter

AudioNodeListener

Emitters and Listeners• Spatial audio source• Can be input or submix

node• Supported # of emitters

depends on multiple factors

• Orientation, Position, Velocity

Emitter

• Spatial receiver of audio• Only supports 1

listener/AudioGraph• Orientation, Position,

Velocity

Listener

DEMO: Spatial Audio - AudioGraph

{Put on headphones to experience Spatial Audio}

http://aka.ms/spatialspheredemocode

DEMO: Spatial Audio – AudioGraph

📢📄

🔀🔀💠

FileInputNode DeviceOutputNode

SubmixNode

ReverbAudioEffect

AudioNodeEmitter

Gain: 0.125

Gain: 1

HRTF-based audio rendering with headphones greatly improves when headphones can track head orientation

Reduces:• Front-Back confusion• Sound Source Elevation confusion

Head movements from 2° to 32° may be required to ensure that spatial information is conveyed with high accuracy.

17

HRTFs and Head Orientation Tracking

DEMO: Head Tracking and AudioGraph Listener Position

Powered by:

Jabra Intelligent Headset

Advanced sensor pack and dynamic 3D audio - and exciting new apps platform!

https://intelligentheadset.com/

{Put on headphones to experience Spatial Audio}

http://aka.ms/spatialspheredemocode

DEMO: Spatial Audio – AudioGraph

📢📄

🔀🔀💠

FileInputNode DeviceOutputNode

SubmixNode

ReverbAudioEffect

AudioNodeEmitter

Gain: 0.125

Gain: 1

AudioNodeListener

• Use AudioGraph to add spatial audio to UWP apps

• Emitters position sound sources in space• Listeners where you hear the audio• Head tracking increases the ability for users

to localize sound• It’s easy to enable spatial audio in your

UWP apps!

Summary

• Download the Spatial Audio Sphere Sample (http://aka.ms/spatialspheredemocode)• Add Spatial Audio Sound to your UWP today

• Re-visit Build on Channel 9.• Continue your education at

Microsoft Virtual Academy online.

Call to Action