State of Augmented Reality & HoloLens

PowerPoint Presentation

AUGMENTED REALITY

Hkan [email protected]@haqwin

1

Physical reality

Augmented reality

Virtual realityvirtuality continuum

Lets not hope our augmented reality will be this overwhelmingBut still the ability to be nudged with ongoing phone call informationAnd if you like messages and notificationMaybe just not all at once3

Lets not hope our augmented reality will be this overwhelmingBut still the ability to be nudged with ongoing phone call informationAnd if you like messages and notificationMaybe just not all at once4

HISTORY

But before we go what we can experience now and in the future we should start out by looking into some history

5

Ivan SutherlandFather of computer graphicsMITSketchpad

Ivan Sutherland Well know graphics workSketchpadLight pen UI and ability to sketch on screenPaper on a head mounted display

6

1968

University of UtahWith Bob Sproul he created the first HeadMountedDisplay as he figured that would be the best way to experience a 3D model

7

The ultimate displaySimple wireframes

8

Sword of Democles

As it was held in place with a large metal construct from above + to be able to track the movement9

1980

EyeTapSteve Mann

Steve Mann

11

Steve Mann with one of his inventions early iterations of EyeTapHeadMounted computerBut also a pioneer in AR

12

1990

Augmented RealityTom Caudell

The work with AR has been developed by Boeing as a way to help in aviation industryResearcher Tom Caudell coined the term AR

It was used to help tech + Handling the service+ but also more full exoskeleton devices to remote control system

14

NASA X-38 escape podThat was created to sit on the ISS and be able to safely get up to 7 people back to earthThis could be remote controlled and here we see an overlay of the landing site and obstacles on top of the thick cloud to be able to have a safe landingUnfortunately it was put on hold and the project was cancelled

15

NASA X-38 escape podThat was created to sit on the ISS and be able to safely get up to 7 people back to earthThis could be remote controlled and here we see an overlay of the landing site and obstacles on top of the thick cloud to be able to have a safe landingUnfortunately it was put on hold and the project was cancelled

16

2000

Up until this point most work has been done in aviation industry and on universities but around 2000 things changed

17

Hirokazu KatoARToolKit

Japanese Hiro ccretaed ARToolKitAnd now and it started to spread18

You were able to overlay an image through theThe printed media started to take some interest around nowThrough a web cam or laptop with cam19

both for the common developer and in more commercial and consumer use and we started to se ad campaigns with this techniques

20

21

2013

google

Google presented their glass23

Although very limited and actually nor much more than a head mounted display it show some of the future in head mounted displays and self containment

That brings us up till now

24

NOW

HUD displays and Aviation + medical are currently heavy users of AR already but for consumer and enterprise users there are not as many devices out yet25

Meta 2

The META 2

90-degree field of view.2560 x 1440 display resolution.Sensor array for positional tracking.

26

Project Tango

Googles system that uses depth cameras and acceelerometes to handle the sorrundings

27

Project Tango

Googles system that uses depth cameras and acceelerometes to handle the sorrundings

28

HoloLens

Physical reality

Augmented reality

Virtual realityMixed reality

Inside the machine

31

Output

See through lenses2 HD light engines Pupilar distance calibration

Surround speakersSpatial sound engine

32

Output



33

Output



34

Output



35

Sensors

4 environment sense cameras1 depth camera1 12MP HD video camera1 ambient light sensor1 inertia measurement unit4 microphones

36

Sensors


37

Sensors


38

Sensors


39

Sensors


40

Its the combination of stuff that makes it uniqueNo strings attached - only this device - no external computers 500g headheld device with everything in it

But that is of course not what makes it unique - its the mixed reality mode where the real world and the digital world meet

41

Processing

Just an ordinary Intel based Windows 10 machineWith an extra powerful HPU - 24 DSP cores, capable of 1 trillion calculations per second (using 1GB mem)+ Small laptop grade CPU for all the other stuff (the HPU is around 200 x as powerful as a normal CPU) (using 2GB mem)

And the usual PC related stuff.. WiFi, Bluetooth, 60G disk and all that

42

Processing

Just an ordinary Intel based Windows 10 machineWith an extra powerful HPU - 24 DSP cores, capable of 1 trillion calculations per second (using 1GB mem)+ Small laptop grade CPU for all the other stuff (the HPU is around 200 x as powerful as a normal CPU) (using 2GB mem)

And the usual PC related stuff.. WiFi, Bluetooth, 60G disk and all that

43

INTERACTION

InteractionGazeTapHoldBloomSpeech

Speach

Possibilities (add leap motion?)


Speach



Speach



Speach


Interaction

GazeTapHoldBloomSpeech

SelectPlace Face meBigger/smaller- Hey Cortana - record video, take picture, etc

Advantages

Self contained - Inside out - Windows holographicNo wires! - Knows its surroundings

The goal here is to push the tech. The usages can be as wide as ArchitectureMedicine Home furniture / decoratingManufacturing

50

HURDLES

What are the biggest hurdles for widespread use of AR/MR?

Price of course but that will change quicker than expectedINTERACTION

51

Finger gestures

On screen interaction

Simplified gesture set

Interaction

The interaction are quite like the Leap Motion sensor barHand and finger input, seems to be quite detailed but I havent been able to confirm that myself

Speech?

Consistency and StandardsDiscoverability FeedbackAffordance

Interaction

Consistency and StandardsThis is something we have to work on right now. How do you navigate, how do you interact. How do you interact with the digital part of a hologram. The gestures for the HoloLens is just the beginning.

DiscoverabilityThe discoverability need to be close to the real world. But you want to be able to manipulate and interact in ways that are not limited by the reality. This is the real challange.

FeedbackThe user should get feedback on actions. And in mixed reality this feedback need to work with real objects.Give virtual object some snapping/persistanse or visual queues when interacting with real objects

Affordance - Depends on context;What makes sense for different scenarios, in simulation, scaling do not make sense. But in analysis or some creative activities it makes perfectly sense. Is the object rooted in the real world?

FUTURE

Rony Abovitz

Magic Leap

The $1B vaporware?

Rony Abovitz

[T]he hardware projects an image directly onto your retina through an optics system built into a piece of semitransparent glass (the product wont fry your eyeballs; its replicating the way we naturally observe the world instead of forcing you to stare at a screen). The hardware also constantly gathers information, scanning the room for obstacles, listening for voices, tracking eye movements and watching hands.55

HoloLens 2

- Electrochromic dimming module- Better gesture and finger control

- FoV- More power / battery56

QUESTIONS?

Hkan [email protected]@haqwin

57

Hkan [email protected]@haqwinTHANKS!

58

State of Augmented Reality & HoloLens

Technology

Transcript of State of Augmented Reality & HoloLens