Achmea technight - HoloLens development
-
Upload
alexander-meijers -
Category
Technology
-
view
38 -
download
0
Transcript of Achmea technight - HoloLens development
![Page 1: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/1.jpg)
HoloLens developmentAchmea TechnightApeldoorn, februari 2017
Alexander Meijers
![Page 2: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/2.jpg)
Even kennismaken...• Alexander Meijers• Architect @ ETTU
Apps | Hololens | Azure SharePoint | Office 365 | Dev | Functional |
Architecture Love the Community Speaker | Blogger | 3 kids Fitness |
Airsoft Warhammer 40K
@ameijers [email protected]
![Page 3: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/3.jpg)
AgendaSESSIE 1
• History, emplacement and fields of application
• Microsoft HoloLens inside out• HoloLens development• Unity Explained
• Camera location, position and forward
SESSIE 2
• Holographic Objects and the gaze • Gestures and pinning objects• Spatial perception• Meshes to planes
![Page 4: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/4.jpg)
History, emplacement and fields of application
![Page 5: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/5.jpg)
History of HoloLens• Codenamed Project HoloLens• Chief inventor Alex Kipman
• Official title is technical fellow• He dreamed up Kinect in at the end of 2007
and set a vision which incorporated HoloLens
• It is the start of a transforming world• In the new reality, sensors will be anywhere • A visual computing platform controlled by speech and gesture
Satya Nadella - “The emergence of the next computing interface”
![Page 6: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/6.jpg)
Digital workspace“Mixed reality is becoming part of the Smart Workplace”
Modern workplace
Advanced intranet
Traditional intranet
Mail & Calendaring
Mobile access
Document management
Unified Communications
Enterprise Search
Selfservice applicaties
Business proces automation
Office applications
Microblogging
Instant Messaging
Social Networking
CRM
Supply chain management
Extranet collaboration
NewsSocial Business
tools
RSS feeds
Intranet collaboration
HR systems Corporate information
PhonebookPolicies and regulations
Line of Business Systems
Corporate App store
Smart workplaceAugmented/Virtual reality
Internet of Things
Machine Learning
Virtual Digital Assistants
![Page 7: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/7.jpg)
“Augment reality is a live direct or indirect view of a physical,
real-world environment whose elements are augmented by
computer generated sensory input such as sound, video,
graphics and data”
![Page 8: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/8.jpg)
Different approaches
• Augment Reality (AR)• Mixed reality• Interact with the physical, real-
world environment• Virtual Reality (VR)
• No transparency to the physical real-world environment
![Page 9: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/9.jpg)
Fields of application•Enables the user to visualize surrounding which is not directly visible
•Get visual insight of an object or model
•Modelling of non-existing objects
visualization
•Enrich real-world environment objects with information
•Show information as part of the real-world on a wall or table
•Show additional information to objects, persons or activities
Provisioning of information
•Support through linked view, voice and gestures
•Meetings around Holographic objects
•Working together on projects whereby visualization is an important part
•Working together when distance is involved
Working together
•Experience learning •Getting more realistic teaching by moving 2D presentations in books to 3D holographic models
Education
•Holographic media•Enrich media with additional holographic objects
Media & entertainment
•Play games as part of your real-world environment
•Game interaction with your walls, ceiling and floor
Gaming
![Page 10: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/10.jpg)
Detecting electrical wiring
• A company needs to add additional electrical wiring.
• Detailed wiring plans are projected over a wall or ceiling
• Based on detailed plans shown through HoloLens he can decide where to add
Example of visualization
![Page 11: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/11.jpg)
Elevator maintenance
• Detailed information about the elevator through a dashboard
• A 3D representation of parts of the elevator to allow quickly find the problem
• Ability to contact others through phone and mail for help
Example of visualization
![Page 12: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/12.jpg)
Enhance your BI desktop
• Getting better insights of BI data through 3D models
• Extend the desktop with 2D/3D information screens
Example of provisioning of information
![Page 13: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/13.jpg)
Working together on car safety
• Sharing the same view, scenarios and data of a car simulation with sensors
• Getting help from each other and make it easier to explain on what you are working on
Example of working together
![Page 14: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/14.jpg)
Learning about the human body
• Getting a realistic 3D projection of the human body
• It allows easy layering of the body and mark different body parts
• By walking around the projection and viewing and marking body parts will help students to learn more quickly
Example of education
![Page 15: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/15.jpg)
…
• Become part of different squares in Rome
• Rome is viewed as a 360 degree view around you in all directions
• Images are enhanced wih holographic additions like the inside of the church and the fountain in the front.
Example of media and entertainment
![Page 16: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/16.jpg)
Mixed reality during gameplay
• HoloLens allows us to play games which become part of our real-world environment
• Fragments is a CSI game in which you, together with virtual people solve crime delicts
• Virtual people use your room as if there are actually there
Example of gaming
![Page 17: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/17.jpg)
Microsoft HoloLens inside out
![Page 18: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/18.jpg)
Microsoft Hololens• First Holographic computer running on
Windows 10• Contains a CPU, GPU and HPU• Human understanding
• Gesture and voice input• Gaze tracking• Spatial sound
• Uses wi-fi to connect and has no wires attached
• Design for comfort
![Page 19: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/19.jpg)
Microsoft Hololens specifications• Windows 10 device based on
32 bit architecture• 64GB flash • 2GB memory• 1GB Holographic Processor Unity (HPU)
• First of its kind• Device is more powerfull than a laptop
• No overheating due to warm air flows to the sides• 2-3 hours active and 2 weeks standby• Weight 579g
![Page 20: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/20.jpg)
Microsoft Hololens specifications• Contains depth camera
• Field of vision that spans 120 by 120 degrees
• 18 sensors flooding the device with terabytes of data every second
• Tricks your brain into perceiving holographic images• Light engine in which light particles are
bouncing million times• Photons enter the two lenses• Ricochet some between layers of blue,
green and red glass• Hitting back of the eye
![Page 21: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/21.jpg)
Microsoft Hololens specifications• Device can project a
hologram into a room and keep it locked in position• Called pinnnig
• Objects are not moving relative to you but you can move around the object• Allows you to view the
object from any angle
![Page 22: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/22.jpg)
Microsoft Hololens specifications• Device scans your
environment and builds a digital model in real time• It allows HoloLens to see
different surfaces, walls and ceiling
• Possible to simulate a physical space• E.g. Projecting a terain
over your floor
![Page 23: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/23.jpg)
Sensors, optics and speakersEnvironment
camera
High definition lenses
Spatial soundwith
speakers
Depth camera
Video camera
![Page 24: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/24.jpg)
Integration coming up• Windows Insiders
will unexpected see a “Holographic” entry on the main page of Settings
• Fast ring only• Build 15007• It has been removed
from the current builds
![Page 25: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/25.jpg)
HoloLens development
![Page 26: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/26.jpg)
Development tools• Visual Studio 2015 Update 3• Windows 10 SDK (version 1511 or later)
• There is no separate SDK for HoloLens• Unity 5.5 or higher• HoloLens emulator
• HoloLens Emulator (build 10.0.14393.0)• Hyper-V• Contains DirectX project templates for
Visual Studio• HoloLens device
![Page 27: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/27.jpg)
Vuforia• Allows you to develop holographic apps
which recognize specific things in the environment and attach experiences to them• Recognizes specific images and objects
in the environment
• Vuforia engine• Extend the capabilities of your
holographic app• Includes support for OpenGL ES 3.x• Uses Cloud recognition services
![Page 28: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/28.jpg)
Build Lifecycle HoloLens project
• Create Unity Project• Configure HoloLens
settings• Create scene
Unity
• Configure build settings
• Build and generate Visual Studio project
Convert• Open project with
Visual Studio• Pair with HoloLens• Build Visual Studio
project
Visual Studio
• Deploy application• Start Application
HoloLens
![Page 29: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/29.jpg)
Develop for HoloLens• HoloLens Device
• Developer mode needs to be turned on
• Pair the device with your development machine
• Retrieve IP address of HoloLens Device• Ask Cortana• Settings Network & Internet
Advanced options
![Page 30: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/30.jpg)
Windows Device Portal• 3D View• Mixed Reality Capture• Performance
• Performance tracing• System performance• Processes
• Apps• Maintenance• Crash dumps
• Additional tools• Logging• File Explorer• Kiosk Mode• Simulation• Virtual Input
![Page 31: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/31.jpg)
Unity Explained
![Page 32: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/32.jpg)
Unity explained Scene
Main Camera
DirectionalLight
Scripts
Components
Script Coupled as component
Game Objects
Assets
![Page 33: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/33.jpg)
Unity Explained - Prefabs• Predefined Game Objects• Contains components with
predefined values• Prefabs used in Scene
inherit the components and settings
![Page 34: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/34.jpg)
Unity explained – Script for GameObject• Derived from MonoBehaviour• Contains methods which are called
by Unity through the a broadcast messaging system
• void Awake() – Called when class is instantiated
• void Start() – Called when object is enabled once before the first Update()
• void Update() – Called once per frame
• More methods are available
![Page 35: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/35.jpg)
Unity explained – Message systempublic void BroadcastMessage(string methodName, object parameter = null, SendMessageOptions options = SendMessageOptions.RequireReceiver);
• Used for calling methods on GameObjects• Calls the methods also on its Children• Examples
• this.BroadcastMessage(“OnSelect”);• focusedObject.BroadcastMessage(“OnSelect”);
![Page 36: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/36.jpg)
Components• Transform
• Position of object• Rotation of object• Scale (size) of object
• Mesh Renderer & Mesh Filter• Renders the object• Mostly done using a mesh
• Mesh Collider• Determines the collision for the object
• Script• Perform actions on the object• Handle messages of the object
![Page 37: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/37.jpg)
Paint 3D• Creates fbx files• Easy to import• Need some additional work to
use them in Unity• Removing nodes• Moving meshes up• Set all components at highest
level> Mesh Render> Mesh Filter> Mesh Collider> Other
http://www.remix3d.com
![Page 38: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/38.jpg)
Positioning Unity and HoloLens
![Page 39: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/39.jpg)
forward (x,y,z)Camera (0, 0, 0)Rotation (x,y,z)
Camera (1, -1, 4)Rotation (x,y,z)
HolographicObject Pinned
HoloLensmoved to
new location
View directionGaze
Positioning Unity and HoloLens
![Page 40: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/40.jpg)
{{
Democamera location,
position and forward
![Page 41: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/41.jpg)
Wat zijn onze conclusies?• Nog geen volwassen product• Integratie met SharePoint Online loopt voorop• Alleen toepassen voor eenvoudige data verwerking
![Page 42: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/42.jpg)
AgendaSESSIE 1
• History, emplacement and fields of application
• Microsoft HoloLens inside out• HoloLens development• Unity Explained
• Camera location, position and forward
SESSIE 2
• Holographic Objects and the gaze • Gestures and pinning objects• Spatial perception• Meshes to planes
![Page 43: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/43.jpg)
{{
DemoHolographic objects
and the gaze
![Page 44: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/44.jpg)
{{
Demogestures and
pinning objects
![Page 45: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/45.jpg)
{{
Demospatial perception
![Page 46: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/46.jpg)
{{
Demomeshes to planes
![Page 47: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/47.jpg)
Developer resources
![Page 48: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/48.jpg)
GitHub development resources• Galaxy Explorer - application build by
Microsoft as part of the Share Your Idea campaign.
• HoloLens Companion Kit - This is a toolkit style repo for code bits and components that *do not run on HoloLens* but pair with HoloLens to build experiences.
• HoloToolkit-Unity – Toolkit for HoloLens and contains Unity specific components
![Page 49: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/49.jpg)
Other resources• Windows Dev Center – Documentation about
learning the fundamentals of holographic development
• Academy – hands on with code, tutorials and lessons
• Windows Holographic Developer Forum – Forum about HoloLens App Development
• Unity tutorials – Learning about the engine, services and production
• Windows blog – Lots of business articles found about HoloLens
• Vuforia – Examples, articles and support on the use of HoloLens with Vuforia
![Page 50: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/50.jpg)
![Page 51: Achmea technight - HoloLens development](https://reader038.fdocuments.in/reader038/viewer/2022102720/58a272201a28ab94628b5f1f/html5/thumbnails/51.jpg)
Bedankt voor jullie aandacht! Alexander Meijers, ETTU
<the end/>