Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07...

14
Unity & AR Lab 07 Renato Mainetti Jacopo Essenziale [email protected] [email protected]

Transcript of Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07...

Page 1: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Unity & AR

Lab 07

Renato MainettiJacopo Essenziale

[email protected]@unimi.it

Page 2: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

From reality to VR(passing by AR)

2

video see-through deviceHUD

HMDNothing

Page 4: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

How does AR typically works: (wiki)

• A key measure of AR systems is how realistically they integrate augmentations with the real world. The software must derive real world coordinates, independent from the camera, from camera images. That process is called image registration which uses different methods of computer vision, mostly related to video tracking.

4

Page 5: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Feature image recognition

5

Usually those methods consist of two parts:• The first stage is to detect interest points, fiducial markers or optical flow in the

camera images. This step can use feature detection methods like corner detection, blob detection, edge detection or thresholding and/or other image processing methods.

• The second stage restores a real world coordinate system from the data obtained in the first stage. Some methods assume objects with known geometry (or fiducial markers) are present in the scene. Mathematical methods used in the second stage include projective (epipolar) geometry, geometric algebra, rotation representation with exponential map, kalman and particle filters, nonlinear optimization, robust statistics.

Page 6: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Vuforia:

6

Page 7: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Hololens, Vuforia & Unity

7

Vuforia’s support for HoloLens is implemented in version 6.1 of the Vuforia Unity extension

Page 8: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Deploy to:

• Andorid

• iOs

• …

• Hololens

8

Page 9: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Hololens

9

Weight: 579 gramsCPU: quad-core Intel Atom x5-Z8100 1,04 GHz, GPU & HPU (Holographic Processing Unit)

Ram: 2 GB of RAMDisk: 64 GBBattery: 16.500 mWh (2 hours)

Audio surround (spatial sound)MemsLight sensor4 cameras (ambient analisys)1 Depht cam1 cam HD (2 megapixel) photo and videos.4 mic (speech recognition)

Page 11: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Vuforia target

11

Page 12: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Setup:• Create an account

• Register your app and obtain a valid key

• Add some image to the db and evaluate features

• Download the db

• Integrate in Unity importing the unity package

• Configure Vuforia with the app key and activate the dbimported

• Add an ARcamera prefab

• Add an imageTarget prefab, specifying the target

• Parent an object to the imageTarget

• Play

12

Page 13: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Texturing in blender:How to cheat the brain…

13

Change the light color in unity (match the natural colour of the image target) to obtain the best result.

Page 14: Unity & AR - unimi.itmainetti.di.unimi.it/teaching/vrlab_2018/lessons/L7/L07.pdfUnity & AR Lab 07 Renato Mainetti Jacopo Essenziale renato.mainetti@unimi.it jacopo.essenziale@unimi.it.

Demo scenes and game prototype:

14