Leap motion UI for EcoVR - Australian National...
Transcript of Leap motion UI for EcoVR - Australian National...
![Page 1: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/1.jpg)
Leap motion UI for EcoVR
Zena Wolba
![Page 2: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/2.jpg)
Recap of EcoVR
![Page 3: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/3.jpg)
Recap of EcoVR“proof of concept to explore new methods of visualising complex time-series environmental data, on the landscape.”
❖ A 2015 Techlauncher project led by Dr. Tim Brown
❖ Created using the Unreal Engine 4
❖ Compatible with Oculus Rift DK2 and Xbox/PS controller.
![Page 4: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/4.jpg)
The Data
● 20 node wireless mesh sensor network measuring humidity, temperature, soil moisture
● Dendrometers measuring trunk diameter
● Gigapixel images, UAV fly overs
● Dual Wavelength Echidna Lidar (DWEL) and Zebedee for creating dense pointclouds
Image & info from http://www.wenfo.org/wald/wp-content/uploads/2015/02/2-Tim_Brown_Sensors.pdf
![Page 5: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/5.jpg)
Data collection
○
The Site
![Page 6: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/6.jpg)
![Page 7: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/7.jpg)
![Page 8: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/8.jpg)
And the Leap Motion?❖ Developed by Leap Motion Inc.
❖ The Leap Motion controller is a USB peripheral device
❖ Tracks hands and fingers using infrared cameras and software
![Page 9: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/9.jpg)
![Page 10: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/10.jpg)
Aim of this project...❖ Substitute the clunky
Keyboard/Mouse/Xbox controller with the more intuitive Leap controller
❖ Hand gestures to toggle menus, press buttons, adjust sliders and more!
![Page 11: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/11.jpg)
Aim of this project...
![Page 12: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/12.jpg)
Aim of this project...
![Page 13: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/13.jpg)
Video!
![Page 14: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/14.jpg)
Design❖ Built upon the Leap Motion
plugin in Unreal Engine 4.11
❖ Designed to be intuitive and VR user friendly.
❖ Necessary to create a 3D interface - UI that exists in the virtual world.
❖ Some aesthetic design choices inspired by Unity 3D widgets.
![Page 15: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/15.jpg)
Innovation❖ Very limited support for 3D
interfaces in UE4.
❖ Created an all new 3D button widget for UE4, designed specifically for VR/Leap Motion.
❖ Also modified existing UE4 slider widget to be used with Leap Motion.
![Page 16: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/16.jpg)
Interactions❖ Menus toggled with
hand gestures.
❖ All widgets designed to respond to touch.
❖ Gives freedom to use VR freely.
![Page 17: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/17.jpg)
How does it work? - Menus❖ Uses UE4 functions which
get and manipulate vectors.
❖ Uses up and right vectors as axis.
❖ Then rotates the forward vector about these axis to decide where to place widgets.
![Page 18: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/18.jpg)
How does it work? - Buttons❖ 3D object.
❖ Floats in a fixed location in front of the player.
❖ Physics enabled - responds to the user’s touch.
❖ 2 types: normal and toggle button.
❖ Hover text changes colour to register touch.
![Page 19: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/19.jpg)
How does it work? - Sliders❖ Uses existing UE4 slider
template.
❖ Masked on a 2D plane.
❖ Also hovers in front of player and responds to the player’s touch.
❖ Had to implement algorithm to translate touch location to a point on the slider.
![Page 20: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/20.jpg)
How does it work? - Sliders
CenterTouch location
x
![Page 21: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/21.jpg)
What’s next?❖ Implement locomotion.
❖ Create a means to handle input for users without VR headsets or Leap Motion (using existing UI).
❖ This UI will replace the old EcoVR UI.
![Page 22: Leap motion UI for EcoVR - Australian National Universitycourses.cecs.anu.edu.au/courses/CSPROJECTS/16S1...Developed by Leap Motion Inc. The Leap Motion controller is a USB peripheral](https://reader036.fdocuments.in/reader036/viewer/2022062415/5fdee3b74f829967a97a8422/html5/thumbnails/22.jpg)
Thank you for watching.
Thank you for watching!