Developing for Leap Motion
-
Upload
iris-classon -
Category
Technology
-
view
118 -
download
1
description
Transcript of Developing for Leap Motion
Taking the Leap with Leap Motion
Iris Classon
About me
• Technical Evangelist at Telerik (WinRT)• Software Developer at Dotnet Mentor• Microsoft MVP C# and member of MEET• MCSD, MCPD, MCTS• Organizer Sweden Pluralsight Study Group (Gothenburg)
• Started learning programming from 0 summer 2011 • Licensed clinical Dietitian and Int.Lic. Personal Trainer
Contact details
• www.irisclasson.com• Twitter• @IrisClasson
• Email• Telerik – [email protected]• Dotnet Mentor – [email protected]
What is the Leap Motion?
To get started
• Mac, Windows, Linux (latest OS – more or less)• Developer Unit of Leap Motion• Preorder or apply to get one• Purchase: release projected for 22nd of July• https://www.leapmotion.com/
• SDK • Can’t be shared
• Forum, samples and documentation available• Projects on GitHub
• Breaking changes introduced between SDK versions- might need to update the code
Good to know
• Still in Beta• Developer units vs. consumer units:• The size and weight • Final hardware components & casing• Field of view approx 20% wider in consume version
Do not forget
• Dispose Frame object when done with it• RemoveListener from Controller when app closes• Dispose Controller afterwards
• Add Leap.DLL, LeapCSHarp.DLL & LeapCSHarp.NET4.0.DLL• Set copy output to Copy Always• Reference the LeapCSHarp.NET4.0.DLL
When in trouble w. the device
• USB-cable has been known to cause trouble• Change USB port and/or cable• Connect the Leap before running the executable• Make sure there is light and the device is clean• Calibrate screen• Use the Leap Vizualiser to check if it works
The Leap story
• Small startup based in San Fransisco• Venture capital• Has grown from just a few people to 80 employees and still hiring• Asus and HP on board
• 40 000 Devs signed up• 12 000 dev units shipped so far• Still shipping
Demo
The Airspace Store
• Storefront to gather all the apps• Dev decides on pricing, distribution & platform supported• Non-exclusive distribution• Min price starts at jus below 1 USD• Industry standard at 70% to the developer• Apps manually tested before publishing
Available languages
• C++• C#• Objective C• Java• Python• JavaScript• (Ruby => hack)
How the API works
• Leap motion detects fingers/tools
• Data captured as frames continously
• Listener class handles events from the Frame class
• Controller is the connection between device and app
• To listen for the updates on the Frame data: controller.AddListener(listener)
• Gestures must be enabled through the controllercontroller.EnableGesture(Gesture.GestureType.TYPESWIPE)
ID, Timestamp, Hands,Fingers, Tools,Gestures
Frame
Translation,RotationAxis,RotationAngle,Scalefactor
Motion factors
Code – C#class SampleListener : Listener
public override void OnFrame(Controller controller) { // Get the most recent frame and report some basic information Frame frame = controller.Frame(); SafeWriteLine("Frame id: " + frame.Id + ", timestamp: " + frame.Timestamp + ", hands: " + frame.Hands.Count + ", fingers: " + frame.Fingers.Count + ", tools: " + frame.Tools.Count + ", gestures: " + frame.Gestures().Count);
if (!frame.Hands.Empty) { // Get the first hand Hand hand = frame.Hands[0];
// Check if the hand has any fingers FingerList fingers = hand.Fingers; if (!fingers.Empty) { // Calculate the hand's average finger tip position Vector avgPos = fingers.Aggregate(Vector.Zero, (current, finger) => current + finger.TipPosition); avgPos /= fingers.Count; SafeWriteLine("Hand has " + fingers.Count + " fingers, average finger tip position: " + avgPos); } ………
private static void Main(string[] args) {
var listener = new SampleListener(); var controller = new Controller(); controller.AddListener(listener);
// Remove the sample listener when done controller.RemoveListener(listener); controller.Dispose(); }
Code – JavaScript // Display Frame object data var frameOutput = document.getElementById("frameData");
var frameString = "Frame ID: " + frame.id + "<br />" + "Timestamp: " + frame.timestamp + " µs<br />" + "Hands: " + frame.hands.length + "<br />" + "Fingers: " + frame.fingers.length + "<br />" + "Tools: " + frame.tools.length + "<br />" + "Gestures: " + frame.gestures.length + "<br />";
// Frame motion factors if (previousFrame) { var translation = frame.translation(previousFrame); frameString += "Translation: " + vectorToString(translation) + " mm <br />";
var rotationAxis = frame.rotationAxis(previousFrame); var rotationAngle = frame.rotationAngle(previousFrame); frameString += "Rotation axis: " + vectorToString(rotationAxis, 2) + "<br />"; frameString += "Rotation angle: " + rotationAngle.toFixed(2) + " radians<br />";
……
// Setup Leap loop with frame callback functionvar controllerOptions = {enableGestures: true};
Leap.loop(controllerOptions, function(frame) { // Body of callback function})
Making sense of movement
The screen
Screen class
- contains computer screen information
- Position, orientation & vector Leap/PC
- Information from OS
Can be invalid
Device should to be calibrated with the screen
The tool
• Based on:• Length• Shape• Width
The gestures
• Predifined gestures:• Circle• Swipe • Key Tap (downward movement)• Screen Tap (forward movement)
Turn, twist and tilt – and fist• Use Motion factors from frame class
• Translation,• RotationAxis,• RotationAngle,• Scalefactor
UX guidelines
• Avoid complexity• Use what is considered natural• Think outside the box• Positive experience for the user• Feedback• No clutter• Desctructive action• Navigation and interaction – keep them apart
Q and A
Infra red light safety concerns• IR occours naturally in the physical world• Widely used in a variety of devices• No health risk found with general exposure of IR
Using Leap as a mouse out of the box • Developer unit no, consumer unit yes
• CPU usage• Will be lower when released• New updates ’inactivates’ the device when not used, activated when it registers
motion
Thank youIris Classon