Using Vitruvius to build Kinect WPF Step by step with image! On april 25th 2017, i just found out that this guy called, VANGOS PTERNEAS, is a Microsoft Most Valuable Professional in the Kinect technology. Vangos is the owner of LightBuzz Software agency. His own website is found here: https://vitruviuskinect.com/ He also has his very own Youtube channel with tutorials and side projects he is uptaking found here: https://www.youtube.com/channel/UCI5MH26KFTbAI9ZN6pDVR0g You can read more about his latest work for kinect here: https://vitruviuskinect.com/blog/ So glad i found him! Had difficulties modifying the Microsoft sample code to suit our project’s needs. In this tutorial, you will be learning how to integrate hand gesture on your wpf projects for custom gesture such as Wave Left, Wave Right, Swipe Left, Swipe Right, Swipe Up, Swipe Down, Zoom In, Zoom Out, and Jump. If you have ever tried learning the kinect by looking at the documentation for kinect sdk v2, you would know it is a pain to first record raw video using the kinect, KSConvert it to xef format, go to Visual Gesture Builder and create your vgb solution, train a couple of recording so it reaches a good confidence level before implementing it in your WPF project. And it is SOOOO confusing to implement it to existing projects as you can tell by now. With Vitruvius you can do it in a couple of minutes. What is Vitruvius? Vitruvius is a set of easy-to-use Kinect utilities that will speed-up the development of your projects. Supports WPF and Windows Store. Requires Kinect 2. Prerequisites:

● Kinect for XBOX v2 sensor with an adapter (or Kinect for Windows v2 sensor) ● Kinect for Windows v2 SDK ● Windows 8.1 or higher ● Visual Studio 2013 or higher ● A dedicated USB 3 port ● Visual Studio 2015 and above

1

Vitruvius Gesture In WPF: We will be using this external library to get the swipe gesture. In this program we would be able to change the label text to the gesture that triggers it. In this demo we will be use Here is a step-by-step guide: STEP #1 - CREATE A NEW WPF PROJECT ON VISUAL STUDIO Go to File > New > Project…

Fig 1

Under installed, click Templates > Visual C# > Windows Classic Desktop, You should be able to choose WPF App (.NET Framework). It is selected in blue in fig 2. I will be naming the application called VitruviusGestureExample, you can name it anything you want. You can also choose the location where you want to store it and click OK.

2

Fig 2 You should see something like this. Otherwise open the MainWindow.xaml on the right (Fig 3)

Fig 3

STEP #2 - CREATE A LABEL ON A WINDOW FORM We would be creating a simple label on the windows form. The purpose is to see that our hand gestures would change the label’s text. Click on the Toolbox on the left to find the label. If you can’t find the toolbox, (Ctrl + Alt + X).

3

Fig 4 Drag and drop the label in the center of the windows form

Fig 5

Now we would change the property of the label by adding a name and increasing the font size. Name

Fig 6

4

Change the Font Size property to 40 or something more visible.

Fig 7

STEP #3 - INSTALL LIBRARIES WE NEED Now we need to install Vitruvius package so we can import it into our code. First under Tools > NuGet Package Manager > Package Manager Console.

Fig 8

5

Type Install-Package lightbuzz-vitruvius beside the PM> and press Enter.

Fig 9

Once it is completed, Let us add the microsoft kinect reference too. Right click on the project > Add > Reference...

Fig 10

6

Under Assemblies > Recent > and check Microsoft Kinect version 2.0.0.0 or above. And click OK.

Fig 11

STEP #4 - IMPORT THE LIBRARIES AND START CODING! Navigate to the MainWindow.xaml.cs, You can navigate on the solution explorer (Ctrl + Alt + L ) under MainWindow.xaml or if it is already opened, you can also select it on the tab. (Fig 4)

Fig 12

Lets import the libraries needed for this project!

using Microsoft.Kinect; using LightBuzz.Vitruvius;

Fig 13

7

Now add the sensors, Frame reader and Gesture Controller under the MainWindow class.

KinectSensor _sensor; MultiSourceFrameReader _reader; GestureController _gestureController;

Fig 14

Inside the MainWindow() method below InitializeComponent(); Add these lines in, don’t worry about the errors as we have not create the methods yet.

_sensor = KinectSensor.GetDefault(); if (_sensor != null) { _sensor.Open(); _reader = _sensor.OpenMultiSourceFrameReader( FrameSourceTypes.Body); _reader.MultiSourceFrameArrived += Reader_MultiSourceFrameArrived; _gestureController = new GestureController(); _gestureController.GestureRecognized += GestureController_GestureRecognized; }

Fig 15

Now to create the Reader for multi frame method, in this case we will only be trying to detect the body frames as stated above in Fig 15.

8

void Reader_MultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e) { var reference = e.FrameReference.AcquireFrame(); // Body using (var frame = reference.BodyFrameReference.AcquireFrame()) { if (frame != null) { Body body = frame.Bodies().Closest(); if (body != null) { _gestureController.Update(body); } } } }

Fig 16

Let us create the other method that will trigger when a gesture is recognized.

void GestureController_GestureRecognized(object sender, GestureEventArgs e) { var gesture = e.GestureType; switch (gesture) { case (GestureType.JoinedHands): break; case (GestureType.Menu): break; case (GestureType.SwipeDown): lbDisplay.Content = gesture.ToString(); break; case (GestureType.SwipeLeft): lbDisplay.Content = gesture.ToString(); break; case (GestureType.SwipeRight):

9

lbDisplay.Content = gesture.ToString(); break; case (GestureType.SwipeUp): lbDisplay.Content = gesture.ToString(); break; case (GestureType.WaveLeft): break; case (GestureType.WaveRight): break; case (GestureType.ZoomIn): break; case (GestureType.ZoomOut): break; } }

In this example we would be only doing the swipe gesture, you can customise the code or use other gesture to trigger anything you want. The code now looks somewhat complete, but before you get too excited and run the program, this error would occur.

Fig 17

10

To solve this error, we first have to set the solution platform. First click on the down arrow for Any CPU > Configuration Manager

Fig 18

In the Configuration Manager, under Platform, click on <New…>

Fig 19

11

Make sure the new platform is x64 and press OK.

Fig 20

Now you are ready to run the program!

Once a swipe gesture is detected, it should show the gesture that triggered the event! As shown below.

Fig 21

12

I will be uploading the entire project on my github page here : https://github.com/bingcheng45/VitruviusGestureExample

13