Lecture 4. Greenfoot 1 Pablo Romero, Department of Informatics Kinect in Greenfoot.

Lecture 4. Greenfoot 1 Pablo Romero, Department of Informatics

Kinect in Greenfoot

Kinect in Greenfoot 2 Pablo Romero, IIMAS, UNAM

Lecture plan

Installing kinect drivers The sample scenarios

The Painting scenario Modifying the code


The scenarios

Painting Ball Depth Mirror Trail Xray


The Painting scenario

World Classes KinectWorld

• PaintWorld

Actor Classes Instructions Label Brush Thumbnail

Other Classes Joint KinectClient UserData Point3D


KinectWorld The basic world class for kinect scenarios Make a subclass for the specific scenario

super(…) in constructor super.act() in act()

Methods isConnected() getTrackedUsers() getAllUsers() getThumbnail() getDepthAt(int x, int y)


KinectClient

Establishes communication and fetches data from the kinect

Used by the world classes (kinectWorld and its specific subclass)

Methods IsConnected(), update(), disconnect() getUsers() getThumbnail()


UserData Holds data from the kinect for one user

id, state, an array of joints, image, etc. getTrackedUsers(), getAllUsers() from

kinectWorld return an array of UserData Used frequently by other classes Methods

drawStickFigure, getAllJoints, getNearestJoint, getHighestJoint

getters and setters for its fields


Joint

Holds data for one joint as a Point3D 15 different types of joints Used by UserData but also frequently by

other classes Methods

getX(), getY(), getJointIndex()


So far

Greenfoot environment provides a connection to use kinect KinectWorld and KinnectClient main classes

provided to interact with kinect Particular application world created as a

subclass of KinectWorld UserData, Joint and Point3D also useful, re-

usable classes


The design

Informal presentation Game should be challenging

Without kinect (with xbox control, intellectual part)

Only with physical part Both intellectual and physical

Intellectual and physical well integrated


Exercise Now change the program so that a the person’s

image and not the stick figure is displayed In the act

• Comment drawing the stick figure• Create a user image (getCombinedUserImage)• Scale that image to the width and height• Draw that image (after drawing the background)


The brush

In the constructor Creates a green dot as a greenfoot image

In the act Positions and paints the dot on the right

hand’s coordinates If the left hand is higher than the head clears

the screen


Exercise How would you change the behaviour of the

brush so that it paints as if the image had been flipped both vertically and horizontally? In Brush.java

• Modify the coordinates of the brush so that they are deducted from the dimensions of the world (getWidth() and getHeight() from World)


Exercise How would you change the behaviour of the brush so

that what it paints falls down the screen until it disappears? This can be done independently from the previous exercise In the act of Brush.java

• Create an arrayList to store the coordinates of each position of the right hand

• Increment each of the y coordinates in the arrayList every time to make them fall. If they go out of the screen delete them


Abstraction and modularization

Modularization is the process of dividing a whole into well-defined parts, which can be built and examined separately, and which interact in well-defined ways.

Abstraction is the ability to ignore details of parts to focus attention on a higher level of a problem.


Falling drawings Create a class for the coordinates Create an ArrayList to a group of coordinates

Declare the ArrayList at the beginning of Brush.java Initialise it in the constructor Store the coordinates in the act method

Modify the Y values of the ArrayList Redisplay the modified coordinates


Big exercise Write a simple game that requires

people to remember a set of postures. The game could consist of two parts: A program to record the set of postures to

use in the game The game proper where the set of

postures recorded before are compared with the user’s postures


Recording postures Use the starting program for postures. This

program tracks a user, displays the instructions of the game and a white background. It has a PostureWorld, subclass of KinectWorld, that does those three tasks. Additionally it has a Posture, subclass of Actor that has the user’s image as its image and displays it

Give the user a 3 sec. warning to make a pose and make a beep sound

Record the postures in a file as a set of joint 2D coordinates


The second step

if (leftHandUp == -1) { leftHandUp = System.currentTimeMillis();} else if (System.currentTimeMillis() - leftHandUp > 5000)

postureWarning = new Label("3",100);

import java.awt.Toolkit;

…

Toolkit.getDefaultToolkit().beep();


Handling errors

Not always programmer’s error File processing particularly error-prone

Missing or corrupt files Lack of appropriate permissions

Errors of this type have to be handled and reported (exception handling) Done with the try statement


The try statement

Code catching an exception must protect the call with a try statement:

try { Protect one or more statements here.}catch(Exception e) { Report and recover from the exception here.}


The try statement

try { addressbook.saveToFile(filename); tryAgain = false;}catch(IOException e) { System.out.println("Unable to save to " + filename); tryAgain = true;}

1. Exception thrown from here

2. Control transfers to here


Text input-output

Input-output is particularly error-prone. It involves interaction with the external

environment. The java.io package supports input-

output. java.io.IOException is a checked

exception.


Text output to file

Use the FileWriter class. Open a file. Write to the file. Close the file.

Failure at any point results in an IOException.


Text output to filetry { FileWriter writer = new FileWriter("name of file"); while(there is more text to write) { ... writer.write(next piece of text); ... } writer.close();}catch(IOException e) { something went wrong with accessing the file}


The postures game

Displaying the postures Comparing the player’s posture to the

recorded posture


Displaying the postures

Different program but can use the starting program The Posture world class responsible for the

overall game• Reading the data file to fill in an ArrayList of

UserData• Displaying the postures

The Posture class responsible for drawing the stick figure and changing the posture


Text input from file Use the FileReader class. Augment with BufferedReader for line-

based input. Open a file. Read from the file. Close the file.

Failure at any point results in an IOException.

Use the split string method


Text input from filetry { BufferedReader reader = new BufferedReader(new FileReader("filename")); String line = reader.readLine(); while(line != null) { do something with line line = reader.readLine(); } reader.close();}catch(FileNotFoundException e) { the specified file could not be found}catch(IOException e) { something went wrong with reading or closing}


Filling in the ArrayList

Use the UserData class as it can draw the stick figure.

Fill in an ArrayList of UserData Just fill in the joints Make the positionWorld and positionScreen

the same The z coordinate can be missing Confidence can be 0


Displaying the postures

In the act of PostureWorld (remember the act is within a loop)

Use a field to keep the index of the current posture and to avoid out of bounds

Use another field to indicate whether it is time to change the posture (similar to the one to keep the elapsed time when warning the user to make a pose)

Lecture 4. Greenfoot 1 Pablo Romero, Department of Informatics Kinect in Greenfoot.

Documents

Transcript of Lecture 4. Greenfoot 1 Pablo Romero, Department of Informatics Kinect in Greenfoot.