Unique Two-Handed Interaction Techniques

Unique Two-Handed Interaction Techniques

Manipulating graphic objects on large displays

By:

Martin A. N. Khamala

Student

Matrikelnummer: 952366

Interactive Media Systems (MA)

Faculty of Art and Design

Hochschule Augsburg, Augsburg University of Applied Sciences (AUAS)

Friedberger Str. 2, D - 86161 Augsburg

Email: [email protected]

Mobile: +49 1520 536 9830

Antony Sande

Student

Matrikelnummer: 952374

Interactive Media Systems (MA)

Faculty of Art and Design

Hochschule Augsburg, Augsburg University of Applied Sciences (AUAS)

Friedberger Str. 2, D - 86161 Augsburg

Email: [email protected]

Mobile: +49 1520 536 9176

This report is submitted in partial fulfillment of the requirement of the Interactive Media Systems

Masters course in Design.

Date: 5-February, 2016

Unique Two-handed Interaction Techniques - Manipulating graphic objects on large displays 2 | 31

Table of Contents

Acknowledgements 3

Abstract 4

1.0 Introduction 5

2.0 A use case in Education 6

3.0 Methodology 7

4.0 Findings 15

5.0 Conclusion and Extended Work 16

6.0 Appendices: Demo Video 17

7.0 Appendices: Research Instruments 18

8.0 Bibliography 31

List of Figures

Figure 1: Cue the system from inside the near zone: 12

Figure 2: Highlight and select from the near zone: 12

Figure 3: Move Transformation from the near zone: 12

Figure 4: Highlight and select: 13

Figure 5: Scale Transformation: Figure 13

Figure 6: Cue the pointing zone: 14

Figure 7: Highlight and Select in the pointing zone: 14

Figure 8: Transform the object using two-handed gestures. 14

List of Tables

Table 1: Sampling Table of success 11

Table 2: Sampling Table of success 15

List of Charts

Chart 1: Comparison sheet of success rates between activity 1 and 2 16


ACKNOWLEDGEMENTS

We acknowledge with appreciation the contribution of all the people who played a part in the success of

this study. Special mention goes to our lecturer and project supervisor, for his assistance with the

proposal ideation, conceptualization, planning, development and evaluation over the semester.

1. Prof. Dr. Michael Kipp Lecturer and course supervisor, Interaction Engineering

Also, deserving of mention are the following officers,

2. Herald Lesti The Faculty of Informatik

3. Prof. Dr. Phil. Doris A. Binger: Text, Culture and Media Studies, International Faculty Coordinator

University of Applied Sciences Augsburg and Faculty of Design

4. Finally, we wish to acknowledge the following students who accepted to participate in evaluating our

prototype:

Christian Spiegel, WirschaftInformatik student

Stefan Berner, Interactive Media Systems (MA) student

Stefan Reichinger, Interactive Media Systems (MA) student

Patrick Schroeter, Interactive Media Systems (MA) student

Sebastian Harter, Interactive Media Systems (MA) student


ABSTRACT

Gesture interactions are being deployed increasingly, in Human Computer Interaction, in an attempt to

create more natural interfacing with the plethora of new digital environments. If these new applications

are to successfully enter into use, in our day to day lives, their development must be underpinned in

equal measure, by robust and intuitive interfacing methods. This research implementation looks at two-

handed spatial gestures and moots the question that seeks to inherit the merits of isotonic gesture

recognition on touch screens, in physical space.

The advantage of evolving unique gesture sets, from 2D isotonic interactions is that there exists the real

possibility, that the new styles will be easy to learn and intuitive to use. This research explores this

possibility by building a prototype that responds to two-handed gestures sets. The underlying purpose is

to construct manipulation styles to transform computer graphics. These styles are comprised of unique

combinations of two-handed gesture inputs, location data, and proximity information. To keep the

system as intuitive as possible, this implementation draws from the extensive library of information

available in two finger isotonic gesture actions.

The prototype is evaluated in terms of gesture recognition and orientation, while its use is demonstrated

with the execution of two object transformations: specifically, move and scale. Users will be tested on

how successfully they manipulate the graphics using the prototype’s predefined styles.


1.0 INTRODUCTION

Background

This study proposes to examine natural and guided human interaction with 2D objects on a projected

display. Emphasis will be laid on using proxemic interaction and two hands to simulate two finger

isotonic gestures in physical space. Input data will be captured using a motion sensing device. The data

throughput will be processed using a gesture recognition method and output to manipulate graphic

objects using “move” and “scale” transformations.

1.1 The Opportunity for Research

During presentations, there is often a need to hold viewer’s attention while navigating on-screen

information by repositioning, enlarging, rotating or revealing an area of focus. Any break from the

presentation to access a laptop to interface with the information comes at a cost of attention by placing

demands on viewer's physical and cognitive resources.

This research study’s innovation is the evolution and implementation of unique manipulation styles that

provide natural two-handed interfacing while transforming graphics on a large projected display area.

1.2 The Research Question

The contention is, whether the physical manipulation of computer graphics using two-handed

manipulation styles on large displays can benefit from leveraging the merits of isotonic gestures already

in widespread use.

The hypothesis asserts that Human Computer Interaction techniques are translatable from a 2D realm,

as with multi-touch screens to a physical 3-dimensional space without the loss of the natural know-how

in intuitively manipulating graphic objects. The researchers proffer the view that, to meaningfully

emulate 2D gesture recognition, manipulation styles in physical space must take into consideration

location, distance, and orientation input data to cue and execute a unique combination of gestures.


1.3 The Objective

The objective of this research implementation is to investigate the suitability of using natural, physical

two-handed gestures that mimic 2D isotonic gestures, for use in transforming a rectangular object on

large display screens.

1.4 The Scope

The scope of the project is limited to building a prototype to demonstrate Human Computer Interaction

techniques and testing the validity of the assertion made in the hypothesis about two-handed gesture

movements.

The project’s success will be determined by the complete implementation of the prototype, the

demonstration its function and the completion of the documentation of the project findings from

usability tests.

1.5 Assumptions

The main assumption that presentation methods do not go far enough in offering presenter’s ways to

hold viewer attention on specific areas as they navigate areas on-screen will be tested. Often time’s

presenter’s retreat to their laptops to manage viewer prompted manipulation interactions involving

repositioning, zooming, rotating and revealing of information.

2.0 Use case for Education

In blended teaching environments, teachers using presentation technology in the classroom experience

few navigational problems when the presentation is linear. But what happens during a question and

answer session when to aid in expounding a concept, the teacher needs to zoom into and reposition a

particular part of a diagram on-screen. User prompted explanations such as these, are usually preceded

by a trip to and from a laptop. Software on laptops allows the presenter to set the navigation style

accordingly. Should the explanation require a reference from another part of the diagram, another trip to

and from will be prompted. Herein lies a situation in which learner attention is broken each time.


3.0 METHODOLOGY

The development of the interface technique

i) Through a series of brainstorming sessions, it was determined that the simplest hand gestures to effect

two-handed interactions are those that mimic isotonic gestures, which through widespread usage over

the passage of time, have become intuitive to most users.

ii) The next step determined the implementation concept for the project. The idea was to project a graphic

onto a large display area. An individual designated to a focus group test would move the object using his

hands.

iii) The next step was to determine a set of manipulation styles correlating to paired gesture actions that

allow for selection in both the near and pointing zone in the experiment. From the sensor hardware

limitations, the near zone was set to 30 cm from the display area and the pointing zone was set from

30cm to 3m the display area.

The manipulation styles that were used were segmented according to proximity zones:

In the near zone (set at 0-30 cm)

a) Orientation: use facial recognition data to determine intent. The idea being to establish that the user is, in

fact, facing the presentation and, therefore, intends to interact with it.

Proxemics: use distance input data to determine that the user is in the near zone.

Highlight: use the location data of the first hand, as input data to determine whether there is a hit

between the hand and the object.

Select: use a closing fist gesture, made with the other hand-form below the waist line, as a grabbing

action to select.

Move: use location input data, to detect the motion of the first hand, to update the position of the

projected graphic object.

Scale: use location input data, to detect the distance between both hands, to update the distance between

hands along a connection line between them.

In the pointing zone (set at 30-300 cm)


iv) Orientation: use facial recognition data to determine intent. The idea being to establish that the user is, in

fact, facing the presentation and, therefore, intends to interact with it.

Proximity: use distance input data to determine that the user is in the pointing zone.

Highlights: using the location data of the first elbow and hand as input data to determine whether there is

a hit between the arms projection and the object.

Select: using a closing fist gesture, made with the other hand-form below the waist line, as a grabbing

action to select.

Move: using location input data, to detect the motion of the first arm, to update the position of the

projected graphic object.

Scale: using location input data, to detect the distance between both hands, to update the distance

between hands along a connection line between them.

v) Next, a developer must determine a methodology for calibrating the screen to let the correlate the

projected display to the virtual space.

The scheme of work.

To program a pair of hand gestures, a developer must track each hand's movement change. This proved

to be tricky when you consider that the Kinect sensors tracking system at times failed to track the hand

movement. This irregular sensor detection resulted in inconsistent event handling on the code. For

example, the coordinates for any hand movements close to the limit of the kinetics’ field of view would

be dropped from the motion tracking feedback.

To decide on when to draw on the screen, a developer must track the hand action. To confirm that a

selection between the real hand the virtual graphic object has been registered,

The developer must analyze the hand's real position and relate its position in virtual space. This is done

using a linear interpolation formula, that relates at any one time, to one of the four corners of the display

box.

This step is repeated rapidly to update the graphic position as often as possible for the action of:

a) Moving the box

A developer must calculate the new position of the hand in relation to the last position. This describes a

vector in the line of motion of the hand and can be used to reposition the graphic object.


b) Scaling the box

A developer must capture the coordinates of both hands and track the change in distance between them.

vi) UX testing

The size of our sample group was 5 respondents. All are male and are attending university.

The user context for the test was a medium sized room of approximately 12 x 12m.

The ambiance was quiet.

Each test comprised two activities, and each activity comprised of 9 tasks. (Ref. Appendix for details).

The results for each respondent was documented after the test.

3.2 Interaction hardware and software

We expect to make use of:

- Microsoft Kinect v2 Sensor device, for live motion data capture.

- A projector to build a prototype demonstrating the projects outcomes.

- A laptop to create a desktop application to analyze and implement the new gesture data.

Prototype

In order to track the presenter’s body, the Microsoft Kinect v2 Sensor, an integrated set of sensors

designed to detect human movement, voice and gesture, was used. Its depth sensing technology, a high-

definition colour camera, and the infra-red emitter were the features used in the prototype.

The prototype is a Windows 8.1 App, with the interface developed using XAML (Extensible Application

Markup Language) and the logic developed in C# programming language using Microsoft Visual Studio

2013, and it was built on the Kinect for Windows Software Development Kit (SDK) 2.0.

The KinectSensor exposes a set of Sources that provide streams of a specific data type. Typically a

Reader is attached to a Source and the FrameArrived event of the Reader is subscribed to. Using this, we

were able to subscribe to the BodyFrameSource which allowed us to track all the body joints at an

output range of 30 Frames per Second (FPS). Since the Frame objects are being created and destroyed at


a rapid rate, we incorporated Reactive Extensions, which are a great way to handle the torrent of event

data that the Kinect generated. Adding a reactive extension layer above the raw Kinect event handlers

allowed us to abstract away the messy details and simplify the application significantly.

The Kinect Sensor tracks up to 6 people, but we only needed one person to control the presentation. The

BodyFrame returns the set of bodies that the Kinect is currently tracking. From the array of bodies, we

then calculated the closest body to the presentation (by comparing distance data). We then passed the

reference onto a second Source; the HighDefinitionFaceFrameSource, which allowed us to access the

face alignment of the tracked user. With this were able to know when they were facing the presentation.

We then tracked the user's hands, spine, wrists and elbows coordinates, which we used in manipulating

the object.

The setup involved a Kinect Sensor mounted on the ceiling beside a projector that was projecting

presentation on a wall. The Kinect sensor was placed approximately 3.5 Metres from the wall.

In order for the Kinect to detect the presentation area, the user had to perform calibration first by

touching each of the four corners of the presentation area with their right hand. Then the system will

map the presentation area with the Kinect’s Coordinates. After that, the user can start interacting with

the presentation by using his hands as a pointer. In order to grab the object, we monitored the

HandStates from the Kinect and detected whether it was Open or Closed (object grabbed).

3.3 Mapping Methods

We suggest the following interface interactions that combine distance and location proximity input data

with two unique spatial gestures, that will allow presenters to intuitively navigate on-screen information.


Table 1 - Sense, select, and transform gestures

Event

Application

function

Manipulation Style

Interaction

outcome Cue Technique

Proximity input Gesture

combination

Cue the interaction

from the near zone

Orientation is facing the display at a

distance of 0 – 30 cm from the display

screen

Hold hand over the

selectable object

Selectable object

highlights

Cue the interaction

from the pointing

zone

Orientation is facing the display at a

distance of 0.3 – 3m from the display

screen

Point to the

selectable object

Selectable object

highlights

Move object with

selecting hand Distance of hand from display

Touch or hover

with hand and

move

Move the display

screen

Enlarge object with

selecting hand Distance of hand from display

Touch or hover

with one hand and

move the other arm

up and down

Scale the display

screen with selecting

hand as anchor and

the other hand as

scale slider

Move object with

selecting hand

Determines the (x, y, z) coordinates of

the shoulder and hand

Point to select, and

then concurrently

move the left hand

up/down

Calculates the target

position of the

pointing hand

Enlarge object with

selecting hand

Determines the (x, y, z) coordinates of

the shoulder and hand

Point to select, and

then concurrently

tilt the hand

up/down

Calculates the

amount of zoom and

is applied

progressively.


3.4 Manipulation style list

Figure 1: Cue the system from inside the near zone: Use the first hand using an open hand gesture.

Figure 2: Highlight and select from the near zone: Select the object with the first hand (left) and use the

second (right) hand to make a grabbing fist gesture.


Figure 3: Move Transformation from the near zone: Move the object by repositioning it using the first

hand. Note the second hand is still in a fist action to keep the selection valid.

Figure 4: Highlight and select: Use the first hand (left) to hover over the graphic and the second (right)

hand to make a selection by using a grabbing action.

Figure 5: Scale Transformation: Open arms apart to enlarge and bring them together to reduce.


Figure 6: Cue the pointing zone: Step away from the display to stand between 30 to 300cm.

Figure 7: Highlight and Select in the pointing zone: The first-hand highlights the second-hand selects.

Figure 8: Transform the object using two-handed gestures. Move the first hand while the other is in a fist

to reposition the graphic. And move both hands simultaneously to reduce or apart to enlarge.


4.0 FINDINGS

The respondents were guided on how to complete an activity and were then timed to see how long it

took to complete a task. An early finishing time (under 2 minutes) done correctly translated to a perfect

score of 2, whereas a late finishing time completing the tasks after 2 minutes translated to a score of 1.

Completing the task after 2 minutes in more than 1 attempt or not completing the task translated to a

score of 0.

Activity 1 Highlight using first hand, select using second hand, reposition by moving first hand

Activity 2 Highlight using first hand, select using second hand, scale by moving both hands

For both activities, the active Orientation is facing the display

Table 2: Sampling Table of success

Activity 1 Activity 2

Name Score Attempts

Success Score Attempts

Success

rate rate

1 Sebastian Spiegel 14 8 1.75 14 13 1.08

2 Stefan Berner 14 8 1.75 11 13 0.85

3 Sebastian Harter 16 8 2.00 19 11 1.73

4 Patrick Schroeter 11 8 1.38 15 8 1.88

5 Stefan Reichinger 11 15 0.73 12 17 0.71

Score guide Success rate scale 2.00 perfect

1.5 to 2 good

1.0 to 2 above average

0.5 to 1 average

0 to 0.5 below average

All respondents are male, between the age of 24 to 30 years and university students.


Activity 2

Activity 1

i. Most respondents achieved a higher success rate with Activity 1 which was done in close proximity to

the display area, far easier than activity 2.

ii. The comparison of both activity results appeared to be consistent as they are outside the margin of error.

iii. More than half the respondents scored an above average score in activity one, whereas more than half

scored a below average performance in activity 2.

3 CONCLUSION AND EXTENDED WORK

From the analysis of the results, it seems that the manipulation of the graphics was achieved from close

proximity with less effort that at a distance using the pointing action. From respondents comments on the

questionnaire, it would seem the difficulty arose from the unsteadiness of the pointer action.

Respondents, in general, attributed the failure to complete the tasks quickly was attributed to a difficult

selection process. This could be attributed to the sensor when provided inconsistent tracking

functionality.

From the completion of the exercise to the point of data gathering means the objective was achieved and

the outcome, to implement and test the hypothesis was successful.

In closing, the manipulation of graphic objects was not as natural compared to isotonic gesture action.

The selection and movement action were made more difficult by hardware limitations.

On the matter of the two handed selection approach, respondents, in general, felt it was not natural, and a

one handed approach would be more intuitive.

-

0.50

1.00

1.50

2.00

2.50

SebastianSpiegel

Stefan Berner SebastianHarter

PatrickSchroeter

StefanReichinger

Suce

ss r

ate

sco

res

Names of respondents

Chart 1: Comparison sheet of sucess rates between activity 1 and 2


4 APPENDIX

4.1 Link to video demonstration: https://vimeo.com/154373014

https://vimeo.com/154373014


4.2 Questionnaire template

Research Instrument:

Data capture sheet No:

Project Title: Date:

Two-Handed Interaction

Techniques 29 Jan 16

Supervisor: Researchers:

Test: iteration No. A. Sande

Prof. Michael Kipp M. Khamala

Most Viable Product 1 Course: Unit:

Respondent Name: Age Gender Handedness

Interactive Media

Systems (MA)

Interactive

Engineering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1

2

3

Activity 1

Task 1: Orientation: Face away from the display in the neutral zone, then turn to the display

Task 2: Highlight object - single-handed gesture, standing less than 30cm from the display zone

Task 3: Select object - 2 hand gesture, make a fist with the other hand while highlighting with the first hand.

Task 4: Move the object to a new location on the display using the highlighting hand.

Task 5: While still in the display zone, highlight the object again using a single-handed gesture.

Task 6: Select the object by making a fist using the other hand while highlighting with the first hand.

Task 7: Enlarge the object by parting hands in a 2 handed gesture.

Task 8: Reduce the object size by bringing hands together in a 2 handed gesture.

Activity 2

Task 1: Orientation: Face away from the display in the neutral zone, then turn to the display

Task 2: Highlight object - single-handed gesture, standing in the pointing zone: 30cm to 3m from the display.

Task 3: Object selection - 2 handed gesture, make a fist with the other hand while pointing with the first hand.


Task 4: Move the object to a new location on the display

Task 5: While in the pointing zone, highlight the object by pointing again using a single-handed gesture.

Task 6: Select the object by making a fist using the other hand while pointing with the first hand.

Task 7: Enlarge the object by parting hands in a 2 handed gesture.

Task 8: Reduce the object size by bringing hands together in a 2 handed gesture.

Performance Index:

2: complete success

1: moderate success

0: failure

Recorded by: Name Signature:


Questionnaire No:


Towards natural,

2-handed gestures for 2D

interactions

29 Jan 16






Interactive Media

Systems (MA)

Interactive

Engineering

The Interaction prototype is a 2D interaction system that uses 2 hand interfacing to move and scale screen objects.

The user objective during the experiment was to move the object around and as quickly and efficiently as possible.

Having completed the activities, please answer the questions below:

1 Have you interacted with sensor technology before, to manipulate or navigate content?

Yes No A little

2 To what extent do you feel that the prototype was helpful in moving the screen object.

Very helpful Helpful Not helpful impossible

3 How natural where the 2 hand gestures in manipulating the object.

Very natural natural Not natural unnatural

4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of

efficiency.

5 Little effort needed Some effort needed A lot of effort needed Too much effort needed


6 Which activity was easier for you to execute?

Activity 1 while in the display zone Activity 2 while in the pointing zone

Explain:

7 Which task was most difficult for you to execute?

Ref. the back of the sheet for the task list.

Explain.

8 What would you suggest as improvements to the whole system?

9 Is there a user case in which you see yourself using the prototype system?

Thank you.


5.2.1 User 1


Data capture sheet No: 001

Project

Title: Date:



Supervis

or:

Resear

chers:

Test: iteration No. A.

Sande

Prof.

Michael

Kipp

M.

Kham

ala Most Viable Product 2 Course: Unit:


Interactive Media

Systems (MA)

Interac

tive

Sebastian Spiegel 30 Male Right Engin

eering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1 2 2 2 1 2 2 1 2 2 2 2 1 1 1 0 2

2 0 1 1 1

3 0



Yes No x A little


Very helpful x Helpful Not helpful impossible



Very natural x natural Not natural unnatural

4 How efficient was the interaction between you and the prototype? Use the amount of effort you had to apply as a measure of efficiency.

5 Little effort needed

x Some effort needed

A lot of effort needed

Too much effort

needed


x Activity 1 while in the display zone Activity 2 while in the pointing zone

Explain:

At the Activity second I can't control the point. It flickered too much


4, act. 2


Explain.

Because I can't move it I can't grab it good.


Not answered


Not answered












Interactive Media

Systems (MA)

Interactive

Engineering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1 2 2 2 2 2 2 1 1 2 2 0 1 0 0 0 0

2 0 1

3 1 2 2



Yes No x A little


Very helpful x Helpful x Not helpful impossible




efficiency.

5 Little effort needed Some effort needed x A lot of effort needed Too much effort needed




Explain:

Tracking worked on the first try



Explain.

Tracking of arm didn't work well


Additional Hardware may be needed on your wrist for better tracking


Maybe at exhibitions and conventions












Interactive Media

Systems (MA)

Interactive

Sebastian Harder 25 Male Right Engineering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1

2 2 2 2

3



x Yes No A little







efficiency.


Some effort needed

x A lot of effort needed

Too much effort needed



Explain:

The selection in Activity 2 was more difficult


3, act 2


Explain.

Selection didn't work properly (maybe I wasn't in the correct pointing zone)


I think it would be more natural to use 1 hand while moving, pointing, fist, then move


Maybe for interactive presentations












Interactive Media

Systems (MA)

Interactive

Patrick Schroeter 24 Male Right Engineering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1 2 2 2 1 1 1 1 1 2 2 2 2 2 2 2 1

2

3



x Yes No A little







efficiency.


x Some effort needed

A lot of effort needed

Too much effort needed


Activity 1 while in the display zone x Activity 2 while in the pointing zone

Explain:

You have a better overview because of the distance from the screen,


3, act. 2

Ref. the back of the sheet for task list.

Explain.

It did not work the way I expected


More precise hardware


Dashboard to move and resize information












Interactive Media

Systems (MA)

Interactive

Stefan Reichinger 25 Right Engineering

Att

emp

ts


Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

Tas

k 1

Tas

k 2

Tas

k 3

Tas

k 4

Tas

k 5

Tas

k 6

Tas

k 7

Tas

k 8

Act

ivit

y

Co

mp

leti

on

tim

e

(min

:sec

)

1 2 1 1 0 0 0 0 0 2 2 0 0 0 0 2 0

2 1 1 1 1 1 2 0 0 0 0 2 0

3 2 0 1 1



x Yes No A little


Very helpful Helpful x Not helpful impossible




efficiency.

5 Little effort needed Some effort needed x A lot of effort needed Too much effort needed


Activity 1 while in the display zone x Activity 2 while in the pointing zone


Explain:



Explain.

Selecting the object


Just better selection and hand detection


Presentations


5.0 BIBLIOGRAPHY

1. Towards user-defined multi-touch gestures for 3D objects; Authors: Sarah Buchanan, Bourke Floyd,

Will Holderness, Joseph J. LaViola; Publisher: ACM New York, NY, USA ©2013

2. Toward localizing audiences' gaze using a multi-touch electronic whiteboard with PieMenu; Authors:

Kazutaka Kurihara, Naoshi Nagano, Yuta Watanabe, Yuichi Fujimura, Akinori Minaduki, Kushiro,

Hidehiko Hayashi, Yohei Tutiya; Publisher: (13–16, 2011), Palo Alto, California, USA. Copyright 2011

ACM

3. Adaptive gesture recognition with variation estimation for interactive systems; Authors: Baptiste

Caramiaux, Nicola Montecchio, Atau Tanaka, and Fred´ eric Bevilacqua. 2014. Publisher: ACM Trans.

Interact. Intell. Syst. 4, 4, Article 18 (December 2014),

4. The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies; Authors:

Nicolai Marquardt1, Robert Diaz-Marino2, Sebastian Boring1, Saul Greenberg

Publisher: UIST’11, October 16–19, 2011, Santa Barbara, CA, USA. Copyright © 2011

5. The use and abuse of PowerPoint in Teaching and Learning in the Life Sciences: A Personal

Overview; Author: ALLAN M JONES; Publisher: Life Sciences Teaching Unit, Old Medical School,

University of Dundee, Dundee, DD1 4HN, UK

6. Cognitive Supports for Analogies in the Mathematics Classroom

Authors: Lindsey E. Richland,1* Osnat Zur,2 Keith J. Holyoak2; Publisher: Department of Education,

University of California, Irvine, CA 92697, USA.

7. Virtual sensors: rapid prototyping of ubiquitous interaction with a mobile phone and a Kinect; Lauren

Norrie, Roderick Murray-Smith; Publisher: ACM, August 2011

8. Kinect analysis: a system for recording, analyzing and sharing multimodal interaction elicitation

studies; Authors: Michael Nebeling, Carnegie Mellon University; David Ott, ETH Zurich; Moira C.

Norrie, ETH Zurich; Publisher: ACM New York, NY, USA ©2015

9. Teaching natural user interaction using OpenNI and the Microsoft Kinect sensor

Authors: Norman Villaroman, Dale Rowe, Bret Swan; Publisher: ACM, October 2011

Unique Two-Handed Interaction Techniques

Documents

Transcript of Unique Two-Handed Interaction Techniques