8/14/2019 Gestural Interaction

1/13

8/14/2019 Gestural Interaction

2/13

Scene AugmentingIntelligence

LARGE SCALE RICH INFORMATIONSPACE

UNCOVERING PATTERNS FOR DECISION

MAKING

FINDING THE STORY IN THE GAPS

MEDIATIONAL MEANS

MATERIAL OF MANIPULATION

GESTURALINTERFACE

HighFidelity

SELECTIONNAVIGATION

SHAPING

WHOLEBEING

INTELLIGEN

CEANALYSIS

INTUITION

LayeredExpression

VisualConfiguration

Geek

Grok

8/14/2019 Gestural Interaction

3/13

Human Communications

Layered channels of expression

Voice with tonal variation

Facial qualifiers

Body positions

Spatial distance to objects

Hand movements

Writing, drawing

8/14/2019 Gestural Interaction

4/13

8/14/2019 Gestural Interaction

5/13

Technology as MediationalMeans

Remote versus Local communication

Asynchronous versus synchronous

Human to Computer to Human

Technology provides the object,symbols, material to manipulate

Need to learn the affordance of theinterface

8/14/2019 Gestural Interaction

6/13

Improving Fidelity of HCI

Signals Add gestural interface to complement

symbol and voice commands.

Initially only hand movement Future include facial expression

Response

Visual screen that fill peripheral vision Spatial Audio

Expressive 3D object in motion

8/14/2019 Gestural Interaction

7/13

8/14/2019 Gestural Interaction

8/13

Whole being Intelligence

Intelligence is in the nervous systemas well as the brain

Human judgment under uncertainty

uses whole being intelligence Much information is ambiguous and

vague

Human comprehension is based onimpressions of patterns

Interpretation of representations

needs advances

8/14/2019 Gestural Interaction

9/13

Augmenting HumanIntelligence

HCI as a moderating socialengagement

Two negotiating parties

Fusing interpretations ofrepresentations

Synergistic symbiotic interactionthrough Gestures

8/14/2019 Gestural Interaction

10/13

Scenario

Saul manipulates representationusing gestural interface

Ava manipulated the system

configuration Saul uses intuition to find patterns

while Ava provides analysis to extract

the material and representations The material is large scale rich

information space needing discoveryand analysis to uncover patterns for

decision making

8/14/2019 Gestural Interaction

11/13

Finding the Story

The scenario fosters story telling

Pattern recognition has limitation

People fill the gaps by creating ahidden narrative using interpretiveexplanation

In business the story is left tointuition

The intuition dominants decision

making in real-world situations

8/14/2019 Gestural Interaction

12/13

Dance of GesturalInteraction

Discovery of the aesthetic ofrepresentation and expression thatfosters better augmented human

judgment

Archeology of knowledge as told bythe story of a social experience

manipulating the material of richinformation spaces

8/14/2019 Gestural Interaction

13/13

Research is needed

Whole being intelligence

Cognitive interfaces

Improve Human to Computer toHuman social communications

Adopt Gestural Interfaces to enhanceexpression

Goal should be to improve humanjudgment under uncertainty wheninformation space is large