Interaction Techniques for Common Tasks in Immersive Virtual Environments

44
Interaction Techniques Interaction Techniques for Common Tasks in for Common Tasks in Immersive Virtual Immersive Virtual Environments Environments Design, Evaluation, and Application Design, Evaluation, and Application Doug A. Bowman April 27, 1998

description

Interaction Techniques for Common Tasks in Immersive Virtual Environments. Design, Evaluation, and Application Doug A. Bowman April 27, 1998. Introduction Methodology Travel Selection/Manipulation Application Remaining Work. Vision. Immersive VEs for productivity - PowerPoint PPT Presentation

Transcript of Interaction Techniques for Common Tasks in Immersive Virtual Environments

Page 1: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Interaction Techniques for Interaction Techniques for Common Tasks in Immersive Common Tasks in Immersive

Virtual EnvironmentsVirtual EnvironmentsDesign, Evaluation, and ApplicationDesign, Evaluation, and Application

Doug A. Bowman

April 27, 1998

Page 2: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 2

VisionVision

Immersive VEs for productivity

Complex applications for real work

Example: immersive modeling and design

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 3: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 3

DefinitionsDefinitions

Interaction Technique (IT): Method used to complete a task via a human-computer interface (hardware & software)

Immersive VE: A real-time 3D synthetic environment that appears to surround the user in space√ HMD with head tracking, CAVEX“Fishtank VR”, MUDs, Multimedia apps

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 4: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 4

A Brief History of VEsA Brief History of VEs

1968: Sutherland’s Ultimate Display Hardware advances

– displays – trackers – 3D graphics– input devices – haptics – 3D audio

Software advances– view culling – level of detail– VE toolkits – collision detection

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 5: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 5

VE ApplicationsVE Applications

In Use:– architectural walkthrough– phobia treatment– games (e.g. 1st person shooter)

Proposed:– information visualization and retrieval– modeling and design– constructivist education

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 6: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 6

Interaction: the Interaction: the Distinguishing FactorDistinguishing Factor

Current applications– may involve movement through VE– may involve shooting or pointing

Proposed applications– require 3D navigation and selection– require 6 DOF manipulation (object placement)– require large command spaces

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 7: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 7

How to improve VE How to improve VE InteractionInteraction

better design of techniques systematic evaluation (formative and

summative) in the context of applications and

requirements

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 8: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 8

Universal Tasks: TravelUniversal Tasks: Travel

Viewpoint Motion Control: The user’s interactive control of the position and orientation of his viewpoint

Wayfinding: Cognitive process of determining a route, using landmarks, maps, etc.

Navigation: VMC + Wayfinding

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 9: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 9

Universal Tasks: Universal Tasks: Selection & ManipulationSelection & Manipulation

Selection: Specification of one or more objects from a set– as the object of a command– to begin manipulation

Manipulation: Specification of the position, orientation, and/or scale of an object

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 10: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 10

Why not natural Why not natural interaction?interaction?

Term “VR” implies replication of real world

Why not use well-developed human skills to accomplish tasks in VEs?– travel: walking or driving– selection & manipulation: grasp and place

These mappings are intuitive, but too limited

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 11: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 11

Interaction Techniques Interaction Techniques and Input Devicesand Input Devices

input devices are only the hardware component of an IT

input device does not determine IT many ITs can be implemented with a single

input device we will not design or evaluate devices we will design and evaluate ITs for common

VE input devices

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 12: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 12

Problem Statement: Problem Statement: I I will...will...

analyze universal tasks and create taxonomies of techniques

design new techniques based on these formal frameworks

design, implement, and conduct formal evaluations of IT performance

apply the results to a complex and useful VE application

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 13: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 13

Design and Evaluation Design and Evaluation MethodologyMethodology

Taxonomization and Categorization Guided Design Performance Measures Range of Evaluation Methods Testbed Evaluation

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 14: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 14

Taxonomization and Taxonomization and CategorizationCategorization

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Task

Subtask

Technique Component

Task analysis Consider techniques for

low-level subtasks Promotes deeper

understanding of task Framework for design Framework for evaluation

Page 15: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 15

Guided DesignGuided Design

Design new techniques based on taxonomy, not simply intuition

Choose a component for each low-level subtask

Easy to see holes in design space

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Task

Subtask

Technique Component

1 2

3 4

Page 16: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 16

Evaluation MethodsEvaluation Methods

Range of performance metrics (quantitative and qualitative; productivity and user-centric)

Range of methods (user studies, usability evaluation, formal experiments)

Consideration of outside factors (characteristics of task, environment, user, system that might affect performance)

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 17: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 17

Testbed EvaluationTestbed Evaluation

testbed: representative set of tasks and environments

evaluate techniques for overall performance in a wide range of situations

vary technique components and outside factors

measure several performance variables generalizable and replicable

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 18: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 18

Summary of MethodologySummary of MethodologyIntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Taxonomies Perf. Metrics Outside Factorsspeedaccuracycomfort...

environment densityuser’s reachtask difficulty...

Initial Evaluation and Design

Page 19: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 19

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Taxonomies Perf. Metrics Outside Factors

Applications

speedaccuracycomfort...

environment densityuser’s reachtask difficulty...

requirements

Initial Evaluation and Design

Summary of MethodologySummary of Methodology

Page 20: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 20

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Taxonomies Perf. Metrics Outside Factors

TESTBED EVALUATION

Applications

speedaccuracycomfort...

environment densityuser’s reachtask difficulty...

requirements

Initial Evaluation and Design

Summary of MethodologySummary of Methodology

Page 21: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 21

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Taxonomies Perf. Metrics Outside Factors

TESTBED EVALUATION

PerformanceMeasurements/

ModelsApplications

speedaccuracycomfort...

environment densityuser’s reachtask difficulty...

requirements

choice oftechniques

Initial Evaluation and Design

Summary of MethodologySummary of Methodology

Page 22: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 22

Informal EvaluationInformal Evaluation

based on observations– default gaze-directed steering– lack of published work

based on our own applications– Conceptual Design Space– Virtual GIS

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 23: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 23

Initial TaxonomyInitial Taxonomy

Task: Move from the current location to the desired location

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Viewpoint MotionControl

Direction/TargetSelection

Velocity/AccelerationSelection

Conditions of Input

gaze-directedpointingphysical props

gestureslow in, slow outphysical props

start/stop buttonsautomatic start/stopconstant movement

Page 24: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 24

Performance MeasuresPerformance Measures

Quantitative (e.g. speed, accuracy) Qualitative (e.g. presence) User-Centric (e.g. ease of use, comfort)

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

IT Apps

Quality Factors-speed-accuracy-cognitive load-presence-spatial awareness- ...

Page 25: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 25

Simple ExperimentsSimple Experiments(Bowman, Koller, and Hodges, VRAIS ‘97)(Bowman, Koller, and Hodges, VRAIS ‘97)

Absolute Motion– no difference between gaze and

pointing Relative Motion

– pointing superior to gaze Spatial Awareness

– teleportation causes disorientation– any continuous motion does not

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 26: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 26

Expanded FrameworkExpanded Framework

Absolute vs. Relative Motion– same techniques, different

results– highlights need to consider

outside factors Consider task, user, system,

and environment characteristics

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Task

Pe

rfo

rma

nce

absolute relative

gaze-directed steering

pointing

Page 27: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 27

Complex ExperimentComplex Experiment Does travel IT affect cognitive load? Task: gather as much info as

possible Variables:

» IT: gaze, pointing, torso» Environment: 1-, 2-, or 3-

dimensional» System: collision detection

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 28: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 28

Guided DesignGuided Design

Taxonomy: “tour” technique– environmental target selection– gesture-based velocity selection– explicit or automatic stop inputs

Intuition: travel based on manipulation– cross-task technique– still fits in taxonomy

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 29: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 29

Final Framework and Final Framework and TestbedTestbed

Rework taxonomy to be more general– task analysis: 2 basic position-setting methods

are specifying destination, specifying trajectory– distinction allows better fitting of techniques

VMC Testbed– still in design stage– based on evaluation framework

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 30: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 30

Initial TaxonomyInitial Taxonomy

Based on metaphor, not task Arm-extension metaphor

– touch and place object with virtual hand– hand may extend beyond normal range

Ray-casting metaphor– point at object to select– manipulate by attaching to virtual light ray

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 31: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 31

Informal EvaluationInformal Evaluation(Bowman and Hodges, I3DG ‘97)(Bowman and Hodges, I3DG ‘97)

Studied six techniques (4 AE, 2 RC) Simple user study (comments, observations) Eleven subjects used techniques to place

furniture in a room Results

– AE excels at manip., RC better at selection– selection & manipulation should be separated

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 32: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 32

HOMER TechniquesHOMER Techniques

Hand-Centered Object Manipulation Extending Ray-Casting

Hybrid technique Select: ray-casting Manipulate: v. hand

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Time

Page 33: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 33

HOMER TechniquesHOMER Techniques

Hand-Centered Object Manipulation Extending Ray-Casting

Hybrid technique Select: ray-casting Manipulate: v. hand

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Time

Page 34: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 34

HOMER TechniquesHOMER Techniques

Hand-Centered Object Manipulation Extending Ray-Casting

Hybrid technique Select: ray-casting Manipulate: v. hand

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Time

Page 35: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 35

HOMER TechniquesHOMER Techniques

Hand-Centered Object Manipulation Extending Ray-Casting

Hybrid technique Select: ray-casting Manipulate: v. hand

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Time

Page 36: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 36

Formal TaxonomyFormal Taxonomy

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Selection

Manipulation

Release

Indication of Object

Indication to Select

Attachment

Positioning

Orientation

Indication to Release

Final ObjectPosition/Orientation

touchocclude

buttongesture

hand moves to objectuser scales to touch object1-to-1hand motion mapping

match tracker orientationindirect control

buttongesture

remain in placeuse physics model

Page 37: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 37

Evaluation FrameworkEvaluation Framework

Performance measures similar to travel Important outside factors:

– task characteristics: DOFs to manipulate– user characteristics: reach, spatial ability– system characteristics: constraints used

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 38: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 38

Guided DesignGuided Design

testbed implemented to allow arbitrary combinations of technique components

4608 possible combinations - reduced to 667 via dependencies and constraints

Taxonomy: gaze-based HOMER with separate positioning and orientation

Intuition: manipulation based on travel (cross-task technique)

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 39: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 39

Selection/Manipulation Selection/Manipulation TestbedTestbed

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Tasks that test all important aspects of a select/manip. IT

Selection variables: distance, size, density

Manip. variables: distance, accuracy, DOFs required

Page 40: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 40

Application Case Study: Application Case Study: Immersive DesignImmersive Design

Verify evaluation results in a complex VE application

Design system involves all universal tasks

Choose ITs based on testbed results and specified application requirements

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 41: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 41

Interaction RequirementsInteraction Requirements

Travel– exploration and goal-based movements– spatial awareness, info gathering, ease of use

Selection– accuracy at a distance, speed, comfort

Manipulation– expressibility, accuracy, ease of use

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 42: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 42

Three Levels of Three Levels of Interaction DesignInteraction Design

Naive design– taken from CDS application(in D. Bertol, Designing Digital Space)

– gaze-directed steering, ray-casting “Intuitive” design iteration

– current implementation– pen & tablet, pointing, Go-Go technique

Final, systematic design

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 43: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 43

Remaining WorkRemaining Work

complete design and evaluation framework for travel

design, implement, and run travel testbed complete and run selection/manipulation

testbed modify application interaction design and

verify with a usability study

IntroductionMethodologyTravelSelection/ManipulationApplicationRemaining Work

Page 44: Interaction Techniques for Common Tasks in Immersive Virtual Environments

Doug Bowman - VE Interaction Techniques 44

ContributionsContributions

formal understanding of tasks/techniques testbeds for future evaluations performance results and models new interaction techniques useful and usable immersive design

application