4/23/10

1

Professor Charles Rich Computer Science Department rich@wpi.edu

IMGD 4000 (D 10) 1

Input Controls: Beyond the Keyboard, Mouse and Joypad

Technical Game Development II

For more: Take Prof. Lindeman’s new course on “HCI for Real, Virtual and Teleoperated Environments” (IMGD 3XXX) in A10...

mimicry

novelty traditional

generic

keyboard

mouse joypad

specialized

recent

steering wheel

joystick

Emotive

IOM

Wiimote

exercise equipment

musical instruments

camera

VR foot pads

2 IMGD 4000 (D 10)

4/23/10

2

Embrace alternative controllers

[From Harmonix presentation, IMGD Seminar 2/7/08] 3 IMGD 4000 (D 10)

We haven’t made a joypad game since 2003

Camera Guitar Microphone

[From Harmonix presentation, IMGD Seminar 2/7/08] 4 IMGD 4000 (D 10)

4/23/10

3

We’ve noticed some big user benefits…

Guitar + Drums + Microphone!

[From Harmonix presentation, IMGD Seminar 2/7/08] 5 IMGD 4000 (D 10)

They look like what they do

[From Harmonix presentation, IMGD Seminar 2/7/08] 6 IMGD 4000 (D 10)

4/23/10

4

It’s obvious how to use them

[From Harmonix presentation, IMGD Seminar 2/7/08] 7 IMGD 4000 (D 10)

Joypads

  Generic by design •  So can’t “look like what it does” •  No mental model for how to use it

[From Harmonix presentation, IMGD Seminar 2/7/08] 8 IMGD 4000 (D 10)

4/23/10

5

Today’s joypads are intimidating

[From Harmonix presentation, IMGD Seminar 2/7/08] 9 IMGD 4000 (D 10)

These controllers are inviting

[From Harmonix presentation, IMGD Seminar 2/7/08] 10 IMGD 4000 (D 10)

4/23/10

6

They level the playing field

[From Harmonix presentation, IMGD Seminar 2/7/08] 11 IMGD 4000 (D 10)

Ensure intuitive controls

[From Harmonix presentation, IMGD Seminar 2/7/08] 12 IMGD 4000 (D 10)

4/23/10

7

Intuitive Controls

  It just works   This is our responsibility   Employ usability principles

[From Harmonix presentation, IMGD Seminar 2/7/08] 13 IMGD 4000 (D 10)

Natural Mapping Spatial relationship between control and result is consistent

[From Harmonix presentation, IMGD Seminar 2/7/08] 14 IMGD 4000 (D 10)

4/23/10

8

[From Harmonix presentation, IMGD Seminar 2/7/08] 15 IMGD 4000 (D 10)

IMGD 4000 (D 10) 16

Other Physical Input Devices

  Foot (dance) pads [video]

  Exercise equipment

•  www.exerciseinabox.com [video]

•  Diamond Park [video]

4/23/10

9

Dance Pad

IMGD 4000 (D 10) 17

www.exerciseinabox.com

IMGD 4000 (D 10) 18

4/23/10

10

Diamond Park

IMGD 4000 (D 10) 19

IMGD 4000 (D 10) 20

Inertial and IR Sensing

  Wiimote •  sports games [video]

•  finger tracking [video]

•  head tracking [video]

4/23/10

11

Wii Tennis

IMGD 4000 (D 10) 21

Wii Finger Tracking

IMGD 4000 (D 10) 22

4/23/10

12

Wii Head Tracking

IMGD 4000 (D 10) 23

IMGD 4000 (D 10) 24

Using Cameras and Computer Vision

  EyeToy for PS2 •  AntiGrav (Harmonix) [video]

–  face tracking –  template matching –  rough pose estimation

  Webcam •  open source software, e.g., OpenCV •  cf. robotics

4/23/10

13

AntiGrav

IMGD 4000 (D 10) 25

IMGD 4000 (D 10) 26

Augmented Reality

  Eye of Judgement [video]

  Lab Demos [2 videos]

4/23/10

14

Eye of Judgement

IMGD 4000 (D 10) 27

Augmented Reality

IMGD 4000 (D 10) 28

4/23/10

15

Augmented Reality

IMGD 4000 (D 10) 29

IMGD 4000 (D 10) 30

Virtual Reality

  Prof. Lindeman’s TactaVest [video]

part of the “Playstation 6”

4/23/10

16

TactaVest

IMGD 4000 (D 10) 31

IMGD 4000 (D 10) 32

Neural and Bio Feedback

  Heart rate and skin conductance •  Wild Divine IOM [video]

–  http://www.meditations-uk.com/products/wilddivine.html

  Neural activity (EEG) •  EmotivEPOC [video]

–  http://www.emotiv.com

4/23/10

17

Wild Divine IOM

IMGD 4000 (D 10) 33

Emotiv

IMGD 4000 (D 10) 34

4/23/10

18

IMGD 4000 (D 10) 35

Other Input Control Ideas

  Speech •  cheap, easy to get •  slow, unreliable (esp. in noisy env.)

  “Embodied Gaming” •  e.g., robosoccer with Sony Aibos

IMGD 4000 (D 10) 36

4/23/10

19

IMGD 4000 (D 10) 37

Input Controls Summary

  What can you do with _____ that you couldn’t do before?

  What’s more fun with _____ than with previous technologies?

  Does _____ enhance game play, rather than replacing traditional controller?

[From Z. Drake, GDC’08]