CE-4026, New Interfaces, by David Brebner

NEW INTERFACES

DAVID BREBNER

New InterfacesGestural & Cinematic experiences

3 | NEW INTERFACES | NOVEMBER 22, 2013 | CONFIDENTIAL

GESTURAL AND CINEMATIC

� The continuum of interaction

From precise to casual

Content Creation & Productivity Content Consumption and Entertainment



� Interaction with others

Individual Online Together



� Ergonomics

Lots of publicly available research and

recommendations

Very little publicly available research and no industry

guidelines

Un-answered questions…

• How will people use large form factor systems?

• How long do they use these devices for?

• Do they stand or sit?

• What angle is their arm, wrist, hand?

• What % of interaction is touch vs. keyboard vs.

mouse?

• How do the ratios change depending on task?



Gestural Cinematic

FunctionMinimal

Consistent

Obvious

What does the App do?

NEW INTERFACES



Gestural Cinematic

Function

Responsive

Animated

Explanatory

Revelatory

Delightful

How does the App look?

NEW INTERFACES



Gestural Cinematic

Function

Manipulate

Direct

Responsive

Reversible

Hinted

Consistent

How do you interact?

NEW INTERFACES



� Gestural

‒ Direct manipulation – not memorizing gestures

‒ Touch, voice, 3D and more

‒ Sweeping movements

‒ Allow for mastery

� Cinematic

‒ Great visuals

‒ Show actions with animation and state

‒ Use lighting, particles, focus, depth of field to highlight elements

‒ Use warps and motion to transition between elements

‒ Use 3D content and environments but provide 2D workspaces

NEW INTERFACES



� Give feedback of action triggered

� Provide a way to undo the action

� Actions to be commutative (any order)

� Physical ‘manipulation’ should be obvious and

reversible (move, rotate, resize etc)

� Actions should be repeatable and reliable (don’t rely on gestures which the sensor cannot repeat)

� Reveal the gestures and their function (with hints)

ACTIONS

Classic animation techniques like bounce, squash &

stretch, follow through are now being used (often on

abstract shapes) to illustrate actions



� Using a 3D camera can have lighting issues

� It’s NOT touch at a distance

� Because there is no control surface it is hard

distinguishing between actions and movement

� Support spatial control surfaces

� Add speech for actions

� We need to find new gestural rules

� 3D Gesture has tracking but no ‘action’

� Fingers, hands and arms obscuring the view of

the screen

SPATIAL GESTURE



� Touch

� Multi touch

� Near touch

� Touch + pressure

� Stylus + touch

� Stylus + pressure + angle

� 3D camera

� Gaze + voice

� Wearable's + GPS + Gyro

GESTURAL INPUT



� Reach towards the screen and receive localised hints

� Touch the screen to manipulate (move, resize, rotate,

drag, pick)

� Reach towards the screen with pen and receive hints

� Touch the screen to act (cut, draw, circle)

� Also speak to act on any visible hints

VOICE, PEN, NEAR TOUCH, TOUCH



� GUI point and click replaced with touch is

hard for users

� New interfaces with touch, supporting fluid

dragging and picking is a big improvement

� New interfaces with multi-touch, allowing

multi finger gestures like resizing and

rotation is better

� Multi-user is a new challenge on larger

devices, and requires the designer to set

aside localised and orientated workspaces

for multiple users

TOUCH - MULTITOUCH - MULTIUSER


NATURAL USER INTERFACE

� NUI is not a fixed toolkit (like a GUI)

� Don’t just mimic GUI or web apps

� NUI is still abstract, it’s not real world

� NUI should seem an extension of the body

� NUI can leverage previously learned skills

� Involves game aspects

‒ enjoyable interaction

‒ a path to skilled usage

‒ but NOT challenge

� Direct manipulation

� Aim for skilled and fluid actions

IT’S A BROAD CONCEPT – NOT A DETAILED FRAMEWORK

Technology stack is

growing & improving


HIGH SPEED ACCURATE MULTI TOUCH


FLEXIBLE PERFORMANT COMPUTE ARCHITECTURES


A GROWING RANGE OF GESTURE TECHNOLOGIES


A GROWING RANGE OF VOICE CAPABILITIES

Client side – cloud side – hybrid


ATTENTION TRACKING

� Turned

away

� Paying

attention


VR


GPU EFFECTS

Fluid

simulation

Paint, glass, chrome, plastic, environment, AO

BUT needs hardcore development


UMAJIN APP BUILDER


UMAJIN APP BUILDERANYONE CAN BUILD NEW INTERFACES


NEW INTERFACESMANY SCREENS

From 4” to 80”


NEW INTERFACESTOUCH, VOICE, GESTURE

Demo…


DISCLAIMER & ATTRIBUTION

The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors.

The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap

changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software

changes, BIOS flashes, firmware upgrades, or the like. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD

reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of

such revisions or changes.

AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY

INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.

AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE

LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION

CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

ATTRIBUTION

© 2013 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices,

Inc. in the United States and/or other jurisdictions. SPEC is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Other names

are for informational purposes only and may be trademarks of their respective owners.

CE-4026, New Interfaces, by David Brebner

Technology

Transcript of CE-4026, New Interfaces, by David Brebner