04/08/2023 1Joe Fletcher MIX09, Las Vegas

Untold Stories of Touch, Gesture, & NUIJoe FletcherDesign Manager, Microsoft Surface

04/08/2023 2Joe Fletcher MIX09, Las Vegas

04/08/2023 3Joe Fletcher MIX09, Las Vegas

Touch is not good

for everythingIt is not an end all solution

Touch is a computing evolution

It is an addition element for users to interact with their

content in new and unique ways

Touch is great for specific aspects

New technology has enabled touch systems, which can enhance certain activities

04/08/2023 4Joe Fletcher MIX09, Las Vegas

04/08/2023 5Joe Fletcher MIX09, Las Vegas

Microsoft Surface

Microsoft Surface is a revolutionary computer that responds to natural hand gestures and real-world objects, creating unique new ways for people to collaboratively interact with digital content.

04/08/2023 6Joe Fletcher MIX09, Las Vegas

Microsoft Surface

04/08/2023 7Joe Fletcher MIX09, Las Vegas

04/08/2023 8Joe Fletcher MIX09, Las Vegas

Vision DesignStrategy

04/08/2023 9Joe Fletcher MIX09, Las Vegas

Surface Massive Multi-touch, Multi-user

Windows 7

PC’s, Form Factors, Technology

Unique Spot

04/08/2023 10Joe Fletcher MIX09, Las Vegas

Larger Surface Team

04/08/2023 11Joe Fletcher MIX09, Las Vegas

Touch is not good for

everything

Touch is an evolution

Touch is great for specific

aspects

04/08/2023 12Joe Fletcher MIX09, Las Vegas

TouchThe act of directly touching your digital content

GestureHand motions to take an action

ManipulationAlteration or freeform movement of digital content using touch

Natural User InterfaceThe combination of these and other elements the create a unique way to interact with your digital content

04/08/2023 13Joe Fletcher MIX09, Las Vegas

History and Ecology

Surface Design

Principles

Touch Challenges

04/08/2023 14Joe Fletcher MIX09, Las Vegas

History and Ecology Surface Design Principles

Touch Challenges

04/08/2023 15Joe Fletcher MIX09, Las Vegas

Why Evolve?

04/08/2023 16Joe Fletcher MIX09, Las Vegas

Command Line Interface (CLI)

04/08/2023 17Joe Fletcher MIX09, Las Vegas

Graphic User Interface (GUI)

04/08/2023 18Joe Fletcher MIX09, Las Vegas

Natural User Interface (NUI)

04/08/2023 19Joe Fletcher MIX09, Las Vegas

Natural User Interface (NUI)

04/08/2023 20Joe Fletcher MIX09, Las Vegas

Natural User Interface (NUI)

04/08/2023 21Joe Fletcher MIX09, Las Vegas

Microsoft Surface and NUI

Microsoft Surface computing uses sensing and display technology to infuse

everyday surfaces with digital content and is comprised of the following

unique attributes:

Object recognitionDirect interaction Multi-touch Multi-user

04/08/2023 22Joe Fletcher MIX09, Las Vegas

NUI vs.

Web 2.0

04/08/2023 23Joe Fletcher MIX09, Las Vegas

NUI & Touch should not be a novelty

04/08/2023 24Joe Fletcher MIX09, Las Vegas

04/08/2023 25Joe Fletcher MIX09, Las Vegas

Input Ecology

Keyboard

Good forAccuracy

ConLimited

Mouse

Good forMoving across the screen efficiently

ConIndirect

Stylus

Good forFreeform input

ConLow accuracyNeed large controls

Single point

Good forDirect object selection

ConNot for quick inputLow accuracyNeed of large controlsMis-hits from fingernails of perceived hit area

Multi point

Good forManipulating objectsSocial

ConNot for inputLow accuracyNeed large controlsSupport of hands and screen

04/08/2023 26Joe Fletcher MIX09, Las Vegas

[Partial] Touch Ecology

Multi point (screen)

Reacts to direct input

Multi Touch(screen)

Reacts to multiple inputs

Orients UI to the direction of the touch

Spatial recognition of system

NUI(Gesture & Physical/Digital)

Reacts to multiple inputs

Spatial Recognition - Orients UI to the direction of the touch

Object recognition - Blurring physical and digital

04/08/2023 27Joe Fletcher MIX09, Las Vegas

Tap is not the new click

≠

04/08/2023 28Joe Fletcher MIX09, Las Vegas

Context, task, and use define the solution

04/08/2023 29Joe Fletcher MIX09, Las Vegas

Context, task, and use define the solution

Touch is not a solution in search of a problem

04/08/2023 30Joe Fletcher MIX09, Las Vegas

Touch is not good for

everything

Touch is an evolution

Touch is great for specific

aspects

04/08/2023 31Joe Fletcher MIX09, Las Vegas

History and Ecology

Surface Design Principles

Touch Challenges

04/08/2023 32Joe Fletcher MIX09, Las Vegas

Create an inflection point

04/08/2023 33Joe Fletcher MIX09, Las Vegas

The start of almost every successful technological innovation is marked by an inflection point, which defines a moment of dramatic change.

04/08/2023 34Joe Fletcher MIX09, Las Vegas

Companies spend 90% of their energy trying to imitate

04/08/2023 35Joe Fletcher MIX09, Las Vegas

Microsoft Surface and principle driven design

Principle Driven Design

04/08/2023 36Joe Fletcher MIX09, Las Vegas

Microsoft Surface and principle driven design

Using history to develop the future

Principle Driven Design

04/08/2023 37Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

C L I G U I N U I

04/08/2023 38Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

Recall Recognition Intuition

C L I G U I N U I

04/08/2023 39Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

Recall Recognition Intuition

Directed Exploratory Contextual

C L I G U I N U I

04/08/2023 40Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

Recall Recognition Intuition

Directed Exploratory Contextual

High-Low Double Medium Fast Few

C L I G U I N U I

04/08/2023 41Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

Recall Recognition Intuition

Directed Exploratory Contextual

High-Low Double Medium Fast Few

Disconnected Indirect Unmediated

C L I G U I N U I

04/08/2023 42Joe Fletcher MIX09, Las Vegas

Text Graphics Objects

Recall Recognition Intuition

Directed Exploratory Contextual

High-Low Double Medium Fast Few

Disconnected Indirect Unmediated

Static Responsive Evocative

C L I G U I N U I

04/08/2023 43Joe Fletcher MIX09, Las Vegas

Intuition

Contextual

Fast Few

Unmediated

Evocative

N U I

Principle of the Super Real

Principle of Contextual Environments

Principle of Scaffolding

Principle of Direct Manipulation

Principle of Performance Aesthetics

04/08/2023 44Joe Fletcher MIX09, Las Vegas

Microsoft Surface and principle driven design

Principle Driven Design

04/08/2023 45Joe Fletcher MIX09, Las Vegas

Microsoft Surface and principle driven design

Three Super Principles

04/08/2023 46Joe Fletcher MIX09, Las Vegas

Social Multiple simultaneous users

04/08/2023 47Joe Fletcher MIX09, Las Vegas

04/08/2023 48Joe Fletcher MIX09, Las Vegas

Social Multiple simultaneous users

Seamless Digital and Physical combined

04/08/2023 49Joe Fletcher MIX09, Las Vegas

Seamless

04/08/2023 50Joe Fletcher MIX09, Las Vegas

Social Multiple simultaneous users

Seamless Digital and Physical combined

Spatial Kinesiology

04/08/2023 51Joe Fletcher MIX09, Las Vegas

04/08/2023 52Joe Fletcher MIX09, Las Vegas

Overcome Homeostasis

04/08/2023 53Joe Fletcher MIX09, Las Vegas

Touch is not good for

everything

Touch is an evolution

Touch is great for specific

aspects

04/08/2023 54Joe Fletcher MIX09, Las Vegas

History and Ecology

Surface Design Principles Touch Challenges

04/08/2023 55Joe Fletcher MIX09, Las Vegas

Let’s become practitioners

04/08/2023 56Joe Fletcher MIX09, Las Vegas

PC’s, Form Factors, Technology

Unique Spot

04/08/2023 57Joe Fletcher MIX09, Las Vegas

Challenge Form Factor & Technology

04/08/2023 58Joe Fletcher MIX09, Las Vegas

Touch Hardware Ecology

04/08/2023 59Joe Fletcher MIX09, Las Vegas

Touch Hardware Ecology

Capacitive Capacitive Infrared Vision system Stylus

04/08/2023 60Joe Fletcher MIX09, Las Vegas

Touch Hardware Ecology

CapacitiveN-Trig (2-10)

CapacitiveN-trig (2-10)

InfraredHP (2)

Vision systemSurface (52)

StylusWacom (1)

04/08/2023 61Joe Fletcher MIX09, Las Vegas

Challenge Patterns

04/08/2023 62Joe Fletcher MIX09, Las Vegas

FireFoxSwipe Left: Go back in history

Swipe Up: Go to the top of the page

04/08/2023 63Joe Fletcher MIX09, Las Vegas

Apple iPhoneSwipe Left: Go Forward in History

Swipe up: Go to the bottom of a page

04/08/2023 64Joe Fletcher MIX09, Las Vegas

Challenge Orientation

04/08/2023 65Joe Fletcher MIX09, Las Vegas

Horizontal Vertical

04/08/2023 66Joe Fletcher MIX09, Las Vegas

Challenge Cardio Effect

04/08/2023 67Joe Fletcher MIX09, Las Vegas

Challenge Paradigms

04/08/2023 68Joe Fletcher MIX09, Las Vegas

To button or

not to button

04/08/2023 69Joe Fletcher MIX09, Las Vegas

Challenge Getting people to touch

04/08/2023 70Joe Fletcher MIX09, Las Vegas

Exploration Instruct

04/08/2023 71Joe Fletcher MIX09, Las Vegas

Challenge Visualizing Touch

04/08/2023 72Joe Fletcher MIX09, Las Vegas

Is it working?

04/08/2023 73Joe Fletcher MIX09, Las Vegas

Future Where do we go?

04/08/2023 74Joe Fletcher MIX09, Las Vegas

Your effort to remain what you are is what limits youGhost in the Shell

04/08/2023 75Joe Fletcher MIX09, Las Vegas

Debating the future of touch is useless

04/08/2023 76Joe Fletcher MIX09, Las Vegas

Creating the future is more interesting

04/08/2023 77Joe Fletcher MIX09, Las Vegas

Touch is not good for

everything

Touch is an evolution

Touch is great for specific

aspects

04/08/2023 78Joe Fletcher MIX09, Las Vegas

Take advantage of Microsoft Surface

• Available from the Surface Community siteDownload SDK Workstation Edition

• Learn how to build applications from scratch on Surface by viewing videos on Surface.comDevelop for Surface

• Interested in purchasing a Surface? Email to sbizdesk@microsoft.comPurchase a Surface

04/08/2023 79Joe Fletcher MIX09, Las Vegas

Future Vision

2019

04/08/2023 80Joe Fletcher MIX09, Las Vegas

04/08/2023 81Joe Fletcher MIX09, Las Vegas

Las Vegas, Thank you!Joe Fletcher

joeflet@microsoft.com

Design Manager, Microsoft Surface

Please Complete an Evaluation FormYour feedback is important!

Evaluation forms can be found on each chairTemp Staff at the back of the room have additional evaluation form copies