Super-Resolution / Image Registration Overviekasim/eng_design/vr/Lecture19-User-interfaces.pdf ·...

Department of Signal Processing

User interfaces

SGN-5406 Virtual Reality 2012Atanas Boev

based on material byStanislav Stankovic and Ismo Rakkolainen

1

SGN-5406 Virtual Reality 2012


Outline


The UIproblem History Modern UI

design UI in VRThe users Humanfactors

Mechanical UI

Computer UI

GUI elements

What is UI? User types

Subjectivefactors

Subjectiveexperiments

Evaluation

Cyber-sicknessProblems

Touch UI

Navigation

Manipulation

2

Modern UItrends

Modern pitfalls


OVERVIEWUser interface

3



What is UI?

• UI - the “way” users interact with the system• Input - users change the state of a system• Output – system indicates its state to the users

• The “way” (UI paradigm) – includes:• UI protocol – how the control behaves• UI presentation – what hardware controls are used

• Also called HMI – human-machine interface• UI layer – the software that implements the UI behaviour• UI layout - how controls are positioned• etc...


4


Problems with UI design

• All UI design isdictated by technology• No technology is perfectly suited

for human anatomy

• There is no “natural” way ofinteraction• All interaction are learned habits• Pen and paper is natural to

people because they are trainedto use it.


5



• People do not know whatthey want• People know what they do not

want

• Focus groups fail• Most people like voice

recognition, but will not use it.• Most people like DVB-H, but

will not use it.• Most people like AR maps, but

will not use it.


6



• UI design should be done by designers• However, designers are not sure what what people want• Still, designers know (slightly) more than engineers

• UI quality is hard to measure• Quality is hard to measure• Subjective quality is hard to quantify• Performance can be quantified• Subjective tests are not an easy thing to do• Subjective tests are not an easy thing to analyse


7


MECHANICALUI history

8



Time

• Round dial• Imitates sun movement• “Noon” is on top

• 12 sections and threehands• Hour hand• Minute hand (appeared in 1500)• Sweep hand (appeared in 1800)

• Multiple dials can bepresent


9


Unconventional dials


24h dial Decimal dial Chemical elements dial

10


Digits on mechanical clocks


11


The corpus clock

• Mostly mechanical• Grasshopper escapement

• Time is displayed viabacklit slits• Slits open and close• Backlight passes trough• Position of light indicates

the time


http://youtu.be/cCqGtvTA36k

12

http://youtu.be/cCqGtvTA36k


Steering

• Steering wheel• First used in 1894• Spoke design allows seeing

the dashboard• Integrated controls

• Car pedals• Brake in the centre• Clutch away from the

throttle minimizes errors• Foot rest

• Ergonomic design• Not “untuitive”

• Follows car principes, ratherthan mind intuition


Clutch, brake, throttle

13


Steering wheel

• Early cars were steeredwith tiller• Like in a boat

• Modern versions exist• Steering wheel is the

standard• Quite effective• Drivers are trained to use it

• “Meet in the middle”• Technology adapts to man• Man adapts to technology


Clutch, brake, throttle

14


Formula one wheel


• Smaller wheel, many controls• Meant for high speed and low manoeuvrability

15


The “design for interface” problem(a.k.a. Asimov’s pitfall)

• The reason for building humanoid robots?• Instead of making robot car and robot tractor, make

normal car and normal tractor and a robot to drivethem (Asimov’s books)

• Technology advances much faster thaninterfaces• By the time humanoid robot is possible, self driving

car will be very cheap to do

• Interface should adapt to technology• Technology should NOT adapt to the interface• Example – handwriting recognition is a flop, voice

recognition… maybe?


16


EARLY COMPUTER UIUI history

17



Early computer UI


Input Output

Punched card – a column of holesrepresents an entry: command,number or symbol

Teletype – receives numbers,prints characters, basically acomputer-controlled typewriter

18


Early computer UI

• Evolutionary approach• Early computers were

seen as typewriters withTV screen

• Meant to replacecalculators

• Meant to replacetypewrites (text editing)

• Interface devices• Keyboard• Display• Joystick, mouse


19


Text based UI


20

• Keyboard shortcuts – each letter represents a command (saveskeystrokes)

• Same letter means different commands depending on the context


Text based UI

• All input is done trough keyboad• Might work in

• Popular with text-centric tasks• Text editing – vi, vim, e, jed...• Line based text editors – work on one line at a time• Screen based text editors – freely go up and down

trough the text

• Fast interaction (faster than mouse)• Steep learning curve

• When programming was highly specialised task• and “teletype operator” was a job position


21


COMPUTER GUIUI history

22



GUI elements - Windows

• “WIMP” – windows, icons, menus, pointer• The classical mouse-driven GUI• Developed by Xerox PARC• “Popularised” by Apple

• GUI elements• Windows• Menus• Icons• Controls• Tabs• Toolbox


23


GUI Windows

• Single window mode

• Tiling windows• Windows do not overlap

• Stacking windows• Windows can overlap• Windows can scroll their

contents• Program does not need (but

is good to) to adapt todifferent window sizes


Single window mode

Tiling windows

Stacking windows


Icons

• Icon – a graphicalrepresentation of a file• Files appear as “objects”• Executable files – custom

icons• Data files – one icon per

file type


Win 3.1 icons, 32x32px

OSX icons, 1024x1024 (HiDPI) icons

Win XP icons, 64x64px

25


Toolbox

• Floating menu• Meant for graphic editors

(Photoshop, Corel, etc)

• “Tool options”• Additional context menu• Options of the currently

selected tool


Toolbox

Tool options

26


GUI widgets

• Widgets• The “active elements” of a GUI

• Also, their appearance and behaviour• “widget” = “window gadget”

• List of GUI widgets• Button• Window controls (maximize, minimize,

close)• Dropdown menu• Text box (plain text, password)• Date selection• Etc, etc, etc…


27

WebOS widgets Symbian widgets

Windows 95 widgetsOSX “Aqua” widgets


Case study – Microsoft Word

• Word 5.0 – screen-based text interface• UI elements

• Document window• Command menu• Status bar


28



• Word 5.5 –text+mouse

• UI elements• Drop-down

menu• Document

window• Command menu• Scroll bars• Status bar• Context help line

(botton)• GUI in text mode

• Dropdown is aGUI element

• Scrollbars areGUI widgets


29



• Word 6.0, Word 95,Word 2000• GUI interface• No major UI changes• Mostly re-skinned

version

• MS had dominantposition on themarket• No need for innovations


Word 6.0 / Win 3.1

Word 95 / Win 95

Word 2000 / Win 2000

30



• Word 2007• Ribbon interface• Similar to tabbed

combo-box• Replaces drop-

down menus• Ribbon adapts

(“reflows”) tovarious displaywidths

• Easier to findcommands(compared todropdown)

• Ribbon can be“folded” bydouble-clicking ona tab


31


TOUCH-BASED UIUI history

32



Touch-based UI

• Allows interaction with nokeyboard• “Keyboard space can be used as screen

estate”• Also, cheaper and more durable• Does not have tactile feedback

• Allows direct manipulation• Requires bigger touch-targets (finger is

less precise than cursor)• Fingers obstruct the drawing – needs

special text-selection tools


33


Keyboard alternatives

• Keyboard overlays• External keyboards• Touch screen-based tactile overlay

• Gesture-based keyboards• Swype – allows drawing trough letters

insteac of tapping them• Graffiti – special alphabet that is easy to

recognize by the system• Simplified version of handwriting

recognition• Requires training


34


Home screens


35

iOS home screen• Status bar / notifications• Icons• Dock

Android home screen• Status bar / notifications• Icons / widgets• App drawer

Windows phone home screen• Live tiles• Vertically scrollable• Comic book style frames


SUBJECTIVE FACTORSThe users

36



User types


37


Age

• Users of different age groups have differentneeds• Different abilities• Different style

• Some factors are obvious• bigger buttons for old people• brighter images for kids

• Some factors are not obvious• Older people use index finger to push buttons• Younger people use thumbs• Older people prefer to pay cash• Younger people do not wear watches


38


Social background

• In some cultures public photography notacceptable• “Female college students carrying camera phones

on campus in any college in Saudi Arabia couldface a SR500 fine and 3-year suspension”

• Camera sound cannot be turned off in Japan

• In some countries insulting the governmentis a crime, downloading mp3 is not

• In some other countries, it’s the opposite


39


User experience

• User experience (UX) - the way a person feelsabout using a product, system or service• Quality of experience (QoE) – subjective measure a

system performance (“Do users approve?”)• Quality of service (QoS) – objective measure of system

performance (“Do engineers approve?”)

• User-centered design (UCD) – UI design whichis focused on user need and limitations• Design is a function of the user expectations and needs• It is critically important to perform correct test and

analysis• Otherwise design will be based on what users seem

to want• … if the users know what they want in the first place.


40


Subjective tests

• Subjective tests have to adhere to a well-documented experimental protocol e.g.• ITU-T BT.500 Methodology for the Subjective

Assessment of the Quality of Television Pictures• ITU-T BT.1438 Subjective video quality assessment

methods for multimedia applications

• Experiments need to follow a structuredsequence for all participants• A trial is a single instance of the experiment.• A sequence of repeated trials constitutes a session.• Sessions (and sometimes trials) are separated by rest

periods for the participant in the study


41


MODERN UI TRENDSModern UI design

42



Tiling window managers

• Single window interfaces• Most mobile OSes show one

app on full screen

• Tiling windows• Windows 8 modern UI allows

running multiple apps side-by-side

• Cascading windows• Galaxy note supports

“floating widgets”• Similar to MacOS 6 “Desk

accessories”


43

MacOS Deskaccessory

Floating notepad window

Win8 UI, two apps side-by-side


“Responsive” design

• UI elements actively resize• If the element is selected• If the windows size changes• Used in web-design

• The same webpage works onmany screen resolutions

• Used in mobile OS• The same software works on

phones and tablets

• Example – the “accordion”• Widgets stretches on selection and

contracts on losing focus


44

Accordion widget, “Header 2” is selected


Infinite canvas

• Invented by Scott McCloud• Modern technology allows for having a

multidimensional narrative

• Possibly used as a basis of Metro Modern UI


45


Drag-to-refresh

• Refresh a list of events by dragging it downwards• First used in Tweetie (bought by Twitter)• Nowadays used by many mobile apps


46


Skeuomorphism

• Skeuomorphic design• Create the outlook of one

(expensive) material usinganother (cheaper) material

• In GUI – using ultra-realistic widgets,mimicking real objects• Shadows• Textures• Mimicking realistic object

behaviour (e.g. rotating knob)


47


MODERN UI PITFALLSModern UI design

48



Skeuomorphism

• Skeuomorphism can easily be overdone• Instead of “luxurious” created “cheap” feel• E.g. fake “leopard” prints


49

UnderstatedOverstated


Volume control

• iOS “mute” button does not mute all sounds• Android has 6(!) separate volume controls

• Media• Notifications• Alarms• System (keyclick)• Headphones• Calls

• One does not simply mute (into Mordor)• Lots of embarrassing situations• Nokia “profiles” disappeared in WP7+


50


Acceleration sensors

• Accelerations sensors are often overused• Games are controlled with device tilt• Hard to do precision control• Hard to follow the action on the screen• Tilt-control lunar lander (?)

• Display orientation• Two modes: locked/unlocked• No way to set display orientation• Hard to use while laying on the sofa• Impossible to use in zero-g


51


VR EXPERIENCEUser interfaces in virtual reality

52



VR experience design


53

Factors which affect the VR experience

• Aims of VR experience design• Immersive experience• Easy to perceive (no sensory overload)• Easy to control (easy to grasp and navigate)


Interaction tasks

• Traditional interaction• Imitation of real objects (see skeuomorphic design)• Imitation of familiar 2D GUI (see 3D dropdown menus)

• Traditional interaction might not be useful in a VE• E.g. using walking for map or areal navigation – might be too tiring• Using text based menus in VE – text might be seen from hard-to-

read angles• VE navigation needs new concepts and paradigms

• VE tasks• Navigation: travel + wayfinding• Selection: pointing, choosing one or more objects• Manipulation: Grabbing, object position, orientation, scale• Communication: with users, with UI agents• Data input: symbols, text, numbers• Commands: menus, changing of system state• Visual information: system status


54

UI agent


NAVIGATION IN VRUser interfaces in virtual reality

55



Navigation

• Navigation = travel + wayfinding• Wayfinding – finding the path to go from A to B• Travel – actual moving of the point-of-view

• Travel methafors• Natural methaphors

• walking, bicycling,• Also Disney raft ride

• Steering – specifying the direction• Steering wheel, joystick, gaze directed

• Target based – e.g. teleport• Point at place, enter coordinates, choose from list

• Orbital movement


56


Navigation

• Wayfinding metaphors• Natural metaphors

• Landmarks• Previous knowledge• Memorable place names

• 3D map-based• Bird-eye views• Constrained travel – cannot leave the predefined path• Path-following – leave trail, place markers


57


VR Navigation UI


58

Point places on aminiature model of theworld


VR Navigation UI


59

Select steering direction


MANIPULATION IN VRUser interfaces in virtual reality

60



Selection / manipulation

• Selection of direction• Using pointer• Using gaze• Using crosshair• Torso-directed (slant of the body towards desired direction)• Landmark-directed

• Selection of item• Point to select - direct contact between object and avatar• Using 3D cursor• Using aperture – e.g. two fingers• Using menu• Using name – e.g. speech recognition

• Manipulation• Apply force to virtual objects• Gesture-based control of size (e.g. pinch to zoom), color, texture• Using menu – modifying object properties


61


EVALUATIONHuman factors in VR

62



Human factors in VR

• Multimodal interfaces (with graphics, sound and hapticfeedback), realistic modeling, and increased systemperformance can produce better-quality simulations

• Are VR systems, UIs really more efficient for a specifictask?• Claims about increased performance need to be backed up by

proper user studies• User’s performance when interacting with the simulation• User’s response to the technology in order to iteratively improve the

VR system or the particular application design• It is necessary to understand why some user responses lead to

simulator sickness, what are its causes, and what can be done tominimize its effects

• Courses at TUT: http://www.cs.tut.fi/ihte/• Design and evaluation of 3D-UI: http://www.3dui.org/


63


Human factors research

• Human factors research consists of systematic studiesby multi-disciplinary teams of engineers andpsychologists to gauge• Which tasks are more suitable for users• Which user characteristics influence the VR simulation performance• How VR technology should be improved to better meet user needs• What kind of designs enhance user performance• The negative societal impact from the users’ misuse on the

technology, etc.• No comprehensive model of human behavior exists,

owing to its multidimensionality as well as largeindividual variability

• The fairness of a simulation is qualitative and it cannotbe easily quantified mathematically.

• Also, determining the performance of a VR simulation issomewhat subjective


64


Human factors research

• Human factors research can be divided into• Usability – studies how to improve a system or

application• User performance – aims to measure the user’s

response to a given simulation and a particularhardware system

• User safety – aims to better understand simulationsickness causes and effects as well as increase theuser’s safety

• Sociological studies – aim to measure factors relatedto effects of VR on other users and the society


65


Selection / manipulation techniques


66


Menu selection in VR



CYBERSICKNESSHuman factors in VR

68



Effects of VR Simulations onusers

• Indirect effects• neurological, psychological, sociological, or cybersickness• E.g., addiction, lack of real human relationships, narrow social life, etc.

• Direct effects• affect mainly the user’s visual system (dominant),• also the user’s auditory, skin, and musculoskeletal systems

• Examples• Visual system can direct high-intensity light at the user’s eyes. The exposure

will result in corneal burns, retinal burns, and other injuries• The laser used in miniature wearable retinal displays• Bright lights pulsed at low frequency may trigger an induced “absence” state

or seisure• Auditory system affects the user if the simulation noise level is too high• Haptic interfaces can apply high level of forces or push the user’s limbs

beyond anatomical range limits• Motion platforms that move ankles beyond normal rotation angles• Increased risk of skin disease when VR interfaces (e.g., HMDs, gloves) are

shared between several users


69


Cybersickness – what is it?

• Cybersickness• perhaps the most troublesome effect of VR simulation to the user• A form of motion sickness that results from VR interaction

• Main symptoms• eye strain• disorientation• postural instability• sweating• drowsiness• Nausea

• Cause• information from several sensorial channels is not in agreement

• Cybersickness vs motion sickness• Motion sickness may be induced without immersion (e.g. by riding a

roller-coaster in an amusement park• Cybersickness is caused by exposure to virtual environments (e.g.

by immersive visual environment)



Cybersickness - causes

• Main reason – inner ear• Conflicting multimodal information which involves the vestibular

sensors• Sense of balance perceives one thing, eyes/ears/skin perceive

another• Causes

• Rendering errors• System latency• Misaligned HMD optics• Vection

• Fast scene movement on a large FOV displays• Movement occupies central and peripheral vision• Eyes perceive movement, vestibular sense does not



Cybersickness - susceptibility

• Age-dependent• People aged 2-12 are the most susceptible to

cybersickness• People at age 25 are half as susceptible than people

at age 18

• Gender dependent• Female users 3X more susceptible than male users

• Medication dependent• Mind affecting medications affect susceptibility to

cybersickness• Small doses of alcohol may decrease it• Large doses of alcohol may cause motion sickness



Cybersickness - adaptation

• Brain can adapt to contradicting sensorialstimuli• Repeated VR exposures can cause adaptation, and

decrease susceptibility to sybersickness• Brain learns to treat abnormal stimuli as normal

• Speed of adaptation can be increased• By active interaction• Gradual and incremental VR exposure


Super-Resolution / Image Registration Overviekasim/eng_design/vr/Lecture19-User-interfaces.pdf ·...

Documents

Transcript of Super-Resolution / Image Registration Overviekasim/eng_design/vr/Lecture19-User-interfaces.pdf ·...