Heuristic Evaluation on Mobile Interfaces

8/18/2019 Heuristic Evaluation on Mobile Interfaces

1/20

Research ArticleHeuristic Evaluation on Mobile Interfaces: A New Checklist

Rosa Yáñez Gómez, Daniel Cascado Caballero, and José-Luis Sevillano

Department o Computer echnology and Architecture, E S Ingenieria In ormatica, Universidad de Sevilla, Avenida Reina Mercedes s/n. Seville, Spain

Correspondence should be addressed to Jos´ e-Luis Sevillano; [email protected]

Received June ; Accepted August ; Published September

Academic Editor: Mohammad S. Obaidat

Copyright © Rosa Ýañez Gómez et al. Tis is an open access article distributed under the Creative Commons AttributionLicense, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Te rapid evolution and adoption o mobile devices raise new usability challenges, given their limitations (in screen size, battery li e, etc.) as well as the speci c requirements o this new interaction. raditional evaluation techniques need to be adapted in order

or these requirements to be met. Heuristic evaluation (HE), an Inspection Method based on evaluation conducted by expertsover a real system or prototype, is based on checklists which are desktop-centred and do not adequately detect mobile-speci cusability issues. In this paper, we propose a compilation o heuristic evaluation checklists taken rom the existing bibliography butreadapted to new mobile inter aces. Selecting and rearranging these heuristic guidelines offer a tool which works well not just orevaluation but also as a best-practices checklist. Te result is a comprehensive checklist which is experimentally evaluated as adesign tool. Tis experimental evaluation involved two sofware engineers without any speci c knowledge about usability, a groupo ten users who compared the usability o a rst prototype designed without our heuristics, and a second one afer applying theproposed checklist. Te results o this experiment show the use ulness o the proposed checklist or avoiding usability gaps evenwith nontrained developers.

1. Introduction

Usability is the extent to which a product can be used witheffectiveness, efficiency, and satis action in a speci ed context o use [ ]. While usability evaluation o traditional browsers

rom pc environments—desktop or laptop—has been widely studied, mobile browsing rom smartphones, touch phones,and tablets present new usability challenges [ ]. Additionally,mobile browsing is becoming increasingly widespread as away o accessing online in ormation and communicatingwith other users. Speci c usability evaluation techniquesadapted to mobile browsing constitute an interesting andincreasingly important study area.

Usability evaluation assesses the ease o use o a website’s unctions and how well they enable users to per orm their tasksefficiently [ ]. o carry out this evaluation, there are severalusability evaluation techniques.

Usability evaluation techniques canbe classi edas shownin Figure [ – ]. Over real systems or prototypes, thebest alternatives are evaluations conducted by experts, alsoknown as Inspection Methods, or evaluations involving

users, which are divided into inquiry methods and testingmethods depending on the methodology adopted. Witha more academic ocus, predictive evaluation offers somepredictions over the usability o a potential and not-yet-existent prototype.

Heuristic evaluation (HE) is an inspection method basedon evaluation over real system or prototype, conducted by experts. Te term “expert” is used as opposed to “users” butin many cases evaluators do not need to be usability experts[ , ]. In HE, experts check the accomplishment o a givenheuristic checklist. Due to its nature, this inspection cannotbe per ormed automatically.

HE, like other usability assurance techniques, has totake into account the act that usability is not intrinsically objective in nature but is rather closely intertwined with anevaluator’s personal interpretation o the arte act and his orher interactionwith it [ ]. But, evaluationscanbe designed tocompensate or personal interpretation as much as possible.

Moreover, inspection methods are ofen criticized oronly being able to detect a small number o problems intotal together with a very high number o cosmetic ones [ ].

Hindawi Publishing Corporatione Scientic World Journal

Volume 2014, Article ID 434326, 19 pageshttp://dx.doi.org/10.1155/2014/434326

http://dx.doi.org/10.1155/2014/434326http://dx.doi.org/10.1155/2014/434326


2/20

Te Scienti c World Journal

Usability evaluation techniques

PredictiveSimulation

Analytical modelling

Over real system or prototype

Conducted by usersTesting methods

Question-asking protocolCodiscovery learningShadowing method

Inquiry methods

Focus groupField observationQuestionnaire

Conducted by experts/inspection methods

Pluralistic walkthroughCognitive walkthroughHeuristic evaluationGuidelines or standards inspection

F : Classi cation o some usability evaluation techniques.

But, HE presents several advantages over other techniques: itsimplementation is easy, ast, and cheap, and it is suitable orevery li e-cycle sofware phase and does not require previousplanning [ ]. Furthermore, it is not mandatory or evaluators

to be usability experts [ , ]. It is possible or engineersor technicians with basic usability knowledge to drive anevaluation. Furthermore, regarding the number o evaluators,Nielsen demonstratedempirically that between three and veexperts should be enough [ ].

Because o all these advantages, HE is a convenientusability evaluation method: the worst usability con icts aredetected at a low cost. But, traditional HE checklists aredesktop-centred and do not properly detect mobile-speci cusability issues [ ].

In this study, we propose a heuristic guideline centred inmobile environments based on a review o previous litera-ture. Tis mobile-speci c heuristic guideline is not only an

evaluation tool but also a compilation o recommended best-practices. It can guide the design o websites or applicationsoriented to mobile devices taking usability into account.

Te ollowing section describes the methods ollowedto de ne the mobile heuristic guideline. Ten, Results andDiscussion section isdivided according to thestepsde nedinthe methodology. We have included a brie discussion o theresults or each task. Te nal sections include Conclusionsand Future Work, Acknowledgments, and Re erences.

2. Methods

o obtain a heuristic guideline centred in mobile environ-

ments and based on a review o previous literature, we willollow a six-step process.

( ) A clear de nition o the problem scope is necessary as a rst step to de ne and classi y the specialcharacteristics o mobile interaction.

( ) Next, we rearrange existing and well-known heuris-tics into a new compilation. We can reuse heuristicguidelines rom the literature and adapt them to thenew mobile paradigm because heuristic checklistsderive rom human behaviour, not technology [ ].Tis heuristics is general checks that must be accom-plished in order to achieve a high level o usability.

( ) Afer building this new classi cation o heuristics,we will develop a compilation o different proposedsubheuristics. “Heuristic” in this paper re ers to aglobal usability issue that must be evaluated or taken

into account when designing. In contrast, the term“subheuristic” re ers to speci c guidelines items. Temain difference between the two concepts lies in thelevel o expertise required o the evaluator and theabstraction level o the checklist. Te resulting selec-tion o subheuristics in this step takes into accountsome o the mobile devices restrictions presented inthe rst step. But, the result o this stage does notinclude many mobile speci c questions, as they arenot covered in traditional heuristic guidelines.

( ) Te ourth step in this work consists o enrichingthe list with mobile-speci c subheuristics. Tis sub-heuristics isgleaned rom mobileusabilitystudies andbest practices proposed in the literature.

( ) One urther step is required to homogenize theredaction and ormat o subheuristics in order tomake it use ul or nonexperts.

( ) Finally, we conduct an evaluation o the use ulness o the tool as an aid in designing or mobile.

Tis process differs slightly rom the methodology proposedby Rusu et al. [ ], but we can subsume their phases whenestablishing new usabilityheuristics in our proposed method.

It is worth remarking that popular mobile operatingsystems are now providing usability guidelines [ , ]which ocus mainly on maintaining coherent interaction andpresentation through applications over the whole plat orm.Tese guidelines could in some cases enrich certain aspectso our proposal, although we have opted to keep it essen-tially agnostic o speci c plat orms aesthetics or coherence-determined restrictions.

Additionally, inter aces or mobile are mainly dividedinto web access and native applications. We do not restrictour study to a speci c kind o inter ace. Again, the goalis to elaborate a guideline which is independent o speci ctechnologies. Te interaction between users and mobileinter aces is similar regardless o the piece o sofware they are using.


3/20


Speci c constraints on mobile

(F) Adoption

PersonalizationComfortAdoptionAcceptancePrivacy

(E) Limited processing capability and power

Limited memory Network connectivity Battery life

(D) Multidevice access

(C) Type of tasks (speci cally different)

(c.6) Killing time(c.5) Playing games(c.4) Transaction

(c.3) CommunicationChat rooms/discussion roomsSocial networking sitesChecking email

(c.2) Browsing

Reading newsWatching videosUndirected or conditioned

(c.1) Searching

Compare/chooseUsing the web to nd something speci c/less speci c searchFinding/information gathering

(B) Mobility and varying context

(A) Limited input/output facilities

OutputLimited screen size and different display resolutionsUnreliability of wireless networksLimited bandwidth

InputDisplay resolutionSmall screen sizeData entry methods

F : Speci c constraints on mobile.

3. Results and Discussion

. . Problem Scope De nition. Users are increasingly adopt-ing mobile devices. According to statistics o Pew Internet & American Li e Project [ ], only in the USA % o adultsown smartphones and % o adults own a cell phone o some kind. Additionally, % o smartphones owners accessthe Internet or email on their handheld— % on a typicalday. A urther % say that they mostly go online using theirsmartphone, rather than a computer. Tis survey shows thatphones operating on the Android plat orm are currently themost prevalent type, ollowed by iPhones and Blackberry devices.

Mobile usability involves different kind o devices, con-texts, tasks, and users. Te compilation o a new heuristicguideline needs a restriction and de nition o the scope o the user-inter ace interaction.

Devices can be divided in three types [ ]:

(i) eature phones: they are basic handsets with tiny screens and very limited keypads that are suitablemainly or dialing phone numbers;

(ii) Smartphones: phones with midsized screens and ullA–Z keypads;

(iii) touch phones/touch tablets: devices with touch-sensitive screens that cover almost the entire ront o the phone.

In our study, we have ruled out eature phones because the

interaction and inter ace design are deeply restricted andthey are gradually being abandoned by a wide range o users.We have also ruled out smartphones because interactionis dramatically different due to the keyboard and they arecommonly constrained to enterprise use. Tis study ocuseson the ubiquitous touch phones and touch tablets. In thiswork, we use the term “touch phones” to re er to both phonesand tablets because they share a similar interaction paradigmand the constraints we describe in Figure .

Mobile interactions de ne a new paradigm characterizedby a wide range o speci c constraints: hardware limitations,context o use, and so orth. All these restrictions have beenstudied in the bibliography in order to de ne the issues thatmust be overcome to improve usability. According to theliterature, the main constraints when designing or mobiledevices are (Figure ):

(A) limited input/output acilities [ – ]: these limita-tionsare imposed bydata entry methods, small screensize, display resolution, and available bandwidth, aswell as unreliability o wireless networks;

(B) mobility and varying context [ – ]: traditionalusability evaluation techniques have ofen relied onmeasures o task per ormance and task efficiency.Such evaluation approachesmay notbe directly appli-cable to the ofen unpredictable, rather opportunis-tic and relatively unstable mobile settings. Mobile


4/20


devices use is on-the-run and interactions may takerom a ew seconds to minutes, being highly context-

dependent. Environmental distractions have a signi -icant effect on mobile inter aces usability and hencethey needto be taken intoaccount [ ].Context o useinvolves background noise, ongoing conversations,

people passing by, and so on. Distractions can beauditory, visual, social, or caused by mobility.Te context o use is so in uent in the interaction thatmany authors propose testing in the eld as indis-pensable to study interaction with mobile devices[ ]. Laboratory testing seems incapable o com-pletely assuring usability in this mobile paradigm.Some attempts to cover this contextual in ormationhave been documented in the literature: Po et al. [ ]proposed inclusiono contextual in ormation intotheheuristic evaluation proposed by Nielsen and Molich[ ];Bertinietal.[ ] discussed thecapacityo expert-based techniques to capture contextual actors in

mobile computing. Indeed, it is not trivial to integratereal-world setting/context into inspection methodswhich are conceived as laboratory testing techniques.In any case, laboratory testing andexpert-based tech-niques are complementary. Both approaches can beused in preliminary analysis and design o prototypesbut, even more in mobile than when dealing withold desktop interaction paradigms, they need to becomplemented with users-based testing.

(C) ype o asks: in mobile environments, typicaltasks are relatively different rom traditional desktopdevices. From the origins o mobile devices, conceptssuch as “personal space extension” [ ] previewed

new uses o mobile terminals.Te literature has tended to classi y mobile tasks onthe basis o searching/browsing categories and alsoaccording to management o known in ormation ornewin ormation.It is important to note that pre-literature does not widely consider touch terminalswhich incorporate new tasks. Having taken all thisinto account, we can classi y tasks as ollows:

(i) search [ – ]:

(a) in ormation gathering [ ];(b) using the web to more or less speci c

search;(c) compare/choose [ ];

(ii) browsing [ , , – ]:(a) undirected or conditioned browsing [ ];(b) watching videos [ ];(c) reading news [ ];

(iii) communication [ , , ]:(a) checking email [ ];(b) social networking sites [ ];(c) chat rooms/discussion rooms [ ];

(iv) transaction [ , , ]: although it is morecommon to use a mobile device to browse

the web or to per orm some shopping-relatedsearch than shopping in earnest [ ].

(v) playing games [ ];(vi) killing time [ ].

Some literature includes other kinds o task like“maintenance” [ ] or “housekeeping” [ ] that havenot been included in our classi cation because the

requency o realization is too low and these kinds o tasks do not de ne new kinds o interactions.

(D) Multidevice access: user’s amiliarity with a web page[ ] helps them to construct a mental model basedon the structural organization o the in ormation,such as visual cues, layout, and semantics. When asite is being designed or multidevice access, a majorconcern is to minimize user effort to reestablish theexisting mental model. Tis new way o workingaround structuredin ormation thatmustbe deliveredthrough so many different inter ace restrictions hasbeen studied as a new paradigm known as ResponsiveDesign [ ].

(E) Limited processing capability and power [ – ]: theselimitations include battery li e, network connectivity,download delays, and limited memory.

(F) Adoption [ ]: adoption o mobile technology by users is based on perceived privacy, acceptance o technology, com ort, and capacity o personalization.Different levels o adoption determine differentgroupo users interacting in a very different way withthe inter ace. Tis may not seem to be a mobile-speci c restriction but the wide variety o mobile

devices, touchable or keyboard-based, with differentsizes and presentation models, makes the range o users requiring different approaches much broader.

. . Rearrangement o raditional Heuristics. Te rst rear-rangement o traditional heuristics in this step is mainly based on the review o the literature by orrente [ ] where theauthor selected the most in uent heuristics guidelines [ , –

]. Tis compilation gives a total o heuristics guidelinesconsisting globally o heuristics and subheuristics. Weneed to rearrange this list o items into a new classi cationwhich is coherent to our purpose. In this step, we only takeintoaccount“heuristics” andno “subheuristics”andthisgivesus the heuristic list shown in Figure .

Our rearrangement ocuses on literature coincidences(i.e., when the same concept or category is included indifferent works in the literature, perhaps under differentnames) and tries to propose a coherent, exhaustive, andcom-plete ramework o heuristics that could be used to arrange

urther identi ed subheuristics. Literature coincidences oreach heuristics are as ollows:

( ) visibility o system status [, , ]: other bibliogra-phy re erences include this concept as “ rack State”[ ], “Give eed-back” [ ], or “Feedback” [ ];

( ) match between system and the real world [ , ];


5/20


Usability heuristics

(13) Privacy (12) Pleasurable and respectful interaction with the user(11) Skills

(10) Help and documentation(9) Help users recognize, diagnose, and recover from errors(8) Aesthetic and minimalist design(7) Flexibility and efficiency of use

(6) Recognition rather than recall

(5) Error prevention(4) Consistency and standards(3) User control and freedom

(2) Match between system and the real world(1) Visibility of system status

F : Proposed heuristic list.

( ) user control and reedom [ , , , ]: other bibli-ography re erences include this concept as “Supportthe user control” [ ], “Te eedback principle” [ ],“Autonomy” [ ], or “Visible Navigation” [ ];

( ) consistency and standards [ , , , ] also citedas “Maintain Consistency” [ ], “Structure Principle”[ ], “Reuse Principle” [ ], “Consistency” [ ], or“Learnability” [ ];

( ) error prevention [ , , ] also cited as “ olerancePrinciple” [ ];

( ) recognition rather than recall [ , ] also cited as“Reduce recent memory load or users” [ ], “Struc-tureprinciple” [ ], “Reuse principle” [ ], “Minimizethe users’ memory load” [ ], or “Anticipation” [ ];

( ) exibility and efficiency o use [, ] also citedas “Simplicity principle” [ ] or “Look at the user’sproductivity not the computer’s” [ ];

( ) aesthetic and minimalist design [ , ];

( ) help users recognize, diagnose, and recover romerrors [ , , ] also cited as “Good error messages”[ ] or “In case o error, is the user clearly in ormedand not over-alarmed about what happened and how to solve the problem?” [ ];

( ) help and documentation [ , , , – ];

( ) skills [ ] also cited as “Prepare workarounds or re-quent users” [ ], “Shortcuts” [ ], or “Readability”[ ];

( ) pleasurable and respect ul interaction with the user[ ] also cited as “Simplicity principle” [ ], “Simpleand natural dialog,” or “Speak the user’s language”[ ]: this point also includes any accessibility ques-tions that could enrich usability allowing a moreuniversal access, such as “Color blindness” [ ];

( ) privacy [ ].

. . Compilation o Subheuristics rom raditional General Heuristic Checklists. As de ned be ore, “heuristic” in this

paper re ers to a global usability issue which must beevaluated or taken into account when designing. In contrast,the term “subheuristic” re ers to speci c guidelines items. Inthis third step, we ocus on locating subheuristics rom theliterature.

Te rst group o potential heuristics is the sub-heuristics proposed in the re erences selected by orrente[ ]. Among these sub-heuristics we exclude those that donot t well with the previously described mobile constraints.For example, subheuristics re erred to desktop data entry methods is obviously discarded. In contrast, this re erringto screen use optimization is particularly relevant. Otherdiscarded amounts o subheuristics include some proposed[ ] with speci c response times which do not apply in amobile and varying context. We also discard coincidences

between different authors proposals.Tus, rom a total o amounts o subheuristics pro-

posed by the re erences [ , – ] selected by orrente [ ],in this study, we obtain a rst selection o subheuristics.

In order to maintain consistency in our classi cation,some subheuristics has been moved rom their originalheuristic parents, and new subcategories have been addedso that semantically related amounts o subheuristics aregrouped together. Te nal ramework, shown in Figure ,builds on that presented in the previous section.

Itis also important to recall that at this stage,subheuristicsredactions have beenkept unchanged rom their correspond-ing re erences. In the nal compilation, these redactions will

be modi ed in order to homogenize the whole guideline aswe planned or step in our methodology.Te nal list o subheuristics is as ollows:( ) visibility o system status:

system status eedback:

( ) is there some orm o system eedback or every operator action? [ ]

( ) i pop-up windows are used to display errormessages, do they allow the user to see the eldin error? [ ]

( ) in multipage data entry screens, is each pagelabeled to show its relation to others? [ ]


6/20


Subheuristics from general heuristic checklists

(13) Privacy

(12) Pleasurable and respectful interaction Input data(11) Skills(10) Help and documentation

(9) Help users recognize, diagnose and recover from errors

(8) Aesthetic and minimalist designMenusIconsMultimedia content

(7) Flexibility and efficiency of use Search


MenusInput/output dataGeneral visual cuesMemory load reduction

(5) Error prevention

(4) Consistency

System response consistency Functional goals consistency Menus/task consistency Naming convention consistency

Design consistency Input eldsMenus

(3) User control

Menus controlUndo/cancelationProcess con rmationSome level of personalizationExplorable interfaces

Output of numeric informationSimplicity MenusNavigational structureMetaphors/mental models

(1) Visibility of system status

Selection/input of dataResponse timeLocation informationSystem status feedback

(2 ) Match between system and the real world(mental model accuracy)

F : First ramework or classi cation o detected subheuristics.

( ) are high in ormative contents placed in highhierarchy areas? [ ]

location in ormation:

( ) is the logo meaning ul, identi able, and suffi-ciently visible? [ ]

( ) is there any link to detailed in ormation aboutthe enterprise, website, webmaster . . . ? [ ]

( ) are there ways o contacting with theenterprise?[ ]

( ) in articles, news, reports . . . are the author,sources, dates, and review in ormation shownclearly? [ ]

response times:

( ) are response times appropriate or the userscognitive processing? [ ]

( ) are response times appropriate or the task? [ ]

( ) i there are observable delays (greater thanfeen seconds) in the system’s response time, is

the user kept in ormed o the system progress?[ ]

( ) latency reduction [ ];

Selection/input o data:

( ) is therevisual eedback in menusordialog boxesabout which choices areselectable? [ ]. We willmerge this statement with the ollowing: “DoGUI menus make obvious which item has beenselected?” [ ], “Do GUI menus make obviouswhether deselection is possible?” [ ], “Is there visual eedback in menus or dialog boxes aboutwhich choice the cursor is on now?” [ ], and“I multiple options can be selected in a menuordialog box, is there visual eedback about whichoptions are already selected?” [ ]

( ) is the current status o an icon clearly indicated?[ ]


7/20


( ) is there visual eedback when objects areselected or moved? [ ]

( ) are links recognizable? Is there any characteri-zation according to the state (visited, active, . . . )?[ ]

( ) match between system and the real world (Mentalmodel accuracy):

metaphors/mental models:

( ) use o metaphors [ ];( ) are icons concrete and amiliar? [ ]( ) i shape is used as a visual cue, does it match

cultural conventions? [ ]( ) do the selected colours correspond to common

expectations about color codes? [ ]

navigational structure:

( ) i the site uses hierarchical structure, are depthand height balanced? [ ]

( ) navigation map [ ], also known as site map ortable o contents;

menus:

( ) are menu choices ordered in the most logicalway, given the user, the item names, and the task variables? [ ]

( ) do menu choices t logically into categories thathave readily understood meanings? [ ]

( ) are menu titles parallel grammatically? [ ]( ) in navigation menus, are the number o itemsand terms by item controlled to avoid memory overload? [ ]

simplicity:

( ) do related and interdependent elds appear onthe same screen? [ ]

( ) or question and answer inter aces, are ques-tions stated in clear, simple language? [ ]

( ) isthe languageused thesame targetusersspeak?[ ]. We will merge this statement with the

ollowing: “Is the menu-naming terminology consistent with the user’s task domain?” [ ]( ) is the language clear and concise? [ ]. We will

merge this statement with the ollowing: “Doesthe command language employ user jargon andavoid computer jargon?” [ ]

( ) does the site ollow the rule “ paragraph = idea”? [ ]

output o numeric in ormation:

( ) does the system automatically enter leading ortrailing spaces to align decimal points? [ ]

( ) doesthe systemautomatically enter a dollarsignand decimal or monetary entries? [ ]

( ) does the system automatically enter commas innumeric values greater than ? [ ]

( ) are integers right-justi ed and real numbersdecimal-aligned? [ ]

( ) user control:

explorable inter aces:

( ) can users move orward and backward betweenelds or dialog box options? [ ]

( ) i the system has multipage data entry screens,can users move backward and orward amongall the pages in the set? [ ]

( ) i the system uses a question and answer inter-ace, can users go back to previous questions or

skip orward to later questions? [ ]

( ) clearly marked exits [ ];( ) is the general website structure user-oriented?

[ ]( ) is there any way to in orm userabout wherethey

are and how to undo their navigation? [ ]

some level o personalization:

( ) can users set their own system, session, le, andscreen de aults? [ ]

process con rmation:

( ) when a user’s task is complete, does the systemwait or a signal rom the user be ore process-ing? [ ]

( ) are users prompted to con rm commands thathave drastic, destructive consequences? [ ]

undo/cancelation:

( ) can users easily reverse their actions? [ ] Alsoound as “Do unction keys that can cause

serious consequences have an undo eature?”[ ] and “Is there an“undo” unction at the levelo a single action, a data entry, and a complete

group o actions?” [ ]( ) can users cancel out o operations in progress?

[ ]

menus control:

( ) i the system has multiple menu levels, is therea mechanism that allows users to go back toprevious menus? [ ]

( ) are menus broad (many itemson a menu) ratherthan deep (many menu levels)? [ ]

( ) i users can go back toa previousmenu, can they change their earlier menu choice? [ ]


8/20


( ) consistency:

designing consistency:

( ) are attention-getting techniquesused with care?[ ]

( ) intensity: two levels only [ ];( ) color: up to our (additional colors or occa-

sional use only) [ ];( ) are there no more than our to seven colors, and

are they ar apart along the visible spectrum?[ ]

( ) sound: sof tones or regular positive eedback,harsh or rare critical conditions [ ];

( ) i the system has multipage data entry screens,do all pages have the same title? [ ]

( ) do online instructions appear in a consistentlocation across screens? [ ]

( ) have industry or companystandardsbeenestab-lished or menu design, and are they appliedconsistently on all menu screens in the system?[ ]

( ) are there no more than twelve to twenty icontypes? [ ]

( ) has a heavy use o all uppercase letters on ascreen been avoided? [ ]

( ) is there a consistent icon design scheme andstylistic treatment across the system? [ ]

menus:

( ) are menu choice lists presented vertically? [ ]( ) i “exit” is a menu choice, does it always appear

at the bottom o the list? [ ]( ) are menu titles either centered or lef-justi ed?

[ ]

input elds:

( ) are eld labels consistent rom one data entry screen to another? [ ]

( ) do eld labels appear to the lef o single eldsand above list elds? [ ]

( ) are eld labels and elds distinguished typo-graphically? [ ]

naming convention consistency:

( ) is the structure o a data entry value consistentrom screen to screen? [ ]

( ) are system objects named consistently across allprompts in the system? [ ]

( ) are user actions named consistently across allprompts in the system? [ ]

menu/task consistency:

( ) are menu choice names consistent, both withineach menu and across the system, in grammati-cal style and terminology? [ ]

( ) does the structure o menu choice names match

their corresponding menu titles? [ ]( ) does the menu structure match the task struc-

ture? [ ]( ) when prompts imply a necessary action, are

the words in the message consistent with thataction? [ ]

unctional goals consistency:

( ) where are the website goals? Are they wellde ned? Do content and services deliveredmatch these goals? [ ]

( ) does thelook & eel correspond with goals,char-acteristics, contents and services o the website?[ ]

( ) is the website being updated requently? [ ]

system response consistency:

( ) is system response afer clicking links pre-dictable? [ ]

( ) are nowhere links avoided? [ ]( ) are orphan pages avoided? [ ]

( ) error prevention:

( ) are menu choices logical, distinctive, and mutu-ally exclusive? [ ]

( ) are data inputs case-blind whenever possible?[ ]

( ) does the system warn users i they are about tomake a potentially serious error? [ ]

( ) do data entry screens and dialog boxes indicatethe number o character spaces available in a

eld? [ ]( ) do elds in data entry screens and dialog boxes

contain de ault values when appropriate? [ ]

( ) recognition rather than recall:

memory load reduction:

( ) high levels o concentration are not necessary and remembering in ormation is not required:two to feen seconds [ ];

( ) are all data a user needs on display at each stepin a transaction sequence? [ ]

( ) i users have to navigate between multiplescreens, does the system use context labels,menu maps, and place markers as navigationalaids? [ ]


9/20


( ) afer the user completes an action (or group o actions),does the eedback indicate that thenextgroup o actions can be started? [ ]

( ) are optional data entry elds clearly marked?[ ]

( ) do data entry screens and dialog boxes indicate

when elds are optional? [ ]( ) is page length controlled? [ ]

general visual cues:

( ) or question and answer inter aces, are visualcues and white space used to distinguish ques-tions, prompts, instructions, and user input?[ ]

( ) does the data display start in the upper-lefcorner o the screen? [ ]

( ) have prompts been ormatted using white space, justi cation, and visual cues or easy scanning?

[ ]( ) do text areas have “breathing space” aroundthem? [ ]

( ) are there “white” areas between in ormationalobjects or visual relaxation? [ ]

( ) does the system provide visibility; that is, by looking, can the user tell the state o the systemand the alternatives or action? [ ]

( ) is size, bold ace, underlining, colour, shading,or typography used to show relative quantity orimportance o different screen items? [ ]

( ) is colour used in conjunction with some otherredundant cue? [ ]

( ) is there good colour and brightness contrastbetween image and background colours? [ ]

( ) have light, bright, saturated colours been usedto emphasize data and have darker, duller, anddesaturated colours been used to deemphasizedata? [ ]

( ) is the visual page space well used? [ ]

input/output data:

( ) on data entry screens and dialog boxes, aredependent elds displayed only when neces-sary? [ ]

( ) are eld labels close to elds, but separated by atleast one space? [ ]

Menus

( ) is the rst word o each menu choice the mostimportant? [ ]

( ) are inactive menu items grayed out or omitted?[ ]

( ) are there menu selection de aults? [ ]( ) is there an obvious visual distinction made

between “choose one” menu and“choose many”menus? [ ]

( ) exibility and efficiency o use:

search:

( ) is the searching box easily accessible? [ ]( ) is the searching box easily recognizable? [ ]

( ) is there any advanced search option? [ ]( ) are search results shown in a comprehensivemanner to the user? [ ]

( ) is the box width appropriated? [ ]( ) is the user assisted i the search results are

impossible to calculate? [ ]

( ) aesthetic and minimalist design:

( ) Fitt’s Law []: the time to acquire a target is aunction o the distance to and size o the target;

( ) is only (and all) in ormation essential to deci-sion making displayed on the screen? [ ]

( ) are eld labels brie , amiliar, and descriptive?[ ]

( ) are prompts expressed in the affirmative, anddothey use the active voice? [ ]

( ) is layout clearly designed avoiding visual noise?[ ]

multimedia content:

( ) does the use o images and multimedia contentadd value? [ ]

( ) are imageswell sized? Are theyunderstandable?

Is the resolution appropriate? [ ]( ) are cyclical animations avoided? [ ]

icons:

( ) hasexcessivedetail in icon designbeen avoided?[ ]

( ) is each individual icon a harmonious membero a amily o icons? [ ]

( ) does each icon stand out rom its background?[ ]

( ) are all icons in a set visually and conceptually distinct? [ ]

menus:

( ) is each lower-level menu choice associated withonly one higher level menu? [ ]

( ) are menu titles brie , yet long enough to com-municate? [ ]

( ) help users recognize, diagnose and recover romerrors;

( ) help and documentation:

( ) are online instructions visually distinct? [ ]


10/20


( ) do the instructions ollow the sequence o useractions? [ ]

( ) i menu choices are ambiguous, does the sys-tem provide additionalexplanatory in ormationwhen an item is selected? [ ]

( ) i menu items are ambiguous, does the sys-

tem provide additionalexplanatory in ormationwhen an item is selected? [ ]

( ) is the help unction visible, or example, a key labeled HELP or a special menu? [ , ]

( ) is the help system inter ace (navigation, pre-sentation, and conversation) consistent withthe navigation, presentation, and conversationinter aces o the application it supports? [ ]

( ) navigation: is in ormation easy to nd? [ ]( ) presentation: is the visual layout well designed?

[ ]( ) conversation: is the in ormation accurate, com-

plete, and understandable? [ ]( ) is the in ormation relevant? ([ ], Help and

documentation) [ ] It shouldbe relevantin theollowing aspects [ ]: goal-oriented (what can

I do with this program?), descriptive (what isthis thing or?), procedural (how do I do thistask?), interpretive (why did that happen?), andnavigational (where am I?);

( ) is there context-sensitive help? [ , ]( ) can the user change the level o detail available?

[ ]( ) can users easily switch between help and their

work? [ ]

( ) is it easy to access and return rom the helpsystem? [ ]

( ) can users resume work where they lef off aferaccessing help? [ ]

( ) i a FAQs section exists, are the selection andredactiono questionsand answerscorrect? [ ]

( ) skills:

( ) do not use the word “de ault” in an applicationor service; replace it with “Standard,” “Use Cus-tomary Settings,” “Restore Initial Settings,” orsome other more speci c terms describing whatwill actually happen [ ];

( ) i the system supports both novice and expertusers, are multiple levels o error message detailavailable? [ ]

( ) i the system supports both novice and expertusers, aremultiple levels o detail available? [ ]

( ) are users the initiators o actions rather than theresponders? [ ]

( ) do the selected input device(s)match user capa-bilities? [ ]

( ) are important keys (e.g., EN ER, AB) largerthan other keys? [ ]

( ) does the system correctly anticipate andpromptor the user’s probable next activity? [ ]

( ) pleasurable and respect ul interaction:

( ) protect users’ work [ ], also as “For data entry screens with many elds or in which sourcedocuments may be incomplete, can users savea partially lled screen?” [ ]

( ) do the selected input device(s) match environ-mental constraints? [ ]

( ) are typing requirements minimal or questionand answer inter aces? [ ]

( ) does the system complete unambiguous partialinput on a data entry eld? [ ]

( ) privacy:

( ) areprotectedareas completely inaccessible? [ ]( ) can protected or con dential areas be accessed

with certain passwords [ ]( ) is there in ormation about how personal data is

protected and about contents copyright? [ ]

. . Compilation o Mobile-Speci c Subheuristics. Te ourthstep in this work is to enrich the list with mobile-speci csubheuristics. Te subheuristic list obtained in the previoussection does not include many mobile speci c questionsbecause, as mentioned be ore, traditional heuristics does notusually cover these issues. New mobile-speci c questionshave been added into this list, taken rom mobile usability

studies and best practices that actually do not provide HE.Our approach allows us to include these new items intotheir corresponding categories, enriching the heuristic withmobile-speci c issues. Some new categories had to be addedto the original heuristic ramework to include new mobile-speci c subheuristics. Te nal ramework is shown inFigure .

As we mentionedearlier, notall mobile devices have beenconsidered; we discarded eatured phones because they arerarelyused or tasks other than phone calls andshortmessageservices (SMS) and theyare gradually being abandonedapart

rom speci c groups o users such as elderly or cognitively impaired people. We also discarded smartphones (phones

with midsized screens and ull A–Z keypads) because theinteractivity with these devices is dramatically different romthat o touch phones and they are commonly constrainedto enterprise use. Tis study is centred in touch phones andtablets which are very popular nowadays and similar rom ausability point o view.

Tis ourth step adds new subheuristics to the compi-lation:

( ) visibility o system status:

System status eedback:

( ) All the items on a list should go on the samepage: i the items are text-only and i they are


11/20


Subheuristics for mobile

(13) Privacy

(12) Pleasurable and respectful interaction

Banking

Shopping

Input data

(11) Skills

(10) Help and documentation

(9) Help users recognize, diagnose and recover from errors

(8) Aesthetic and minimalist design

Navigation

Orientation

Menus

Icons

Multimedia content

(7) Flexibility and efficiency of use Navigation

Search


Navigation

Menus

Input/output data

General visual cues

Memory load reduction

(5) Error prevention Fat- nger syndrome

(4) Consistency

Orientation

System response consistency

Functional goals consistency

Menus/task consistency Naming convention consistency

Design consistency Input elds

Menus

(3) User control

Menus control

Undo/cancelation

Process con rmation

Some level of personalization

Explorable interfaces

(2) Match between system and the real world (mental model accuracy)

Output of numeric information

Simplicity

Menus

Navigational st ructure

Metaphors/mental models

(1) Visibility of system status

Presentation adaptationSelection/input of data

Response time

Location information

System status feedback

F : Second ramework or classi cation o detected subheuristics.

sorted in an order that matches the needs o thetask [ ];

( ) i a list o items can be sorted according todifferent criteria, provide the option to sort thatlist according to all those criteria [ ];

( ) i a list contains items that belong to differentcategories, provide lters or users to narrow

down the number o elements that they need toinspect [ ];

( ) i the list contains only one item, take the userdirectly to that item [ ];

( ) i the list contains items that download slowly (e.g., images), split the list into multiple pagesand show just one page at a time [ ];


12/20


( ) i an article spans several pages, use paginationat the bottom. Have a link to each individualpage, rather than just to the previous and thenext ones [ ];

location in ormation:

( ) whenever you have physical location in orma-tiononyourwebsite, link itto a map and includea way o getting directions [ ];

response time:

( ) splash screens too long [ ];( ) download time [ ]: “Progress bar is pre erable”

and “Alternative entertainment i downloadtime is greater than seconds”;

selection/input o data:

( ) lowdiscoverability (active areas that do not look touchable): users do notknow that something istouchable unless it looks as i it is [ ];

( ) swiping [ ]: swiping is still less discoverablethan most other ways o manipulating mobilecontent, so we recommended includinga visiblecue when people can swipe. And swipe ambi-guity should be avoided: the same swipe gestureshould not be used to mean different things ondifferent areas o the same screen:

( ) expandable menus should be used sparingly.Menu labels should clearly indicate that they expand to a set o options [ ];

presentation adaptation:

( ) detect i users are coming to your site on amobile phone and direct them to your mobilesite [ ];

( ) include a link to your mobile site on your ullsite. It can direct mobile users who were not re-directed to your mobile site [ ];

( ) include a link to the ull site on the mobile page[ ];

( ) match between system and the real world:

navigational structure:

( ) too much navigation ( MN) [ ];

( ) user control and reedom:

explorable inter aces:

( ) accidental activation (lack o back button) [ ];( ) include navigation on the homepage o your

mobile website [ ];

( ) consistency and standards:

orientation:

( ) about constraining orientation: users tend toswitchorientation whenan impasse occurs and,i the application does not support them, their

ow is going to be disrupted, and they are goingto wonder why it is not working [ ];

( ) navigation (horizontal and vertical) must beconsistent across orientations. Some applica-tions use a different navigation direction in thetwo orientations; or instance, they use hori-zontal navigation in landscape and use verticalnavigation in portrait [ ];

( ) inconsistent content across orientations [ ]:“Same content,” “Keep location,” and “I a ea-ture is only available in one orientation, in ormusers”;

( ) error prevention

( ) accidental activation (lack o back button) [ ];

at- nger syndrome:

( ) touchable areas are too small [ ]. Research hasshown thatthe best target size or widgets is cm× cm or touch devices [ ];

( ) crowding targets: another at- nger issue thatwe encountered requently is placing targets tooclose to each other. When targets are placed tooclose toeach other, userscaneasily hitthe wrongone [ ];

( ) padding: although the visible part o the targetmaybe small, there is some invisible targetspacethat i a user hits that space, their tap will stillcount [ ];

( ) when several items are listed in columns, oneon top o another (see the time example below),users expect to be able to hit anywhere in therow to select the target corresponding to thatrow. Whenever a design does not ul l thatexpectation, it is disconcerting or users [ ];

( ) do not make users download sofware that isinappropriate or their phone [ ];

( ) JavaScript and Flash do not work on many phones; do not use them [ ];

( ) recognition rather than recall:

Memory load reduction:

( ) the task ow should start with actions that areessential to the main task. Users should be ableto start the task as soon as possible [ ];

( ) the controls that are related to a task shouldbe grouped together and re ect the sequence o actions in the task [ ];


13/20


navigation:

( ) usebreadcrumbson sites with a deep navigationstructure (many navigation branches). Do notuse breadcrumbs on sites with shallow naviga-tion structures [ ];

( ) Flexibility and efficiency o use:

search:

( ) a search box and navigation should be presenton the homepage i your website is designed orsmartphones and touch phones [ ];

( ) the lengtho the search box shouldbeatleast thesize o theaveragesearch string. We recommendgoing or the largest possible size that will t onthe screen [ ];

( ) preserve search strings between searches. Useautocompletion and suggestions [ ];

( ) do not use several search boxes with differentunctionalities on the same page [ ];

( ) i the search returns zero results, offer somealternativesearchesor a link to thesearch resultson the ull page [ ];

navigation:

( ) use links with goodin ormation scent (i.e., linkswhich clearly indicate where they take theusers)on your mobile pages [ ];

( ) use links to related content to help the usernavigate more quickly between similar topics[ ];

( ) aesthetic and minimalist design:

( ) recognizableapplication icons to be ound in thecrowded list o applications [ ];

multimedia content:

( ) getting rid o Flash content [ ];( ) carousels [ ]: avoid using animated carousels,

but i they must be used, users should be able tocontrol them;

( ) do not use image sizes that are bigger than thescreen. Te entire image should be viewablewith no scrolling [ ];

( ) or cases where customers are likely to needaccess to a higher resolution picture, initially display a screen-size picture and add a separatelink to a higher resolution variant [ ];

( ) when you use thumbnails, make sure the usercan distinguish what the picture is about [ ];

( ) usecaptions or images that are part o an articlei their meaning is not clear rom the context o the article [ ];

( ) do not use moving animation [ ];( ) i you have videos on your site, offer a textual

description o what the video is about. [ ];( ) clicking on the thumbnail and clicking on the

video title should both play the video [ ];( ) indicate video length [ ];( ) speci y i the video cannot be played on the

user’s device [ ];( ) use the whole screen sur ace to place in orma-

tion efficiently [ ]: “Popovers or displayingin ormation restricts size o rame where in or-mation will be shown” and “Small modal viewspresent the same size constraints”;

orientation:

( ) desktop websites have a strong guideline toavoid horizontal scrolling.But or touch screens,horizontal swipes are ofen ne [ ];

navigation:

( ) do not replicate a large number o persistentnavigation options across all pages o a mobilesite [ ];

( ) Help users recognize, diagnose, and recover romerrors:

( ) o signal an input error in a orm, mark thetextbox that needs to be changed [ ];

( ) help and documentation:

( ) ocus on one single eature at a time. Presentonly those instructions that are necessary ortheuser to get started [ ];

( ) skills:( ) pleasurable and respect ul interaction:

input data:

( ) users dislike typing. Compute in ormation orthe users. For instance, ask only or the zip codeand calculate state and town; possibly offer a list

o towns i there are more under the same zipcode [ ];( ) be tolerant o typos and offer corrections. Do

not make users type in complete in ormation.For example, accept “ Main” instead o “Main St.” [ ];

( ) save history and allow users to select previously typed in ormation [ ];

( ) use de aults that make sense to the user [ ];( ) I theapplication doesnot store any in ormation

that is sensitive (e.g., credit card), then the usershould de nitely be kept logged in (log outclearly presented) [ ];


14/20


( ) minimize the number o submissions (andclicks) that the userneeds togo throughin orderto input in ormation on your site [ ];

( ) When logging in must be done, use graphicalpasswords at least some o the time, to getaround typing [ ];

( ) Do not ask people to register ona mobilephone;skipping registration should be the de aultoption [ ];

( ) When logging in must be done, have an optionthat allows the user to see the password clearly [ ];

shopping:

( ) when you present a list o products, use imagethumbnails that are big enough or the user toget some in ormation out o them [ ];

( ) on a product page, use an image size that ts the

screen. Add a Link to a higher resolution imagewhen the product requires closer inspection[ ];

( ) offer the option to email a product to a riend[ ];

( ) offer the option to save the product in a wish list[ ];

( ) on an e-commerce site, include salient linkson the homepage to the ollowing in ormation:locations and opening hours (i applicable),shipping cost, phone number, order status, andoccasion-based promotions or products [ ];

banking and transactions:

( ) whenever users conduct transactions on thephone, allow them to save con rmation num-bers orthat transaction byemailing themselves.I the phone has an embedded screen-capture

eature, show them how to take a picture o theirscreen [ ];

( ) privacy:

( ) or multiuser devices, avoid being permanently signed in on an application [ ];

( ) I the application does store credit card in or-mation, it should allow users to decide i they want to remain logged in [ ]. Ideally, when theuser opts to be kept logged in, he/she should geta message in orming o the possible risks

. . Final New Mobile-Speci c Heuristics. Te nal com-pilation o heuristics and subheuristics, which is shownin Appendix A (Supplementary Material available onlineat http://dx.doi.org/ . / / ), gives a total o heuristics and subheuristics ( + ). In this nalcompilation, we have omitted intermediate classi cationsintroduced during the discussion. Also, semantically related

items have been merged into a single item ollowing themostcommon presentation o heuristics guidelines in literature.Wording has been corrected to offer a homogeneous collec-tion o heuristics questions.

Tis nal mobile heuristics can be used as a tool toevaluate usability o mobile inter aces. In its current version,

possible answers or the proposed questionsare “yes/no/NA.”Te number o “yes” answers provides a measure o theusability o the inter ace. Other approaches in the literatureinclude more elaborates ratings that have to be agreedbetween evaluators [ ].

. . Empirical est o the New Mobile-Speci c Heuristics. Tegoal o our testwas to per orm an evaluationo the use ulnesso the proposed heuristics as a tool ordesigners andsofwareengineers with no speci c knowledge and experience o usability.

Te usecase designwasas ollows: twosofware engineerswithout any speci c knowledge about usability were asked to

designan inter ace ora tabletapplication having a unctionaldescription in a low- delity prototype designed or a desktop version o the application. Over their proposed inter acedesign theyused ourheuristics as an evaluation and re exiontool. In view o the results o the evaluation, they were askedto develop a new prototype. Finally, both inter aces weretested with a small group o users to compare their usability.

Tis empirical test o use ulness o the proposed usability list was divided into the ollowing phases:

( ) prototype : developing an inter ace prototype ori-ented to tablet access rom a given PC-desktop low-

delity unctional design (prototype , P );( ) HE o P using the proposed heuristics as the basis or

an oriented discussion between designers;( ) prototype : evolution o P xing usability gaps

detected in phase (prototype , P );( ) Empirical comparison o prototypes: users’ testing o

P and P .

. . . Prototype Developing. Te unctional description o the desktop version used to build the prototypes evaluated inthis testing was provided by Project PROCUR@ [ ], an e-care and e-rehabilitation plat orm ocused on neurodegener-ative diseases patients, their carers, and health pro essionals.Te project is based on the deployment o three social spaces

or researchandinnovation (SSRI) [ ] in the threevalidationscenarios: Parkinson’s disease SSRI, acquired brain damage(ABD) SSRI, and Alzheimer’s disease SSRI. Te unctionaldescription corresponds to this latter SSRI and provides velow- delity inter ace descriptions rom the point o view o ve pro les: patients, relatives, doctors, caregivers, andsanitary personnel, respectively.

Te subjects o these experiments were two sofwareengineering students preparing their end o degree project.Tey had never been trained in usability but had knowledgeabout sofware li e-cycles and design techniques. P was theresult o a rst tablet-inter ace adaptation without usability training. Te tablet ormat was imposed because a bigger


15/20


screen size is specially convenient or the target users (i.e.,elderly people with low vision capability and motor control).

Tis rst adaptation included two main groups o changes: unctional re nement and new inter aceadaptation.Functional re nement required changes that were notpartic-ularly relevant to this work. However, adaptations to the new

inter ace involved decisions adopted by designers withoutknowledge o usability, guided only by their common sense.At a later stage, some o these decisions were con ronted withthe HE new tool and not all o them were maintained. Tesedecisions are described in Figure .

Figure shows an example o the inter ace change.

. . . Prototype : Heuristic Evaluation. Once Prototype was designed, the next step was to evaluate its usability. Teobjective was not the evaluation itsel but how the designersre ected on its usability.

When per orming a HE using such tool, one has to makecertain decisions about the scoring o each subheuristics. Inthis case, the experts were asked to use a ponderation whichwould allow the prioritization o heuristic item relevance

or the speci c evaluated inter ace. Experts are marked with values rom to : or accomplished heuristic items, orthose corresponding to usability gaps, or heuristic itemswhich were not evaluable in the actual sofware li e-cyclephase, and or questions not applicable to the inter ace.

Applying a Delphi-based [ ] approximation, bothexperts were asked to independently evaluate the inter aceusing the list. Aferwards, the results o the evaluations werecon ronted and the experts had to agree in the case o itemswith different scorings.

In the independent evaluation, the level o coincidenceo the experts was moderate and in the nal HE scoring,where both experts agreed, the results were as ollows: items scored , items scored , scored , and scored

. Tis nal result established a huge number o items as“not applicable.” Tis may have been because the heuristicswas intended to be as general as possible, not ocusing onany speci c kind o application, and it there ore included anexhaustive list o checks.

Te most important result rom this evaluation was thatexperts were orced to re ect on each item in the heuristicguideline. For each not accomplished question, they learntwhich usabilitygaps hadto be avoided in the inter ace design.Tis learning provided a wider knowledge background when

it came to designing next prototype.

. . . Prototype Building. Prototype was not only a serieso modi cations to Prototype butalso a complete revision o the whole inter ace concept. Tis global re exion was guidedby the expert discussion rom the previous section.

Te most speci c changes which x detected usability gaps are shown in Figure but, as mentioned, the overallappearance anddesign have changed dramatically ( Figure ).

. . Empirical Comparison o Prototypes. Te empirical com-parison o the two prototypes was intended to evaluate

whether P designed using the proposed HE tool was betterin any way than P .

Tisempiricalstudyinvolved usersso theexperiment hadto be designed care ully to obtain valid results. Te approachincluded a test design, a pilot phase to check the test design,the execution o the test itsel , and a phase to analyze the

collected data.Several decisions were taken in the design phase.

( ) Wizardo Oz [ ] (WO) was chosen as the evaluationtechnique because the prototypes are developed onpaper and are well suited to presentation throughhuman intervention to the users.

( ) o develop WO technique, users were asked to per-orm a task-guided interaction. Te experts selected

three unctional tasks that users had to carry outinteracting with the inter ace. Te tasks were repre-sentative enough to be use ul in this test. Tey werebrie y described to the users so that they were able toaccomplish them by exploring the inter ace withoutstep-by-step guides.

( ) en userswere selected with the characteristics shownin able .

( ) Te experimentation adopted an inner group [ ]design: hal o theusersinteracted with P rst andtheother hal with P rst. Tis was to avoid as much aspossible correlations due to learning o the inter aces.

( ) Lastly, users were asked to give eedback about theiroverall eelings abouteach inter ace toprovideuswithsome conclusions related to user experience beyondusability.

Te pilot phase consisted o a simulation o the nalexperiment using two dummy users. Tis phase was very use ul or consolidating o task description and helping toimprove Wizard’s skills managing prototypes and or thewhole experiment.

Final test execution detected serious usability gaps inP and serious gaps in P (which are also included in the

rst ) as can be seen in able . When asked about generalsatis action, % o users stated they were more satis edwith P prototype interaction.

4. Conclusions and Future Work

In this paper, we have presented a compilation o heuristicevaluation checklists readapted to mobile inter aces. Westarted our work by reusing heuristics rom desktop heuris-tics evaluation checklists, which is allowed because “heuristicchecklists change very slowly because they derive romhuman behaviour, not technology” [ ]. In act, in the nalproposal o this work, the amount o reused heuristics romthe literature is % o the total proposed subheuristics.Te rest are best-practices and recommendations or mobileinter aces not initially conceived as part o a usability tool.

In the nal collection o heuristics, the most in uentauthor is Nielsen [ , – ]. While it is not a long list o heuristics, it is exhaustive enough to be a use ul categoriza-tion or urther research. However, in our work, Nielsen’s


16/20


(2) New interface adaptation

(2.8) Tabulated information edited through touching areas(2.7) Tactile keyboard appears when necessary (2.6) Replacement of information shown through menus to a navigable screens tree(2.5) Replacement of login screen adapted to touchable interface(2.4) Replacement of page navigation for scrollbar(2.3) Addition of scrollbars when screen dimensions are exceeded(2.2) Visual buttons instead of links

(2.1) Links to external pages are remove or replace for speci c information(1) Functional re nement

P1 design decisions

F : Design decisions in prototype .

F : Prototype rom desktop version description.

(10) Bigger buttons with coherent style and distribution along the interface

(9) Horizontal scrollbars replaced by visual clues of pagination and use of “switch” movement

(8) Logout functionality marked in red color and warning of closing added

(7) Redistribution of visual elements to use the whole screen

(5) Navigation buttons are replaced by icons referred to mental models

(4) Navigations buttons are avoided as much as possible and include search tools over navigation

(3) Search boxes with visual clues of touchable areas

(2) Touch keyboard style replaced old desktop version

(1) Show password option added

P2 resolution of usability gaps(6 ) List and tables are replaced when possible for lists of buttons + icons

F : Prototype main changes.

F : Prototype global concept changes rom the rst version.


17/20


18/20


Te speci city o the items collected in the tool means that itcan be used as a re erence guide to help conceive more usableinter aces andnot just as a reactiveevaluation tool orexistingprototypes. Future work should look into this to con rm thispartial result.

Furthermore, other aspects related to the suitability o

this guideline need to be validated. For instance, an experts-guided review could evaluate the completion, coherence,and adequacy o the heuristic checklist. Tis review couldbe carried out through questionnaires, experts panels, orsome kind o Delphi-based surveys [ ]. Another highly interesting question is the empirical comparison o generalheuristics and this mobile-speci c heuristics when analysingmobile inter aces.

Conflict of Interests

Te authors declare that there is no con ict o interestsregarding the publication o this paper.

Acknowledgments

Tis work has been supported by project PROCUR@-IP -- - , unded by the program INNPAC O o

MINECO and FEDER unds and by the ele onica Chair“Intelligence in Networks” o the Universidad de Sevilla,Spain. Te authors would like to thank the members o project PROCUR@, engineers Anabel Hern ández Luis andLidia Romero Martı́nez who participated in the empiricalevaluation and Dr. V ı́ctor Dı́az Madrigal and Dr. Jos é Mar-iano Gonz ález Romano or their support in the initial phaseso this work.

References

[ ] International Organization or Standardization (ISO), “ISO- : Ergonomic requirements or office work with

visual displayterminals (VD s)—part : guidanceon usability,”http://www.iso.org/iso/iso catalogue/catalogue tc/cataloguedetail.htm?csnumber= .

[ ] R. Inostroza, C. Rusu, S. Roncagliolo, C. Jim´enez, and V. Rusu,“Usability heuristics or touchscreen-based mobile devices,” inProceedings o the th International Con erence on In ormation

echnology (I NG ' ), pp. – , Las Vegas, Nev, USA, April.

[ ] J. Wang andS. Senecal, “Measuring perceivedwebsite usability,” Journal o Internet Commerce, vol. , no. , pp. – , .

[ ] J. Karat, “User-centered sofware evaluation methodologies,” inHandbook o Human-Computer Interaction, M. Helander, . K.Landauer, and P. Prabhu, Eds., pp. – , Elsevier,New York,NY, USA, .

[ ] D. Zhang, “Overview o usability evaluation methods,”http://www.usabilityhome.com .

[ ] J. Heo,D.Ham, S. Park, C. Song, andW. C. Yoon, “A ramework or evaluating the usability o mobile phones based on multi-

level, hierarchical model o usability actors,” Interacting withComputers , vol. , no. , pp. – , .

[ ] M. C. S. orrente, Sirius: Sistema de evaluacíon de la usabilidad web orientado al usuario y basado en la determinación de tareas

cr ́ıticas [Ph.D. thesis], Universidad de Oviedo, Oviedo, España,.

[ ] M. Y. IvoryandM. A. Hearst, “Testateo the art in automatingusability evaluation o user inter aces,” ACM Computing Sur-veys, vol. , no. , pp. – , .

[ ] J. Nielsen and R. Molich, “Heuristic evaluation o user inter-aces,” in Proceedings o the SIGCHI Con erence on Human

Factors in Computing Systems (CHI ’ ), J. C. Chew and J.Whiteside, Eds., pp. – , ACM, New York, NY, USA, .

[ ] J. Nielsen, “Guerrilla HCI: Using Discount Usability Engineer-ing to Penetrate the Intimidation Barrier,” Nielsen’s Alertbox,

, http://www.nngroup.com/articles/guerrilla-hci/ .[ ] R. Agarwal and V. Venkatesh, “Assessing a rm’s Web presence:

a heuristic evaluation procedure or the measurement o usabil-ity,” In ormation Systems Research, vol. , no. , pp. – ,

.[ ] C.-M. Karat, R. Campbell, and . Fiegel, “Comparison o

empirical testing and walkthrough methods in user inter aceevaluation,” in Proceedings o the SIGCHI Con erence on HumanFactors in Computing Systems (CHI ’ ), pp. – , ACM,

New York, NY, USA, May .[ ] J. Nielsen, “Heuristic evaluation,” in Usability Inspection Meth-ods, J. Nielsen and R. L. Mack, Eds., John Wiley & Sons, New York, NY, USA, .

[ ] R. Budiu and N. Nielsen, Usability o iPad Apps and Websites,Nielsen Norman Group, nd edition, .

[ ] C. Rusu, S. Roncagliolo, V. Rusu, and C. Collazos, “A method-ology to establish usability heuristics,” in Proceedings o the

th International Con erences on Advances in Computer-HumanInteractions (ACHI ’ ), pp. – , IARIA, .

[ ] Android Design Guidelines, https://developer.android.com/de-sign/index.html .

[ ] iOS Human Inter ace Guidelines. Designing or iOS , ,https://developer.apple.com/library/ios/documentation/user-experience/conceptual/MobileHIG/index.html .

[ ] Smartphone Adoption and Usage, http://www.pewinternet.org// / /smartphone-adoption-and-usage/.

[ ] J. Nielsen, Mobile Usability Update, Nielsen’s Alertbox, ,http://www.nngroup.com/articles/mobile-usability-update/ .

[ ] M. Dunlop and S. Brewster, “Te challenge o mobile devicesor human computer interaction,” Personal and Ubiquitous

Computing , vol. , no. , pp. – , .[ ] D.Zhang andB. Adipat,“Challenges,methodologies,andissues

in the usability testing o mobile applications,” International Journal o Human-Computer Interaction, vol. , no. , pp. –

, .[ ] R. Looije, G. M. te Brake, and M. A. Neerincx, “Usability engi-

neering ormobile maps,” in Proceedings o the th International Con erence on Mobile echnology, Applications, and Systemsand the st International Symposium on Computer HumanInteraction in Mobile echnology (Mobility ’ ), pp. – ,ACM, New York, NY, USA, .

[ ] D. Zhang, “Web content adaptation or mobile handhelddevices,” Communications o the ACM , vol. , no. , pp. – ,

.[ ] R. Budiu and J. Nielsen, Usability o Mobile Websites: Design

Guidelines or Improving Access to Web-Based Content and Services Trough Mobile Devices, Nielsen Norman Group, .

[ ] A. S. siaousis and G. M. Giaglis, “An empirical assessment o environmental actors that in uence the usability o a mobilewebsite,” in Proceedings o the th International Con erence on


19/20


Mobile Business and th Global Mobility Roundtable (ICMB-GMR ’ ), pp. – , June .

[ ] H. B. Duh, G. C. B. an, and V. H. Chen, “Usability evaluationor mobile device: a comparison o laboratory and eld tests,”

in Proceedings o the th International Con erence on Human-Computer Interaction with Mobile Devices and Services (Mobile-HCI’ ), pp. – ,ACM, Helsinki,Finland, September .

[ ] S. Po, S. Howard, F. Vetere, and M. Skov, “Heuristic evaluationand mobile usability: bridging the realism gap,” in MobileHuman-Computer Interaction—MobileHCI , vol. o Lecture Notes in Computer Science, pp. – , Springer, Berlin,Germany, .

[ ] E. Bertini, S. Gabrielli, and S. Kimani, “Appropriating andassessingheuristics ormobile computing,” in Proceedingso theWorkingCon erence on Advanced Visual Inter aces (AVI ’ ), pp.

– , ACM, New York, NY, USA, May .[ ] Y. Cuiand V. Roto,“How peopleuse theweb onmobile devices,”

in Proceeding o the th International Con erence on WorldWideWeb (WWW ' ), pp. – , New York, NY, USA, April

.[ ] L. D. Catledge and J. E. Pitkow, “Characterizing browsing

strategies in the World-Wide web,” Computer Networks and ISDN Systems, vol. , no. , pp. – , .

[ ] C. W. Choo, B. Detlor, and D. urnbull, “A behavioral modelo in ormation seeking on the web,” in Proceedings o the ASIS Annual Meeting Contributed Paper , .

[ ] J. B. Morrison, P. Pirolli, and S. K. Card, “A axonomic analysiso what world wide web activities signi cantly impact people’sdecisions and actions,” in Proceeedings o the Con erence onHuman Factors in Computing Systems(CHIEA ’ ), pp. – ,ACM, New York, NY, USA, April .

[ ] A. J. Sellen, R. Murphy, and K. L. Shaw, “How knowledgeworkers use the web,” in Proceedings o the SIGCHI Con erenceon Human Factors in Computing Systems(CHI’ ), pp. – ,ACM, New York, NY, USA, .

[ ] B. MacKay, C. Watters, and J. Duffy, “Web page trans or-mation when switching devices,” in Mobile Human-Computer Interaction—MobileHCI : Proceedings o th International Symposium, MobileHCI, Glasgow, UK, September – , ., vol. , pp. – , .

[ ] M. Kellar, C. Watters, and M. Shepherd, “A goal-based classi -cation o web in ormation tasks,” Proceedings o the AmericanSociety or In ormation Science and echnology , vol. , pp. – ,

.[ ] E. Marcotte, “A list apart,” Responsive Web Design, ,

http://alistapart.com/article/responsive-web-design/ .[ ] B. Shneiderman and C. Plaisant, Designing the User Inter ace:

Strategies or Effective Human-Computer Interaction, Addison-

Wesley, Reading, Mass, USA, .[ ] D. Pierotti, “Heuristic evaluation—a system checklist,” ech.Rep., Xerox Corporation, Society or echnical Communica-tion, .

[ ] L. Constantine, Devilish Details: Best Practices in Web Design,Constantine and Lockwood, .

[ ] K. Instone, “Usability engineering or the web,” W C Journal,.

[ ] B. ognazzini, “First principles o interaction design,”Interaction Design Solutions or the Real World, ,http://www.asktog.com/ .

[ ] Y. Hassan Montero and F. J. Mart́ın Fernández,“Gúıa de Evaluacíon Heur ı́stica de sitios web,” ,http://www.nosolousabilidad.com/articulos/heuristica.htm .

[ ] Y. Hassan, M. Ferńandez, J. Francisco, and G. Iazza, DiseñoWeb Centrado en el Usuario: Usabilidad y Arquitectura de laIn ormacíon, vol. , , http://www.up .edu/hipertextnet/ .

[ ] L. Olsina, G. La uente, and G. Rossi, “Speci ying quality characteristics and attributes or websites,” in Web Engineering,Sofware Engineering and Web Application Development , S.Murugesan and Y. Deshpande, Eds., pp. – , Springer,London, UK, .

[ ] J. Nielsen, “Severity ratings or usability problems,” Nielsen’sAlertbox, ,http://www.nngroup.com/articles/how-to-rate-the-severity-o -usability-problems.

[ ] November , http://innovation.logica.com.es/web/procura .[ ] , http://www.researchspaces.eu .[ ] H. A. Linstone and M. uroff, Delphi Method: echniques and

Applications, Addison-Wesley, Reading, Mass, USA, .[ ] N. O. Bernsen, H. Dybkjær, and L. Dybkjær, “Wizard o Oz

prototyping: when and how?” Working Papers in CognitiveScience and HCI, .

[ ] J. Lazar, J. H. Feng, and H. Hochneiser, Research Methods inHuman-Computer Interaction , John Wiley & Sons, New York,NY, USA, .


20/20

Submit your manuscripts athttp://www.hindawi.com

Heuristic Evaluation on Mobile Interfaces

Documents

Transcript of Heuristic Evaluation on Mobile Interfaces