Usability Testing2014 Highlights & Findings

Ecaterina Moraru — 26 July 2014 —

Why is good to do usabilitysessions?

You confirm issues you've suspected

Get amazed by new things you discover

“”

·

·

SCENARIOScenario #4

PERIOD27 May - 12 June 2014

VERSIONXE 6.0

SKINColibri

DURATION9:45:45 h

FEATURESProfile, Navigation, Pages, Wikis,Applications

Usability Sessions

3/44

Tasks

1 | First Impressions

2 | Profile Editing

3.1 | Wiki Navigation

3.2 | Page Editing

3.3 | Page Hierarchy

4 | Sending Message

6 | Welcome Editing

7.1 | Wiki Creation

7.2 | Application Install

5 | Conclusions

Tasks

5/44

Task 1

INSTRUCTIONS HYPOTHESES

What are your firstimpressions of XWiki?

What do you think you cando with it?

You can scroll, hover, butplease don't click onanything yet.

Introduction

Testing: Homepage

Status: Not-logged-in

Status: Logged-in

Task 1

·

·

·

·

7/44

Task 2

INSTRUCTIONS HYPOTHESES

Imagine this is your firstday at work and XWiki isused internally.

You want to customiseyour profile and add someinformation about yourself.How can you do that?

UCUC11 : Measure the performance of 'menu-

button' vs. 'icon-link' (#tmEdit vs.

.editProfileCategory)

9/9 'icon-link' usage

Testing: Profile

9/9 Success

Task 2

·

·

·

9/44

(video)

Add hints to Profile Preferences Specify default values

Task 2 - Profile Editing

Safari Power SaverClick to Start Flash Plug-in

· · 10/44

Task 3

INSTRUCTIONS SUBTASKS

You've been given the assignment to add somecontent (text+image) for the Marketing team.

Task 3.1

Task 3.2

Task 3.3

INSTRUCTIONS HYPOTHESES

First let's browse a bit theproduct and locate wherethe Marketing material isfound.

UCUC22 : Measure the performance of Search,

Search Suggest vs. Wiki Index

7/9 Search Suggest

2/9 Wiki Index

Testing: Navigation

3/9 couldn't find

Marketing without

help

Task 3

Task 3.1

·

·

·

·

12/44

(playlist)

No suggest for partial words

No Wiki results in Search

Pressing Enter too fast

3/9 Suggest worked in

first try

4/9 Suggest failed in

first try

Task 3.1 - Navigation - Search Suggest

Safari Power SaverClick to Start Flash Plug-in

·

·

·

·

·13/44

(video)

Harder to find under menu

No Wikis gadget on Homepage

No Applications panel entry

1/9 Used only Wiki Index

2/9 Found it alone after

Suggest fail

Task 3.1 - Navigation - Wiki Index

Safari Power SaverClick to Start Flash Plug-in

·

·

·

·

·

14/44

Task 3.2

INSTRUCTIONS HYPOTHESES

You need to create a pagecalled 'XWiki Enterprise' (inthe Marketing location).

As a result, the page shouldlook similar to:

UCUC33 : In order to edit, you need to be a

member of the wiki. Measure task

success.

UCUC44 : Measure 'Add image' dialog naming.

UCUC55 : Measure table performances.

Testing: Page Creation

Testing: Page Editing

Extended: wiki syntax

Extended: macro usage

Task 3.2

·

·

·

·

16/44

EXPECTED RESULT0/9 expected result

3/9 created Wiki

2/9 edited Sandbox

ACTUAL RESULTS

x2 x2

Task 3.2 - Page Creation

·

·

·

17/44

(video)

EXPECTED RESULT ACTUAL RESULTS

Task 3.2 - Page Creation

Safari Power SaverClick to Start Flash Plug-in

18/44

(video)

EXPECTED RESULT ACTUAL RESULTS

Task 3.2 - Page Creation

Safari Power SaverClick to Start Flash Plug-in

19/44

(video)

EXPECTED RESULT ACTUAL RESULTS

Task 3.2 - Page Creation

Safari Power SaverClick to Start Flash Plug-in

20/44

(video)

ACTUAL RESULTS ACTUAL RESULTS

Task 3.2 - Page Creation

Safari Power SaverClick to Start Flash Plug-in

21/44

Task 3.2 - Page Creation

Challenges:Finding the Add buttonCreating a Page and not something elseCreating the page in the right location

Issues:Problems finding the Add buttonMissing Add and Edit actions from IndexesDifficulties in finding Wiki Index and understand current locationConfusion between Page and Wiki conceptsDepending on the current location, having automatically Parent setMissing permissions by not completing the Join this wiki step

····

·······

22/44

(video)

4/9 used 'Join this wiki'

Confusing between Move - Copy -Rename - Export

Missing 'Add - Page' option

Missing Rights to move pages

Task 3.2 - Join this wiki

Safari Power SaverClick to Start Flash Plug-in

·

·

·

·

23/44

(playlist)

Preference for 'External image'

Save confirmation needed

Improvements: Single click, Pre-filled label, etc.

Task 3.2 - Page Editing

Safari Power SaverClick to Start Flash Plug-in

·

·

·

24/44

Task 3.3

INSTRUCTIONS HYPOTHESES

Create another page called'Products'.

Link the 2 pages one fromanother in order to be ableto navigate between them.

Is it possible also to makean hierarchy?

(where 'Products' page ison an upper level than'XWiki Enterprise' page)

UCUC66 : Measure discoverability for 'Page

Parent' functionality.

UCUC77 : Asses the clearness of hierarchy

display (breadcrumb, document

index tree, document information,

etc.).

Testing: Page Linking

7/9 WYSIWYG Page

Linking

Testing: Page Parent

1/9 Page Parent

Task 3.3

·

·

·

·

26/44

(video)

Missing visibility for Parent

Cross wikis issues

Tree is hidden and ambiguous

Task 3.3 - Page Parent

Safari Power SaverClick to Start Flash Plug-in

·

·

·

27/44

Task 4

INSTRUCTIONS HYPOTHESES

You would like to let themanager know that thecontent has been addedand the page is ready.

How would you do that?

UCUC88 : Evaluate alternative communication

methods.

8/9 Send Message

1/9 Send Page by mail

Testing: Send Message

Testing: Page

Comments

Testing: Send Page by e-

mail

Task 4

·

·

·

·

·

29/44

Task 6

INSTRUCTIONS HYPOTHESES

Go to the Home page andremove the first paragraphfrom the 'Welcome to yourwiki' section.

UCUC99 : As simple users the ability to edit

inline is not available. Since the

default editor is WYSIWYG, the user

will not be able to remove the

paragraph without removing the

whole Dashboard macro.

8/8 failed to edit the text

2/8 changed the title

2/8 deleted gadget

Testing: Dashboard

Testing: Inline Editing

Task 6

·

·

·

·

·

31/44

(playlist)Task 6 - Welcome Editing - Main

Safari Power SaverClick to Start Flash Plug-in

32/44

(playlist)Task 6 - Welcome Editing - Dashboard

Safari Power SaverClick to Start Flash Plug-in

33/44

Task 7.1

INSTRUCTIONS HYPOTHESES

You would like to create adedicated place forDevelopment material, justlike the one existing for theMarketing team.

Create such a location.

UCUC1010 : Evaluate the preference for wiki or

space solution.

3/9 Space instead of Wiki

UCUC1111 : Evaluate Distribution Wizard's

impact and friendliness.

6/9 Later

1/9 Never

2/9 Install

Testing: Wiki Creation

Testing: Space Creation

Testing: Distribution

Wizard

Prereq: X user has 'Add

Wiki' right

Task 7.1

·

·

·

·

·

·

·

·

35/44

(video)Task 7.1 - Wiki Creation

Safari Power SaverClick to Start Flash Plug-in

36/44

Task 7.2

INSTRUCTIONS HYPOTHESES

The Development teamcould use a Calendarapplication.

Find out how to install thisapplication on your wiki.

UCUC1212 : Evaluate multiple Calendar

applications naming ambiguity.

6/8 Full Calendar Macro

1/8 Calendar Panel

Application

1/8 Mocca Calendar

Application

Testing: Administration

Testing: Application

Panel

Testing: Extension

Manager

Task 7.2

·

·

·

·

·

·

38/44

(playlist)

Display only Applications

Add a Search button

Provide entries in Suggest

Task 7.2 - Application Install

·

·

·

39/44

Task 5

INSTRUCTIONS HYPOTHESES

Thinking about your overallexperience with XWiki,what did you like about theproduct?

What did you not like?

Would you use it in thefuture? Why or why not?

Conclusions

Task 5

·

41/44

Questions?

Thank you to all the participants and for your attention

Ecaterina Moraru — 26 July 2014 —