Post on 20-Dec-2015
Case study - usability evaluationHowell Istance
SOFT3057 - Interactive Systems
ELISE II - Web- based Image Retrieval System
This allows users to enter search terms which relate to descriptions of images stored in image collections e.g. ‘sunflower’
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
Methods proposed for usability evaluation of ELISE II
Structured expert reviewing Use of cut-down version of cognitive walkthrough technique
Observation of users completing standard tasks Studied via a number of user trials
Extended in-situ testing Use of ELISE II in normal work during 1 month Participant keeps diary of system usage
SOFT3057 - Interactive Systems
User trials – pilot testing
Used 2 graphics designers Insufficient scope in the tasks used to test the advanced
search and the browse facilities Requires additional tasks which require these to be used
SOFT3057 - Interactive Systems
Participants in evaluation trials
Limerick 5 librarians, 1 student ULB 10 librarians DMU 2 graphics designers (pilot), 2 lecturers, 2 research
students V&A 1 researcher, 2 cataloguers, 2 curators Tilburg 1 librarian, 2 cataloguers, 1 manager
SOFT3057 - Interactive Systems
Procedures for user trials
introduction to the trial and its purpose initial interview to establish background and nature of
image use initial exploration phase formal explanation of the functions search for last image actually used Searches for 3 sample images post-trial interview
SOFT3057 - Interactive Systems
Macro Usability Issues
How well does the system support the user in formulating a query?
Are the retrieval results presented in such a way that enables the user to comprehend the relevance of the results to his search needs?
Is the user supported in making advanced queries? Can the image retrieval system be integrated into the
working environment of the user?
SOFT3057 - Interactive Systems
Search strategies
Most – began with restrictive search (several terms and boolean operators), removed terms successively of increase number of hits, only if no hits found then different generic term used
Few – began with generic term and added terms to reduce hits
Very few – used browse facility to check search term Many tried to minimise numbers of record retrieval in view
of long wait
SOFT3057 - Interactive Systems
Support in formulating query
Level of information about individual collections, type of images and search terms
Truncation not supported Language of collections not clear Order of precedence in Boolean expression evaluation
collections visible to user or one large virtual collection?
SOFT3057 - Interactive Systems
Assessing relevance of search results
Order of image retrieval unclear Detail of retrieved images Local copies of image records Decision to buy image records
SOFT3057 - Interactive Systems
SOFT3057 - Interactive Systems
Support for Advanced search
Understanding meaning of index fields Information about individual collections Complexity of search expressions
SOFT3057 - Interactive Systems
Integration with user’s normal working environment
Speed of image retrieval and system reliability Saving retrieved images General usefulness
SOFT3057 - Interactive Systems
Micro usability issues
Consistency Appropriate user support Minimal user input Reduce user memory load Feedback Flexibility Navigational support
SOFT3057 - Interactive Systems
Navigation
Mechanisms inconsistent For example..
No obvious means of returning to the simple search page from the search results page
Clicking on buttons containing the numbers of hits to view the record list is not obvious
Requires consistent mechanism to move query – search – view sequence,
Consistency in object used, in labelling, in location
SOFT3057 - Interactive Systems
System feedback
System messages are too small in size appear in a non-prominent part of the page language and content is often very obscure
e.g ERROR: HTTP return error – Broker not found=?HTTP/1.1 500 Internal Server error..
empty return – contact the administrator - meaningless to most users
SOFT3057 - Interactive Systems
Login Facility
Location of command not prominent Feedback about outcome (success or failure) is poor Dialogue box persists on-screen with no visible change
during login attempt User encouraged to quit rather than logging off
Have separate login page
SOFT3057 - Interactive Systems