Geoportal Usability Evaluation

Xin HE, Hans Persson, Anders Östman

FPX, IHT, HIG

Sweden

Geoportal Usability Evaluation

Xin HE, Hans Persson, Anders Östman

FPX, IHT, HIG

Sweden

GEOTEST

2

GeoTest

• A project jointly initiated by

– National Land Survey of Sweden

– University of Gävle

– Future Position X

• A GIS cluster organization in Gävle, Sweden

3

[GeoTest] Aims

• Measuring Swedish SDI

• Contributing the implementation of INSPIRE in Sweden

4

[GeoTest] Scopes

• Performance

• Usability

• Feasibility

• (Cost & Benefit)

• (Data quality)

5

GEOTEST USABILITY EVALUATION

6

GeoTest Usability Evaluation

• ISO standards

• Institute for Humane Technology, Sweden

• FPX GeoUsability Lab

• Established network of test panels

7

[GeoTest] Applying ISO Standards

• Evaluate usability of GI services by applying standard

frameworks

– ISO 9241-11: Guidance on Usability

– ISO 9241-210: Human-centered design for interactive

system

– ISO 20282-3: Test method for measuring usability

8

GEOPORTAL

9

Geoportal

• An access point to different sources of GI on the web

• Many Member States provide access to their

infrastructures through geoportals

10

Geoportal Examples

11

USABILITY

12

Usability

• “Extent to which a product can be used by specified

users to achieve specified goals with effectiveness,

efficiency and satisfaction in a specified context of

use.”

• (ISO 9241-11:1998, definition 3.1)

13

[Usability] Effectiveness

• “the accuracy and completeness with which users

achieve specified goals”

• (ISO 9241-11:1998, definition 3.2)

14

[Usability] Efficiency

• “the resources expended in relation to the accuracy

and completeness with which users achieve goals”

• (ISO 9241-11:1998, definition 3.3)

15

[Usability] Satisfaction

• “freedom from discomfort, and positive attitudes

towards the use of the product”

• (ISO 9241-11:1998, definition 3.4)

16

USABILITY EVALUATION

17

Usability Framework (ISO 9241-11)

18

Evaluating Usability

• Understanding the goal for which the tool is designed

• Context of use in test

– Test panel

– Test tasks

• Usability measures

19

Usability Measures

• Effectiveness

– Level of completion

– Errors

• Efficiency

– Time

– (Cognitive workload)

• Satisfaction

– Psychometric questionnaire, e.g. SUMI and SUS

20

USABILITY EVALUATION OF A SWEDISH

GEOPORTAL

21

Geodata.se

22

Test Panel

23

Test Tasks

24

Usability Measures

25

SUMI

• Software usability measurement inventory (Kirakowski

and Corbett, 1993)

26

GeoUsability Lab

• 13.3 inches laptops

• Windows-based

• IE 8 installed

27

GeoUsability Lab

• Recording

– Task completion

– Time spent

– Keyboard strokes

– Mouse movements and

clicks

– Facial and oral expressions

28

[Results] Effectiveness and Efficiency

29

[Results] User Satisfaction

30

[Results] Design weakness

31

[Results] Differences in age groups

32

CONCLUSIONS

33

Conclusions from the pilot study

• Usability consideration should be a part of design

• Usability test detects design weakness

• Furthermore, it helps in understanding

– The activity

– Users

• It works for GI applications such as geoportals

34

Future works

• More tests

• Smarter method

– Test panel (Public, expert users, user experts?)

– Psychometric measurement

– Statistical inference

• Context of using geodata

35

Conclusions

• Feedbacks

• Design weakness detection

• Benchmarking

• Best practice and specification

36

Any questions?Any questions?