203CR PORTFOLIO 2
NAME: Arunas Bedzinskas
STUDENT ID: 3790007
COURSE: BSc Computing
BLOG URL: http://arunasbedzinskas.wordpress.com/203cr-usability/
Portfolio 2
User – Centered Design of a Pervasive Interface
Introduction
User-centered design (UCD) is an approach to design that grounds the
process in information about the people who will use the product (Usability
Professionals’ Association, 2009). User – Centered Design concentrates on users
through planning, designing and producing the product. In this report I am writing
about the usability test carried out with a user – centered design of a pervasive
interface of house heating system.
Problem space
There are lots of different types of home heating systems. Some of them are
modern like using boilers of hot water, others are heated with electricity and some
people even heat their houses using wood or peat. I would like to talk about
heating with electricity or from boilers. They usually have screens to control the
heating, full of different buttons and gauges. Personally, I find them really hard to
understand (picture below). In the picture you can see a controller with up and
down, select, set and reset buttons. When I tried to use the device, arrows made
temperature go up or down, I did not understood why do I need select button and
set button did not respond to my press. Other people find them hard as well,
according to anita9 ―There is a thermostat downstairs and each room has a wall
vent with an individual fan control. When you turn on the fan the room warms up -
but even if you don't turn it on, a little bit of warmth comes out as long as the
thermostat is on. So do we save any heating energy by keeping some of the fans
off? Or are they just blowing heat that's being produced anyway? What is behind
the vents in the rooms?‖ (GardenWeb, 2011). Of course, technicians can help you
with that and solve the problem you have in few seconds. But I think people do not
really want to call a technician every time they feel like it is too warm or too cold,
or maybe they want to save some money by turning it off for a while. Doing it by
yourself is usually complicated because the control interface on the boiler is really
hard to understand. Although, users can just turn off the controller straight on the
radiator, but that just disables the heat coming into your radiator, the boiler still
keeps working. That’s why I decided to design some pervasive interface prototypes
of heating systems that every person could use.
Methodology
According to W3C (Web Accessibility Initiative) ―In UCD, all development
proceeds with the user as the center of focus. Rubin depicts the User-Centered
Design Process as follows: The users are in the center of a double circle. The inner
ring contains: Context; Objectives; Environment and Goals. The outer ring
contains: Task Detail; Task Content; Task Organization and Task Flow‖ (W3C,
2008). Goals of my research are to find out whether users would use this
technology and if it is simple enough to use it for a regular person. For that I
produced two prototypes of my interface – low – fi and mid - fi.
Prototypes
First prototype (low – fidelity) is for users getting introduced with the idea of
prototype, show some basic interactions and abilities of the interface. Moreover, it
is ―… fast way to mock up an interface — no coding required, finds a wide variety of
Figure 1 – Example of heating control in my house.
problems in an interface, including many of the serious ones, allows an interface to
be refined based on user feedback before implementation begins‖ (Paper
Prototyping, 2003). That is why I implemented it as a paper prototype.
Second prototype (mid –fidelity) is for fixing mistakes of the first prototype, getting
users to enjoy using this prototype, identifying further improvements.
Usability Methods
There was more than one concept that I had to think about when I was
running usability tests of my prototypes. I had to choose between measuring
Qualitative or Quantitative data. Although, ―Each of these approaches has strengths
and weaknesses, and each can benefit from our combining them with one another‖
(UX matters, 2013). So I decided to measure both, because it would provide results
from wider perspective in order to understand if prototypes are really easy to
understand and satisfying user needs. With quantitative data I was recording right /
wrong user paths and ask for help. This was crucial, because this device is
supposed to be easy to use. I had lots of qualitative data recorded though, because
I wanted to sense if the users are happy with results (satisfaction), prototype
learnability, which parts are confusing or not obvious.
Measuring Data
Data was measured in two ways. Users after tests were provided with a
questionnaire and I recorded data in data sheet as well. Questionnaires were used
to detect satisfaction of the users, opinion about the prototype and suggestions for
improvement of the device.
In the data sheets that I had, I collected information such as detecting ease of use,
user comments, parts where user was confused, frustrated, happy or behaved in
any other way.
Figure 2 – Blank Questionnaire
Both questionnaire and data collection sheets that were filled up by users are based
in appendixes section.
Conceptual Frameworks
There are various conceptual frameworks that designers can think that they
are important or not. But even for people who do not have any specific physical or
mental characteristics that affect computer use, it has been found that adopting
universal design principles can reduce fatigue, increase speed, decrease errors, and
decrease learning time for all users. In many ways, universal design addresses the
larger issues of usability by making things easier for everyone (Usability First,
2013). I think some of them are very important for a pervasive interface. One of
key principles is flexibility or in other words – solution path redundancy. As I came
up with some heating devices problems – they are tricky to use, usually you do not
Figure 3 – Blank Data Capture Sheet
know what to press to do something you need or if you press something wrong –
how to go back. That is why I tried to make as much different ways to complete the
tasks for the users and in this way make the prototype as much flexible as possible.
Another principle that is very important to me is being simple and intuitive. Even
though I tried to make device as flexible as possible, it is always better to get
everything right from the first time. So I simplified the navigations and functions as
much as possible to ensure this is much easier way to set up heating in the house.
Users
To test my prototypes, obviously I needed potential users. Users that participated
in this test were male Coventry university students and staff. I decided that it is
enough for me to test each prototype with five users, because, according to Jeff
Sauro, ―The five user number comes from the number of users you would need to
detect approximately 85% of the problems in an interface, given that the
probability a user would encounter a problem is about 31%‖ (Measuring Usability,
2010). One of important things of the test is detecting problems of an interface and
trying to fix them in the second prototype, so five users should be enough.
Usability Test Procedure
Tests were carried out in Coventry University. Participated users were provided with
consent form, test information sheet. After the test was finished a questionnaire
was provided for getting feedback about the prototype. Moreover, a prototype was
provided for conducting the test. Myself I had a data capture sheet, to record
particular information about test procedure and contestants. During first test with
paper prototype I was also showing screens and interactions when users ―presses‖
buttons and links in the prototype. During second test I did not had to show
screens and interactions, because Mid – Fidelity prototype was done using Microsoft
Power Point software. After second tests users were provided some extra time just
to try out prototype by themselves, just to further understand the idea and have
fun with this interface. Consent forms, data capture sheets and questionnaires are
provided in the appendices section of this document.
Usability Test Tasks
First, I thought this user – centered design will be for controlling house heating.
However, later on I decided to improve functionality to air conditioning and water
temperature control as well. In this case, I had to make two tasks of the test,
because one test would test only little functionalities of this device and I would not
have enough data to make this pervasive interface as much easily usable as
possible. Even though, both prototypes had same tasks during the tests in order to
check if the improvements of the first prototype have increased usability of
pervasive interface. Both tasks began in the main screen (after first task user has
to start from the main screen again). First task was to set a kitchen heating to
twenty three degrees Celsius for next two hours. This will help me to find out if
users find simple to set regular heating for short amount of time. Second task was
to set living room air conditioning to eighteen degrees Celsius for every week day
from four pm to ten pm. The reason of this task is to find out, if it is simple enough
for users to complete harder tasks, identify possible issues.
UCD Process
Initial User Study
As I lived in my parent’s house I did not have to worry about house heating.
But since I moved to study to Coventry University, I started to live in rented house
sharing it with other people. And last December, my heater stopped working. My
roommates and I tried many things to make it work playing with handles and
pressures and other things that appeared on the boiler. Even though when we
called a technician, it occurred that we wouldn’t able to fix it even if we knew what
handle or button does what. And that is how I came up with creating new interface
for house heating. Moreover, I talked with my neighbors as well about this
situation, and they answered that a technician set everything for their house
heating and they try not to touch it because it is hard to understand what does
what. So I understood that I am not alone with this problem.
Requirements Gathering (Low – Fi Prototype)
In requirements gathering, user experience researchers gather information to
lay groundwork for additional research, inform design, direct development, and
provide test goals (TECED, 2012). The main purposes of my work were to create an
easily usable, modern interface of the house heating system. Users need their
heating system very simplified. They should be able to change temperature of their
house and get used to the interface without any big problems occurring in their
way. Also, the interface should be modern, because for younger people new
modern interfaces are easier to understand than the old ones. Moreover, it is easier
to fix problems occurring with the interface, if it is created using modern
technologies.
Interactive Paper Prototype
With prototyping, the key is to create an interface quickly. To test out a
completely new design creating a paper prototype is the fastest way to go (User
Interface Engineering, 2000). Idea that instantly came up to my head was the
simple modern prototype with touch screen device.
I decided to create interface with three sections – two small horizontal sections one
on the top and one on the bottom of screen where you could switch between rooms
and what you want to control (heating, water, air conditioner).
Figure 4 – Paper Prototype - the beginning or start
screen – this has lock unlock and other direction
buttons. Date and time is shown as well.
The middle section has all the content - amount of time, temperature to set up. I
also decided that a home screen is needed – to make device look better and
availability to lock and unlock it. After that, I came up that I also need a button or a
separate link if users want to set various times for heating and temperature during
the week period for saving purposes (for example if all people are not at home
during that time, temperature falls down to save money and comes up again in
around time when people come back to home).
Figure 5 – Paper Prototype - screen to set up regular heating in the
living room. Pressing the box with temperature will edit it; pressing
amount of time drop down box will let you choose from one of the
default values from a drop down menu. Run and stop buttons to
start and stop the selected heating. Set up varied heating button
leads user to new window to set up weekly heating.
For entering values of temperature I decided to add additional key pad and for
entering amount of time – drop down boxes.
Figure 6 – Paper Prototype - screen to set up weekly conditioning
and heating. Table: first row – week days to set up; second row –
heating/conditioning start time; third row – heating/conditioning end
time; fourth row – temperature of the room. Buttons: add another
timetable – brings up another timetable to add further times for
varied heating/conditioning; start – starts the selected program;
repeat all days – sets up all week days same values as the first
column of the table.
User Evaluation (Low – Fi Prototype)
The test of paper prototype with users showed me that the interface is not as
usable as I expected it to be. I noticed many things that I need to fix in order to
improve user satisfaction and usability of the prototype. Main problems:
1. Lock device button. Tests showed that all of the participants did not notice
the unlock button in the corner of the device.
2. Varied Conditioning. All users were struggling when they had to set
conditioning in weekly basis.
3. Navigation. Few users were unhappy about navigation in the interface (no
back button, etc.).
4. Help. One user told that it would be very good to have help somewhere in
the interface.
Problems of interface affected satisfaction result as well. Some people even seemed
disappointed with the certain parts of the interface. Even though overall result of
first task was ―happy‖, the second task satisfaction was worse than expected – two
out of five was ―neither‖ and one was ―disappointed‖.
Figure 7 – Paper Prototype - same heating screen just with the
numbers keypad on the bottom to modify the temperature value.
Requirement Gathering (Mid – Fi Prototype)
In order to make much better both satisfactory and effectiveness results, I
had to fix as much problems in the mid – fi prototype as possible. Lock device
button was changed to an open source lock icon which should be easier noticeable
(problem one).
Varied conditioning button was changed to weekly conditioning (problem two).
Navigation problems were solved by adding home and back buttons for certain
windows (problem three).
Figure 8 – PowerPoint Prototype -
Beginning or start screen. Locked device.
Figure 9 – PowerPoint Prototype - screen
where you can set up regular kitchen
heating.
Figure 10 – PowerPoint Prototype - screens
which appear when the set up weekly air
conditioning button is pushed.
Help Icon was added to almost every page and a separate help page was created
for every one of them (problem four).
Interactive Digital Prototype
I decided to create my mid – fi prototype using Microsoft PowerPoint 2007.
Even though it is not a professional prototype creation tool, it is quite good to
create interactive and simple prototypes. I decided to use same design as I used for
paper prototype except I fixed the issues that occurred during the user test (you
can see them above). The font I chose was grey with some highlight of light grey
for pressed buttons and chosen pages.
User Evaluation (Mid – Fi Prototype)
The results of user tests were much better this time. Overall satisfaction
result of the mid – fi prototype was rated ―happy‖ from all users. Moreover, the
overall wrong path number during the tests decreased from one point eight in first
prototype first task to zero point six in second prototype and from one point eight
in first prototype second task to one point six in second prototype which is great
progress. Test showed that with some fixed mistakes in from first prototype made
the second prototype not only more effective, but much more satisfactory as well.
Requirements Gathering
Even though the results of the second interface tests were promising, there
are still some features that could be improved or changed. Few users were
confused about second task part, where they had to set times for weekly air
conditioning. They were not sure whether to write them according to twenty four
hour clock or twelve hour clock. This possibly happened because in the task sheet
the requirements for task completion were displayed using twelve hour clock and
device is using twenty four hour clock. So the task description may was misleading
and that lead to more wrong paths during second task. Although, people that live in
Great Britain are usually using twelve hour clock, so this might affected the test
results as well. Moreover, most users still made mistakes when they had to set up
weekly heating. More fixes might be needed with direction to weekly heating page.
Future Work
I think the most logical thing to do for the future would be developing the
interface up to the high – fidelity prototype. This could be done using some tools to
create an app for a touch screen device and test it with the users. Also, this
interface still has some things that need updating and fixing, so user satisfaction
and efficiency result with the device would be even higher. Essential fixes should
include updating weekly heating, so users would not get confused when they want
to set it up, possibility to decide whether to use twenty four hour or twelve hour
clock. Furthermore, a deeper user case study must be made to make sure people
need this type of technology and they would use it.
Total words: 3019
References
GardenWeb (2011) Don’t understand how my heating system works [online]
available from
<http://ths.gardenweb.com/forums/load/hvac/msg1114503821697.html> [13
March, 2013]
Measuring Usability (2010) Why you only need to test with five users? [online]
available from <http://www.measuringusability.com/five-users.php> [03 March,
2013]
Paper Prototyping (2003) What is Paper Prototyping? [online] available from
<http://www.paperprototyping.com/what_prosandcons.html> [02 March, 2013]
TECED (2012) Requirements Gathering [online] available from
<http://teced.com/services/user-interface-design/requirements-gathering/> [07
March, 2013]
Usability First (2013) Principles of Accessible and Universal Design[online] available
from <http://www.usabilityfirst.com/about-usability/accessibility/principles-of-
accessible-and-universal-design/> [02 March, 2013]
Usability Professionals’ Association (2009) What is User – Centered Design? [online]
available from
<http://www.usabilityprofessionals.org/usability_resources/about_usability/what_is
_ucd.html> [02 March, 2013]
User Interface Engineering (2000) Five Paper Prototyping Tips [online] available
from <http://www.uie.com/articles/prototyping_tips/> [07 March, 2013]
UX matters (2013) Strengths and Weaknesses of Quantitative and Qualitative
Research [online] available from
<http://www.uxmatters.com/mt/archives/2012/09/strengths-and-weaknesses-of-
quantitative-and-qualitative-research.php> [02 March, 2013]
Web Accessibility Initiative (2008) Notes on User Centered Design Process [online]
available from <http://www.w3.org/WAI/redesign/ucd> [02 March, 2013]
Appendices
Verifiable User Data – Consent forms
Questionnaires
Low – Fidelity
Mid – Fidelity
Verifiable User Data – Photography of User Participating in Test
The Low – Fidelity Interactive Paper Prototype
Same main screen as shown above. Extra button shows up only if the user
tries to press one of the three main buttons without unlocking the device.
Same heating screen just with a drop down menu to modify amount of time
value.
Same heating screen just with an extra pop up box which shows that the
heating phase is already started, it also shows how much time left till it ends
and what temperature is set.
Same heating screen just with the kitchen selected. Used for solution path
redundancy and wrong path availability.
Air conditioning screen with living room selected. Additional buttons – on –
off button, Fan speed – sets the fan speed so the room cools faster.
Same page as shown above, with additional changes that was used for task
two of the interface test.
Same page as shown above, with additional number keypad, that was used
to enter values for the timetable.
Same page as shown above. With additional message that same values are
applied to all days as the ones entered in the first column.
Same air conditioning page with additional box that shows the current
program that is set up.
Same air conditioning page just for the kitchen. Used for solution path
redundancy.
Screen to set up water temperature in the kitchen (tap). Temperature box to
set temperature and start button to start the program.
Screen where you can select water temperature for living room – null path,
not implemented.
Screen where you can add additional rooms for the device, not used for the
test.
Mid – Fidelity Prototype
NOTE: Only screens, which were used during tests are shown, because it consists
94 different screens.
Screen that shows up if user tries to use direction buttons if the device is not
unlocked.
Screen that appears when user unlocks the device.
Screen to select a temperature for a regular kitchen heating program.
Screen to select amount of time for the kitchen heating program.
Screen, that appears when the start button is pushed, shows the current kitchen
heating program.
Screen to set up regular air conditioning for living room.
Screen to set up temperature for regular kitchen air conditioning for living room.
Same screen as shown above has additional keypad in the bottom of the screen to
set up different values in the table. Appears when the table box is chosen.
Same screen as shown above except it has all filled boxes in the timetable. This can
be made by entering values in the MON column and pressing repeat all days button.
Screen that appears when the varied air conditioning program starts.
Screen to set up water temperature for kitchen. Used just for displaying
functionality of the prototype.
Same screen as shown above, additional box is displayed which shows that current
program of water program in the kitchen.
Null path; showed when user presses set up room link in the screens above.
Help screen. Most of the screens in the prototype have their unique help screen to
increase functionality and efficiency of the prototype.
Data Capture Forms
PERFECT PATH (Low - Fi)
Quantitative data:
Number of touch screen presses (path ½) 12 18
Number of wrong presses, wrong paths - -
Qualitative data:
Confusion -
Happiness -
Ask for help -
Verbal comments -
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: None
PERFECT PATH (Mid - Fi)
Quantitative data:
Number of touch screen presses (path ½) 7 21
Number of wrong presses, wrong paths - -
Qualitative data:
Confusion -
Happiness -
Ask for help -
Verbal comments -
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: None
USER 1
Quantitative data:
Number of touch screen presses (path ½) 20 25
Number of wrong presses, wrong paths 5 2
Qualitative data:
Confusion
Yes, Did not notice the room choices first, changed room
after the information completed in different room,
so had to repeat it in the correct room as well. Did not notice that device should be unlocked before doing actual
task.
Took some time to find the varied heating button
Happiness Quite satisfied with results Seemed unhappy with the
varied heating path.
Ask for help - -
Verbal comments “Oh well” “Dont know”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: Overall, user made mistakes, but was quite happy with the idea of device.
USER 2
Quantitative data:
Number of touch screen presses (path ½) 14 21
Number of wrong presses, wrong paths 2 1
Qualitative data:
Confusion
While entering details in the wrong room about the heating
realised that he is in the wrong room.
Took to think how to find the weekly air conditioning
settings, even though did not make any incorrect key
presses.
Happiness Looked quite happy about the prototype and tasks.
Ask for help - -
Verbal comments “Oh I see”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: This user looked really satisfied with the device and the tasks he got and completed successfully.
USER 3
Quantitative data:
Number of touch screen presses (path ½) 13 24
Number of wrong presses, wrong paths 1 4
Qualitative data:
Confusion Did not notice that unlocking screen is needed.
Got confused with setting air conditioning for weekly basis. Started entering values in main screen until tried varied heating button.
Happiness Happy Frustrated while unsuccessful.
Ask for help - -
Verbal comments - “What should I do” “I’m going to fail”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: User was confident what he was doing until the part with air conditioning, changes needed!
USER 4
Quantitative data:
Number of touch screen presses (path ½) 12 19
Number of wrong presses, wrong paths 0 1
Qualitative data:
Confusion Was not happy with the value entering, needs highlighting.
Took time to notice varied conditioning, started entering regular conditioning before noticing the varied one.
Happiness Looked very frustrated during all test.
Ask for help 1 1
Verbal comments “How do I know if I entered correct value”
“Am I supposed to enter information for every day or there is a quicker way”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
X
Other: Very critical user, lots of suggestions for improvement.
USER 5
Quantitative data:
Number of touch screen presses (path ½) 17 19
Number of wrong presses, wrong paths 1 1
Qualitative data:
Confusion Unlocking button Did not notice at first that you
can select a room for that.
Happiness Looked happy and satisfied during all this test.
Ask for help - -
Verbal comments “Great idea”, “I like this”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: Took time to notice varied conditioning.
MID – FI
USER 1
Quantitative data:
Number of touch screen presses (path ½) 13 24
Number of wrong presses, wrong paths 1 0
Qualitative data:
Confusion Not satisfied with 24 hour clock
Happiness Happy, familiar with interface
Ask for help - -
Verbal comments “Oh, I remember that”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: User looked satisfied with prototype capabilities.
USER 2
Quantitative data:
Number of touch screen presses (path ½) 10 27
Number of wrong presses, wrong paths 0 2
Qualitative data:
Confusion Get stuck when you have to choose weekly conditioning, seem
lost.
Happiness Did not show any with facial expression
Ask for help - -
Verbal comments None
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: User got into prototype faster than using the low – fi.
USER 3
Quantitative data:
Number of touch screen presses (path ½) 11 33
Number of wrong presses, wrong paths 1 3
Qualitative data:
Confusion Confused with air conditioning, struggled a while till tried
weekly conditioning button.
Happiness Happy, except was frustrated during the part with weekly air
conditioning mentioned above.
Ask for help - -
Verbal comments “Hope I won’t crash it”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: Looked more self confident, but made a lot mistakes.
USER 4
Quantitative data:
Number of touch screen presses (path ½) 11 25
Number of wrong presses, wrong paths 0 1
Qualitative data:
Confusion -
Happiness -
Ask for help - Not realised that it should be
24 hour clock.
Verbal comments “It is good”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: User seemed much more satisfied with second prototype comparing to the first one.
USER 5
Quantitative data:
Number of touch screen presses (path ½) 9 21
Number of wrong presses, wrong paths 1 1
Qualitative data:
Confusion Same mistake as in low – fi prototype – entered temperature in
regular air conditioning screen and after that chose weekly conditioning button.
Happiness Seemed satisfied with the device.
Ask for help - -
Verbal comments “It is good”
Satisfaction:
User looked
Very disappointed
Disappointed Did not show
with facial expression
Satisfied Very satisfied
x
Other: Might need fixes with the weekly conditioning button, because, despite the changes, users still
get confused.
Top Related