Best practice for working Best practice for working with human participantswith human participants

CS/IT Honours projects CS/IT Honours projects 20072007

Judy RobertsonJudy Robertson

Overview

Why should you care? Principles of working with human

participants About working with children and

other vulnerable users

Why work with human participants?

You’ll need to do this: If you’re doing a survey/ market research

type project (IT students) If you’re trying to evaluate software If you’re trying to design new software Feedback from users tells you if your

software is fit for purpose If it’s not, why spend so much time

building it?

Why the fuss?

Running a study with users requires careful planning

Analysis of user data is time consuming to do properly

There is more to this than “giving people a questionnaire”!

There are procedures you should follow to make sure you do it ethically

Ways to work with users Paper prototyping (simulating an interface

on paper) Log files Expert evaluation Observation – watching users at work Diaries – ask users to fill in regular notes

about using your software Focus groups (leading a group discussion) Interview (in depth, fewer users) Questionnaire (lots of users, broad

overview needed)

Tips Formative evaluation – finding out what users

think before software is finished. Exploratory – require depth not loads of users

Summative evaluation – Proving hypothesis about software usability when it is finished. Often requires lots of people

If you are trying to prove a hypothesis you need to go beyond graphs or averages. You’ll need statistical tests (e.g t-test)

Avoid the word “significant” in your analysis section unless you are using it technically to comment on the results of statistical tests.

Principle: Informed consentPrinciple: Informed consent

Participants should be told:Participants should be told:– The aim of the researchThe aim of the research– The process they will go throughThe process they will go through– Any risks/consequences involvedAny risks/consequences involved– Why their participation is necessaryWhy their participation is necessary– What information will be gatheredWhat information will be gathered– How and to whom will this information How and to whom will this information

be reportedbe reported

Consent from participantsConsent from participants

Participants:Participants: can choose to stop at any time can choose to stop at any time

without giving a reasonwithout giving a reason shouldn’t be made to take part under shouldn’t be made to take part under

duressduress are entitled to know how and why are entitled to know how and why

their data is stored, what it is used their data is stored, what it is used for, and who might see it.for, and who might see it.

Issue of deceptionIssue of deception

Researcher’s responsibility to Researcher’s responsibility to participantsparticipants

Researchers should minimise discomfort to Researchers should minimise discomfort to participants and stop if they show distressparticipants and stop if they show distress

Minimal intrusion – only ask questions which Minimal intrusion – only ask questions which are relevant to current research questionsare relevant to current research questions

Incentives for taking part only in exceptional Incentives for taking part only in exceptional cases and shouldn’t be harmful (ie. No cases and shouldn’t be harmful (ie. No sweeties!)sweeties!)

Data should be kept confidential, secure and Data should be kept confidential, secure and anonymous (See Data protection guidelines)anonymous (See Data protection guidelines)

Give participants feedback on the outcome Give participants feedback on the outcome of the researchof the research

Working with children or vulnerable Working with children or vulnerable usersusers

Conducting research with children is Conducting research with children is rewarding, but it requires careful rewarding, but it requires careful planning.planning.

We’ll look atWe’ll look at Why it’s important to follow procedures Why it’s important to follow procedures

for getting consent;for getting consent; How to gain the necessary permission to How to gain the necessary permission to

carry out your research with children;carry out your research with children; Tips for working with children;Tips for working with children;

Why is all this necessary?Why is all this necessary?

Following through the correct procedure can Following through the correct procedure can be time consuming and frustrating. But it’s be time consuming and frustrating. But it’s necessary because:necessary because:

It’s the law – Protection of Children It’s the law – Protection of Children (Scotland) Act 2003;(Scotland) Act 2003;

Children need protection – the welfare of Children need protection – the welfare of the child is always the first prioritythe child is always the first priority

Those working with children need Those working with children need protection.protection.

Permission to work with childrenPermission to work with children

Working with children in schools and other Working with children in schools and other organisations requires careful planning organisations requires careful planning and permission from various sources:and permission from various sources:

Organisation in which you are workingOrganisation in which you are working Local authority (in some cases)Local authority (in some cases) ParticipantsParticipants Parents/ guardiansParents/ guardians MACS (see Peter King)MACS (see Peter King)

Consent from child participantsConsent from child participants Follow the principle of Follow the principle of voluntary informedvoluntary informed

consent.consent. Children have a right to express their views in Children have a right to express their views in

matters concerning them AND they have a right matters concerning them AND they have a right to be listened to (UN Convention on the Rights of to be listened to (UN Convention on the Rights of The Child).The Child).

This information should be age appropriate. For This information should be age appropriate. For younger children, you may have to read the younger children, you may have to read the information out loud to them.information out loud to them.

Parents should also give their informed consentParents should also give their informed consent

Permission from organisationPermission from organisation

You should visit the head teacher of the You should visit the head teacher of the school (or head of organisation) in school (or head of organisation) in advance to plan the project. Ask about:advance to plan the project. Ask about:

Parental permission slipsParental permission slips Photographic/ video permissionPhotographic/ video permission Do you need to notify the local authority?Do you need to notify the local authority? Do you need to get a Disclosure Scotland Do you need to get a Disclosure Scotland

background check?background check? Will there be a member of staff with you at Will there be a member of staff with you at

all times?all times? Which classroom will be used?Which classroom will be used?

Permission from local authorityPermission from local authority Edinburgh City Council require a Edinburgh City Council require a

Disclosure Scotland background check;Disclosure Scotland background check; Disclosure Scotland is the organisation Disclosure Scotland is the organisation

which administers background checks for which administers background checks for people who work with children (voluntary people who work with children (voluntary and paid);and paid);

If you are working in another local If you are working in another local authority, check with the school to find out authority, check with the school to find out whether you need a Disclosure Check.whether you need a Disclosure Check.

PLEASE NOTE – Disclosure applications can PLEASE NOTE – Disclosure applications can take 6 weeks or more to be processed.take 6 weeks or more to be processed.

Tips for working with childrenTips for working with children

Do you have any advice based on Do you have any advice based on previous experiences?previous experiences?

Tips for working with childrenTips for working with children

Make sure there is a member of staff Make sure there is a member of staff from the organisation with you at all from the organisation with you at all timestimes

Keep a professional distance Keep a professional distance between you and the participantsbetween you and the participants

Be careful about physical contactBe careful about physical contact Be careful about jokesBe careful about jokes Dress appropriatelyDress appropriately

InterviewsInterviews Ask children if they would like to be Ask children if they would like to be

interviewed individually or with a friend;interviewed individually or with a friend; Ask the children if it is OK to audio record Ask the children if it is OK to audio record

them;them; Tell them it’s OK to ask you turn off the Tell them it’s OK to ask you turn off the

recorder during the interview;recorder during the interview; Suggestion – if the child is talking quietly, Suggestion – if the child is talking quietly,

paraphrase what they just said when they paraphrase what they just said when they finished finished – A) to check you understood themA) to check you understood them– B) So you can decipher what they said later on.B) So you can decipher what they said later on.

QuestionnairesQuestionnaires Make sure the vocabulary and sentence Make sure the vocabulary and sentence

structure is age appropriate;structure is age appropriate; You may need to read the questions aloud You may need to read the questions aloud

if the participants can’t read yet;if the participants can’t read yet; Make sure each isn’t confusing, contains a Make sure each isn’t confusing, contains a

single question and is unambiguous;single question and is unambiguous; Pupils should have enough space to fill in Pupils should have enough space to fill in

their answers in privacy;their answers in privacy; Pupils should be given a sealed envelope Pupils should be given a sealed envelope

to return their questionnaires so that the to return their questionnaires so that the teacher can’t see their answers.teacher can’t see their answers.

Focus groupsFocus groups Explain the “rules” of the focus group to begin Explain the “rules” of the focus group to begin

withwith– Everyone gets a sayEveryone gets a say– Listen when other people are talkingListen when other people are talking– Group members are asked to respect each otherGroup members are asked to respect each other– The researcher will keep the information confidentialThe researcher will keep the information confidential– Other groups members are asked to keep the Other groups members are asked to keep the

information confidentialinformation confidential Ask the group if it’s OK to record what is saidAsk the group if it’s OK to record what is said The tape will be turned off on requestThe tape will be turned off on request

What’s wrong here?

Including non-anonymised questionnaire forms in appendix

Asking people in a questionnaire if they download music illegally

Running a study to see if VR helps agoraphobics

Any questions?Any questions?

Further informationFurther information If you’re not taking Human Factors and Usability, If you’re not taking Human Factors and Usability,

you can sign up to get course materials at you can sign up to get course materials at http://advancedinteractiondesign.ning.comhttp://advancedinteractiondesign.ning.com

Ethics podcast: Ethics podcast: http://www.macs.hw.ac.uk/~judy/HF/Podcasts/EthicsPodcashttp://www.macs.hw.ac.uk/~judy/HF/Podcasts/EthicsPodcast.mp3t.mp3

Disclosure ScotlandDisclosure Scotlandhttp://www.disclosurescotland.co.uk/

Scottish Council for Research in Education ethics guidelinesScottish Council for Research in Education ethics guidelineshttp://www.sera.ac.uk/docs/SERA_Ethical_Guidelines_final.pdf

ACM ethics guidelinesACM ethics guidelines– http://www.acm.org/about/code-of-ethicshttp://www.acm.org/about/code-of-ethics

Ask me if you want example consent forms etc.Ask me if you want example consent forms etc.