The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the...
-
Upload
raymond-chandler -
Category
Documents
-
view
215 -
download
0
Transcript of The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the...
![Page 1: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/1.jpg)
The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.
![Page 2: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/2.jpg)
Two types of evaluation:1. Summative Evaluation2. Formative Evaluation
Is evaluation necessary?
When do we need it?
Quantitative vs. Qualitative Data
![Page 3: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/3.jpg)
To increase the quality of research, we must avoid the following effects during the observational study:
1. Hawthorne effect2. Observer effect3. Halo effect
![Page 4: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/4.jpg)
Verbal Protocols
Designing Observations:◦ Writing a verbal protocol◦ How to conduct a session◦ Analyzing a protocol/transcript
![Page 5: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/5.jpg)
We ask users to speak out load and mention why they had done an action.
2 components:◦ Talk aloud: verbalize silent decision◦ Think aloud: verbalize whatever thoughts occur
during this task.
![Page 6: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/6.jpg)
Select:◦ Tasks◦ Users◦ Environment◦ One or two significant functional requirements
Observe (at least 3 users)
Give users their set of tasks to complete
Conduct a think-aloud study
Keep protocols (transcripts for each user)
Record users comments, etc. (p. 141)
![Page 7: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/7.jpg)
Description of the environment List of tasks completed by the user Users’ background & demographic details Record and write up users’ comments, body
language, facial expressions The aspects of the interface that these
responses relate to should also be detailed.
![Page 8: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/8.jpg)
Choose the tasks Select users – wrong users lead to
misleading information Explain the purpose to the users Conduct the evaluation – for example:
◦ What are you looking at now?◦ What just happened?◦ What are you going to do next? Why?
![Page 9: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/9.jpg)
The data that you will have at the end of a session is known as the transcript, which details the physical actions and verbal commentary that the user has made.
When analyzing a transcript of an evaluation session, the aim is to categorize the comments according to:◦ Frequency◦ Fundamentality
![Page 10: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/10.jpg)
Experiments define a hypothesis
2 stages of experiments:◦ Implementing the experiment◦ Analyzing the results
Advantages: systematic with a repeatable approach to testing based on scientific rigor.
Disadvantages: includes a reduced consideration of specific variables, questions which are hard t relate to real-world holistic problems and an artificial setting which may lack real-world validity.
In testing we strive for realism
![Page 11: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/11.jpg)
CW is an approach to formative evaluation without users.◦ Preparing a CW◦ Conducting a CW◦ Experiments in Support of Design◦ Dependent and Independent Variables◦ Assigning Subjects◦ Statistics◦ Summary of user experimentation
![Page 12: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/12.jpg)
Interviews◦ Structured interview◦ Unstructured interview
Questionnaires:◦ Open Q’s◦ Closed Q’s:
Simple checklist: Y/N or N/A Ranked order: (select your preference) Multi-point rating scale: strongly agree/disagree
![Page 13: The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f2c5503460f94c47753/html5/thumbnails/13.jpg)