Usability Study of the Apply Texas Application including Interviews with High School Counselors
description
Transcript of Usability Study of the Apply Texas Application including Interviews with High School Counselors
![Page 1: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/1.jpg)
Usability Study of theApply TexasApplication
including Interviews with High School Counselors
Sandra Martinez, Ph.D.Apply Texas Team
![Page 2: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/2.jpg)
Performed Spring 2010 by Garrett Stettler as his Master’s project for UT School of Information
Garrett observed, interviewed, and videotaped 15 participants
He interviewed 5 high school counselors
Usability Study
![Page 3: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/3.jpg)
Getting a third party to perform a usability study is more effective than using developers because:◦ They have less personal involvement in the
website, which produces less emotional response.
◦ They are not biased by over-familiarity with the website
◦ They have different expectations than developers
Usability Study – the 3rd party
![Page 4: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/4.jpg)
They evaluate a product by testing it on actual users.
They discover errors and areas for improvement by observing people using the product.
They measure efficiency, accuracy, recall, and emotional responses of test subjects.
How are usability studies helpful?
![Page 5: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/5.jpg)
The Apply Texas Committee, in authorizing the study, charged the Apply Texas team with four goals:◦ Fix simple errors
◦ Fix big, urgent problems
◦ Create list of usability issues for the Apply Texas committee to prioritize
◦ Create list of suggested changes for the Apply Texas committee
How the study will be used
![Page 6: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/6.jpg)
8 postsecondary-bound Texas high school students
2 postsecondary-bound high school students from other states
2 current college students (transfer)
1 prospective graduate student
1 “non-traditional” adult returning to school
1 pilot participant
Participants
![Page 7: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/7.jpg)
1) Navigation 2) Copy process 3) Help 4) Confusion and frustration 5) Accuracy and completion
Areas of research for Apply Texas usability study
![Page 8: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/8.jpg)
Tasks by Group Group A (“new”
users)1. Create account, input
profile information
2. Start and complete a new application
3. Enter essay information
4. Enter scholarship information
5. Logout
Group B (“returning” users)
1. Retrieve forgotten password
2. View/edit a saved profile
3. View/edit a saved application
4. View/edit saved essays
5. Copy an existing application
![Page 9: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/9.jpg)
Critical concerns
Major concerns
Moderate concerns
Minor concerns
Levels of Concern
![Page 10: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/10.jpg)
Description of critical concerns:
Critical data may be lost, or
The user may not be able to complete the task, or
The user may not want to continue using the application
Critical concerns (None found)
![Page 11: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/11.jpg)
Description of major concerns: Users can accomplish the task but only with
considerable frustration and/or unnecessary steps Non-critical data may be lost The user will have great difficulty in circumventing
the problem Users can overcome the problem only after they
have been shown how to perform the task
Major concerns (2 found)
![Page 12: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/12.jpg)
Error handling
◦ Error messages are grouped at the top of the page.
◦ Errors are not highlighted within the page itself
◦ Participants tended to skim over error messages and used a “trial-and-error” approach to correcting the error
Major concerns (1 of 2)
![Page 13: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/13.jpg)
Major concerns (1 of 2)
![Page 14: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/14.jpg)
Error handling recommendations
◦ Errors should be highlighted at the top of the page and at the location in the page where the error has occurred.
◦ If possible, identify the error as the user is typing
Major concerns (1 of 2)
![Page 15: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/15.jpg)
Home / My Account / Logout
◦ Navigating to the main account page was confusing
◦ Attempting to log out could be confusing
Major concerns (2 of 2)
![Page 16: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/16.jpg)
Major concerns (2 of 2)Current view of top toolbar
Currently links to logout page
Currently links to logout page
Currently links to application listing page
Currently links to help page
![Page 17: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/17.jpg)
Major concerns (2 of 2)
For new cycle, links to logout page
For new cycle, links to application listing page
Links to application listing page (no change)
For new cycle, name changed to Logoff from Home/Logoff
Changes for 2011-2012 cycle
![Page 18: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/18.jpg)
Description of moderate concerns:
The user will be able to complete the task in most cases but will undertake some moderate effort in getting around the problem
The user may need to investigate several links or pathways through the system to determine which option will allow them to accomplish the intended task
Users will most likely remember how to perform the task on subsequent encounters with the system
Moderate concerns (5 found)
![Page 19: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/19.jpg)
Helper text and other indicators◦ Every participant failed to notice help text at
some point during the test.
Moderate concerns (1 of 5)
Helper Text
![Page 20: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/20.jpg)
Moderate concerns (1 of 5)
Color of helper text changed from gray to blue
![Page 21: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/21.jpg)
Buttons or important information appear ‘below the fold’
Some participants had difficulty completing a task if the continue button required scrolling
Moderate concerns (2 of 5)
![Page 22: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/22.jpg)
Moderate concerns (2 of 5)
Continue button off the screen
![Page 23: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/23.jpg)
Moderate concerns (2 of 5)
For new cycle, continue button moved to top of page
![Page 24: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/24.jpg)
Confusing terminology on essay page
Participants were unsure about clicking ‘Save’ if they had not entered an essay.
Some participants were unsure about the difference between save and submit.
Moderate concerns (3 of 5)
![Page 25: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/25.jpg)
Moderate concerns (3 of 5)
Use of the term “save” before anything is entered
![Page 26: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/26.jpg)
Moderate concerns (3 of 5)
Changed to Enter Topic A
![Page 27: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/27.jpg)
Moderate concerns (3 of 5)
Changed to Edit after first save
![Page 28: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/28.jpg)
Difficulty using Perform High School/College Search function
◦ Over one-third of participants clicked the “perform search” button before entering any text
◦ Two participants failed to select the radio button
◦ Several participants did not initially use the search feature
Moderate concerns (4 of 5)
![Page 29: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/29.jpg)
Moderate concerns (4 of 5)
![Page 30: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/30.jpg)
The Apply Texas team recommends reworking this page as follows:
1. Make the “perform high school name search” button more prominent
2. Ask applicants to enter the name of the school in a new window when the button is clicked
3. When the applicant chooses a radio button, automatically close the window and fill in information on the application
Moderate concerns (4 or 5)
![Page 31: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/31.jpg)
Top Menu and Quick Links
◦ Some participants were confused about the Menu bar and the Quick Links menu, and how they differed
◦ Participants generally did not use the navigation bar at the top for moving between Admissions/Scholarships/Essays/Submit
Moderate concerns (5 of 5)
![Page 32: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/32.jpg)
Moderate concerns (5 of 5)
![Page 33: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/33.jpg)
The Apply Texas team recommends reworking the navigation and links:
◦ Make the navigation links as a side navbar
◦ Add quick links to the top menu bar with a “mouseover” dropdown.
Moderate concerns (5 of 5)
![Page 34: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/34.jpg)
Moderate concerns (5 of 5)
Example of a page with a side navigation bar
![Page 35: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/35.jpg)
Description of minor concerns:
The concern is an irritant for the user
There is a cosmetic problem
There is a typographical error
Minor concerns ( 4 found)
![Page 36: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/36.jpg)
Confusing/inconsistent terminology
◦ 1) Applicants were unsure what was being asked for “Mini or short” semester
Minor concerns (1 of 4)
![Page 37: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/37.jpg)
Confusing/inconsistent terminology◦ 2) Participants were unsure about the difference
between ‘community and volunteer service’ and ‘extracurricular activities’
◦ 3) Under extracurricular activities, participants were unclear what ‘activity level’ meant
◦ 4) One participant was unclear whether ‘household size’ included only children, or all persons in household.
Minor concerns (1 of 4)
![Page 38: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/38.jpg)
Minor concerns (1 of 4)For mini/short semester, the following paragraph was added for the new cycle as clarification:
![Page 39: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/39.jpg)
Constraining employment form
◦ Two participants felt the employment form would be difficult to use, especially for applicants who have recurring summer jobs.
Minor concerns (2 of 4)
![Page 40: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/40.jpg)
Custom Questions◦ Many participants found the custom question
page to be irritating, particularly for institutions with many custom questions.
◦ The page did not cause confusion, just user irritation as it requires scrolling down the page for each question in order to click the ‘save, acknowledge question button.’
Minor concerns (3 of 4)
![Page 41: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/41.jpg)
Minor concerns (3 of 4)
![Page 42: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/42.jpg)
Suggestion: Schools with many custom questions may want to consider moving some of their questions to other modules instead of grouping them all on the custom question page.
Minor concerns – 3 of 4
![Page 43: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/43.jpg)
Yes/No/Reset Answer
◦ Several participants were confused by the Yes/No/Reset Answer response.
Minor concerns – 4 of 4
One suggestion would be to change the button labels from Reset Answer – Not Applicable to No Answer – Not Applicable
![Page 44: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/44.jpg)
Overall, Apply Texas is a fairly usable product, with no critical concerns
Many improvements to reduce frustration could be done fairly easily
Some improvements will require reworking certain modules or processes
And now, let’s hear what the high school counselors had to say --
Usability Study
![Page 45: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/45.jpg)
Suggestions for improving Apply Texas:◦ Give applicants a warning that if they select to send a paper
resume, schools may not process their application until the paper resume is received.
◦ Allow more space in online resumes, or provide an upload option into Apply Texas
◦ Provide another mechanism for retrieval of lost usernames, passwords, and application id numbers.
◦ Provide an online application deadline table equivalent to the one available in the paper copy of the Apply Texas application.
High School Counselor interviews
![Page 46: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/46.jpg)
High School Counselor interviews
Application deadline table
![Page 47: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/47.jpg)
Suggestions for improving the High School Counselor System
◦ Different navigation options
◦ Different documentation on what data is available
◦ The ability to separate four year and two year information
High School Counselor interviews
![Page 48: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/48.jpg)
The Apply Texas team would like to thank Garrett Stettler for his time, effort, and excellent results.
We would also like to thank the Apply Texas committee for authorizing this study.
We are confident that implementing his results will make the application easier and less frustrating for our applicants.
Usability Study
![Page 49: Usability Study of the Apply Texas Application including Interviews with High School Counselors](https://reader036.fdocuments.in/reader036/viewer/2022062310/56816267550346895dd2d42b/html5/thumbnails/49.jpg)
Questions?Usability Study