2009 CDC Conference Presentation Ellery
21
INShape Indiana: A Case Study in Evaluation Strategies for Evaluating Health-Related Website Usability Peter J. Ellery Fisher Institute for Wellness and Gerontology Jennifer Bott Miller School of Business Kristin Ritchey School of Psychology Lori Byers College of Communication, Infor mation and Media 3 rd National Conference on Health Communication, Marketing and Media Atlanta, GA / August, 3009 Jane Ellery Fisher Institute for Wellness and Gerontology
description
2009 CDC Marketing Conference in Atlanta,GA
Transcript of 2009 CDC Conference Presentation Ellery
INShape Indiana: A Case Study in Evaluation Strategies for Evaluating Health-Related Website
Usability
Peter J. Ellery Fisher Institute for
Wellness and Gerontology
Jennifer BottMiller School of
Business
Kristin RitcheySchool of Psychology
Lori ByersCollege of
Communication, Information and Media
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009
Jane Ellery Fisher Institute for
Wellness and Gerontology
2
3 Essential Strategies Needed for the Effective Assessment and Evaluation of Health Information Websites
Use a Theoretically Based Approach to Your Assessment and Evaluation Process.
Use “Micro” Qualitative Research Methods to Determine the Interactive Experience of the User.
Use Quantitative “Macro” Research Methods to Determine the Overall Experience of your Clients as a Group.
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 30093 Key Strategies for Evaluating Website Usability
4
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009
Without a theoretical frame of reference, usability data is just a series of numbers, words, or as in this case … black marks.
The Importance of a Theoretical Framework
5
Having a frame of reference also helps prevent data from being distorted by other data, and from incorrect interpretations from being made.
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009The Importance of a Theoretical Framework
6
Meaningful
Is what you do adding value and meaning to your life?
Is the world around you
ordered, consistent, structured, and clear? Does it conform with
the way you understand the world
to be?
Do you have the time and resources needed
to cope with the problems you face on
a daily basis?
Comprehensible
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009The Importance of a Theoretical Framework
Salutogenesis: A Theoretical Framework for Determining Usability
7
General Resistance Resources (GRR’s) give rise to, or reinforce, a strong Sense of Coherence.
Deficits in GRR’s Diminish our Sense of Coherence.
GRR’s are properties of a person, a collective, or a situation that allow individuals to successfully cope and thrive.
The one quality GRRs have in common is that they all foster repeated life experiences that help one see the world as ‘making sense,’ cognitively, interactively and emotionally.
Some examples of GRR’s include time, money, ego, cultural background, family support, intelligence, etc.
3rd National Conference on Health Communication, Marketing and MediaAtlanta, GA / August, 3009The Importance of a Theoretical Framework
8
3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Assessing the Individual Experience
9
Indiana State Government supported health and wellness program that is promoted through Indiana businesses and organizations.
Primarily a web-based health and wellness promotion program that also utilizes motivational/informational emails to participants.
The program currently has a participant database for over 63,000 “Hoosiers” of which approximately 70% are female participants.
3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009Assessing the Individual Experience
10
The Qualitative Portion of the Study Design Involved:
Developing a “Task-Based Interview Guide” that included questions related to how meaningful, manageable and comprehensible the website was to the subject;
Selection of a demographically similar subject sample;
Interviewing and recording subjects while attempting to complete the tasks from the interview guide on the INShape Indiana website;
Collecting ASL eye tracking technology data showing user eye movement around the web pages; and,
The use of Morae Usability software to capture subject voice and facial expression while the website is being used;
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
11
The participants in the study were asked to complete tasks and provide their thoughts and opinions related to different aspects of the website.
Tasks and activities were developed based on the following website sections:• The landing page or initial “home” page for
the website• Signing up and the support information
section of the website• The tools and resources section of the
website• The current program support information
Participants completed tasks and activities that were considered typical and common to users of this kind of website.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
12
16 volunteer subjects participated in the study. Group demographics and characteristics included:
• All were Ball State University employees currently enrolled in INShape Indiana• 3 to 1 Female to Male ratio (which was representative of the INShape Indiana
Membership gender makeup)• A variety of education levels
3 (Some College Education); 4 (Bachelors Degree); & 9 (Graduate or Terminal Degree)
• A variety of age levels 3 (18-24); 4 (25-34); 3 (35-44); 0 (45-54); & 6 (55 and older)
• A variety of computer use and weekly internet access time 2 (2 to 3 times per week); 3 (4 to 5 times per week); & 11 (everyday)
• All but one had been to the website prior to the study
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
13
• Links to the left and right were readily noticed;
• And, the central image does draw viewer attention.
• However, the web page content did not appear to catch the subject’s attention very much.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
14
• Links to the left and right were readily noticed;
• And, the central image does draw viewer attention.
• However, the web page content did not appear to catch the subject’s attention very much.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
15
• Links to the left and right were readily noticed;
• And, the central image does draw viewer attention.
• However, the web page content did not appear to catch the subject’s attention very much.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
16
– Good information that is relevant to the user related to tobacco cessation, nutrition and physical activity.
– Having the opportunity to ask questions for themselves and for others is a good feature.
– Unsubscribing was difficult. Separate it from the subscription process.
– Add headlines to your stories to draw reader’s attention
– INShape Partners is confusing. Consider offering activities and events related to programs being conducted.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
17
– Good information that is relevant to the user related to tobacco cessation, nutrition and physical activity.
– Having the opportunity to ask questions for themselves and for others is a good feature.
– Unsubscribing was difficult. Separate it from the subscription process.
– Add headlines to your stories to draw reader’s attention
– INShape Partners is confusing. Consider offering activities and events related to programs being conducted.
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
18
– Well organized with good information but too much on the landing page (Manageable).
– The calorie counter, newsletter and videos and the website overall is supportive (Meaningful).
– Links, photos and information all need to be relevant to the user. For example, “What’s happening in my County?” (Comprehensible)
– Unsubscribe needs to be a stand alone link on the web site. (Comprehensible)
– Easy to wander into other sections on the government side of the site. (Manageable)
– Partner page appears to be more about advertising than useful links for the user. (Meaningful)
Assessing the Individual Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
19
Assessing the Overall Group Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
20
Adapted from Antonovsky’s Orientation Toward Life Survey.
Comprised of rating items (1 to 5) for 31 statements related to the meaningfulness, manageability and comprehensibility of the website.
Administered on-line to users of the INShape Indiana website and completed by almost 400 participants.
Responses were used to reduce the number of survey items and begin the validity and reliability testing process.
Preliminary analysis of the data suggests a one factor structure consistent with Antonovsky’s suggestion that “Meaningfulness,” “Manageability,” and “Comprehensibility” are interconnected and may be more appropriately viewed as the single factor “Sense of Coherence.”
Assessing the Overall Group Experience 3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
21
Formulating a theoretical framework is essential as we work toward a common understanding of “usability.”
Website evaluation should include qualitative research methods that look at individuals actually using the website.
ASL Eye Tracking and Morae Video are great support tools in qualitative research looking at a users experience on a website but should always be viewed in context with the qualitative data collected from the subject.
An overall group evaluation of your clientele should provide a benchmark measure of their sense of coherence when using the website, and offer some indication as to those aspects of the website that make it difficult to use.
3rd National Conference on Health Communication, Marketing and Media - Atlanta, GAAugust, 3009
