Health Literacyon the World Wide Web

Research Team

• Principal Investigator: Frances Wen, PhD• Co-Investigators:

Jagan Bokka, MD Bhupinder Walia, MD

Michelle Farabough, MSKM Danny Stout, MHR

Anna Jaffe, BS

Introduction• An estimated 20% of population will be

65 or older by 2030.• Alzheimer’s in US expected to triple over

next 50 years.

• Demand for information on how to navigate and cope with issues faced in later life is expected to increase.

• “Alzheimer’s, dementia, and memory loss” among the top health topics searched online.

Health Literacy“A patient’s ability to obtain, process, and

understand basic health information and services needed to make appropriate health decisions.” – AMA

– Gaining attention last 10 yrs… vulnerable population– Important part of health outcomes

Aim“Evaluate the readability and suitability of education materials found on 15 frequently accessed health care websites targeted at individuals seeking information about Alzheimer’s, dementia, and memory loss.”

Website Selection

STEP ONE: “Page Rank”• Two most widely used Internet search

engines Google and Yahoo• Search terms “Alzheimer’s” “dementia”

and “memory loss”• Top 15 websites for

each term on each search engine

We identified the top 15 unique websites.

Websites Evaluated

• Governmental: NIH• Nonprofit: Alzheimer’s Association• Popular Medical: WebMD, Mayo Clinic,

Medicine Net, Everyday Health • Others: Wikipedia, Health NY Times,

Real Age

Webpage Selection

STEP TWO• Added search term “education” to search

terms for each website• Identified unique url

“jumping off” point

Educational value

STEP THREE• Evaluate education materials as if

recommending to patient

Evaluation Methods

1. SAM: Suitability Assessment of Materials an instrument used by health care

professionals to assess six main categories:– Content – Literacy demand: SMOG– Graphics– Layout/typography – Learning Stimulation – Motivation

1a. SMOG: Standard Measure of Gobbledegook a commonly used literacy evaluation tool

instrument to assess reading levels – Figured into SAM “Literacy demand”

2. DISCERN an instrument used by health care professionals to assess consumer health information relative to…– Reliability (authority*) and whether or not you

should consider it a trusted source– Specific details about the treatment choices– Overall quality rating of the media

*more about this at the end of the presentation

3. QUIS: Questionnaire for User Interface Satisfaction an instrument used to measure overall reaction,

overall website, terminology and information, learning, and website capabilities

Procedure• Two investigators independently

reviewed/assessed materials using SAM, DISCERN and QUIS to evaluate suitability.

• Another investigator used SMOG to evaluate readability and incorporate results in SAM.

• Interrater agreement analysis and descriptive analyses were conducted.

Interrater Agreement: SAM• Interrater agreement = .78

• < .40: No agreement• .40-.59: Moderate agreement• .60-.79: Substantial agreement• >.80: Outstanding agreement

• Raters agreed on the SAM so the results could be averaged between raters.

Results: SAM Adequate

• SAM scores: 0=Not Suitable, 1=Adequate, 2=Superior• SMOG scores (an item on the Literacy Demand scale) were all

determined to be at or above the 10th grade reading level.

Content

Litera

cy Dem

and

Graphics

Layo

ut and Ty

pography

Learn

ing Stimulati

on, Motiva

tion

Cultural

Appropriaten

ess0

0.4

0.8

1.2

1.6 1.371.2 1.1

1.361.23 1.17

Interrater Agreement: DISCERN

• Interrater agreement = .48• < .40: No agreement• .40-.59: Moderate agreement• .60-.79: Substantial agreement• >.80: Outstanding agreement

• Raters did not agree very well on their scoring of the DISCERN, so their ratings had to be analyzed separately.

Results: DISCERN Above Average

DISCERN scores were slightly above average for both raters. On average, Rater 1 scored websites slightly higher than Rater 2.

Is the p

ublicati

on relia

ble

Quality

of inform

ation on tr

eatm

ent c

hoices

Overal

l rating

3

3.4

3.8

4.2 4.03

3.54 3.63.683.46 3.33

Rater 1 Rater 2

Interrater Agreement: QUIS

• Interrater agreement = .73• < .40: No agreement• .40-.59: Moderate agreement• .60-.79: Substantial agreement• >.80: Outstanding agreement

• Raters agreed on the QUIS so the results could be averaged between raters.

Results: QUIS Above Average

• QUIS scores 0-9; higher scores represent higher user friendliness• The average website scored above average on website usability.

02468 6.21 6.72 6.67 7.27 7.55

Discussion

• 15 educational websites with information about age-related cognitive impairment evaluated. Overall the suitability and readability of these materials are adequate.

• Future direction: User analysis

• Research focused on health literacy is critical to improve health care outcomes.

Clinical Implications

“Authority”

– Date of creation/update– History/longevity of publication or website – Author or source: academic, government– “Facework”: links or citations– Accuracy of information– Objectivity and coverage: how in-depth

Questions?

Websites Used• NIH (and all associated separated listings):

http://www.nia.nih.gov/Alzheimers/Publications/UnderstandingMemoryLoss/• WebMD: http://www.webmd.com/alzheimers/tc/dementia-topic-overview• MayoClinic: http://www.mayoclinic.com/health/dementia/DS01131• MedicineNet: http://www.medicinenet.com/alzheimers_disease_causes_stages_and_symptoms/article.htm• Health NY Times:

http://health.nytimes.com/health/guides/disease/alzheimers-disease/overview.html?8qa&scp=1-spot&sq=alzheimer%27s&st=nyt

• Alzheimer’s Association: http://www.alz.org/alzheimers_disease_10_signs_of_alzheimers.asp• Wikipedia: http://en.wikipedia.org/wiki/Alzheimer's_disease• Yahoo Health: http://health.yahoo.net/galecontent/alzheimers-disease• Everyday Health: http://www.everydayhealth.com/senior-health/alzheimers/index.aspx• Med Help: http://www.medhelp.org/medical-information/show/393/Alzheimers-disease• Right Health:

http://www.righthealth.com/topic/memory_loss,_dementia,_alzheimer's/overview/adam20?fdid=Adamv2_000760

• Real Age: http://www.realage.com/check-your-health/disease-prevention/understanding-alzheimers-disease?click=main_sr

• Wellsphere: http://www.wellsphere.com/caregiving-article/how-the-loss-of-memory-works-in-alzheimer-8217-s-disease-and-how-understanding-this-could-help-you/1339201

• Healthline: http://www.healthline.com/adamcontent/alzheimers-disease• Family Doctor: http://familydoctor.org/online/famdocen/home/seniors/common-older/124.html