The Health Information Literacy Knowledge Test (HILK):

Construction and results of a pilot study

Leibniz Institute for Psychology Information (ZPID)

Trier, Germany mayer@zpid.de s1juholz@uni-trier.de

Anne-Kathrin Mayer & Julia Holzhäuser

Health information literacy (HIL) comprises a set of abilities needed to recognize a health information need, search and evaluate relevant health information, and to use this information to make appropriate health decisions

relevant for people´s autonomy regarding health decisions

Everyday HIL is usually assessed by self-report measures or measures of basic reading skills and numeracy (e.g., TOFHLA, Parker et al., 1995; REALM, Davis et al., 1993)

So far no validated achievement test exists in the international research literature which is not focused on the academic context

Need for a knowledge test which goes beyond basic literacy and which is appropriate for adults of different age groups with middle to higher levels of education (adequate level of reading skills is assumed)

Importance of a knowledge test for research purposes: measurement of HIL in the general population to identify people with low HIL, determine a need for training programs of HIL and evaluate such interventions

Background

Development of an economic test of knowledge about seeking and evaluating everyday health information of adults based on a skill decomposition

Empirical construction in two steps:

1) Expert study: examination of the correctness of the answers and refinement of item formulations

2) Pilot study: considerable shortening of the test based on psychometric properties of the items and first examination of its validity

Aims

Construction of items was based on a skill decomposition derived from models of information problem solv-

ing as well as general models of information literacy (e.g. IPS, Brand-Gruwel et al., 2009; Big6, Eisenberg

& Berkowitz, 1990)

Skill decomposition (Four skills with two subskills each):

1. Definition of information need (1.1 Define the information problem / 1.2 Identify information needed),

2. Planning the search (2.1 Knowledge of information resources / 2.2 Determine search strategy),

3. Accessing information sources (3.1 Identify type of source / 3.2 Gain access (full text)),

4. Scanning information (4.1 Assessment of relevance and quality / 4.2 Orientation within source)

Questions require participants, e.g., to recognize adequate sources to satisfy a specific information need, or

to evaluate health materials with regard to their relevance or quality

The test focuses on different types of information sources and providers, e.g. books, libraries, and the inter-

net

Multiple choice questions with three answer options and an “I don´t know”-category (number of correct

answers ranging from 0 to 3)

Example items:

Test Construction

Subjects: N = 11 psychologists from the field of information literacy research (6 male, 5 female)

Materials: The experts worked on a preliminary paper-and-pencil test version of the HILK with 57 items

Results:

Four items were dropped because experts did not reach an acceptable consensus about the correct

answers

Experts’ feedback was used to refine several item formulations

Resulted in a revised test version with 53 items

Expert Study

Subjects and Procedure:

N = 138 psychology students, 70.3% Bachelor, 28.3% Master

Age: between 18 and 33 years (M = 22.61, SD = 2.71)

87% female and 13% male

Data was collected in computer laboratories or classrooms at the University of Trier in groups of up to 20 sub-

jects; the sessions took between 90 and 120 minutes

Participants were paid for their participation

Materials:

HILK: revised version with 53 items

Test of domain-specific scholarly information literacy (Leichner et al., 2013; k = 35)

Battery of self-report questionnaires assessing :

1) self-efficacy beliefs related to health-related information: SWE-IB-16 (Behm, 2015; k = 16), eHEALS (Soellner et al., 2014; k = 8)

2) current health status SF-12 (Bullinger & Kirchberger, 1998; k = 12) 3) health related control beliefs: FEGK (Ferring & Filipp, 1989; k = 29) 4) health anxiety: MK-HAI (Bailer & Witthöft, 2006; k = 14)

The HILK could be successfully shortened to a 24-item version based on psychometric properties of the items

(Item-total correlations range from rit = .11 to .38)

Reliability:

The shortened version demonstrates acceptable internal consistency (Cronbach’s Alpha = .71)

Guttman´s Lambda 6 of λ6 = .78 more suitable measure of reliability for heterogeneous constructs

Validity and associations with other measures:

The HILK is moderately correlated with the scholarly information literacy test for psychology (see Table 1 for

correlations)

sign for convergent validity

Significant correlation with a measure of self-efficacy beliefs related to health-related information (SWE-IB-

16), but not with self-efficacy beliefs focused on the internet and electronic resources (eHEALS)

No significant correlations found between achievement in the HILK and health related control beliefs, health

anxiety and the current health status

Significant group difference in HILK test performance between Bachelor (M = .79; SD = .09) and Master stu-

dents (M = .82; SD = .11; t(134) = 1.69; p < .05, one-tailed)

Results

Pilot Study

3rd European Health Literacy Conference, November 17-18, Brussels

References Bailer, J. , Witthöft, M. (2006). Modifizierte Kurzform des Health Anxiety Inventory (MK-HAI) [The health anxiety inventory - German modified version]. In A. Glöckner-Rist (Ed.), ZUMA-Informationssystem. Elektronisches Handbuch sozialwissenschaftlicher Erhebungsinstrumente. Version 10.0. Mannheim: Zentrum

für Umfragen, Methoden und Analysen. Behm, T. (2015). Informationskompetenz und Selbstregulation: Zur Relevanz bereichsspezifischer Selbstwirksamkeitsüberzeugungen [Information competence and self-regulation: The relevance of domain-specific self-efficacy] . In A.-K. Mayer (Ed.) Informationskompetenz im Hochschulkontext – Interdisziplinäre

Forschungsperspektiven (S. 151-162). Lengerich: Pabst Science Publishers. Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 53(4), 1207-1217. Bullinger, M., & Kirchberger, I. (1998). Fragebogen zum Gesundheitszustand - Manual [Health survey - Manual]. Göttingen: Hogrefe-Verlag. Davis, T. C., Long, S. W., Jackson, R. H., Mayeaux, E. J., George, R. B., Murphy, P. W., & Crouch, M. A. (1993). Rapid estimate of adult literacy in medicine: a shortened screening instrument. Family medicine, 25(6), 391-395. Eisenberg, M. B., & Berkowitz, R. E. (1990). Information Problem Solving: The Big Six Skills Approach to Library & Information Skills Instruction. Norwood, NJ: Ablex Publishing Corporation. Ferring, D., & Filipp, S. H. (1989). Der Fragebogen zur Erfassung gesundheitsbezogener Kontrollüberzeugungen (FEGK): Kurzbericht [The Health-Related Locus of Control Questionnaire (FEGK)]. Zeitschrift für Klinische Psychologie, Psychopathologie und Psychotherapie, 18, 285-289. Leichner, N., Peter, J., Mayer, A.-K., & Krampen, G. (2013). Assessing information literacy among German psychology students. Reference Services Review, 41(4), 660-674. doi:10.1108/RSR-11-2012-0076 Parker, R. M., Baker, D. W., Williams, M. V., & Nurss, J. R. (1995). The test of functional health literacy in adults. Journal of General Internal Medicine, 10(10), 537-541. Soellner, R., Huber, S., & Reder, M. (2014). The concept of eHealth Literacy and its measurement. Journal of Media Psychology, 26, 29-38.

SWE-IB-16

α = .85

eHEALS

α = .77

ILT

α = .67

FEGK internal

α = .83

FEGK external

α = .81

SF-12 physical

SF-12 mental

MK-HAI

α = .89 HILK .15* -.02 .49** .07 -.08 .06 .17 -.08

Table 1. Correlations between HILK test performance and self-efficacy beliefs related to health-

related information (SWE-IB-16 and eHEALS), information literacy test (ILT), health related control be-

liefs (FEGK), current health status (SF-12) and health anxiety (MK-HAI).

* p < .05; ** p < .01 (one-tailed)

Book 1 Book 2 Book 3

Additional materials provided in a supplement Questions

The 24-item version of the HILK is an economic test of knowledge about seeking and evaluating everyday

health information which may be used for research purposes

More evidence is needed before applying it in research:

Studies needed with people from different age groups, fields of studies, and without academic background

Stronger indicators of validity needed, e.g. search tasks or measures of the general cognitive level

Conclusion

8. Look at the book covers on page 3 in the supplement. Which book contains

or which books contain probably the most balanced information (naming pros

and cons) about the topic “health outcomes of sport in older age”?

True Not true

Book 1

Book 2

Book 3

I don´t know

22. Please look at the list of contents on page 5 in the supplement. You would

like to find other publications about the treatment options for lung cancer as

easily as possible. How do you proceed?

True Not true

I read the chapter „Therapeutic approaches for lung cancer” and look up the cited works in the section “References”.

I go systematically through the section “References“ and look for all the sources which relate to treatment options accord-ing to the title.

I look for “treatment options“ in the index.

I don´t know

50. You want to find out if a website contains reliable information. How do you

proceed?

True Not true

I look for the legal notice for finding out which person or or-ganization is responsible for the content of the webpage.

I pay attention if the website is clearly designed and uses an appealing design.

I take care if the website contains advertisements – if yes, the information is useless.

I don´t know

11.2.2 The DNA structure .................................................................................................................. 13

11.2.3 The role of telomeres ............................................................................................................. 16

11.2.4 The human immune system ................................................................................................... 19

11.3 Cardiovascular diseases ......................................................................................................... 24

11.3.1 The structure of the cardiovascular system ........................................................................... 24

11.3.2 Hypertension .......................................................................................................................... 26

11.3.3 Embolism and stroke .............................................................................................................. 29

11.3.4 Coronary ................................................................................................................................. 33

11.4 Lung cancer ............................................................................................................................. 37

11.4.1 Lung anatomy ......................................................................................................................... 37

11.4.2 Risk factors for lung cancer ..................................................................................................... 40

11.4.3 Diagnostic procedures for lung cancer ................................................................................... 42

11.4.4 Therapeutic approaches for lung cancer ................................................................................ 45

11.5 Summary ................................................................................................................................... 47

References ................................................................................................................................ 50

Outlook .................................................................................................................................... 64

Glossary ................................................................................................................................... 17

Index ........................................................................................................................................ 20