The impact of information skills training on independent literature searching activity and requests...

DOI:10.1111/j.1471-1842.2009.00871.x

The impact of information skills training onindependent literature searching activity and requestsfor mediated literature searchesJohn Addison‡, Steven William Glover* & Christine Thornton†

*Christie Hospital NHS Foundation Trust—Kostoris Library, Wilmslow Road, Withington; †Central Manchester University

Hospitals, NHS Foundation Trust—Trust Library Service, Manchester and ‡Pennine Acute Hospitals, NHS Trust—Education

Centre Library, Oldham, UK

Abstract

Background: Most NHS library services routinely offer both mediated searches and information skills

training sessions to their users. We analyse the impact of these two services on the amount of literature

searching demonstrated by users of hospital- based library services in the north-west of England.

Methods: Data for (1) mediated literature searches, (2) number of library users attending information skills

training sessions, (3) amount of library staff time devoted to information skills training, and (4) number of

Athens-authenticated log-ins to databases were obtained from statistical returns for 2007, and analysed for

significant correlations.

Results: There was evidence of quite strong correlations between the two measures of training activity

and the number of mediated literature searches performed by library staff. There was weaker evidence of

correlation between training activity and total literature searching activity.

Discussion: Attending training sessions may make some library users aware of the difficulty of complex

literature searches and actually reduce their confidence to perform their own complex searches indepen-

dently. The relationships between information skills training, mediated literature searches, and independent

literature searching activity remain complex.

Keywords: Libraries, Hospital, Library Services, Medline, Information Storage and Retrieval

Key Messages

Implications for Practice

C

M

ªH

d Both approaches of training and mediated searching have strengths and weaknesses in their own

right but do not adequately service the user population as efficiently as required.d Training offers hands-on practical skills to users, but these skills decline rapidly when not used regu-

larly and require constant updating.d Mediated searching is more effective for the user but would be a major burden to librarian time if

not resourced adequately.

Implications for Policy

d Strategic decisions about maximising the efficiency and effectiveness of NHS library services in sup-

porting EBP will need to take into account that increasing this number will require diverting staff

time from other duties, or investing in additional skilled staff.d To ensure organisations use the best evidence users must be made aware that library information ser-

vices offer both training and mediated searching services.d Information professionals working in health libraries need the skills to be capable of training users

and providing expert searching services to suit the needs of the client.

orrespondence: Steven William Glover, Christie Hospital NHS Foundation Trust—Kostoris Library, Wilmslow Road, Withington, Manchester

20 4BX, UK. E-mail: [email protected]

2010 The authors. Health Information and Libraries Journal ª 2010 Health Libraries Group

ealth Information and Libraries Journal, 27, pp.191–197 191

Impact of information skills training, S. W. Glover192

Background to Dialog, and again from Dialog to Search

Many NHS library services (23 of 29 Acute Hospi-

tal NHS library services in the NW of England) rou-

tinely offer users both face-to-face information

skills training in how to plan and conduct literature

searches of bibliographic databases, and a mediated

literature searching service, whereby a member of

the library staff conducts a search on behalf of a

user. Indeed, both services are seen as part of the

core business of libraries, to the extent that they are

included as part of the NLH annual statistical

returns, and reports of such activity form part of the

Helicon library accreditation procedures.1 Philo-

sophically, these two services may be considered to

be at opposing ends of a spectrum that runs from

the do-it-yourself approach at one end, to the ask

the information professional to do it for you

approach at the other. In practice, there is no reason

why the two should not complement each other, but

it is reasonable to enquire as to the extent of the

return that investment in each provides. This paper

attempts to analyse evidence of the effect of infor-

mation skills training on subsequent user behaviour.

An effective training service aims to equip users

to conduct high-quality literature searches indepen-

dently, thus removing the necessity for trained

users to seek assistance with future searches. How-

ever, in practice, there are a number of reasons

why this is not always so:d Discussion at the Library and Information for

Health Network North West’s Trainers Group

meetings demonstrate that training sessions

vary in duration, content and delivery.
d Without regular practice all learners tend to lose
skills.2 Infrequent database users are likely

to lose or forget information retrieval skills.
d Even those users who regularly undertake their
own literature searches will, by definition

not be as familiar with, or adept at, literature

searching as expert library staff who conduct

literature searches on behalf of many differ-

ent users.
d Time constraints mean that busy clinicians may
prefer to ask the library service to conduct a

search on their behalf.
d The application of NHS procurement regulations
result in regular changes to database plat-

form as users moved from Webspirs ⁄OVID

ª 2010 The authors. Health Informat

He

2.0 within the relatively short period of

8 years. Consequent changes in appearance

and functionality may disconcert even the

most confident of users.

It is reasonable to expect that the provision of

training would have some impact on user behav-

iour, both in terms of independent literature

searching activity, and in the number of requests

for mediated searches that are made to library

services.

This paper attempts to shed light on four

hypotheses:d Organisations whose library services staff train a

higher proportion of their users will see a

greater amount of literature searching activ-

ity.
d Organisations whose library services staff spend
a greater amount of time delivering informa-

tion skills training see a greater amount of

literature searching activity.
d Organisations whose library services staff train a
higher proportion of their users will receive

fewer requests for mediated literature

searches.
d Organisations whose library services staff spend
a greater amount of time delivering informa-

tion skills training will receive fewer

requests for mediated literature searches.

This quantitative approach offers a quick snap-

shot of existing practice, made especially manage-

able as the statistics needed are readily available.

Results may well raise questions around reasons

for users’ behaviour, but answers to these are not

attempted here.

What is already known about this topic

The importance of training to support evidence-

based practice (EBP) and continuing professional

development (CPD) and the role of librarians in

providing that training have long been accepted.3,4

The EMPIRIC Project5 looked at views on the

effectiveness of information skills training. The

findings are discussed by Brettle et al. in an evalu-

ation of the qualitative results in 2007.6 The

authors cite three studies7–9 which suggest that

training is valued by those receiving it.

ion and Libraries Journal ª 2010 Health Libraries Group

alth Information and Libraries Journal, 27, pp.191–197

Impact of information skills training, S. W. Glover 193

Despite the value placed on training, there is

some debate about its effectiveness. Schwartz10

reported increased search activity following train-

ing. Ikeda11 and Kronick et al.12 demonstrated that

hands-on training improved end user searching.

Garg and Turtle13 evidenced some impact on the

search skills of medical students. This was borne

out by Robinson and Lawson14 whose study of

training within the primary care setting found that

course attendance was a precursor to increased use

of resources.

However, even where studies suggest that train-

ing does impact on the ability to search more

effectively, other evidence, often within those same

studies, questions the level of effectiveness.7,13

There are also issues around subsequent search fre-

quency15,16 and the long-term impact of train-

ing.17–19

Measures of ‘effectiveness’ are often based on

the subjective self-evaluation carried out at the end

of training sessions. As Harwell et al.20 points out,

this is a weakness and evaluations should be based

on actual use of resources following training rather

than on ‘self-perceived proficiency and intentions

to use online bibliographic databases…’.

Methods

We aimed to obtain statistical data for four usage

indicators for all hospital trusts in the north-west

of England (NW) for the period April 2007 to

March 2008 inclusive. Usage indicators comprised:

log-ins to the Dialog suite of databases, mediated

literature searches performed by library staff, num-

ber of staff receiving information skills training,

and library staff time spent delivering information

skills training.

The NW has 29 hospital trusts. The Isle of Man

service was excluded from this research both for

political ⁄geographical reasons, and because it

offers the same Athens organisational identity to

all its users, which include social care as well as

health staff. An additional five trusts were

excluded because their NLH statistical returns did

not include the required information. This left 23

services, comprising three specialist trusts, five

teaching trusts and 15 general acute trusts which

do offer information skills training. These 23 ser-

vices form the basis of our subsequent analyses.

ª 2010 The authors. Health Information and Libraries Journal ªHealth Information and Libraries Journal, 27, pp.191–197

Access to the Dialog suite of online biblio-

graphic databases was authenticated by the Athens

Access Management System (AMS). Each hospital

trust has its own unique Athens organisation iden-

tifier that is embedded in their users’ Athens

accounts. The AMS records how many times each

individual account is used to log-in to an Athens-

authenticated resource over a given period. Litera-

ture searching activity data was obtained by using

the statistical reports feature on the Athens admin-

istrator interface. Mediated literature searches, and

user training statistics were obtained from the

annual National Library for Health (NLH) statisti-

cal returns of library activity [formerly collected

by the NHS Library and Knowledge Development

Network (LKDN)] submitted by all NHS library

services.

Literature searching activity was defined as

Athens-authenticated log-ins to the Dialog suite

of databases which was provided to NHS users

during this period and includes log-ins made by

library staff as well as by users themselves. In the

case of the specialist and teaching trusts some

clinical staff are University employees and are

entitled to HE Athens-authenticated resources,

which, they may use in preference to NHS

resources. Therefore, literature searching activity

carried out in these organisations may be under-

represented. More generally, some members of

staff submit lists of document requests on print

outs of searches performed on PUBMED, or the

BMA’s own MEDLINE+. These are not Athens-

authenticated resources, and usage of them is not

recorded in the figures.

Two measures of information skills training are

included in the NLH statistical return. Section 10.1

asks for Library staff hours spent on user educa-tion per year, and 10.2 for Number of library

users receiving education per year. No guidance is

offered as to what constitutes user education, in

terms of duration, qualification or expertise of trai-

ner, or even subject matter. (Several library ser-

vices offer training sessions on critical appraisal,

attendance at which is unlikely to improve litera-

ture searching techniques. In addition, several NW

NHS library services were actively involved in

training public library staff in the use of electronic

health resources during this period. Public librari-

ans are not entitled to register for NHS Athens

2010 Health Libraries Group


accounts or to use associated Athens-authenticated

resources, with the result that the measurable

impact of training on subsequent literature search-

ing activity by this group would be invisible for

the purposes of this current research.) Despite

these qualifications, these returns do offer consid-

erable evidence as to the extent and prevalence of

training.

It is clear that not all library services keep

detailed records of the amount of training they

give. Indeed, the NLH guidance recognises this,

and suggests that ‘If regular education…statistics

are unavailable, it is recommended that this infor-

mation be obtained by undertaking a sample study

for one month, and multiplying the resulting figure

by 12’’. Figures submitted by individual library

services do not indicate whether they represent an

exact count, or an estimate based on a ‘sample’

month multiplied by 12. Efforts were made to

ensure that all statistics reported the same activity.

Three library services who submitted outlying

results for numbers of users trained were contacted

Table 1 NHS Trusts, full-time equivalent employees, number of

training, and numbers of users trained, April 2007 to March 2008

NHS Trust WTE

Aintree Hospitals 3518

Blackpool Fylde & Wyre Hospitals 3559

Bolton Hospitals 3173

Cardiothoracic Centre 1180

Christie 1631

Central Manchester University Hospitals 7555

Countess of Chester 2651

East Cheshire 1802

Lancashire Teaching Hospitals 5187

Liverpool Women’s Hospital 1249

North Cheshire Hospitals 3321

North Cumbria Acute Hospitals 3202

Pennine Acute Hospitals 8607

Salford Royal Hospitals 4042

South Manchester University Hospitals 3972

Southport and Ormskirk Hospitals 2547

St Helens & Knowsley Hospitals 3793

Stockport NHS Foundation Trust 3225

Tameside & Glossop Acute 2095

Trafford Healthcare 1730

University Hospitals Morecambe Bay 3972

Wirral University Teaching Hospital 4903

Wrightington, Wigan & Leigh 3773

Total 80 687

ª 2010 The authors. Health Info

and asked to confirm the figures they had submit-

ted. Two of these services subsequently agreed that

they had misinterpreted the instructions at 10.2

and resubmitted revised figures.

Similarly, Mediated literature searches are

included as a separate category (9.1) on the NLH

statistical return. The guidance states that a search

is defined as ‘a distinct question or information

need for which any number of databases or other

resources may be consulted’. Two library services

returned search totals which enumerated each

occasion of rerunning a saved search for alerting

purposes as a separate search, resulting in totals

which immensely exceeded those of other services.

Once contacted, these services supplied revised

totals. Search totals submitted by the remaining

organisations were assumed to accord to the

definition quoted.

In summary: for the reasons described above:

lack of clarity in the guidance for collecting and

submitting statistics, lack of compliance with what

guidance is provided and library users who insist

Dialog log-ins, mediated searches, library staff time spent on

Dialog

Log-ins

Mediated

Searches

Training

(hours)

Training

(users)

1450 115 160 303

1592 68 46 177

3156 260 130 132

801 500 200 250

2483 267 103 286

4208 591 229 483

971 43 35 68

1038 16 120 175

4248 378 90 311

673 65 60 50

944 398 49 228

2135 105 126 336

6514 467 602 767

2313 357 65 63

2640 52 105 198

1540 150 430 950

2505 90 60 160

2161 150 184 316

1038 10 88 67

965 23 104 104

2039 231 165 331

1992 64 107 270

1292 54 50 250

48 698 4454 3466 7031

rmation and Libraries Journal ª 2010 Health Libraries Group

Health Information and Libraries Journal, 27, pp.191–197


on using non-Athens-authenticated resources, the

accuracy and reliability of the statistics collected is

not absolute. Nevertheless, they offer a basis on

which to analyse any correlations between training

and literature searching.

Once totals for each of the measures of activity

described had been obtained for each of the 23

trusts, an attempt was made to allow for the dif-

fering sizes of each organisation. This was

achieved by dividing each of the measures by the

number of full-time equivalent members of staff

employed by each organisation. This is a fairly

crude measure, as all these organisations employ

a large number of staff who are not involved in

delivering patient care or healthcare management.

One solution to this might have been to restrict

the analysis to numbers of library members, or

numbers of clinicians employed by each organisa-

tion. However, there would be some problems

with this as not all services keep up-to-date mem-

bership records; it is not uncommon for non-

library members and non-clinical staff to attend

Table 2 Dialog log-ins, mediated searches, library staff time spen

equivalent employee by NHS Trust, April 2007 to March 2008

NHS Trust

Dialog

log-ins ⁄ employee

Aintree Hospitals 0.4122

Blackpool Fylde & Wyre Hospitals 0.4473

Bolton Hospitals 0.9946

Cardiothoracic Centre 0.6788

Christie 1.5224

Central Manchester University Hospitals 0.5570

Countess of Chester 0.3663

East Cheshire 0.5760

Lancashire Teaching Hospitals 0.8190

Liverpool Women’s Hospital 0.5388

North Cheshire Hospitals 0.2843

North Cumbria Acute Hospitals 0.6668

Pennine Acute Hospitals 0.7568

Salford Royal Hospitals 0.5722

South Manchester University Hospitals 0.6647

Southport and Ormskirk Hospitals 0.6046

St Helens & Knowsley Hospitals 0.6604

Stockport NHS Foundation Trust 0.6701

Tameside & Glossop Acute 0.4955

Trafford Healthcare 0.5578

University Hospitals Morecambe Bay 0.5133

Wirral University Teaching Hospital 0.4063

Wrightington, Wigan & Leigh 0.3424

Totals 0.6035


training sessions or request mediated literature

searches; and results might be distorted in the

case of services which go out of their way to

encourage membership amongst staff whose

interests in the provisions of the service may be

limited, e.g. use of PCs.

Results

Table 1 lists the name of each trust, and indicates

their number of whole time equivalent employees.

The remaining four columns give the raw scores

for each of the four indicators described.

Table 2 is the result of dividing each of the four

indicators by the number of whole time equivalent

employees working at that trust.

Table 3 presents the degree of correlation (Pear-

son Correlation Co-efficient) between each of the

two measures of training, and the relevant user

behaviour.

We found weak evidence that the number of

users trained correlates with the amount of

t on training, and number of users trained per full-time

Mediated

searches ⁄ employee

Training

hours ⁄ employee

Proportion of

employees trained

0.0327 0.0455 0.0861

0.0191 0.0129 0.0497

0.0819 0.0410 0.0416

0.4237 0.1695 0.2119

0.1637 0.0632 0.1754

0.0782 0.0303 0.0639

0.0162 0.0132 0.0257

0.0089 0.0666 0.0971

0.0729 0.0174 0.0600

0.0520 0.0480 0.0400

0.1198 0.0148 0.0687

0.0328 0.0394 0.1049

0.0543 0.0699 0.0891

0.0883 0.0161 0.0156

0.0131 0.0264 0.0498

0.0589 0.1688 0.3730

0.0237 0.0158 0.0422

0.0465 0.0571 0.0980

0.0048 0.0420 0.0320

0.0133 0.0601 0.0601

0.0582 0.0415 0.0833

0.0131 0.0218 0.0551

0.0143 0.0133 0.0663

0.0552 0.0410 0.0871


Table 3 Pearson’s correlation coefficient r between the two training measures and the two user behaviour outcomes (CI

95%), P-value

Mediated searches Dialog log-ins

Training (users) 0.42 (0.01, 0.71), P = 0.043 0.27 ()0.16, 0.61), P = 0.215

Training (hours) 0.59 (0.24, 0.81), P = 0.003 0.23 ()0.20, 0.59), P = 0.291


organisation-wide literature searching as measured

by log-ins to Dialog.

We also found weak evidence that the amount

of library service staff time spent delivering infor-

mation skills training correlates with the amount of

organisation-wide literature searching as measured

by log-ins to Dialog.

There was quite strong evidence that training

measured in terms of numbers of users correlates

with a demand for mediated literature searches.

We found strong evidence to suggest that the

number of hours devoted to training by library

staff correlates with the number of mediated litera-

ture searches performed by library staff.

Discussion

The results contradicted our expectations, as they

suggest that neither investment of library staff

time in training users nor attendance by users at

information skills training sessions has little impact

on the number of independent literature searches

performed. (Whether the training improved the

quality of users’ literature searching is an interest-

ing question, but one which lies outside the scope

of this current paper.)

The correlation between training hours delivered

by library staff and demand for mediated searches

was particularly unexpected. We had expected pre-

cisely the opposite: that information skills training

would enable users to become more independent

in their literature searching behaviour leading to a

reduced demand for mediated literature searches.

The true picture may, of course, be more compli-

cated than this: the choice between doing searches

oneself, and deciding to refer them to a member of

the library staff does not have to be a once-and-

for-all-time either ⁄or: both may be appropriate in

different circumstances. In addition, attendance at

a training session may also serve to remind users

that library staff provide a mediated literature

ª 2010 The authors. Health Info

searching service, use of which may be preferable

to relying on hard-won, but perhaps shakily

grounded, skills of one’s own.

We speculate that providing more extended

training may make users aware of the complexity

of literature searching, and reduce their confidence

to perform their own searches in situations where

they have a complex question to investigate, or

where they think doing a thorough search really

matters.

One piece of evidence lends some support to

this speculation. In 2008 an unpublished internal

online survey21 was conducted by the library ser-

vice at Pennine Acute in order to gauge user satis-

faction with its mediated literature searching

service. Over 70% of respondents (i.e. staff who

had requested mediated literature searches in the

past year) stated that they had attended literature

searching training at some point in their careers

(16% of them within the past year). Regrettably,

the authors of this questionnaire did not follow up

with the interesting question of why they had

requested a mediated search rather than performing

the search independently.

The number of mediated searches performed

represents less than 9.2% of the total literature

searching activity performed in the services sur-

veyed, and represents a low number of searches

per employee in virtually all cases.

To ensure that the current deployment of library

services offers optimum value, additional research

should be conducted investigating: (a) the reasons

users give for requesting mediated searches rather

than performing the searches themselves, and (b)

comparing the search results of librarians with

those of trained library users.

References

1 Fowler, C. & Trinder, V. 2002 Accreditation of library and

information services in the health sector. HeLicon.

rmation and Libraries Journal ª 2010 Health Libraries Group

Health Information and Libraries Journal, 27, pp.191–197


Accessible at: http://www.lihnn.nhs.uk/document_uploads/

Accreditation_Reports/accreditation_criteria.pdf.

2 Arthur, W., Bennett, W., Stanush, P. L. & McNelly, T. L.

Factors that influence skill decay and retention. A

quantitative review and analysis. Human Performance 1998,

11, 57–101.

3 NHS Executive. Library and Information Services: Health

Service Guidelines, HSG (97) 47, 1997. Accessible at:

http://www.dh.gov.uk/prod_consum_dh/groups/

dh_digitalassets/@dh/@en/documents/digitalasset/

dh_4012399.pdf (accessed 8 December 2008).

4 Davies, R. & Urquhart, C. J. EVINCE: Establishing the

Value of Information to Nursing Continuing Education.

Report of the EVINCE Project. BLRIC Report 44. Boston

Spa: BLDSC, 1997.

5 Brettle, A., Hulme, C. & Ormandy, P. Effective Methods of

Providing Information for Patient Care: EMPIRIC Project –

Baseline Data Collection Survey. Salford: University of

Salford, 2004.

6 Brettle, A., Hulme, C. & Ormandy, P. Effectiveness of

information skills training and mediated searching:

qualitative results from the EMPIRIC project. Health

Information and Libraries Journal, 2007, 24, 24–33.

7 Brettle, A. Information skills training: a systematic review

of the literature. Health Information and Libraries Journal,

2003, 20(Suppl. 1), 3–9.

8 Urquhart, C. M., Massiter, C., Thomas, R., Sharp, S. &

Smith, J. Getting information to Vocational Trainees: Report

of the GIVTS Project. London: British Library and

Information Commission Research Report, 1999.

9 Wozar, J. A. & Worona, P. C. The use of online information

resources by nurses. Journal of the Medical Library

Association, 2003, 91, 216–221.

10 Schwartz, D. G. Medline training for medical students in the

clinical curriculum. Academic Medicine, 1992, 67, 1040–2446.

11 Ikeda, N. R. Impact of end-user search training on pharmacy

students: a four-year follow-up study. Bulletin of the

Medical Library Association, 1992, 80, 124–130.

12 Kronick, J., Blake, C., Munoz, E., Heilbrunn, L.,

Dunikowski, L. & Milne, W. K. Improving on-line skills and


knowledge; a randomized trial of teaching rural physicians to

use on-line medical information. Canadian Family

Physician, 2003, 49, 312–317.

13 Garg, A. & Turtle, K. M. Effectiveness of training health

professionals in literature search skills using electronic

health databases – a critical appraisal. Health Information

and Libraries Journal, 2003, 20, 33–41.

14 Robinson, S. & Lawson, S. Evaluating the impact

of information skills training within primary care.

Health Information and Libraries Journal, 2005, 22,

63–65.

15 Vogel, E. W., Block, K. R. & Wallingford, K. T. Finding the

evidence: teaching medical residents to search MEDLINE.

Journal of the Medical Library Association, 2002, 90, 327–

330.

16 Erickson, S. & Warner, E. R. The impact of an individual

tutorial session on MEDLINE use among obstetrics and

gynaecology residents in an academic training programme: a

randomized trial. Medical Education, 1998, 32, 269–273.

17 Trinder, V. M., Fleet, G. E. & Gray, A. E. Evaluating the

impact of library user training programmes across Thames

Valley Strategic Health Authority in the UK. Health


18 Booth, A. Sole searching – wasting our time? Health


19 Cheng, G. Y. T. Educational workshop improved

information-seeking skills, knowledge, attitudes and the

search outcome of hospital clinicians: a randomised

controlled trial. Health Information and Libraries Journal,

2003, 20, 22–23.

20 Harwell, T.S., Law, D.G., Ander, J.L. & Helgerson, S.D.

Increasing state public health professionals’ proficiency in

using PubMed. Journal of the Medical Library Association,

2008, 96, 134–137.

21 Pennine Acute Hospitals NHS Trust Library Service.

Literature searching questionnaire (PAT): survey monkey.

Report to Pennine Acute Trust Library Group, November

2008.

Received 18 April 2009; Accepted 22 September 2009


The impact of information skills training on independent literature searching activity and requests...

Documents

Transcript of The impact of information skills training on independent literature searching activity and requests...