Pilot Study to Test the Use of a Mobile Device in the Clinical Setting to Access Evidence-Based...

Original Article

Pilot Study to Test the Use of a Mobile Devicein the Clinical Setting to Access Evidence-BasedPractice Resources

Jenny Morris, RN, BSc, PhD, Veronica Maynard, BSc, MSc, PhD, CSP

ABSTRACTAim: To test the feasibility and acceptability of the use of a mobile device to access evidence-based

practice (EBP) resources in the clinical setting.Methods: A pretest/posttest design was used with a convenience sample of 2nd- and 3rd-year pre-

registration undergraduate health care students in the United Kingdom. Questionnaires were used tomeasure (1) feasibility and acceptability of the mobile device and (2) perceptions of the development ofEBP knowledge and skills. The study took place during the students’ clinical practice and involved twomeetings at the beginning and end of the placement period. A Web page was developed to support theprocess and provide links to key EBP resources.

Results: Nineteen undergraduate physiotherapy and nursing students took part in the study. The mainfindings indicated a generally low level of utilisation of the mobile device in the clinical setting, primarilydue to practical difficulties associated with accessing the Internet and the small size of the screen.Consequently, the majority of the students used personal computers (PCs) to access EBP resources.Through the process, students reported improvements in their knowledge and skills in relation to EBPand the appraisal of clinical guidelines.

Conclusions: Students were able to complete the EBP activity using either the mobile device or PCand reported improvements in their knowledge and skills in relation to EBP and the appraisal of clinicalguidelines. Findings suggest that for undergraduate health care students, rapid access to online evidencein the clinical environment is not necessarily essential for the integration of the EBP process into practice,or for the development of EBP knowledge and skills, provided there is easy access to such evidence atsome point during the placement period.KEYWORDS clinical practice, evidence-based practice (EBP), mobile device, undergraduate education, knowl-edge and skills

Worldviews on Evidence-Based Nursing 2010; 7(4):205–213. Copyright ©2010 Sigma Theta Tau International

Jenny Morris, Senior Lecturer and Veronica Maynard, Senior Lecturer, both at theFaculty of Health and Social Work, University of Plymouth, Plymouth, England.

Address correspondence to Jenny Morris, Faculty of Health and Social Work,University of Plymouth, Knowledge Spa, Truro TR1 3HD, England; [email protected]

The authors thank the Centre for Excellence in Professional Placement Learningat the University of Plymouth in England for funding the study, and all the adultbranch nursing and physiotherapy students at the University of Plymouth whotook the time and effort to participate. We would also like to thank Rob Stilwelland Jeremy Black, Learning Technologists at the University of Plymouth, for theirassistance with the use of the mobile technology.

Accepted 21 July 2009Copyright ©2009 Sigma Theta Tau International1545-102X1/09

INTRODUCTION

Evidence-based practice (EBP) is embedded in most un-dergraduate health care preregistration and postquali-

fying programmes, and is also taught at postgraduate level.To become skilled in EBP, students are taught how to de-velop focused questions using frameworks such as Patient,Intervention, Comparison, Outcome (PICO) (Nollan et al.2005), how to search efficiently using search terms, how toselect appropriate information both in terms of the topicand in terms of the nature of the evidence likely to beyielded, how to critically appraise primary and secondarysources of research, and how to apply appraised evidenceto practice.

The importance of EBP to health care curricula is re-flected in the Sicily statement that recommended “. . . all

Worldviews on Evidence-Based Nursing �Fourth Quarter 2010 205

Mobile Technology and Evidence-Based Practice

practitioners understand the principles of evidence-basedpractice, implement evidence based policies, and have acritical attitude to their own practice and to evidence”(Dawes et al. 2005, p.3). The value of reading and criticallyevaluating research-based articles has long been recognisedwithin medicine, although the research findings regardingthe benefits gained from teaching these skills are mixed(Norman & Shapiro 1998; Taylor et al. 2000).

The journal club format used traditionally in medicaleducation is now recognised as a useful tool for developingcritical appraisal skills in a variety of health care studentgroups (Owen et al. 2001; Goodfellow 2004; Dobrzanska& Cromack 2005; McQueen et al. 2006). However, jour-nal clubs also have their limitations in that many sufferfrom nonattendance and lack of preparation or active par-ticipation by those attending (Cramer & Mahoney 2001;Dobrzanska & Cromack 2005) and have been criticisedfor not encouraging deep learning in participants (Khan &Gee 1999).

Developing critical appraisal skills however, is only oneaspect of EBP. Changing the focus of the journal clubfrom one of critical appraisal to that of evidence-basedmedicine has demonstrated increased knowledge and sat-isfaction amongst the participants (Cramer & Mahoney2001). Indeed, Phillips and Glasziou (2004) have shownjournal clubs that are driven by a question relating to thepractice situation that learners find themselves in and thatfocus on appraising the evidence and generating writtenoutput are considered to be more successful than others indeveloping the key skills required for EBP.

It is evident, however, that teaching the wider skillsassociated with EBP will not necessarily bring about achange in behaviour. Several studies of different profes-sional groups have shown that EBP educational interven-tions may well improve knowledge, but not necessarilyapplication in the clinical setting (Forsetlund et al. 2003;Stevenson et al. 2004; McCluskey & Lovarini 2005). Itis now recognised that there is a need to integrate EBPteaching within the practice setting, thus enabling stu-dents to identify topics that have arisen from uncertaintyaround issues from their own practice and from case con-ferences (Haines & Nicholas 2003; Portney 2004). Theimportance of integrating the teaching of evidence-basedmedicine within the practice setting has been the subjectof a systematic review that showed integrated courses ledto a greater improvement in critical appraisal skills, atti-tudes to evidence-based medicine and behaviour changereflected in reading habits and how information resourceswere used (Coomarasamy & Khan 2004).

The findings of a case study reported by Morris andMaynard (2009) supported results from other studies inhighlighting the importance of integrating the teaching and

learning of EBP in the clinical environment (Mangrulkaret al. 2002; Coomarasamy & Khan 2004; Portney 2004).Morris and Maynard (2009) also identified the difficultiesfaced by preregistration nursing students and their mentorsin accessing the electronic resources needed to search andretrieve relevant research evidence; thus highlighting somepractical issues around integrating the EBP process in aclinical environment. Several approaches to overcomingsuch difficulties have arisen with the rapid advances in thedevelopment of mobile technologies.

The use of handheld computers containing concisesummaries of evidence- and decision-making tools wasseen as one way of overcoming this problem and of pro-moting the practice of evidence-based medicine anywhere(Ebell & Rovner 2000). A study by Leung et al. (2003)looked at the use of a handheld computer clinical decisionsupport tool, coupled with a brief teaching intervention,to improve the learning of evidence-based medicine in agroup of medical students. They found that the provisionof a clinical decision support tool on a mobile device, suchas a personal digital assistant, had the potential to improvethe learning of evidence-based medicine, increase currentand future use of evidence, and enhance students’ confi-dence in the clinical decision-making process.

In an attempt to improve access to information in theclinical setting, some health care practitioners are lookingto mobile handheld devices to increase the speed of ac-cess to EBP resources that will facilitate clinical decisionmaking (Lu et al. 2005; Shannon et al. 2005; Cocosila &Archer 2006; Krauskopt 2006).

There has been little published work on the impact ofusing mobile devices within preregistration education fornursing and the allied health professionals; and the re-search that has been published has focused primarily onimplementation and use, rather than outcome(s) (Eastes2001; Goss and Carrico 2002; Huffstutler et al. 2002;Scordo et al. 2003). Typical benefits that were reported in-cluded the usefulness in the clinical setting in the contextof access to information and resources and the mobilityand size of the device (White et al. 2005).

Building on the findings from our previous study, in par-ticular the difficulty students had in accessing electronicresources in the workplace, the aim of this pilot project wasto test the feasibility and acceptability of the use of a mobiledevice to access EBP resources in the clinical setting.

METHODS

DesignA pretest/posttest design was used in this pilot study, whichwas undertaken in the southwest of England.

206 Fourth Quarter 2010 �Worldviews on Evidence-Based Nursing


Selection of a Mobile DeviceThere were several key considerations regarding the se-lection of a mobile device in terms of the present projectand also future use. Key considerations included the va-riety and geographical spread of clinical placements andthe local National Health Service (NHS) service providers.The student groups selected for this study have clinicalplacements throughout the southwest of England and thecommunication networks in these areas are often poor andvery patchy, thus limiting the communication technologiesthat could be relied upon. Furthermore, within the NHSservice providers themselves, not all staff, let alone stu-dents, has guaranteed access to a computer workstationor even access to the Internet when there was access to acomputer. The most important feature for this study was adevice that had mobile network communications such asgeneral packet radio service (GPRS), which would allowstudents to access resources wherever they had a phonesignal.

In addition, it was also important that the device wascapable of displaying content clearly with good accessibil-ity features such as zooming of text. The ability to transferdocuments onto the device with minimal modification orediting was also seen as important, as was the ability toallow students to make notes and if required transfer theseonto a workstation.

Taking the above considerations into account, the Fac-ulty Learning Technologist determined that the HP iPAQwas the best all-around device to cater for our needs. TheHP iPAQ supports the following file formats: MicrosoftWord, Excel, PowerPoint, and Adobe portable documentformat (PDF), all commonly used within the university;windows media video and audio files are also supported.Furthermore, it was felt that the design of the device wasbalanced between reasonable portability and good screensize. Along with this, it has many communication tech-nologies built in, including wireless Internet and GPRSmobile Internet. Whilst not specifically required for thisproject, the device also has additional features such as abuilt-in camera, Bluetooth, and global positioning system(GPS), making the device extremely flexible. A handoutfor the students that illustrated the key functions of thedevice was provided by the project team. Ten iPAQs wereprovided for the study by the funding organisation.

Project Web Page to Support LearningThe Web site designed for our earlier study (Morris & May-nard 2009) was modified to primarily include Web links tokey EBP resources on developing search strategies, accessto and critical appraisal of clinical guidelines, and preap-praised summaries of primary research (http://www.ebp-case-study.org.uk). To ensure that the Web sites were opti-

mised for display on the iPAQ, a tool developed by Googlewas used that automatically rescaled text and removed ta-bles or frames that made pages too wide to display or pre-vented the page from displaying at all. An additional benefitof optimising the Web pages in this way was faster loadingtimes for the Web pages and significantly smaller down-loads, resulting in a more cost-effective use of the iPAQ.The Web site used was http://www.google.com/gwt/n. Thelink to our project Web site was placed on each of theiPAQs used by the students.

Identification of Programmes and Timingof ImplementationA review of placement information for students on theadult branch nursing, physiotherapy, occupational ther-apy, and dietetics programmes indicated that students onthe physiotherapy and adult branch nursing programmeswould be most suitable due to the timing and length oftheir placements in the 2nd and 3rd years. For the project,students were required to be in practice for 4 to 5 weeks.Due to the timeframe of the project, the fieldwork neededto be undertaken between January and July 2008.

SampleAll 2nd- (n = 49) and 3rd-year (n = 69) physiotherapystudents, and 2nd- (n = 48) and 3rd-year (n = 27) adultbranch nursing preregistration students based at the mainuniversity campus were contacted by letter, which outlinedthe aims and procedure of the study and asked if theywould be willing to take part. E-mail confirmation wasreceived from those students interested in taking part inthe study; this was taken as informed consent.

EthicsEthical committee approval for the study was obtainedfrom the University Faculty Ethics Committee.

Data CollectionThe focus of the pilot project was on the feasibility andacceptability of using a mobile device in the practice set-ting in order to access EBP resources. To test the feasibilityof using the device in practice, and the acceptability tothe students, two questionnaires were developed for thestudy based upon the information gained from the reviewof the literature and also the previous work of the authors(Morris & Maynard 2009). Face and content validity wereestablished as part of the development process. One of thequestionnaires was designed to describe the students’ expe-riences of using the iPAQ in the context of the project andwas based on some of the items from the Mobile Phone Us-ability Questionnaire (Ryu & Smith-Jackson 2005). Thisquestionnaire contained 10 questions that addressed the



students’ previous experience of using a mobile device,12 Likert-style statements to quantify their view of theease of use of the iPAQ, and then questions specificallyrelated to the project with regard to the EBP sites theyaccessed, and their opinion about the advantages and dis-advantages of using the iPAQ in the context of the project.The second questionnaire was designed to evaluate stu-dents’ perceptions of the development of their EBP skillsas a consequence of participating in the project. This ques-tionnaire contained questions about the ease of selectinga guideline for critical appraisal, the extent to which par-ticipating in the study contributed to the management ofthe type of patients that were the focus of the selectedguideline, and questions that related in the main to the de-velopment of key EBP skills. A copy of both questionnairescan be obtained from the authors upon request.

Data AnalysisDescriptive analyses were undertaken on the informationcollected from both questionnaires. The focus was on re-porting students’ perceptions of the feasibility and accept-ability of using a mobile device in the clinical setting.

EBP ActivityAn EBP activity, which involved the appraisal of a clinicalguideline identified by the students, was introduced to pro-vide the vehicle for testing the feasibility and acceptabilityof the mobile device in a standardised manner.

Students were first required to identify a clinical guide-line related to an aspect of care in their practice area inconsultation with their practice mentor as necessary andto e-mail the title of the guideline to either JM or VM alongwith a brief rationale for selection. Students then criti-cally appraised the selected guideline using the Appraisalof Guidelines Research and Evaluation (AGREE) appraisalinstrument (AGREE Collaboration n.d.). This process alsorequired the students to check the evidence base of the rec-ommendations contained within the guideline to ensurethe recommendations were appropriate and based uponup-to-date research evidence, and to indicate whether theguideline should be (1) strongly recommended, (2) rec-ommended with provisos or alterations, (3) not recom-mended, or (4) that they were unsure. The students wereasked to send the completed guideline appraisal spread-sheet to either JM or VM.

ImplementationThe fieldwork took place in three phases according to whenthe students were in practice. The first phase involved 2nd-year students from the adult branch nursing program, thesecond phase involved 2nd-year physiotherapy studentsand 3rd-year adult branch nursing students, and the final

phase involved 3rd-year physiotherapy students. We had10 iPAQs available to us for the duration of the project,which limited the numbers of students we could recruit ateach phase.

For each phase, two meetings were arranged betweenthe students and the project team. The first meeting tookplace 2 or 3 weeks prior to the students starting theirplacement and was scheduled during a theory day whenthe students were at the university. At this meeting thenature and purpose of the project was further outlined tothe students and training provided in use of the iPAQ.The students were shown the Web page designed for theproject and the key electronic resources were identified.Guidance was also provided on the kind of practice-basedguideline that would be suitable for the project, and howto use the AGREE instrument that was to be used to crit-ically appraise the guideline (AGREE Collaboration n.d.).The second meeting, at which the two evaluation question-naires were completed, took place as soon after the end ofthe placement period as possible when the students wereattending the University.

RESULTS

ParticipantsA total of 30 students took part in the study, of whom19 completed the study and 11 withdrew due to pres-sures of work and placement commitments. Five 2nd-yearand four 3rd-year students from the adult branch nurs-ing programme, and three 2nd-year and seven 3rd-yearphysiotherapy students completed the study. The nurs-ing students were either 5 or 6 weeks into their place-ment when the project started, and all were in practice inone of the following acute settings: acute medicine, minorinjuries unit, orthopaedic theatres, gynaecology, or car-diac intensive care. The physiotherapy students were justabout to go out into placement when the project startedand were in practice in one of the following areas: respira-tory/medical wards, neurological rehabilitation, commu-nity learning disabilities, elderly care, intensive care unitand surgical wards, acute stroke ward, musculoskeletaloutpatients, or physiotherapy outpatients.

Usability of the iPAQFour of the 19 students had used an iPAQ prior to thisproject. Thirteen of the students “rarely” used the iPAQthroughout the project because they were concerned aboutpossible theft (n = 3), problems with Internet connec-tion (n = 5), speed of connection, the absence of mo-bile signal where they were staying, and difficulty becauseof the size of the screen. Nine of the students used theiPAQ at home, four in the placement area and four whilst



travelling. The EBP Web sites most often used by the stu-dents were those that related to clinical guidelines (n = 12)such as the National Institute for Health and Clinical Excel-lence, the AGREE collaboration (AGREE), Royal Collegeof Physicians, Scottish Intercollegiate Guideline Network,Chartered Society of Physiotherapy, and the general EBPresources accessible via the project Web page, which stu-dents used to refresh their skills (n = 5). However, most ofthe students reported that they used personal computers(PCs) (either at the university or at home) as opposed totheir iPAQs to access these Web sites and resources. Oneof the students used the iPAQ to access the local hospitalintranet.

The students were asked to list three of the most use-ful aspects associated with using the iPAQ and cited: idealfor EBP; portable with access to the Internet on the moveor whilst away (n = 6); games, phone, and camera (n =3); access to resources on demand (n = 2); and small andcompact (n = 1). In terms of the disadvantages and prac-tical difficulties in using the iPAQ, 10 students indicatedthat they had experienced practical difficulties in usingthe iPAQ, with only 5 students indicating that they hadn’texperienced any problems. Students cited concerns aboutinsurance and losing the device or having it stolen, diffi-culties and slow speed in accessing the Internet, the smallscreen and difficulty reading off the screen, and difficultymoving between Web pages on the iPAQ. Overall, 11 ofthe students did not think there was any “added value” inusing the iPAQ in the way designed for the project. Table 1shows the results relating to the perceived ease of use ofthe iPAQ. It is interesting to note from the informationcontained in Table 1 that the students felt the iPAQ was

TABLE 1Students’ rating of perceived ease of use of the iPAQ

STRONGLY STRONGLYAGREE AGREE DISAGREE DISAGREE

1. I found it easy to learn to operate the iPAQ. 3 9 2 12. I found it easy to move from one application to another using the iPAQ 1 7 4 3

(e.g., Microsoft Word to the Internet).3. I found it easy to access the information I needed for the EBP study from the iPAQ. 1 3 3 44. I had difficulty accessing the Internet using the iPAQ. 5 3 2 55. The iPAQ screen size was adequate. 2 5 3 46. The characters on the screen were easy to read. 1 4 3 27. The amount of information displayed on the screen was adequate. 6 7 28. The organisation of information on the screen was clear. 5 8 29. It was easy to download information from the Internet. 1 2 5 6

10. I found the EBP project page useful for linking to key resources. 3 7 4 111. I felt confident in using the iPAQ whilst on placement. 2 5 612. Overall, the iPAQ was useful. 6 4 5

Note: Figures relate to the number of students; totals do not all sum 19 due to missing information and the fact that not all students used the iPAQ for the project.

relatively easy to use in terms of general basic operation.When questioned about its value in relation to the project,however, the overall view was that the iPAQ was of limitedbenefit.

In terms of the guideline appraisal activity introducedas the exercise for testing the mobile device, nine studentsconsidered it “very easy” to select a guideline from prac-tice and only one student used her mentor to assist withthis. Twelve students thought it “quite easy” and five “veryeasy” to complete the guideline appraisal process using theAGREE instrument. Eleven of the students thought thatappraising the guideline improved their knowledge aboutthe type of case addressed in the guideline in that theywere “able to appreciate the greater range of presentationsfor this condition,” “gained an understanding of the guide-line as well as the challenges in implementing in practice,”and experienced “more in-depth knowledge gained of sub-ject.” One of the physiotherapy students reported that theexercise increased her knowledge about the appropriatetreatment techniques and plan for patients with low backpain; and another reported that it gave her “an awarenessof the amount of literature and relevance to each patientgroup and has highlighted the importance of following andusing guidelines.” Ten students thought that completingthe appraisal activity influenced their approach to the careof their patients “to a great extent” or “to some extent,”commenting that they were “more aware of challenges inimplementing in practice and how the process of creat-ing and/or reviewing guidelines were” and that “it has en-couraged me to look at guidelines more, even if you/youreducator think you know what should be done.” A num-ber of students commented that completing the guideline



appraisal activity enhanced their understanding of the ev-idence base around particular treatment approaches, forexample, “I felt that I had more knowledge into why I’mmanaging the patient in a certain way after looking at thereasoning/evidence behind it,” and that the activity “hasincreased my awareness of the availability of guidelines toaid patient treatment and care.”

Fourteen students indicated that completion of theguideline appraisal activity enhanced their understandingof the relevance of EBP to the clinical situation with com-ments such as “. . . I have a better understanding of theneed for evidence based practice in modern health care,”and that the exercise has “. . . highlighted the importanceof using guidelines when working in practice and impor-tance of keeping up to date with evidence.” Several stu-dents reported that the guideline appraisal process hadenabled them to identify situations in which the evidencefor practice may be limited or questionable. For example,one student commented: “I now understand that the evi-dence available may not be of the highest quality or quan-tity”; “(I) just expect guidelines to be right, when actuallythere are some factors not addressed”; and “the guidelines Ifound were lacking in evidence and as a result had unclearpurpose and no supporting evidence.”

In relation to the development of EBP skills, the stu-dents reported that taking part in the project, specificallycompleting the guideline appraisal activity, helped the de-velopment of their EBP skills around the application ofevidence to practice (n = 14), critical appraisal of evidence(n = 13), search and retrieval of evidence (n = 7), andthe formulation of a searchable question (n = 3). Six ofthe students thought the activity was “of great value” andtwelve thought it was “of some value” in developing theirknowledge around the critical appraisal of practice-basedguidelines. This was reflected in comments such as: “(I)found the AGREE tool informative and quite easy to use”;“(it) has given me a tool to evaluate guidelines in the fu-ture”; “(it) has helped me understand the degree to whichguidelines are developed and processes involved”; “(it)hasmade me think about what I’m reading rather than justaccepting it”; “having never appraised guidelines before, Ifound it a useful practice and one that I may need to usein my continuing career”; and “I feel it has been useful inallowing me to continually question whether what I do isreally evidence based.”

All students who took part completed the appraisal ofa guideline that they had selected from practice and wereable to appraise the guideline using the scoring methodoutlined in the AGREE instrument. Three of the five2nd-year nursing students had identified areas within theguideline where provisos or alterations were required. Forexample, one student commented that the guideline re-

quired more up-to-date research, better referencing andresources to support the guideline. One of the 3rd-yearphysiotherapy students had indicated that she would notrecommend the set of guidelines that she had reviewed onnoninvasive ventilation, commenting that they were verybrief and “difficult to follow with little or no informationon the source . . . implications of use or misuse or targetusers.”

DISCUSSION

The aim of this pilot study was to test the feasibility andacceptability of the mobile device as support for accessingEBP resources in the practice setting. The advantages ofusing mobile devices have been reported by others (e.g.,White et al. 2005; Honeybourne et al. 2006; Johnson 2008)suggesting that such devices could be used to provide as-sistance with clinical decision making in the clinical area.The results from this pilot study suggested that the mainbenefits of using the iPAQ were associated with flexibilityand the fact that it could be used anywhere and at anytimeto access the Internet (e.g., whilst travelling, at home, orin the placement area). However, although students foundthe iPAQ easy to operate, they reported a number of practi-cal difficulties associated with accessing the Internet to ob-tain the information needed for the study and with viewingdocuments due to the way information was organised onthe screen. The main difficulties reported related to prob-lems around the speed of connection, difficulties due to thesize of the screen, and the associated difficulties with read-ing research articles, all of which have been cited elsewhere(Cocosila & Archer 2006). Some students highlighted thebenefit of easy access to the hospital intranet and the EBPproject Web site, but the majority of students experiencedsome difficulty with accessing and downloading informa-tion from the Internet whilst in practice, particularly if theinformation was in PDF. It should be noted, however, thatthe majority of the students who took part in this studyrarely used the iPAQ in the clinical setting for the studymainly due to the limitations described earlier. This is sim-ilar to findings reported by Johnston et al. (2004) in theirstudy of medical undergraduates.

In addition to these limitations, a number of studentsreported concerns about possible theft of the device fromthe placement area and consequently did not feel confidentabout using the iPAQ whilst in practice. The issue of con-cern around the theft of portable devices from a practicearea is not something that has been reported by previousauthors.

Overall, although a number of potential benefits of thistype of device were identified, there appeared to be noadded value to using the iPAQ in the way identified for this



study. However, in retrospect, a limitation in the designof the pilot study was that we could not ensure that thestudents completed the guideline appraisal activity usingonly the iPAQ, and a number of the students chose to useeither their home or university PCs.

One clear benefit identified by the students was thevalue of the guideline appraisal process regardless ofwhether this was completed using the iPAQ or PC, witha number of students reporting that the appraisal processhelped in the development of their knowledge around thecritical appraisal of research-based guidelines, and in thedecision-making process around the adoption of guidelinesand recommendations. Several students were sufficientlyconfident with the appraisal process to make recommenda-tions for continued use of the guidelines in their practice.In addition, a number of students reported that this typeof exercise improved their knowledge about the type ofcase addressed by the guideline and highlighted to themthe challenges of implementing research evidence intopractice.

In terms of EBP skill development, the students reportedthat this type of guideline appraisal exercise helped thedevelopment of skills in the search and retrieval of evi-dence, critical appraisal of evidence, and the applicationof evidence to practice. Most important, the majority ofstudents reported the value in developing their knowledgearound the critical appraisal of practice-based guidelinesand that participation enhanced their understanding of therelevance of EBP to the clinical situation. A number of stu-dents were able to identify areas within a guideline whereprovisos or alterations, or further research, was requiredbefore considering utilization in practice.

This was a pilot study and as such any conclusionsdrawn must be tentative and subject to further research.The sample size was relatively small, although we did haverepresentation from different professional groups and dif-ferent year groups, and we were limited to 10 in eachgroup by the number of mobile devices available through-out the duration of the project. Thirty students agreed totake part out of a possible 193, although one-third of thestudents were unable to complete the study due to pres-sures of work and placement commitments. The use ofself-reporting measures to record development of knowl-edge and skills in relation to EBP is recognized as a po-tential limitation of this study and has been highlighted inrelation to much educational research into the effective-ness of EBP teaching initiatives (Hatala & Guyatt 2002).In our previous study, we did use such a measure (Morris& Maynard 2009), but as the focus of this study was onthe feasibility of using a mobile device in a clinical setting,a decision was made to focus on changes in perceptionsand attitudes, which have been argued as being important

precursors to influencing behaviour change in relation toEBP (Johnston et al. 2003).

CONCLUSIONS AND RECOMMENDATIONS

Despite the limited use of the iPAQ throughout the study,the students were able to complete the guideline appraisalprocess and also reported improvements in their knowl-edge and skills in relation to EBP and the appraisal ofclinical guidelines. Whilst one of the objectives of our pi-lot study was to test the use of a mobile device in theclinical setting, it became clear that the students were re-porting additional benefits as a result of taking part inthe study. The physiotherapy students, for example, hadnot previously been introduced to the critical appraisalof clinical guidelines and using the AGREE instrument,thus the project provided an opportunity to develop theseskills in the practice setting. Additionally, some of thenursing students were able to make recommendationsfor improving the guidelines they reviewed, thus demon-strating improved skills and confidence in the appraisalprocess.

Initial discussion with the programme leaders of thosestudents who participated in the study has generated in-terest in this kind of initiative and we are now consideringhow it could be formally integrated into the curriculum toensure the continued development of EBP skills in prereg-istration health care students and to provide a larger baseupon which to evaluate the impact on learning.

ReferencesAGREE Collaboration. (n.d.). Appraisal of Guidelines

for Research and Evaluation (AGREE) instrument. Re-trieved November 19, 2007, from http://www.agreecollaboration.org.

Cocosila M. & Archer N. (2006). Physician interactionwith mobile and wireless handheld devices: Data en-try and retrieval challenges. International Journal ofHealth care Technology and Management, 7(3/4), 233–251.

Coomarasamy A. & Khan K. (2004). What is the evidencethat postgraduate teaching in evidence based medicinechanges anything? A systematic review. British MedicalJournal, 329, 1017–1021.

Cramer J. & Mahoney M. (2001). Introducing evidencebased medicine to the journal club, using a structuredpre- and post-test: A cohort study. Medical Education, 1,6–9.

Dawes M., Summerskill W., Glasziou P., Cartabellotta A.,Martin J., Hopayian K., Porzsolt F., Burls A. & OsborneJ. (2005). Sicily statement on evidence-based practice.



BMC Medical Education, 5(1), doi: 10.1186/1472-6920-5-1. Retrieved April 21, 2008, from http://www.bio-medcentral.com/1472-6920/5/5.

Dobrzanska L. & Cromack D. (2005). Developing a jour-nal club in the community setting. British Journal ofCommunity Nursing, 10, 374–377.

Eastes L. (2001). Use of the personal digital assistant forpoint-of-care trauma documentation. Journal of Emer-gency Nursing, 27, 516–518.

Ebell M. & Rovner D. (2000). Information in the palm ofyour hand. Journal of Family Practice, 49, 243–251.

Forsetlund L., Bradley P., Forsen L., Nordheim L.,Jamtvedt G. & Bjørndal A. (2003). Randomised con-trolled trial of a theoretically grounded tailored inter-vention to diffuse evidence-based public health practice.BMC Medical Education, 3, doi: 10.1186/1472-6920-3-2.

Goodfellow L. (2004). Can a journal club bridge the gapbetween research and practice? Nurse Educator, 29, 107–110.

Goss L. & Carrico R. (2002). Get a grip on patient safety.Journal of Infusion Nursing, 25, 274–279.

Haines S. & Nicholas J. (2003). Teaching evidence-basedmedicine to surgical subspecialty residents. Journal ofthe American College of Surgeons, 197, 285–289.

Hatala R. & Guyatt G. (2002). Evaluating the teaching ofevidence-based medicine. Journal of American MedicalAssociation, 288(9), 1110–1112.

Honeybourne C., Sutton S. & Ward L. (2006). Knowledgein the palm of your hands: PDAs in the clinical setting.Health Information and Libraries Journal, 23, 51–59.

Huffstutler S., Wyatt T. & Wright C. (2002). The use ofhandheld technology in nursing education. Nurse Edu-cator, 27, 271–275.

Johnson C. (2008). Nurses and the use of personal digitalassistants (PDAs) at the point of care. Design of ElectronicText, 1, 1–7.

Johnston J.M., Leung G., Fielding R., Tin K., Ho L.-M.& Ho L-M. (2003). The development and validation ofa knowledge, attitude and behaviour questionnaire toassess undergraduate evidence-based practice teachingand learning. Medical Education, 37, 992–1000.

Johnston J.M., Leung G., Tin K., Ho L.-M. & Lam W.(2004). Fielding R. Evaluation of a handheld clinicaldecision support tool for evidence-based learning andpractice in medical undergraduates. Medical Education,38, 628–637.

Khan K. & Gee H. (1999). A new approach to teaching andlearning in journal club. Medical Teacher, 21, 289–293.

Krauskopt P.B. (2006). Even techno-phobic NPs can usePDAs. The Nurse Practitioner, 31, 48–52.

Leung G., Johnston J., Tin K., Wong I., Ho L.-M., LamW. & Lam T.-H. (2003). Randomised controlled trial

of clinical decision support tools to improve learningof evidence based medicine in medical students. BritishMedical Journal, 327, 1090–1095.

Lu Y.C., Xiao Y., Sears A. & Jacko J.A. (2005). A reviewand a framework of handheld computer adoption inhealth care. International Journal of Medical Informatics,74, 409–422.

Mangrulkar R., Saint S., Chu S. & Tierney L. (2002). Whatis the role of the clinical “pearl?” The American Journalof Medicine, 113, 617–624.

McCluskey A. & Lovarini M. (2005). Providing educa-tion on evidence-based practice improved knowledgebut did not change behaviour: A before and after study.BMC Medical Education, 5, doi:10.1186/1472-6920-5-40.

McQueen J., Miller C., Nivison C. & Husband V. (2006).An investigation into the use of a journal club forevidence-based practice. International Journal of Ther-apy and Rehabilitation, 13, 311–317.

Morris J. & Maynard V. (2009). The feasibility of introduc-ing an evidence based practice cycle into a clinical area:An evaluation of process and outcome. Nurse Educationin Practice, 9, 190–198.

Nollan R., Fineout-Overholt E. & Stephenson P. (2005).Asking compelling clinical questions. In B. Melnyk &E. Fineout-Overholt (Eds.), Evidence-based practice innursing and health care. A guide to best practice (1st ed.,pp 25–37). Philadelphia, PA: Lippincott, Williams &Wilkins.

Norman G. & Shapiro P. (1998). Effectiveness of in-struction in critical appraisal (evidence-based medicine)skills: A critical appraisal. Canadian Medical AssociationJournal, 158, 177–181.

Owen S., Wheway J. & Anderson M. (2001). The use ofa journal club and clinical seminars on a 4-year under-graduate, pre-registration mental-health nursing degree.Nurse Education Today, 21, 297–303.

Phillips R. & Glasziou P. (2004). What makes evidence-based journal clubs succeed? Evidence Based Medicine,9, 36–37.

Portney L. (2004). Evidence-based practice and clinical de-cision making: It’s not just the research course anymore.Journal of Physical Therapy Education, 18, 46–51.

Ryu Y.S. & Smith-Jackson T.L. (2005). Reliability andvalidity of the Mobile Phone Usability Questionnarie(MPUQ). Journal of Usability Studies, 2, 39–53.

Scordo K., Yeager S. & Young L. (2003). Use of personaldigital assistants with acute care nurse practitioner stu-dents. AACN Clinical Issues, 14, 350–362.

Shannon T., Feied C., Smith M., Handler J. & GillamM. (2005). Wireless handheld computers and voluntaryutilization of computerized prescribing systems in the



emergency department. Journal of Emergency Medicine,31, 309–315.

Stevenson K., Lewis M. & Hay E. (2004). Do physiothera-pist’ attitudes towards evidence-based practice change asa result of an evidence-based educational programme?Journal of Evaluation in Clinical Practice, 10, 207–217.

Taylor R., Reeves B., Ewings P., Binns S., Keast J. &Mears R. (2000). A systematic review of the effectivenessof critical appraisal skills training for clinicians. MedicalEducation, 34, 120–125.

White A., Allen P., Goodwin L., Breckinridge D., DowellJ. & Garvy R. (2005). Infusing PDA technology intonursing education. Nurse Educator, 30, 150–154.


Pilot Study to Test the Use of a Mobile Device in the Clinical Setting to Access Evidence-Based...

Documents

Transcript of Pilot Study to Test the Use of a Mobile Device in the Clinical Setting to Access Evidence-Based...