Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support
-
Upload
savannah-holland -
Category
Documents
-
view
44 -
download
8
description
Transcript of Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support
![Page 1: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/1.jpg)
Leading a Horse to Water:Using Automated Reminders to Increase
Use of Online Decision Support
James J. Cimino and Dmitriy Borovtsov
NIH Clinical Center
andDepartment of Biomedical Informatics
Columbia University College of Physicians and Surgeons
![Page 2: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/2.jpg)
Knowledge Resources are Underused
Clinicians (physicians) with questions in clinical practice:
– sought answers only 45% of the time
– found answers only 34% of the time
– used online resources 18% of the time
* Ely JW, Osheroff JA, Maviglia SM, Rosenbaum ME. Patient-care questions that physicians are unable to answer. J Am Med Inform Assoc. 2007 Jul-Aug;14(4):407-14.
![Page 3: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/3.jpg)
Health Resources
• Available from main menu
• Static list of resources
• No assistance with retrieval
![Page 4: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/4.jpg)
![Page 5: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/5.jpg)
![Page 6: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/6.jpg)
Infobuttons
• Links inserted next to clinical information
• Resources selected based on likely information need
• Links are customized to assist with retrieval
![Page 7: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/7.jpg)
![Page 8: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/8.jpg)
Log File of HR vs IB 2006
2006 Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
7000
6000
5000
4000
3000
2000
1000
0
Monthly Usage
![Page 9: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/9.jpg)
Log File of HR vs IB 2006, 2007
2006 2007Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
7000
6000
5000
4000
3000
2000
1000
0
Monthly Usage
![Page 10: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/10.jpg)
Educational Interventions
• Orientation for new housestaff
• Orientation for new medical students
• Infobutton exercises for new medical students
![Page 11: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/11.jpg)
Educational Interventions
2006Medical StudentsHousestaff
Jun Jul Aug Sep Oct Nov Dec
Monthly Usage
450
400
350
300
250
200
150
100
50
0
![Page 12: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/12.jpg)
Educational Interventions
2006 2007Medical StudentsHousestaff
Jun Jul Aug Sep Oct Nov Dec
450
400
350
300
250
200
150
100
50
0
Monthly Usage
![Page 13: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/13.jpg)
![Page 14: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/14.jpg)
![Page 15: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/15.jpg)
Usage of IB vs HR in Inpatient Meds
Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
1400
1200
1000
800
600
400
200
0
Monthly Usage
![Page 16: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/16.jpg)
Usage of IB in Outpatient Meds
Health ResourcesInfobuttons
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
800
700
600
500
400
300
200
100
0
Monthly Usage
![Page 17: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/17.jpg)
Hypothesis
Users of the Health Resources page in a particular context will be especially receptive to a suggestion to use Infobuttons in the same context.
![Page 18: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/18.jpg)
SendReminder
No
StopYesReminder
Sent in past2 months?
No
StopYesIB Used
in past2 months?
InterventionHR used in OM
HR = Health Resources PageOM = Outpatient Medications
![Page 19: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/19.jpg)
E-MailDate: Fri, 2 Mar 2007 06:00:11 -0500 (EST)From: James Cimino <[email protected]>To: [email protected]: Getting Drug Information while Using WebCIS
Dear WebCIS User:
I am writing to let you know about a handy feature in WebCIS called the "infobutton". You will see it in places like the Outpatient Medication list - it is a little purple circle with a white letter "i". If you click on it, it will give you a list of topics that you can select to get more information about the drug next to the infobutton.
I hope you'll find it useful.
--Jim Cimino, Infobutton Project Manager
![Page 20: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/20.jpg)
Reminders Sent
• 522 messages• 371 users
– 80 attending physicians– 189 housestaff– 29 nurses– 24 students– 49 other/unknown
• 173 users received two messages• 1 user received three messages• Average 89.9 days between messages
![Page 21: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/21.jpg)
Usage of Infobuttons by Reminder Recipients
• 371 recipients of 552 reminders• 111 (20.1%) eventually used IBs in OM• 52 (9.4%) “early responders”
– 26 housestaff– 18 attending physicians– 8 other/unknown
• 201 used either IB or HR within one month• 25.9% “early responders with opportunity”• 9 early responders used IBs 8 to 97 times in
subsequent months
![Page 22: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/22.jpg)
Intervention versus Control
552 Intervention525 Control
Days to First Use of Infobuttons after First Use of Health Resources Page
P<0.05 by Day 4
![Page 23: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/23.jpg)
Effect of delay
0
100
200
300
400
1-2Days
3-5Days
>5Days
RemindersSent
EarlyResponders
![Page 24: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/24.jpg)
Adverse Effects
• 522 reminders
• 12 replies– All were positive– Most were thanking us for infobuttons
![Page 25: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/25.jpg)
Discussion
• Traditional training method appears to have limited effect
• Simple, spam-like reminder appears to have an effect on a substantial fraction (26%) of users
• Sustained effect in some (17%) of the early responders
![Page 26: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/26.jpg)
Limitations
• Impersonal message
• Timing of reminder less-than ideal
• However: Logfile+e-mail approach is application independent
• Quasi-experimental design
• Convenience sample of controls
• However: Strong temporal association between intervention and outcome
![Page 27: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/27.jpg)
Conclusions
• Automated, context-specific reminders are technically feasible, requiring few resources
• Reminders to use online information resources appear to be effective
• Timing of message is critical to success
![Page 28: Leading a Horse to Water: Using Automated Reminders to Increase Use of Online Decision Support](https://reader035.fdocuments.in/reader035/viewer/2022062304/568138a8550346895da06604/html5/thumbnails/28.jpg)
Acknowledgments
This work was supported in part by:National Library of Medicine Grant R01LM07593
NIH Clinical Center intramural research funds
National Library of Medicine intramural research funds
The authors also thank:Dr. Jianhua Li for technical support
Dr. Rick Gallagher for log files
Dr. Krystl Haerian for statistical support