BMJ Open · visual field testing, electroretinography), numerical data (e.g. autorefraction,...

$: BMJ Open · visual field testing, electroretinography), numerical data (e.g. autorefraction, keratometry, and biometry) and ophthalmic image data (e.g. fundus photography, optical$
For peer review only

Assessing the efficacy of the electronic patient record system EDeR: Implementation study - Study protocol

Journal: BMJ Open

Manuscript ID: bmjopen-2012-002478

Article Type: Protocol

Date Submitted by the Author: 11-Dec-2012

Complete List of Authors: Job, Oliver; Eye Clinic Cantonal hospital of Lucerne 6000 Lucerne 16, Switzerland, Bachmann, Lucas; medignition Inc, Schmid, Martin; Eye Clinic Cantonal hospital of Lucerne 6000 Lucerne 16, Switzerland, Thiel, Michael; Eye Clinic Cantonal hospital of Lucerne 6000 Lucerne 16, Switzerland, Ivic, Sandra; medignition Inc,

<b>Primary Subject Heading</b>:

Medical management

Secondary Subject Heading: Ophthalmology

Keywords:

Information management < BIOTECHNOLOGY & BIOINFORMATICS, Information technology < BIOTECHNOLOGY & BIOINFORMATICS, Cataract and refractive surgery < OPHTHALMOLOGY, Glaucoma < OPHTHALMOLOGY, Neuro-ophthalmology < OPHTHALMOLOGY, Medical retina < OPHTHALMOLOGY

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on June 5, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2012-002478 on 10 A

pril 2013. Dow

nloaded from

http://bmjopen.bmj.com/


1

Assessing the efficacy of the electronic patient record system EDeR:

Implementation study - Study protocol

Oliver Job 1, Lucas M. Bachmann

2 *, Martin K Schmid

1, Michael A. Thiel

1, Sandra Ivic

2

[1]

Eye Clinic

Cantonal hospital of Lucerne

6000 Lucerne 16, Switzerland

[2]

Medignition Inc. Research Consultants

Postrasse 15, 6300 Zug, Switzerland

* Corresponding Author

Prof . Lucas M. Bachmann, MD, Ph.D. Medignition Inc, Postrasse 15, 6300 Zug, Switzerland

e-mail Adresses:

Oliver Job: [email protected]

Lucas M. Bachmann: [email protected]

Martin K. Schmid: [email protected]

Michael A. Thiel: [email protected]

Sandra Ivic: [email protected]

Page 1 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on June 5, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2012-002478 on 10 April 2013. D

ownloaded from



2

ABSTRACT

Introduction Despite many innovations in information technology, many clinics still rely on

paper based medical records. Critics claim that they are hard to read, because of illegible

handwriting and uncomfortable to use. Moreover, a chronological overview is not always

easily possible, content can be destroyed or get lost. There is an overall opinion that electronic

medical records should solve these problems and improve physicians’ efficiency, patients’

safety and reduce overall costs in practice. However, to date, the evidence supporting this

view is sparse.

Methods and Analysis In this protocol we describe e a study exploring differences in speed

and accuracy when searching clinical information using the paper based patient record or the

EDeR. Designed as a randomised vignette study, we hypothesise that the EDeR increases

efficiency, i.e. reduces time on reading the patient history and looking for relevant

examination results, helps finding mistakes and missing information quicker and more

reliably. In exploratory analyses we aim at exploring factors associated with a higher

performance.

Ethics and Dissemination The ethics committee of the Canton Lucerne, Switzerland,

approved this study. We presume that the implementation of the electronic medical record

software EDeR will have a positive impact on the efficiency of the doctors, which will result

in an increase of consultations per day. We believe that the results of our study will provide a

valid basis to quantify the added value of an electronic medical record system in an

ophthalmologic environment.

238 words

Page 2 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3

Article focus

To compare the gain in

• Efficiency

• Accuracy

When introducing an electronic patient record system vs. paper based records

Key messages

The chosen design

• has the potential to validly address the added value of an electronic patient record

system

• allows extrapolating the economic gain of introducing the system

Strengths and limitations

Study design allows examining the effects

• in a realistic clinical setting

• using relevant clinical problems

• using data of real patients

Economic consequences can only be addressed indirectly.

Page 3 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



4

BACKGROUND

Despite substantial effort to promote electronic data management in patient care, a large

proportion of healthcare providers still rely on paper based patient records. The American

Academy of Ophthalmology performed a survey concerning the adoption rate of electronic

medical records within their members and found an adoption rate of only 12%1.

The main problem with paper based patient records is the handwriting and the missing

overview of treatment and patient history. The handwriting is sometimes hard to read for the

physician in charge and the missing overview could potentially lead to needless examinations

and errors in diagnosis, prescription and treatment2, because of the absence of consistent

information on changes in state of health, treatment and the parameters measured. As a result

of these problems, there was the idea to support clinical daily routine with the help of

computers, as it is reality in many fields of service.

In the last decade, there was enormous development and investment in the field of

electronically supported medical records. It seems to be a key strategy to improve health care

in the different specialties3. Electronic medical records are supposed to improve physicians’

workflow and to offer a better overview on patient’s history. Today many authors believe that

this innovation should improve safety and reduce costs4.”A better understanding of the impact

of the electronic medical records on workflows is vital to understanding what the technology

really does offer that is new and unique” 5.

The cantonal hospital of Lucerne has developed new software addressing the special needs of

the ophthalmologic clinic. The idea was to create a tool, which offers a complete overview on

the patient history once a patient has been referred from another ophthalmologist or general

practitioner. The software includes physician’s drawings, imaging, measurement data (e.g.

visual field testing, electroretinography), numerical data (e.g. autorefraction, keratometry, and

biometry) and ophthalmic image data (e.g. fundus photography, optical coherence

Page 4 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



5

tomography). These data are essential to support clinical diagnosis to track disease

progression, and to plan treatment9.

In order to explore the impact on productivity of physicians using electronic health records, it

has been shown, that there is a statistically significant increase in the average monthly patient

visit volume of 9 visits per examined provider per month3, or the reduction of time spent on

administration by nursing staff6. The collected data also could be used for clinical research

and clinical studies7. In general, there is an existing hypothesis, that EMR should improve

patient’s safety, but there are neither data nor suggestions available, how this should be

measured. Overall however, there are only few studies, which show evidence, that the

implementation of electronic medical records has a major impact on the efficiency 4 8

,

especially in the field of ophthalmology. There seems to be also the demand for more studies

with prospective and randomized experimental designs (Fontaine et al), instead of surveys.

Possible barriers for a successful use of EMR are costs and time loss for implementation,

issues concerning the security and privacy of the patient’s data, lack in training and support of

the staff using the EMR software4 8

. The main objective should be the elimination of obstacles

for a successful implementation.

In this protocol we describe e a study exploring differences in speed and accuracy when

searching clinical information using the paper based patient record or the EDeR. Designed as

a randomised vignette study, we hypothesise that the EDeR increases efficiency, i.e. reduces

time on reading the patient history and looking for relevant examination results, helps finding

mistakes and missing information quicker and more reliably. In exploratory analyses we aim

at exploring factors associated with a higher performance.

Page 5 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



6

METHODS / DESIGN

Study design

The study is planned as a prospective simulated study.

Study location

The study is planned to take place at the cantonal hospital of Lucerne in the clinic of

ophthalmology.

Study population

The study population will be divided in two groups. One group will consist of senior

physicians of the ophthalmologic clinic of the cantonal hospital in Lucerne, Switzerland. The

other group will be formed by junior doctors, who are usually younger and more computer

adepts, but have less clinical experience.

Inclusion criteria

Ophthalmologist of the cantonal hospital Lucerne eye clinic who voluntarily agreed to

participate in the study

Experimental design

The study is planned as a prospective, randomised study using patient vignettes. The aim is to

measure the time needed to solve a given task and the accuracy of the result. For each out of

five different ophthalmologic sub-specialities (Neuro-ophthalmology, Corneal, Glaucoma,

Uveitis and orthoptics) we will create five tasks based on a patient example. Thus, we will

create 25 pairs of cases either represented in paper or electronic form.

We defined the following five task types: i) To detect the medication dose 3 months after

onset of treatment ii) Decision about the adjustment of a treatment based on the impairment

Page 6 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



7

or persistent state of data monitored iii) Question about the acquisition of underlying diseases.

iv), Question about the detection of implausible data, which could be important for the further

treatment, v) Question about the completeness of additional information. The Table provides

examples of tasks used for Neuro-Ophthalmology.

The patient model will come from original paper forms to assure that cases are comparable to

the real life situation. The cases, slightly modified, will be transferred from paper into the

electronic patient record system. During the experiment, a research assistant will monitor

physicians and measure the time required to complete each task. Each physician from each

division has to solve all of the tasks. The evaluation will take place at two time points;

immediately after implementation of EDeR and one year after.

Detailed Research Plan

a) Creating the patient vignettes

Of each specialization we will collect five typical patient records on paper.

b) Checking for plausibility and technical feasibility

After creation of the vignettes, two to three individuals will be selected to evaluate the tasks.

They will solve the tasks and afterwards fill out a questionnaire, which will consist of

questions about the difficulty, comprehensibility and technical feasibility of the task.

c) Design matrix

We will generate random sequence coding whether the task is started with the paper or the

electronic version of the patient record. A second random sequence will be generated to code

for the sequence with which tasks will be presented to the participants. These two measures

will eliminate order effects.

Page 7 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8

d) At the assessment

The participant will be sitting in a consultation room and the assessor will be seated next to

him or her to avoid distractions.

We will design a reply form into which each participant enters his or her response to the task.

This reply form will be handed over to the assessor after completion. Time will be taken by

the assessor after completing each entry into this form.

There will be no communication between participant and assessor

e) The role of the assessor

1. The assessor will measure the time the participant needs to complete the reply form for

each task and each mode (paper based, electronic).

2. The assessor will take care of a protocol based procedure.

3. If any inconveniences of technical or administrative manner occur, the assessor will solve

them.

f) End of the assessment

All forms will be collected and the data will be entered in anonymised fashion into a

spreadsheet for further statistical analysis.

Sample size calculation

By implementing a novel electronic health record system it is possible to observe a

productivity gain of the physicians3. We hypothesise that our productivity gain should be one

additional consultation per day assuming that one consultation lasts about 20 minutes and a

physician sees 25 patients daily. Comparing the two methods, paper and electronic medical

record, the overall gain in time should be 60 seconds on average when using the electronic

medical record EDeR.

Page 8 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



9

Therefore, we are planning a study of the continuous response variable time from matched

pairs of study subjects. We presume that the difference in the response of matched pairs is

normally distributed with standard deviation 45. If the true difference in the mean time of

matched pairs is 60 seconds, we will need to study 7 pairs of subjects to be able to reject the

null hypothesis that this response difference is zero with probability (power) 0.8. The Type I

error probability associated with this test of this null hypothesis is 0.05.

Data collection

Point of time of data collection

The first data acquisition will take place shortly after the implementation of EDeR. The

defined tasks will be available in paper form and the physicians will fill them out with the

help a medical record, either the paper form or the electronic form.

Additionally, there will be someone from the study personal, who sits with the physicians in

the same room and measure the time they need to complete each task.

Baseline data

Baseline data will consist of information about the physicians’ age, extent of previous training

in ophthalmology, degree of specialization and self-reported level of general computing skills.

Follow-up data

At the second time point, one year after the first assessment, the experiment will be repeated

using the same protocol.

Page 9 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



10

Discussion

This paper describes the rationale, methodology and design of a randomised, prospective

simulation study using patient vignettes based on real patients. This study will compare the

paper based medical records with electronic medical record software EDeR in terms of

efficiency, safety and cost reduction in the daily routine of an ophthalmological clinic. The

experiment will consist of 25 tasks, which will be solved by two groups of participants; senior

and junior doctors.

We presume that the implementation of the electronic medical record software EDeR will

have a positive impact on the efficiency of the doctors, which will result in an increase of

consultations per day. We believe that the results of our study will provide a valid basis to

quantify the added value of an electronic medical record system in an ophthalmologic

environment.

Page 10 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



11

References

1. Chiang MF, Boland MV, Margolis JW, Lum F, Abramoff MD, Hildebrand PL. Adoption

and perceptions of electronic health record systems by ophthalmologists: an American

Academy of Ophthalmology survey. Ophthalmology 2008;115(9):1591-7; quiz 97 e1-

5.

2. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med

2003;348(25):2526-34.

3. Cheriff AD, Kapur AG, Qiu M, Cole CL. Physician productivity and the ambulatory EHR

in a large academic multi-specialty physician group. Int J Med Inform;79(7):492-500.

4. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of

eHealth on the quality and safety of health care: a systematic overview. PLoS

Med;8(1):e1000387.

5. Vishwanath A, Singh SR, Winkelstein P. The impact of electronic medical record systems

on outpatient workflows: a longitudinal evaluation of its workflow effects. Int J Med

Inform;79(11):778-91.

6. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on

time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc

2005;12(5):505-16.

7. Murphy EC, Ferris FL, 3rd, O'Donnell WR. An electronic medical records system for

clinical research and the EMR EDC interface. Invest Ophthalmol Vis Sci

2007;48(10):4383-9.

8. Fontaine P, Ross SE, Zink T, Schilling LM. Systematic review of health information

exchange in primary care practices. J Am Board Fam Med;23(5):655-70.

9. Chiang MF, Boland MV, Brewer A, Epley KD, Horton MB, Lim MC, et al. Special

requirements for electronic health record systems in ophthalmology.

Ophthalmology;118(8):1681-7.

Page 11 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



12

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Lucas M. Bachmann was involved in the conception and design of the study, drafted the

protocol, supervised the revisions and approved the final manuscript. Oliver Job was involved

in conception and design of this study, revised the draft critically for intellectual content and

approved the final revised manuscript. Martin K Schmid and Michael A. Thiel were involved

in the conception of the study, revised the draft critically for intellectual content and approved

the final manuscript. Sandra Ivic was involved in the conception and design of the study,

drafted the protocol and approved the final manuscript.

Acknowledgments

None

Main Sponsors:

Cantonal hospital of Lucerne, 6000 Lucerne 16, Switzerland

Page 12 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13

Table Description of context, task and contrast examined for the example of Neuro-

Ophthalmology

Task 1 Context Regular control visit Task To detect the medication dose after a certain of time after onset of treatment Contrast EdER: 6x daily 3 dragées à 60 mg Mestinon®

Paper record: 15x daily 3 dragées à 60 mg Mestinon®

Task 2 Context Regular control visit Task Decision about the adjustment of a treatment based on the impairment or persistent state of data

monitored - "In a case of Myasthenia gravis, the decision to adjust the steroid dose is made 4 weeks after starting therapy. If there has been a significant improvement in eye position, recorded with the coordimetry of Hess Weiss, the steroids can be tapered off. Is the eye position unchanged, or even worse, the steroid dose should be increased or an additional medication should be introduced. In the present case: Would you increase or decrease the dose of the steroids 4 weeks after onset of the therapy?"

Contrast EdER: There has been a significant improvement in the eye position; therefore the steroids can be tapered off. Paper record: The eye position is unchanged; the steroid dose should be increased.

Task 3 Context Post-op medication Task Question about the detection acquisition of underlying diseases.

Question: "In case of an allergic reaction to Bactrim® the administration of Diamox® often also produces allergic signs due to a cross-reaction. Diamox® should be avoided in such a situation. Is there a contraindication to administer Diamox® after a cataract surgery in the present case?"

Contrast EdER: The section with the patient history, containing information about allergies and underlying diseases, is blank. Although the patients has a sulfonamide allergy. Paper record: There is a short note about an allergic reaction to Bactrim®.

Task 4 Context Error in documentation Task "Do you find any implausible data regarding visual field recordings during the last 3 years of

documentation in the present case of a patient with a pituitary adenoma?"

Contrast EdER: The series of visual fields will show a constant bitemporal field Paper record: The series of visual fields will show one hemianopic visual field within the series of bitemporal fields. This change in just one visual field within an otherwise stable series is implausible and indicates an error of documentation)

Task 5 Context Completeness of clinical information Task In a patient with an idiopathic intracranial hypertension the diagnosis is made by the fact of

(bilateral) papilledema without visual afferent defects and raised intracranial opening pressure measured by lumbar puncture. Is it correct to make the diagnosis of an idiopathic intracranial hypertension in the present case?"

Contrast EdER: The necessary information to diagnose idiopathic intracranial hypertension is present. Paper record: No lumbar puncture, diagnosis not possible, because of missing information.

Page 13 of 13


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from




Journal: BMJ Open

Manuscript ID: bmjopen-2012-002478.R1


Date Submitted by the Author: 04-Mar-2013



Medical management

Secondary Subject Heading: Ophthalmology

Keywords:





jopen.bmj.com

/B

MJ O


pril 2013. Dow

nloaded from



1





1, Michael A. Thiel

1, Sandra Ivic

2

[1]

Eye Clinic



[2]





e-mail Adresses:






Page 1 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



2

ABSTRACT


paper based medical records. Critics, however, claim that they are hard to read, because of

illegible handwriting, and uncomfortable to use. Moreover, a chronological overview is not

always easily possible, content can be destroyed or get lost. There is an overall opinion that

electronic medical records should solve these problems and improve physicians’ efficiency,

patients’ safety and reduce overall costs in practice. However, to date, the evidence

supporting this view is sparse.

Methods and Analysis In this protocol we describe a study exploring differences in speed






performance.







238 words

Page 2 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3

Article focus


• Efficiency

• Accuracy


Key messages

The chosen design


system








Page 3 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



4

BACKGROUND




medical records within their members and found an adoption rate of only 12% 1.




and errors in diagnosis, prescription and treatment 2, because of the absence of consistent

information on changes in state of health, treatment and the parameters measured. As a

solution of these problems, there was the idea to support clinical daily routine with the help of






this innovation should improve safety and reduce costs 4.”A better understanding of the

impact of the electronic medical records on workflows is vital to understand what the

technology really does offer that is new and unique”5 6

. However, a systematic review

published in 2008 and covering the evidence until 2005 concluded that none of 20 studies

provided a formal cost effectiveness analysis of electronic patient record systems in general. 7





Page 4 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



5




progression, and to plan treatment 8.



visit volume of 9 visits per examined provider per month 3, or the reduction of time spent on

administration by nursing staff 9. The collected data also could be used for clinical research

and clinical studies 10. In general, there is an existing hypothesis, that EMR should improve



implementation of electronic medical records has a major impact on the efficiency4 11

,


with prospective and randomized experimental designs, instead of surveys 11

.



the staff using the EMR software 4 11

. The main objective should be the elimination of

obstacles for a successful implementation.

In this protocol we describe a study exploring differences in speed and accuracy when






Page 5 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



6

METHODS / DESIGN

Study design


Study location


ophthalmology. The Eye clinic is run by six consultant ophthalmologists, 12 senior

physicians, and 14 junior doctors.

Study population

The study population will be divided into two groups. One group will consist of senior




Inclusion criteria


participate in the study will be included. Each participant will receive a general introduction

on the usage of the EDeR of half an hour prior to the assessment. The training will cover

aspects of data collection and saving very briefly. The training will be such, that the basic

functions of EDeR are known.

Primary outcome

Gain in speed, when performing the task using the EDeR system

Secondary outcome

Accuracy of replies given using the EDeR system vs. the paper record

Page 6 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



7

Experimental design





create 25 pairs of cases either represented in paper or electronic form. The tasks are chosen on

the basis of the relevance in clinical practice.

Type of tasks





treatment, v) Question about the completeness of additional information. Table 1 provides



the real life situation. The full patient record, containing a slight modification for the contrast

examined in a specific task, will be transferred from paper into the electronic patient record

system. During the experiment, a research assistant will monitor physicians and measure the

time required to complete each task. Each physician from each division has to solve all of the

tasks. The evaluation will take place at two time points; immediately after implementation of

EDeR and one year after.

Design Matrix Summary

This is a two group (EDeR vs. paper based) cross-over (randomly assigning the sequence

electronic vs. paper-based per task) study. Time (primary endpoint) and accuracy (secondary

endpoint) are the two outcome variables. Intervention is the EDeR that is compared with the

Page 7 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8

paper-based patient record. We assess tasks within 5 clinical subgroups (without examining

subgroup effects) and perform the experiment in two groups with different ophthalmologic

expertise (junior doctors vs. senior doctors). Each assessor will complete 50 tasks (i.e. 25

tasks based on the electronic patient record and 25 based on the paper patient record).

Electronic and paper-based vignettes are slightly different for each task. If an assessor checks

the same vignette a second time (e. g. in the electronic version), he or she must read the

content again, because the answer is different to the first time (e. g. in the paper-based

version). The material for the paper-based-group comes from real patients records and the

tasks were selected based on clinical relevance. For Summary see Figure 1








c) Design matrix






Page 8 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



9





the assessor after completing each entry into this form. In total, each participant will go

through 25 pairs of tasks. In case of a technical problem with the electronic system, the

assessment will be suspended and the remaining tasks will be completed after restoration of

the system.







them.






productivity gain of the physician 3. We hypothesise that our productivity gain should be one



Page 9 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



10









At the time of writing this protocol 10 senior physicians and 9 junior doctors were willing to

participate in the study.

Data collection




help of a medical record, either the paper form or the electronic form.

Additionally, there will be someone from the study personnel, who sits with the physicians in


Baseline data

Prior to the examination baseline data will be collected of each participant. Baseline data will

consist of information about the physicians’ age, extent of previous training in

ophthalmology, degree of specialization and self-reported level of general computing skills.

Follow-up data

Page 10 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



11


using the same protocol. In the interval between the two measurements, all physicians will be

mainly working with the EDeR system.

Page 11 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



12

Discussion






and junior doctors.





environment.

Page 12 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13

References



Academy of Ophthalmology survey. Ophthalmology 2008;115(9):1591-7; quiz 1597

e1-5.


2003;348(25):2526-34.


in a large academic multi-specialty physician group. Int J Med Inform 2010;79(7):492-

500.


eHealth on the quality and safety of health care: a systematic overview. PLoS Med

2011;8(1):e1000387.



Inform 2010;79(11):778-91.

6. Vishwanath A, Winkelstein P, Singh SR. The impact of electronic health records on

outpatient flow: an evaluation of pre- and post-implementation expectations. AMIA

Annu Symp Proc 2006:1130.

7. Uslu AM, Stausberg J. Value of the electronic patient record: an analysis of the literature. J

Biomed Inform 2008;41(4):675-82.


requirements for electronic health record systems in ophthalmology. Ophthalmology

2011;118(8):1681-7.



2005;12(5):505-16.



2007;48(10):4383-9.


exchange in primary care practices. J Am Board Fam Med 2010;23(5):655-70.

Page 13 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



14

Competing interests


Conflict of interest

None declared









Acknowledgments

None

Main Sponsors:


Page 14 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



15

Table 1 Description of context, task and contrast examined for the example of Neuro-

Ophthalmology















Page 15 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



16

Figure 1 Study Flow and Design Features

Page 16 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



# Reviewer: M.I. Lopez Galvez Our reply

1 The article only includes the protocol not the results. It is not enough

Thank you. This is the protocol for a study that will be still executed. BMJ open offers publication of protocols and we think that this is very useful to share and discuss methods of studies before their execution.

2 I suggest to publish it with the results with the analysis included There is no conclusion

Please see our reply to query 1

Reviewer: Frank Larkin, Our reply

1 It may be inappropriate to evaluate clinician performance using the electronic record before any necessary learning time using the software has elapsed. I realise that the planned second analysis at one year would probably identify any such effect due to inexperience.

Thank you for the favorable assessment of our proposal.

2 Refs #3-5, 8, 9 are incomplete (year of publication). Thank you – the references have been revised.

3 This manuscript describes the design of a prospective simulated study to examine the factors associated with improved performance using an electronic clinical record versus paper records. They term this a randomised vignette study. The design is based on specific clinical decision points in consultations in a range of ophthalmology sub-specialties. While ophthalmology is itself a small specialty, it does lend itself to electronic patient record keeping better than many others for a number of reasons, including the high proportion of images which are used as diagrams or digital photographs in records (for example fundoscopy findings). My opinion is that the findings in this type of study would be of broad interest in clinical medicine more generally.

Thank you! No reply required.

4 Having indicated my support for this study, it would be of much higher value if the results were included. I encourage the authors to add their findings and discuss these. The manuscript is short as submitted.

Thank you. We would like to refer to our reply to query number 1 of Dr. Lopez Galvez

5 As the authors indicate, there has been little research previously published in this field. More critical analysis of those previous reports which they cite would improve the manuscript.

Thank you – no reply required.

6 Methods & design, P9 line 7: Is it valid to assume standard distribution of any differences in responses to matched vignette pairs?

Thank you. We checked again with our statistician and he confirmed the adequacy of our approach.

7 Time of data collection, P9 line 22: If the authors have not set out to examine for any clinician learning effect with software, why not assess competence with software before the first comparative analysis?

Thank you for this important query. We fully agree and clarified this point in the revised paper. (p.8, 1

st para) “Prior to the examination

baseline data will be collected of each participant.”

Page 17 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



Reviewer Jürgen Stausberg Our reply

1 The authors present a well-thought-out study protocol to assess the electronic patient record system EDeR to be implemented in their ophthalmological department. Main focus of this study is to “explore differences in speed and accuracy when searching clinical information using the paper based patient record or the EDeR”. The authors also talk about patient´s safety, which should be improved by an electronic medical record (EMR), and the possible barriers for a successful use of the EMR.

Thank you for the favorable assessment of our protocol. No reply required.

2 As far as the reviewer understands, the plan is as follows. Patient vignettes are the observational units. The study measures the time needed to solve tasks related to those vignettes. The reviewer assumes that tasks and vignettes are related 1 to 1. Otherwise, if a patient vignette is used for several tasks, the tasks become observational unit. Time is the outcome variable. Intervention is the EMR that is compared with the paper-based patient record. Therefore, the study implements a two group design. The authors assume that the intervention saves 60 seconds time in the mean with a standard deviation of 45 seconds. Instead of randomly assigning vignettes to the groups, the study implements a crossover design in randomly assigning the sequence (EMR first, paper second vs. paper first EMR second) to vignettes. That design is underpinned by a formal correct sample size calculation leading to 7 observational units needed. To make it more complex, the authors include subgroups as well as several observers for each measurement. They include 5 subgroups for clinical specialties related to the patient vignettes and 2 subgroups of level of expertise related to the observers. At least, they will have 2 x 5 x 5 (five tasks each clinical subspecialty) x 2 (EMR and paper) = 500 measurements. To address less reliability having only one observer in each subgroup (experts and beginners) the authors intend to recruit more than the two for the study. To make it more serious, the vignettes are slightly modified according to the group. So if an observer checks the same vignette a second time (e. g. in the EMR), he must read the content again, because the answer is different to the first time (e. g. in the paper-based version). The material for the paper-based-group comes from real patients records; the material for the EMR-group is entered into the EMR specifically for the study.

This is an excellent analysis of the proposed method. Thank you – no reply required.

Page 18 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3 This long description makes clear that the proposed study protocol characterizes a complex study design. Unfortunately, the complexity is not well addressed in the text. Furthermore, it is unclear whether the design is really appropriate to answer the research question. Therefore I recommend to improve the readability of the text by clarifying the points made above on the one hand and to rethink the study design according to the research question on the other hand.

Thank you for this query – we agree that the description was too brief. We amended the protocol as follows lending substantially from Prof Stausberg’s excellent summary in point 2: (p 8, last. para.) “Design Matrix Summary

This is a two group (EDeR vs. paper based) cross-over (randomly assigning the sequence electronic vs. paper-based per task) study. Time (primary endpoint) and accuracy (secondary endpoint) are the two outcome variables. Intervention is the EDeR that is compared with the paper-based patient record. We assess tasks within 5 clinical subgroups (without examining subgroup effects) and perform the experiment in two groups with different ophthalmologic expertise (junior doctors vs. senior doctors). Each assessor will complete 50 tasks (i.e. 25 tasks based on the electronic patient record and 25 based on the paper patient record). Electronic and paper-based vignettes are slightly different for each task. If an assessor checks the same vignette a second time (e. g. in the electronic version), he or she must read the content again, because the answer is different to the first time (e. g. in the paper-based version). The material for the paper-based-group comes from real patients records and the tasks were selected based on clinical relevance.”

4 It would be helpful if the authors refer to an established schema for study design rather than to describe the details in own words.

Thank you – we are unaware of a similar application and would be grateful, if you could indicate a paper covering the same topic and applying a similar method.

5 However, the study idea of the authors in assessing the efficacy of EDeR is essential and a current and important topic that must be discussed. It offers a good basis to explore other essential usability and efficacy questions regarding the use of EMRs.

Thank you, no reply required.

6 As long as the time needed to fulfill the tasks is the only measurement, the study is solely related to efficacy as the authors denote time saving. Then, the study does not measure accuracy. Consequently, accuracy should be deleted a study goal.

We fully agree that the protocol did not clearly specify that accuracy was a secondary outcome of this study. We amended the protocol as follows. (p. 6. 3 para.) “Primary outcome


Secondary outcome


”

7 A figure should be added to explain the several entities in the study (vignettes, tasks, sub-specialties, user groups).

We agree with the Reviewer and added a figure to the amended manuscript.

Page 19 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8 A major concern with the study design is a potential bias introduced by the unblinded recording of the vignettes into the EMR. The reviewer assumes that the result is an optimized representation of the vignettes in the EMR. This optimized representation will be different from the information available in the EMR in routine use. Comparing the optimized representation of the vignettes in the EMR with an ordinary representation in a paper-based-record is unfair. Addressing this potential bias in the study design is mandatory.

Thank you. The Reviewer raises an important issue. Data managers at the Hospital will enter the complete available patient record into EDeR. Thus, the content will be identical in both exposures. We agree that this aspect was not clearly stated in the initial submission. We therefore revised the paper as follows: (p. 7, 2. para) “The patient model will come from original paper forms to assure that cases are comparable to the real life situation. The full patient record, containing a slight modification for the contrast examined in a specific task, will be transferred from paper into the electronic patient record system.”

9 The authors should clarify whether the participants of the study will be trained or not in the implemented EDeR prior to the assessment.

Thank you. We fully agree and amended the protocol as follows: (p.6, 5 para)“

Ophthalmologist of the cantonal hospital

Lucerne eye clinic who voluntarily agreed to participate in the study will be included. Each participant will receive a general introduction on the usage of the EDeR of half an hour prior to the assessment.”

10 Study population: How big the study population is or will be? For instance, how many ophthalmologists are found in the eye clinic of the cantonal hospital Lucerne? How many junior and senior ophthalmologists are there? How many ophthalmologists already voluntarily agreed to participate in the study? What is the expectation of the recruitment rate?

Thank you for this query. We provide some information in the revised paper. (p.6. 2

nd para)

“The study is planned to take place at the cantonal hospital of Lucerne in the clinic of ophthalmology. The Eye clinic is run by six consultant ophthalmologists, 12 senior physicians, and 14 junior doctors.”

And (p. 9, 3rd para) “At the time of writing this

protocol 10 senior physicians and 9 junior doctors were willing to participate in the study.”

11 For each out of five different ophthalmologic sub-specialties, the authors will create five tasks based on a patient example. Did some literature lead the choice of these five task types? Or how did the authors come to the definition of the task types? What motivated this choice? The authors should specify if these five tasks are the main consultation´s tasks in their ophthalmology department or if these tasks are generally error-prone when using the paper-based form.

Thank you again for this important query. We specified the selection of topics in the amended manuscript as follows: (p.7, 1

st para) “The tasks

are chosen on the basis of the relevance in clinical practice.”

12 Each physician from each division has to solve all of the tasks. So, a physician from a specific division will have to solve <5 or 25?> tasks using paper-based records and <5 or 25?> tasks using the EDeR in a randomized sequence. Please specify if the physicians will have to solve all <10 or 50?> tasks at once, that is without break.

Thank you. We clarified this as follows: (p.8, 3rd

para) “We will design a reply form into which each participant enters his or her response to the task. This reply form will be handed over to the assessor after completion. Time will be taken by the assessor after completing each entry into this form. In total, each participant will go through 25 pairs of tasks.”

Page 20 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13 “The evaluation will take place at two time points; immediately after implementation of EDeR and one year after.” Please specify if the physicians are already trained in EDeR at the first time point. What about the second time point (one year after)? Also specify if the physicians will be using EDeR during that year or would you keep training them to prepare them to the second evaluation?

Please see our reply to your query 9. Before the first exposure, only a general introduction on the usage of EDeR will be provided that will last about 30 minutes. The training will cover aspects of data collection and saving very briefly. The training will be such, that the basic functions are known. We amended the paper as follows. (p. 6, 3

rd para.) “The training will cover

aspects of data collection and saving very briefly. The training will be such, that the basic functions of the EDeR are known.”

On page 10, 3rd para we also revised as follows:

“At the second time point, one year after the first assessment, the experiment will be repeated using the same protocol. In the interval between the two measurements, all physicians will be mainly working with the EDeR system.”

14 At the assessment/the role of the assessor. Let´s suppose a technical problem occurs after a physician has already performed 8 tasks. This problem unfortunately forces the break of the assessment. Shall the physician start all tasks from the beginning at another time point or may he perform only the last tasks later?

Thank you very much for this important query. We fully agree and added this statement on p.9 1st para: “In case of a technical problem with

the electronic system, the assessment will be suspended and the remaining tasks will be completed after restoration of the system.”

15 Please check, if the sample size estimation holds true in view of the several subgroups as well as the issue of multiple testing introduced with the subgroups.

We will only make statistical comparisons on the main effect. Subgroup analyses will be performed in exploratory analyses. Any subgroup result will be marked in the paper as such and the issue of power will be discussed.

16 Having 20 minutes time for a whole consultation, 60 seconds time reduction with a single task seems unrealistic. Please clarify how the total time reduction for a whole consultation is estimated by observing small tasks.

The reviewer raises an interesting question. We performed a small survey among the senior staff at our institution and there was a consensus that a meaningful difference should be at least 1 Minute / Patient. Therefore, this delta was chosen. We do agree that the advantage in terms of speed will not necessarily come from one single task in clinical practice. But still this could be the case for example when checking the number of interventions a patient received over a longer period of time combined with a judgment of treatment effect. Arguably, there is no strong basis for our statistical assumptions but they seem rational to us. Therefore, we would like to keep the paper unchanged on this point.

17 “We presume that the implementation of the electronic medical record software EDeR will have a positive impact on the efficiency of the doctors, which will result in an increase of consultations per day.” Good and fine, but what is more important: the increase of consultations per day (through decrease of consultation time: 1 min gained) or the improvement of patients healthcare (improved quality of consultation, patient safety)?

Please also see our reply to your query 6. We agree, therefore accuracy is assessed as a secondary outcome in this study. We realize that accuracy is only an indirect measure of “improved quality of consultation and patient safety” and will design further studies investigating this more directly, once this study showed a superiority of EDeR. We kept the manuscript unchanged.

Page 21 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



18 Abstract – Introduction: replace the second sentence with “Critics, however, claim that they are hard to read, because of illegible handwriting, and uncomfortable to use.”

Thank you. We revised the Abstract accordingly.

19 Abstract – Methods and Analysis: there is an 'e' too much in the first sentence.

Well spotted – thank you!

20 Background: second paragraph, last sentence: ‘As a result of these problems’ should be ‘As a solution to these problems’.

Thank you – we revised the paper accordingly.

21 Background: third paragraph, last sentence: the second word 'understanding' should be 'understand'.


22 Last paragraph, first sentence: there is an 'e' too much in the first sentence (In this protocol we describe e a study exploring…).


23 Methods/Design – Study population, first sentence: change ‘in’ in ‘into’.


24 Methods/Design – Data collection – Point of time of data collection: Second part of the second sentence should be rewritten. Did you mean: … with the help ‘of‘ a medical record?


25 Even if there is just one table provided in the paper, the authors should give it a number when referencing to it.


26 References related to the evaluation of medical informatics interventions are missing.

Thank you – we checked again and added the reference of your excellent 2008 Systematic Review published in J Biomed Inform. to the background section. We also added some additional references to underpin our statements better.

27 The selection of references according to the research question is arbitrary.

Please see our reply to your query 26

28 It is unclear whether a conflict of interest exists due to the involvement of the authors in the development of the EMR-system. Please include a statement related to the involvement of the authors.

Thank you – we fully agree and amended the paper accordingly. We declare to have no conflict of interest. (p.13, last para)

Page 22 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



1



Oliver Job 1, Lucas M. Bachmann 2 *, Martin K Schmid 1, Michael A. Thiel 1, Sandra Ivic 2

[1]

Eye Clinic



[2]





e-mail Adresses:






Page 23 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



2

ABSTRACT








Methods and Analysis In this protocol we describe e a study exploring differences in speed






performance.







238 words

Page 24 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3

Article focus


• Efficiency

• Accuracy


Key messages

The chosen design


system








Page 25 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



4

BACKGROUND




medical records within their members and found an adoption rate of only 12%1.%

1.




and errors in diagnosis, prescription and treatment2, 2, because of the absence of consistent


resultsolution of these problems, there was the idea to support clinical daily routine with the

help of computers, as it is reality in many fields of service.





this innovation should improve safety and reduce costsIt seems to be a key strategy to

improve health care in the different specialties3. Electronic medical records are supposed to

improve physicians’ workflow and to offer a better overview on patient’s history. Today

many authors believe that this innovation should improve safety and reduce costs4.”A better

understanding of the impact of the electronic medical records on workflows is vital to

understanding what the technology really does offer that is new and unique” 5.

4.”A better

understanding of the impact of the electronic medical records on workflows is vital to

understand what the technology really does offer that is new and unique”5 6

. However, a

systematic review published in 2008 and covering the evidence until 2005 concluded that

Page 26 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



5

none of 20 studies provided a formal cost effectiveness analysis of electronic patient record

systems in general. 7








progression, and to plan treatment9. 8.



visit volume of 9 visits per examined provider per month3, or the reduction of time spent on

administration by nursing staff6. The collected data also could be used for clinical research

and clinical studies7. In general, there is an existing hypothesis, that EMR should improve



implementation of electronic medical records has a major impact on the efficiency In order to

explore the impact on productivity of physicians using electronic health records, it has been

shown, that there is a statistically significant increase in the average monthly patient visit

volume of 9 visits per examined provider per month 3, or the reduction of time spent on





Page 27 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



6

implementation of electronic medical records has a major impact on the efficiency4 84 11,


with prospective and randomized experimental designs (Fontaine et al), instead of surveys.

, especially in the field of ophthalmology. There seems to be also the demand for more studies

with prospective and randomized experimental designs, instead of surveys 11.



the staff using the EMR software 4 84 11



In this protocol we describe e a study exploring differences in speed and accuracy when






Page 28 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



7

METHODS / DESIGN

Study design


Study location




Study population

The study population will be divided ininto two groups. One group will consist of senior




Inclusion criteria






Primary outcome


Secondary outcome


Page 29 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8

Experimental design







Type of tasks





treatment, v) Question about the completeness of additional information. The Table 1

provides examples of tasks used for Neuro-Ophthalmology.


the real life situation. The cases, slightly modifiedThe full patient record, containing a slight

modification for the contrast examined in a specific task, will be transferred from paper into

the electronic patient record system. During the experiment, a research assistant will monitor

physicians and measure the time required to complete each task. Each physician from each

division has to solve all of the tasks. The evaluation will take place at two time points;

immediately after implementation of EDeR and one year after.





Page 30 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



9





Electronic and paper-based vignettes are slightly different for each task. If an assessor checks

the same vignette a second time (e. g. in the electronic version), he or she must read the

content again, because the answer is different to the first time (e. g. in the paper-based

version). The material for the paper-based-group comes from real patients records and the

tasks were selected based on clinical relevance. For Summary see Figure 1








c) Design matrix





Page 31 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



10









the system.







them.






productivity gain of the physicians3.physician

3. We hypothesise that our productivity gain

should be one additional consultation per day assuming that one consultation lasts about 20

Page 32 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



11

minutes and a physician sees 25 patients daily. Comparing the two methods, paper and

electronic medical record, the overall gain in time should be 60 seconds on average when

using the electronic medical record EDeR.









Data collection





Additionally, there will be someone from the study personalpersonnel, who sits with the

physicians in the same room and measure the time they need to complete each task.

Baseline data




Follow-up data

Page 33 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



12




Page 34 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13

Discussion






and junior doctors.





environment.

Page 35 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



14

References



Academy of Ophthalmology survey. Ophthalmology 2008;115(9):1591-7; quiz 97 e1-

5.


2003;348(25):2526-34.


in a large academic multi-specialty physician group. Int J Med Inform;79(7):492-500.


eHealth on the quality and safety of health care: a systematic overview. PLoS

Med;8(1):e1000387.



Inform;79(11):778-91.



2005;12(5):505-16.



2007;48(10):4383-9.


exchange in primary care practices. J Am Board Fam Med;23(5):655-70.


requirements for electronic health record systems in ophthalmology.

Ophthalmology;118(8):1681-7.

References




e1-5.


2003;348(25):2526-34.



500.



2011;8(1):e1000387.

Page 36 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



15



Inform 2010;79(11):778-91.





Biomed Inform 2008;41(4):675-82.



2011;118(8):1681-7.



2005;12(5):505-16.



2007;48(10):4383-9.



Page 37 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



16

Competing interests



None declared









Acknowledgments

None

Main Sponsors:


Page 38 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



17


Ophthalmology















Page 39 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



18


Formatted: Font: Times New Roman

Page 40 of 40


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from




Journal: BMJ Open

Manuscript ID: bmjopen-2012-002478.R2


Date Submitted by the Author: 11-Mar-2013



Medical management

Secondary Subject Heading: Ophthalmology, Health informatics

Keywords:





jopen.bmj.com

/B

MJ O


pril 2013. Dow

nloaded from



1





1, Michael A. Thiel

1, Sandra Ivic

2

[1]

Eye Clinic



[2]





e-mail Adresses:






Page 1 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



2

ABSTRACT














performance.







238 words

Page 2 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3

Article focus


• Efficiency

• Accuracy


Key messages

The chosen design


system








Page 3 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



4

BACKGROUND


























Page 4 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



5













,



.












Page 5 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



6

METHODS / DESIGN

Study design


Study location




Study population





Inclusion criteria






Primary outcome


Secondary outcome


Page 6 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



7

Experimental design







Type of tasks


















Page 7 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8




tasks based on the electronic patient record and 25 based on the paper patient record). The

electronic records are exact representations of an existing paper-based record. Patient records

differ only in respect to one contrast that is changed for a specific task. Thus, besides that

task-specific detail, the remaining content of each pair of patient records is identical. But, if

an assessor checks the same vignette a second time (e. g. in the electronic version), he or she

must read the content of the paper-based record again, because the correct answer, i.e. a

specific lab value, is different. The tasks were selected based on clinical relevance. For

Summary see Figure 1








c) Design matrix





Page 8 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



9









the system.







them.








Page 9 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



10












Data collection







Baseline data




Follow-up data

Page 10 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



11




Page 11 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



12

Discussion






and junior doctors.



consultations per day. It can be argued that choosing volunteers for this study could lead to

exaggerated results. Nevertheless we believe that the results of our study will provide a

rational basis to quantify the added value of an electronic medical record system in an


Page 12 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13

References

1. Chiang MF, Boland MV, Margolis JW, et al. Adoption and perceptions of electronic health

record systems by ophthalmologists: an American Academy of Ophthalmology

survey. Ophthalmology 2008;115(9):1591-7; quiz 1597 e1-5.


2003;348(25):2526-34.

3. Cheriff AD, Kapur AG, Qiu M, et al. Physician productivity and the ambulatory EHR in a

large academic multi-specialty physician group. Int J Med Inform 2010;79(7):492-500.

4. Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health

care: a systematic overview. PLoS Med 2011;8(1):e1000387.



Inform 2010;79(11):778-91.





Biomed Inform 2008;41(4):675-82.

8. Chiang MF, Boland MV, Brewer A, et al. Special requirements for electronic health record

systems in ophthalmology. Ophthalmology 2011;118(8):1681-7.

9. Poissant L, Pereira J, Tamblyn R, et al. The impact of electronic health records on time

efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc

2005;12(5):505-16.



2007;48(10):4383-9.

11. Fontaine P, Ross SE, Zink T, et al. Systematic review of health information exchange in

primary care practices. J Am Board Fam Med 2010;23(5):655-70.

Page 13 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



14

Competing interests



None declared









Acknowledgments

None

Main Sponsors:


Page 14 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



15


Ophthalmology















Page 15 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



16

Figure legend


Page 16 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



1





1, Michael A. Thiel

1, Sandra Ivic

2

[1]

Eye Clinic



[2]





e-mail Adresses:






Page 17 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



2

ABSTRACT














performance.







238 words

Page 18 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



3

Article focus


• Efficiency

• Accuracy


Key messages

The chosen design


system








Page 19 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



4

BACKGROUND


























Page 20 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



5













,



.












Page 21 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



6

METHODS / DESIGN

Study design


Study location




Study population





Inclusion criteria






Primary outcome


Secondary outcome


Page 22 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



7

Experimental design







Type of tasks


















Page 23 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



8





Electronic and paper-based vignettes are slightly different for each task. IfThe electronic

records are exact representations of an existing paper-based record. Patient records differ only

in respect to one contrast that is changed for a specific task. Thus, besides that task-specific

detail, the remaining content of each pair of patient records is identical. But, if an assessor

checks the same vignette a second time (e. g. in the electronic version), he or she must read

the content of the paper-based record again, because the correct answer, i.e. a specific lab

value, is different to the first time (e. g. in the paper-based version).. The material for the

paper-based-group comes from real patients records and the tasks were selected based on

clinical relevance. For Summary see Figure 1








c) Design matrix



Page 24 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



9











the system.







them.





Page 25 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



10















Data collection







Baseline data

Page 26 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



11




Follow-up data




Page 27 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



12

Discussion






and junior doctors.



consultations per day. WeIt can be argued that choosing volunteers for this study could lead to

exaggerated results. Nevertheless we believe that the results of our study will provide a

validrational basis to quantify the added value of an electronic medical record system in an


Page 28 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



13

References




e1-5.


2003;348(25):2526-34.



500.



2011;8(1):e1000387.



Inform 2010;79(11):778-91.





Biomed Inform 2008;41(4):675-82.



2011;118(8):1681-7.



2005;12(5):505-16.



2007;48(10):4383-9.



Page 29 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



14

Competing interests



None declared









Acknowledgments

None

Main Sponsors:


Page 30 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



15


Ophthalmology















Page 31 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



16


Page 32 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from



125x90mm (300 x 300 DPI)

Page 33 of 33


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


http://bmjopen.bm

j.com/

BM


jopen-2012-002478 on 10 April 2013. D

ownloaded from


BMJ Open · visual field testing, electroretinography), numerical data (e.g. autorefraction,...

Documents

Transcript of BMJ Open · visual field testing, electroretinography), numerical data (e.g. autorefraction,...