Answering Clinical Questions Improves UpToDate Patient...

Executive summary: Improved care results in cost savingsSeveral tools are currently being deployed in provider environments to improve

quality and efficiency of care. Chief among these are electronic medical records

(EMR), computerized physician order entry (CPOE), E-prescribing, and medication

bar-coding systems.

However, it is imperative that quality improvement initiatives also include a

clinical knowledge system. Physicians generate one question for every two or

three patient encounters and two-thirds of those questions go unanswered.

If physicians had access to the appropriate clinical information system, and

could answer clinical questions as they arise, some 40 percent of management

decisions would be changed, improving patient care. Regularly updated electronic

clinical knowledge systems have decreased barriers to answering these questions.

However, there has previously been little evidence as to whether they have an

impact on health outcomes.

The Hospital Clinical Knowledge Systems and Health Outcomes Study was

designed to evaluate whether a leading clinical knowledge system (UpToDate®)

was associated with improvement in patient safety and efficiency in hospitalized

patients, and to estimate the financial impact associated with its implementation.

The study demonstrated a strong association between hospital quality and

efficiency and use of UpToDate. Hospitals with access to UpToDate (n=424)

performed significantly better on risk adjusted measures of patient safety

(p=0.0163) and complications (p=0.0012) and had significantly shorter length

of stay (by an average 0.167 days per discharge, 95% confidence interval 0.081

to 0.252 days, p<0.0001) compared with hospitals without access. These

associations persisted after adjusting for hospital characteristics known to be

associated with the outcomes including geographic location, teaching status,

and discharge volume.

Answering Clinical Questions Improves Patient Safety and Saves MoneyThe Hospital Clinical Knowledge Systems and Health Outcomes Study

Peter A.L. Bonis, M.D.

www.wolterskluwerhealth.com

Hospitals with access to UpToDate performed significantly better on patient safety, complications, and length of stay resulting in a strong return on investment.

UpToDate

1.

The study also confirmed a strong correlation between how much UpToDate was

used at each hospital reflecting a “dose-response” effect. For example, a doubling of

usage from 26,000 to 52,000 topic reviews annually in a hospital with 20,000 annual

admissions could save an additional 520 hospital days each year.


Research study supports the value of UpToDate

BackgroundClinical knowledge systems help providers answer clinical questions. A large body

of evidence has demonstrated that most clinical questions are not pursued even

though the answers to the questions would have a favorable impact on management

decisions.1-25 Physicians generate about one question for every two to three

patient encounters and two-thirds of those questions go unanswered. If answered,

approximately 40 percent of management decisions would be changed.11 The ability

to answer clinical questions quickly has the potential to improve patient safety and

hospital efficiency.

The Hospital Clinical Knowledge Systems and Health Outcomes Study was designed

to assess whether the use of a leading clinical knowledge system, UpToDate, (see

Appendix 1) was associated with quality of patient care and the economic implications

of decreased length of stay and complications, and increased patient safety.26

MethodsSolucient (now a subsidiary of Thomson Healthcare) is a well recognized leader in

measuring hospital performance. Each year it reports on the top performing hospitals

in the United States (the Solucient Top 100® report, now the Thomson Top 100®

report) based upon analysis of a database that contains information regarding inpatient

Medicare admissions from 3,091 short-term general, non-federal hospitals in the

United States (i.e. approximately 75 percent of the 4,200 acute, non-federal hospitals

in the United States).

Metrics used in the 100 Top Hospitals

differentiate hospitals on a variety of

performance dimensions such as quality

and efficiency. They have been used

in many previous studies evaluating

healthcare quality.27-35

UpToDate provided Solucient with

a list of hospitals that had online

access to UpToDate from 2000 through

2005 as well as usage data detailing

how much UpToDate was used at

each hospital. Solucient compared

outcomes in Medicare recipients who

were treated at hospitals with online

access to UpToDate to those in acute

care hospitals included in the Solucient

100 Top Hospitals® database.

The outcomes evaluated were risk-

adjusted for mortality, complications,

the Agency of Healthcare Research

and Quality Patient Safety Indicators,

and hospital length of stay. These

outcomes were chosen because they

reflected the types of benefits that might

be expected from a knowledge system

that has been designed to answer clinical

questions. Details on the study methods

and statistical analyses are provided

in Appendix 2.

2.

For more specific information on the chart above, see page 3.

ResultsHospitals with access to UpToDate (n=424) performed significantly better than other

hospitals in the Solucient database (n=3,091) on risk-adjusted measures of patient

safety (p=0.0163) and complications (p=0.0012), and had significantly shorter length

of stay (by an average 0.167 days per discharge, 95% confidence interval 0.081

to 0.252 days, p<0.0001). These associations persisted after adjusting for hospital

characteristics known to be associated with the primary outcomes including geographic

location, teaching status, and discharge volume.


In addition, all of these associations correlated significantly with how much UpToDate

was used (reflected in “hits per week” [HPW], the average number of topic reviews

viewed each week) supporting a “dose-response” relationship. The graphs depict the

relationship between hospital discharge volume, the amount that UpToDate was used,

and the observed benefits on complications and length of stay. The cut points on HPW

reflect the quartiles of the data.

The study demonstrated a strong linear

association between hospital efficiency

and safety, suggesting that UpToDate

represents an important component

of hospital quality. UpToDate allows

providers to answer questions quickly,

leading to changes in decision-making

that can improve management

and efficiency.11,17,36

In addition, UpToDate is an important

factor in medical knowledge acquisition

among medical residents.37 These effects,

along with time savings and early

adoption of efficient diagnostic and

management strategies, may explain

the observed benefits on patient safety

measures and length of stay.

Financial implications of improved efficiency and patient safety

The improved efficiency and patient

safety benefits associated with use of

UpToDate would be expected to have

substantial economic benefits which

were estimated by an analysis performed

by UpToDate.

The frequency and costs associated

with adverse events are somewhat

more difficult to estimate. The Solucient

data, which are based on medical claims,

likely underestimate the frequency

of adverse events since claims-based

data are known to be insensitive to

adverse events.38

The cost of an adverse event was

estimated to be $30,936 based on a

study of preventable medical injuries in

28 hospitals in the United States adjusted

to 2006 dollars.39 Notably, substantial

additional financial benefits would be

expected for payers and from a reduction

in costs to society, factors that were not

included in the models.

3.

The financial impact of UpToDate is directly associated with the amount that UpToDate

is used at each hospital. Use grows spontaneously year after year (both in the number

of users and the amount of use for each user) at hospitals that subscribe. Thus, the

economic benefits described above would grow proportionally to the rate of adoption

and the amount that UpToDate is used. Efforts to stimulate use of UpToDate could

further increase the benefits.


Relationship between use of UpToDate, complications avoided and hospital days saved each year for a hospital with 20,000 annual discharges. The graph on the left shows that the number of complications that can be prevented increases with greater use of UpToDate. The complications evaluated considered the underlying severity of illness. They included conditions that should not occur and thus represent substandard care, and complications that occurred with a frequency more than expected. The complication methodology has been used in many studies evaluating patient safety. The graph on the right shows that the expected number of hospital days that can be saved increases with greater use of UpToDate. The estimates were based upon the regression model and considered case mix and hospital characteristics known to be associated with hospital length of stay. (See Appendix 2).

As the data suggest, these systems

can improve care and hospital efficiency

producing a large return on investment

comparable to if not exceeding the

expected return on investment for

implementation of CPOE systems.

Thus, they should be considered

complementary and an essential

component of efforts to improve

hospital efficiency and safety.

Whether the observed associations

with hospital quality and efficiency

are applicable to all electronic knowledge

resources is unclear. It is likely that

the benefits from an electronic clinical

knowledge resource would be related

to the degree to which it is adopted

and used. Spontaneous usage levels for

UpToDate increase year after year as a

result of increased use among providers

and an increasing number of users.

The return on investment compares favorably with other technological innovations

aimed at improving the quality, safety, and efficiency of care. Relatively few studies

have directly evaluated the return on investment for information technology

initiatives.40,41 A landmark study involving six Massachusetts community hospitals

estimated that implementation of a CPOE system could save each hospital $2.7

million annually.42 The one-time average total cost of a CPOE system was estimated

to be $2.1 million with an annual incremental operating cost of $435,000. Most of

the savings were anticipated to result from avoidance of preventable adverse drug

events, which were estimated to occur in 10 percent of hospitalized patients. The

economic case for CPOE systems is strong based upon these data but the magnitude

of expected benefits is less than those associated with the implementation of clinical

knowledge systems.

Clinical knowledge systems can also be implemented far more easily than decision

support tools embedded within CPOE or EMR systems. The challenges involved in

successful implementation of EMR and CPOE systems with acceptable decision support

are well-recognized.43,44 By contrast, implementation of clinical knowledge systems can

be accomplished easily and with enthusiastic acceptance by providers.

The chart depicts the median topics viewed each week at 114 hospitals with 18,000 to 25,000 admissions each year. Usage shown is for the year 2006-2007 and is stratified according to how many continuous years each hospital had subscribed. Use increased significantly (p<.0001).

4.


Use of UpToDate was an independent

predictor of performance. Remarkably,

use for 20 minutes a day was associated

with a comparable increase in IM-ITE

scores as an entire year of residency.

Methods to increase use Increased benefits from clinical

knowledge support systems can be

realized by increasing their usage. For

example, a doubling of UpToDate usage

from 500 to 1000 topic reviews per

week in a hospital with 20,000 annual

discharges could result in more than

520 hospital days saved each year (and

more than $800,000 in annual savings).

Increasing use can be accomplished by:

• Increasing awareness and training

• Reducing the click distance to the

resource by integrating it into the

EMR system

• Rewarding providers for use by granting

continuing medical education credits

Conclusions and recommendationsMany factors are involved in hospital

quality and efficiency. The value of

clinical knowledge systems such as

UpToDate as part of efforts to improve

care has not been fully recognized.

The Hospital Clinical Knowledge

Systems and Health Outcomes Study

demonstrated that use of an electronic

clinical knowledge system was associated

with hospital quality and efficiency.

Such systems can improve care and

efficiency and are well-accepted by

providers. The benefits on patient

safety and efficiency also translate into

a large return on investment, and require

minimal resources to be implemented.

That return on investment can be

continually improved by efforts to

increase usage.

The march toward broad implementation

of electronic medical record, CPOE,

E-prescribing, and medication bar-coding

systems is continuing. The costs and

complexity involved in implementing

such systems have overshadowed the

comparatively easily acquired benefits

of implementing clinical knowledge

support systems.

Such systems can be implemented

rapidly leading to almost immediate

improvements in patient care. In

addition, these systems are important

for education of trainees and continuing

medical education of providers.

The acceptance of UpToDate among

providers has translated into widespread

adoption. Over 88% of academic medical

centers in the United States subscribe to

UpToDate and nearly 320,000 healthcare

providers use it worldwide. About 98

percent of hospitals that subscribe

to UpToDate continue to renew their

subscription year after year. UpToDate

users at these hospitals depend on it for

their practices. Hospitals that subscribe to

UpToDate frequently report that providing

access is important for recruiting and

retaining their physicians.

Studies of users in community hospitals

have demonstrated that use is not

confined to doctors. UpToDate is also

used extensively by nurses, nurse

practitioners, physician assistants, and

pharmacists. This permits UpToDate to

become the standard of care, thereby

helping to ensure consistent care that

is well-integrated across all caregivers.

This degree of use means that UpToDate

impacts more than 90 million patient

visits a year in more than 140 countries.

Use has an impact on learning The widespread adoption and use

of UpToDate has had an impact on

learning among providers and trainees.

A study performed by the Mayo Clinic,

for example, examined the impact of

learning habits (use of UpToDate and

conference attendance) on medical

knowledge acquisition as measured

by the Internal Medicine In-training

Examination (IM-ITE).37 The IM-ITE

assesses medical knowledge acquisition

during residency; scores correlate

with subsequent performance on the

American Board of Internal Medicine

Certification Examination.

The most frequently used methods to retrieve clinical information were Medline (73%) and UpToDate (70%) in a study of residents from the University of Colorado Health Sciences Center. From Schilling, L. Acad Med 2005; 80:51.

5.

Appendix 1: Description of UpToDate

UpToDate is an evidence-based, peer reviewed information resource—available via the

Web, desktop, and PDA. UpToDate is designed to answer clinical questions quickly and

provide synthesized recommendations for treatment. Over 90 million patient-related

problems are researched each year with UpToDate.

The UpToDate community includes our faculty of more than 3,800 leading physicians,

peer reviewers, and editors and nearly 320,000 users. Our faculty writes topic

reviews that include a synthesis of the literature, the latest evidence, and specific

recommendations for patient care. Our users provide feedback to the editorial group.

This community’s combined efforts result in the most trusted, unbiased medical

information available.

UpToDate is available through institutional or individual subscriptions. Providers at

hospitals that subscribe to UpToDate can access it through an Internet connection

at any terminal within the facility.

UpToDate contentUpToDate includes more than 7,400 clinical topics, each an average of 10 pages of

text, figures, references, and abstracts, as well as drug and drug interactions programs,

guidelines, patient handouts, and clinical calculators. Access to the content of UpToDate

is through a search screen with a search box. UpToDate uses a proprietary search

engine. The search engine is not static but undergoing continual adjustments to

provide the best search results given the content offering. The search engine can handle

multiple concepts from a “natural language” question like: Treatment of hypertension

in a pregnancy. The search concepts here are: treatment, hypertension, and pregnancy.

During the search, the entire database gets reorganized in order of relevancy.

AuthorsAll topics in UpToDate are written by the listed authors in conjunction with a deputy

editor. Authors are identified as experts by the Editors-in-Chief, our editorial staff,

and participating societies.


Exceptions are guidelines from major

societies, which are added to UpToDate

in their original form. All material is

originally prepared by the contributing

author(s) whose name(s) and affiliation(s)

appear in the upper left corner of

each topic.

This material is reviewed extensively by

our physician editors and peer reviewers

for accuracy and completeness of the

literature search, and for consistency

with all aspects of the editorial policy.

Physician editors suggest changes

to ensure that the topics summarize

the relevant evidence, and that the

recommendations are consistent with

the evidence, with our understanding of

patients’ values and preferences, and with

UpToDate’s editorial policy. Some of the

content may be taken from other topics

in UpToDate. In such cases, the text is

hyperlinked to the topic from which

it originated.

EvidenceUpToDate follows a hierarchy of evidence

consistent with most evidence-based

resources. At the top of the hierarchy are

randomized trials of high methodological

quality, followed by randomized trials with

methodological limitations, observational

studies, and unsystematic clinical

observations. Inferences are stronger

when the evidence is summarized in

systematic reviews of the literature that

present all relevant data.

Each topic has an author who is an

expert in the area discussed, and at

least two separate physician reviewers.

This group works together to perform

a comprehensive review of the

literature and carefully select studies

for presentation based upon the quality

of the study, the hierarchy of evidence

discussed above, and clinical relevance.

6.

When current, high-quality systematic reviews are available, UpToDate topics and

recommendations rely heavily on these reviews. When such reviews are unavailable,

UpToDate summarizes the key studies bearing on the clinical issues at hand. Systematic

reviews and the design of primary studies (randomized trial, observational studies) are

often identified explicitly in the text with the relevant data. However, in cases where

either the type of study or the data are not stated explicitly, users can click on the

reference and bring up the Medline abstract to obtain this information. Evidence is

derived from a number of resources, including but not limited to:

• Hand-searching of over 400 peer reviewed journals

• Electronic searching of databases including MEDLINE, The Cochrane Database,

Clinical Evidence, and ACP Journal Club

• Guidelines that adhere to principles of evidence evaluation described above

• Published information regarding clinical trials such as reports from the Food and Drug

Administration, as well as other sources of information produced by federal agencies

such as the Centers for Disease Control and Prevention and the National Institutes

of Health

• Proceedings of major national meetings

• The clinical experience and observations of our authors, editors, and peer reviewers

RecommendationsUpToDate strives to provide a “Summary and Recommendations” section

(see figure below) where appropriate.


Structured questionsUpToDate’s process of arriving at

recommendations involves constructing

a structured clinical question. That

structure includes carefully defining

the patient population of interest, the

alternative management strategies, and

the outcomes of importance to patients.

Values and preferencesA fundamental principle of evidence-

based medicine, as described by Dr.

Gordon Guyatt from McMaster University,

is that “Evidence alone is never sufficient

to make a clinical decision. Decision

makers must always trade the benefits

and risks, inconvenience, and costs

associated with alternative management

strategies, and in doing so consider the

patient’s values.”45

This principle has led some evidence-

based resources to avoid making specific

recommendations for patient care, since

the recommendation needs to account

for all of the factors cited. UpToDate

has taken a different approach. It is the

policy of UpToDate to make specific

recommendations for patient care

whenever possible. Recommendations

in UpToDate are based upon a synthesis

of evidence including that from clinical

trials and clinical experience; whenever

possible, the evidentiary basis for

recommendations is stated explicitly.

When there is no published systematic

evidence available (eg, prednisone dosing

regimen in pulmonary sarcoidosis),

recommendations are based upon

the unsystematic clinical observations

of our experts and reviewers, and

on pathophysiologic rationale.

7.


UpToDate recommendations identify situations in which different decisions might be

appropriate for patients with different values and preferences. Furthermore, UpToDate

recognizes that recommendations will not apply to every patient, and counts on

clinicians to evaluate the recommendations in light of the individual circumstances

of their patient. Nevertheless, UpToDate feels that giving clinicians access to

recommendations based on a sophisticated understanding of the clinical issues,

the best evidence, and a consideration of patient values and preferences, allows

them to make informed decisions with and for their patients.

GradingUpToDate began grading recommendations for treatment and screening in 2006.

This is a continuing process and not all such recommendations have yet been graded.

Graded recommendations appear in the Summary and Recommendations sections

at the end of topics.

UpToDate uses the UTD-GRADE format, a modification of the GRADE system.46,47

Grades have two components, a number (1 or 2) reflecting the strength of the

recommendation and a letter (A, B, or C) reflecting the quality of the evidence

supporting that recommendation.

A Grade 1 recommendation is a strong recommendation to do (or not do) something,

where the benefits clearly outweigh the risks (or vice versa) for most, if not all patients.

A Grade 2 recommendation is a weaker recommendation, where the risks and benefits

are more closely balanced or are more uncertain. The majority of recommendations will

be Grade 2 recommendations. UpToDate uses wording that reflects the strength of the

recommendation: strong (Grade 1) recommendations are “recommended” and weak

(Grade 2) recommendations are “suggested.”

Grade A evidence means high-quality evidence that comes from consistent results

from well-performed randomized controlled trials, or overwhelming evidence of some

other sort (such as well-executed observational studies with very strong effects). Grade

B evidence means moderate-quality evidence from randomized trials that suffer from

serious flaws in conduct, inconsistency, indirectness, imprecise estimates, reporting bias,

or some combination of these limitations, or from other study designs with special

strength. Grade C evidence means low-quality evidence from observational evidence,

or from controlled trials with several very serious limitations.

8.

Moving from evidence to recommendationsThe following table presents the criteria that UpToDate authors and editors

consider when weighing the advantages and disadvantages of treatments to

decide on a recommendation, and grade the strength of that recommendation.

UpdatingUpToDate performs a continuous comprehensive review of resources previously

outlined to keep the program updated. All of the topics in UpToDate are revised

whenever important new information is published, not by any specific time schedule.

Updates are integrated carefully, with specific statements as to how the new findings

should be applied clinically.


Each topic has a date indicating

the most recent time the topic has

been reviewed and/or modified.

On average, approximately 40 percent

of the topics are updated during each

four-month cycle.

Peer review The Deputy Editor for a specialty,

as well as the Editor-in-Chief and/

or Section Editors assigned to a topic,

review the entire UpToDate content,

including all new topics, updates, and

recommendations. In addition, each

UpToDate specialty has assembled

a group of peer reviewers, often in

conjunction with a sponsoring specialty

society, that is responsible for reviewing

selected topics in each specialty. Finally,

any comments from users of UpToDate

are formally addressed with changes

made as necessary.

Policy review UpToDate’s policies and procedures

are continuously reviewed in consultation

with our Evidence- Based Medicine

Advisory Group. Members of this group

include Dr. Gordon Guyatt and Dr.

Roman Jaeschke from McMaster

University, Dr. Holger Schünemann

from the Italian National Cancer Institute

in Rome/McMaster University, and Dr.

Yngve Falck-Ytter from Case Western

Reserve University.

9.


The database permits analysis of

hospital-level data for two general

types of complications: 1) “Conditions

of concern” in which there are outcomes

that should not occur and thus represent

substandard care (e.g. air embolism); 2)

“Expected complications” (e.g. venous

thrombosis) that (when occurring in

higher than expected frequencies) may

indicate opportunities for improving

patient safety.

Complication rates can be compared

across hospitals after adjusting for

differences in the severity of illness,

geographic location, hospital size and

teaching status, and community setting

(urban versus rural). Thus, facilities

are compared to other facilities with

similar characteristics. The complications

methodology has been extensively

validated and used in many studies

evaluating patient safety.49-51

Mortality Risk-adjusted mortality is determined

based upon normative comparisons using

patient-level data to control for case mix

and severity of illness.52,53 Thus, patients

are compared to other patients with

similar characteristics and co-morbid

conditions. The risk-adjusted mortality

index has been extensively validated.52,53

Length of stayDetermination of average length of

stay was based upon severity-adjusted

diagnostic related groups (DRGs).54,55

Severity-adjusted DRGs are intended

to distinguish discharges that are

clinically similar and require comparable

resources (e.g. diagnostic, therapeutic,

and nursing services).

Severity of illness is based upon the

patient’s medical condition or evidence

of physiologic decompensation. The

DRG system is used to determine

reimbursement by the Centers for

Medicare and Medicaid Services and

most private insurers.

EligibilityHospitals included in the Solucient

database were required to have a Centers

for Medicare and Medicaid Services (CMS)

Medicare Provider Analysis and Review

(MEDPAR) dataset for 2003 to 2004, CMS

standard analytic files outpatient dataset

for 2002 and 2003, and a CMS cost

report for 2004. Hospitals were excluded

from the Solucient database if a current

Medicare cost report was unavailable,

if they were specialty hospitals (e.g.

children’s, women’s, psychiatric, substance

abuse, rehabilitation, cardiac, orthopedic,

long-term acute care), had fewer than

25 acute care beds, 500 total facility

admissions, or 100 Medicare patient

discharges in fiscal year 2004, or had

Medicare average lengths of stay longer

than 30 days. All UpToDate hospitals

fulfilling these criteria were included

in the analysis.

Weighting strategyHospitals that subscribed to UpToDate

were compared to the Solucient database

on bed size, teaching status, and

geographic region. These are variables

known to be associated with the

primary outcome measures. Two

weighting strategies were used to allow

the model coefficients to be applicable

to a national distribution of acute-care

hospitals. Because all the outcome

variables were based on discharge-

level observations, weighting for

hospital volume was done to account

for differences across hospital

inpatient volume.

Appendix 2: Hospital clinical knowledge systems and health outcomes study methods26

Patient safety indicatorsThe patient safety indicator (PSI)

scores represent a set of measures

on potential complications and adverse

events on hospitalized patients. They

were developed with a comprehensive

literature synthesis, analysis of ICD-9

codes, and review by expert clinicians

and implementation of risk adjustment

and empirical studies.48 The Agency for

Healthcare Research and Quality (AHRQ)

provides free, publicly available tools

necessary to determine PSI scores and

they are a widely used measure of patient

safety. The AHRQ Patient Safety Indicators

are used in at least 9 states for public

reporting on hospital performance. The

rationale is that hospitals that show good

performance on these measures are likely

to be providing good quality of care.

There are 23 AHRQ PSIs that are relevant

for provider-level (hospital-level) analysis.

Seven, which are birth-related, are not

relevant to a Medicare population and

were thus excluded. Five require “Cause

of Injury” or E-codes, which are not coded

consistently across acute-care hospitals in

the United States, and were therefore not

included in the study.

ComplicationsSolucient has constructed a database

containing normative, case-level data

on a national level; it contains more than

21 million annual patient discharge levels.

The case-level data include age, sex, race,

payer, length of stay, clinical grouping

(Diagnosis Related Groups or Refined

Diagnosis Related Groups), comorbid

conditions, and hospital

identification information.

10.

Weighting by hospital class was done to derive model coefficients that would better

reflect the national distribution of hospitals on bed size category, teaching status, and

region. Thus, the analysis was performed without weighting, by weighting for discharge

volume only, and by weighting for discharge volume and hospital class. Hospital class

was categorized as small community (bed size 25-99), medium community (bed size

100-249), or large community (bed size >250). Teaching hospitals were classified as

standard teaching (bed size >250 and resident-to-bed ratio of >0.03 or total residency

programs >3), or major teaching (bed size >400 and residents-to-beds ratio >0.25 and

>10 residency programs, or resident-to-bed ratio >0.60 if fewer residency programs).

Statistical analysisHospital-specific performance on mortality, complications, and patient safety were

represented by a z-score for each measure, while severity-adjusted length of stay

(LOS) was based upon the Yale Refined Diagnosis-Related Group methodology.28,56

Severity-adjusted LOS was represented in the models in units of days. Hospital-level,

risk-adjusted mortality, complications, and PSI were based on two years of data (2003

and 2004) to increase the precision of hospital-level measures through larger sample

sizes; LOS was based on 2004 data only.

The PSI composite measure was created by calculating the average of the normalized

z-scores for 11 individual AHRQ PSI indicators that were appropriate for a Medicare

population aged 65 years and older, and did not depend on E-codes. As noted above,

use of cause of injury codes, or E-codes, varies extensively from hospital to hospital,

thus PSI measures that used E-codes were omitted.

The measure-specific z-scores were calculated by subtracting the expected number

from the observed number and dividing by the standard error.57,58 The mortality,

complications, and PSI z-scores, as well as the severity-adjusted LOS, were used

as outcome variables in the analyses.

Separate linear regression models were developed for each combination of outcome

and weighting strategy. All models included adjustment variables described above

to control for hospital bed size category, teaching status, and geographic region

(i.e. Northeast, Midwest, South, and West).

The primary predictor variables of interest were: 1) the UpToDate status of the hospital

and, 2) in separate models, the average number of topic reviews per week (hits per

week, or HPW). HPW refers to the average number of topic reviews that were viewed

at each institution each week. All comparisons were between UpToDate hospitals versus

all nonUpToDate hospitals in the Solucient database.


Data regarding the average hits per

week were also matched to the

Solucient database to determine

whether associations with the primary

outcomes correlated with the amount

that UpToDate was used at each facility.

Thus, UpToDate status was evaluated

in two ways: 1) as a binary variable

indicating whether a given hospital was

using UpToDate and 2) as a continuous

variable representing HPW. Hospitals that

were not UpToDate users were assigned

a zero HPW. The relationship between

the primary outcome measures and HPW

was presented graphically by converting

changes in z-scores associated with

UpToDate HPW to risk-adjusted rates.

11.


Wolters Kluwer Health | ProVation® Medical800 Washington Avenue NorthSuite 400Minneapolis, MN 55401

ProVationMedical.com888.952.6673612.313.1500

References

1. Alper BS, White DS, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med 2005;3(6):507-13.

2. Barrie AR, Ward AM. Questioning behaviour in general practice: a pragmatic study. BMJ 1997;315(7121):1512-5.

3. Bergus GR, Emerson M. Family medicine residents do not ask better-formulated clinical questions as they advance in their training. Fam Med 2005;37(7):486-90.

4. Chambliss ML, Conley J. Answering clinical questions. J Fam Pract 1996;43(2):140-4.

5. Cheng GY. A study of clinical questions posed by hospital clinicians. J Med Libr Assoc 2004;92(4):445-58.

6. Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med 1985;103(4):596-9.

7. Currie LM, Graham M, Allen M, Bakken S, Patel V, Cimino JJ. Clinical information needs in context: an observational study of clinicians while using a clinical information system. AMIA Annu Symp Proc 2003:190-4.

8. D’Alessandro DM, Kreiter CD, Peterson MW. An evaluation of information-seeking behaviors of general pediatricians. Pediatrics 2004; 113(1 Pt 1):64-9.

9. Del Mar CB, Silagy CA, Glasziou PP, et al. Feasibility of an evidence-based literature search service for general practitioners. Med J Aust 2001;175(3):134-7.

10. DeZee KJ, Durning S, Denton GD. Effect of electronic versus print format and different reading resources on knowledge acquisition in the third-year medicine clerkship. Teach Learn Med 2005;17(4):349-54.

11. Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians’ clinical questions: obstacles and potential solutions. J Am Med Inform Assoc 2005;12(2):217-24.

12. Ely JW, Osheroff JA, Ebell MH, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319(7206):358-61.

13. Ely JW, Osheroff JA, Ebell MH, et al. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ 2002;324(7339):710.

14. Fozi K, Teng CL, Krishnan R, Shajahan Y. A study of clinical questions in primary care. Med J Malaysia 2000;55(4):486-92.

15. Gorman P. Information needs in primary care: a survey of rural and nonrural primary care physicians. Medinfo 2001;10(Pt 1):338-42.

16. Gorman P. Information needs in primary care: a survey of rural and nonrural primary care physicians. Medinfo 2001;10(Pt 1):338-42.

17. Graber MA, Randles BD, Ely JW, Monnahan J. Answering clinical questions in the ED. Am J Emerg Med 2008;26(2):144-7.

18. Green ML, Ciampi MA, Ellis PJ. Residents’ medical information needs in clinic: are they being met? Am J Med 2000;109(3):218-23.

19. Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med 2005;80(2):176-82.

20. Jerome RN, Giuse NB, Gish KW, Sathe NA, Dietrich MS. Information needs of clinical teams: analysis of questions received by the Clinical Informatics Consult Service. Bull Med Libr Assoc 2001;89(2):177-84.

21. Kim GR, Bartlett EL, Jr., Lehmann HP. Information resource preferences by general pediatricians in office settings: a qualitative study. BMC Med Inform Decis Mak 2005;5:34.

22. Koonce TY, Giuse NB, Todd P. Evidence-based databases versus primary medical literature: an in-house investigation on their optimal use. J Med Libr Assoc 2004;92(4):407-11.

23. Leff B, Harper GM. The reading habits of medicine clerks at one medical school: frequency, usefulness, and difficulties. Acad Med 2006;81(5):489-94.

24. Ramos K, Linscheid R, Schafer S. Real-time information-seeking behavior of residency physicians. Fam Med 2003;35(4):257-60.

25. Swinglehurst DA, Pierce M, Fuller JC. A clinical informaticist to support primary care decision making. Qual Health Care 2001;10(4):245-9.

26. Bonis PA, Rind DM, Pickens G, Foster D. Association of a Clinical Knowledge Support System with Improved Patient Safety, Reduced Complications and Shorter Length of Stay among Medicare Beneficiaries in Acute Care Hospitals in the United States. Int J Med Inform 2008:(in press).

27. Dickinson T, Riley J, Zabetakis PM. External validation of compliance to perfusion quality indicators. Perfusion 2004;19(5):295-9

28. Edwards N, Honemann D, Burley D, Navarro M. Refinement of the Medicare diagnosis-related groups to incorporate a measure of severity. Health Care Financ Rev 1994;16(2):45-64.

29. Griffith JR, Alexander JA, Jelinek RC. Measuring comparative hospital performance. J Healthc Manag 2002;47(1):41-57.

30. Griffith JR, Knutzen SR, Alexander JA. Structural versus outcomes measures in hospitals: a comparison of Joint Commission and Medicare outcomes scores in hospitals. Qual Manag Health Care 2002;10(2):29-38.

31. Kirchheimer B. Hospitalists make a mark. Emerging specialty linked to shorter lengths of stay and improved mortality, but not to lower costs, Solucient finds. Mod Healthc 2006;Suppl:8- 10, 2, 4 passim.

32. McDonagh KJ. Hospital governing boards: a study of their effectiveness in relation to organizational performance. J Healthc Manag 2006;51(6):377-89; discussion 90-1.

33. Nallamothu BK, Saint S, Ramsey SD, Hofer TP, Vijan S, Eagle KA. The role of hospital volume in coronary artery bypass grafting: is more always better? J Am Coll Cardiol 2001;38(7):1923-30.

34. Proenca EJ, Rosko MD, Dismuke CE. Service collaboration and hospital cost performance: direct and moderating effects. Med Care 2005;43(12):1250-8.

35. Safavi K. The measurement conundrum. J Healthc Manag 2006;51(5):287-90.

36. Lucas BP, Evans AT, Reilly BM, et al. The impact of evidence on physicians’ inpatient treatment decisions. J Gen Intern Med 2004;19 (5 Pt 1):402-9.

37. McDonald FS, Zeger SL, Kolars JC. Factors Associated with Medical Knowledge Acquisition During Internal Medicine Residency. J Gen Intern Med 2007;22(7):962-78.

38. Resar RK, Rozich JD, Classen D. Methodology and rationale for the measurement of harm with trigger tools. Quality & safety in health care 2003;12 Suppl 2:ii39-45.

12.


Wolters Kluwer Health | ProVation® Medical800 Washington Avenue NorthSuite 400Minneapolis, MN 55401

ProVationMedical.com888.952.6673612.313.1500

39. Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry 1999;36(3):255-64.

40. Krohn R. In search of the ROI from CPOE. J Healthc Inf Manag 2003;17(4):6-9.

41. Menachemi N, Brooks RG. Reviewing the benefits and costs of electronic health records and associated patient safety technologies. Journal of medical systems 2006;30 (3):159-68.

42. Adams M, Bates DH, Coffman GA, Everett W. Saving Lives, Saving Money: The imperative for computerized physician order entry in Massachusetts Hospitals. In: Massachusetts Technology Collaborative and the New England Healthcare Institute; 2008.

43. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14(4):415-23.

44. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals. Health affairs (Project Hope) 2004;23(4): 184-90.

45. McAlister FA, Straus SE, Guyatt GH, Haynes RB. Users’ guides to the medical literature: XX. Integrating research evidence with the care of the individual patient. Evidence-Based Medicine Working Group. JAMA 2000;283(21):2829-36.

46. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ 2004;328(7454):1490.

47. Guyatt G, Gutterman D, Baumann MH, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest 2006;129(1):174-81.

48. AHRQ. Quality Indicators--guide to Patient Safety Indicators. version 21, Revision 3 ed: Agency for Healthcare Research and Quality; 2005.

49. Iezzoni LI, Daley J, Heeren T, et al. Identifying complications of care using administrative data. Med Care 1994;32(7):700-15.

50. Iezzoni LI, Davis RB, Palmer RH, et al. Does the Complications Screening Program flag cases with process of care problems? Using explicit criteria to judge processes. Int J Qual Health Care 1999;11(2):107-18.

51. Weingart SN, Iezzoni LI, Davis RB, et al. Use of administrative data to find substandard care: validation of the complications screening program. Med Care 2000;38(8):796-806.

52. DesHarnais S, McMahon LF, Jr., Wroblewski R. Measuring outcomes of hospital care using multiple risk-adjusted indexes. Health Serv Res 1991;26(4):425-45.

53. DesHarnais SI, Chesney JD, Wroblewski RT, Fleming ST, McMahon LF, Jr. The Risk-Adjusted Mortality Index. A new measure of hospital performance. Med Care 1988;26(12):1129-48.

54. Fetter RB. Casemix classification systems. Aust Health Rev 1999;22(2):16-34; discussion 5-8.

55. Freeman JL, Fetter RB, Park H, et al. Diagnosisrelated group refinement with diagnosis- and procedure-specific comorbidities and complications. Med Care 1995;33(8):806-27.

56. MacKenzie T. Patient classification systems: An evaluation of the state of the art. 1991.

57. Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD. Judging hospitals by severity-adjusted mortality rates: the influence of the severity-adjustment method. Am J Public Health 1996;86(10):1379-87.

58. Iezzoni LI, Shwartz M, Ash AS, Hughes JS, Daley J, Mackiernan YD. Using severity-adjusted stroke mortality rates to judge hospitals. Int J Qual Health Care 1995;7(2):81-94.

13.

Answering Clinical Questions Improves UpToDate Patient...

Documents

Transcript of Answering Clinical Questions Improves UpToDate Patient...