Impact of Physician Scorecards onEmergency Department Resource Use,Quality, and EfficiencyShabnam Jain, MD, MPHa,b, Gary Frank, MDb, Kelly McCormick, MSPHc, Baohua Wu, MSc, Brent A. Johnson, PhDc

abstract BACKGROUND AND OBJECTIVES: Variability in practice patterns and resource use in theemergency department (ED) can affect costs without affecting outcomes.ED quality measures have not included resource use in relation to EDoutcomes and efficiency. Our objectives were to develop a tool forcomprehensive physician feedback on practice patterns relative to peers andto study its impact on resource use, quality, and efficiency.

METHODS: We evaluated condition-specific resource use (laboratory tests; imaging;antibiotics, intravenous fluids, and ondansetron; admission) by physicians at2 tertiary pediatric EDs for 4 common conditions (fever, head injury, respiratoryillness, gastroenteritis). Resources used, ED length of stay (efficiency measure),and 72-hour return to ED (return rate [RR]) (balancing measure) were reportedon scorecards with boxplots showing physicians their practice relative to peers.Quarterly scorecards were distributed for baseline (preintervention, July2009–August 2010) and postintervention (September 2010–December 2011).Preintervention, postintervention, and trend analyses were performed.

RESULTS: In 51 450 patient visits (24 834 preintervention, 26 616 postintervention)seen by 96 physicians, we observed reduced postintervention use ofabdominal and pelvic and head computed tomography scans, chest radiographs,intravenous antibiotics, and ondansetron (P , .01 for all). Hospital admissionsdecreased from 7.4% to 6.7% (P = .002), length of stay from 112 to 108 minutes(P , .001), and RR from 2.2% to 2.0%. Trends for use of laboratory tests andintravenous antibiotics showed significant reduction (P , .001 and P , .05,respectively); admission trends increased, and trends for use of computedtomography scans and plain abdominal radiographs showed no change.

CONCLUSIONS: Physician feedback on practice patterns relative to peers results inreduction in resource use for several common ED conditions withoutadversely affecting ED efficiency or quality of care.

There has been considerable focus onoverall resource use and cost of care inour nation’s emergency departments(EDs), particularly for nonemergentconditions that often make up a largeshare of the case mix in the typical ED.Additionally, wide variation in practiceand resource use that cannot beexplained by patient-related factors hasbeen demonstrated in both adults andchildren, including emergency

medicine patients.1–12 Excessive use ofresources in health care has not beenfound to improve quality or outcomes,but it does affect cost.11,13–16 Therecent Choosing Wisely initiativeencourages every specialty to considerreducing use of tests and proceduresthat are often unnecessary andsometimes can be harmful.17

Traditional ED quality measures haveincluded measures of timeliness such

aDepartment of Pediatrics and Emergency Medicine, EmoryUniversity School of Medicine, Atlanta, Georgia; bChildren’sHealthcare of Atlanta, Atlanta, Georgia; and cDepartment ofBiostatistics, Rollins School of Public Health, EmoryUniversity, Atlanta, Georgia

Dr Jain conceptualized the study, designed thescorecards, interpreted the data, and drafted theinitial manuscript; Dr Frank assisted with scorecardimplementation and data review and revised themanuscript; Ms McCormick assisted with dataanalysis and revised the manuscript; Mr Wu workedon data analysis, generated the summary tables andgraphs, and revised the manuscript; Dr Johnsonanalyzed the data and revised the manuscript; andall authors approved the final manuscript assubmitted.

www.pediatrics.org/cgi/doi/10.1542/peds.2014-2363

DOI: 10.1542/peds.2014-2363

Accepted for publication Mar 6, 2015

Address correspondence to Shabnam Jain, MD,MPH, Emory University, Children’s Healthcare ofAtlanta, Pediatric Emergency Medicine, 1645 TullieCircle NE, Atlanta, GA 30329. E-mail: sjain@emory.edu

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online,1098-4275).

Copyright © 2015 by the American Academy ofPediatrics

FINANCIAL DISCLOSURE: The authors have indicatedthey have no financial relationships relevant to thisarticle to disclose.

FUNDING: Partially funded by the Children’sHealthcare of Atlanta Research Fund.

POTENTIAL CONFLICT OF INTEREST: The authors haveindicated they have no potential conflicts of interestto disclose.

QUALITY REPORT PEDIATRICS Volume 136, number 3, September 2015 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

as length of stay, time to antibiotics,boarder time, safety measures such aserrors, hand-washing, and measuresof patient-centeredness such as left-without-being-seen rates and patientsatisfaction. Unexpected return to EDsoon after an initial ED visit isa measure of effectiveness of care thatis monitored in most EDs. Recentstudies have suggested theimportance of measuring efficient useof resources for common pediatric EDconditions.18,19 Efforts to streamlineresource use by standardizingpractice through evidence-basedguidelines have been ongoing, butsignificant degrees of variation inpractice remain.4,5,19 Current EDquality measures do not includemeasures of efficient use of resourcesor how they correlate with EDoutcomes. In recently proposedindicators of quality in pediatric EDs,none of the 62 evaluated unnecessarytests.20 A lack of system incentivessuch as quality measures has beennoted as one of the reasons foroverdiagnosis that can result ina cascade of effects that do not alwaysimprove outcomes.21

Audit and feedback of physicianpractice has been used as a tool toimprove resource use and careefficiency; when physicians areprovided with data on how theypractice, they may be more likely toconsider practice changes.22–24

Although insurers have been profilingproviders’ resource use patterns, suchdata are often not available toindividual clinicians, usually are notacuity-adjusted, and do not includea combination of metrics that balanceresource use and outcome measures.Balanced scorecards have been used inhealth care to provide a comprehensivepicture of performance at theorganizational level but have not beenstudied at the individual providerlevel.25–28 The impact of providingphysicians with comprehensive andbalanced feedback on their practicepatterns relative to peers in the samesetting is not known. In this qualityimprovement initiative, our objectives

were to develop a tool forcomprehensive feedback to EDphysicians on their practice patternsrelative to peers and to evaluate theimpact of such a physician feedbacktool on ED resource use and associatedED quality measures, ED length of stay(efficiency measure), and return to ED(balancing measure).

METHODS

Setting and Scorecard Development

The study was conducted at a largepediatric health care system’s 2tertiary-level EDs with a combinedannual census of .150 000 pediatricvisits. Both sites are staffed by pediatricemergency medicine physicians andurgent care pediatricians and haveelectronic medical records (EMRs). Wehave previously documented largevariation in practice in the ED evenafter adjusting for patient (eg, acuity,age) and temporal factors.4 We usedthis persistent variation in practice tohighlight individual physicianperformance relative to the range ofpractice of their peers. We did this bycreating a comprehensive and balancedscorecard showing physicians theirresource use and ED quality metrics,relative to their peers, for 4 commonED conditions. These conditions werechosen because they are some of themost common presentations topediatric EDs. Institutional guidelinesfor the management of fever have beenin place since 2004 and for the 2 mostcommon respiratory illnesses (asthmaand bronchiolitis) since 2006.Guidelines for the management of mildhead injury were implemented in mid-2011, and there are no institutionalguidelines for the management ofchildren with gastroenteritis-likeillness. Although guidelines caninfluence practice patterns, suchchange takes time, especially if notassociated with measurement ofcompliance with guidelinerecommendations. During our studyperiod, there were no new efforts toreinforce compliance to guidelines.

Scorecard Inclusion Criteria

Four common ED conditions wereincluded in the scorecard. To avoidbias based on final diagnosis, webased inclusion criteria on patientpresenting complaints. Professionalcoders classified patient chiefcomplaints into “admitting” diagnosesusing International Classification ofDiseases, Ninth Revision, ClinicalModification codes. The 4 conditionsand the corresponding chiefcomplaint International Classificationof Diseases, Ninth Revision, ClinicalModification codes were Feverunspecified (780.60) (age .2 monthsonly because infants ,2 months oldreceive routine screening tests andare often admitted); Head Injuryunspecified (959.01) (age .3 monthsbecause institutional guidelines applyonly to infants .3 months old);Gastroenteritis-like symptoms:Vomiting alone (787.03), PersistentVomiting (536.2), Diarrhea (787.91),Dehydration (276.51); andRespiratory illness: Other Dyspneaand Respiratory Abnormality(786.09), Cough (786.2), Wheezing(786.07), Unspecified Asthma(493.90), with exacerbation (493.92),with status asthmaticus (493.91).

Acuity Adjustment and PeerComparisons

Studies on ED provider feedback haveunderscored the importance ofadjusting for acuity, diagnosis, andpatient outcomes, and theyrecommend making comparisons ofresource use rates and outcomes forindividual physicians against peer-based norms.24 In our study, weachieved acuity adjustment byincluding only patients in EmergencySeverity Index category 3(midacuity).29,30 The potential forpractice variation is highest for thesemidacuity patients, and the 4 conditionsincluded in the scorecard representnearly 40% of all midacuity patientsseen in our ED. Peer groupperformance was shown on thescorecard as a standard boxplot with25th, 50th, and 75th percentiles, with

PEDIATRICS Volume 136, number 3, September 2015 e671 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

dashes for 1.5 interquartile ranges. Abold diamond indicated an individualphysician’s performance against thepeer group’s performance shown onthe boxplot (Fig 1).

Quality Measures Included in Scorecard

Three measures of ED quality of carewere included in the scorecard:resource use, ED length of stay, and72-hour return rate (Table 1).Resource use, a process measure, wasmonitored specific to each condition.Additionally, because EDs account for.50% of hospital admissions, andarguably this is one of the mostexpensive resource use decisionsmade by an ED physician and isconsidered discretionary, admissionrate to the hospital was measured foreach of the 4 conditions.31 ED lengthof stay (LOS; measured as time fromphysician assuming care of thepatient until exit from the ED) wasincluded as a timeliness measure.Rate of 72-hour return to the ED forthe same condition was included asa balancing quality measure. Thescorecard also included the totalnumber of patients seen by eachphysician during the reporting period.

Exclusion Criteria

Patients who left without being seenwere excluded. Performance data ofphysicians who saw ,10 patients

during a reporting period in anycondition were not included in thepeer group (boxplot) calculation forthat condition, to prevent proportionswith low denominators from undulyinfluencing percentiles. Furthermore,physicians who saw ,25 patientsduring either the preintervention orpostintervention phase wereexcluded from analysis because thesephysicians were not regular EDphysicians, or they left the practice atsome point, and therefore did notmeaningfully contribute to changes inpractice patterns.

Physician Attribution

Physical attribution was based on theattending physician assigned to thepatient during the ED visit. Whenthere was transfer of care during theED visit, resource use decisions wereassigned to the first physician,whereas the disposition decision and72-hour return were assigned to thesecond (dispositioning) physician.

Intervention

The intervention (feedback viascorecards) began in September 1,2010. Feedback was providedthrough 3 related interventions. Thefirst was quarterly distribution ofscorecards electronically. The firstscorecard was for the reportingperiod July 2009 to June 2010, and

subsequent scorecards weredistributed quarterly, with eachreporting quarter representinga previous 12-month rolling average.The second intervention waspresentation of scorecards foreducational purposes at practicemeetings. Third, physician directorsof both EDs regularly reviewedwith physicians their performance ona 1-on-1 basis, particularly withphysicians who had outlier practicepatterns. The 14-month period fromJuly 2009 to August 2010 (just beforethe first scorecard was distributed)was the preintervention phase, andthe 16-month period from September2010 to December 2011 was thepostintervention phase.

Data Source

Data were obtained from EMRs andadministrative data that are stored inan institutional data warehouse. TheEMRs have electronic signaturecapture that allows for accuratephysician attribution; computerizedphysician order entry ensured thatresource use was accurately assignedto the ordering physician. Data wereaggregated at the level of individualproviders, who were identified byblinded codes not accessible toinvestigators.

Statistical Analysis

To assess the effect of this scorecardintervention, we conceived 2complementary statistical analyses:compare the aggregate group-leveldifference in resource use before andafter the intervention, and comparephysician-level serial trends beforeand after the intervention to evaluatelonger-term effects of theintervention. The former analysisallows comparison of aggregategroup-level averages for 11 resourcesand outcomes of interest before andafter the intervention. For thisanalysis, we used x2 tests (forcategorical data), 2-sample t tests (fornormally distributed continuousdata), and Wilcoxon rank-sum tests(for skewed continuous data, eg,

FIGURE 1A boxplot showing an individual physician’s performance (shown as a blue diamond) against thepeer group’s performance.

e672 JAIN et al by guest on June 16, 2020www.aappublications.org/newsDownloaded from

LOS). The latter analysis isa secondary analysis of findings toassess the trends both before andafter intervention and to determinewhether there are statisticaldifferences in these 2 trends. For thisanalysis, we used piecewise(generalized) linear mixed models tomodel the scorecard outcomes asa function of each resource oroutcome on the scorecard whileaccounting for physician-levelclustering and effect of time. Eachmonth, each physician was a repeatedmeasure, and each physician startedat his or her own baseline. A Poissondistribution was used to modelresponses that are counts. The effectof the intervention was measured bythe change in the slope of the trendlines before and after theintervention, showing change inmonthly trends over time (asopposed to change in practice ata single point in time), reflectingsustained effect. Results were notedin terms of marginal effects (eg, thechange in the probability thata patient receives a test before andafter intervention). We performed allstatistical analyses on SAS (SASInstitute, Inc, Cary, NC) statisticalsoftware.32

The institutional review boardexempted this study from reviewbecause only aggregate data wereused, with no physician or patient

identifiers and no patient-levelintervention.

RESULTS

During the study period from July2009 to December 2011, a total of336 294 patients were seen in theEDs, of whom 128 691 were inEmergency Severity Index 3 acuity. Ofthese, 51 450 patients met inclusioncriteria (preintervention, 24 834;postintervention, 26 616). Ninety-sixphysicians saw these patients, witha mean of 536 patients per physician.Figure 2 shows a typical scorecard ofa physician during the study periodfor all 4 conditions.

Table 2 shows the overallpreintervention–postinterventionresults for the various measuresreported on scorecards. Resourcecategories shown reflect only thosethat were included in the qualityimprovement initiative and reportedto providers via the scorecard.Overall, statistically significantreduction was noted in use ofabdominal and pelvic computedtomography (CT) scans, head CTscans, chest radiographs, intravenous(IV) antibiotics, and IV ondansetron(P , .01 for all). Hospital admissionrates decreased from 7.4% to 6.7%(P = .002). The median ED LOS for the4 included conditions decreased from112 minutes to 108 minutes(P , .001). The 72-hour return rate

changed from 2.2% to 2.0%; thoughnot statistically significant, the ratedid not increase as use of tests andtherapies during the initial visitdecreased.

Table 3 summarizes the trends overtime for significant findings in Table 2.Statistically significant decrease intrends over time was noted for use oflaboratory tests (P , .001) and foruse of IV antibiotics (P = .02);admission trends showed an increase(P = .02). Other resource categoriesdid not show statistically significantchange in trends over time during thestudy period. Figure 3 shows trendsover time as monthly estimates ofrates of laboratory tests performed,IV antibiotic use, hospital admissions,IV ondansetron use, abdominal CTscans, and abdominal radiographsperformed before and after theintroduction of physician-levelscorecards.

DISCUSSION

Our study shows reduction inresource use for several commonlyseen conditions in the pediatric EDafter physicians were provided withfeedback on practice patterns,including resource use and qualitymetrics, relative to their peers. Therewas a small improvement inefficiency (LOS), and reducedresource use did not adversely affectquality of care (return rate).

TABLE 1 Quality Measures Included in Scorecard

Fever Gastroenteritis-like Illness Head Injury Respiratory Illness

Resources Laboratory tests Laboratory testsCBC CBCBMP or CMP BMP or CMPBlood cultureCRP

Imaging Imaging Imaging ImagingChest radiograph Abdominal radiograph Head CT scan Chest radiographAbdominal radiograph Abdominal CT scan

IV fluidsIV ondansetron

Hospital admission: reported for all 4 conditionsED LOSa Reported for all 4 conditions72-h return rateb Reported for all 4 conditions

BMP, basic metabolic profile; CBC, complete blood cell count; CMP, comprehensive metabolic profile; CRP, C-reactive protein.a ED LOS measured as time from physician assuming care of patient until exit from the ED.b 72-h return rate: Return to the ED for same condition within 72 h of an initial visit.

PEDIATRICS Volume 136, number 3, September 2015 e673 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

However, results were notconsistent, and some trendspredated the intervention.

Traditional ED quality measuresinclude operational metrics such asturnaround time and LOS, timeliness

measures such as time to antibioticsin newborns or patients with sicklecell disease and fever, and outcome

FIGURE 2Scorecard for 1 physician for all 4 conditions (fever, gastroenteritis-type illness, respiratory illness, and head injury) during the entire study period. Abx,antibiotics; KUB, kidneys, ureter, bladder; pts, patients; XR, radiograph.

e674 JAIN et al by guest on June 16, 2020www.aappublications.org/newsDownloaded from

measures such as patient satisfactionand left-without-being-seenrates.33–36 Many of these measuresdo not reflect physician-relatedclinical decision-making in the careprovided to the patient and are oftennot directly controlled by EDphysicians. Stang et al20 recentlydeveloped evidence-based quality

indicators for high-acuity pediatricconditions that can be applied to EDsettings where children are seen.Most of the measures they suggestare efficiency measures.

Our primary goal was to studywhether providing feedback to EDphysicians, on factors that theycontrol (testing and treatment

decisions for common midacuitypresentations for which practiceguidelines are available), can affectresource use. We have previouslyshown wide variation in physicianuse of common resources in our ED;higher resource use was associatedwith greater LOS but did not reducereturn to ED.4 In a recent study,

FIGURE 2Continued.

PEDIATRICS Volume 136, number 3, September 2015 e675 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

Kharbanda et al14 observedvariation in quality measures forpatients presenting to 21 USpediatric EDs with commonconditions and noted that highercosts (reflective of overall resourceuse) were not associated with lowerhospitalization or ED revisit rates.Coon et al21 noted that practicevariation may be an importantindicator for overdiagnosis, whichcan lead to a number of downstreameffects that do not improve patientoutcomes but can cause harm. Inour current study, we tookadvantage of variation in practice atthe local level to highlight outlierpractice patterns in both directionsof high and low resource use. Suchbenchmarking of providers relativeto peers can highlight opportunitiesfor improvement; when it is done atthe local level, it is likely to be moremeaningful to clinicians practicingin the same setting. A unique featureof our scorecard is that it isbalanced and comprehensive, notonly providing data on resource use

but also providing return rates asa balancing measure and LOS as anefficiency measure. We alsoadjusted for both acuity and patientcomplaint, and we used common EDconditions for which institutionalguidelines were available to informpractice. We used 2 separatemethods of analysis: an unadjustedpreintervention–postinterventionanalysis to evaluate aggregategroup-level differences in practicepatterns and a rigorous, adjustedmodel to evaluate changes in truetrends at the individual physicianlevel. Whereas the former analysiscan indicate whether overallresource use differed in the 2 timeperiods, the latter analysis canidentify ongoing sustained changesin trends of practice. The latteranalysis also controls forpreexisting trends (eg, if resourceuse was already decreasing in thepreintervention period) and is morerobust in adjusting for physician-level random effects and repeatedmeasures over time.

After providing feedback for justover 15 months, we observed overallreduction in the use of severalresources by using an aggregatecomparison between the 2 timeperiods. Trend analysis showedsimilar reduction in resource useover time for some but not all ofthese categories. This reduction mayhave been caused by several factors.Some trends reversed in thepreintervention andpostintervention period; forexample, trends for laboratoriesshowed increasing use before theintervention and decreasing useafter (Fig 3). Although overall use oflaboratory tests did not changeduring the study, the change intrends reflects statisticallysignificant change in practice(reduction in use) after theintervention (Table 3). Trends suchas use of IV ondansetron andabdominal and pelvic CT scanspredated the intervention (Fig 3);some of these may have resultedfrom overall awareness of issuesrelated to costs of IV medication,radiation exposure of CT scans, andthe like. Reduction in use of CT scansfor minor head injury during 2005 to2009 was noted in a recent study byMannix et al.11 Our secondaryanalysis of trends helps elucidatelong-term changes in behavior at theindividual physician level andsustainability of practice changes forresources whose use is alreadychanging. We conjecture that ourintervention kept the momentum ofchange moving in the desireddirection for most but not allresource use. For example, foradmission to hospital we noteda true discrepancy between thestatistically significant decrease inoverall admission rates in thepostintervention period and thestatistically significant increase intrends in admission rates. Althoughit is conceivable that this increase inadmission rate trends may be relatedto lower resource use, the decreasein overall admission rates does not

TABLE 2 Overall Preintervention and Postintervention Results for the Quality, Efficiency, andResource Use Measures Reported on Scorecards

Resource or Outcome Preintervention Postintervention P

IV antibiotics, n (%) 1218/10 114 (12.0) 937/8691 (10.8) .007IV ondansetron, n (%) 562/4827 (11.6) 478/5937 (8.1) ,.001Abdomen or pelvis CT scan, n (%) 56/4827 (1.2) 37/5937 (0.6) .003Head CT scan, n (%) 534/2055 (26.0) 522/2740 (19.1) ,.001Chest radiograph, n (%) 2470/7838 (31.5) 2592/9248 (28) ,.001Abdominal radiograph, n (%) 742/4827 (15.4) 972/5937 (16.4) NSLaboratory tests per 100 patients, mean (SD) 71.13 (124.9) 68.74 (121.6) NSIV fluids, n (%) 1826/4827 (37.8) 2292/5937 (38.6) NSHospital admission, n (%) 1842/24 834 (7.4) 1788/26 616 (6.7) .002LOS in minutes, median (interquartile range) 112 (69–168) 108 (66–166) ,.00172-h return rate, n (%) 549/24 834 (2.2) 542/26 616 (2.0) NS

NS, not significant.

TABLE 3 Change in Trends Before and After Intervention for Selected Resource Use MeasuresReported on Scorecards

Resource Use Measure Change in Trends (slope afterintervention 2 slope before intervention)

Interpretation of Change P

Laboratory tests(per 100 patients)

20.021 Decreasing trend in use ,.0001

IV antibiotics, % 20.027 Decreasing trend in use .02Hospital admissions, % 0.020 Increasing trend in use .02IV ondansetron, % 20.021 No change .21Abdominal CT scans, % 0.047 No change .37Abdominal

radiographs, %0.0116 No change .4

e676 JAIN et al by guest on June 16, 2020www.aappublications.org/newsDownloaded from

support that possibility. Use ofabdominal radiographs was the onlyresource category that showed anactual increase in use after theintervention, with a somewhat

increasing trend in thepostintervention period (Fig 3).Although neither the increase inoverall use of abdominal radiographsnor the increase in trends is

statistically significant, this changemay be related to a reduction in useof abdominal CT scans, withphysicians replacing 1 imagingmodality with another.

FIGURE 3Trends over time in rates of laboratory tests performed, IV antibiotic use, hospital admissions, IV ondansetron use, abdominal CT scans, and abdominalradiographs performed before and after introduction of physician-level scorecards.

PEDIATRICS Volume 136, number 3, September 2015 e677 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

Use of IV antibiotics showeda significant decrease after theintervention, both in overall rate and insignificant change in trends (Tables 2and 3, Fig 3). In these common,nonsevere pediatric conditions, use ofantibiotics is often not indicated, anduse of IV antibiotics may bea cautionary approach of someproviders and therefore amenable tochange in practice. Decreased use ofbroad-spectrum IV antibiotics wouldhave additional downstream benefitsincluding reducing costs, side effects,and resistance.

A limitation of our study is that it isa single-center study and may not begeneralizable to other settings. Giventhe increasing use of EMRs nationally,and of standard systems of triageacuity assignment that can be usedfor severity adjustment, along withnearly universal tracking of returnrates and LOS in most EDs, we thinkscorecards similar to ours can beeasily adapted for use in many EDs.Additionally, trends for resource usein some categories predated theintervention, making interpretationdifficult. We did not make any otherchanges in the ED during the studyperiod that would affect resource use.The EDs did get a new EMR in mid-July 2011; the only foreseeableimpact of that change pertaining tothis study would be an increase inLOS as providers learned a newsystem. We saw an increase in overallLOS for a few months afterimplementation of the new EMR (dataavailable); however, over the entirestudy period, the LOS specific to thepatients included in the scorecardsshowed a decrease.

The modest and inconsistent effectsof the scorecards in our studyunderscore the difficulty of changingphysician behavior. It remains to beseen how other factors, such asmonetary incentives and medicolegalissues, none of which changed duringthis study, can influence resourceuse in addition to comprehensive,balanced feedback to providers.

CONCLUSIONS

Our study shows reductions inresource use for several commonlyseen conditions in the pediatric EDafter we provided ED physicians withfeedback on practice patterns,including resource use and qualitymetrics, relative to peers. Reducedresource use did not adversely affectED efficiency (LOS) or quality of care(return rate).

Feedback on practice patternsrelative to peers can influenceprovider practice for patients outsidethe middle acuity range and the4 conditions we studied, thus havinga broader impact. Also, feedback tophysicians on clinical factors thatthey can control, and for which thereis evidence to base practice on, maybe more meaningful than qualitymeasures that focus primarily onoperational metrics. Suchperformance measurement can beused for activities such as the JointCommission’s Ongoing ProfessionalPractice Evaluation. With additionalrefinement, comprehensive, severity-adjusted performance measures thatencompass both quality and resourceuse can be used to measure value atthe individual provider level and toidentify high-value providers.

ACKNOWLEDGMENTS

We thank Robert Massey and SetonMcRae for data acquisition andmanagement.

ABBREVIATIONS

CT: computed tomographyED: emergency departmentEMR: electronic medical recordIV: intravenousLOS: length of stay

REFERENCES

1. Goldin AB, Garrison M, Christakis D.Variations between hospitals inantireflux procedures in children. Arch

Pediatr Adolesc Med. 2009;163(7):658–663

2. Goldman RD, Scolnik D, Chauvin-Kimoff L,et al Practice variations in the treatmentof febrile infants among pediatricemergency physicians. Pediatrics. 2009;124(2):439–445

3. Hampers LC, Faries SG. Practicevariation in the emergency managementof croup. Pediatrics. 2002;109(3):505–508

4. Jain S, Elon LK, Johnson BA, Frank G,Deguzman M. Physician practicevariation in the pediatric emergencydepartment and its impact on resourceuse and quality of care. Pediatr EmergCare. 2010;26(12):902–908

5. Landrigan CP, Conway PH, Stucky ER,Chiang VW, Ottolini MC. Variation inpediatric hospitalists’ use of proven andunproven therapies. J Hosp Med. 2008;3(4):292–298

6. Lantos JD, Meadow W. Variation in thetreatment of infants born at theborderline of viability. Pediatrics. 2009;123(6):1588–1590

7. Powell EC, Hampers LC. Physicianvariation in test ordering in themanagement of gastroenteritis inchildren. Arch Pediatr Adolesc Med.2003;157(10):978–983

8. Song Y, Skinner J, Bynum J, Sutherland J,Wennberg JE, Fisher ES. Regionalvariations in diagnostic practices. N EnglJ Med. 2010;363(1):45–53

9. Tieder JS, Cowan CA, Garrison MM,Christakis DA. Variation in inpatientresource utilization and management ofapparent life threatening events. JPediatr. 2008;152(5):629–635, e621–e622

10. Wennberg JE. Variation in use ofMedicare services in regions andselected academic medical centers: ismore better? 2005. Available at: http://www.commonwealthfund.org/publications/fund-reports/2005/dec/variation-in-use-of-medicare-services-among-regions-and-selected-academic-medical-centers–is-more-b. AccessedOctober 28, 2010

11. Mannix R, Meehan WP, Monuteaux MC,Bachur RG. Computed tomography forminor head injury: variation and trendsin major United States pediatricemergency departments. J Pediatr. 2012;160(1):136–139, e131

e678 JAIN et al by guest on June 16, 2020www.aappublications.org/newsDownloaded from

12. Tieder JS, McLeod L, Keren R, et alVariation in resource use andreadmission for diabetic ketoacidosis inchildren’s hospitals. Pediatrics. 2013;132(2):229–236

13. Fisher ES, Wennberg DE, Stukel TA,Gottlieb DJ, Lucas FL, Pinder EL. Theimplications of regional variations inMedicare spending. Part 2: healthoutcomes and satisfaction with care.Ann Intern Med. 2003;138(4):288–298

14. Kharbanda AB, Hall M, Shah SS, et al.Variation in resource utilization acrossa national sample of pediatric emergencydepartments. J Pediatr. 2013;163(1):230–236

15. Florin TA, French B, Zorc JJ, Alpern ER,Shah SS. Variation in emergencydepartment diagnostic testing anddisposition outcomes in pneumonia.Pediatrics. 2013;132(2):237–244

16. Knapp JF, Simon SD, Sharma V. Variationand trends in ED use of radiographs forasthma, bronchiolitis, and croup inchildren. Pediatrics. 2013;132(2):245–252

17. Choosing Wisely: an initiative of the ABIMFoundation. 2013. Available at: www.choosingwisely.org. Accessed March 15,2014

18. Guttmann A, Razzaq A, Lindsay P,Zagorski B, Anderson GM. Developmentof measures of the quality of emergencydepartment care for children usinga structured panel process. Pediatrics.2006;118(1):114–123

19. Knapp JF, Simon SD, Sharma V. Quality ofcare for common pediatric respiratoryillnesses in United States emergencydepartments: analysis of 2005 NationalHospital Ambulatory Medical Care SurveyData. Pediatrics. 2008;122(6):1165–1170

20. Stang AS, Straus SE, Crotts J, JohnsonDW, Guttmann A. Quality indicators forhigh acuity pediatric conditions.Pediatrics. 2013;132(4):752–762

21. Coon ER, Quinonez RA, Moyer VA,Schroeder AR. Overdiagnosis: how our

compulsion for diagnosis may beharming children. Pediatrics. 2014;134(5):1013–1023

22. Jamtvedt G, Young JM, Kristoffersen DT,O’Brien MA, Oxman AD. Does telling peoplewhat they have been doing change whatthey do? A systematic review of theeffects of audit and feedback. Qual SafHealth Care. 2006;15(6):433–436

23. Forrest CB, Fiks AG, Bailey LC, et al.Improving adherence to otitis mediaguidelines with clinical decision supportand physician feedback. Pediatrics. 2013;131(4). Available at: www.pediatrics.org/cgi/content/full/131/4/e1071

24. Ahwah I, Karpiel M. Using profiling forcost and quality management in theemergency department. Healthc FinancManage. 1997;51(7):48, 50–53

25. Grant L, Proctor T. Measuring quality:how to empower staff to take control.Nurs Times. 2011;107(7):22–25

26. Kruskal JB, Reedy A, Pascal L, Rosen MP,Boiselle PM. Quality initiatives: leanapproach to improving performance andefficiency in a radiology department.Radiographics. 2012;32(2):573–587

27. Meliones JN, Alton M, Mericle J, et al10-year experience integrating strategicperformance improvement initiatives:can the Balanced Scorecard, Six Sigma(R), and Team Training all thrive ina single hospital? In: Henriksen K, BattlesJB, Keyes MA, Grady ML, eds. Advances inPatient Safety: New Directions andAlternative Approaches (Vol. 3:Performance and Tools). Rockville, MD:Agency for Healthcare Research andQuality (US); 2008

28. Wicks AM, St Clair L. Competing values inhealthcare: balancing the (un)balancedscorecard. J Healthc Manage. 2007;52(5):309–323; discussion 323–324

29. Durani Y, Brecher D, Walmsley D, AttiaMW, Loiselle JM. The Emergency SeverityIndex Version 4: reliability in pediatric

patients. Pediatr Emerg Care. 2009;25(11):751–753

30. Gilboy N, Tanabe T, Travers D, RosenauAM. Emergency Severity Index (ESI): ATriage Tool for Emergency DepartmentCare, Version 4. ImplementationHandbook 2012 Edition. Rockville, MD:Agency for Healthcare Research andQuality; 2011

31. Pitts S, Niska R, Xu J, Burt C. NationalHospital Ambulatory Medical CareSurvey: 2006 emergency departmentsummary. National Health Stat Rep. 2008;7:1–38

32. SAS Institute Inc. SAS/STAT 9.2 User’sGuide. Cary, NC: SAS Institute Inc; 2008

33. Hung GR, Chalut D. A consensus-established set of important indicatorsof pediatric emergency departmentperformance. Pediatr Emerg Care. 2008;24(1):9–15

34. Welch SJ, Asplin BR, Stone-Griffith S,Davidson SJ, Augustine J, Schuur J;Emergency Department BenchmarkingAlliance. Emergency departmentoperational metrics, measures anddefinitions: results of the SecondPerformance Measures andBenchmarking Summit. Ann Emerg Med.2011;58(1):33–40

35. Welch SJ, Stone-Griffith S, Asplin B,Davidson SJ, Augustine J, Schuur JD;Second Performance Measures andBenchmarking Summit; EmergencyDepartment Benchmarking Alliance.Emergency department operationsdictionary: results of the SecondPerformance Measures andBenchmarking Summit. Acad EmergMed. 2011;18(5):539–544

36. Pham JC, Kirsch TD, Hill PM, DeRuggerioK, Hoffmann B. Seventy-two-hour returnsmay not be a good indicator of safety inthe emergency department: a nationalstudy. Acad Emerg Med. 2011;18(4):390–397

PEDIATRICS Volume 136, number 3, September 2015 e679 by guest on June 16, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2014-2363 originally published online August 10, 2015; 2015;136;e670Pediatrics 

Shabnam Jain, Gary Frank, Kelly McCormick, Baohua Wu and Brent A. JohnsonQuality, and Efficiency

Impact of Physician Scorecards on Emergency Department Resource Use,

ServicesUpdated Information &

http://pediatrics.aappublications.org/content/136/3/e670including high resolution figures, can be found at:

Referenceshttp://pediatrics.aappublications.org/content/136/3/e670#BIBLThis article cites 29 articles, 9 of which you can access for free at:

Subspecialty Collections

ine_subhttp://www.aappublications.org/cgi/collection/evidence-based_medicEvidence-Based Medicinesubhttp://www.aappublications.org/cgi/collection/quality_improvement_Quality Improvemente_management_subhttp://www.aappublications.org/cgi/collection/administration:practicAdministration/Practice Managementfollowing collection(s): This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtmlin its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprintshttp://www.aappublications.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

by guest on June 16, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2014-2363 originally published online August 10, 2015; 2015;136;e670Pediatrics 

Shabnam Jain, Gary Frank, Kelly McCormick, Baohua Wu and Brent A. JohnsonQuality, and Efficiency

Impact of Physician Scorecards on Emergency Department Resource Use,

http://pediatrics.aappublications.org/content/136/3/e670located on the World Wide Web at:

The online version of this article, along with updated information and services, is

ISSN: 1073-0397. 60007. Copyright © 2015 by the American Academy of Pediatrics. All rights reserved. Print the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elk Grove Village, Illinois,has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on June 16, 2020www.aappublications.org/newsDownloaded from