ORIGINAL CONTRIBUTION

Prioritizing Clinical and Nonclinical Issues

for Emergency Department Quality Assessment Studies

Geoffrey Gibson, PhD Anne L. Kaszuba, BA Baltimore, Maryland

There is a need at both the emergency department and national level for a sys- tematic method to prioritize clinical and nonclinical areas for research and quality assessment. A method has been developed and applied that allows frequently presenting diagnoses and management issues to be rated in terms of morbidity generated, efficacy of (optimal) clinical intervention, discrepancy be- tween optimal and current treatment, and the impact of an evaluative intervention study on improving care. Also, a summary research priority score was derived. When this method was used by physicians, nurses and administrators at the Johns Hopkins Hospital Adult Emergency Department, a reasonable level of agreement between the three responding groups emerged. Gibson G, Kaszuba AL: Prioritizing clinical and nonclinical issues for emergency department quality as- sessment studies. JACEP 7:293-296, August, 1978. emergency d#partment, priorities.

I NTRODUCTION

The Joint Commission on Accreditation of Hospitals, Professional Services Review Organizations, and the Emergency Medical Services (EMS) Systems Acts of 1973 and 1976 have mandated a minimum number of medical audits and are now turning their attention to outpatient and emergency departments. In addition, emergency medicine specialty status and the t ra ining requirements of, residency programs urge greater concern with emergency department quali ty assessment studies.

Despite this, there is no appropriate methodology, or even an acceptable set of guidelines, to help emergency personnel decide which clinical or administra- tive problems can be most usefully evaluated. Clinical areas with extremely low prevalence, thus accounting for minimal morbidity or mortality, are often cho- sen for audit. House officers and emergency department directors, often tempted by research fundings and the growing prestige of emergency medicine, may be opportunistic rather than analytic, and fragmented rather than accumulative in their research strategies.

As 1 have indicated elsewhere, 1 the National Center for Health Services Research focuses for EMS research 2,s are different from what Congress apparently intended in the 1973 Act, a discrepancy that could have been avoided if either

From the Health Services Research and Development Center, Department of Emergency Medicine, Johns Hopkins University and Medical Institutions, Baltimore, Maryland.

Supported in part by Grant No. HS 01907 from the National Center for Health Services Research, DHEW, to the Health Services Research and Development Center, Johns Hopkins Medical Institu- tions. Presented at the Fifth Annual ACEP/EDNA Scientific Assembly in San FranciscO, November, 1977.

Address for reprints: Geoffrey Gibson, PhD, 624 North Broadway, Baltimore, Maryland 21205.

7:8 (August) 1978 JACEP 293/9

T a b l e 1 R E S P O N S E S T O Q U E S T I O N S C O N C E R N I N G D I A G N O S T I C C O N D I T I O N *

CONDITIONS

1. Asthma

2. Head cold/sore throat

3. Nausea and vomiting

4. Headache

5. Nonpleuritic chest pain

6, Skin rash of new onset

7. Hypertension

8. Shortness of breath

9, Diabetes

10. Pregnancy

11. Vaginal discharge

12. Problem with urination

t3. Rectal pain/bleeding '

14, Hematemesis

15. Convulsions

16. Lacerations

17. Low back pain

18. Pleuritic chest pain

19. Closed head trauma

20. Fracture of hand/wrist

21. Sprained ankle

22, Chronic alcohol abuse

23. Anxiety/depression

24. Burns

25. Foreign body in eye

M o r b i ~ Ratings

Total MDs Numes Admin n=20 n=11 n=5 n=4

2.1t 2.2 2.0 2.2 1.1~ 1,1 1.4 .9 1.7 2.0 1,2 1.7

.7 .8 .4 .5 1,9 2.0 1.6 2.0

.7 .8 .5 ,8 1,3 1.4 1.2 1.5

.6 .5 .4 1,0 1.7 1.7 2,0 1.2

.8 .8 1.0 .5 1,2 1.2 1.2 1.5

.4 .4 .4 .6 1.6 1.4 1.8 2.0

.7 .7 1.1 .0 2.5 2,6 2.6 2.0

.9 .8 .9 1.1 2.2 2.4 2~6 1.2 1.0 1.1 .9 .5 2.0 2.2 2.0 1.7 1.1 1.2 .8 .5 1.4 1.4 1.2 1 . 7

.6 .7 ,4 .6 1.4 1,3 1.2 1.7 .6 .5 .5 .9

1.9 1.7 2.2 2.0 .6 ,8 .4 1.0

3,3 3.7 3.2 1.7 1.3 1,1 1,3 .6 3.0 3.3 3.0 2.2 1.4 1.3 1.6 1.9 2.3 2.1 2,2 3.0

.9 .8 .8 1.1 2.6 2.7 2.4 2.5 1.0 1.2 .5 1.0 2.6 2.6 2.8 2.2

.8 .9 ,8 .5 2.6 2.2 3.6 2.7 1.4 1.2 1.1 1,7 3.3 3.4 3.6 2.7 1.3 1.2 1.5 1.7 2.8 2:9 2.2 3.2 1.2 1.0 1.6 1.2: 3.7 3.9 3.4 3.7 1.5 1.4 2.2 1.5 2.6 2.4 2.8 3.0 1.2 .9 1.3 1.8 3.0 3.0 3.6 2.2 1.0 .9 ,9 .9 1.8 2.0 1.8 1.5

.7 .8 .8 .6

* Listed in descending order of frequency.

Discrepancy Ratings

Total MDs Nurses Admln

t Score is average of responses.

3.8 4.4 3.6 2.7 1,4 .9 1.7 1.5 2,2 2.3 2.6 1.7 1.5 1.4 22 .5 2.7 2.8 2.6 2.5 1.2 .9 1,8 1.3 2.1 2.4 2.2 1.5 1.2 1.1 1.6 .6 2.6 2.9 2.6 1.7 1.1 .9 1.5 .5 2.9 2,8 3.2 2.7 1.4 1.1 2.0 1.5 3.3 3.8 2.4 3.2 1.3 1.1 1,5 .9 3.1 3.7 2,6 2.2 1.3 1.0 1,3 1,2 3.0 3.5 2.4 2,5 1.2 ,9 1.5 1,3 2.3 2.7 2.0 1,2 1.2 1.1 1.4 .5 3.2 3.6 2.8 2.7 1.4 1.1 2.0 1.5 3.1 3.5 2.4 2.7 1.4 1.1 1.7 1.5 3.1 3.5 3.0 2.2 1.1 .9 1.6 .5 3.0 3.5 2.6 2.0 1.5 1.5 t.5 1,4 2.9 3.1 2.4 3,2 1.6 1.5 1.9 1.2 4.1 4.5 3.4 4.0 1.2 .7 1.7 1.4 2.0 2.3 1.8 1.7

.9 .9 1.3 .5 2.6 2.9 2.6 1.7 1.0 .7 1.5 .5 2.5 2,8 2.6 1,5 1.3 1.5 1.1 .6 3.6 4.2 3.6 2.2 1.6 1.2 1.7 ~1.9 3.2 4.1 2.4 2.0 1# 1.3 1.9 1.4' 1,7 1.7 1.4 2.2

.8 1.8

.9 3.4 1.2 3.5 1.6

Total

1.1 .8 .4 .5 .6 .5 .7 .7 .7 .7

1.3 .7

1.2 .9 .7 .6

1.0 1.1

.9 1.1

.6

.8

.7

.7

.7

.7

.6 ,5

1.0 .6 .6 .8 .7 .8 .8 .9 .7 .9 .5 .8 .6 .7

1.5 .6 .5 1.5 1.0

1.7 2.2 1.7 1,8 .6 1.3 .9 .9

3.8 3.4 2.2 .8 1,2 .9 1.2 .8 4.1 3.4 2.2 ,4 1.2 1 .7 1.9 .5

:1: Second line is standard deviation.

Efficacy Ratings

MDs Nurses Admin n=11 n=5 n=4

1.2 1,0 1,0 .4 1,4 ,8 .4 ,2 .7 .5 ,4 .6 .5 .4 1.0 ,5 .5 .0 .7 ,6 .7 .8 .5 .6 .8 .2 1,5 ,6 .4 .7

1.6 .6 1,0 ,5 .5 1.0

1.5 1.2 ,5 .8 1.1 .6

1.0 .2 .3 .4 .4 .6

1.3 ,8 ,5 1.0 1.3 1.0 1.1 1.0 .5 1.1 1# .6

.5 .2 t.0

.8 .5 1.0

.9 .2 1.0

.7 .4 1.0

.8 .4 .7

.7 .5 1,1

.7 .4 .3

.5 .5 .6 1.3 .6 .5

.5 .5 .7

.4 .4 1,3

.5 .5 1.5

.7 .6 1.0

.9 .9 .0

.9 .6 .5 1.0 .9 .7 .6 .5 1.3 ,8 1.0 1.5 .6 .2 .5 .8 .4 1.0 .9 .4 .2 .7 .5 .5

1.4 1.8 1.0 .8 1.3 1.0

1.9 2.2 1.2 .9 .8 .9 .7 1.7 .2 .6 .5 .5 ,3 .7 .5 .5 .6 .7

EvaluativeRatings

Total MDs NumesAdmln n = ~ n=11 n=5 n=4

1.4 1.4 1.4 1,7 .8 .7 1.1 .9 .5 ,5 .4 .7 .8 ,8 .5 1.1 .6 .5 .8 .7 .7 ,7 .4 .9 ,9 ,6 1.2 t.7 .9 .8 .9 .6

1.2 1,2 1.0 1.7 .9 1,1 ,8 .6

1,4 1.5 1,0 1.3 .9 .9 ,8 1.1

1,8 2.0 1,4 2.0 1.1 .9 1,3 1.7 1.2 1.2 1,2 1.0

.9 .9' 1.5 .0 1.4 1.4 1.2 2.5 1.0 .9 1.1 .7

.8 .9 1.0 .5 1.1 1.2 1.4 .6 .7 .7 .2 1.5 ,7 .6 .4 .7 ,9 1.1 .2 1.7 .7 .6 .4 ,6 .7 .6 6 1.0 .8 .8 .9 1.0

1.0 1.t .8 1.0 • 6 .7 .8 .0

1.8 2,2 1.0 1.7 1.0 .7 1.0 1.1 1.3 1.4 .2 2.7 1.0 .8 .4 .6 1.0 t.2 .2 2.0 1.0 .9 .4 .0 1,0 1.4 .2 1.0 1.1 1.1 .4 .0 • 9 .8 1.0 1,3 ,7 .6 1.0 .6 ,8 1.0 .2 1.3 .8 .8 .4 .6 ,9 1.2 .2 1.0 ,8 .9 .4 .8

1,5 1.4 1.4 2.0 1,1 1,1 1.3 .8 1,8 1.7 1.5 2.3 1.1 1.2 1.3 .6 1.6 1.0 1.6 1.0 .7 .6 ,9 .0 .7 .9 .7 ,2 .0 .6 .6 .5

the Congress or the National Center had available an effective method for p r io r i t i z ing EMS re sea rch topics. Congress mandated an EMS action- oriented research program while the National Center produced a method- developing program in ambula tory care, quality assessment, and EMS. Congress apparen t ly wanted feas- ibili ty confirmation and implemen- tation of technology from the research program; the National Center con- centrated on test ing relat ive effec- tiveness.

The National Center has stated

tha t it is: 3 " . . . imperat ive tha t a process for ident i fy ing feasible re- search projects of potentially high so- cial ut i l i ty [be] incorporated into the research effort . . . Yet, there is no widely accepted scheme for weight- ing and ordering, in terms of relative impor tance , the myr i ad of hea l t h care problems, t h a t from t ime to t ime, a t t r ac t the a t t en t ion of the public as well as those in the field it- self."

M E T H O D S

To address these deficiencies, a

s tudy was u n d e r t a k e n at the emergency department at The Johns Hopkins Hosp i t a l to develop and apply a methodology to priori t ize clinical and administrat ive topics for qual i ty assessment studies. Physi- cians (residents and faculty), nurses (director, assistant director and shift head nurses) , and admin i s t r a to r s ( regis t ra rs and coordinators) were asked to rate the importance of 25 diagnostic conditions and 11 man- agement issues with regard to the extent of morbidity associated with each, efficacy of medical care, the

10/294 JACEP 7 : 8 ( A u g u s t ) 1978

T a b l e 2 R E S P O N S E S T O Q U E S T I O N S O N M A N A G E M E N T I S S U E S *

Morbidity Ratings Discrepancy Ratings

Total MDs Nurses Admln Total MDs Nurses Admin MANAGEMENT ISSUES n=2O n=20

1. Triaging (within emergency 1 . l t 1.0 1.0 1,7 1.7 department) .5 .0 .0 .9 .9

2. Patient waiting time 1.4 1.2 1.4 1 .O 2.2 .8 .6 ,5 1.4 1.0

3. Scheduling procedures for- 1.8 1.4 2.4 2.2 2.1 OPD follow-up 1.0 .7 1.1 1.2 .9

4. Patient compliance with 2.6 2,5 3.0 2.2 2.4 scheduled follow-up appointments 1.1 .7 1.6 1.5 .8

5. Patient walk-outs 1,4 1.4 1.2 1.3 1.4 ,5 .5 .4 .6 .7

6. Nonurgent utilization of the 1.7 1.7 1.8 1.5 2.2 emergency department 1.2 1.3 t .3 1 .O .9

7. Waiting time for lab test 1.7 1.4 1.8 2.5 2.2 results I .O .9 .8 1.8 .8

8. Nursing exit interviews 1.5 1.3 1.2 2.2 1.1 .7 .5 .4 .9 ,9

9. Nursing audit 1.5 1.3 1.2 2.2 .9 .8 .5 .4 1.2 .8

10. Patient advocacy program 1.3 1.2 1.4 1.7 1.0 .5 .4 .5 .6 .8

11. Referrals to primary care 1.3 1.3 1.2 1.5 .9 center .6 .5 .4 1.0 .8

n=11 n=5 n=4

1.8 1.2 2.2 .9 .8 .9

2.4 1.8 2.0 .8 1.3 1.4

2.2 2.0 2.2 .7 1.2 .9

2.7 2.2 2.O ,5 1.3 .8

1.5 1.2 1.5 .5 .8 1.0

2.4 2.2 2.0 .8 1.3 .8

2.3 1.8 2.5 ,6 1.3 ,6 .9 1.4 1.5 .9 .9 1.0 .8 1.2 1.0 ,8 .8 1.0 .9 1.0 1.2 .8 .7 .9

1.0 1.0 .7 .8 .7 .9

Effcacy Ratings

Total MDs Nurses Admin

2.2 2.5 1.4 2,5 1.4 1.4 .9 1.7 3.1 3.3 2.4 3.7 1.4 1.4 1.5 .9 2.6 2,4 2.6 3.0

.7 .8 .9 .O 2.8 2.8 2.7 3.0

.9 1.2 .9 .0 1.9 1.8 1,2 3.0

.9 .6 .4 .8 3,0 3.0 2.8 3.2 1.6 1.8 1.6 t .2 2.7 2.4 2.6 3.5 1.4 1.5 1.5 1,3 2.1 2.0 2.0 2.5 1.3 1.2 1.7 1.3 2.1 1.7 2.5 2.7 1.2 .8 1.7 1.5 1.9 1.5 2.6 2.2 1.4 1,2 1.8 1.2 2.0 1.9 2.0 2.5 1.5 1.6 1.2 1.7

EvaluaUve Ratings

Total MDs NursesAdmin n = ~ n=11 n=5 n=4

1,6 1.5 1.4 2.2 1.0 1,0 1.1 .9 1.8 1.8 1.8 2.0 1.0 1.0 1,1 .8 1,9 2.1 1.6 2.0

,9 .9 1.1 .8 1,9 2.0 1.8 1.7 1.0 1.0 1.3 .5 1.2 1.2 1.0 1.2 .9 1.1 .2 ,9

1.8 2,0 1.6 1.7 1.0 1.1 1.1 .9 1~8 1,6 2.0 2.0 1.0 1,0 1.4 .8 1.2 1,1 1.4 1.2 1.0 1.1 1.1 ,5 1,2 .9 .8 1.7 ,9 .9 1.2 .9

1,0 .8 1.2 1.5 .9 .9 .8 1.0

1.0 1.1 1,0 1.0 .7 .8 .7 ,O

* Some, but not all, of the following management issues will be applicable to all patients in each emergency department. For each issue, consider only those patients whose visit is or should be affected by the issue. For example, in assessing the extent of morbi~lity related to mismanagement of triaging within the emergency department, consider only patients who are, or should he triaged. The judgment to he made, therefore is: Of all patients who are or should be triaged, what proportion experiences significant morbidity due to mismanagement of triaging?

t Score is average of responses. :1: Second line is standard deviation.

disparity between optimal care and p r e s e n t l y r e n d e r e d care, a n d the l ikelihood of a qua l i t y assessment s tudy be ing able to improve care. The 25 d iagnost ic condi t ions were the most frequent complaints at The Johns H o p k i n s A d u l t E m e r g e n c y Depar tmen t while the 11 adminis- t ra t ive issues were the nonc l in ica l problems most frequently nominated by emergency depar tment staff. Usa- ble responses were received from 11 physicians, five nurses, and four ad- minis t ra tors , a response rate of 71 percent . Fo l l owing the successful work of Wil l iamson 4 in developing a smal l group consensus by c l in ica l staffs for h i s q u a l i t y a s s e s s m e n t studies, for each of the 25 diagnoses and the 11 m a n a g e m e n t issues, re- spondents were asked to answer the following, in the i r judgement , and check the a p p r o p r i a t e r e sponse category:

1) What proport ion of pa t ien ts p r e s e n t i n g to your emergency de- par tment with each condition experi-

* Significant morbidity is defined as two or more days of activity limitation, ie, days kept from work, school or daily ac- tivity,

enced significant morbidity* caused by the condition?

1 = 0-20%, 2 = 21-40%, 3 = 41- 60%, 4 = 60-80%, 5 = 81-100%

2) Wha t propor t ion of pa t ien ts p r e s e n t i n g to your e me r ge nc y de- pa r tment with each condition should be signif icantly improved after opti- ma l e m e r g e n c y d e p a r t m e n t t rea t - ment, (ie, if every th ing clinically ap- propriate was done)?

I = 0-20%, 2 = 21-40%, 3 = 41- 60%, 4 = 61-80%, 5 = 81-100%

3) How much difference is there b e t w e e n o p t i m a l t r e a t m e n t a nd t r ea tmen t tha t is current ly given in your emergency depar tment for each condition?

0 = none, 1 = a l i t t le , 2 = a moderate amount , 3 = a great deal.

4) What is the probable impact of the evaluat ive research (ie, tes t ing an in te rven t ion or change in proce- dures) in reducing the difference be- tween op t imal t r e a t m e n t and tha t which is c u r r e n t l y g i v e n in y o u r emergency depar tment for each con- dition?

0 = none, 1 = a l i t t le , 2 = a moderate amount , 3 = a great deal.

An overall summary score was ca l cu la t ed for each d iagnos i s and

issue and for each of the responding groups (physicians, nurses, and ad- ministrators) giving equal weight to each of the four criteria. In addition, respondents were g iven the oppor- tun i ty to ~write in" diagnoses and is- sues of par t icular evaluat ive impor- tance to them.

R E S U L T S

The mean scores and s t andard devia t ions when respondents ra ted the 25 diagnoses are presented (Ta- ble 1).

For morb id i ty , alcohol abuse , fracture of hand/wrist , hematemesis, bu r ns a n d convuls ions were ra ted highest (Table 1). There was an in- verse association between frequency and morb id i ty and genera l agree- ment between physicians, nurses and to a lesser extent administrators.

In terms of efficacy, lacerations, a s t hma , a nd h a n d / w r i s t f r a c tu r e s were rated as most improvable by op- t imal emergency depar tment treat- m e n t (Table 1), phys i c i ans r a t i n g each of the three higher t han nurses. In te res t ing ly , phys ic ians ra ted the efficacy of op t ima l e me rgency de- pa r tmen t t r ea tmen t more highly for 22 of the 25 diagnoses than did the nurses or administrators .

7:8 ( A u g u s t ) 1 9 7 8 JACEP 2 9 5 / l t

For the perceived gap between optimal and present emergency de- par tment care, the results offer a somewhat enthusiastic and uncriti- cal view of emergency department care in the minimal discrepancies seen between present and optimal care (Table 1), with the exception of chronic aIcohol abuse and anxiety/ depression.

• On the other hand, there was an apparent pessimism about the likeli- hood of evaluation and intervention studies reducing the gap between op- t imal and present care (Table 1). This pessimism could be due to either a pessimism about evaluative studies or an optimism about the minimal gap between optimal and actual care that the respondents be- lieved existed at the Hopkins emergency department. In any event, convulsions, anxiety/depression, and hypertension were identified as high impact areas. There seems to be a reasonable consensus between physi- cians, nurses and administrators.

This process was repeated for the 11 management issues. The respond- ents believed that management is- sues were much less important than presenting complaints in generating s ignif icant morbidi ty (Table 2). Nevertheless , pa t ien t compliance and referrals to other outpat ient clinics were identified as the most important morbidity-creating issues. Patient waiting time, nonurgent use, and pa t ien t compliance with scheduled follow-up appointments were regarded as areas where opti- mal managemen t could most im- prove patient outcomes (Table 2). Pa- t ient compliance, pa t ient wai t ing time, nonurgent use and wait ing time for laboratory test results were the areas with the greatest discrep- ancy between optimal and current practice (Table 2). Pa t ien t com- pliance and outpatient department (OPD) appointment scheduling was the area most likely to be improved by evaluative research (Table 2).

A summary score was calculated for each diagnosis and management issue by equal weighting of each of the four criteria. These scores range from a possible low score of 0 to a possible high of 25. The diagnoses re- ceiving highest priority were convul- sions, chronic alcohol abuse, anxiety/depression and asthma (Ta- ble 3). Physicians rated convulsions, hypertension and as thma highest while nurses ra ted anxiety/ depression and chronic alcohol abuse

Table 3 SUMMARY SCORES FOR DIAGNOSES

Mean Median Standard Condition Rank* Score Score Deviation

Convulsions 1 13.43 12.08 4.10 Chronic alcohol abuse 2 13,01 13.75 4.76 Anxiety/depression 3 12.92 11.32 4.62 Asthma 4 12.81 12.71 4.37 Hypertension 5 12.56 12.92 2.73 Burns 6 12.17 10.83 3.78 Lacerations 7 12.06 11.67 4.15 Diabetes 8 11.55 10.49 4.83 Fracture of hand/wrist 9 11.50 11.46 3.11 Hematemesis 10 11.01 10.56 3.99

* Of the 25 diagnoses rated, the ten that ranked highest are shown.

Table 4 SUMMARY SCORES FOR MANAGEMENT ISSUES

Mean Median Standard Management Issues Rank* Score Score Deviation

Patient compliance with scheduled follow-up appointments 1 15.28 16.04 5.50

Nonurgent utilization of the emergency department 2 14.42 13.54 5.94

Patient waiting time 3 14.15 13.33 5.23 Scheduling procedures for OPD

follow-up 4 14.20 14.58 4.58 Waiting time for lab test results 5 13.90 14.37 5.24

*Of the 11 management issues rated, the live that ranked highest are shown,

highest. The priorities given to man- agement issues related to morbidity are summarized (Table 4). Patient compliance and nonurgent use re- ceived highest priority by all respon- dents as well as by physicians when considered separately.

CONCLUSION

The method we have described is probably more important than the results associated with its use at Hopkins.

This method may well have po- tential for use by the Research Com- mittees of the American College of Emergency Physicians and the Uni- versity Association for Emergency Medicine in identifying high priority areas in emergency medicine, initiat- ing well focused, collaborative stud- ies, securing federal financial support

for them, and in enlarging a distinct body of emergency medicine knowl- edge through highly selective and effective prioritizing techniques.

REFERENCES

1. Gibson G: Emergency medical services research: Methodology development or substantive application. Health Serv Res 12:44-55, 1977. 2. A Program for Research in E M S Re- search. National Center for Health Ser- vices Research, Department of Health, Education and Welfare, April, 1975. 3. The Program in Health Services Re- search. National Center for Health Ser- vices Research, Department of Health, Education and Welfare, September, 1976. 4. Williamson J: Formulating priorities for quality assurance activity: description of a method and its application. JAMA 239:631-637, 1978.

12/296 JACEP 7:8 (Aug ust) 1978