OR~IGINAL CONTRIBUTION

Measures of Emergency Ambulance Effectiveness:

Unmet Need and Inappropriate Use

Geoffrey Gibson, PhD Baltimore, Maryland

For Heal th S y s t e m s A g e n c i e s to fulf i l l the i r n e e d s a s s e s s m e n t a n d faci l - ity regu la t ion f u n c t i o n s u n d e r the N a t i o n a l H e a l t h P l a n n i n g a n d Re- sources D e v e l o p m e n t A c t o f 1974 (PL 93-641), p r o c e s s m e a s u r e s are needed to e v a l u a t e the o v e r a l l e f f i c i e n c y o f an e m e r g e n c y a m b u l a n c e system. In E r i e C o u n t y , N e w Y o r k , u n m e t a m b u l a n c e need , d e f i n e d as the p r o p o r t i o n o f e m e r g e n c y p a t i e n t s w h o c l i n i c a l l y n e e d a m b u l a n c e transportation but do n o t r e c e i v e it, w a s f o u n d to b e 55%. I n a p p r o p r i a t e ambulance use , d e f i n e d as t h e p r o p o r t i o n o f e m e r g e n c y p a t i e n t s r e c e i v - ing a m b u l a n c e c a r e w h o d id n o t c l i n i c a l l y n e e d it, w a s f o u n d to be 30 %. These t w o m e a s u r e s w e r e c r o s s - t a b u l a t e d w i t h p a t i e n t a n d v i s i t c h a r a c - teristics to i d e n t i f y the d e t e r m i n a n t s o f u n m e t n e e d and i n a p p r o p r i a t e use and to g e n e r a t e r e c o m m e n d a t i o n s for s y s t e m c h a n g e s to r e d u c e b o t h types of a m b u l a n c e i n e f f e c t i v e n e s s .

Gibson G: Measures of emergency ambulance effectiveness: unmet need and inappropriate use. JACEP 6:389-392, September, 1977. Health Planning and Resources Development Act of 1974; ambulance, effectiveness.

INTRODUCTION

The Emergency Medical Services Systems (EMSS) Act of 1973 (Public Law 93-154) and the Amendments of 1976 are a major federal in i t ia t ive to

Revised and updated version of paper pre- seated at American Public Health Associa- tion meetings, New Orleans, Louisiana, October, 1974.

~ Upported in part by Grant No. HS 1907, from the National Center for

Health Services Research DHEW to Health Services Research and Develop- i t nt Center, The Johns Hopkins Univer-

Y and Medical Institutions, Baltimore, aryland.

ddress for reprints: Geoffrey Gibson, , HSR & D Center The Johns Hopkins

edical Institutions '624 N Broadway, 13altimore Maryland' 21205.

J ~ P 6:9 (sept)1977

improve emergency medical services (EMS) at state., regional and local levels. These are also a po t e n t i a l source of p r o g r a m g u i d e l i n e s and evaluat ive cr i ter ia for EMS compon- cuts. The EMSS Act requires tha t an emeTgency ambulance service in an EMS system shall

include an adequate number of neces: sary ground, air and water vehicles and ather transportation facilities to meet the individual characteristics of the sys- tem's service area. Such vehicles and facilities must meet appropriate standards relating to location, design, performance and equipment; and the operators and other personnel for such vehicles and facilities must meet.appropriate training and experience reqdi~ments. 1

While this wording is more specific t han previous legislative defini t ion

of the ambulance subsystem, it does not sufficiently guide the approval of EMS gran t applications, the monitor- ing of funded activities, or continu- ing a s ses smen t of the adequacy of ambulance services and the impact on tha t adequacy of the EMSS Act. S imi l a r ly , the leg is la t ive word ing does not enable a communi ty to as- sess its ambulance system to deter- mine what, if any, improvements are indicated or the impact of program intervent ions . This is regrettable in l ight of the assessment and evalua- t ion responsibi l i t ies of Heal th Sys- tems Agencies (HSAs) ar is ing out of Pub l ic Law 93-641, The N a t i o n a l Heal th P l a n n i n g and Resources De- velopment Act of 1974.

The major federal legislation and regulat ions use input criteria almost e x c l u s i v e l y to define adequacy of a m b u l a n c e services (ie, the avai l - ab i l i ty of resources) with on ly one process c r i t e r i o n ( resource u t i l i - zation) and no outcome criterion (pa- t ient ' s hea l th status). Input criteria include t r a i n i n g levels of ambulance p e r s o n n e l , t he p resence of radio communicat ions; ambulance first aid and e x t r i c a t i on equ ipmen t , ambu- l a nc e d e s i g n factors r e f e r r i n g to headroom, and avai labi l i ty on a 24- h o u r basis. The one example of pro- cess cr i ter ia is a suggested response t ime (ambulance station to accident site) of 30 minu tes or less for 95% of

389/13

ambulance runs in rura l areas and of ten minu tes in u rban areas.

Resea rch and e v a l u a t i o n l i t e ra - ture 2 on emergency ambulance sys- tems shares most of the l imita t ions of the legislative regula t ions on ambu- lance adequacy. Cur ren t evaluat ive c r i t e r i a for e m e r g e n c y a m b u l a n c e services

• de f ine a m b u l a n c e a d e q u a c y m e r e l y as c o m p l i a n c e w i t h specified inpu t resources and un- crit ically assume that if such re- sources are present they will be used, and therefore will affect pa t ien t hea l th outcomes.

• conta in only one process measure, response t ime , which does no t measure the qual i ty or quan t i t y of service provided but only vehi- cle speed and service area geo- graphic constraints .

• fail to provide re levant informa- t ion on p a t t e r n s of a m b u l a n c e use to ind ica t e needed sys t em changes.

• do no t i n c l u d e a n y ou tcome measures because of research de- sign difficulties in controll ing for exogenous var iab les , ad ju s t i ng for i l lness / in ju ry severity, and, therefore, be ing able to a t t r ibute a known and measurable portion of the patient 's outcome to a known and m e a s u r a b l e aspect of the emergency ambulance service.

• ignore the impor tan t issue of ap- p ropr ia teness of cu r r en t ambu- lance use, and conversely wheth- er pa t i en t s need ing ambu lance transportation are not receiving it.

Unmet Ambulance Need and Inappropriate Ambulance Use

We suggest two new measures of appropr ia teness of ambulance uti l i- za t ion : u n m e t need a n d i n a p p r o - priate use (Figure 1). Unmet need is t h a t p ropor t ion of e m e r g e n c y pa- t ients who need ambulance care bu t do not receive it (c/a 4- c) for one of several reasons: 1) they did not re- quest it; 2) they did not know how to access the system; or 3) their request for ambulance services was denied• Although such pat ients should be fol- lowed to d e t e r m i n e t h e i r course , unme t need is an attractive over-all i nd ica to r of the emergency ambu-

14/390

Patient Used Ambulance Service

Patient Needed Ambulance Service

Yes No Yes a b No c d

Unmet Ambulance Need = c/a~+c Inappropriate Ambulance Use = b/a + b

Fig. 1. Distribution of emergency department patients by use of and need for ambulance service.

lance system's effectiveness• Wi th in the context of t h e Nat ional Heal th P l a n n i n g and Resources Develop- men t Act of 1974, "unmet ambulance need" will be a useful measure for HSAs for their in i t ia l needs assess- m e n t in developing ~mul t i - ins t i tu - t i ona l sys t ems for coord ina t ion or consolidation of ins t i tu t iona l hea l th services [ inc lud ing . . . emergency m e d i c a l . . , se rv ices] a nd the de- v e l o p m e n t of effective methods of e d u c a t i n g the ge ne r a l publ ic con- ce rn ing proper personal . . . hea l th care and methods for effective use of ava i lab le hea l th services" (Section 1502). In addit ion, HSAs, as regu- lators, will have to assess the need for new or al tered medical facilities and services. A signif icant source of program proposals will be g ran t ap- p l i c a t i o n s by local EMS s y s t e m s under t h e Emergency Medical Serv- ices Systems Act of 1973. HSAs and the associa ted s ta te agencies may wish to use '~unmet ambulance need" and ~inappropriate ambulance use" as cr i ter ia in judg ing EMS proposals and de te rmin ing priorities. In addi- tion, emergency physicians may find these measures and the insights they provide u s e f u l in a s s e s s i n g the adequacy of ambulance services at the i r facilities or wi th in their EMS systems.

Pat ients fal l ing into the group of inappropria te a m b u l a n c e use - - the proportion of all ambulance pat ients who do not need such care - - (b/a 4- b) (Figure 1) do so for only one rea- son, the ambulance dispatcher inap- propriately accepted the request for ambulance. This cri terion measures the misal location of expensive ambu- lance resources and, more specifical- ly, of s c r e e n i n g eff ic iency. Such ~false posi t ive" errors (pa ra l l e l ing the "false negat ive" errors denoted in un me t ambulance need) may result

from the d i spa tche r ' s inab i l i ty or unwi l l ingness to secure an accurate history of the patient, his belief that the pa t ient may be exaggerating the symptoms, or the caller's inability or unwi l l i ngnes s to convey accurately the u r ge nc y of the s i tua t ion . Pre. cisely to al leviate these difficulties, the D H E W p r o g r a m guidelines under the EMSS Act call for the use of ~nurse screeners" at central ambu. lance dispatch points.

MATERIALS AND METHODS

Measures of ambu lance use sup ported by data were developed by the Western New York EMS Project for which the author served as Director of Research and E v a l u a t i o n part. t ime b e t w e e n Ju ly , 1972 and De- cember 1973. The EMS project, cov- e r ing seven New York counties and two n o r t h e a s t P e n n s y l v a n i a c0un. ties, functioned under a grant from the Lakes Area Regiona l Medical Program, and subsequent ly , under Section 1202 and 1203 grants from DHEW. The t h r e e m a i n program areas are 1) t r a i n i n g of ambulance and hospital staff; 2) communications equ ipmen t for hospitals and ambu- lances, and 3) research and evalua- tion. The in i t i a l effort was i n Erie C o u n t y (Buffalo) . R e s e a r c h and e v a l u a t i v e b a s e - l i n e ac t iv i t ies in- cluded on-site surveys of all ambu: lance and hospi ta l facilities, chart review of all pat ients treated in all hospital emergency departments dur, ing four on ~-week periods, review of ambulance records on all emergency r u n s ( even w h e n no p a t i e n t was t r anspor t ed ) for the same periods, i n t e r v i e w of a l l p a t i e n t s in each emergency depar tment dur ing thel~ t r e a t me n t for a 24-hour period, an" follow-up interviews with a sample of these pa t ients three months later.

6:9 (SePt) 1977 ~ P

Table 1 DISTRIBUTION OF

EMERGENCY PATIENTS BY NEED FOR AMBULANCE

n %

No 20,523 85 probably no 652 3 undecided 344 1 probably yes 1,231 5 Yes 1,458 6

TOTAL 24,208 100

No data 386

Table 2 AMBULANCE USE BY AMBULANCE NEED

Yes* n %

Ambulance Needed

No** Total n % n

Ambulance Yes 1224 45 563 3 1,787 Used No 1465 55 20,956 97 22,421

2689 100 21,519 100 24,208

*"Yes" and "Probably Yes" (Table 1) .... No" and "Probably No" and "Undecided" (Table 1)

%

7 93

100

The data is based in the 24,594 emergency depar tment medical rec- ords abstracted for all patients pre- senting at 23 hosp i t a l s in Er ie County d u r i n g the per iods Sep- tember 12 to 18 and December 12 to 18, 1971 and March 12 to 18 and June 12 to 18, 1972 a n d o n the 888 patients interviewed: A highly skilled and experienced t r a u m a nurse did the i n t e rv i ews and r e v i e w e d the clinical abstracts prepared on each of the 24,594 emergency d e p a r t m e n t patient medical records con ta in ing information on pat ient age, sex, pre- senting complaint, associated symp- tomatology and time of presentation. The nurse did not know whether the patient received ambulance service and she was asked to judge whether each patient clinically required am- bulance transportat ion.

RESULTS

Of the patients reviewed, 85% did not need an ambulance; 3% probably did not need an ambulance. Six per- cent c e r t a i n l y and 5% p r o b a b l y needed an ambulance (Table 1). The aggregated "need" judgment is com- pared to the actual provision of am- bulance care (Table 2) in a format that corresponds to Figure 1. Thus, of the 2,689 pa t ien t s need ing ambu- lance service, 1,465 (55%) did not re- Ceive it, ie, unmet ambulance need is 55%. Of the 1,787 patients receiving arabulance care, 563 (31%) did not need it, ie, inappropriate a m b u l a n c e Use is 31%.

Program eva lua t ion should help explain how variance within the pro- gram operation, or client characteris- tics, influences the evaluat ion. By identifying such characteristics, rec- °~aa]endations for change can be

~[]~P 6:9 (Sept)1977

Table 3 AGE

% % Inap- Age Unmet propriate

in Years Need Use

0- 4 71 39 5-18 73 48

19-39 54 39 40-64 52 27 65+ 42 20

Table 4 SEX

% % Inap- Unmet propriate

Sex Need Use Men 54 27 Women 54 36

Table 5 INSURANCE COVERAGE

Type

Blue Cross/Blue Shield -Medicaid

Medicare Welfare Worker's Compensation Other None

% % Inap- Unmet propriate Need Use

58 30 49 30 44 21 49 48 79 58 60 20 61 28

Table 6 VISIT CHARACTERISIC BY SHIFT

Midnight - - 8 am 8 am - - 4 pm 4 pm - - Midnight

% % In- Unmet appro- Need priate

Use

42 27 55 32 57 32

Table 7 VISIT CHARACTERISTIC

BY DAY OF WEEK

Sunday Monday Tuesday Wednesday Th u rsday Friday Saturday

All Patients

% % Inap- Unmet propri- Need ate

Use

50 28 58 33 52 25 53 37 49 28 58 35 52 32

53 31

391/15

Table 8 PATIENT RESIDENCE

% %

Unmet Inappropriate Need Use

Buffalo 53 32 Grand Island 42 12 Tonawanda City 35 29 Town of Tonawanda 54 19 Kenmore 65 .22 Amherst 27 24 Cheektowaga 52 37 West Senaeca 70 34 Lackawanna 62 45 Clarence 50 44 Lancaster 45 0 Orchard Park 54 8 Hamburg 37 30 Elma 50 0 Aurora 0 0

TOTAL (Suburbs) 35 29 TOTAL (Erie County) 55 31

Buffalo Districts: South Side 61 18 Model Cities 46 39 East Side 57 34 North Side/University 63 43 Black Rock/Riverside 29 17 West Side 54 22 Lower West Side 46 38

Table 9 PHYSICIAN'S JUDGMENT - - AMBULANCE NEED

Patient's Use of Ambulance

Yes No

TOTAL

Ambulance Need Yes No Total

55 23 78 50 681 731

105 704 809

Unmet Ambulance Need = 48% Inappropriate Ambulance Use = 29%

made in this part icular case, aimed at reducing unmet ambulance need and inappropr i a t e ambulance use. Both unmet ambulance need and in- appropr ia te ambulance use decline with increasing pat ient age, suggest- ing differential thresholds of age used by both dispatchers and citizens for ca l l ing and d i spa tch ing an ambu- lance (Table 3). Sex differences are only noticeable in the rate of inap- propriate ambulance use with women exhibi t ing a h igher rate (Table 4).

Pat ients with no health insurance have a higher rate of unmet ambu- lance need and a lower rate of inap- propriate use when compared with insured patients, perhaps indicating highly selective screening by the dis- pa t che r (Table 5). U n m e t need is highest dur ing the evening shift, the period of highest demand for ambu- lance service (Tables 6 and 7).

Measures should be presented for each ambulance unit so tha t each can

16/392

e v a l u a t e its pe r fo rmance : Als0, measures are given for each political jurisdiction within Erie County aa~ each of the seven subareas withi~ Buffalo (Table 8). While there is lit. tle difference between the city a~ d the suburbs with regard to inappr0. priate ambulance use (32% vs 2~ respectively), unmet ambulance need is subs t an t i a l l y h igher in the city (53%) t h a n in the suburbs (35%t.

Since one nurse made the "a~bu. lance need" judgments on the 24,594 p a t i e n t s f rom re t rospec t ive docu. mentat ion, an a t tempt was made to reproduce the measures with the at. tending physicians who treated the 888 i n t e r v i e w e d pa t i en t s . Each physician was asked to judge if the patient should have received ambu. lance t r a n s p o r t a t i o n (Table 9). In this group, 48% had unmet ambu. lance needs and 29% had inappr0. priate ambulance use, values com. parable with the nurse 's judgment on a different sample.

Several l imitat ions of these data should be noted. The judgments on t he c l in ica l need for ambulance t r a n s p o r t a t i o n were made by on~ t r a u m a nurse retrospectively. And, while we know that she made reli. able (ie, reproducible) judgments and tha t her assessments broadly corre. spond to those made by attending physicians, the judgments were not made off the basis of explicit and pro. spective standards, nor even on an implicit case-by-case panel review. In addition, the assessment referred only to the clinical need for ambulance t ranspor ta t ion to the emergency de, partment; it did not refer explicitly to the need for clinical intervention by the a m b u l a n c e personnel , nor even to the detailed basis for the pre. sumed need. At best, the "need as. sessment" judgments are gross aggre gates appropriate for a system-wide evaluat ion but lacking in conceptual and operat ional detail.

REFERENCES

1. Emergency Medical Services System: Program Guidelines. Health Services Administration, Public Health Admi~" istration, US Department of Health, Education and Welfare, 1973. 2. Gibson G: Evaluative criteria for emer" gency ambulance systems. Social S cie#e and Medicine 7.426-454, 1973.

6:9 (Sept)1977