Michael L Washington, PhDDeputy Director, Preparedness Modeling Unit

Industrial & Systems EngineerCenters for Disease Control and Prevention

August 2009

Evaluating the Capacity, Efficiency, and Cost of a

Mass Influenza/ Pneumococcal Vaccination

Clinic Via Simulation DIMACS/MBI US - African BioMathematics Initiative: Workshop on

Economic Epidemiology, 3 Aug 2009

Engineering Jokes

Engineers aren’t boring people, we just get excited over boring things.

Half Full or Half Empty?• To the optimist, the glass is half full. • To the pessimist, the glass is half empty. • To the engineer, the glass is twice as big

as it needs to be.

Agenda

• Measurements to evaluate model or actual clinic (including efficiency)

• Estimating the capacity of a mass vaccination clinic

• Issues with the optimized clinic

Measurements

• How do you evaluate a good and efficient clinic?

• First, let us evaluate a mock clinic and learn about a technique to improve the efficiency of a clinic.

Line Balance

8 hours or 480 minutesArrival Rate = 1 person per minute1 person per station

FYI. Does not reach a steady state for 1 to 2 hours and queuing theory does not work.

0.1 min 0.7 min 1 min 0.5 min 0.2 min

Which Set-up is Best?And what measure to use?

0.1 min 0.7 min 1 min 0.5 min 0.2 min

0.1 min 0.7 min 1 min 0.5 min 0.2 min

Station 10.8 min

Station 30.7 min

Station 21 min

0.1 min 0.7 min 1 min 0.5 min 0.2 min

Station 12.5 mins

1

2

3

Station 1 Station 2 Station 3 Station 4 Station 5

Some Measures of Performance

• Number of people served • # of people served or treated over time

(Throughput)• Average time people are waiting or in the

clinic• Average # of people in clinic or station (WIP)• Resource utilization – people and equipment• Cost• Customer satisfaction• Effort per person (i.e., $/person or CE ratio)

Measures

Measurement Pros Cons

Number of people served

Simple - count peopleFamiliar

No measure of efficiencyNo customer satisfaction

measure

Throughput Simple - people/hrFamiliar

No measure of efficiencyNo customer satisfaction

measure (a Honda plan - 36 cars/hr)

Exit rate Simple – ID bottleneckHelps with things

down the line

No measure of efficiencyCan be miss leading

Measures

Measure Pro Con

Clients time in the clinic

Measure of efficiency and customer satisfaction

Measure variabilityIf measured per stations,

measure trainingEasy way to ID bottleneck

Additional data requiredDifficult to collect per

stationNo standardsNot = throughput

Ave # of people in clinic (or station)

Measures of efficiencyIf measured per station,

can identify bottleneckCan easily measure for

the entire clinic

May have to be done electronically

Difficult to measure per station manually.

Measures

Measure Pro Con

Resource utilization

Measures efficiencyID over- or understaffing Can be calculated post-

event if the right data are collected

Does not provide information about one main goal

Cost Measure of resources needed to accomplish an objective

Compare across clinics

It takes time to consider all costs

Some consider it unimportant in an emergency (which I think is a mistake)

Measures

Measure Pro Con

Customer satisfac-tion

Being able to measure how the people you serve feel about your service is always a great idea.

Surveying clients is trick, and clients can be very fickle.

$/person Probably the bestMeasure the effort

involved in treating one person

Easy to get cost if documentation is good

May be difficult to collect cost, especially indirect cost

No standardDon’t worry about $ in

emergencies (mistake)

Results (100 Runs)Mean (min, max)

Measurement 1 2 3

Number of people served

454 (411, 510)

454 (409, 494)

192 (175, 213)

Throughput(client/hr)

57 (51, 64)

57 (51, 62)

24 (22, 27)

Exit rate 1 1 2.5

Results (100 Runs)Mean (min, max)

Measurement 1 2 3

Clients time in the clinic (min)

19 (7, 57)

19 (9, 45)

145 (122, 172)

Average # of people in the clinic (people)

26 (0, 96)

28 (2, 81)

287 (219, 342)

Resource utilization (%)

48 (40, 56)

80 (71, 88)

100 (99, 100)

Results (100 Runs)Mean (min, max)

Measurement 1 2 3

Direct Cost ($28/hr/staff)

$1,120 $672 $224

Indirect Cost ($18/hr/client)

$155,372($52,478, $523,967)

$160,795($64,059, $396,121)

$502,961($387,009,

659,256)

Total Cost $156,492($53,598, $525,089)

$161,467($64,722, $396,792)

$503,186($387,233, $659,480)

Results (100 Runs)Mean

Measurement(easiest)

1 2 3

Direct Cost/client

2.47 1.48 1.17

Indirect Cost/client

343 354 2,626

Total Cost/client

345 356 2,627

Agenda

• Measurements to evaluate model or actual clinic (including efficiency)

• Estimating the capacity of a mass vaccination clinic

• Issues with the optimized clinic

Simulation Model

• Evaluate one clinic design

• The main measure is the # of people vaccinated

• We will examine other measures in evaluating the clinic

• Try to improve the clinic design based upon the main measure

Optimizing a Clinic

• History– Anticipated vaccination 15,000 in 17 hours – Only 8,300 showed up and were vaccinated– Could they have vaccinated 15,000 with

current design and staff

• Simulation (very difficult with queuing)– Arrival rate was not consistent– Violate a big rule in queuing (service rate >

arrival rate)– Looked to optimize staff placement

Clients

• “Medicaid” – Assumed to be retired, > 65 years old

• “Special”– A sub-population of “Medicaid”– Usually the physically challenged

• “Cash” – Normal work force

Office

500 sq. ft. Office

300 sq. ft.

RN R

RN RN

R R R R R R

CASH

RNRN RNRN RNRN RNRN RNRN

R

Educational Display

Ed

Disp

lay

Work Station Vaccine PrepStaff Break

Area

Pneumonia Shot

Special Needs

C C

10%

20%

90%

10%

70%

10%90%

EnterExit

C C

C

Special Need

Cash

Regular Medicare

C = CopierRN = ShotsR = Registration

Staff Sits Patient Stands or Sits

Key

Staff

• Special Copy• Special Flu Form• Special Flu Vac• Pnu Form

Yellow = RNWhite = Support staffContractor/Volunteer not included (EMT, security, runners, etc)

• Pnu Vac• Medicaid Copy• Medicaid Flu Form• Reg Flu Vac• Cashier

InputsService Time at Each Station

Station # of Staff a

Type of Staff Distribution b

(SD)(minutes)

Source n

Special Flu Copy 1 AP Beta 0.40 (0.813) EO -

Special Flu Registration 2 AP Gamma 2.63 (0.930) TS 102

Special Flu Vaccination 2 RN Lognormal 1.13 (0.590) TS 158

Pnu Registration 2 AP

Medicare Client Lognormal 1.01 (0.416) TS c

Special Client Gamma 2.18 (3.140) TS c

Pnu Vaccination 4 RN

Medicare Client Lognormal 1.18 (0.561) TS c

Special Client Lognormal 1.13 (0.590) TS c

Medicare Copy 4 AP Beta 0.40 (0.813) EO -

Medicare Registration 18 AP Lognormal 1.01 (0.416) TS 402

Medicare/Cash Flu Vaccination 20 RN Lognormal 1.18 (0.561) TS 895

Cashier 3 AP Beta 0.43 (0.084) EO -

Facility Costs

• Printing• Vaccine• Copiers • T-Shirts• Signs/Banners• Food• Advertising

• Tables/Chair Rental• Rent• Supplies• Medical Billing• Sharps Removal• Refrigerator Rental

Some Cost

• Salary– Nurses– Support Staff

• Usage – Every client usedVaccine, printing, copies, supplies, etc…

InputsSupplies Type Distribution Minimum ($) Maximum ($)

Influenza Vaccine a Uniform 9.71 19.94

Pneumococcal Vaccine a Uniform 14.24 24.19

Copying/Printing Cost b 0.10

General Supplies b Varies form $0.20 to $0.25 per station

Staff Type Distribution Salary (SD) - $/hr

Registered Nurses c Lognormal 23.1 (4.5)

Administrative Personnel c Lognormal 13.3 (3)

Clients Type Distribution Salary (SD) - $/hr

Special d Lognormal 21.5 (79.5)

Medicare d Lognormal 21.5 (79.5)

Cash c Lognormal 18.1 (15)

Why worry about Client salary?

Optimizing a Clinic

Maximize: # of people vaccinated

ST: Support staff <= 30

RNs <= 26

Staffing per station >= 1

Only RNs can vaccinate

Only support staff provide support function

Tool

• Arena 11.0 Discrete-event computer simulation

• OptQuest 11.0– Neural network– Scatter search (use infeasible values)– Tabu search (cannot use previous

values)

 Original Model

Optimized Model

Arrival Intensity (%) 80 40 80 140

Max Client Vaccinated 13,138 13,039 14,817 15,096

Special Flu Vaccination 2 1 1 1

Pnu Vaccination 4 2 2 2

Medicare Registration 18 15 15 15

Medicare/ Cash Flu Vaccination 20 23 23 23

Cashier 3 4 4 4

Others* 9

Total Staff 56 54 54 54* Special Flu Copy (1), Special Flu Registration (2), Pnu Registration(2), Medicare Copy (4)

Arrival intensity is the % increase in arrival rate above the original arrival rate.

0

3,000

6,000

9,000

12,000

15,000

0 20 40 60 80 100 120 140 160Intensity of Arrival Increase (%)

Clie

nts

All Clients

Special

Medicare

Cash

All Clients (Opt)

Special (Opt)

Medicare (Opt)

Cash (Opt)

Number Vaccinated

Time in Clinic

Medicare follows path similar to Special.

Throughput rateOriginal – 770 clients/hrOptimal – 880 clients/hr

0

50

100

150

200

250

300

350

0 20 40 60 80 100 120 140 160

Intensity of Arrival Increase (%)

Min

ute

s

Special

Cash

Special (Opt)

Cash (Opt)

Time in Clinic

Maximum ThroughputOriginal – 770 clients/hrOptimal – 880 clients/hr

0

50

100

150

200

250

300

350

0 20 40 60 80 100 120 140 160

Intensity of Arrival Increase (%)

Min

ute

s

SpecialMedicareCashSpecial (Opt)Medicare (Opt)Cash (Opt)

Cost

$

$100,000

$200,000

$300,000

$400,000

$500,000

$600,000

$700,000

$800,000

$900,000

0 20 40 60 80 100 120 140 160Intensity of Arrival Increase (%)

Material

Special

Medicare

Cash

Material (Opt)

Special (Opt)

Medicare (Opt)

Cash (Opt)

Other CostOther Cost Amount

Contract staff $2,882

Copiers 600

T-Shirts 1,043

Signs/Banners 350

Food 997

Ads 1,150

Tables/chairs 391

Rent 2,000

Medical billing 5,284

Sharp removal 375

Refrigerator Rental 80

Total $15,152

CE Ratio (Opt Model)

0102030405060708090

100

0 20 40 60 80 100 120 140 160

Intensity of Arrival Increase (%)

$/p

ers

on

Va

c

All CostNon Opp Cost

Agenda

• Measurements to evaluate model or actual clinic (including efficiency)

• Estimating the capacity of a mass vaccination clinic

• Issues with the optimized clinic

Issues with Optimization

• Targeted group with:– Little processing times– Few stations to visit– Larger numbers

• Alternative objective functions and constraints could have limited this disparity at the expense of efficiency

Issues with OptimizationObj function (instead of max # vacc)

– Max revenue – focus on one group of clients (who pays the most)

– Min cost – vaccinate no one– Max profit – we are the government– Max societal benefit minus cost –

programming dependent (societal perspective)

– Min time in clinic (client’s perspective)

Issues with Optimization

ConstraintsLimit the optimization to where no one spends more than a specific amount of time in the clinic; however, this also decreases efficiency if you want to maximize the # vacc

Issues with Optimization

Result (from the optimized model)– Elderly suffer: small number and slow

Still good to separate the elderly from others

– High resource utilization if want 15,000More staff are needed and capacity (safety) issue

– Planners did a good job in designing the clinic

Modeling Resources

• We must do this– Do we have enough– How much training is need– Where are they and how to allocate

them• Clinics and hospitals• Logistics, distribution, transportation• Appropriate supplies

• What is the right design or process

Thank You

Opportunities

• PE Fellowship

• EIS

• ORISE

• Other fellowship programs– Presidential Management Intern– Informatics– Many more

PE Fellowship

• Across the CDC, 2 Year program• Deadline Feb 1• Ph.D. (econ, decision sciences, HSR,

IE, OR, or related fields)• Accept non-US Citizens• Pay GS12 (2009), step 1 in ATL

$70,399 US ($83,714 after graduation)• Have a publication list