Conducting Meaningful Cost AnalysisMichael T. Braun, Ph.D. Satomi Wakita, Ph.D.

Children and Family Research Center, School of Social Work at the University of Illinois at Urbana-Champaign

2017 CQI Conference

Purpose

Ø reviewwhatstepsareneededtoperformacostanalysis,inparticularfocusingonthe“ingredients”ofdata.

Ø talkaboutthebarriersforcalculatingcostsandconductingtheanalysis.

1.CQIandcostanalysis2.Ingredientsofcost3.Ourproject4.Barriersforcalculatingcosts

5.Whatwelearned6.Questionsanddiscussion

Outline“Costanalysisisathoroughdescriptionofthetypeandamountofallresourcesusedtoproduceprogramservices.”

Source: Yates, B. T., Delany, P. J., & Dillard, D. L. (2001). Using cost " procedure " process "outcome analysis. In Bruce A. Thyer (Ed.), the Handbook of Social Work Research Methods(pp. 207-238). Thousand Oaks, CA: Sage.

1.Identifyand

understandtheproblem

2.Researchthesolution

3.Developthetheoryofchange

4.Adaptordevelopthe

solution

5.Implementthesolution

6.Monitorandassess

thesolution

Source: CQI Training Academy. (2014). CQI cycle of learning and improvement tip guide.

Figure 1. CQI Cycle of Learning and Improvement

Strategiesforcalculatingcosts

Personnel (salaries and benefits)

Materials and supplies

Depreciation costs of durable equipment

Facilities / office space

Travel Utilities

1. Itemize resources used to implement the program.

2. Collect data on resource costs.

Direct costq Employee salariesq Employee benefitsq Initial service costsq Costs associated with

servicesq Costs of out-of-home

placement

Direct cost(continued) q Time the worker spent

on the - Assessment- Interview- Communication

(e.g., email, phone call, etc.)

Indirect costq Administrativeq Management

§ Training§ Accounting§ HR

q Infrastructure costs§ IT

q Facilities

3. Collect data on resource allocation.

Category ActivityServicestoenrolledfamilies Parentgroupmeeting

• Proportion of salaries/fringe benefits for the caseworker

• Value of any volunteer labor• Educational materials• Client support materials• Cars• Computers/laptops• Office supplies• Office space

4. Estimate the value of direct and indirect costs.

5. Create an estimate of total costs.

6.Estimatecostsofindividualprogramcomponentsoractivities.

7.Estimatecostsperparticipant.

8.Conductanalysesandexaminecostvariation.

Source:Calculating the Costs of Child Welfare Services Workgroup. (2013). Cost analysis in program evaluation: A guide for child welfare researchers and service providers. Washington, DC: Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.

üFamiliarizeoneselfwiththeinterventionsthatwillbeevaluated:Whatarethegoals?

üDecidewhetheracostanalysisisworthdoing.üRecognizetheaudience.Whataretheirneeds? Whatprogramorservicewillbeanalyzed?

üWhatdaterangewillbecovered?üWhatcomparisonswillwemake?

Checklist for establishing a cost analysis

Ø Costevaluationwasoneoftheoutcomesintheevaluationproject.

Ø Totestthetheorythatanewprogramwouldproducehighercostsduringtheinitial assessmentandlowerthecostsduringthefollow-up period.

Cost analysis for our evaluation project

Initialcaseandfollow-upserviceperiodsforthecostanalysis

Initial Report Date Assessment Close Date

365 Days after Initial Report Date

Initial Case Period Follow-up Period

Analysisdesign(unit=family)Treatment Group

Comparison Group

Total cost

Total cost

Selectrandomly

ØFirst,weasked:Whatdowecareabout?ØWecareaboutpiecesthatchangewiththeNewProgram

ØWedon’tcareaboutanythingthatisthesamebetweenBusinessasUsualandtheNewProgram

ØForexample,nonewfacilitiesareneeded,nonewsupplies

Cost Types

ØSoaskyourself,WhatdoIcareabout?ØDoIcareaboutcoststostarttheprogramFROMSCRATCHsomewhereelse?

ØOrarethesecostsmoreaboutsmallchangestoBusinessasUsual?

ØForexample,let’ssaytheprogramisaboutmotivationalinterviewing

Cost Types

ØSecond,weasked:OfthethingsweDOcareabout,whatcanweaccess?ØStatelevelevaluationsmeanlotsofdifferentsystemsandlotsofdifferentgatekeepers

ØForexample,wecouldn’tgetinformationaboutexactemployeesalary

Cost Types

ØSoaskyourself,WhatcanIgetaccessto?ØIdeally… Butofcourse,wedon’tliveinanidealworld!

ØForexample,howmuchburdencanweputonworkers?

Cost Types

ØThird,weasked:Howaccuratecanourcostcategorymeasuresbe?ØOfthethingswedocareaboutandcangetaccessto,howaccurateistheinformationwehave?

ØForexample,workertimespentwithfamilies

Cost Types

ØSoaskyourself,HowaccuratecanIgetwithoutgoingcrazy?ØDoyouwanttotaketheAccountantApproach,theBackofEnvelopeApproach,orsomethinginbetween?

Cost Types

ØAccountforeverypossiblecostineverypossibleway!

ØYoumightneedthisapproach!

The Accountant Approach

ØQuickanddirtyØThismightbethebestyoucando

The Back of Envelope Approach

Your own back of envelope calculations are unlikely to involve geometry.

ØMostlikely,youwillchooseacombinationofthetwo

ØSomecategoriesallowexactcalculationsØOthersrequireroughapproximationsØThat’sokay,aslongasyounoteyourcertaintyaboutallcategories

Or somewhere in between

ØBacktoourexperience…ØWorkertimewithfamilieswasalistofworkercontactswithfamiliesØButwecouldnotknowA)howlongthecontacttook,B)ifthecontactwasfacetofaceorviasomeothermethod,andC)ifworkersreliablyusedthesamecontacttypeforsimilarcontacts

Cost Types

ØSowhattodo?ØFindthemostcommoncontacttypesØUsean“expertpanel”andaskthemfortheirbestestimates!

ØAveragetheirestimatesforanaveragetimeforeachcontacttype

Cost Types

ØThisresultedinacostanalysiscontainingthisdata:

1. Workercontactswithfamilies*averagetimepercontact*averageworkersalary=Costofworkertime(forinitialandfollow-upperiods)

2. Servicecostsperfamily(forinitialandfollow-upperiods)

The Result

ØAndwhatdidwefind?1. Workersdidspendmoretimewithfamiliesin

thenewprogram2. Thismeantthenewprogramcostmoreinthe

beginning3. Butalsonoevidencethatnewprogram

reducedcostsinthefollow-upperiod

The Result

ØThiswasthehardestquestiontoanswerØAfewthoughts…1. Peopleunderstandcostmoreintuitivelythan

othermeasures2. ButweKNOWthatthisisonlyarough

estimateoftruecost3. Sowehavetobecautiousinanyconclusions

weoffer

Is this useful at all?

What we learned

ØGarbageIn/GarbageOut(GIGO)ØWatchoutforwaysinwhichvarianceiscapped

ØAndwatchoutfortimesvarianceishuge!ØConsiderthereasonsyoumay(ormaynot)wanttoconvertallunitsintodollarsandcentsØOryoumightsay,MakeyourcostanalysisunitsDollarsand(Common)Sense

ThankYou!Whatquestionsdoyouhave?

Otherquestionstothinkabout/discuss:1. Whatistheroleofcostanalysisinyour

organization?2. Whataresomebarriersinconductingcost

analyses?3. Anythoughtsonmakingyourcostanalysis

moremeaningful?

Michael Braunmtb@illinois.edu

Satomi Wakitawakita@illinois.edu

ReferencesØ CalculatingtheCostsofChildWelfareServices

Workgroup.(2013).Costanalysisinprogramevaluation:Aguideforchildwelfareresearchersandserviceproviders.Washington,DC:Children’sBureau,AdministrationforChildrenandFamilies,U.S.DepartmentofHealthandHumanServices.

Ø Levin,H.M.&McEwanP.J.(2001).Cost-effectivenessanalysis:Methodsandapplications(2nd ed.).ThousandOaks,CA:Sage.