Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s...
Transcript of Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s...
![Page 1: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/1.jpg)
Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness
Team and the Division for Heart Disease and Stroke Prevention at the Centers for Disease
Control and Prevention. We are fortunate to have Alberta Mirambeau as today’s presenter.
Alberta is a health scientist on the Evaluation and Program Effectiveness Team. My name is
Martha Bose and I am today’s moderator. I am an Evaluation Fellow here on the same
team as Alberta. Today Alberta will be presenting on how to conduct a cost analysis.
*Note: Screen magnification settings may affect document appearance.
1
![Page 2: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/2.jpg)
2
![Page 3: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/3.jpg)
Just to give you a quick overview, today I’ll be talking about conducting a cost analysis and
sharing some key principles related to conducting a cost analysis. Then I’ll walk through an
example. I’ll close with a few remarks for your consideration and then share some tools
and resources that you can reference after this presentation.
3
![Page 4: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/4.jpg)
Before we actually talk about conducting a cost analysis I want to pause and talk about the
purpose of economic evaluation. Economic evaluation in public health programs focuses
on the resources used to implement a program or intervention and the health impacts of
that program.
Using the simplified logic model before you, economic evaluation focuses on the resources
and inputs and then factors changes in health outcomes so that decision makers are able to
use it as a tool to make choices and comparisons about whether the outcomes yielded
from a program are worth the resources used to produce those results. There are various
types of techniques when applying economic evaluation. There is a cost analysis, cost
benefit analysis, and cost effectiveness analysis among many others. Obviously today I’ll be
spending more time on conducting a cost analysis.
4
![Page 5: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/5.jpg)
Let’s start by defining a cost analysis. Borrowing from the earlier image, we see that the
cost analysis focuses primarily on the inputs and resources and it doesn’t yet factor the
outcomes of a program so inherently the emphasis of a cost analysis is on the costs.
Therefore, conducting a cost analysis is the first step to applying any economic evaluation
to your program. It’s also beneficial to conduct a cost analysis when truly answering the
question: how much does it cost to implement program or intervention X?
5
![Page 6: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/6.jpg)
Here are a few key concepts related to cost analyses. With any evaluation you want to start with a
purpose and a plan and this point is ideal to seek the consultation and services of a health economist.
While the expertise of an economist will be needed for a majority of your economic evaluation efforts
or the more advanced analyses it’s still important to be very involved as a program administrator, so
that you’re much more informed and prepared to undergo a cost analysis by understanding a few key
principles.
One of those principles has to do with which costs are counted or in other words from whose
perspective? Everyone has a cost to pay so there are a few perspectives to consider when determining
those costs. Think about the pubic health system perspective, which used when you are looking at how
much funding the public health system needs to just start up and operate a program. Another example
is the insurer perspective and in this case they are most interested in the cost incurred through claims
or reimbursements. Whereas the societal perspective looks at costs incurred by all parties and this is
the most common perspective.
So let’s say you’re responsible for implementing a YMCA lifestyle modification program. From the
societal perspective the health insurance, if a health insurance agency is also partially funding that
program have a cost. The YMCA has also has costs—maybe for the facilities or for employee time; Even
if the program is free, the participants may have a cost whether through transportation costs, time lost
from work or daycare expenses. The societal perspective accounts for all of these hidden costs incurred
by someone in society.
Another point I want to mention about conducting a cost analysis is the timeline in which you do it. If
you are collecting the data while the program is being implemented then the costs are being captured
prospectively. If the program period has already taken place and you’re looking at costs after the fact
then that’s a retrospective approach. Assessing costs prospectively is ideal because it allows for a more
systematic and accurate data collection process.
6
![Page 7: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/7.jpg)
On this slide are a few key terms and in essence when we’re talking about conducting a
cost analysis we are operationalizing resources, which translates into the workforce,
workspace, et cetera. These can then be itemized through your cost categories. It is
through the development of the cost categories you start to determine which costs you are
going to count in your cost analysis. Then you consider the costs or the value of those
resources.
We can then go a step further with costs by looking at them in two ways: financial or
economic. Financial costs refers to what commonly comes to mind when we think of costs,
such as the market value or the price tag of a resource. The financial costs of running a
program are more easily identified in a budget sheet. On the other hand, economic costs
are those costs that do exist but there may not be a direct exchange of funding so I refer to
these as your hidden costs. For example, just because the time of a volunteer is not
reflected in a budget sheet doesn’t mean that there isn't a monetary value for their time.
Although the financial costs are a convenient source for conducting a cost analysis they also
give an incomplete picture of the resources that it takes to implement a program. If there
is a resource that is needed to implement a program then that cost should also be
accounted for and when you account for both financial and economic costs, you are
strengthening the accuracy and credibility of your economic analysis.
7
![Page 8: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/8.jpg)
A cost analysis is the first step to applying economic evaluation to your program and it’s
important to understand that a sound economic evaluation stems from a sound cost
analysis. It is simply the first step.
8
![Page 9: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/9.jpg)
Now we can actually go into the example. Today I’ll be talking a bit about the steps that we
use for conducting a cost analysis of a community health worker program. This is just one
approach and one method to conducting a cost analysis and it is not to be implied as the
only way to do so.
9
![Page 10: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/10.jpg)
On this slide I lay out the major steps that we took in this process and bolded the steps that
I think are most important to remember when conducting your own cost analysis; the other
steps are ideal but not necessary. I will walk through each step in the next few slides.
10
![Page 11: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/11.jpg)
As I mentioned before, with program evaluation or economic evaluation you always want
to define the purpose and the scope. In this instance, we wanted to articulate the full range
of cost for this program and this helped us define what the project was and what it wasn’t
and right away we were able to see what our limitations would be.
We decided that the cost analysis would be based on just the community health worker
program component, which is part of a much larger program; and we were going to do it
retrospectively and look at a one year period. We also decided to use the public health
system perspective—meaning we weren't going to account for other costs such as those
incurred by the participants or insurance companies.
11
![Page 12: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/12.jpg)
The second step that we took was a literature scan. We conducted a literature scan and
assessment and in essence you can translate this into doing your homework. This helped us
to really understand what other programs have done in the literature and how they did it.
We determined which studies were relevant to our particular program and what ideas we
could borrow from cost analyses that were conducted previously.
12
![Page 13: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/13.jpg)
After that, we developed the cost categories. This picture shows the series of steps that we
took. We used information from the literature scan, looked at other cost analysis tools,
reviewed program documents, and dialogued with the program staff. With each step there
was a different version of the cost categories as we identified what data were and were not
available. This process was certainly an iterative process and should be reflective of your
program.
13
![Page 14: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/14.jpg)
On this slide are the items that we actually identified as our cost categories. I don’t expect
you to be able to read every detail on this chart; we’ll have these slides available on our
website for you to review at your own leisure. What I’d like to emphasize here is that the
cost categories that you eventually land on should be reflective of your program.
Let’s say, for instance, the lifestyle interventions of the WISEWOMAN program differs from
state to state; therefore, one would expect that while they are all working to accomplish
similar goals, their program and therefore their costs are different. So you will want to
ensure that your cost categories are reflective of your program’s particular setting and
activities.
14
![Page 15: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/15.jpg)
In our cost analysis we also conducted a time study and this helped us to be able to look at
how the community health workers were allocating their time to specific activities. Now, it
is not an essential step but it is ideal because this activity enabled us to have another layer
of information on how costs were being expended and used in terms of the community
health worker’s time.
15
![Page 16: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/16.jpg)
Once you have done all these different activities, then you want to actually collect the data.
When possible, we used original data source and used industry standards for the economic
costs. Industry standards were used for items such as volunteer time and workspace.
16
![Page 17: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/17.jpg)
Then, lastly, you calculate your costs. I have noted here that we conducted a sensitivity
analysis as part of our endeavor and this helped us to look at different scenarios from the
least expensive program scenario to the most expensive program setting for implementing
the program.
17
![Page 18: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/18.jpg)
In summary, here are a few things that I’d like to share with you that we learned from this
process of conducting a cost analysis:
• You definitely want to be systematic and thorough when identifying your cost categories
and ensuring that they reflect your particular program.
• Remember to consider variables that may not be reflected in your budget, such as in-
kind support, volunteer time
• And when possible use original or actual data sources.
While most decision makers are most interested in the benefit or results of program costs,
which is usually provided through cost effectiveness or cost benefit analysis, it remains
essential that cost analyses are conducted with accuracy and a degree of rigor. And most
importantly, that they are considered as the first step to any type of economic evaluation.
18
![Page 19: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/19.jpg)
Here are a few tools and resources for your review and for referencing information related
to economic evaluation. The first items listed served as the primary resource from which I
referenced a majority of the information shared in the earlier parts of this presentation.
The 5-part podcast can be found on our Division web-site and provides a module
specifically on conducting a cost analysis. The other resources should also serve you well
for learning more about economic evaluation.
19
![Page 20: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/20.jpg)
Question 1: You mentioned that you recognize the limitations of the project. Could you elaborate on
what those limitations were?
Response: When we started to determine the scope of the project and what was feasible to do we
realized that by doing it retrospectively rather than prospectively we were going to be limited in the
data that were going to be available to us; right away, that’s a limitation. We also limited the project
scope to just one year. The program has been in existence for over three years and an ideal situation
would be to look at those entire three-year periods and to come up with an average of the costs—that
way we have a more realistic understanding of the costs rather than just relying on one year. As we all
know with the market and how costs fluctuate, an analysis of all three years would have provided a
much more accurate and comprehensive understanding of the costs. And obviously the cost analysis
was just for this one setting so we don’t have any group to compare or another program that is similar in
scope and context to compare the costs to; that also adds another limitation to the project.
Question 2: What were the benefits of adding a time study to the cost analysis?
Response: It allowed us to look at the distribution of time that the community health workers were
devoting to certain activities. Therefore, not only were we able to know what proportion of the costs are
committed to documenting activities or what proportion of the costs are committed to direct patient
support; we were also able to see the reach. It was another way of looking at how often and to what
extent the community health workers were coming in contact with individuals. It also allowed us to look
at the time and scope of activities of the volunteers and not just the community health workers. This
way we had an idea of not only of what the volunteers do but how they were contributing to the
community health workers’ activity. Thus, there were a few benefits for adding that component.
20
![Page 21: Welcome to today’s coffee break presented by the Evaluation and …€¦ · Welcome to today’s coffee break presented by the Evaluation and Program Effectiveness Team and the](https://reader034.fdocuments.in/reader034/viewer/2022050123/5f530a69f1126027e34ad668/html5/thumbnails/21.jpg)
If anyone thinks of additional questions or topics related to economic evaluation or ways
you would like to receive more TA about economic evaluation please feel free to email us at
the address listed on the slide.
21