Post on 04-Dec-2014
description
Slide 1
Making Sense of the Bottom Line: A Guide to Reading Cost-Benefit Reports
April 24, 2012
Gary VanLandingham, Director, Results First, Pew Center on the States Joshua Rinaldi, Policy Analyst, Cost-Benefit Analysis Unit, Vera Institute of Justice
Slide 2 • April 25, 2012
This webinar will teach you to:
• Use a structured framework to read cost-benefit reports.
• Quickly identify the key components of a cost-benefit analysis that affect the bottom line.
• Understand how these details and the CBA are relevant to policy discussions and decisions.
Slide 3 • April 25, 2012
Why is cost-benefit analysis important?
• Resources are scarce and getting scarcer.
• People want to know how to use resources more efficiently to get the most “bang for the buck.”
• Cost-benefit analysis (CBA) provides information to help weigh the pros and cons of investments.
Slide 4 • April 25, 2012
Making Sense of the Bottom Line
Gary VanLandingham Director
Results First
Joshua Rinaldi Policy Analyst
Vera Institute of Justice
Slide 5 • April 25, 2012
Results First
Results First is partnering with states to assess and advance policy options that benefit residents and improve states’ fiscal health. Results First is an initiative of the Pew Center on the States and the John D. and Catherine T. MacArthur Foundation, with additional support from the Annie E. Casey Foundation.
Slide 6
The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB) is a project of the Vera Institute of Justice funded by the U.S. Department of Justice’s Bureau of Justice Assistance.
• Website (cbkb.org) • Cost-benefit analysis toolkit • Snapshots of CBA literature • Podcasts, videocasts, and webinars • Roundtable discussions • Community of practice
Slide 7 • April 25, 2012
Agenda
• CBA basics 5 minutes
• A guide to reading CBA reports 30 minutes
• Recap 5 minutes
• Questions and answers 15 minutes
Slide 8 • April 25, 2012
Housekeeping items
• If you need Webinar support or for troubleshooting: Type questions into the chat box Call 800-843-9166 Send e-mail to help@readytalk.com
• This webinar is being recorded.
• The recording and PowerPoint slides will be posted
on cbkb.org.
Slide 9 • April 25, 2012
Housekeeping items
Questions
Use the chat feature to send us your questions at any time during the webinar.
We will address your questions midway through the presentation and again at the end.
Slide 10 • April 25, 2012
CBA Basics
Slide 11 • April 25, 2012
What is cost-benefit analysis?
A type of economic analysis that compares the costs and benefits of policies and programs, and
• Identifies efficient uses of resources
• Uses dollars as a common measurement
• Presents a long-term picture
Slide 12 • April 25, 2012
The CBA process
1. Determine the impact of the initiative.
2. Determine which perspectives to include.
3. Measure costs and benefits (in dollars).
4. Compare costs and benefits.
5. Assess the reliability of the results.
For a glossary of CBA terms: cbkb.org/basics/glossary
Slide 13 • April 25, 2012
A Guide to Reading Cost-Benefit Reports
Slide 14 • April 25, 2012
A guide to reading cost-benefit reports
1. Getting your bearings 2. Surveying the landscape 3. Looking at the details 4. Making sense of your surroundings
Slide 15 • April 25, 2012
Cost-benefit analysis of CEO
• The Center for Employment Opportunities (CEO) is a New York City–based employment program for formerly incarcerated people.
• We will refer to examples from a cost-benefit analysis of CEO to illustrate how this guide takes you through a CBA report.
Slide 16 • April 25, 2012
Step 1 Getting Your Bearings
Slide 17 • April 25, 2012
Getting your bearings
• What’s the bottom line? • What does the report cover?
Slide 18 • April 25, 2012
What’s the bottom line?
• Look at the executive summary for the “bottom line” comparison of costs and benefits, expressed as:
• Net Present Value (NPV) or Net Benefits total benefits – total costs
• Benefit-Cost Ratio (BCR) total benefits/total costs
• Return on Investment (ROI) total benefits/(total benefits – total costs)
• For more information: cbkb.org/toolkit/reporting-cost-benefit-results
Slide 19 • April 25, 2012
What’s the bottom line?
• Look at the executive summary for the NPV, net benefits, BCR, or ROI.
ES, page 18
ES, page 19
Slide 20 • April 25, 2012
What does the report cover?
The executive summary may discuss:
• Measurements and components of the costs and benefits
• Assumptions on which costs and benefits are based
• Omissions from the study
Slide 21 • April 25, 2012
What does the report cover?
• Table of contents
• Section headings
• Tables
• Appendices
Page vii
Slide 22 • April 25, 2012
Step 2 Surveying the Landscape
Slide 23 • April 25, 2012
Surveying the landscape
• What data did the CBA use?
• Which perspectives were included?
• What is the breakdown of costs and benefits?
Slide 24 • April 25, 2012
What data did the CBA use?
Identify:
• How recent the data is
• How specific the data is to the jurisdiction
• How costs and benefits were measured
Slide 25 • April 25, 2012
Data for the CEO cost-benefit analysis
Evaluation • Three-year random-assignment study
Budgets • CEO financial statements • New York City budget documents • New York State budget documents
Interviews • CEO staff
Literature • Research literature on victimization costs
The data section is on page 56 of the CEO report.
Slide 26 • April 25, 2012
Which perspectives were included?
• CBAs examine several perspectives to present a comprehensive picture of a program’s net impact to society.
• In criminal justice, CBAs often consider the perspectives of: Taxpayers Victims Program participants
Slide 27 • April 25, 2012
Perspectives in the CEO analysis
Page 67
Slide 28 • April 25, 2012
What is the breakdown of costs and benefits?
• Understanding which perspectives are included and what data was used will help you identify the benefits included in the analysis.
Page 67
Slide 29 • April 25, 2012
Questions
Use the chat feature to send us
your questions.
Slide 30 • April 25, 2012
Step 3 Looking at the Details
Slide 31 • April 25, 2012
Looking at the details
• Does the report use marginal costs?
• What time period was analyzed?
• What discount rate was used?
• Which impacts were not monetized?
Slide 32 • April 25, 2012
Does the report use marginal costs?
• Marginal costs (and benefits) are incurred because of changes in units of activity from an existing level of operations.
• For more information: cbkb.org/toolkit/marginal-costs
Cost per day of keeping one inmate in prison in New York in 2011
Average cost Marginal cost
$164.59* $27.53**
* Source: Vera’s “The Price of Prisons” report (2012) ** Source: CEO report (2012)
Slide 33 • April 25, 2012
What time period was analyzed?
• Identify the period used in the report to collect or estimate costs and benefits.
• Costs and benefits should be measured for as long as they last.
Page 6
Slide 34 • April 25, 2012
What discount rate was used?
• Discounting is a technique that translates future costs and benefits into present-day values.
• Criminal justice CBAs usually use rates of 3% or 5%.
• The CEO report used a 3% discount rate.
Slide 35 • April 25, 2012
Why is discounting important?
• A higher discount rate (relative to a lower discount rate) will more greatly reduce future benefits or costs in present-value terms.
• The discount rate selected is of particular importance in CBAs with long time frames where a large share of the costs and benefits accrue in future years.
• For more information: cbkb.org/toolkit/discounting
Slide 36 • April 25, 2012
Which impacts were not monetized?
• If a CBA does not monetize certain impacts, it may discuss them qualitatively.
Page 57
Slide 37 • April 25, 2012
Step 4 Making Sense of Your Surroundings
Slide 38 • April 25, 2012
Making sense of your surroundings
• How does the study address uncertainty? • What are the study’s limitations? • How does this all add up?
Slide 39 • April 25, 2012
How does the study address uncertainty?
• CBAs rely on estimates and assumptions.
• Uncertainty in estimates or assumptions means uncertainty in the results.
Slide 40 • April 25, 2012
How does the study address uncertainty?
• Sensitivity analysis provides information about the degree to which results will change if the CBA’s underlying assumptions are modified.
• Sensitivity analysis can be used to present a range of results.
Slide 41 • April 25, 2012
Uncertainty in the CEO analysis
• In a sensitivity analysis, the CEO results were tested by replacing the benefits calculated with the “worst-case” values.
• The CBA was performed again using these worst-case values to determine whether it still resulted in a positive bottom line.
Slide 42 • April 25, 2012
Uncertainty in the CEO analysis
Page 122
Slide 43 • April 25, 2012
What are the study’s limitations?
The report should describe any limitations of the analysis.
Page 57
Page 56
Slide 44 • April 25, 2012
How does all this add up?
• Limitations are a part of any analysis and do not invalidate a CBA. But it is important that the limitations are clear.
• Uncertainty is a part of any analysis. Still, it is important that the range of possible cost-benefit results is clear, so that policymakers can make well-informed decisions.
• Benefits are not necessarily budget savings.
Slide 45 • April 25, 2012
Recap
Slide 46 • April 25, 2012
Putting it all together
• Don’t panic. There is a logical approach to reading cost-benefit reports.
• Use this guide to determine whether the CBA you’re reading is based on sound data and assumptions.
• Use the guide to ask questions to better understand the CBA results.
Slide 47 • April 25, 2012
A guide to reading cost-benefit reports
1. Getting your bearings 2. Surveying the landscape 3. Looking at the details 4. Making sense of your surroundings
Slide 48 • April 25, 2012
1. Getting your bearings
• What’s the bottom line? • What does the report cover?
Slide 49 • April 25, 2012
2. Surveying the landscape
• What data did the CBA use?
• Which perspectives were included?
• What is the breakdown of costs and benefits?
Slide 50 • April 25, 2012
3. Looking at the details
• Does the report use marginal costs?
• What time period was analyzed?
• What discount rate was used?
• Which impacts were not monetized?
Slide 51 • April 25, 2012
4. Making sense of your surroundings
• How does the study address uncertainty? • What are the study’s limitations? • How does this all add up?
Slide 52 • April 25, 2012
Questions
Use the chat feature to send us your questions.
Slide 53 • April 25, 2012
Contact information
Gary VanLandingham
gvanlandingham@pewtrusts.org
Joshua Rinaldi jrinaldi@vera.org
212-376-5201
Slide 54 • April 25, 2012
Sources
MDRC. (2012). More Than a Job: Final Results from the Evaluation of the Center for Employment Opportunities (CEO) Transitional Jobs Program
Vera Institute of Justice. (2012). The Price of Prisons: What Incarceration Costs Taxpayers
Slide 55
This project is supported by Grant No. 2009-MU-BX K029 awarded by the Bureau of Justice Assistance. The Bureau of Justice Assistance is a component of the Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office of Sex Offender Sentencing, Monitoring, Apprehending, Registering, and Tracking. Points of view or opinions in this document are those of the author and do not represent the official position or policies of the United States Department of Justice.
Slide 56 • April 25, 2012
Thank you!