Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results
description
Transcript of Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results
![Page 1: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/1.jpg)
Washington State Low Income Weatherization Program Evaluation Calendar Year 2011DRAFT ResultsPrepared by:Rick Kunkle
July 2013
![Page 2: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/2.jpg)
Purpose of EvaluationIdentify and document Weatherization Program outcomes, benefits and costs to:• Assure prudent use of funds (accountability)• Improve the quality and effectiveness of program
services• Assess progress toward Weatherization Program
outcomes as measured by key performance measures
![Page 3: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/3.jpg)
Weatherization Program OverviewServes low income families by installing energy efficiency measures and making health and safety improvements and necessary repairs• Services provided in Washington since 1977• Managed by the Housing Improvement and
Preservation Unit in the Department of Commerce• Commerce contracts with 28 local agencies to
deliver services • Funded by a mix of state and local sources• The American Recovery and Reinvestment Act
(ARRA) significantly increased funding, primarily in 2010 and 2011
![Page 4: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/4.jpg)
Previous Evaluation Findings• Oak Ridge National Laboratory completed an impact
evaluation for Washington in 2001 showing energy savings compared favorably with other states
• WSU Energy Program completed Washington evaluations for 2006 and FY2010• Benefits exceeded costs• Production increased significantly in FY2010 from
ARRA funding• Recommendations for improving data collection,
estimates of benefits, and cost tracking and allocation
• Commerce has made investments and progress in addressing evaluation recommendations
![Page 5: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/5.jpg)
Evaluation ApproachThe evaluation covers calendar year 2011• Project Data: from the Weatherization information
Data System (WIDS) for projects with final inspections in 2011
• Program Data: from Commerce including expenditures, agency work plans, and historical data
![Page 6: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/6.jpg)
Bottom Line Results for 2011• Expenditures were $48 million. This is more than
twice 2005-2008 annual expenditures. Over half was from ARRA
• Production was 7,451 units. This is a little less than the peak in 2010, but still more than twice pre-ARRA production. Two-thirds of production was multi-family units
• Over 12,000 people lived in housing that was weatherized
• On average, 9 improvement measures were installed in each weatherized unit
• Estimated energy savings is $1.17 million/year. On average this is $157/unit. Other benefits are estimated to be $198/unit
• The Program benefit-cost ratio is estimated to be 1.12, meaning that benefits slightly exceed costs
![Page 7: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/7.jpg)
What is in this Presentation
• Program Delivery• Program Benefits• Program Costs• Program Cost-Effectiveness• Summary and Recommendations
![Page 8: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/8.jpg)
Key Findings – Program Delivery
![Page 9: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/9.jpg)
Historical Production Trends
![Page 10: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/10.jpg)
Monthly Production Trends
![Page 11: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/11.jpg)
Production by Agency in 2011
![Page 12: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/12.jpg)
Agency Service Delivery
Type of Agency
Delivery Method
# of agencies % of FY10 production
![Page 13: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/13.jpg)
Funding Sources
![Page 14: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/14.jpg)
Number of Funding Sources
![Page 15: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/15.jpg)
Project Delivery Time
![Page 16: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/16.jpg)
Average Days to Completion by Agency
![Page 17: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/17.jpg)
Heating Fuel
![Page 18: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/18.jpg)
Weatherization Measures by Unit
![Page 19: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/19.jpg)
Weatherization Measure Categories
![Page 20: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/20.jpg)
Top Weatherization Measures
![Page 21: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/21.jpg)
Top Weatherization Measures by Building Type
![Page 22: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/22.jpg)
Tier 2 Weatherization Measures
![Page 23: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/23.jpg)
Household Demographics
![Page 24: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/24.jpg)
Household Poverty Level
![Page 25: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/25.jpg)
Accomplishments, Challenges, Recommendations
Each year, local agencies produce a work plan that summarizes their processes, procedures, accomplishments, and challenges• Accomplishments:
• Energy savings, comfort, health and safety and housing repair benefits of their weatherization work
• Partnerships, cooperation and referrals• Challenges
• Decreasing funding along with increasing costs from wage and other requirements (reporting, monitoring, training, etc.)
• Recommendations• Clearer communication and more transparent decision-
making about funding availability and allocations• Streamlining program requirements
![Page 26: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/26.jpg)
Quality Assurance Inspections Specifications Correction Factor
![Page 27: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/27.jpg)
Key Findings – Program Benefits
![Page 28: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/28.jpg)
Energy Savings Analysis in WIDSDeemed savings estimates:• Developed savings coefficients by measure, building
type, fuel source, and heating zone• Used savings coefficients with measure data in WIDs in
savings calculations for each measure• Accounted for interactions between measures that save
heating energy• Energy savings are calculated for each weatherization
project with caps and checks• Energy cost savings based on average Washington
energy costs by fuel source• Energy benefits calculations use weighted average
measure life – 34 years
![Page 29: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/29.jpg)
Total Energy SavingsFuel Source Building Type
![Page 30: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/30.jpg)
Total Energy Savings by MeasureCategory Measure
![Page 31: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/31.jpg)
Energy Savings per Unit by Agency
![Page 32: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/32.jpg)
Energy Cost SavingsFuel Source Building Type
![Page 33: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/33.jpg)
Energy Cost Savings per Unit
![Page 34: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/34.jpg)
Other Benefits CalculationsThis is the same approach used for the FY2010 Evaluation• Utility benefits > estimated from best practices review
• Reduce delinquent bills and bad debt write-off• Service shut offs and reconnects
• Participant benefits > estimated from best practices review• Water/wastewater savings• Increased property value• Fewer moves• Health, safety and comfort benefits
• Societal benefits • Reduce greenhouse gas and pollutant emissions >
directly estimated
• Improvements to the economy > estimated from evaluation of Weatherization Program for Pacific Power
![Page 35: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/35.jpg)
Program Benefits
$/Household/Yr Mid Percent of Total
Utility 16 4%
Participant 103 29%
Societal - Economic 60 17%
Societal - Emissions 19 5%
Other Total 198 56%
Energy 157 44%
Total 355 100%
![Page 36: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/36.jpg)
Key Findings – Program Costs
![Page 37: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/37.jpg)
Program Expenditures by Fund Source
![Page 38: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/38.jpg)
Program Expenditures by Category
![Page 39: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/39.jpg)
Direct Project Cost by Fund Source
![Page 40: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/40.jpg)
Direct Project Cost by Measure Category
![Page 41: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/41.jpg)
Direct Average Unit Cost by Agency
![Page 42: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/42.jpg)
Distribution of Direct Unit Cost
![Page 43: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/43.jpg)
Cumulative Direct Unit Cost Curve
![Page 44: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/44.jpg)
High Cost Projects
![Page 45: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/45.jpg)
Average Direct Unit Cost
![Page 46: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/46.jpg)
Total Direct Cost Expenditures
![Page 47: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/47.jpg)
Total Unit Cost Estimate
![Page 48: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/48.jpg)
Why are per unit costs higher in 2011 compared to FY2010?
Many of the reasons could be associated with ARRA• More comprehensive weatherization• Wage requirements• Reporting requirements• Quality assurance requirements• Others?
![Page 49: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/49.jpg)
Key Findings – Program Cost Effectiveness
![Page 50: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/50.jpg)
Per Unit Cost Divided by Energy Savings
![Page 51: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/51.jpg)
Simple Energy Payback
![Page 52: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/52.jpg)
Cumulative Direct Unit Cost and Energy Savings
![Page 53: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/53.jpg)
Cost-Effectiveness Analysis Approach
• Cost-Effectiveness = total benefits per unit/total costs per unit
• Program benefits occur over time and are converted to a present value• 30 year average measure life• 2.3% discount factor (OMB)• 1% fuel escalation factor
• A high, mid and low scenario are considered to account for uncertainty in the benefits estimates
• Alternate approaches with different costs and benefits are also considered
![Page 54: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/54.jpg)
Cost-Effectiveness Analysis Results($/unit costs)
Present Value Mid Low High
Emissions Benefit $421 $369 -
Economic Benefit $1,313 $685 $1,967
Utility Benefit $338 $76 $674
Participant Benefit $2,233 $920 $4,588
Total Non-Energy $4,305 $2,050 $7,229
Energy Benefit $3,934 $2.992 $4,571
Total Benefit $8,239 $5,042 $11,800
Total Cost $7,362 $7,362 $7,362
Benefit-Cost Ratio 1.12 0.68 1.60
![Page 55: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/55.jpg)
Cost-Effectiveness Alternate Approaches
($/unit costs)Present Value Energy Only Plus
No Admin or T&TA $
Direct Cost Only
Plus No H&S and WRR
Emissions Benefit 0 0 0 0
Economic Benefit 0 0 0 0
Utility Benefit 0 0 0 0
Participant Benefit 0 0 0 0
Total Non-Energy 0 0 0 0
Energy Benefit $3,934 $3,934 $3,934 $3,934
Total Benefit $3,934 $3,934 $3,934 $3,934
Total Cost $7,362 $6,414 $4,780 $3,724
Benefit-Cost Ratio 0.53 0.61 0.82 1.06
![Page 56: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/56.jpg)
Summary and Recommendations
![Page 57: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/57.jpg)
Some Interesting Results for 2011• Higher production during ARRA is due to large multi-
family units• Top 9 producers account for 80% of production• City and county government agencies account for over
half the production, but less than half the agencies• Compared to FY2010 there is evidence of more
comprehensive weatherization• Ceiling, wall, and floor insulation and air sealing account
for 55% of the estimated energy savings• Large multi-family units account for 60% of the units,
but only a quarter of the energy savings and a quarter of total direct costs
• The 10% of highest cost units account for 32% of total direct costs and 27% of total energy savings
![Page 58: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/58.jpg)
Conclusions to Think About• The total program unit cost estimate
increased in 2011 compared to FY2010• Large multi-family units contribute to higher
production, more people served, and lower costs, but they also have fewer measures installed and produce less energy savings
• High cost units contribute significantly to total Program costs, but they also have more measures installed and greater energy benefits
• Energy cost benefits are less than other benefits
• While high production agencies had lower unit costs, they did not have lower unit costs/energy savings
![Page 59: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/59.jpg)
Recommendations• We need to develop better ways to track
Program expenditures for Commerce• The Program unit cost estimate includes an
estimate of local Program operations costs that did not go directly to unit weatherization. We need to check the accuracy of this estimate and identify what these cost are for and how to track them
• Identify ways to reduce the trend towards higher costs, which reduces Program cost-effectiveness if benefits do not increase
• How do we measure success? Identify key Program metrics and use WIDS data or other sources to document Program success
• Consider looking at the benefits from high cost units
![Page 60: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/60.jpg)
Next Steps
• Review and discuss the evaluation results• Make needed revisions and corrections to the
evaluation • Complete an executive summary for the
evaluation (text document)• Share the evaluation with the weatherization
network and stakeholders• Apply the experience from the 2011
evaluation to plan for and conduct the 2012 evaluation
![Page 61: Washington State Low Income Weatherization Program Evaluation Calendar Year 2011 DRAFT Results](https://reader031.fdocuments.in/reader031/viewer/2022011717/568166d2550346895ddae56e/html5/thumbnails/61.jpg)