Designing a gas program impact evaluation
-
Upload
zondits -
Category
Engineering
-
view
266 -
download
3
description
Transcript of Designing a gas program impact evaluation
Main Headquarters: 120 Water Street, Suite 350, North Andover, MA 01845 With offices in: NY, ME, TX, CA, OR
www.ers-inc.com
HOW TO DESIGN A GAS PROGRAM IMPACT EVALUATIONJonathan B. Maxwell, Energy & Resource Solutions (ERS)Kathryn Parlin, West Hill Energy & Computing
04/10/2023
Examine results for 13 gas evaluations Realization rates Variation of realization rates Net-to-gross Non-gas impact
Ramifications on evaluation designs
AGENDA
2
04/10/2023
Gas evaluation less mature than electric
Gas realization rates somewhat lower than electric
Evaluated project savings varies widely from reported—even in large custom programs
Account for interaction with other fuels
Account for non-energy benefits & costs
KEY POINTS
3
04/10/2023
GAS PROGRAMS EXAMINED
4
Program or Portfolio or Targeted Measure Type
No. Progra
msSample
Size
Residential single family new construction
1 25
Multifamily retrofit 1 6
C/I new construction & retrofit 2 48
C/I performance contracting 1 6
Commercial retrocommissioning 4 34
Industrial 1 29
Agriculture 1 30
Specialized (pipe insulation, bid program)
2 73
TOTAL 13 251
04/10/2023
Applicants typically estimated site-specific savings Expectation of good estimates
Evaluators estimated site-specific savings for all Majority “enhanced” level of
evaluation engineering rigor Many program types & administrators;
many evaluation engineering firms Actor bias unlikely
PROGRAM & EVALUATION TYPES
5
04/10/2023
0.68 median portfolio realization rate
Lower RR than for similar electric portfolios 5 of 13 portfolios had both electric & gas
RRs 0.70 median elec RR for the 5 0.53 median gas RR for the 5
SAVINGS REALIZATION RATES
6
.08 .08 .21 .33 .53 .64 .68
.72 .91 .92 .93 .98 1.07
04/10/2023
Measures variation of realization rates (Stratified ratio estimation) Lower is better Higher requires larger sample to get high
precision Unrelated to magnitude of realization rate
0.4 to 1.0 typical in EE evaluation 0.4 to 0.6 typical for electric EE
programs with site-specific analysis
ERROR RATIO - DEFINITION
7
04/10/2023
ERROR RATIO - ILLUSTRATION
8
04/10/2023
ERROR RATIO - RESULTS
9
Median 1.04 Shown with 6 outlier projects removed
04/10/2023
Why are error ratios so poor? Less mature programs Difficult for applicants (& evaluators) to
measure Baseline less clear (11% projects with 0
RR) Fuel switching Inherently difficult-to-predict measures
(RCx)
What to do? Intensively study gas measures Do not increase sample sizes at expense of
rigor per site
ERROR RATIO - OBSERVATIONS
10
04/10/2023
Similar methods used as with electric Mostly enhanced for these portfolios
0.85 median NTG factor 0.31 to 1.09 range for NTG factor More consistent than realization
rate
NET-TO-GROSS
11
04/10/2023
Results from 3 portfolio evaluations
Infrequent, but significant when it occurs
Customers add ~22% in utility bill savings
Should include in benefit-cost calculations
GAS MEASURE IMPACT ON ELECTRICITY, STEAM, OIL
12
Portfolio TypeElectric Impacts (kWh/ MMBtugas)
Other Energy Impacts (MMbtu/ MMBtugas)
Commercial New Construction 1.8 0.00
C/I Retrofit 15.1 0.02
Manufacturing--Ag-Food 18.5 na
04/10/2023
Results from 4 program evaluations
Infrequent but significant when it occurs Labor & water most common savings Customer save up to $0.75 for every $1 gas
saved Should include in benefit-cost calculations
IMPACT ON NON-ENERGY COSTS
13
Group Delivery ProgramNon-Energy Impact
(/ MMBtugas)
C/INew Construction $0.00
Existing Facilities $9.46
Loan Fund $1.22
1-4 Res. ENERGY STAR Homes $0.91
Multifamily Multifamily Building (Existing) $0.06
Summary - Evaluation Planning
Gas programs are less mature than electric
Expect large error ratios--evaluated savings varies widely from reported
Don’t sacrifice digging deeper for more sites Invest in enhanced M&V until programs mature
Allow time for vigorous feedback with program staff
Account for interaction with other fuels
Account for non-energy benefits & costs
aesp.org
04/10/2023
Gas programs are less mature than electric
Expect large error ratios--evaluated savings varies widely from reported
Don’t sacrifice digging deeper for more sites Invest in enhanced M&V until programs
mature
Allow time for vigorous feedback with program staff
Account for interaction with other fuels
Account for non-energy benefits & costs
SUMMARY - EVALUATION PLANNING
15