ERCOT Load Research Sampling Round 2 Comparison of Sample Means to Profile Models Presented to the...

ERCOT Load Research Sampling ERCOT Load Research Sampling Round 2Round 2

Comparison of Sample Means to Profile ModelsComparison of Sample Means to Profile Models

Presented to the PWG Meeting of May 26, 2010

Overview

• Background Information• Energy Comparison• Comparisons of Scaled LRS Means vs

backcasts – Interval data and daily average profiles

• Summary Statistics by Profile Type and Weather Zone

• Summary

2

Background InformationAnalysis period 07/01/2008 – 12/31/2009

◦ 18 month period to maximize interval data availability due to Hurricane Ike

◦ Ike exclusion: Sept 11, 2008 – Oct 31, 2008

◦ Analysis: one calendar month per run

Ratio estimation used in this analysis

Auxiliary variable is calendarized kWh for the month

3

Xxw

ywY

l

iii

l

iii

1

1

kWhIntervalforEstimateSampleY

kWhCalendarizedMeanPopulationX

stratalofoutWeightiStratumwi

iStratumforkWhMeanIntervalforEstimateSampleyi

iStratumforkWhCalendarizedMeanforEstimateSamplexi

Background Information

• An ESI ID is included in the analysis (both for the sample and population) for a month if:– Status is Active or De-energized at the time of the

study and Status is Active the entire study month– At least 90% of the interval data was available– Calendarized kWh available covering the whole month

• Treatment of Profile ID migrations– Eliminated from sample and population RES → BUS, NM BUS → RES, NM, BUSIDRRQ WZone or Voltage changes New ESI IDs since sample selection

4


• Sample and population stratification for analysis based on a combination of original stratum and current Profile ID

5

Current New

Prof ID WZone Strat Prof ID StratRESHIWR Coast 1 RESHIWR 1

RESHIWR Coast 2 RESHIWR 2

RESHIWR Coast 3 RESHIWR 3

RESLOWR Coast 1 RESHIWR 4



Original Stratum


• Migrations due to annual validation were modeled in the Round 2 analysis.

• Premises with advanced meters were removed from the population at the time of analysis (~400,000.)

6

Background Information• Two or more sample points are required

per stratum.◦ Required for sample precision calculation◦ Exception: 1 ESI ID in sample and population for the

stratum◦ Stratum eliminated if insufficient sample points to

perform the calculations◦ All other strata included in analysis.

• Mean Absolute Percent Error (MAPE) was calculated for each interval:MAPE = ABS(backcast-LRS_Mean)/LRS_Mean * 100

7

Background Information• Precision is a measure of the variability of the sample

estimate about the sample mean• Precision for each Profile Type and Weather Zone

combination was impacted by:– sample sizes based on estimated Error Ratios and varying levels of

accuracy depending upon the load in the cell

– the number of recorders actually installed

– sample points with a minimum of 90 percent data were used in the calculations

– sample point migration into and out of the strata

– advanced meters were removed from the population at the time of analysis.

8

Background Information• Profile Scaling

– The Round 2 estimates of the Mean were scaled so that their monthly energy equals the backcast energy.

– The scaled profiles provide a good indication of the allocation of energy across the day and month.

9

Results

• The following slides include: – Comparison of monthly energy (kWh) differences– Profile comparisons of scaled Round 2 Means vs backcast

profiles for selected weeks in the analysis period– Comparisons of scaled Round 2 Means vs backcast

profiles for daily average profiles across the analysis period

– Summaries of Mean Absolute Percent Error (MAPE) between the Round 2 Means and the backcasts for both scaled and un-scaled Round 2 Means.

– Summary of Precision by Profile Type and Weather Zone– Summary of Sample Sizes by Profile Type and Weather

Zone.

10

LRS Means vs Backcast Energy ComparisonLRS Means vs Backcast Energy Comparison

11

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Mon

thly

kW

h

2009

BUSHILF/NCENT

Sum_kWh_LRS

Sum_kWh_bkcst

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000


Mon

thly

kW

h

2009

BUSHILF/COAST

Sum_kWh_LRS

Sum_kWh_bkcst

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000


Mon

thly

kW

h

2009

BUSHILF/SCENT

Sum_kWh_LRS

Sum_kWh_bkcst

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000


Mon

thly

kW

h

2009

BUSHILF/SOUTH

Sum_kWh_LRS

Sum_kWh_bkcst


12

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000


Mon

thly

kW

h

2009

BUSLOLF/COAST

Sum_kWh_LRSSum_kWh_bkcst

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000


Mon

thly

kW

h

2009

BUSLOLF/NCENT


0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000


Mon

thly

kW

h

2009

BUSLOLF/SCENT


0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000


Mon

thly

kW

h

2009

BUSLOLF/SOUTHSum_kWh_LRSSum_kWh_bkcst


13

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000


Mon

thly

kW

h

2009

BUSMEDLF/COAST


0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000


Mon

thly

kW

h

2009

BUSMEDLF/NCENT


0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000


Mon

thly

kW

h

2009

BUSMEDLF/SCENT

Sum_kWh_LRS

Sum_kWh_bkcst

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000


Mon

thly

kW

h

2009

BUSMEDLF/SOUTH



14

0

100

200

300

400

500

600

700

800

900

1,000


Mon

thly

kW

h

2009

BUSNODEM/COAST


0

100

200

300

400

500

600

700

800

900

1,000


Mon

thly

kW

h

2009

BUSNODEM/NCENT


0

100

200

300

400

500

600

700

800

900

1,000


Mon

thly

kW

h

2009

BUSNODEM/SCENT


0

100

200

300

400

500

600

700

800

900

1,000


Mon

thly

kW

h

2009

BUSNODEM/SOUTH



15

0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESHIWR/COASTSum_kWh_LRSSum_kWh_bkcst

0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESHIWR/NCENT


0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESHIWR/SCENT


0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESHIWR/SOUTH



16

0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESLOWR/COASTSum_kWh_LRSSum_kWh_bkcst

0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESLOWR/NCENT


0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESLOWR/SCENT


0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

2,000

2,200

2,400


Mon

thly

kW

h

2009

RESLOWR/SOUTH


Profile Comparisons• A PowerPoint file has been prepared for each

profile type containing scaled LRS Means vs backcast comparisons for each profile type/weather zone combination

• These files are appendices to this presentation and are posted as key documents

• Two weeks containing peaks were selected: – July 13-19 2009 (2009 summer peak is on 7/13)– December 7-13 2009 (2009 winter peak is on 12/10)

• Three weather zones with the highest load were selected– Coast, North Central, South Central

17

PROFILE COMPARISONSWEEK OF JULY 13-19 2009

2009 Summer Peak occurred on July 132009 Summer Peak occurred on July 13

18

12

13

14

15

16

17

18

19

20

21

Mon 13 Tue 14 Wed 15 Thu 16 Fri 17 Sat 18 Sun 19 Mon 20

kWh

Time

Week of July 13-19 2009BUSHILF COASTBUSHILF COAST

BackcastScaled R2 Means

14

16

18

20

22

24

26

28


kWh

Time

Week of July 13-19 2009BUSHILF NCENTBUSHILF NCENT


12

13

14

15

16

17

18

19

20


kWh

Time

Week of July 13-19 2009BUSHILF SCENTBUSHILF SCENT


3

4

5

6

7

8

9

10

11

12


kWh

Time

Week of July 13-19 2009BUSMEDLF COASTBUSMEDLF COAST


4

5

6

7

8

9

10

11

12

13

14


kWh

Time

Week of July 13-19 2009BUSMEDLF NCENTBUSMEDLF NCENT


3

5

7

9

11

13


kWh

Time

Week of July 13-19 2009BUSMEDLF SCENTBUSMEDLF SCENT


0

1

2

3

4

5

6


kWh

Time

Week of July 13-19 2009BUSLOLF COASTBUSLOLF COAST


0

1

2

3

4

5

6


kWh

Time

Week of July 13-19 2009BUSLOLF NCENTBUSLOLF NCENT


0

1

2

3

4

5

6


kWh

Time

Week of July 13-19 2009BUSLOLF SCENTBUSLOLF SCENT


0.0

0.1

0.2

0.3

0.4

0.5

0.6


kWh

Time

Week of July 13-19 2009BUSNODEM COASTBUSNODEM COAST


0.0

0.1

0.2

0.3

0.4

0.5

0.6


kWh

Time

Week of July 13-19 2009BUSNODEM NCENTBUSNODEM NCENT


0.0

0.1

0.2

0.3

0.4

0.5

0.6


kWh

Time

Week of July 13-19 2009BUSNODEM SCENTBUSNODEM SCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2


kWh

Time

Week of July 13-19 2009RESHIWR COASTRESHIWR COAST


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4


kWh

Time

Week of July 13-19 2009RESHIWR NCENTRESHIWR NCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4


kWh

Time

Week of July 13-19 2009RESHIWR SCENTRESHIWR SCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6


kWh

Time

Week of July 13-19 2009RESLOWR COASTRESLOWR COAST


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6


kWh

Time

Week of July 13-19 2009RESLOWR NCENTRESLOWR NCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4


kWh

Time

Week of July 13-19 2009RESLOWR SCENTRESLOWR SCENT


PROFILE COMPARISONSWEEK OF DECEMBER 7-13 2009

2009 Winter Peak occurred on December 102009 Winter Peak occurred on December 10

37

8

9

10

11

12

13

14

15

16


kWh

Time

Week of December 7-13 2009BUSHILF COASTBUSHILF COAST


10

11

12

13

14

15

16

17

18

19

20


kWh

Time

Week of December 7-13 2009BUSHILF NCENTBUSHILF NCENT


8

9

10

11

12

13

14

15

16


kWh

Time

Week of December 7-13 2009BUSHILF SCENTBUSHILF SCENT


2

3

4

5

6

7

8

9

10


kWh

Time

Week of December 7-13 2009BUSMEDLF COASTBUSMEDLF COAST


2

3

4

5

6

7

8

9

10


kWh

Time

Week of December 7-13 2009BUSMEDLF NCENTBUSMEDLF NCENT


2

3

4

5

6

7

8

9

10


kWh

Time

Week of December 7-13 2009BUSMEDLF SCENTBUSMEDLF SCENT


0

1

2

3

4

5


kWh

Time

Week of December 7-13 2009BUSLOLF COASTBUSLOLF COAST


0

1

2

3

4

5


kWh

Time

Week of December 7-13 2009BUSLOLF NCENTBUSLOLF NCENT


0

1

2

3

4

5


kWh

Time

Week of December 7-13 2009BUSLOLF SCENTBUSLOLF SCENT


0.0

0.1

0.2

0.3

0.4

0.5


kWh

Time

Week of December 7-13 2009BUSNODEM COASTBUSNODEM COAST


0.0

0.1

0.2

0.3

0.4

0.5


kWh

Time

Week of December 7-13 2009BUSNODEM NCENTBUSNODEM NCENT


0.0

0.1

0.2

0.3

0.4

0.5


kWh

Time

Week of December 7-13 2009BUSNODEM SCENTBUSNODEM SCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2


kWh

Time

Week of December 7-13 2009RESHIWR COASTRESHIWR COAST


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8


kWh

Time

Week of December 7-13 2009RESHIWR NCENTRESHIWR NCENT


0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6


kWh

Time

Week of December 7-13 2009RESHIWR SCENTRESHIWR SCENT


0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


kWh

Time

Week of December 7-13 2009RESLOWR COASTRESLOWR COAST


0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


kWh

Time

Week of December 7-13 2009RESLOWR NCENTRESLOWR NCENT


0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


kWh

Time

Week of December 7-13 2009RESLOWR SCENTRESLOWR SCENT


0

5

10

15

20

25

30

35

40

12AM 03AM 06AM 09AM 12PM 03PM 06PM 09PM 12AM

kW

Interv al Ending

Average Day Across the Study PeriodBUSMEDLF COASTBUSMEDLF COAST


0

5

10

15

20


kW

Interv al Ending

Average Day Across the Study PeriodBUSMEDLF FWESTBUSMEDLF FWEST


0

2

4

6

8

10

12

14

16


kW

Interv al Ending

Average Day Across the Study Period

BUSLOLF COASTBUSLOLF COAST


0

2

4

6

8

10

12

14

16


kW

Interv al Ending

Average Day Across the Study Period

BUSLOLF NCENTBUSLOLF NCENT


0

1

2

3

4


kW

Interv al Ending

Average Day Across the Study PeriodRESHIWR COASTRESHIWR COAST


0

1

2

3

4


kW

Interv al Ending

Average Day Across the Study PeriodRESHIWR NCENTRESHIWR NCENT


0

1

2

3

4


kW

Interv al Ending

Average Day Across the Study PeriodRESLOWR COASTRESLOWR COAST


0

1

2

3

4


kW

Interv al Ending

Average Day Across the Study PeriodRESLOWR NCENTRESLOWR NCENT


Summary Summary

• The current models, based on Round 1 data in 2004-2006, are “best fit” regressions on the Round 1 estimated means.

• Round 2 is a “full sample” with sample sizes based on estimated Error Ratios and targeted ±15% accuracy at 90% confidence. (Round 1 was a “half sample”.)

• Ratio estimation analysis was used to calculate the Round 2 means.

• How well do the Round 1 models estimate the Round 2 means?

64

Summary by Profile Type and WZoneSummary by Profile Type and WZone

65

Unscaled Round 2 LRS Means

COAST EAST FWEST NCENT NORTH SCENT SOUTH WEST MEAN

BUSHILF 7.8 36.7 12.6 10.6 7.3 36.2 24.3 36.0 21.4

BUSLOLF 10.9 8.7 11.1 17.4 8.9 13.4 12.7 18.5 12.7

BUSMEDLF 6.9 5.4 23.1 6.3 23.3 10.2 6.8 35.4 14.7

BUSNODEM 17.7 9.7 10.5 6.2 12.5 9.0 17.5 18.6 12.7

RESHIWR 12.8 12.9 12.1 11.4 16.3 11.7 9.6 12.5 12.4

RESLOWR 9.4 13.1 11.2 7.9 9.3 9.4 14.7 10.7 10.7

MEAN 10.9 14.4 13.4 10.0 12.9 15.0 14.3 22.0 14.1

MEAN ABSOLUTE PERCENT ERROR (MAPE)

Scaled Round 2 LRS Means


BUSHILF 3.2 6.2 3.7 2.8 2.9 3.2 4.6 3.1 3.7

BUSLOLF 7.2 8.2 10.3 15.5 8.0 12.3 11.4 9.0 10.2

BUSMEDLF 5.9 4.9 10.1 4.2 6.9 5.0 5.6 4.4 5.9

BUSNODEM 27.9 9.0 8.2 5.3 12.3 8.6 16.5 13.7 12.7

RESHIWR 8.6 9.5 11.7 8.0 10.8 10.7 7.8 10.1 9.6

RESLOWR 6.9 11.4 8.7 6.1 8.5 8.6 9.5 9.9 8.7

MEAN 9.9 8.2 8.8 7.0 8.2 8.1 9.2 8.4 8.5

MEAN ABSOLUTE PERCENT ERROR (MAPE)

Summary by Profile Type and WZoneSummary by Profile Type and WZone

66

COAST EAST FWEST NCENT NORTH SCENT SOUTH WEST

BUSHILF 91 105 94 136 102 100 109 106

BUSLOLF 392 130 147 426 145 156 195 159

BUSMEDLF 290 128 114 343 122 113 128 110

BUSNODEM 157 141 134 417 120 106 88 104

RESHIWR 494 123 125 604 125 120 371 130

RESLOWR 633 116 121 601 118 124 133 115

MEAN SAMPLE SIZE


BUSHILF 4.4 8.8 4.3 2.8 3.7 2.7 5.0 6.6 4.8

BUSLOLF 7.8 12.5 13.2 6.9 13.1 12.3 11.6 11.2 11.1

BUSMEDLF 4.4 6.9 8.4 4.0 6.2 8.0 8.3 8.0 6.8

BUSNODEM 13.1 14.4 13.8 7.9 17.1 14.7 20.3 19.2 15.1

RESHIWR 7.2 14.3 13.3 6.7 13.8 14.1 7.5 12.6 11.2

RESLOWR 4.8 11.3 12.2 5.0 11.0 10.3 12.3 12.1 9.9

MEAN 6.9 11.4 10.9 5.5 10.8 10.4 10.8 11.6 9.8

MEDIAN PRECISION

Summary Summary • MAPE’s using scaled Round 2 Means by Profile Type

and Weather Zone range between 2.8 % (BUSHILF, NCENT) and 27.9% (BUSNODEM, COAST) with the mean for all profile types and weather zones = 8.5%.

• Mean MAPE’s using un-scaled Round 2 Means by Profile Type and Weather Zone range between 6.2 % (BUSNODEM, NCENT) and 36.7% (BUSHILF, EAST) with the mean for all profile types and weather zones = 14.1%.

• Mean Precisions by Profile Type and Weather Zone range between 2.7 % (BUSHILF, SCENT) and 20.3% (BUSNODEM, SOUTH) with the Mean Precision for all profile types and weather zones = 9.8%.

67

SummarySummary

• Reasons the current models do not do a better job of estimating the Round 2 means.• The population has changed in the 3 years since

the Round 1 models were developed.• Because Round 1 was a “half sample”, there were

cells in the Round 1 analysis that had the minimum number of sample points required to do the analysis thereby lowering the accuracy of the estimate.

• The minimum sample size in the Round 2 sample is 88 points in the BUSNODEM/SOUTH cell.

68

Questions for DiscussionQuestions for Discussion

69

ERCOT Load Research Sampling Round 2 Comparison of Sample Means to Profile Models Presented to the...

Documents

Transcript of ERCOT Load Research Sampling Round 2 Comparison of Sample Means to Profile Models Presented to the...