Computing Electricity Consumption Profilesfrom Household Smart Meter Data

Omid ArdakanianUniversity of Waterloo

Waterloo, Ontario, Canadaoardakan@uwaterloo.ca

Negar Koochakzadeh⇤

OracleVancouver, BC, Canada

negar@koochakzadeh.net

Rayman Preet SinghUniversity of Waterloo

Waterloo, Ontario, Canadarmmathar@uwaterloo.ca

Lukasz GolabUniversity of Waterloo

Waterloo, Ontario, Canadalgolab@uwaterloo.ca

S. KeshavUniversity of Waterloo

Waterloo, Ontario, Canadakeshav@uwaterloo.ca

ABSTRACTIn this paper, we investigate a critical problem in smartmeter data mining: computing electricity consumption pro-files. We present a simple, interpretable and practical pro-filing framework for residential consumers, which accountsfor variations in electricity consumption at di↵erent timesof day and at di↵erent external temperatures. Our approachis to isolate the e↵ect of external temperature on electricityconsumption and apply a time-series autoregressive model tothe remaining signal. The proposed profiles may be used formaking personalized energy-saving recommendations, de-tecting outliers, and generating very large realistic data setsfor testing the scalability of smart meter data managementsystems. Using predictive power as a metric for the accu-racy of consumption profiles, we show, using a real dataset of 1000 homes, that our approach results in improvedroot-mean-squared prediction error compared to existing ap-proaches.

1. INTRODUCTIONSmart electricity meters are rapidly replacing conventional

meters in many parts of the world. Smart metering systemso↵er many operational advantages for energy utilities andpolicy makers, including

• enabling automated collection of fine-grained (typ-ically half-hourly or hourly) consumption readings,thereby eliminating the need for utilities to send outestimated bills or to dispatch personnel to customerpremises and manually read the meters,

• enabling dynamic pricing schemes that depend on thetime-of-day in order to reduce demand for electricity

⇤Work done while the author was a Postdoctoral Fellow atthe University of Waterloo.

(c) 2014, Copyright is with the authors. Published in the Workshop Proceed-ings of the EDBT/ICDT 2014 Joint Conference (March 28, 2014, Athens,Greece) on CEUR-WS.org (ISSN 1613-0073). Distribution of this paper ispermitted under the terms of the Creative Commons license CC-by-nc-nd4.0.

during peak times.

However, exploiting smart metering systems to their fullestalso requires mining the vast amounts of collected consump-tion data to obtain insights into grid operations and con-sumer behaviour [2, 4, 7, 12,16,21].

In this paper, we address the problem of computing elec-tricity consumption profiles from smart meter data, with afocus on residential customers. The residential sector con-tributes a significant fraction to the total electricity demand(30 percent in Canada [5]) and greenhouse gas emissions(see, e.g., [19,24] for United States statistics). Furthermore,in many regions, residential consumers are significant con-tributors to peak demand; e.g., in Ontario, Canada, resi-dential air conditioning load is a major contributor to peakdemand, which occurs in the afternoon of hot summer week-days [23].

We argue that consumption profile generation is a fun-damental smart meter data mining operation that electric-ity providers, resellers and consultants can perform, with atleast the following applications:

• Conducting “virtual energy audits” and making per-sonalized recommendations for saving electricity basedon the trends identified in the profiles.

• Clustering households based on the features capturedby the profiles. This may be used to understand di↵er-ent classes of consumers and to design targeted energyconservation and peak reduction programs for di↵erentclasses.

• Generating real-time alerts if new consumption read-ings do not match the expected consumption predictedby the profiles. A related application is to identify con-sumers with “suspicious” load profiles that do not fitin any cluster, which could indicate electricity theft,malfunctioning meters or the presence of specializedequipment such as electric vehicle chargers.

• Generating realistic synthetic data based on the avail-able real data. This may be used as input to grid simu-lation, transformer sizing, forecasting and pricing mod-els, or to create very large realistic data sets for testingthe scalability of smart meter data management sys-tems.

140

1.1 Challenges and ContributionsIn order to be useful for the above applications, we argue

that consumption profiles must satisfy the following criteria.

• First, they must be accurate, i.e., able to describe andpredict consumption with reasonable accuracy.

• Second, they must be easily interpretable; a complexmachine learning model may be accurate, but if it isnot interpretable, then actionable energy-saving rec-ommendations cannot be easily inferred from it.

• Third, they must be practical, and therefore theyshould only require data that are easily available toutilities, such as hourly smart meter readings andweather. While household characteristics (e.g., homesize and age, number of appliances, number of occu-pants, etc.) and consumer demographics could be use-ful, this information is typically not available to utili-ties due to privacy regulations and cannot be easily ob-tained without intrusive measurements and question-ing.

The challenge in computing accurate and interpretableprofiles from smart meter data is that residential electricityconsumption depends on many factors, including the timeof day, weather and the occupants’ daily routines. Broadlyspeaking, prior work can be divided into two approaches.One is to compute various aggregate statistics from histori-cal consumption data that account for typical daily activity;examples include the average, maximum, minimum and vari-ance of hourly or daily consumption, ratios of night-to-dayor morning-to-evening consumption, or identifying the hourof day when peak consumption usually occurs. The otherapproach has been to correlate consumption with externaltemperature, e.g., using piecewise linear regression, and usethe correlation coe�cients as representatives for the coolingand heating e�ciency of homes.

In this paper, we propose a simple and practical tech-nique that combines the best features of existing methods,and accounts for both temperature and activity in an ac-curate and interpretable fashion. The idea is to remove thee↵ects of external temperature and outliers from the rawconsumption data1, and compute typical hourly consump-tion values from the remaining signal using a time seriesauto-correlation model. This gives us typical consumptionlevels of a given home at di↵erent times of the day, indepen-dent of temperature and robust to “noise” (e.g., time periodswhen the home was empty or unusually busy).

Specifically, we make the following contributions in thispaper:

• We propose a simple and interpretable technique forcomputing electricity consumption profiles from house-hold smart meter data, which may be used for per-sonalized recommendations, forecasting, classification,and as input to simulation models.

• Using predictive power as a metric for accuracy andhence the representativeness of consumption profiles,

1Here, by outliers we mean consumption readings that aremuch lower or higher than the average consumption for thegiven home, and thus do not correspond to the typical levelof activity in this home.

we compare the proposed method to several existingapproaches. Using a real data set, consisting of a yearof hourly smart meter readings from 1000 homes insouthern Ontario, Canada, we show that our approachoutperforms existing approaches in terms of the root-mean-squared prediction error.

1.2 RoadmapThe remainder of this paper is structured as follows. Sec-

tion 2 gives the intuition and an overview of our solution;Section 3 presents the details of our consumption profile al-gorithm; Section 4 describes our experimental results; Sec-tion 5 discusses related work; and Section 6 concludes the pa-per and discusses open problems in smart meter data man-agement.

2. INTUITION AND SOLUTIONOVERVIEW

In this section, we give an overview of our solution andwe explain the intuition behind it. The input to our prob-lem consists of two time series: 1) periodic (e.g., hourly)timestamped electricity consumption readings from a givenhome for some period of time (e.g, 6 months or a year), and2) a corresponding time series with external temperaturemeasurements, with the same granularity and for the sameperiod of time, e.g., from a nearby weather station.

A very simple consumption profile could consist of 24numbers: the average consumption for each hour of the day,aggregated over some or all of the input data. A simple ex-tension is to compute two such profiles: one for weekdaysand one for weekends and holidays. (We could go furtherand compute separate profiles for every day of the week,but, to keep the model simple and easily interpretable, wewill only consider weekday-weekend splits in this paper.)

Hourly averages may reveal some high-level details aboutthe consumption habits of a household, but we can do better.Observe that in climates with summer air conditioning usageand/or winter electric heating, a large part of the electric-ity consumption is temperature-sensitive; e.g., in the UnitedStates, roughly 40 percent of a home’s energy consumptionservers heating and cooling needs [24]. For example, Fig-ure 1 plots the hourly consumption and external temper-ature (measured in degrees Celcius) for a sample home insouthern Ontario, Canada, between April 2011 and Octo-ber 2012 (we omit further details about the data source topreserve privacy). Observe that the peak summer consump-tion of this home is roughly 1.5 kilowatt-hours (kWh) higherthan the peak winter consumption, which is likely due to airconditioning usage. If we could quantify the consumption oftemperature-sensitive loads and remove it from the originalconsumption time series, the remaining consumption wouldgive us a better idea of the occupants’ routines and activi-ties, and thus a more accurate profile.

The problem is that the relationship between consumptionand external temperature is not exact, making it di�cultto estimate the consumption of temperature-sensitive ap-pliances from whole-house smart meter data. Figure 2 plotsthe noon-time energy consumption of the same sample homeas a function of temperature; i.e., each point represents thenoon-time consumption of this home on some day betweenApril 2011 and October 2012 as well as the temperature atthat time. On some days, the noon-time consumption is rel-

141

Figure 1: Hourly consumption of a sample home (blue curvewith Y-axis on the left) and the temperature (green curvewith Y-axis on the right), measured between April 2011 andOctober 2012.

atively high when the temperature is moderate or relativelylow, and vice-versa. Some of these “outliers”may correspondto days when the home was empty or unusually busy. If wecould remove these outliers, we could obtain a more accurateestimate of the temperature-sensitive load, and also a moreaccurate estimate of the remaining (routine and activity)load, which can give a more accurate and robust profile.

Our proposed solution, illustrated in Figure 3, implementsthe above observations. Given the smart meter and temper-ature time series, we will compute 48 numbers: the typi-cal daily consumption values for each hour of the day onweekdays and weekends, after accounting for temperature-dependent load and outliers. The details of our solution arepresented in the next section.

3. COMPUTING CONSUMPTION PRO-FILES

We now describe the proposed solution, beginning with anoverview of the time series model that we employ (Section3.1), followed by a discussion of how outliers and temper-ature e↵ects are taken into account (Section 3.2) and howmodel parameters are chosen (Section 3.3). We then showhow to extract consumption profiles from our time seriesmodel (Section 3.4) and we discuss several applications ofthe proposed profiles (Section 3.5).

3.1 The PARX ModelThe main idea behind our solution is to apply a time series

autoregression model, specifically Periodic Auto Regressionwith eXogenous variables (PARX) [17]. Table 1 lists the sym-bols used in the remainder of the paper; in our case, we have24 “seasons”, each corresponding to a particular hour of theday, as we will be building separate consumption models foreach hour.

In general, a PARX model of order p represents a timeseries in terms of 1) its recent history (the most recent pdata points), 2) exogenous variables, and 3) a white noise

-10 0 10 20 300

1

2

3

4

5

Temperature (C)

Ene

rgy

cons

umpt

ion

at n

oon

Figure 2: Hourly consumption measured at noon in a sam-ple home versus the external temperature. The dashedline represents best linear fit for temperatures higher than20�Celsius.

component. In our case, this can be written as

Yt =pX

i=1

�isYt�i +nX

j=1

jsXjt + Cs + ✏t, t 2 s (1)

where Yt is the electricity consumption at a particular hourat time t, n is the number of exogenous variables (i.e., theXj ’s), ✏t is the value of the white noise component2 at timet, Cs is an intercept term, and s is the “season” index. Themodel parameters Cs,�is, js, and �2

s depend on the season.Intuitively, Equation (1) states the following. Pick a “sea-

son”, i.e., some hour of the day, say, noon. The electricityconsumption at noon is a linear function of the consumptionat noon on the previous p days3, and of the n exogenous vari-ables (such as temperature), plus a constant intercept termand an error term. Since each hour of the day is a separateseason with its own model, the values of the coe�cients �i

and j may be di↵erent for di↵erent hours. That is, thismethod is flexible enough to capture the possibility that atsome hours of the day (e.g,. night-time), temperature hasa stronger relationship with consumption, whereas at otherhours of the day (e.g., dinner-time), the load is more a↵ectedby the occupants’ activities.

3.2 Exogenous VariablesAs mentioned in Section 2, we want to compute the typi-

cal hourly consumption of a household, after accounting fortemperature and “outliers” corresponding to periods of verylow or very high consumption. This is exactly the purposeof exogenous variables. The e↵ects of other unknown fac-tors on the overall consumption (i.e., other appliances, dailyroutines and patterns) will be captured in the resulting con-sumption profile via the auto-regressive part of the model.

2We assume that the white noise process is a sequence of in-dependent and identically distributed random variables withzero mean and finite variance �2

s .3More precisely, it is a function of the previous p week-days for the weekday profile and the previous p week-ends/holidays for the weekend/holiday profile.

142

Hourly'consump-on'

Hourly'temperature'

Remove'outliers'and'temperature5dependent'consump-on'component'

Average'consump-on'for'each'hour'of'day'(weekdays)'

Average'consump-on'for'each'hour'of'day'(weekends)'

Figure 3: Overview of the proposed consumption profile approach.

Yt

The original time series of house-hold electricity consumption attime t

Y ⇤

t

The time series at time t obtainedafter removing the e↵ects of exoge-nous variables, representing tem-perature and outliers

XT1, XT2, XT3Temperature related exogenousvariables

XO1, XO2Occupancy related exogenous vari-ables

�s The standard deviation of season s

✏tThe value of the white noise com-ponent at time t

�is The coe�cient of Yt�i in season s

js

The coe�cient of the jth exogenousvariable in season s

Cs The intercept term of season s

Table 1: List of symbols used in this paper

First, we deal with temperature. Recall Figure 2 and no-tice that the e↵ect of temperature in the summer may bevery di↵erent to the e↵ect of temperature in the winter. Insouthern Ontario, the regression line has a positive slopeat high temperatures, corresponding to the increasing in-tensity of air conditioning usage as temperatures climb. Onthe other hand, the winter e↵ects of temperature are lesspronounced, because the majority of homes, including oursample home, mainly use natural gas for heating.

The above observation implies that we cannot use a sin-gle exogenous variable to account for temperature. Instead,following previous work on modelling the e↵ect of temper-ature on electricity consumption (e.g., [5, 13]), we use threevariables: XT1, XT2, and XT3. They are defined in Equa-tions (2),(3) and (4). The coe�cients of these variables rep-resent the cooling (temperature above 20 degrees), heating(temperature below 16 degrees), and overheating (tempera-ture below 5 degrees) slopes, respectively.

XT1 =

⇢T � 20 if T > 200 otherwise

(2)

XT2 =

⇢16 � T if T < 160 otherwise

(3)

XT3 =

⇢5 � T if T < 50 otherwise

(4)

Now, we show how to handle “outliers” corresponding tounusually low or high consumption. The first step is as fol-lows. For each season (i.e., hour of the day), logically weproduce a plot similar to that in Figure 2, which illustratesthe relationship between temperature and consumption atthat particular hour of the day across di↵erent days, usingall the historical data given as input. For each value of tem-perature, we then compute the 10th and 90th percentiles ofthe consumption values. Using these values, we then definetwo new exogenous variables, XO1 and XO2, as follows.

• At any time t, XO1 is equal to one if the consumptionat t is higher than the 90th percentile of the consump-tion at that hour of the day and that specific temper-ature, as described above. It is zero otherwise.

• Similarly, XO2 is equal to one if the consumption at tis less than the 10th percentile of the consumption atthat hour of day and that specific temperature. It iszero otherwise.

We chose the 10th and 90th percentile values heuristicallyto define XO1 and XO2, corresponding to very low con-sumption (when the home may have been empty for a longwhile) and very high consumption (when the household isunusually busy). We refer to XO1 and XO2 as occupancy-related variables.

Using all five exogenous variables, our PARX model be-comes

Yt =pX

i=1

�isYt�i + 1sXT1t +

2sXT2t + 3sXT3t

+ 4sXO1t +

5sXO2t + Cs + ✏t, for t 2 s (5)

143

3.3 Parameter EstimationThe first parameter that we need to set is p, the number

of previous days to include in the auto-regressive part ofthe model. For each of our 24 seasons (hours of day), wetried di↵erent values of p between 1 and 48, and computedthe Bayesian Information Criterion (BIC) [25]. Based on ourdata set of 1000 homes, p = 3 gave the best results (i.e., thelowest BIC).

Once we have determined an optimal value of p, we usethe standard Ordinary Least Squares (OLS) method to de-rive the coe�cients of the PARX model for each hour ofthe day (repeating the process for weekdays only, and forweekends/holidays).

3.4 Putting it All TogetherHaving presented the details of our PARX framework, we

are now ready to describe how the consumption profiles arederived. First, we compute our PARX model for each hourof day, separately for weekdays and weekends/holidays. Wethen generate a new consumption time series by taking theoriginal values and removing the temperature-sensitive con-sumption as well as the outliers. For each hour of day (andseparately for weekdays and weekends/holidays), we do thissimply by “reversing” the model and subtracting the e↵ectsof exogenous variables. Let Y ⇤

t be the new consumption timeseries after removing temperature- and occupancy-sensitivecomponents:

Y ⇤

t = Yt � 1sXT1t �

2sXT2t � 3sXT3t

� 4sXO1t �

5sXO2t for t 2 s (6)

That is, what remains in Y ⇤

t is just the auto-regressive partof the model.

Finally, we take the hourly averages of the correspondingY ⇤

t ’s, separately for weekdays and weekends/holidays, whichcompletes the discussion of the process shown in Figure 3.Thus, our profiles consist of two vectors of 24 values, wherethe ith value is the typical consumption level of the givenhome at the ith hour of the day, after removing the e↵ectsof exogenous variables.

Figures 4 and 5 illustrate the weekday and weekend pro-files of our sample home. Note that the profiles are easy tointerpret and contain a great deal of useful information . Forexample, it is easy to see that 1) the typical hourly consump-tion is higher on weekdays than weekends, 2) peak weekdayload occurs at 19:00 with a small peak at 9:00, while peakweekend load occurs at 17:00, and 3) the occupants of thishome appear to consume more electricity between 8:00 and11:00 on weekends than weekdays.

3.5 ApplicationsWe conclude this section with a brief description of how

the proposed consumption profiles can be used for two ofthe motivating applications listed in Section 1.

Personalized recommendations for saving electricityNormally, we expect the hourly consumption of a typicalhousehold to decrease at night, when there is little to noactivity in the home. If the consumption profile suggeststhat the nightly consumption of a given household remainshigh, then we can recommend a new refrigerator or anotherappliance that is always on. Note that this recommendation

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 230

0.25

0.5

0.75

1

1.25

1.5

Time of day

Hou

rly e

nerg

y co

nsum

ptio

n (k

Wh)

Figure 4: Weekday consumption profile of a sample home.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 230

0.25

0.5

0.75

1

1.25

1.5

Time of day

Hou

rly e

nerg

y co

nsum

ptio

n (k

Wh)

Figure 5: Weekend/holiday consumption profile of a samplehome.

makes sense because we have removed temperature-sensitiveload before computing the profiles. Otherwise, we would notknow if the high nightly load is caused by heating and cool-ing or by another appliance. Similarly, if the profile showshigh consumption during expensive on-peak hours regard-less of temperature, then we can recommend shifting someactivities (such as laundry) to o↵-peak hours.

Furthermore, we can generate comparative feedback byclustering similar consumers based on the hourly loads con-tained in their profiles and/or the coe�cients of their ex-ogenous variables. For instance, if a household belongs to acluster in which other households have similar hourly loadsbut lower coe�cients of the temperature-related exogenousvariables, then we can hypothesize that this household hasan ine�cient air conditioning system.

144

Generating realistic synthetic data as input to forecast-ing models or to test the scalability of smart meter datamanagement systemsHere, the objective is to create realistic synthetic “house-holds” based on the profiles computed from a real data set.One way to do this is as follows. First, we use a clustering al-gorithm such as k-means to group together similar profiles.To generate a new consumption time series, we randomlychoose a cluster and use its centroid as the consumptionprofile of the new household. We can then choose the ex-ogenous variable coe�cients from a random member of thiscluster, generate a weather forecast time series, and use thisinformation to create the new consumption time series.

4. EXPERIMENTSIn this section, we evaluate the predictive power of our

consumption profiles and compare it to the predictive powerof representative profiling techniques from prior work—onethat focuses on hourly consumption aggregates, one thatuses temperature alone, and one that uses both tempera-ture and hourly averages. We implemented the algorithmsin Matlab.

Our dataset is comprised of aggregate hourly electricityconsumption levels of 1000 homes from a city in south-ern Ontario, Canada. Measurements were taken betweenMarch 2011 and October 2012. We also obtained the ambientair temperature data of that region from the EnvironmentCanada Website.

4.1 MethodologyWe use two thirds of the consumption dataset of each

home as the training data set for building the model, andthe rest, including 170 days from April 2012 to October 2012,as the testing data set for evaluating its predictive power.For instance, to evaluate the predictive power of a modelon April 1, 2012, we use consumption measurements fromMarch 2011 to March 2012 as the training set. We extendthe training set by adding days from the test set which areprior to the day for which we evaluate the predicative power.For example, April 1, 2012 is added to the training set whenwe evaluate predictive power for April 2, 2012.

We predict the consumption of each home using the fol-lowing four profiling approaches, assuming that the hourlytemperature forecast is available one day in advance.

The first is our approach, labeled PARX. We predict thehourly consumption on the test day, for a given hour h, asfollows. First, we look up the average hourly consumptionfor that hour from the profile, call it Ph. We then add in thecontribution of exogenous variables for that hour using thecoe�cients of that hour’s model. This gives us an estimateof the consumption for that hour, call it Yh:

Yh = Y ⇤

h + 1hXT1h +

2hXT2h + 3hXT3h

+ 4h

\XO1h + 5h

\XO2h (7)

Note that in order to use our consumption profiles forpredicting future consumption, we must use estimated val-ues of the occupancy-related exogenous variables, denoted[XO1 and [XO2, since we obviously do not know their truevalues when making the prediction. Here, we simply usethe observed value of these variables in the previous hour

( \XO1h = XO1h�1

and \XO2h = XO2h�1

), although moresophisticated methods could be used to estimate these vari-ables and further improve the predictive power of our ap-proach.

The second approach represents methods that computehourly aggregates from the consumption time series. In par-ticular, we compute hourly averages over the training setand use these for prediction. We call this approach HourlyMean.

The third approach represents methods that focus on thecorrelation between consumption and temperature [5]. Thisalgorithm fits a three-piece linear regression model after re-moving very-low and very-high consumption values and usesthe temperature of the test day to predict consumption. Werefer to this technique as 3-Line.

Finally, the fourth approach, proposed in [13], uses a timeseries model similar to PARX and also takes temperatureinto account (but does not account for outliers, which wedo). We call this algorithm Convergent Vector since it com-putes typical hourly consumption by finding the convergentvector of the input time series.

The real value of the hourly electricity consumption on thetest day is then compared with the predicted values to com-pute the root-mean-square error (RMSE) of each approachfor each day.

4.2 ResultsWe compared the predictive power of the above four ap-

proaches using all 1000 homes in our dataset. Our findingsare as follows.

• PARX outperformed Hourly Mean for 982 homes, 3-Line for 960 homes, and Convergent Vector model for901 homes.

• The average RMSE was 0.70 for PARX, 0.81 for HourlyMean, 0.94 for 3-Line, and 0.77 for Convergent Vec-tor. This means that our model’s RMSE was 14 per-cent lower than that of Hourly Mean, 26 percent lowerthan that of 3-Line, and 9 percent lower than that ofConvergent Vector.

Thus, although 3-Line and Convergent Vector obtaineda slightly lower prediction error for a few homes, on aver-age their prediction error is considerably higher than thatof PARX. This confirms that, in most cases, incorporat-ing historical hourly consumption, temperature dependence,and occupancy dependence results in a more representativemodel. Nevertheless, in some cases, temperature is highlycorrelated with electricity consumption, and therefore incor-porating occupancy does not improve prediction accuracy.This is most likely because the bulk of the electricity con-sumption of these homes serves heating and cooling needs,and thus temperature alone is a very good predictor.

Figure 6 shows the average RMSE on the testing daysfor 20 randomly selected homes along with the RMSE val-ues averaged over all 1000 homes on the testing days. TheRMSE of PARX is lower than the RMSE of the other threeapproaches for all but two of these homes.

5. RELATED WORKA considerable body of previous work has developed

various consumption modelling techniques from household

145

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 avg.0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

3-LineHourly MeanConvergent VectorPARX

Figure 6: Average RMSE of the four approaches on 20 randomly selected homes measured on 170 testing days.

smart meter data, with applications ranging from cluster-ing similar consumers, planning and forecasting, tari↵ de-sign, electricity loss and theft detection, to providing per-sonalized feedback on how to save electricity. Our techniquemay be used in many of these applications in regions wheresome fraction of the electricity consumption is correlatedwith temperature.

From a technical standpoint, previous work has ap-proached the consumption profiling problem from two di-rections. One was to examine historical consumption val-ues and compute various representative aggregates; see, e.g.,[3,8–10,14,20,22]. The other direction has been to focus onthe relationship between electricity consumption and tem-perature; see, e.g., [1, 5]. There are also techniques thatcombine aggregated values with temperature correlations,e.g., [13]. In this paper, our goal was to design a simple andinterpretable but also accurate profiling algorithm by com-bining the best features of existing methods.

In particular, the two approaches most closely related toours are by Espinoza et al. [13] and Birt et al. [5]. Our ap-proach combines the time series modelling approach of [13]with the temperature model of [5], and additionally takesoutliers into account. As we experimentally showed in Sec-tion 4, by combining and enhancing the best features of priormodels, our techniques resulted in a lower prediction error.

In general, energy data management is an emerging fieldof study, with recent work on smart grid data managementand analytics [6, 15], using Hadoop to manage smart meterdata [11], imputing missing data in smart meter time series[18], and symboling representation of smart meter time series[26].

6. CONCLUSIONS AND OPEN PROB-LEMS

In this paper, we described a simple and interpretabletechnique for computing electricity consumption profilesfrom residential smart meter data combined with temper-ature data. Our solution relies on auto-regressive time se-ries modelling with exogenous variables to take into accountvarious factors influencing electricity consumption, such astemperature and the occupants’ daily habits. Experimentalevaluation using a real data set of smart meter readings from1000 homes revealed the advantages of our method over pre-vious work in terms of prediction accuracy.

One limitation of the proposed approach is that it is e↵ec-tive only for regions where some fraction of household elec-tricity consumption is correlated with temperature, such asthose with heavy air conditioning use during the summer.If this is not the case, simpler profiling techniques may beused, such as computing the average electricity consumptionin each hour of the day.

We are currently building a prototype smart meter datamanagement system, in which the proposed consumptionprofiling method will play a central role. We highlight severalopen problems in this area that we intend to study:

• Smart meter data quality: missing values are common,and unusually low or high values may indicate failingmeters that need to be replaced.

• E�cient and scalable smart meter analytics: there isvery little work that focuses on exploiting modern dataanalytics platforms, such as Hadoop, data stream en-gines or time series databases, for smart meter data.

146

• In addition to smart electricity meters, smart wa-ter meters are being introduced in many juris-dictions, including Toronto, Canada, as describedat torontowatermeterprogram.ca. This will enablelarge-scale water data analytics and require smart me-ter data management system to handle water data inaddition to electricity data.

7. REFERENCES[1] A. Albert and R. Rajagopal. Building dynamic

thermal profiles of energy consumption for individualsand neighborhoods. In IEEE Big Data Conf., 2013

[2] AutoGrid. www.auto-grid.com[3] C. Beckel, L. Sadamori, and S. Santini. Towards

automatic classification of private households usingelectricity consumption data. In 4th ACM Workshopon Embedded Sensing Systems for Energy-E�ciency inBuildings (BuildSys), pages 169-176, 2012.

[4] Big Data Energy Services.www.bigdataenergyservices.com

[5] B. J. Birt, G. R. Newsham, I. Beausoleil-Morrison,M. M. Armstrong, N. Saldanha, and I. H. Rowlands.Disaggregating categories of electrical energy end-usefrom whole-house hourly data. Energy and Buildings,50(0):93–102, 2012.

[6] M. Boehm, L. Dannecker, A. Doms, E. Dovgan, B.Filipic, U. Fischer, W. Lehner, T. B. Pedersen, Y.Pitarch, L. Siksnys and T. Tusar. Data Management inthe MIRABEL Smart Grid System. In 1st Workshopon Energy Data Management (EnDM), 2012.

[7] C3 Energy. www.c3energy.com[8] G. Chicco. Overview and performance assessment of

the clustering methods for electrical load patterngrouping. Energy, 42(1):68–80, 2012.

[9] G. Chicco and I.-S. Ilie. Support vector clustering ofelectrical load pattern data. IEEE Trans. on PowerSystems, 24(3):1619–1628, 2009.

[10] G. Chicco, R. Napoli, P. Postolache, M. Scutariu, andC. Toader. Customer characterization options forimproving the tari↵ o↵er. IEEE Trans. on PowerSystems, 18(1):381-387, 2003.

[11] L. dos Santos, A. da Silva, B. Jacquin, M.-L. Picard,D. Worms, C. Bernard. Massive Smart Meter DataStorage and Processing on top of Hadoop, In VLDBBigData Workshop, 2012.

[12] eSmart Systems. www.esmartsystems.com[13] M. Espinoza, C. Joye, R. Belmans, and B. DeMoor.

Short-term load forecasting, profile identification, andcustomer segmentation: A methodology based onperiodic time series. IEEE Trans. on Power Systems,20(3):1622–1630, 2005.

[14] V. Figueiredo, F. Rodrigues, Z. Vale, and J. Gouveia.An electric energy consumer characterizationframework based on data mining techniques. IEEETrans. on Power Systems, 20(2):596–602, 2005.

[15] U. Fischer, D. Kaulakiene, M. E. Khalefa, W. Lehner,T. B. Pedersen, L. Siksnys and C. Thomsen. Real-timeBusiness Intelligence in the MIRABEL Smart GridSystem. In VLDB Workshop on Business Intelligencefor the Real-Time Enterprise (BIRTE), 2012.

[16] Green Button. www.greenbuttondata.org

[17] H. Hurd and A. Miamee. Periodically correlatedrandom sequences: spectral theory and practice, volume355. Wiley-Interscience, 2007.

[18] R.-S. Jeng, C.-Y. Kuo, Y.-H. Ho, M.-F. Lee, L.-W.Tseng, C.-L. Fu, P.-F. Liang, L.-J. Chen. MissingData Handling for Meter Data Management System.In e-Energy Conf., pages 275-276, 2013.

[19] J. Miller. North american power plant air emissions.Technical Report 2-923358-11-2, Commission forEnvironmental Cooperation of North America, 2004.

[20] A. Nizar and Z. Dong. Identification and detection ofelectricity customer behaviour irregularities. InIEEE/PES Power Systems Conference andExposition, pages 1–10, 2009.

[21] OPower. www.opower.com[22] T. Rasanen, D. Voukantsis, H. Niska, K. Karatzas and

M. Kolehmainen. Data-based method for creatingelectricity use load profiles using large amount ofcustomer-specific hourly measured electricity use data.Applied Energy, 87(11):3538-3545, 2010.

[23] I. Rowlands. Demand response in Ontario: exploringthe issues. A report for the Independent ElectricitySystem Operator (IESO) of Ontario, 2008.

[24] U.S. Annual Energy Outlook 2012.www.eia.gov/forecasts/aeo/pdf/0383(2012).pdf.

[25] W. W.-S. Wei. Time series analysis: univariate andmultivariate methods, pages 156–157. Addison-Wesley,2nd edition, 2006.

[26] T. K. Wijaya, J. Eberle and K. Aberer. Symbolicrepresentation of smart meter data. In 2nd Workshopon Energy Data Management (EnDM), pages 242-248,2013.

147