Time series Forecasting

Time series Forecasting

Presented at: Big Data Analy2cs Programme NIT Srinagar (15 March 2017) Presenter: Haroon Rashid [email protected]

Help Yourself!

2

Contents

� What is forecas2ng � Why we need it

�  Forecas2ng approaches �  Forecas2ng evalua2on �  Forecas2ng applica2ons

3

Forecasting: Case Scenario - I

500

1000

Aug 28 Aug 29 Aug 30 Aug 31 Sep 01Timestamp

Powe

r (W

) variableTrain

Forecast

4

Forecasting: Case Scenario - II

5

Find carbon footprint for Toyota Matrix with capacity of 1.8 (L), and mileages of 25 (C) and 31 (H)?

Sta2s2cs of 4 cylinder cars

Table: hWps://www.otexts.org/fpp/1/4

Forecasting Types

� Time series Forecas2ng �  Data collected at regular intervals of 2me �  e.g., Weather, electricity forecas2ng

� Cross-‐Sec2onal Forecas2ng �  Data collected at single point in 2me �  e.g., Carbon emission, disease predic2on

6

Time series Forecas2ng (Energy)

Assumptions

1.  Historical informa2on is available 2.  Past paWerns will con2nue in the future

7

100

200

300

400

Aug 03 Aug 04 Aug 05 Aug 06Timestamp

Powe

r (W

)

500

1000

Aug 27 Aug 28 Aug 29 Aug 30Timestamp

Powe

r (W

)

Forecasting Horizon

1.  Short-‐term forecas2ng: �  Hours to few days ahead

2.  Medium-‐term forecas2ng: �  Few days to months ahead

3.  Long-‐term forecas2ng: �  Months to years ahead

8

I. Short term decision II. Long term investment

9

Why electricity forecasting

2

3

4

5

0 20 40 60 80Months (Approx. 6 years)

Pow

er (g

W)

Sol: Backup Generators 1.  Renewables (Solar) 2.  Diesel Generators 3.  Power plants

Img: hWp://www.installeronline.co.uk/brownout-‐one-‐coming/

Forecasting steps

1.  Problem defini2on �  Purpose (demand supply, Fault detec2on) �  Factors (Weather, occupancy, day type)

2.  Informa2on gathering �  Sense [Sensors: smart meters, temperature, PIR] �  Retrieve [Z-‐wave, Bluetooth, Wi-‐Fi, GSM] �  Store [Mongo DB, MySQL]

3.  Preliminary analysis �  Visualiza2ons [R, Python, Matlab, Plotly]

4.  Choosing & Fieng models 5.  Model evalua2on

10

3. Preliminary analysis

0.5

1.0

0.00.51.01.52.0

0255075

100125

0246

Apartm

ent_1A

partment_2

HVAC

Chiller

StreetLights

Aug 09 Aug 10 Aug 11 Aug 12 Aug 13 Aug 14 Aug 15

Powe

r(KW

)

11

Sunday Saturday

3. Preliminary analysis

500

1000

1500

2000

2012−05−01 2012−11−01 2013−05−01 2013−11−01 2014−05−01 2014−11−01Timestamp

Powe

r (wa

tts)

12

3. Preliminary analysis: Seasonality, trend

13

data

seasonal

trend

remainder

2 4 6 8 10 12

200

400

600

−40

0

40

200

300

400

500

−50−250255075

Time

4. Model fitting: Averaging approach

500

1000

1500

2000

Aug 30 Sep 01 Sep 03 Sep 05 Sep 07 Sep 09Timestamp

Ener

gy (W

)

variableActual

Forecast

Average last 6, RMSE = 251.44

14

Yt+1|1...t =(Y1 + Y2 + ...+ Yt)

length(t)

4. Model fitting: Naïve approach

500

1000

1500

2000


Ener

gy (W

)

variableActual

Forecast

Naive, RMSE = 246.14

15

Yt+1|t = Yt

4. Model fitting: Vertical approach

16

0.1

0.2

0.3

0.4

0 5 10 15 20 25Hour of the Day

0.1

0.2

0.3

0.4


Powe

r (kW

)

Day291623

Forecast of Day 30, ‘15

Weighted Forecasting

5. Model evaluation: Prediction accuracy

Root mean square error (RMSE): Lower is beWer

17

RMSE =

vuut 1

n

nX

i=1

(yi � yi)2n = values

yi = Actual values

yi = Forecast values

y = [713, 711, 652, 522]

y = [751, 713, 711, 652]

RMSE = 73

5 .Model evaluation: Residual diagnostics

−1000

0

1000


Res

idua

ls (W

)

Naive − Residuals

18

Demo1

Summary - I

1.  Time series forecas2ng 2.  Steps in forecas2ng 3.  Forecas2ng models: Naïve, averaging 4.  Model evalua2on 5.  Demonstra2on in R

19

Line fitting

20

●

●

●

●

●

●

●

●

●

●

10

20

30

1 2 3 4 5 6 7 8 9 10X

YLine Fitting

Y = mX + C

Slope = 3, Intercept = 0 Approx.

Linear Regression

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

200

400

600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day

Powe

r (W

)

21

Linear Regression

●

● ●

●

●●

●

●●

●

●

●

●

●

●

200

400

600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day

Powe

r (W

)

Intercept:764.2 Slope:−45.6

22

y = �0 + �1x+ ✏

power = �0 + �1(dayhour) + ✏

Least squares

●

● ●

●

●●

●

●●

●

●

●

●

●

●

200

400

600

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day

Powe

r (W

)

Intercept:764.2 Slope:−45.6

23

NX

i=1

✏

2i =

NX

i=1

(yi � �0 � �1xi)2

Error

Minimize

Evaluation

1.  Standard error: Lower is beWer

2.  Goodness of fit: Higher is beWer

24

ei = yi � yi

se =

vuut 1

N � 2

NX

i=1

e2i

R2 =

P(yi � y)2P(yi � y)2

R_squared: hWps://goo.gl/Xm5gUd

Evaluation

25

Regression: Scenario II

●● ●

●

●●

●

● ●●

●

●

●

●

● ●●

●

●● ●

●

●

●

●

●●

●

●●

●

●● ●

●●

●

●

●●

●●

●

●

●

●

● ●●

●

● ●●

●

●

●

●

●

● ●●

●

● ● ● ● ●

●

● ●

●

●

500

1000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day

Powe

r (W

)

26

Regression: Scenario II

27

●● ●

●

●●

●

● ●●

●

●

●

●

● ●●

●

●● ●

●

●

●

●

●●

●

●●

●

●● ●

●●

●

●

●●

●●

●

●

●

●

● ●●

●

● ●●

●

●

●

●

●

● ●●

●● ● ● ● ●

●

● ●

●

●

500

1000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day

Powe

r (W

)

Residual error:297.01

y = �0 + �1x+ ✏

Polynomial regression

●● ●

●

●●

●

● ●●

●

●

●

●

● ●●

●

●● ●

●

●

●

●

●●

●

●●

●

●● ●

●●

●

●

●●

●●

●

●

●

●

● ●●

●

● ●●

●

●

●

●

●

● ●●

●● ● ● ● ●

●

● ●

●

●

500

1000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day

Powe

r (W

)

Residual error:174.48

28

y = �0 + �1x+ �1x2 + ...+ �1x

p + ✏

Spline regression

29

●● ●

●

●●

●

● ●●

●

●

●

●

● ●●

●

●● ●

●

●

●

●

●●

●

●●

●

●● ●

●●

●

●

●●

●●

●

●

●

●

● ●●

●

● ●●

●

●

●

●

●

● ●●

●

● ● ● ● ●

●

● ●

●

●

500

1000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day

Powe

r (W

)

Multiple regression

−1

0

1

2

3

Aug 28 Aug 29 Aug 30 Aug 31 Sep 01Timestamp

Powe

r (W

) variableTrain

Forecast

Actual

Residual Test RMSE:1.05

30

Demo 2

Summary - II

1.  Regression (Line fieng) 1.  Linear 2.  Non-‐Linear (Cousin : Splines) 3.  Mul2ple

2.  Demonstra2on

31

Auto-regressive (AR) model

� Auto-‐regressive model: � Models future values as a func2on of recent past sequen2al values

� Representa2on: An AR model with past p values is denoted as AR(p)

32

Yt = f(Yt�1, Yt�2, ..., Yt�p, ✏t)

Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p + ✏t

Moving Average (MA) model

33

Yt = �0 + ✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q

Yt = f(✏t, ✏t�1, ✏t�2, ..., ✏t�q)

•  Moving average model: •  Models future values as a func2on of recent

past sequen2al error terms

•  Representa2on: An MA model with past q values is denoted as MA(q)

AR MA model

34

Yt = f(✏t, ✏t�1, ✏t�2, ..., ✏t�q, Yt�1, Yt�2, ..., Yt�p)

Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p+

✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q

•  Auto regressive Moving Average (ARMA) model: •  Models future values as a func2on of recent

past sequen2al values and error terms

•  Representa2on: ARMA(p, q) model

Stationary vs. Non stationary time-series

200

300

400

500

0 20 40 60 80Timestamp

Beer

produ

ction (

mega

litres

)

35

400

450

500

550

600


Beer

produ

ction (

mega

litres)

−100

−50

0

50

100


Beer

produ

ction

(mL)

Non-‐sta2onary Non-‐sta2onary

Sta2onary

Differencing (Non-stat. -> Stat.)

36

−100

−50

0

50

100


Beer

pro

duct

ion

(mL)

200

300

400

500


Beer

pro

duct

ion

(mL)

Yt = Yt � Yt�1

Differencing order (d): Number of 2mes differencing is done

ARIMA model

37

ARIMA is defined by a tuple (p, d, q)

Auto-‐Regressive Integrated Moving Average

AR I MA

Yt = �0 + ✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q

Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p + ✏tYt = Yt � Yt�1

[Order p] [Order d]

[Order q]

ACF/PACF plots

1.  Auto-‐Correla2on Func2on (ACF) Plot: �  Correla2on coefficients of 2me-‐series at different lags �  Defines q order of MA model

2.  Par2al Auto-‐correla2on Func2on (PACF) Plot:

�  Par2al correla2on coefficients of 2me series at different lags

�  Defines p order of AR model

38

ACF/PACF plots

39

500

1000

Aug 29 Aug 30 Aug 31 Sep 01Timestamp

Powe

r (W

)

−0.25

0.00

0.25

0.50

5 10 15Lag

PACF

−0.25

0.00

0.25

0.50

5 10 15Lag

ACF

Data

PACF plot

ACF plot

Model evaluation

� AIC/BIC � Residual errors

40

Demo 3

Seasonality

41

data

seasonal

trend

remainder

2 4 6 8 10 12

200

400

600

−40

0

40

200

300

400

500

−50−250255075

Time

SARIMA (Seasonal ARIMA)

�  SARIMA(p,d,q) (P,D,Q):

�  Order (P,D,Q) handles the seasonality part

42

Complete Forecasting pseudocode

1.  Visualize 2me-‐series data 2.  If data is noisy

�  Apply averaging or naïve model

3.  If data is not sta2onary �  First, sta2onarize data using differencing �  Next, apply any 2me series model

4.  If data is already sta2onary �  Apply any 2me series model

43

Challenges : Outliers

44

0.5

1.0


Powe

r (kW

)

Day26272829

Anomalous usage

0.25

0.50

0.75

1.00

1.25


ForecastActual

Day 30

Challenges: Domain Knowledge

45

100

200

300

400


Powe

r

Day291623

250

500

750


Powe

r (w)

ForecastActual

1

500

1000


Powe

r (w)

Day26272829

250

500

750

1000

1250


Powe

r (w)

ForecastActual

2

Application: Anomaly Detection

46

References

1.  Book: Forecas2ng principles and prac2ce, hWps://www.otexts.org/fpp

2.  Understanding seasonality and trend with code: hWps://anomaly.io/seasonal-‐trend-‐decomposi2on-‐in-‐r/

3.  ARIMA ordering: hWps://people.duke.edu/~rnau/411arim3.htm

4.  Time series Forecas2ng theory: hWps://www.youtube.com/watch?v=Aw77aMLj9uM

5.  Book: Applied predic2ve modeling by kuhn et al. 6.  Book: An introduc2on to sta2s2cal learning by

Gareth et al.

47

Annexure

48

Multidimensional Scaling (MDS)

0

300

600

900

0 5 10 15 20 25Hour of the day

Powe

r (wa

tts)

Day1Day2Day3Day4

49

●

●

●

●

−1500 −1000 −500 0 500 1000 1500

−600

−200

020

060

0

MDS Dimension−1

MD

S D

imen

sion−2

Day1

Day2

Day3

Day4

Dissimilarity/Distance Matrix

Apply MDS

Day1 Day2 Day3 Day4

Day1 0000 2789 1194 2699

Day2 2789 0000 2516 0254

Day3 1194 2516 0000 2371

Day4 2699 0254 2371 0000

dist(dayx

, dayy

) =qP

n=24i=1 (dayi

x

� dayiy

)2

Regression coefficient

�1 =

PNi=1(yi � y)(xi � x)PN

i=1(xi � x)2

�0 = y � �1x

50

Forecasting band

51

200

204

208

0 5 10 15 20 25Hour of the day

Powe

r (wa

tts)

Actual_UsageForecast

Time series Forecasting

Data & Analytics

Transcript of Time series Forecasting