Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications...
Transcript of Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications...
![Page 1: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/1.jpg)
Seasonal-Trend Decomposition in Time Series
POSTECHDepartment of Creative IT Engineering
Doyup Lee
![Page 2: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/2.jpg)
Index• What is Time Series ?
• Stationarity vs. Non-stationarity
• Basic Approaches of Time Series Modeling
• RobustSTL (AAAI 2019 paper)
• Q & A
![Page 3: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/3.jpg)
Caution
• This material with English.
• Many typings and mathematics.
• Need Background Knowledge about Time Series and Optimization.
![Page 4: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/4.jpg)
Caution
• This material with English.
• Many typings and mathematics.
• Need Background Knowledge about Time Series and Optimization.
• But if you concentrate on the presentation and follow me, you can understand and know I am a liar.
![Page 5: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/5.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
![Page 6: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/6.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(X)
![Page 7: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/7.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(X)Data (Observations):X_1, X_2, ……, X_N
Machine Learning Model
![Page 8: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/8.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(X)Data (Observations):X_1, X_2, ……, X_N
Machine Learning Model
sampling
![Page 9: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/9.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
![Page 10: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/10.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(Y|X)
![Page 11: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/11.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(Y|X) Data (Observations):X_1, X_2, ……, X_N Y_1, Y_2, ……, Y_N
![Page 12: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/12.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(Y|X) Data (Observations):X_1, X_2, ……, X_N Y_1, Y_2, ……, Y_N
Machine Learning Model
![Page 13: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/13.jpg)
What Machine Learning Do ?• In general, machine learning estimates probability distribution.
- Generative model learns to estimate joint probability P(X) or P(X, Y)- Discriminative model learns to estimate conditional probability P(Y|X)
P(Y|X) Data (Observations):X_1, X_2, ……, X_N Y_1, Y_2, ……, Y_N
Machine Learning Model
Given test data X
![Page 14: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/14.jpg)
Time Series Data is all around us• Services: User Log
• Finance: Many types of Price
• Manufactoring: Smart Factories
• Health Care: EEG, ECG, …
• Meteorological Data
• Etc….
![Page 15: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/15.jpg)
Because• NO TIME NO LIFE : Our life is defined by “TIME”
• Everything in our life is connected with time changes.- Does a state change or not ?- How it changes ?- Which characteristics in the state according to time change?
![Page 16: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/16.jpg)
Time Series
![Page 17: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/17.jpg)
Time Series• Time Series:
An ordered sequence of values of a variable at equally spaced time intervals.
![Page 18: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/18.jpg)
Time Series• Time Series:
An ordered sequence of values of a variable at equally spaced time intervals.
• Time Series Modeling:Model a stochastic process with autoregressive manners.
![Page 19: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/19.jpg)
Time Series• Time Series:
An ordered sequence of values of a variable at equally spaced time intervals.
• Time Series Modeling:Model a stochastic process with autoregressive manners.
• In the end, time series modeling can be to find probability distribution of a variable at time t, conditioned on past time t-1, t-2, …… .
![Page 20: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/20.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
![Page 21: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/21.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 2
T+1
![Page 22: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/22.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 3
T+1
![Page 23: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/23.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample N
T+1
![Page 24: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/24.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
![Page 25: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/25.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
![Page 26: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/26.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
observation(realization)
Distribution of Ordered (T+1) Series
![Page 27: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/27.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
Distribution of Ordered (T+1) Series
![Page 28: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/28.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
Distribution of Ordered (T+1) Series
God
![Page 29: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/29.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
Distribution of Ordered (T+1) Series
![Page 30: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/30.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
Distribution of Ordered (T+1) Series
(Machine Learning) Model
![Page 31: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/31.jpg)
Time Series Modeling Intuitiontimet=0 t=1억
sample 1
T+1
sample 2
sample N
.
.
.
Distribution of Ordered (T+1) Series
(Machine Learning) Model
Estimate the true distribution based on training data
![Page 32: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/32.jpg)
ARMA (Whittle 1951; Box&Jenkins, 1971)
• ARMA (AutoRegressive Moving Average) is a typical model for time series.
• ARMA(p,q): a generative linear model that combines AR(p) and MA(q)
AR(p) MA(q)
![Page 33: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/33.jpg)
Modeling Process
![Page 34: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/34.jpg)
Modeling Process
![Page 35: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/35.jpg)
Modeling Process
![Page 36: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/36.jpg)
Stationarity vs. Non-Stationarity• Stationary Time Series
- 기간 (Period)에 관계 없이 데이터의 확률 분포가 변하지 않은 시계열 데이터
![Page 37: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/37.jpg)
Stationarity vs. Non-Stationarity• Stationary Time Series
- 기간 (Period)에 관계 없이 데이터의 확률 분포가 변하지 않은 시계열 데이터
![Page 38: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/38.jpg)
Stationarity vs. Non-Stationarity• Nonstationary Time Series
- 기간 (Period)에 따라 데이터의 확률 분포가 변하는 시계열 데이터
![Page 39: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/39.jpg)
Stationarity vs. Non-Stationarity• Nonstationary Time Series
- 기간 (Period)에 따라 데이터의 확률 분포가 변하는 시계열 데이터
![Page 40: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/40.jpg)
Stationarity vs. Non-Stationarity• Nonstationary Time Series
- 기간 (Period)에 따라 데이터의 확률 분포가 변하는 시계열 데이터
![Page 41: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/41.jpg)
Stationarity vs. Non-Stationarity• Nonstationary Time Series
- 기간 (Period)에 따라 데이터의 확률 분포가 변하는 시계열 데이터
• Typical Examples:
- Trends
- Seasonality
- Changes of Variance
![Page 42: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/42.jpg)
Solutions for Nonstationarity• Differencing
• Data Transformation
• Seasonal-Trend Decomposition
• (Deep) Neural Networks
![Page 43: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/43.jpg)
• Differencing
• Data Transformation
• Seasonal-Trend Decomposition
• (Deep) Neural Networks
Solutions for Nonstationarity
![Page 44: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/44.jpg)
Solutions for Nonstationarity• Differencing
• Data Transformation
• Seasonal-Trend Decomposition
• (Deep) Neural Networks
![Page 45: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/45.jpg)
Solutions for Nonstationarity• Differencing
• Data Transformation
• Seasonal-Trend Decomposition
• (Deep) Neural Networks
![Page 46: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/46.jpg)
Solutions for Nonstationarity• Differencing
• Data Transformation
• Seasonal-Trend Decomposition
• (Deep) Neural Networks
https://deepmind.com/blog/wavenet-generative-model-raw-audio/
![Page 47: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/47.jpg)
RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for
Long Time Series
Qingsong Wen, Jingkun Gao, Xiamin Song, Liang Sun, Huan Xu, Shenghuo ZhuAlibaba
AAAI 2019 paper
![Page 48: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/48.jpg)
Summary• Decomposing complex time series into trend, seasonality, and
remainder components is an important task to facilitate time series anomaly detection and forecasting.
![Page 49: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/49.jpg)
Summary• Decomposing complex time series into trend, seasonality, and
remainder components is an important task to facilitate time series anomaly detection and forecasting.
• Limitation of previous researches 1) Ability to handle seasonality fluctuation and shift, and abrupt changes in trend and reminder2) robustness of data with anomalies3) applicability on time series with long seasonality period.
![Page 50: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/50.jpg)
Introduction• ST Decomposition can reveal the underlying insights of a time series and
can be useful in further analysis such as AD and forecasting.
![Page 51: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/51.jpg)
Introduction• ST Decomposition can reveal the underlying insights of a time series and
can be useful in further analysis such as AD and forecasting.
• Without decomposition, it would be missed as its value is still much lower than the unusually high values during a busy period.
![Page 52: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/52.jpg)
Introduction• ST Decomposition can reveal the underlying insights of a time series and
can be useful in further analysis such as AD and forecasting.
• Without decomposition, it would be missed as its value is still much lower than the unusually high values during a busy period.
• Spike & dip anomalies correspond to abrupt change of remainder and the change of mean anomaly corresponds to abrupt change of trend.
![Page 53: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/53.jpg)
Introduction• ST Decomposition can reveal the underlying insights of a time series and
can be useful in further analysis such as AD and forecasting.
• Without decomposition, it would be missed as its value is still much lower than the unusually high values during a busy period.
• Spike & dip anomalies correspond to abrupt change of remainder and the change of mean anomaly corresponds to abrupt change of trend.
• Previous approaches still suffer from less flexibility when seasonality period is long and high noises are observed. Or not feasible on large-size data.
![Page 54: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/54.jpg)
ST decomposition on Real-world
• 3 characteristics of real-world time series 1) Seasonality fluctuation and shift are quite common in real-world time series.2) Most algorithms can’t handle the abrupt change of trend and remainder.3) Most methods are not applicable to time series with long seasonality period and some of them can only handle quarterly or monthly data.
![Page 55: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/55.jpg)
Previous approaches Summary
![Page 56: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/56.jpg)
Robust STL Model Overview• What we want to do
![Page 57: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/57.jpg)
Robust STL Model Overview• A value on time t decomposes into (trend, seasonality, and remainder)
- Seasonality: related pattern which changes slowly or even status constant over time.- Trend: change faster than seasonality.- Remainder: it consists of anomalies(spikes and dips) and white noise.
![Page 58: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/58.jpg)
RobustSTL algorithm
![Page 59: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/59.jpg)
RobustSTL algorithm
• S1. Denoise time series by applying bilateral filtering
![Page 60: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/60.jpg)
RobustSTL algorithm
• S1. Denoise time series by applying bilateral filtering
• S2. Extract trend robustly by solving a LAD regression with sparse regularizations
![Page 61: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/61.jpg)
RobustSTL algorithm
• S1. Denoise time series by applying bilateral filtering
• S2. Extract trend robustly by solving a LAD regression with sparse regularizations
• S3. Calculate the seasonality component by applying a non-local seasonal filtering to overcome seasonality fluctuation and shift
![Page 62: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/62.jpg)
RobustSTL algorithm
• S1. Denoise time series by applying bilateral filtering
• S2. Extract trend robustly by solving a LAD regression with sparse regularizations
• S3. Calculate the seasonality component by applying a non-local seasonal filtering to overcome seasonality fluctuation and shift
• S4. Adjust extracted components (repeat S2 and S3)
![Page 63: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/63.jpg)
S1. Noise Removal
• In real-world applications when time series are collected, the observations may be contaminated by carious types of errors and noises.
• Noise removal is indispensable for trend and seasonality decomposition, robustly.
• Many approaches: low-pass filtering, moving/median average, Gaussian filter.
• The noise removal process “should not” destruct some underlying structuring in trend and seasonal components.
![Page 64: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/64.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
![Page 65: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/65.jpg)
S1. Noise Removal
![Page 66: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/66.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
t t+1t-H t+Ht-1 ……
![Page 67: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/67.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
t t+1t-H t+Ht-1 ……
![Page 68: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/68.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
t t+1t-H t+Ht-1 ……
t
![Page 69: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/69.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
![Page 70: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/70.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
![Page 71: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/71.jpg)
S1. Noise Removal• Bilateral filtering: cadge-preserving filter in image processing.
Use neighbors with similar values to smooth the time series. The abrupt change of trend and spike and dip can be fully preserved.
![Page 72: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/72.jpg)
S2. Trend Extraction
• The joint learning of trend and seasonal components is challenging.
• As the seasonality component is assumed to change slowly, we first perform seasonal difference operation for the despised signal to mitigate the seasonal effects.
![Page 73: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/73.jpg)
S2. Trend Extraction• Then, the seasonal difference is dominated by trend difference because
we assume seasonality and reminder difference are small.
![Page 74: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/74.jpg)
S2. Trend Extraction• Thus, the objective function of trend extraction is to recover the first order
difference of trend signal: LAD (robust to outliers)
SmoothnessTrend change unit
![Page 75: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/75.jpg)
S2. Trend Extraction• Thus, the objective function of trend extraction is to recover the first order
difference of trend signal: LAD (robust to outliers)
SmoothnessTrend change unit
• Second term assumes that the trend difference usually changes slowly but can also exhibit some abrupt level shifts.
• Third term assumes that the trends are smooth and piecewise linear such that sparsity.
![Page 76: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/76.jpg)
S2. Trend Extraction• Objective with matrix form.
![Page 77: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/77.jpg)
S2. Trend Extraction• The optimization problem is equivalent to below.
![Page 78: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/78.jpg)
S2. Trend Extraction• The optimization problem is equivalent to below.
![Page 79: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/79.jpg)
(Reminder) RobustSTL algorithm
• S1. Denoise time series by applying bilateral filtering
• S2. Extract trend robustly by solving a LAD regression with sparse regularizations
• S3. Calculate the seasonality component by applying a non-local seasonal filtering to overcome seasonality fluctuation and shift
• S4. Adjust extracted components (repeat S2 and S3)
![Page 80: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/80.jpg)
S3. Seasonality Extraction• After de-trending, it can be considered as a contaminated seasonality.
• To consider seasonality shift, non-local seasonal filtering is proposed and also consider K neighborhoods centered at seasonal parts.
• In this way, the points with most similar seasonality are automatically found and the seasonality shift problem is solved.
tt-h t+h……t-T t-T+h…t-T-h …t-kT t-kT+h…t-kT-h …
![Page 81: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/81.jpg)
S3. Seasonality Extraction• After de-trending, it can be considered as a contaminated seasonality.
• To consider seasonality shift, non-local seasonal filtering is proposed and also consider K neighborhoods centered at seasonal parts.
• In this way, the points with most similar seasonality are automatically found and the seasonality shift problem is solved.
tt-h t+h……t-T t-T+h…t-T-h …t-kT t-kT+h…t-kT-h …
t
![Page 82: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/82.jpg)
S3. Seasonality Extraction• Non-local Seasonal Filtering
I think the notationhas to be t, not t prime.
And it’s right !
![Page 83: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/83.jpg)
S3. Seasonality Extraction• Non-local Seasonal Filtering
![Page 84: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/84.jpg)
S3. Seasonality Extraction• Robustness of non-local seasonal filtering to outliers.
![Page 85: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/85.jpg)
S4. Final Adjustment• To make seasonal-trend decomposition unique,
RobustSTL makes the sum of seasonality components become zero, using mean shift.
![Page 86: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/86.jpg)
Algorithm Summary
![Page 87: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/87.jpg)
Results - Synthetic Data
![Page 88: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/88.jpg)
Results - Synthetic Data
![Page 89: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/89.jpg)
Results - Real-world Data 1
![Page 90: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/90.jpg)
Results - Real-world Data 2
![Page 91: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/91.jpg)
Implementations• codes: https://www.github.com/LeeDoYup/RobustSTL
![Page 92: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/92.jpg)
Some Open Questions• Real Time applications ?
• Neural Networks ?
• Higher frequency time series ?
• Does really the algorithm work well? (Some problems yet..)
![Page 93: Seasonality Trend Decomposition in Time Series · S1. Noise Removal • In real-world applications when time series are collected, the observations may be contaminated by carious](https://reader035.fdocuments.in/reader035/viewer/2022070714/5ed3af34e22b2e4cd501677e/html5/thumbnails/93.jpg)
Thank you.