Streaming Pattern Discovery in Multiple Time-Series Jimeng Sun Spiros Papadimitrou Christos...

Streaming Pattern Discovery in Multiple Time-Series

Jimeng Sun

Spiros Papadimitrou Christos Faloutsos

PARALLEL DATA LABORATORYCarnegie Mellon University

<your name here> © Apr 21, 2023 http://www.pdl.cmu.edu/ 2

Motivation

• Co-evolving time series (data streams) appear in many different applications—e.g.:• Disk access traffic in network clusters• Internet flow traffic in a network• Temperatures in a large building• Chlorine concentration in water distribution

network

Values are typically correlated

Would be very useful if we could summarize them on the fly


Example

water distribution network

normal operation

Phase 1 Phase 2 Phase 3

: : : : : :

: : : : : :

chlo

rine c

once

ntr

ati

ons

sensorsnear leak

sensorsawayfrom leak

time


• Discover “hidden” (latent) variables for:• Summarization of main trends for users• Efficient forecasting, spotting outliers/anomalies

• Incremental, real-time computation• Limited memory requirements

Goals


Phase 1 Phase 2 Phase 3

: : : : : :

: : : : : :

Example: chlorine measurements

water distribution network

normal operation major leak

chlo

rine c

once

ntr

ati

ons

sensorsnear leak

sensorsawayfrom leak


Phase 1

k = 1

Example: hidden variable

actual measurements(n streams)

k hidden variable(s)

We would like to discover a few “hidden(latent) variables” that summarize the key trends

Phase 1

: : : : : :

: : : : : :

chlo

rine c

once

ntr

ati

ons


Example: hidden variable trackingch

lori

ne c

once

ntr

ati

ons

Phase 1 Phase 1Phase 2 Phase 2



k = 2

: : : : : :

: : : : : :



Example: hidden variable trackingch

lori

ne c

once

ntr

ati

ons

Phase 1 Phase 1Phase 2 Phase 2Phase 3 Phase 3



k = 1

: : : : : :

: : : : : :



Method outline

• Step 1: How to capture correlations?

• Step 2: How to do it incrementally, when we have a very large number of points?

• Step 3: How to dynamically adjust the number of hidden variables?


1. How to capture correlations?

20oC

30oC

Tem

pera

ture

T1

• First sensor

time


1. How to capture correlations?

• First sensor• Second sensor

20oC

30oC

Tem

pera

ture

T2

time


20oC 30oC

1. How to capture correlations

20oC

30oC

Temperature T1

•Correlations:

•Let’s take a closer look at the first three value-pairs…

Tem

pera

ture

T2


20oC 30oC


20oC

30oC

Tem

pera

ture

T2

Temperature T1

•First three lie (almost) on a line in the space of value-pairs…

O(n) numbers for the slope, and One number for each value-pair (offset on line)

offse

t = “h

idde

n va

riabl

e”

time=1

time=2

time=3



20oC 30oC

20oC

30oC

Tem

pera

ture

T2

Temperature T1

•Other pairs also follow the same pattern: they lie (approximately) on this line


Method outline





From hidden variables

Experiments: chlorine concentration

166 streams2 hidden variables (~4% error)

Measurements

Reconstruction

[CMU Civil Engineering]

from sensor


Experiments: chlorine concentration

hidden variables

[CMU Civil Engineering]

• Both capture global, periodic pattern• Second: ~ first, but “phase-shifted”• Can express any “phase-shift”…


Conclusion• Many settings with hundreds of streams, but

• Stream values are, by nature, related• We proposed a method to

• discover hidden variables as summarization of main trends for users

• require only incremental computation without buffering of any past data

• Future work:• Apply on more applications: e.g, performance

monitoring for storage system, network system.


Related work

• Stream SVD [Guha, Gunopulos, Koudas / KDD03]• StatStream [Zhu, Shasha / VLDB02]• Clustering• [Aggarwal, Han, Yu / VLDB03], [Guha, Meyerson,

et al / TKDE],• [Lin, Vlachos, Keogh, Gunopulos / EDBT04], • Classification• [Wang, Fan, et al / KDD03], [Hulten, Spencer,

Domingos / KDD01]• Piecewise approximations• [Palpanas, Vlachos, Keogh, etal / ICDE 2004]


Experiments: Light measurements

54 sensors2-4 hidden variables (~6% error)

measurementreconstruction


Experiments: Light measurements

• 1 & 2: main trend (as before)• 3 & 4: potential anomalies and

outliers

hidden variables

intermittentintermittent


Stream correlations





2. Incremental update

error

20oC 30oC

20oC

30oC

Tem

pera

ture

T2

Temperature T1

• For each new point

• Project onto current line

• Estimate error

New value



error

20oC

30oC

20oC 30oC

Tem

pera

ture

T2

Temperature T1

• For each new point• Project onto

current line• Estimate error• Rotate line in the

direction of the error and in proportion to its magnitude

O(n) time New value



20oC

30oC

20oC 30oC

Tem

pera

ture

T2

Temperature T1

• For each new point• Project onto

current line• Estimate error• Rotate line in the

direction of the error and in proportion to its magnitude


Stream correlationsPrincipal Component Analysis (PCA)

• The “line” is the first principal component (PC) vector

• This line is optimal: it minimizes the sum of squared projection errors


2. Incremental updateGiven number of hidden variables k

• Assuming k is known• We know how to update the slope• (detailed equations in paper)

• For each new point x and for i = 1, …, k :

• yi := wiTx (proj. onto wi)

• di di + yi2 (energy i-th eigenval.)

• ei := x – yiwi (error)

• wi wi + (1/di) yiei (update estimate)

• x x – yiwi (repeat with remainder)

y1

w1

xe1

w1 updated


Stream correlations



• Step 3: How to dynamically adjust k, the number of hidden variables?


T3

3. Number of hidden variables

• If we had three sensors with similar measurements

• Again: points would lie on a line (i.e., one hidden variable, k=1), but in 3-D space

T1

T2

value-tuple space


T3


• Assume one sensor intermittently gets stuck

• Now, no line can give a good approximation

T1

T2

value-tuple space


T3


• Assume one sensor intermittently gets stuck

• Now, no line can give a good approximation

• But a plane will do (two hidden variables, k = 2)

T1

T2

value-tuple space


Number of hidden variables (PCs)

•Keep track of energy maintained by approximation with k variables (PCs):

• Reconstruction accuracy, w.r.t. total squared error

•Increment (or decrement) k if fraction of energy maintained goes below (or above) a threshold

• If below 95%, k k 1

• If above 98%, k k 1

Streaming Pattern Discovery in Multiple Time-Series Jimeng Sun Spiros Papadimitrou Christos...

Documents

Transcript of Streaming Pattern Discovery in Multiple Time-Series Jimeng Sun Spiros Papadimitrou Christos...