Detecting changes in streaming data using adaptive...

DetectingchangesinstreamingdatausingadaptiveestimationDeanBodenham,ETHZürich(jointworkwithNiallAdams)

Loading [MathJax]/jax/output/HTML-CSS/jax.js

OutlineChangedetectioninstreamingdataApplicationsAdaptiveestimationusingforgettingfactorsExtensions

Streamingdata

DatastreamsAdatastreamisasequenceofrandomobservations

observedsequentiallypotentiallyunendingtoomanyobservationstostoreandanalyselater

observationsarriveatahighfrequencyunderlyingdistributionsareoftenunknownunpredictablewhenandhowchangesoccur

, ,… , ,…x1 x2 xN

Changedetection

Changedetectioninstreamingdata

Changedetectioninstreamingdatasequentiallydetect(significant)changesinthestreamdetectthefirstchangepointandthencontinuouslymonitorforfurtherchangepoints

useasfewcontrolparametersaspossible

Changedetectionusingacontrolchart

ChangedetectionusingacontrolchartAcontrolchartconsistsof:statisticscontrollimits ,whereisachangepointif:

for

Wesay:in-controlout-of-control

, ,…z1 z2a, b a < b

τ∈ (a, b)zi i = 1,2,… , τ

∉ (a, b)zτ+1

∈ (a, b) ⇒zi

∉ (a, b) ⇒zi

Changedetectionusingacontrolchart

ChangedetectioninstreamingdataWeassume:observationsrealisationsofrandomvariablesdistributions

Wesaythat arethechangepointsofthedatastream.

, ,…x1 x2, ,…X1 X2

≠ ≠ …D1 D2

, ,… ,X1 X2 Xτ1

, ,… ,X +1τ1 X +2τ1 Xτ2

, ,… ,X +1τ2 X +2τ2 Xτ3

∼ ,D1

∼ ,D2

∼ ,etcD3

, ,…τ1 τ2

Changedetection:examples

Changedetection:literatureOriginalmethod:Shewhartcontrolcharts: ,forparameter

Twopopularmethods:CUSUM:CumulativeSumEWMA:ExponentiallyWeightedMovingAverageBothneedtwocontrolparametersIfthedistributionsareknown,CUSUMhasoptimalityproperties

ParameterchoicesdependonmagnitudeofthechangeModernmethodsusuallyrequiretwoormorecontrolparametersManymethodsonlyfocusonasinglechange

( + Ks, − Ks)x̄ x̄ K

Changedetectioninstreamingdata

sequentiallydetectmultiplechangepointsinastream

DifferencetoofflinemethodsTraditionalmethods Streamingdatamethods

fixedamountofobservations unending

accesstoallofthedataallthetime onepass

(often)prechangedistribution isknown unknown

(often)onlymonitoringforasinglechangepoint multiplechangepoints

Lastpointrelatedtoqualitycontrol:canwestoptheprocess?Streamingdataisoftenjustobserved,cannotbestopped

F1

ContinuousmonitoringMonitorapotentiallyunendingstreamExpectmutiplechangepointsAfterfirstchangepoint,monitorfornextchangepointsUseasfewcontrolparametersaspossible

ApplicationsCybersecurityFinanceBiologyandmedicine

Adaptiveestimationwithforgettingfactors

SamplemeanStream (mostrecent: )

Allobservationshaveequalweight

, ,… ,x1 x2 xN xN

x̄N

x̄N

= [ + + ⋯ + + ]1N

xN xN−1 x2 x1

= ,1N

∑i=1

N

xi

Samplemean:sequentialFor

Settingfor‘mass’and for‘weight’

i = 1,2,…mi

wi

x̄i

= +mi−1 xi

= + 1wi−1

=mi

wi= = 0m0 w0

m w

ForgettingfactormeanBuildingonworkofChristoforosAnagnostopoulosandNiallAdams,

Stream: (mostrecent: )Usingaforgettingfactor

Earlierobservationsdownweighted

, ,… ,x1 x2 xN xN

λ ∈ [0, 1]

x̄N,λ

x̄N,λ

= [ + λ + … + ]1

wN,λxN xN−1 λN−2x2 λN−1x1

= , =1

wN,λ∑i=1

N

λN−ixi wN,λ ∑i=1

N

λN−i

Forgettingfactormean:sequentialFor

Setting

i = 1,2,…mi,λ

wi,λ

x̄i,λ

= λ +mi−1,λ xi

= λ + 1wi−1,λ

=mi,λ

wi,λ= = 0m0,λ w0,λ

Aside:Twospecialcases

But::onlylatestobservation:samplemean

Usuallyonlyconsider

=x̄N,λ1

wN,λ∑i=1

N

λN−ixi

λ ∈ [0, 1]

λ = 0λ = 1

λ ∈ (0, 1)

Asimulationstudy

Changefrom toN(0, 1) N(3, 1)

Asimulationstudy

Averagebehaviourover100trials

Valueof ?Problem:Howdowechoose ?

Onesolution:Adaptiveforgettingfactor

λλ

λ→

Adaptiveforgettingfactors

AdaptiveforgettingfactormeanGivenasequenceofobservations ,andforgettingfactors ,

Theadaptiveforgettingfactormean isdefined,for ,by

, ,… ,x1 x2 xN

, ,… ,λ1 λ2 λN−1m

N, λ→

wN, λ

→

= + ,λN−1mN−1, λ

→ xN

= + 1,λN−1wN−1, λ

→

= 0,m0, λ

→

= 0,w0, λ

→

x̄N, λ

→ N ≥ 1

=x̄N, λ

→

mN, λ

→

wN, λ

→

UpdatingUsegradientdescent:

→ ?λN λN+1

= − η , η ≪ 1λN λN−1∂

∂ λ→ L

N, λ→

DerivativeofForanyfunction involving :

λ→

fN, λ

→ λ→

∂

∂ λ→ f

N, λ→ = [ − ]lim

ϵ→0

1ϵ

fN, +ϵλ

→ fN, λ

→

Example:m

N, λ→

mN, λ

→

mN, +ϵλ

→

= +λN−1mN−1, λ

→ xN

= [( ) ]∑i=1

N ∏p=i

N−1

λp xi

= [( ( + ϵ)) ]∑i=1

N ∏p=i

N−1

λp xi

(1)

(2)

(3)

Example:Δ

N, λ→

ΔN, λ

→

ΔN, λ

→

= = [ − ]∂

∂ λ→ m

N, λ→ lim

ϵ→0

1ϵ

mN, +ϵλ

→ mN, λ

→

= [ ( ) ]∑i=1

N−1 ∑t=i

N−1 ∏p=ip≠t

N−1

λp xi

= + , = 0λN−1ΔN−1, λ

→ mN−1, λ

→ Δ1, λ

→

(4)

(5)

(6)

ChoiceofcostfunctionThefollowingcostfunctionisused:

Inthegradientdescentstep:

LN, λ

→ = [ −x̄N−1, λ

→ xN ]2

λN = − ηλN−1∂

∂ λ→ L

N, λ→

Behaviourofadaptiveforgetting

Asimulationstudy


Asimulationstudy:withAFF


Decidingwhenachangehasoccurred

Expectationandvarianceof

Assume:datastreamissampledfromthei.i.d. ,with

Then:

where and,for ,

X̄N, λ

→

, ,… ,X1 X2 XN

E[ ] = μXi

Var[ ] =Xi σ2

E[ ]X̄N, λ

→

Var[ ]X̄N, λ

→

= μ,

= ( ) ,uN, λ

→ σ2

= 1u1, λ

→ i ≥ 1

= (1 − + ( .ui+1, λ

→1

wi+1, λ

→)2

ui, λ

→1

wi+1, λ

→)2

AdecisionruleAssumingnormalityUsenormalcdftofindcontrollimits(a,b)

Then:nochange

change

Pr[ ≤ a] = α/2, Pr[ ≥ b] = 1 − α/2,X̄N,λ X̄N,λ

∈ (a, b) ⇒X̄N,λ∉ (a, b) ⇒X̄N,λ

ChangedetectionwithAFFmethodAdaptiveforgettingfactor(AFF)methodonecontrolparametercomputedsequentiallyfixedmemoryandcomputationalrequirements(fornow)assumesnormalityfordecisionrule

α

Simulationstudy

PerformancemeasuresStandard:ARL1:Howquicklyisatruechangeisdetected?ARL0:Howlonguntilafalsechangeisdetected?

Forcontinuousmonitoring:DNF:ProportionofdetectionsnotfalseCCD:Proportionofdetectionscorrectlydetected

Simulationstudy

Algo Parameters Values CCD DNF ARL1 ARL0

CUSUM ( , ) (0.50,4.77) 0.85 0.77 24.73 373.38

EWMA ( , ) (0.10,2.814) 0.87 0.78 24.17 420.15

AFF 0.005 0.86 0.79 27.12 819.36

Performanceonastreamofover750,000observationswithapproximately changesofsize ,with

CCD:higherisbetterDNF:higherisbetterARL0:higherisbetterARL1:lowerisbetter

k h

r L

α

5000 δδ ∈ {0.25, 0.5, 1, 3}

Application

Application:ForeignexchangedataGBP/CHFapprox.4yearsofmonitoringevery5minutes330,000observations

Wherearethetruechangepoints?noideacomparedetectedchangepointswithoptimalofflinedetector

ComparingAFFandofflinedetector

Extensions

NaturalextensionstoothersettingsMultivariatecasemonitoreachstream,combinep-values

Changeinthevariance

ExtremevaluedistributionsNonparametricextensions

=s2N,λ

1vN,λ

∑i=1

N

λN−i [ − ]xi x̄N,λ2 (7)

RpackageffstreamavailableonCRAN

https://cran.r-project.org/package=ffstream

Thankyou!

Slidespreparedwith:

rmarkdownreveal.jspdf2svg

References

—andN.M.Adams(2016)Continuousmonitoringforchangepointsindatastreamsusingadaptiveestimation,StatisticsandComputing,—andN.M.Adams(2013)ContinuousmonitoringofacomputernetworkusingmultivariateadaptiveestimationICDMWorkshopsC.AnagnostopoulosC(2010)Astatisticalframeworkforstreamingdataanalysis.PhDthesis,ImperialCollegeLondonS.Haykin(2002)AdaptiveFilterTheory,Prentice-HallR.Killick,P.FearnheadandI.A.Eckley(2012)Optimaldetectionofchangepointswithalinearcomputationalcost,JASAE.Page(1954)Continuousinspectionschemes,BiometrikaS.W.Roberts(1959)Controlcharttestsbasedongeometricmovingaverages,Technometrics

doi:10.1007/s11222-016-9684-8

doi:10.1007/s11222-016-9684-8

Behaviourofadaptiveforgetting

PoemRosesarered,violetsareblue.

Othercolours

chosencolourisdodgerblue,chosencolourisdodgerblue,

Analignλ =

1ϵ + δ

=1ξ

(8)

Anequationwithnumbersλ =

19

(9)

Detecting changes in streaming data using adaptive...

Documents

Transcript of Detecting changes in streaming data using adaptive...