Causality Detection in Time SeriesTushar MehndirattaIDD CSE ( V year)10211026

Overview

Introduction

Detecting Causality Control Experimentation

Granger Causality

Building Causal Relationship Graphs Exhaustive Granger Method

Lasso Granger Method

Forward backward Granger Method

Application of Causal Modelling Brain Imaging

Topic Mining

Anomaly Detection

Conclusion

Why?

Why did the apple fall down instead of going

up?

Why does average temperature rise?

Why did the stock market fall?

Why did a post go viral on facebook?

CAUSE EFFECT RELATIONSHIPS

Cause - Effect Relationships◦ Causality is defined as the relation between two events: cause and effect where

the effect occurs as a consequence of the cause.

◦ Effect is “What happened?” and Cause is “Why it happened?”

◦ e.g. In case of global warming, the increase in Greenhouse gases is the cause and increase in average temperature is the effect.[1]

Characteristics of Causal Relationships◦ Temporal Precedence: It states that the cause occurs prior to the effect. e.g. A person

must smoke first and then he gets lung cancer.

◦ Co-occurrence : Whenever cause happens, effect must also happen. Cause cannot be

isolated from the effect. e.g. Whenever there is a net force on a body, it will accelerate.

Is Causality same as Association then?

Correlation Vs Causation◦ Correlation does not imply Causation

◦ Correlation only means that two events co-exist more often than ordinary chance.[2]

Physics

EconometricsTypes of Data: web metrics , stock prices, sales

(all time series)

MedicineTypes of Data: experiments result, gene

sequences(sequential data), brain signals(time series)

Climate ScienceTypes of Data: weather conditions (spatio-

temporal or temporal data)

Fields of Study

HOW TO DETECT CAUSALITY?

Detecting CausalityTo test if X causes Y

Control ExperimentationAim: To find out what happens to a system when you interfere with it.

Divide subjects randomly into two groups: Test and

Control

Introduce X only in the test group and observe Y in both.

If X causes Y : ((ܺ)݀|ݕ=ܻ)ܲ >

((ܺ)݀!|ݕ=ܻ)ܲimplies Causality

Disadvantage of Control Experimentation◦ Not possible to always carry out the experiment.

◦ Most time series data cannot be manipulated. e.g. Climate, Stock data

◦ Have to resort to statistical methods to determine causality.

HOW TO DO IT IN TIME SERIES?

Time Series◦ A time series is a sequence of data points, measured typically at successive

points in time spaced at uniform time intervals.

Granger Causality ◦ Also known as Predictive Causality.

◦ Granger said that Causality could be reflected by measuring the ability of predicting the future values of a time series using past values of another time series.

◦ Two main principles:

Cause must occur before the Effect.

The Cause can be used to predict the of Effect i.e. Cause has some unique information

about the future values of the effect.

Granger Causality

𝑃[𝑌(𝑡 + 1)|𝛤 𝑡 ≠ 𝑃[𝑌(𝑡 + 1)| 𝛤−X 𝑡

𝛤 𝑡 and 𝛤−X 𝑡 denote the “information in the universe up to time t” and “information in alternate universe up to time t in which X is excluded”.

Suppose X and Y are two time series and for X to cause Y :

Performing the Granger Causality test◦ Model 1: Build model 1 by regressing on the past values of both X and Y

𝐸(𝑌|𝑌𝑡−𝑘 , 𝑋𝑡−𝑘) 𝑌𝑡 = 𝑗=1𝑚 𝛼𝑗𝑌𝑡−𝑗 + 𝑖=1

𝑛 𝛽𝑖 𝑋𝑡−𝑖 + 𝐷𝑡 + 𝜀𝑡

◦ Model 2: Build model 2 by regressing on the past values of Y only

𝐸(𝑌|𝑌𝑡−𝑘) 𝑌𝑡 = 𝑗=1𝑚 𝛼𝑗𝑌𝑡−𝑗 + 𝐷𝑡 + 𝜀𝑡

◦ Check whether the prediction accuracy has significantly increased by performing F-test.[11]

Granger Causality• CONS

It does not take into account the effect of hidden common

causes(confounders)

It assumes that all the relationships are linear in nature and does not account

for non-linear dependencies.

HOW TO DEAL WITH MULTIPLE TIME SERIES?

Relationship Graphs in Time SeriesExtending the concept of Granger Causality to Multiple Time Series

Relationship Graphs◦ Relationship graph has all time series as nodes and an edge between any two

nodes denotes the direction of relationship between the two.

◦ Input: Matrix X of time series

Xlag which is the lagged versions of time series matrix X.

◦ Output◦ Relationship graph between the time series with nodes xi’s each edge from xi to xj if xi

causes xj.

xi xj

Exhaustive Graphical Granger method ◦ Algorithm:

◦ For every pair of nodes(xi,xj) perform the following

Insert an edge xi → xj if Granger (xi,xj, Xlag) = ‘yes’ and Granger (xj,xi, Xlag) = ‘no’

Insert an edge xi ← xj if Granger (xi,xj, Xlag)= ‘no’ and Granger (xj,xi, Xlag) = ‘yes’

Insert an edge xi↔xj, if Granger (xi,xj, Xlag) = ‘yes’ and Granger (xj,xi, X

lag)= ‘yes’

Exhaustive Graphical Granger method ◦ Complexity

A total of N time series with T lags each and P time stamps/sample size, makes the

complexity as O(N2P2T2).

◦ Shortcomings

Not considering the effect of other time series.

Computationally expensive.

The LASSO-Granger Method

LASSO-Least Absolute Shrinkage and Selection Operator

◦ Uses variable selection in Causality Detection

◦ Aim is to identify the subset of time series on which xi is conditionally dependent

and on what lag is it dependent.

◦ Achieved by applying variable selection on the set of time series and the lags

◦ Variable selection is done by LASSO.

LASSO◦ A selection method for linear regression

◦ Selects a subset of variables subject to the following condition

𝑤 = 𝑚𝑖𝑛1

n (𝑤. 𝑥 − 𝑦)2+𝜆 𝑤

Here w is the vector of coefficients, y is the variable to be predicted.

◦ Aim is to minimize the OLS error and the sum of coefficients to prevent over

fitting.

◦ LARS(Least Angle Regression): best method to achieve LASSO.

LARS(Least Angle Regression Shrinkage)

Step 1: Start with û0=0

Step 2: The residual ŷ2-û0 has a greater correlation with x1 than with x2

LARS(Least Angle Regression Shrinkage)

Step 3: Move in the direction of x1

LARS(Least Angle Regression Shrinkage)

Step 4: First LARS estimate : û1 = û0 + ƛx1

where the residual ŷ2-û1

has equal correlation with both x1 and x2

LARS(Least Angle Regression Shrinkage)

Step 5: Move in the direction of Angular bisector of x1 and x2

The Lasso-Granger Method◦ Algorithm

Obtain Xlag(the lagged version of the time series matrix X).

For each xi in X,

y= xi

Performs LASSO (y,Xlag)

Wi : the set of time series for which the coefficients returned by are non-zero.

Add edge (xj, xi) to the graph if xj is in Wj

The Lasso-Granger Method◦ Complexity

Using LARS to solve the lasso problem: O(PN2T2).

◦ Pros.

Computationally less expensive.

Can be used when number of series are quite large as compared to the number of data

points.

Consistency: The probability of Lasso falsely including a non-neighboring feature in its

neighborhood is very small even when the number of features are very large.

Forward Backward Granger Causality◦ Improvement on LASSO-Granger Algorithm

◦ Inspired from Physics

◦ Principle: Reverse time and all the relationships must remain same except for

change in direction, i.e. if xi causes xj with a time lag of k then on reversing time xj

will cause xi with time lag k.

◦ Apply LASSO-Granger on both the forward and backward time series and

combine the results of the two.

Application of Causal ModellingBRAIN IMAGING TOPIC MINING ANOMALY DETECTIO N

Brain Imaging◦ How different portions of the brain affect one another.

Identify the direction and order of influence

◦ Apply Granger Causality to obtain the relationship between different components of

the brain.

Obtain fMRI data from the brain corresponding to a stimulus and divided it into

independent components corresponding to different sections of the brain.

Each independent component corresponds to a time series.

Apply Exhaustive Granger test to obtain the relationship between different time

series.

Brain Imaging◦ Advantages:

No prior assumption about the nodes and their inter-connections.

Measures not only the connections but also the time lags between interactions.

Can work with a large number of regions.

Mining topics based on Causality◦ Identification of topics that are causally related with the non textual data

iteratively.

◦ InCaToMi (Integrative Causal Topic Miner)

◦ Architecture:

Topic modelling module

Causality Module

Feedback

Text Data

InCaToMi: Integrative Causal Topic Miner◦ Topic Modelling Module:

Takes text and number of topics as input.

Creates topics based on word probabilities and the likelihood of each topic in the

document using PLSA algorithm.

Time series of the topic formed by summation of likelihood of each word in the topic for

a day.

InCaToMi: Integrative Causal Topic Miner◦ Causality Module:

Perform the Granger Causality test for the time series for each topic and for each word in

the topic.

Form new candidate topic by selecting the words which are most causally related with the

non textual series.

Use this as prior for the next round of Topic Modelling.

Anomaly Detection◦ Types of Anomalies:

Univariate Anomalies

Dependency Anomalies

◦ Given two sets of data sequences A(training) and B(test) each containing p time

series we have to find data points in B which significantly deviate from the

normal pattern of data sequence.

◦ Algorithm for finding dependency anomalies.

Anomaly Detection

Learning temporal causal graphs by regularization

Finding the Anomaly Score using Kullback-

Leibler (KL) Divergence

Determining Anomalies by

specifying a threshold and finding the

underlying causes

Hypothesis: Causal Graphs of both remain the same

Anomaly Detection

Learning temporal causal graphs by regularization

Finding the Anomaly Score using Kullback-

Leibler (KL) Divergence

Determining Anomalies by

specifying a threshold and finding the

underlying causes

Calculate the graph for A by LASSO Granger method.

When finding the causal graph for B we need to apply additional constraints. This can be done using two methods:a) Neighborhood Similarity: This implies imposing an additional constraint that the values of β(a) should be zero or non-zero only when the value of β(b) are zero or non-zero. Here β(a) and β(b) are the coefficients obtained by running Lasso Granger on set A and Set B respectively.b) Coefficient similarity: The constraint is that the coefficients β(a) and β(b) should be similar.

Anomaly Detection

Learning temporal causal graphs by regularization

Finding the Anomaly Score using Kullback-

Leibler (KL) Divergence

Determining Anomalies by

specifying a threshold and finding the

underlying causes

KL divergence is a measure of how much one distribution differs from another.

Obtain the distributions for the two time series and the anomaly score is calculated using the KL formulae.

Anomaly Detection

Learning temporal causal graphs by regularization

Finding the Anomaly Score using Kullback-

Leibler (KL) Divergence

Determining Anomalies by

specifying a threshold and finding the

underlying causes

• To set a threshold we calculate how a normal time series would score on the anomaly score.

• We slide the window through the reference data and calculate the anomaly scores for each window.

• We them use these to approximate the distribution of anomaly scores that a normal time series should have.

• Given a significance level α, we set the α quantile of the distribution as threshold cutoff.

Conclusion◦ Widespread application of causal relationships motivates the study.

◦ Completely data driven approach. So provides a new outlook in every field

without making any assumptions.

◦ Further Scope:

Applying the model to different domains. e.g Climate and Social media

Predicting anomalous behavior.

References[1] Lashof, Daniel A., and Dilip R. Ahuja. "Relative contributions of greenhouse gas emissions to global warming." (1990): 529-531.

[2] Perry, Ronen. "Correlation versus Causality: Further Thoughts on the Law Review/Law School Liaison." Conn. L. Rev. 39 (2006): 77.

[3] Diks, Cees, and Valentyn Panchenko. Modified hiemstra-jones test for Granger non-causality. No. 192. Society for Computational Economics, 2004.

[4] Granger, Clive WJ. "Investigating causal relations by econometric models and cross-spectral methods." Econometrica: Journal of the Econometric Society (1969): 424-438.

[5] Arnold, Andrew, Yan Liu, and Naoki Abe. "Temporal causal modeling with graphical granger methods." Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2007

[6] Tibshirani, Robert. "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society. Series B (Methodological) (1996): 267-288.

[7] Cheng, Dehua, Mohammad Taha Bahadori, and Yan Liu. "FBLG: a simple and effective approach for temporal dependence discovery from time series data."Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014.

[8] Smith, Delmas, Iwabuchi, Kirk. “Demonstrating causal links between fMRI time series using time-lagged correlation”.

[9] Kim, Hyun Duk, et al. "Incatomi: Integrative causal topic miner between textual and non-textual time series data." Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 2012.

[10] Qiu, Liu, Subrahmanya, et al. "Granger Causality for Time-Series Anomaly Detection." Proceedings of the 12th IEEE international conference on data mining, 2012.

[11] Lomax, Richard G. (2007) Statistical Concepts: A Second Course, p. 10

Thanks!! Any Questions?