Changepoint Models for Event Counts - The University of ... · PDF fileChangepoint Models for...

OutlineIntroduction

Poisson Regression Changepoint ModelsChangepoint Models that Estimate the Numbers of Breaks

Changepoint Models for Event Counts

Patrick T. Brandt

University of Texas, Dallas

July 2010

Patrick T. Brandt Changepoint Models for Event Counts

OutlineIntroduction


1 Introduction

2 Poisson Regression Changepoint ModelsFitting changepoint models with fixed numbers of breaksBrandt and Sandler (2009) (partial) replication

3 Changepoint Models that Estimate the Numbers of Breaks


OutlineIntroduction


Fitting changepoint models with fixed numbers of breaksBrandt and Sandler (2009) (partial) replication

Bayesian Poisson Changepoint Models

Brandt and Sandler (2010) employ this model, developed by Park(2010) and Fruhwirth- Schnatter and Wagner (2006), to analyzechanges in transnational terrorists’ targeting decisions.This model allows the parameters of the Poisson regression tochange over time in m regimes. The changepoints separate out theregimes and are estimated endogenously as part of the model.

No need to pre-specify the (incorrect) breaks

Models nest, so we can compare ones with m to m − 1 andm + 1 regimes.

Can show where the regimes change in a simple time seriesplot.


OutlineIntroduction



Bayesian Poisson Changepoint Regression

Let yt be the number of events at time t with covariates xt andregime-specific regression parameters βst :

yt ∼ Poisson(λt), λt = exp(x ′tβst ) (1)

βst = β1, . . . , βm (2)

s = (s1, s2, . . . , sT ) : st ∈ 1, 2, . . . ,m, t = 1, . . . ,T . (3)

The s indexes the regimes, so we have a vector of values that givethe values of st = i , i = 1, . . . ,m such that if we have m breaks wehave m + 1 parameter regimes.


OutlineIntroduction



Changepoint process

The changepoint process is a restricted Markov process for a givenregime are defined by the inter-arrival times τi to τj for regimesi < j .

The st variables index the regime at time t, such thatPr(τ1|yt) = Pr(st = 2|Y )− Pr(st−1 = 2|Y ), t = 2, . . . ,T .

These probabilities are collected in a transition matrix, P:

P =

p11 p12 0 · · · 00 p22 p23 · · · 0...

.... . .

......

0 0 0 pm−1.m−1 pm−1,m

0 0 0 0 1

, (4)


OutlineIntroduction



Prior and Posterior

The prior is

βst ∼ N(0, 1) (5)

pj,j+1 ∼ Beta(0.25(T )/(j + 1), 0.25) (6)

where pi,i+1 is a Beta prior for the transition from the i th regime.

The posterior is non-standard. It is approximated by re-writing thePoisson process in terms of the inter-arrival times between events, τtj forregime j :

τtj ∼ Exp(λt) =Exp(1)

λt. (7)

where Exp(·) is an exponential distribution.


OutlineIntroduction



Posterior mixture process

These inter-arrival times are functions of the regression function.Substituting the regression function from (1) into (7) and takinglogarithms of (7) yields

log(τtj) = x ′tβst + εtj , εtj ∼ log(Exp(1)). (8)

This is a regression with exponential error terms. Thisnon-standard posterior is approximated using a mixture of fivenormal densities for each regime (Fruhwirth-Schnatter andWagner, 2006)


OutlineIntroduction



Example

ITERATE data on terrorist targets, monthly from 1968–2007

Construct monthly time series for four target groups (“type ofimmediate victim”)

OfficialsMilitaryBusinessPrivate Parties

Covariates

Nature of the victim: number of events involving people,number involving property, number unknown.Logistical success: number completed, aborted or stopped.

Replication code for this example is athttp://www.utdallas.edu/~pxb054000/pbrandt/Replication_files/BrandtSandlerJCR2010.zip


http://www.utdallas.edu/~pxb054000/pbrandt/Replication_files/BrandtSandlerJCR2010.zip

http://www.utdallas.edu/~pxb054000/pbrandt/Replication_files/BrandtSandlerJCR2010.zip

OutlineIntroduction



Identifying the number of changepoints

For each of the data series, we estimated models with one to fourchangepoints using 10000 draws from the posterior. We thencomputed Bayes factors to determine the optimal number ofchangepoints for each series:

ChangepointsSeries Number Dates

Officials 2 1974:10, 1993:3Military 3 1970:2, 1979:5, 1995:11Business 2 1973:3, 1990:4

Private Parties 3 1968:11, 1973:6, 1996:6


OutlineIntroduction



Officials Changepoints

1970 1980 1990 2000

020

4060

Offi

cial

s

1970 1980 1990 2000

0.0

0.4

0.8

Reg

ime

Pro

babi

litie

s

8.2 8.5 2.2


OutlineIntroduction



Military Changepoints

1970 1980 1990 2000

05

1015

Mili

tary

1970 1980 1990 2000

0.0

0.4

0.8

Reg

ime

Pro

babi

litie

s

0.4 1.8 3.5 1.2


OutlineIntroduction



Business Changepoints

1970 1980 1990 2000

020

4060

Bus

ines

s

1970 1980 1990 2000

0.0

0.4

0.8

Reg

ime

Pro

babi

litie

s

4.8 7.2 3.4


OutlineIntroduction



Private Parties Changepoints

1970 1980 1990 2000

020

40

Priv

ate

Par

ties

1970 1980 1990 2000

0.0

0.4

0.8

Reg

ime

Pro

babi

litie

s

1.4 4.9 9.2 5.2


OutlineIntroduction



Changepoint Results

Changepoint vary across target types.

Changepoint locations roughly parallel those hypothesized.

Changepoints depend on the relative costs of and benefits ofvarious targets over time.

See declines in the number of attacks on officials and military,which hardened in the 1970s.

Attacks on business fall by half once state-sponsorshipdeclines in the mid-1990s.

Private parties still face the highest risk: 1.5 to over 4 timesthe rate seen in the other target types.

Find regime specific covariate effects that fit our theory (seethe paper).


OutlineIntroduction



Changepoint model example

Changepoint models can be fit using various R packages. Theseinclude

bcp

SLC

sac

strucchange

MCMCpack

Since I strongly advocate fitting and reporting these models from aBayesian framework, what follows is a simple example usingMCMCpack.


OutlineIntroduction



Simulated data changepoint example

The R code in changepoint-sim.R in thechangepoint-example folder simulates a simple changepointprocess and fits the relevant model using a Bayesian method. Hereis the code for the data generation and plotting:

# Load the MCMCpack library

library(MCMCpack)

# Simulate a time series of counts with two changes in the mean

set.seed(10805)

n <- 50

y <- c(rpois(n, 3), rpois(n, 1), rpois(n, 10))

# Plot the data

par(mfrow=c(1,2))

plot(ts(y), ylab="Counts")

abline(v=50)

abline(v=100)

plot(cumsum(ts(y)), ylab="Cumulative Counts", xlab="Time", type="l")

abline(v=50)

abline(v=100)


OutlineIntroduction



Simulated data

Time

Cou

nts

0 50 100 150

05

1015

20

0 50 100 1500

200

400

600

Time

Cum

ulat

ive

Cou

nts


OutlineIntroduction



Estimation of the Bayesian changepoint models

# Estimate the models via MCMC sampling

m1 <- MCMCpoissonChange(y ~1, m=1, c0=1, d0=1, marginal.likelihood="Chib95")



#Summarize the means for each changepoint

summary(m1)

summary(m2)

summary(m3)

# Compute the Bayes factors for the models

print(BayesFactor(m1, m2, m3))


OutlineIntroduction



Selection of the models via Bayes factors

> print(BayesFactor(m1, m2, m3))The matrix of Bayes Factors is:

m1 m2 m3m1 1 0.000401 0.00059m2 2493 1.000000 1.47020m3 1696 0.680181 1.00000

The matrix of the natural log Bayes Factors is:m1 m2 m3

m1 0.00 -7.821 -7.436m2 7.82 0.000 0.385m3 7.44 -0.385 0.000


OutlineIntroduction



Plotting the changepoints

plotState(m2)

Posterior Regime Probability

Time

Pr(

St=

k |Y

t)

0 50 100 150

0.0

0.2

0.4

0.6

0.8

1.0

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●

●

●

●

●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●


OutlineIntroduction



Brandt and Sandler (2009)

Here we’ll go back and do a simplified version of the changepointanalysis in Brandt and Sandler (2009). We’ll look at the skyjackingdata and find the number of changepoints. In the original, we fitthe model described in the next section.

The code for this analysis is in the file changepoint-skyjacks.R.


OutlineIntroduction



Code for the changepoint models

# Read in the data

load("hostages-monthly.RData")

# Get the monthly skyjacks variable

skyjacks <- sm[,"incident"]

# Load the MCMCpack package and the zoo package

library(MCMCpack)

# Plot the data

plot(skyjacks)

# Fit the changepoint models to the data

set.seed(1986)

m1 <- MCMCpoissonChange(skyjacks ~ 1, m=1, c0=1, d0=1, marginal.likelihood="Chib95")






# Check the convergence of the samples

plot(m1); plot(m2); plot(m3)

plot(m4); plot(m5); plot(m6)

# Compute the Bayes factors

BayesFactor(m1,m2,m3,m4,m5,m6)


OutlineIntroduction



Bayes Factor output for model selection

> BayesFactor(m1,m2,m3,m4,m5,m6)

The matrix of Bayes Factors is:

m1 m2 m3 m4 m5 m6

m1 1.00e+00 2.45e-01 7.91e-08 1.79e-09 6.20e-14 3.41e-13

m2 4.09e+00 1.00e+00 3.23e-07 7.30e-09 2.53e-13 1.39e-12

m3 1.26e+07 3.10e+06 1.00e+00 2.26e-02 7.85e-07 4.31e-06

m4 5.60e+08 1.37e+08 4.42e+01 1.00e+00 3.47e-05 1.91e-04

m5 1.61e+13 3.95e+12 1.27e+06 2.88e+04 1.00e+00 5.49e+00

m6 2.93e+12 7.18e+11 2.32e+05 5.24e+03 1.82e-01 1.00e+00

The matrix of the natural log Bayes Factors is:

m1 m2 m3 m4 m5 m6

m1 0.00 -1.41 -16.35 -20.14 -30.41 -28.71

m2 1.41 0.00 -14.95 -18.74 -29.00 -27.30

m3 16.35 14.95 0.00 -3.79 -14.06 -12.35

m4 20.14 18.74 3.79 0.00 -10.27 -8.56

m5 30.41 29.00 14.06 10.27 0.00 1.70

m6 28.71 27.30 12.35 8.56 -1.70 0.00


OutlineIntroduction



Code for plotting and summarizing results

The R script contains a pair of functions to sumamrize and plotthe changepoint results: plotChange and plotChange2.

# Plot the results and get the dates of the changepointsplotChange(m5, name="Skyjacks", start=c(1968,1), freq=12)

plotChange2(m5, name="Skyjacks", start=c(1968,1), freq=12)


OutlineIntroduction



Changepoint plotS

kyja

cks

1970 1980 1990 2000

02

46

8

Reg

ime

Pro

babi

litie

s

1970 1980 1990 2000

0.0

0.4

0.8

0.7 2.6 0.6 4.1 0.8 0.1


OutlineIntroduction



Dates of the changepoints

plotChange2(m5, name="Skyjacks", start=c(1968,1), freq=12)@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@Expected changepoint(s):[1] 1979.75[1] 1981.75[1] 1990.083[1] 1990.667[1] 2003@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@


OutlineIntroduction



Conclusions on skyjacks data

We see that there have been shifts in the number of eventsover time.

Breaks correspond to major shifts in 1) airport security, 2)punishments for skyjacking, 3) shifts in international relations.

General downward trend is evident: skyjacking events arepunctuated by short periods of new skyjackins in the early1980s and early 1990s.


OutlineIntroduction


Decisions about the number of changepoints

The previous, Bayesian Poisson regression changepoint model hasone drawback: it treats the number of breaks as a fixed numberand you have to use a hypothesis test to find the number of breaks.

This is problematic if you have a long series or one where there aresome known breaks, but you want to find others.

Also, from a Bayesian perspective, the number of changepointsand their locations should be treated as parameters and estimated.

But this is a hard problem, since altering the number ofchangepoints in the model (and their locations) is a non-nestedcomparison.


OutlineIntroduction


Example: Simulated data series

Time

Cou

nts

0 50 100 150

05

1015

20

0 50 100 1500

200

400

600

Time

Cum

ulat

ive

Cou

nts


OutlineIntroduction


Changepoint estimation by RJMCMC

One way to address this problem is to use what is called areversible-jump Markov Chain Monte Carlo (RJMCMC) estimationmethod.The RJMCMC method was developed by Green (1995) to modelthe change in the arrival rates of new events.These methods allow one to sample whether the data are bettercharacterized by a model with either k − 1, k, or k + 1changepoints.The basic idea is that we want to fit a step function to thecumulative number of events where we use the fewest number ofsteps (breaks).


OutlineIntroduction


Poisson RJMCMC Changepoint Model

Let yi be the number of events per unit i whereyi = 1, 2, . . . , n ∈ [0, L]. The daily arrival rate for the Poissonprocess for the events varies over time according to the arrival ratestep function w(t).The likelihood for these Poisson events is

n∑i=1

log{w(yi )} −∫ L

0w(t)dt

where n is the number of events and L is the total time units ofthe events.


OutlineIntroduction


Changepoint process

The multiple changepoint part of the model is in the specification of thestep function that describes the arrival rate of the events, w(t):

Suppose there are k steps at intervals 0 < s1 < s2 < . . . < sk < Land the steps take a value or height of hj between [sj , sj+1] forj = 0, 1, 2 . . . , k .

The number of possible steps, k is assumed to be Poissondistributed with mean λ with k ≤ kmax and kmax is the arbitrarymaximum number of breaks.

Under these assumptions, the steps are even-numbered orderstatistics from 2k + 1 points over a uniform interval [0, L]. Theheights of the steps, h0, h1, . . . , hk (which describe the density ofchangepoints) are independent draws from a Γ(α, β) density.


OutlineIntroduction


The “reversible” part

We want to be able to compare the (posterior) probability ofadding or subtracting from k , but this changes the dimension ofthe number of changepoints in w(·). This happens as follows

k to k − 1 (a death step),

k to k + 1 (a birth step),

changing the step height (intensity step)

changing the position of a step (location step).

The changepoint models for k versus k ± 1 are nonnested.


OutlineIntroduction


Brandt and Sandler (2009)

Brandt and Sandler (2009) use these Bayesian RJMCMC methodsto estimate and find the number of changepoints in the earlierhostage taking transnational terrorism series.The code for doing this is computationally intensive and written inFortran (by Peter Green). We will not replicate the examples here,but the code if available for this example by request (it is complex).


OutlineIntroduction


Example

Brandt and Sandler (2009) employ both PAR(p) and RJMCMCchangepoint models to investigate the dynamics of transnationalhostage taking 1968-2005. We look at three time series of hostagetaking events:

1 Kidnappings

2 Skyjackings

3 Other hostage events (e.g., barricade and hostage events)


OutlineIntroduction


Hostage data series0

24

68

Kid

napp

ings

1970 19900

12

34

Sky

jack

ings

1970 1990

02

46

8

Non

−K

idna

ppin

gs

1970 1990

040

080

012

00

Cum

ulat

ive

Kid

napp

ings

1970 1990

010

020

030

0

Cum

ulat

ive

Sky

jack

ings

1970 19900

5010

020

0

Cum

ulat

ive

Non

−K

idna

ppin

gs

1970 1990


OutlineIntroduction


RJMCMC changepoint models

Fit models from 1 to kmax = 50 breaks.

Sampled 20.4 million draws from the posterior distribution.Discarded the first 400,000 draws and summarized the thinneddraws of the last 20 million for a sample of 1 million.

This takes about 13 minutes to simulate two samples of 20.4million draws – for one series.

This analysis took about 50 minutes of computational time.

Results include an estimate of the probability of each numberof breaks or changepoints. We report the number of breakswith the highest posterior probability.


OutlineIntroduction


Kidnap Results

Date

Cum

ulat

ive

num

ber

of k

idna

p ev

ents

020

040

060

080

010

0012

00

Pos

terio

r m

ean

arriv

al r

ate

of k

idna

p ev

ents

0.00

0.05

0.10

0.15

0.20

0.25

1970 1980 1990 2000

1 2 3 4 5 6 7 8 9 10

1 Rise of transnational terrorism2 Small decline in kidnappings3 Lebanon MNF, rise in Middle East kidnappings4 Downturn in Middle East kidnappings5 Algerian/Turkish kidnappings

6 Drop in transnational terrorism7 African / Latin American kidnappings8 Pre 9/11 drop9 Abu Ghraib revelations10 Reduction in Iraqi kidnappings


OutlineIntroduction


Skyjack Results

Date

Cum

ulat

ive

num

ber

of k

idna

p ev

ents

010

020

030

0

Pos

terio

r m

ean

arriv

al r

ate

of k

idna

p ev

ents

0.02

50.

050

0.07

5

1970 1980 1990 2000

1 2 3 4 5 6 7 8

1 PFLP skyjackings demonstration effect2 Metal detectors3 Cuban skyjackings4 Castro 40−yr sentences

5 Soviet skyjackings6 End of Cold War7 Low terrorism year8 Increased airport security


OutlineIntroduction


Non-kidnap Results

Date

Cum

ulat

ive

num

ber

of k

idna

p ev

ents

010

020

0

Pos

terio

r m

ean

arriv

al r

ate

of k

idna

p ev

ents

0.00

50.

010

0.01

50.

020

0.02

50.

030

1970 1975 1980 1985 1990 1995 2000 2005

1 2 3 4

1 Post−Israeli occupation2 Substitution into other hostage events

3 Embassy fortification4 Post−Cold War reduction


OutlineIntroduction


Conclusion

Fitting these kinds of models makes sense in the followingcircumstances:

There is a theoretical reason to positive a change in the meannumber of counts.

The breaks or changes better characterize the process thanany cycles or ARIMA-like process.

One is engaging in an exploratory analysis that needs tocharacterize breaks before cycles.


Changepoint Models for Event Counts - The University of ... · PDF fileChangepoint Models for...

Documents

Transcript of Changepoint Models for Event Counts - The University of ... · PDF fileChangepoint Models for...