FastDTW: Toward Accurate Dynamic Time Warping in

Linear Time and Space

Department of Computer SciencesFlorida Institute of Technology

Stan Salvador and Philip Chan

Outline

• Dynamic Time Warping (DTW)

• Problem Statement

• Related Work for Speeding up DTW

• FastDTW Algorithm

• Evaluation of FastDTW

• Contributions

• Limitations and Future Work

Dynamic Time Warping (DTW)

• Aligns two time series by warping the time dimension

• Warping - expanding/contracting the time dimension

Time

The Dynamic Time Warping Algorithm

• A dynamic programming approach

• Solutions to slightly smaller problems used to find larger solutions

The DTW Cost Matrix

1

1

|X|

|Y|

i

j

Time Series X

Tim

e S

eri

es

Y

Tim

e

Time Series X

Time

Distance of Min-Cost Warp Path

1

1

|Y|

i

j

Time Series X

Tim

e S

eri

es

Y

Tim

e

Time Series X

Time

|X|

Finding Min-Cost Warp Path

)]1,1(),1,(),,1(min[ jiDjiDjiD

1

1

|X|

|Y|

i

j

Time Series X

Tim

e S

eri

es

Y

Tim

e

Time Series X

Time

Advantages of DTW

• DTW is optimal

• An intuitive distance measurement

• Local variation in the time axis is common– Handwriting– Speech– “Events” that start after varying delays

Disadvantages of DTW

• O(N2) time and space complexity

• Only practical for small data sets (<3,000)

• Time series are often very long

• Data mining requires a scalable DTW algorithm

Problem Statement

• We desire an efficient Dynamic Time Warping algorithm– Linear time complexity– Linear space complexity– Warp path is needed in addition to warp

distance– Warp path must be nearly optimal

Does DTW Need to be Faster?

“Myth 3: There is a need (and room) for improvements in the speed of DTW for data

mining applications.”

(Keogh today-9:45am)

• Keogh: many time series

• FastDTW: Long time series

Existing Methods to Speed Up DTW

1. Constraints – only fill in part of the cost matrix

2. Abstraction – sample the data before time warping

Constraints

• Still O(N2) if the window width is a function of input size (linear if the width is constant)

• Assumes a near-optimal warp path stays near the i=j axis

• Accuracy depends on the domain

Sakoe-Chiba Band (Sakoe & Chiba 1978)

Itakura Parallelogram (Itakura 1975)

Abstraction1/51/1 1/1

• O(N) if N pts are sampled down to ≤

• Assumptions– Sampling preserves time series structure– Small deviations from the optimal path cause

little increase in warp-path distance

(Keogh & Pazzani 2000), (Chu et al. 2002)

N

Our FastDTW Algorithm

• A multi-resolution approach inspired by a multi-level graph bisection algorithm (Karypis 1997)

• 3 key operations1. Coarsening – reduce

the resolution of a time series

2. Projection – use a low-res warp path as an initial solution at a higher resolution

3. Refinement – Refine a projected warp path locally adjusting the path

Sample Run of FastDTW

1/8 1/4 1/2 1/1

FastDTW Algorithm

1. Set the resolution to be the coarsest2. Find the initial path using regular DTW3. Repeat

a. Double the resolutionb. Project the path onto the finer resolutionc. Find a path through the projected area (plus a

small radius around the projected area)

4. Until the original resolution is reached

Complexity

• O(N) time

• O(N) space

• Details in the paper

Evaluation Criteria

• AccuracyThe error of an approximate Time Warping algorithm:

% error = where:

approxDist – the warp path distance of the approximate algorithm

optimalDist – the warp path distance of the DTW algorithm

• EfficiencyRuntime (measured in seconds)

100

toptimalDis

toptimalDisapproxDist

Evaluation Procedure (Accuracy)

• Data Sets – UCR Time Series Data Mining Archive (Keogh & Folias 2002), 3 groups used:

1. Random – 45 unrelated time series (earthquakes, random walk, eeg, speech, etc.)

2. Trace – 200 time series simulating nuclear power plant failure (4 classes)

3. Gun – 200 time series of a gun being drawn and pointed (2 classes)

• Procedure1. Run FastDTW, Constraints (Sakoe-Chiba Band), and Data

Abstraction on all pairs within a data set group, also vary the radius

2. Record the average error of all three methods for a group of data and a radius

Radius0 1 10 20 30

FastDTW 19.2% 8.6% 1.5% 0.8% 0.6%

Abstraction 983.3% 547.9% 6.5% 2.8% 1.8%

Band 2749.2% 2385.7% 794.1% 136.8% 9.3%

Average % Error (Accuracy)

Error in Different Data Sets

Accuracy of FastDTW, Bands, and Data Abstraction

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20 25 30radius

Err

or

FastDTW-Random FastDTW-Trace FastDTW - GunAbstraction-Random Abstraction-Trace Abstraction-GunBand-Random Band-Trace Band-Gun

Evaluation Procedure (Execution-time)

• Data Sets– Synthetic sine waves with Gaussian noise– 10 to 180,000 data points

• Procedure1. Run FastDTW and DTW on each data set,

vary the radius for FastDTW

2. Compare the Execution times

Execution TimeExecution Time of FastDTW on Large Time Series

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 160,000 180,000

Length of Time Series

Tim

e (s

eco

nd

s)

DTW

FastDTW (radius=100)

FastDTW (radius=20)

FastDTW (radius=0)

Summary of Contributions

• FastDTW – an approximation of DTW– O(N) time and space complexity– Scales well to long time series– Accurate, 8.6% error if radius=1, 0.8% error if

radius=20

Limitations and Future Work

• Limitations– FastDTW does not always find an optimal solution

• Future Work– Examine using different step sizes between resolutions

– Investigate search algorithms to help improve refinement

– Examine # of cells evaluated vs. accuracy between the FastDTW, Abstraction, and Band algorithms.

Questions?

Thanks to those who helped with this research:

Matt Mahoney (Florida Institute of Technology),

Brian Buckley, Walter Schiefele (Interface & Control Systems)

This research is partially supported by NASA

FastDTW PseudocodeInput: X, Y, radius Output: 1) A minimum distance warp path between X and Y 2) The warped path distance between X and Y

1| // The min size of the coarsest resolution. 2| Integer minTSsize = radius+2 3| 4| IF (|X|≤ minTSsize OR |Y|≤ minTSsize) 5| { 6| // Base Case: for a very small time series run the full DTW algorithm 7| RETURN DTW(X, Y) 8| } 9| ELSE10| {11| // Recursive Case: Project the warp path from a coarser resolution onto the current current resolution. 12| // Run DTW only along the projected path (and also radius cells from the projected path).13| TimeSeries shrunkX = X.reduceByHalf() // Coarsening14| TimeSeries shrunkY = Y.reduceByHalf() // Coarsening15| 16| WarpPath lowResPath = FastDTW(shrunkX, shrunkY, radius)17|18| SearchWindow window = ExpandedResWindow(lowResPath, X, Y, radius) //

Projection19| 20| RETURN DTW(X, Y, window) // Refinement21| }