1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The...
-
date post
18-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The...
![Page 1: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/1.jpg)
1
Spatio-Temporal Outlier Detection in Precipitation Data
Elizabeth Wu, Wei Liu, Sanjay ChawlaThe University of Sydney, Australia
SensorKDD 2008
Sunday, 24th August, 2008
![Page 2: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/2.jpg)
2
Outline
• What is a spatio-temporal outlier?
• Motivation
• Previous Work
• Contributions
• Our Approach
• Future Work
![Page 3: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/3.jpg)
4
What is a Spatio-Temporal Outlier?
• “A spatio-temporal object whose thematic attribute values are significantly different from those of other spatially and temporally referenced objects in its spatial and/or temporal neighborhoods.”
– Cheng and Li (2006)
t=1 t=2 t=3 t=4 t=5
1 2 3 4 5 1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
5
4
3
2
1
![Page 4: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/4.jpg)
5
What is a spatio-temporal object?
• “A time-evolving spatial object whose evolution or ‘history’ is represented by a set of instances (o_id, si, ti) where the spacestamp si is the location of object o_id at timestamp ti.” - Theodoris et. al. (1999)
• Simply put,
time
y co-ordinate
x co-ordinate
time
y co-ordinate
x co-ordinate
A 2D region becomes a 3D region A point becomes a line
![Page 5: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/5.jpg)
6
Data• South American precipitation data
(NOAA)• 10 years (1995-2004)
• 2.5 x 2.5° grids
• 31 latitude x 23 longitude divisions
• 713 grids total
• 2,609,580 possible data values
• Missing data – spatially and temporally
• El Niño Southern Oscillation Data (NOAA)
• Southern Oscillation Index (SOI)
• Measures the difference in Sea Surface Temperature (SST) between Tahiti and Darwin
• The lower the score, the more intense an El Niño event
Figure: Stations used to produce gridded
precipitation fields
![Page 6: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/6.jpg)
7
Motivation
• Why would we be interested in moving outlier regions in precipitation data?• Knowing the location, time and duration of past
extreme precipitation events helps to understand and prepare for future events.
• We can analyse how different phenomenon interact.
• E.g. ENSO and precipitation.
![Page 7: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/7.jpg)
8
Previous Work
• Spatial Scan Statistics• Used to find spatial outliers
• Cluster detection using the spatial scan statistic in spatio-temporal point data (Iyengar, 2004)
• Exact-Grid and Approx-Grid (Agarwal et. al., 2006)• Uses the Kulldorff Spatial Scan Statistic
• Finds the highest discrepancy region (by location and size) in a spatial grid dataset.
• Spatio-temporal outlier detection (Birant and Kut, 2006)
• Limited to finding outliers over a single time period.
time
y co-ordinate
x co-ordinate
![Page 8: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/8.jpg)
9
Contributions
• Extended Exact-Grid and Approx-Grid to find the top-k outliers in a single time period.
• Developed the Outstretch & RecurseNodes algorithm to find outliers that repeatedly appear over several time periods.
• Apply to South American Precipitation data.
• Analyse the behaviour of the outliers against the El Niño Southern Oscilation (ENSO).
![Page 9: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/9.jpg)
10
Our Approach
1. Find the top-k outliers in a spatial grid for each time period• Extend Exact-Grid and Approx-Grid algorithms
2. Use Oustretch to find spatial outliers which extend over several time periods.
3. Use RecurseNodes to extract the sequences from the Outstretch tree.
![Page 10: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/10.jpg)
11
Finding the top-k outliers
• Find every possible region size and shape in the grid.
• Get each region’s discrepancy value to determine which is a more significant outlier.
• Our extension keeps track of the top-k regions rather than just the top-1.
left right
top
bottom
![Page 11: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/11.jpg)
12
• Uses two values:1. Measurement – Number of incidences of an event
• E.g. In how many cells is precipitation extreme?• M – for the whole dataset• m(p) - for the cell p
• mR = ΣpєR m(p) / M
• Baseline – Total population at risk• I.e. How many cells have we recorded values for?• B – for the whole dataset
b(p) - for the cell p
• bR = ΣpєR b(p) / B
• We find the discrepancy for local region R by subsitution into:• When mR > bR
d(mR, bR) = mRlog(mR/bR) + (1-mR)log((1-mR)/(1-bR))
• Otherwise d(mR, bR) = 0
Kulldorff Scan Statistic
![Page 12: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/12.jpg)
13
Kulldorff Scan Statistic: Example
• M = 6 = total # cells with “1” in entire grid
• ΣpєR m(p) = 4= total # cells with “1” in R
• mR = ΣpєR m(p)/M = 0.67
• B = 16= total # cells in entire grid
• ΣpєR b(p) = Sum of b’s in region = 4= total # cells in R
• bR = ΣpєR b(p)/B = 0.25
• Result: d(mR, bR) = 0.3836
1 1 0 0
1 1 0 0
0 0 0 1
0 0 1 0
1 2 3 4
4
3
2
1
1 1 1 1
1 1 1 1
1 1 1 1
1 1 1 1
1 2 3 4
4
3
2
1
![Page 13: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/13.jpg)
14
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 14: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/14.jpg)
15
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 15: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/15.jpg)
16
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 16: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/16.jpg)
17
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 17: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/17.jpg)
18
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 18: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/18.jpg)
19
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 19: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/19.jpg)
20
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 20: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/20.jpg)
21
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 21: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/21.jpg)
22
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 22: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/22.jpg)
23
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 23: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/23.jpg)
24
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 24: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/24.jpg)
25
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 25: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/25.jpg)
26
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 26: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/26.jpg)
27
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 27: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/27.jpg)
28
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 28: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/28.jpg)
29
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 29: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/29.jpg)
30
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 30: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/30.jpg)
31
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 31: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/31.jpg)
32
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 32: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/32.jpg)
33
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 33: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/33.jpg)
34
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 34: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/34.jpg)
35
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 35: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/35.jpg)
36
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 36: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/36.jpg)
37
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 37: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/37.jpg)
38
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 38: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/38.jpg)
39
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 39: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/39.jpg)
40
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 40: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/40.jpg)
41
Finding the top-k outliers: Exact-Grid
left right
bottom
top
Keeps moving top and bottom lines
until all regions have been examined
between the left and right lines…
![Page 41: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/41.jpg)
43
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 42: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/42.jpg)
44
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 43: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/43.jpg)
45
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 44: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/44.jpg)
46
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 45: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/45.jpg)
47
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 46: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/46.jpg)
48
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 47: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/47.jpg)
49
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 48: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/48.jpg)
50
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 49: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/49.jpg)
51
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 50: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/50.jpg)
52
Finding the top-k outliers: Exact-Grid
left right
bottom
top
![Page 51: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/51.jpg)
53
Finding the top-k outliers: Exact-Grid
left right
bottom
Same again…
Top and bottom lines define all possible areas
between the left and right lines…
top
![Page 52: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/52.jpg)
54
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 53: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/53.jpg)
55
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 54: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/54.jpg)
56
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 55: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/55.jpg)
57
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 56: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/56.jpg)
58
Finding the top-k outliers: Exact-Grid
left right
top
bottom
![Page 57: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/57.jpg)
59
Finding the top-k outliers: Exact-Grid
left right
top
bottom
Continue until all regions have been
examined…
![Page 58: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/58.jpg)
60
Finding the top-k outliers: Approx-Grid
• Reduces the time complexity of the algorithm by using only two sweep lines and finding the interval that maximises the discrepancy function
• (See Agarwal et al. (2006) paper).
top
bottom
m(I,j) stores the sum of the
m(p)’s for each column
For each move of a sweep line, run the Linear1D algorithm to find the interval that maximises the
discrepancy function
![Page 59: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/59.jpg)
61
Finding the top-k outliers: Considerations
• Overlapping Regions
![Page 60: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/60.jpg)
62
Finding the top-k outliers: Considerations
• Overlapping Regions – Overlap types
![Page 61: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/61.jpg)
63
Finding the top-k outliers: Considerations
• Chain effect
• One option: Union Solution
d=0.45
d=0.51d=0.54
![Page 62: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/62.jpg)
64
Finding the top-k outliers: Considerations
• Chosen Option: Allow a percentage of overlap
d=0.45
d=0.51
If this overlap is less than allowable_overlap
% then, keep both regions in the top-k list.
![Page 63: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/63.jpg)
65
Outstretch
• Outstretch – find the paths of the outliers over time.
t=1 t=2 t=3 t=4 t=5
1 2 3 4 5 1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
1 2 3 4 5
5
4
3
2
1
5
4
3
2
1
![Page 64: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/64.jpg)
66
Outstretch
• Use Outstretch to find spatial outliers which extend over several time periods.
• Check the same region (slightly stretched to cover more area) in the next time period, to see if another outlier lies in the region.
• If it is, then it is considered to be part of the spatio-temporal outlier, which is now extended over an additional time period.
• Store in a tree data structure.
r
This region (dark green) has been stretched by
r=2 grid cells…
In the next time period, we will check if any
outliers fall in that area.
![Page 65: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/65.jpg)
67
Outstretch
• Store outliers found over subsequent time periods in a tree data structure.
Node
Num Children
Children
1,1 1 {2,2}
1,2 3 {2,2}, {2,3}, {2,4}
1,3 1 {2,1}
1,4 1 {2,4}
2,1 1 {3,2}
2,2 3 {3,1}, {3,4}
2,3 2 {3,3}
2,4 0 -
3,1 0 -
3,2 0 -
3,3 0 -
3,4 0 -
1,1 1,2 1,3
2,1 2,2 2,3
3,1 3,2 3,3
1,4
3,4
2,4
![Page 66: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/66.jpg)
68
Outstretch
1,1 1,2 1,3 1,4
1
2
34
• Stretch the top-k outliers from t=1 by r (their spatial neighbourhood).
![Page 67: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/67.jpg)
69
Outstretch
1,1 1,2 1,3
2,1 2,2 2,3
1,4
2,4
1
2
34
2
1
3
4
• From the top-k in t=2, find those which fall inside the stretched region from the previous period, t=1.
![Page 68: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/68.jpg)
70
Outstretch
1,1 1,2 1,3
2,1 2,2 2,3
3,1 3,2 3,3
1,4
3,4
2,4
1
2
34
2
1
3
4
1
4
3
2
• Stretch the new outliers from t=2 and find the outliers from t=3, that fall in the newly stretched regions.
![Page 69: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/69.jpg)
71
RecurseNodes
• Now that we’ve stored all the sequences in the tree, how do we get them out?
• Use RecurseNodes to extract the sequences from the Outstretch tree.
Node
Num Children
Children
1,1 1 {2,2}
1,2 3 {2,2}, {2,3}, {2,4}
1,3 1 {2,1}
1,4 1 {2,4}
2,1 1 {3,2}
2,2 3 {3,1}, {3,4}
2,3 2 {3,3}
2,4 0 -
3,1 0 -
3,2 0 -
3,3 0 -
3,4 0 -
![Page 70: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/70.jpg)
73
RecurseNodes
• Start at {1,1}
• We notice it has a child {2,2}
• Check {2,2}
• We notice {2,2} has two children {3,1} and {3,4}.
• Check {3,1} first.
• {3,1} has no children. Stop and store sequence:[ {1,1}, {2,2}, {3,1} ]
• Now check {3,4}.
• {3,4} has no children. Stop and store sequence:[ {1,1}, {2,2}, {3,4} ]
• And so on…
Node
Num Children
Children
1,1 1 {2,2}
1,2 3 {2,2}, {2,3}, {2,4}
1,3 1 {2,1}
1,4 1 {2,4}
2,1 1 {3,2}
2,2 3 {3,1}, {3,4}
2,3 2 {3,3}
2,4 0 -
3,1 0 -
3,2 0 -
3,3 0 -
3,4 0 -
![Page 71: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/71.jpg)
74
Results: Exact vs. Approx-Grid Top-k
Exact-Grid Top-k O(n4k)
229s
Approx-Grid Top-k O(n3k)
35s
Length and number of outliers found
Outlier Discovery – Time Taken
• Exact-Grid Top-k:• finds longer
sequences than Approx-Grid Top-k
• Approx-Grid Top-k• Is faster than
Exact- Grid Top-k
![Page 72: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/72.jpg)
75
Results:Mean discrepancy of Exact-Grid Top-k sequences and the mean SOI
• Notice that some of the discrepancies at the centre time period are higher during the more intense El Niño event
• This is showing that there are more extreme extremes during an El Niño event.
![Page 73: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/73.jpg)
76
Results:Mean discrepancy of Approx-Grid Top-k sequences and the mean SOI
• We also find extreme extremes in the Approx-Grid Top-k sequences
![Page 74: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/74.jpg)
77
Future Work
• Evaluate against Other metrics (besides SOI), such as Sea Surface Temperature (SST)
• Point data
• Other data e.g. other precipitation data.
![Page 75: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/75.jpg)
78
Conclusion
• Our contributions:• Top-k extension to Exact and Approx-Grid algorithms
• Outlier sequence discovery over time
• Evaluate using precipitation data
• Compared results to the El Niño Southern Oscillation Index (SOI)
• Results showed:• More extreme extreme values during El Niño periods
• Able to find these with both Exact and Approx-Grid algorithms
![Page 76: 1 Spatio-Temporal Outlier Detection in Precipitation Data Elizabeth Wu, Wei Liu, Sanjay Chawla The University of Sydney, Australia SensorKDD 2008 Sunday,](https://reader036.fdocuments.in/reader036/viewer/2022081515/56649d255503460f949fc645/html5/thumbnails/76.jpg)
79
Questions
• Please ask