2004-09-23 PM Event Detection from Time Series
-
Upload
rudolf-husar -
Category
Technology
-
view
447 -
download
1
description
Transcript of 2004-09-23 PM Event Detection from Time Series
![Page 1: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/1.jpg)
PM Event Detection from Time Series
Contributed by the FASNET Community, Sep. 2004Correspondence to R Husar , R Poirot
Coordination Support by
Inter-RPO WG Fast Aerosol Sensing Tools for Natural Event Tracking, FASTNET
NSF Collaboration Support for Aerosol Event Analysis
NASA REASON Coop
EPA -OAQPS
Event : Deviation > x*percentile
![Page 2: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/2.jpg)
Temporal Analysis
• The time series for typical monitoring data are ‘messy’; the signal variation occurs at various scales and the time pattern at each scale is different
• Inherently, aerosol events are spikes in the time series of monitoring data but extracting the spikes from the noisy data is a challenging endeavor
The temporal signal can be meaningfully decomposed into a
1. Seasonal component with stable periodic pattern
2. Random variation with ‘white noise’ pattern
3. Spikes or events that are more random in frequency and magnitude
Each signal component is caused by different combination of the key processes: emission, transport, transformations and removal
Typical time series of daily AIRNOW PM25 over the Northeastern US
![Page 3: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/3.jpg)
Temporal Signal Decomposition and
Event Detection
• First, the median and average is obtained over a region for each hour/day (thin blue line)
• Next, the data are temporally smoothed by a 30 day moving window (spatial median - red line; spatial mean – heavy blue line). These determine the seasonal pattern.
EUS Daily Average 50%-ile, 30 day 50%-ile smoothing
Deviation from %-ile
Event : Deviation > x*percentile
Median Seasonal Conc.
Mean Seasonal Conc.
Average
Median
• Finally, the hourly/daily deviation from the the smooth median is used to determine the noise (blue) and event (red) components
![Page 4: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/4.jpg)
Seasonal PM25 by Region
The 30-day smoothing average shows the seasonality by region
The Feb/Mar PM25 peak is evident for the Northeast, Great Lakes and Great Plains
This secondary peak is absent in the South and West
![Page 5: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/5.jpg)
Northeast – Southeast Comparison• Northeast and Southeast differ in the pattern of seasonal and event variation• Northeast has two seasonal peaks and more events–values well above the median• Southeast peaks in September and has few values much above the noise
Northeast
Southeast
![Page 6: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/6.jpg)
Causes of Temporal Variation by Region
The temporal signal variation is decomposable into seasonal, meteorological noise and events
Assuming statistical independence, the three components are additive:
V2Total = V2
Season + V2MetNoise + V2
Event
The signal components have been determined for each region to assess the differences
Northeast exhibits the largest coeff. variation (56%); seasonal, noise and events each at 30%Southeast is the least variable region (35%), with virtually no contribution from eventsSouthwest, Northwest, S. Cal. and Great Lakes/Plains show 40-50% coeff. variation mostly, due to seasonal and meteorological noise.Interestingly, the noise is about 30% in all regions, while the events vary much more, 5-30%
![Page 7: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/7.jpg)
‘Composition’ of Eastern US Events• The bar-graph shows the various combinations of species-
events that produce Reconstructed Fine Mass (RCFM) events
• ‘Composition’ is defined in terms of co-occurrence of multi-species events (not by average mass composition)
• The largest EUS RCFM events are simultaneously ‘events’ (spikes) in sulfate, organics and soil!
• Some EUS RCFM events are events in single species, e.g. 7-Jul-97 (OC), 21-Jun-97 (Soil)
Based on VIEWS data
![Page 8: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/8.jpg)
Northeast
![Page 9: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/9.jpg)
Great Lakes
![Page 10: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/10.jpg)
Great Lakes-Plains
![Page 11: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/11.jpg)
Northeast
![Page 12: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/12.jpg)
Great Plains
![Page 13: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/13.jpg)
NorthWest
![Page 14: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/14.jpg)
S. California
![Page 15: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/15.jpg)
Southeast
![Page 16: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/16.jpg)
Southwest
![Page 17: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/17.jpg)
Event Definition:Time Series Approach
• Eastern US aggregate time series
![Page 18: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/18.jpg)
Sulfate
EUS Daily Average
50%-ile, 30 day 50%-ile smoothing
Deviation from %-ile
Event – Deviation > percentile value
Median Seasonal Conc.
Mean Seasonal Conc.
![Page 19: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/19.jpg)
Reconstructed Fine Mass RCFM
![Page 20: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/20.jpg)
Organic Carbon
![Page 21: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/21.jpg)
Eelemental Carbon
![Page 22: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/22.jpg)
SOIL
![Page 23: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/23.jpg)
Nitrate
![Page 24: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/24.jpg)
Temporal Pattern Regional Speciated Analysis - VIEWS
• Aerosol species time series:– ammSO4f– OCf– ECf– SOILf– ammNO3f– RCFM
Regions of Aggregation
![Page 25: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/25.jpg)
Dust
Seasonal + spikes
East – west events are independent
East events occur several times a year, mostly in summer
West events are lest frequent, mostly in spring
US
West
East
![Page 26: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/26.jpg)
Dust
asgasgasfg
Northeast
Southwest
Southeast
![Page 27: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/27.jpg)
Dust
dfjdjdfjetyj
Northwest
S. California
Great Plaines
![Page 28: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/28.jpg)
Amm. Sulfate
wdthehreherh
US
West
East
![Page 29: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/29.jpg)
Amm. Sulfate
stheherheyju
Northeast
Southwest
Southeast
![Page 30: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/30.jpg)
Amm. Sulfate
shheherh
Northwest
S. California
Great Plaines
![Page 31: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/31.jpg)
Organic Carbon
sdhdfhefheryj
US
West
East
![Page 32: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/32.jpg)
Organic Carbon
sdheherh
Northeast
Southwest
Southeast
![Page 33: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/33.jpg)
Organic Carbon
erheryeyj
Northwest
S. California
Great Plaines
![Page 34: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/34.jpg)
Reconstructed Fine Mass
estrhertheryu
US
West
East
![Page 35: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/35.jpg)
Reconstructed Fine Mass
werty3rueru
Northeast
Southwest
Southeast
![Page 36: 2004-09-23 PM Event Detection from Time Series](https://reader034.fdocuments.in/reader034/viewer/2022051514/54c1bfd24a7959a8778b4675/html5/thumbnails/36.jpg)
Reconstructed Fine Mass
wthwrthwerhtr
Northwest
S. California
Great Plaines