Abstract:
Burst detection is the activity of finding abnormal aggregates in data streams. Such aggregates are based on sliding windows over data streams. In some applications, we want to monitor many sliding window sizes simultaneously and to report those windows with aggregates significantly different from other periods. We will present a general data structure for detecting interesting aggregates over such elastic windows in near linear time. We present applications of the algorithm for detecting Gamma Ray bursts in large-scale astrophysical data. Detection of periods with high volumes of trading activities and high stock price volatility is also demonstrated using real time Trade and Quote (TAQ) data from the New York Stock Exchange (NYSE). Our algorithm beats the direct computation approach by several orders of magnitude. 1.
Citations
|
347
|
Models and issues in data stream systems
– Babcock, Babu, et al.
- 2002
|
|
311
|
Efcient similarity search in sequence databases
– Agrawal, Faloutsos, et al.
- 1993
|
|
302
|
The Quadtree and related hierarchical data structures. ACM computing surveys
– Samet
- 1984
|
|
164
|
Wavelet-based histograms for selectivity estimation
– Matias, Vitter, et al.
- 1998
|
|
156
|
Monitoring Streams - A New Class of Data Management Applications
– Carney, Etintemel, et al.
- 2002
|
|
140
|
Maintaining stream statistics over sliding windows
– Datar, Gionis, et al.
- 2002
|
|
135
|
Surfing wavelets on streams: One-pass summaries for approximate aggregate queries
– Gilbert, Kotidis, et al.
- 2001
|
|
133
|
Mining time-changing data streams
– Hulten, Spencer, et al.
- 2001
|
|
131
|
Approximate computation of multidimensional aggregates of sparse data using wavelets
– Vitter, Wang
- 1999
|
|
127
|
Approximate query processing using wavelets
– Chakrabarti, Garofalakis, et al.
- 2001
|
|
127
|
Efficient time-series matching by wavelets
– Chan, Fu
- 1999
|
|
108
|
On computing correlated aggregates over continual data streams
– Gehrke, Korn, et al.
- 2001
|
|
93
|
Bursty and hierarchical structure in streams
– Kleinberg
- 2002
|
|
90
|
StatStream: Statistical monitoring of thousands of data streams in real time
– Zhu, Shasha
- 2002
|
|
43
|
Finding surprising patterns in a time series database in linear time and space
– Keogh, Lonardi, et al.
- 2002
|
|
27
|
TSA-tree: A wavelet-based approach to improve the efficieny of multi-level surprise and trend queries
– Shahabi, Tian, et al.
- 2000
|
|
27
|
Data mining meets performance evaluation: Fast algorithm for modeling bursty traffic
– Wang, Madhyastha, et al.
- 2002
|
|
24
|
Mining Deviants in a Time Series Database
– Jagadish, Koudas, et al.
- 1999
|
|
5
|
Demon: Data evolution and monitoring
– Ganti, Gehrke, et al.
- 2000
|
|
2
|
for the Milagro Collaboration. A search for bursts of tev gamma rays with milagro
– Smith
- 2001
|
|
1
|
The Milagro Collaboration). Evidence for TeV emission from GRB 970417a
– al
- 2000
|
