Document Type
Article
Publication Date
12-2009
Publication Title
Signal Processing
Volume
89
Issue
12
Abstract
We consider the problem of detecting changes in a multivariate data stream. A change detector is defined by a detection algorithm and an alarm threshold. A detection algorithm maps the stream of input vectors into a univariate detection stream. The detector signals a change when the detection stream exceeds the chosen alarm threshold. We consider two aspects of the problem: (1) setting the alarm threshold and (2) measuring/comparing the performance of detection algorithms. We assume we are given a segment of the stream where changes of interest are marked. We present evidence that, without such marked training data, it might not be possible to accurately estimate the false alarm rate for a given alarm threshold. Commonly used approaches assume the data stream consists of independent observations, an implausible assumption given the time series nature of the data. Lack of independence can lead to estimates that are badly biased. Marked training data can also be used for realistic comparison of detection algorithms. We define a version of the receiver operating characteristic curve adapted to the change detection problem and propose a block bootstrap for comparing such curves. We illustrate the proposed methodology using multivariate data derived from an image stream.
First Page
2529
Last Page
2536
Recommended Citation
Kim, Albert Y.; Marzban, Caren; Percival, Donald B.; and Stuetzle, Werner, "Using Labeled Data to Evaluate Change Detectors in a Multivariate Streaming Environment" (2009). Statistical and Data Sciences: Faculty Publications, Smith College, Northampton, MA.
https://scholarworks.smith.edu/sds_facpubs/16
Digital Object Identifier (DOI)
doi.org/10.1016/j.sigpro.2009.04.011
Rights
© the authors
Comments
Peer reviewed accepted manuscript.