Real-time constant memory visual summaries for surveillance

In surveillance applications there may be multiple time scales at which it is important to monitor a scene. This work develops on-line, real-time algorithms that maintain background models simultaneously at many time scales. This creates a novel temporal de-composition of video sequence which can be used as a visualization tool for a human operator or an adaptive background model for classical anomaly detection and tracking algorithms. This paper solves the design problem for choosing appropriate time scales for the decomposition and derives the equations to approximately reconstruct the original video given only the temporal decompo-sition. We present two applications that highlight the potential of video processing; first a visualization tool that summarizes recent video behavior for a human operator in a single image, and second a pre-processing tool to detect "left bags" in the challenging PETS 2006 dataset which includes many occlusions of the left bag by pedestrians.