Looking into the future of residential and office building Mitsubishi Electric Research Labs (MERL) has been collecting motion sensor data from a network of over 200 sensors for a year. The data is the residual traces of year in the life of a research laboratory. It contains interesting spatio-temporal structure ranging all the way from the seconds of individuals walking down hallways, the minutes in the lobbies chatting with colleagues, the hours of dozens of people attending talks and meetings, the days and weeks that drive the patterns of life, to the months and seasons with their ebb and flow of visiting employees. This document describes that dataset, which contains well over 30 million raw motion records, spanning a calendar year and two floors of our research laboratory, as well as calendar, weather, and some intermediate analytic results.

Wide-area context awareness is a crucial enabling technology for next generation smart buildings and surveillance systems. It is not practical to cover an entire building with cameras, however it is di#cult to infer missing information when there are significant gaps in coverage. As a solution, we advocate a class of hybrid perceptual systems that builds a comprehensive model of activity in a large space, such as a building, by merging contextual information from a dense network of ultra-lightweight sensor nodes with video from a sparse network of high-capability sensors. In this paper we explore the task of automatically recovering the relative geometry between a pan-tilt-zoom camera and a network of one-bit motion detectors. We present results for the recovery of geometry alone, and also recovery of geometry jointly with simple activity models. Because we don't believe a metric calibration is necessary, or even entirely useful for this task, we formulate and pursue the novel goal we term functional calibration. Functional calibration is the blending of geometry estimation and simple behavioral model discovery. Accordingly, results are evaluated in terms of the ability of the system to automatically foveate targets in a large, nonconvex space, not in terms of pixel reconstruction error.

Looking into the future of residential and office building Mitsubishi Electric Research Labs (MERL) has been collecting motion sensor data from a network of over 200 sensors for a year. The data is the residual traces of year in the life of a research laboratory. It contains interesting spatio-temporal structure ranging all the way from the seconds of individuals walking down hallways, the minutes in lobbies chatting with colleagues, the hours of dozens of people attending talks and meetings, the days and weeks that drive the patterns of life, to the months and seasons with their ebb and flow of visiting employees. This document describes that dataset, which contains well over 30 million raw motion records, spanning a calendar year and two floors of our research laboratory, as well as calender, weather, and some intermediate analytic results. The dataset was

Video surveillance is a key technology for enhanced protection of facilities such as airports and power stations from various types of threat. Networks of thousands of IP-based cameras are now possible, but current surveillance methodologies become increasingly ineffective as the number of cameras grows. Constructing software that efficiently and reliably deals with networks of this size is a distributed information processing problem as much as it is a video interpretation challenge. This paper demonstrates a software architecture approach to the construction of large scale surveillance network software and explores the implications for instantiating surveillance algorithms at such a scale. A novel architecture for video surveillance is presented, and its efficacy demonstrated through application to an important class of surveillance algorithms. 1

This paper considers the problem of designing an active observer to plan a sequence of decisions regarding what target to look at, through a foveal-sensing action. We propose a framework in which a pan/tilt/zoom (PTZ) camera schedules saccades in order to acquire high resolution images (at least one) of as many moving targets as possible before they leave the scene. An intelligent choice of the order of sensing the targets can significantly reduce the total dead-time wasted by the active camera and, consequently, its cycle time. The grabbed images provide meaningful identification imagery of distant targets which are not recognizable in a wide angle view. We cast the whole problem as a particular kind of dynamic discrete optimization. In particular, we will show that the problem can be solved by modelling the attentional gaze control as a novel on-line Dynamic Vehicle Routing Problem (DVRP) with deadlines. Moreover we also show how multi-view geometry can be used for evaluating the cost of high resolution image sensing with a PTZ camera. Congestion analysis experiments are reported proving the effectiveness of the solution in acquiring high resolution images of a large number of moving targets in a wide area. The evaluation was conducted with a simulation using a dual camera system in a master-slave configuration. Camera performances are also empirically tested in order to validate how the manufacturer’s specification deviates from our model using an off-the-shelf PTZ camera.

We describe in this dissertation, planning strategies which enhance the accuracy with which visual surveillance can be conducted and which expand the capabilities of visual surveillance systems. Several classes of planning strategies are considered: sensor planning, motion planning and temporal planning. Sensor planning is the study of the control of cameras to optimize information gath-ering for performing vision algorithms. The study of camera control spans camera place-ment strategies, active camera (specifically, Pan-Tilt-Zoom or PTZ cameras) control, and, in some cases, camera selection from a collection of static cameras. Camera placement strategies have been employed previously for enhancing vision algorithms such as 3D reconstruction, area coverage in surveillance, occlusion and vis-ibility analysis, etc. We will introduce a two-camera placement strategy that is utilized by a background subtraction algorithm, allowing it to achieve video rate performance and invariance to several illumination artifacts, such as lighting changes and shadows. While camera placement strategies can improve the performance of vision algo-rithms significantly, their utilities are limited in situations where it is more cost-effective

Focus-of-attention camera systems comprised of stationary, far-field master cameras and pan-tilt-zoom slave cameras offer a practical way of consistently tracking all objects across a large area while simultaneously obtaining detailed imagery of activity for wide-area surveillance. In order for such high-resolution data to be useful for motion-based activity analysis, the systems should be able to segment and track moving objects in a high-resolution view with minimal or no delay upon fixating on a given part of the scene. We present a method that exploits the same wide-area adaptive background model used for consistently tracking all objects in a scene to enable a high-resolution segmentation of moving objects. Assuming a known or learned homography between the two views, we use a technique akin to super-resolution to infer from a section of the low-resolution background model its corresponding high-resolution model, which we refer to as the super-resolution background model. Using this method, we are able to segment moving objects from the background in a single high-resolution image.