Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. A method for finding the optical flow pattern is presented which assumes that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. An iterative implementation is shown which successfully computes the optical flow for a number of synthetic image sequences. The algorithm is robust in that it can handle image sequences that are quantized rather coarsely in space and time. It is also insensitive to quantization of brightness levels and additive noise. Examples are included where the assumption of smoothness is violated at singular points or along lines in the image.

Change detection plays a very important role in real-time image analysis, e.g., detection of intruders. One key issue is robustness to varying illumination conditions. We propose two techniques for change detection that have been developed to deal with variations in illumination and background, with real-time capabilities. The foundations of these techniques are based on a vector model of images and on the exploitation of the concepts of linear dependence and linear independence. Furthermore, the techniques are compatible with physical photometry. A detailed description of the proposed detector and three state-of-the art change detectors is also provided. For the purposes of comparison, an evaluation procedure is presented consisting of both objective and subjective parts. This evaluation procedure results in a final performance value for each detector analyzed. Keywords—Background extraction, change detection, illumination invariance, real-time, surveillance, Wronskian. I.

We present a model for recovering the direction of heading of an observer who is moving relative to a scene that may contain self-moving objects. The model builds upon an algorithm proposed by Rieger and Lawton (1985), which is based on earlier work by Longuet-Higgins and Prazdny (1981). The algorithm uses velocity differences computed in regions of high depth variation to estimate the location of the .focus o.f ezpansion, which indicates the observer's heading direction. We relate the behavior of the proposed model to psychophysical observations regarding the ability of human observers to judge their heading direction, and show how the model can cope with self- moving objects in the environment. We also discuss this model in the broader context of a navigational system that performs tasks requiring rapid sensing and response through the interaction of simple task-specific routines.

A novel approach for the computation of optical flow based on an L type minimization is presented. It is shown that the approach has inherent advantages since it does not smooth the flow-velocity across the edges and hence preserves edge information. A numerical approach based on computation of evolving curves is proposed for computing the optical flow field. Computations are carried out on a number of synthetic and real image sequences in order to illustrate the theory as well as the numerical approach.

This paper presents a real-time system for pedestrian tracking in sequences of grayscale images acquired by a stationary camera. The objective is to integrate this system with a traffic control application such as a pedestrian control scheme at intersections. The proposed approach can also be used to detect and track humans in front of vehicles. Furthermore, the proposed schemes can be employed for the detection of several diverse traffic objects of interest (vehicles, bicycles, etc.) The system outputs the spatio-temporal coordinates of each pedestrian during the period the pedestrian is in the scene. Processing is done at three levels: raw images, blobs, and pedestrians. Blob tracking is modeled as a graph optimization problem. Pedestrians are modeled as rectangular patches with a certain dynamic behavior. Kalman filtering is used to estimate pedestrian parameters. The system was implemented on a Datacube MaxVideo 20 equipped with a Datacube Max860 and was able to achieve a peak performance of over 30 frames per second. Experimental results based on indoor and outdoor scenes demonstrated the system's robustness under many difficult situations such as partial or full occlusions of pedestrians.

A model that is capable of maintaining the identities of individuated elements as they move is described. It solves a particular problem of underdetermination, the motion correspondence problem, by simultaneously applying 3 constraints: the nearest neighbor principle, the relative velocity principle, and the element integrity principle. The model generates the same correspondence solutions as does the human visual system for a variety of displays, and many of its properties are consistent with what is known about the physiological mechanisms underlying human motion perception. The model can also be viewed as a proposal of how the identities of attentional tags are maintained by visual cognition, and thus it can be differentiated from a system that serves merely to detect movement.

In this paper we propose a new representation of videos with spatiotemporal blocks. After a given video is decomposed into the spatiotemporal blocks, a dimensionality reduction technique is applied to obtain a compact vector representation of each block gray level values. The block vectors provide a joint representation of texture and motion patterns in videos. Our results on PETS repository videos show that detection and tracking of moving objects is substantially improved if based on spatiotemporal blocks instead on pixels. Thus, we go away from the standard input of pixel values that are known to be noisy and the main cause of instability of video analysis algorithms. 1.

In this paper we propose to use local variation of spatiotemporal texture vectors for motion detection. The local variation is defined as the largest eigenvalue component of spatiotemporal (sp) texture vectors in certain time window at each location in a video plane. Sp texture vectors are computed using a dimensionality reduction technique applied to spatiotemporal (3D) blocks. They provide a compact vector representation of texture and motion patterns for each block. The fact that we go away from the standard input of pixel values and instead base the motion detection on sp texture of 3D blocks, significantly improves the quality of motion detection. This is particularly relevant for infrared videos, where pixel values have smaller range than in daylight color or gray level videos.

Articulated motion is a subset of non-rigid motion in which the object of interest is composed of several rigid components connected to each other by ball and hinge joints. The human body, many animals and insects, and machinery all exhibit such motion. This dissertation addresses the problem of vision-based tracking and analysis of this type of motion. The importance of this problem can be seen in many application domains including surveillance, traffic monitoring, entertainment, user interfaces, medicine, sports, video annotation, and image compression. This dissertation deals with two important subproblems of the general problem: whole-body tracking and motion recognition. In whole-body tracking, the body is tracked as one unit without paying attention to the details of the posture and limbs. Current solutions to this problem suffer from being too sensitive to small changes in the environment. We present a novel approach which reduces these restrictions significantly. This is achieved by separating the concepts of a blob from that of a body and by tracking each independently while maintaining a many-to-many relationship between the two. The approach makes use of the Extended Kalman Filter and outputs trajectory information in world coordinates. The method was tested by tracking pedestrians in a variety of environments and achieved real-time performance and a high degree of robustness. Motion recognition is the high level problem of classifying an action taking place in a video sequence into one of several action categories. Most of the present approaches attempt to perform three-dimensional reconstruction of the articulated shape prior to recognition, which is an inherently difficult problem made even more difficult due to the nonrigidity of the articulated object. W...

INTRODUCTION AND A PERSPECTIVE MUBARAK SHAH Computer Vision Lab Computer ScienceDepartment University of Central Florida Orlando, FL 32816 AND RAMESH JAIN Electrical and Computer Engineering University of California, San Diego La Jolla, CA 92093-0407 1. Introduction Computer vision has started migrating from the peripheral area to the core of computer science and engineering. Multimedia computing and natural human-machine interfaces are providing adequate challenges and motivation to develop techniques that will playkey role in the next generation of computing systems. Recognition of objects and events is very importantin multimedia systems as well as interfaces. We consider an object a spatial entity and an eventatemporal entity. Visual recognition of objects and activities is one of the fastest developing area of computer vision. Objects and events must be recognized by analyzing images. An