Abstract. In this chapter, a new feature-level image fusion technique for object segmentation is presented. The proposed technique approaches fusion as a feature selection problem, utilizing a selection criterion based on mutual information. Starting with object regions roughly detected from one sensor, the proposed technique aims to extracts relevant information from another sensor in order to best complete the object segmentation. First, a contour-based feature representation is presented that implicitly captures object shape. The notion of relevance across sensor modalities is then defined using mutual information computed based on the affinity between contour features. Finally a heuristic selection scheme is proposed to identify the set of contour features having the highest mutual information with the input object regions. The approach works directly from the input image pair without relying on a training phase. The proposed algorithm is evaluated using a typical surveillance setting. Quantitative results, and comparative analysis with other potential fusion methods are presented. 1

Abstract — This paper makes new contributions in motion detection, object segmentation and trajectory estimation to create a successful object tracking system. A new efficient motion detection algorithm referred to as the flux tensor is used to detect moving objects in infrared video without requiring background modeling or contour extraction. The flux tensor-based motion detector when applied to infrared video is more accurate than thresholding ”hot-spots”, and is insensitive to shadows as well as illumination changes in the visible channel. In real world monitoring tasks fusing scene information from multiple sensors and sources is a useful core mechanism to deal with complex scenes, lighting conditions and environmental variables. The object segmentation algorithm uses level set-based geodesic active contour evolution that incorporates the fusion of visible color and infrared edge informations in a novel manner. Touching or overlapping objects are further refined during the segmentation process using an appropriate shapebased model. Multiple object tracking using correspondence graphs is extended to handle groups of objects and occlusion events by Kalman filter-based cluster trajectory analysis and watershed segmentation. The proposed object tracking algorithm was successfully tested on several difficult outdoor multispectral videos from stationary sensors and is not confounded by shadows or illumination variations. Index Terms — Flux tensor, sensor fusion, object tracking, active contours, level set, infrared images. I.

A feature-selection approach to fusing electro-optical and IR imagery

To separate object motion from camera motion in an aerial video, consecutive frames are registered at their planar background. Feature points are selected in consecutive frames and those that belong to the background are identified using the projective constraint. Corresponding background feature points are then used to register and align the frames. By aligning video frames at the background and knowing that objects move against the background, a means to detect and track moving objects is provided. Only scenes with planar background are considered in this study. Experimental results show improvement in registration accuracy when using the projective constraint to determine the registration parameters as opposed to finding the registration parameters without the projective constraint. I.

Abstract—Gaussian Mixture Models (GMMs) are the most popular techniques in background modeling but present some limitations when some dynamic changes occur like camera jitter, illumination changes, movement in the background. Furthermore, the GMM are initialized using a training sequence which may be noisy and/or insufficient to model correctly the background. All these critical situations generate false classification in the foreground detection mask due to the related uncertainty. In this context, we propose to model the background by using a Type-2 Fuzzy Gaussian Mixture Models. The interest is to introduce descriptions of uncertain parameters in the GMM. Experimental validation of the proposed method is performed and presented on a diverse set of RGB and infrared videos. Results show the relevance of the proposed approach.

Detecting moving pedestrians and vehicles with foreground segmentation algorithms is problematic during fluctuating lighting conditions. Edge-based approaches are more robust to lighting than the conventional intensity-based ones. The issue with edge-based approaches though is segmenting the internal foreground areas. In this work a strategy is developed to detect complete foreground areas. Firstly, edge-extraction is performed at multiplescales which increases the initial area detected. To complete the detection of object areas, edgemotion-history-images are introduced. The final segmentation is achieved with a region growing algorithm in the edge-motion-history-image. Examples are shown of the successful extraction of foreground objects through changing lighting conditions. 1