This paper reviews a number of recently available techniques in contentanalysis of visual media and their application to the indexing, retrieval,abstracting, relevance assessment, interactive perception, annotation and re-use of visualdocuments. 1. Background A few years ago, the problems of representation and retrieval of visualmedia were confined to specialized image databases (geographical, medical, pilot experimentsin computerized slide libraries), in the professional applications of the audiovisualindustries (production, broadcasting and archives), and in computerized training or education. The presentdevelopment of multimedia technology and information highways has put content processing of visualmedia at the core of key application domains: digital and interactive video, large distributed digital libraries, multimedia publishing. Though the most important investments have been targeted at the information infrastructure (networks, servers, coding and compression, deliverymodels, multimedia systems architecture), a growing number of researchers have realized thatcontent processing will be a key asset in putting together successful applications. The need for contentprocessing techniques has been made evident from a variety of angles, ranging from achievingbetter quality in compression, allowing user choice of programs in video-on-demand, achieving betterproductivity in video production, providing access to large still image databases or integrating still images and video in multimedia publishing and cooperative work. Content-based retrieval of visual media and representation of visualdocuments in human-computer interfaces are based on the availability of content representationdata (time-structure for

The organization of video information for video databases requires segmentation of a video into its constituent shots and their subsequent characterization in terms of content and camera work. In this paper, we look at these two steps using compressed video data directly. For shot detection, we suggest a scheme consisting of comparing intensity, row, and column histograms of successive I frames of MPEG video using the chi-square test. For characterization of segmented shots, we address the problem of classifying shot motion into different categories using a set of features derived from motion vectors of P and B frames of MPEG video. The central component of the proposed shot motion characterization scheme is a decision tree classifier built through a process of supervised learning. Experimental results using a variety of videos are presented to demonstrate the effectiveness of performing shot detection and characterization directly on compressed video. * Pattern Recognition: Special ...

Abstract not found

This paper examines the issue of direct extraction of low level features from compressed images. Specifically, we consider the detection of areas of interest and edges in images compressed using the discrete cosine transform (DCT). For interest areas, we show how a measure based on certain DCT coefficients of a block can provide an indication of underlying activity. For edges, we show using an ideal edge model how the relative values of different DCT coefficients of a block can be used to estimate the strength and orientation of an edge. Our experimental results indicate that coarse edge information from compressed images can be extracted up to 20 times faster than conventional edge detectors. Keywords: Image compression, DCT domain, interest area extraction, edge detection, feature extraction 1. INTRODUCTION The ability to process and transmit visual information quickly is the current driving force in computer and telecommunication hardware and software developments. The present vi...

Temporal video segmentation is the "rst step towards automatic annotation of digital video for browsing and retrieval. This article gives an overview of existing techniques for video segmentation that operate on both uncompressed and compressed video stream. The performance, relative merits and limitations of each of the approaches are comprehensively discussed and contrasted. The gradual development of the techniques and how the uncompressed domain methods were tailored and applied into compressed domain are considered. In addition to the algorithms for shot boundaries detection, the related topic of camera operation recognition is also reviewed. � 2001 Elsevier Science B.V. All rights reserved.

Temporal segmentation of video is a necessary first step to indexing digital video for browsing and retrieval. A number of different video temporal segmentation algorithms have been published in the literature. There has been little effort to evaluate and characterize their performance so as to deliver a single (or set of) algorithms that may be used by other researchers for indexing video databases. We present results of evaluating a number of these algorithms and characterizing their performance, specifically with respect to robustness to encoder and bitrate changes. The lessons learnt have relevance to algorithm development and evaluation in general.

Extracting information from the ever growing stream of multimedia data is becoming increasingly difficult. One of the main reasons lies within the unstructured way multimedia data are usually presented. Audio-visual material represents a large part of current multimedia material and can be structured in meaningful ways due to the nature of visual communication. This paper surveys several approaches and algorithms that have been recently developed to help in automatically structuring audio-visual data, both for annotation and access

The major problem facing video databases is that of content characterization of video clips once the cut boundaries have been determined. The current efforts in this direction are focussed exclusively on the use of pictorial information, there by neglecting an important supplementary source of content information, i.e the embedded audio or sound track. The current research in audio processing can be readily applied to create many different video indices for use in Video On Demand (VOD), educational video indexing, sports video characterization etc. MPEG is an emerging video and audio compression standard with rapidly increasing popularity in multimedia industry. Compressed bit stream processing has gained good recognition among the researchers. We have also demonstrated feature extraction in MPEG compressed video which implements majority of scene change detection schemes on compressed video. 1 In this paper, we examine the potential of audio information for content characterization ...

Images or videos may be imaged under different illuminants than models in an image or video proxy database. Changing illumination color in particular may confound recognition algorithms based on color histograms or video segmentation routines based on these. Here we show that a very simple method of discounting illumination changes is adequate for both image retrieval and video segmentation tasks. We develop a feature vector of only 36 values that can also be em used for both these objectives as well as for retrieval of video proxy images from a database. The new image metric is based on a color-channel-normalization step, followed by reduction of dimensionality by going to a chromaticity space. Treating chromaticity histograms as images, we perform an effective low-pass filtering of the histogram by first reducing its resolution via a wavelet-based compression and then by a DCT transformation followed by zonal coding. We show that the color constancy step -- color band normalization -- can...

This paper presents a scheme for performing convolution operation directly on compressed images without decompressing them first. The use of such a scheme is demonstrated and discussed by showing the implementation of the Laplacian-of-Gaussian operator for edge detection. We present a complete evaluation of the different parameters involved in this process and show edge detection results on several real images through our proposed scheme. In each case, it is shown that the proposed scheme of directly performing convolution on the compressed data leads to not only a significant computation speedup but also yields better edges. Keywords: Block DCT convolution, DCT domain, edge detection, image compression, Laplacian-of-Gaussian. I. Introduction The lower data rate of compressed video such as motion JPEG and MPEG offers an attractive lowcost possibility of software-based real time processing of digital video needed in many multimedia applications. As a result, several researchers have ...