H.J. Zhang, C.Y. Low, S.W. Smoliar and J.H. Wu, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, in Proc. of ACM MM 1995.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....of 3D polyhedral objects independent of viewpoint. 1. Introduction Histograms have been used widely in image analysis and recognition. Swain and Ballard in [14] used them to identify 3D objects. Currently, they are an important tool for the retrieval of images and video from databases [3] 10] [15]. Some of the reasons for their wide applicability are that they can be computed easily and fast, they achieve significant data reduction, and they are robust to local image transformations. Furthermore, color properties can be related to functionality, and must be considered for a complete ....

H. Zhang, C. Low, W. Smoliar, and J. Wu. Video parsing, retrieval and browsing: An integrated and content--based solution. ACM Multimedia, pages 15--24, 1995.

.... of limited and meaningful informations is a way to resolve a set of challenging problems for recently emerging multimedia applications: video browsing and navigation, content based indexing, video summarization and trailers, storage and transmission bandwidth of digitized video information [14] [9] The access to video is still a hard task due to video s length and unstructured format. Video abstraction and summarization techniques are needed to solve this diculty. Shot boundary detection and key frame extraction are two bases for abstraction and summarization techniques [15] 1] ....

....of shot boundary detection is to segment the video stream into multiple shots. There exist many already e ective shot boundary techniques [2] Beyond the shot level an abstraction level could be constructed by mapping the entire shot to a small number of representative frames, called key frames [14]. Indeed, an index may be constructed from key frames, and retrieval may be directed at key frames, which can subsequently be displayed for browsing purposes. This paper focuses on the key frame extraction techniques. There exist many di erent approaches to extract key frames [14] 15] 1] ....

[Article contains additional citation context not shown here]

H. J. Zhang, C. Y. Low, S.W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. ACM Multimedia, pages 15-24, 1995.

....to show 6 more detailed keyframes below it. This way, the user can quickly move the mouse cursor over any keyframe displayed on the screen, hierarchically drilling up and down the keyframe set. This particular style of keyframe browsing has earlier been mentioned elsewhere [Mills et al. 92] Zhang et al. 95] There are, of course, male users as well. Figure 2: Slide Show browser Figure 3: Hierarchical browser Figure 1: Timeline browser 2.1.2 The Evaluation Suite We used a specially designed, automated web based evaluation suite for the interactive testing, which integrated all the test users ....

Zhang, H., Low, C., Smoliar, S. and Wu, J. Video parsing, retrieval and browsing: an integrated and content-based solution. Proceedings of 3rd ACM International Conference on Multimedia (MM '95), San Francisco, CA, 5-9 November, 1995, 503-512.

....of greater importance, for the purpose of facilitating user s access to the video content. The automatic video structuring represents the fundamental task for many recently emerging multimedia applications: video browsing and navigation , content based indexing and video summarization and trailers [17][7] Commonly, two basis tasks for identifying the low level structure of a video document are performed rst: shot boundary detection and key frames extraction. A shot is de ned as an unbroken sequence of frames recorded from a single camera, which forms the building block of a video. Beyond ....

....extraction. A shot is de ned as an unbroken sequence of frames recorded from a single camera, which forms the building block of a video. Beyond the shot level an abstraction level can be constructed by mapping the entire shot to a small number of representative frames, called key frames [17]. The drawbacks of low level structures, shots and key frames, are that they (1) contain too many entries to be eciently presented to the user for example there are 3225 shots This work is supported by Alcatel CRC Grant Alcatel Inria No. 198G098. Demos of this work are available at ....

H. J. Zhang, C. Y. Low, S.W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. ACM Multimedia, pages 15-24, 1995. 11

....segmentation techniques. A typical automatic video segmentation consists of three important steps. The rst step is shot boundary detection (SBD) A shot boundary is declared if a dissimilarity measure between consecutive frames exceeds a threshold value. Some of the recent SBD techniques are [3, 7, 1, 2, 8 13]. The second step is keyframe selection. For each shot, one or more frames that best represent the shot, termed key frame(s) are extracted to reduce the processing overhead in the next step. For instance, key frames can be selected from one or more pre determined location(s) in a shot. More ....

....processing overhead in the next step. For instance, key frames can be selected from one or more pre determined location(s) in a shot. More frames in the same shot can be selected if they are visually di erent than the previously selected key frames. Recent key frame selection techniques include [14, 15, 2, 3, 16, 17]. The nal step is scene segmentation that groups related shots into a meaningful high level unit termed scene in this paper. We focus on this step for a narrative lm a lm that tells a story [18] Most movies are narrative. In narrative lms, viewers understand a complex story by identifying ....

Zhang, H.J., Wu, J.H., Zhong, D., Smoliar, S.: Video parsing, retrieval and browsing: An integrated and content-based solution. Pattern Recognition (Special Issue on Image Databases) 30 (1997) 643-658

....into smaller and meaningful units for subsequent browsing and retrieval. The effectiveness of this step is, thus, very crucial. The units typically considered in video segmentation are shots and scenes. A video shot is defined as a collection of video frames recorded from a single camera operation [12, 14, 15, 8] while a sequence of shots unified by a common locale, an individual event, or parallel events constitutes a scene . Typical video segmentation consists of three important steps. In the first step, given an input video, a shot boundary is declared if a similarity measure between the boundary ....

....important steps. In the first step, given an input video, a shot boundary is declared if a similarity measure between the boundary frames exceeds a threshold value. Recent years have seen several notable shot boundary detection techniques (SBDs) that either process compressed video data directly [14, 13, 10] or uncompress videos first before processing them [15, 4, 5, 1, 12, 7] In the subsequent step, one or more frames that best represent the shot, termed keyframe (s) are extracted for each shot to reduce the amount of processing overhead in the next step. Finally, shots are grouped into scenes ....

H. J. Zhang, J. H. Wu, D. Zhong, and S. Smoliar. Video parsing, retrieval and browsing: An integrated and contentbased solution. Pattern Recognition (Special Issue on Image Databases), 30(4):643--658, April 1997.

....videos are automatically segmented into a number of logically meaningful video clips by a two step algorithm based on video and audio contents. A subtitle decoder and a word spotter is being incorporated into the system to extract textual information to index video clips by their contents. SWIM [48] is an integrated system for parsing, retrieval, and browsing. The video is segmented into shots and each shot is abstracted into key frames. Visual features, such as colour and texture, are used to represent the contents of key frames. In addition, temporal information is obtained from variations ....

....shot, a small number of representative frames are extracted. In the other approach, called mosaicking, the frames in the shot are superimposed on top of each other to obtain salient stills. Key frame extraction The key frame extraction process is usually rule based. In the work by Zhang et al. [48], the first and last frames are always extracted from the shot. The number of keyframes in addition to these two is specified by the user. Ardizzone et al. 2] extract one frame for every second of the video segment. Arman et al. 5] extract what they call Rframes. Each Rframe consists of a body, ....

[Article contains additional citation context not shown here]

H.J. Zhang, C. Y. Low, S. W. Smoliar, J. H. Wu. Video Parsing, Retrieval and Browsing: An integrated and Content-Based Solution. Proc. of ACM Multimedia'95, San Francisco, Nov.7--9, 1995.

....results that clearly show the bene ts of our approach. Lastly, we give our concluding remarks in Section 5. 2. RELATED WORK Two types of shot boundaries are sharp cuts and gradual transitions. A sharp cut generally corresponds to an abrupt change or camera switch between two consecutive frames [8, 9]. A gradual transition is typically caused by camera movements (i.e. panning, tilting, or zooming) or editing e ects inserted between two cuts. Some examples of editing e ects include fades, wipes, and dissolves. In sensor based videos, there are no editing e ects and few camera movements. ....

....susceptibility to the choice of thresholds [10, 9, 11] Besides this common characteristic, recent existing SBD techniques can be distinguished from one another according to the following criteria. Whether the technique works with uncompressed data 1 [11, 2, 12, 13, 1] or compressed videos [8, 14]. Techniques working directly with compressed videos have bene ts of avoiding a decompression overhead. Whether the technique uses global or local histogram comparison [8, 13, 1, 2] pixel matching [15] camera tracking [11] or other derived features such as likelihood ratio [1] production ....

[Article contains additional citation context not shown here]

H. J. Zhang, J. H. Wu, D. Zhong, and S.W. Smoliar. Video parsing, retrieval and browsing: An integrated and content-based solution. Pattern Recognition (Special Issue on Image Databases), 30(4):643-658, April 1997.

....Universal Server, UDFs of NCR Teradata Object Relational database system, Cartridges of Oracle 8, or Extenders of IBM DB2. The second trend is to build special purpose systems from scratch to support content based queries for only a single media type, such as QBIC [13] and Virage[3] for images, [21, 27, 9, 8, 23] for video, and [1, 6] for audio to name a few. Finally, the third approach is to develop special purpose systems for a specific application which might support content based queries on multiple media types transparently from the user (e.g. Informedia [7] project at CMU) At the USC Integrated ....

H. Zhang, C. Y. Low, S. W. Smoliar, and D. Zhang. Video parsing;retrieval and browsing: An integrated and content-based solution. ACM Multimedia, pages 15--24, 1995. 21

....if they are indexed by differenthumans [Mar77] and they differ for videos, too. To obtain a low cost homogeneous classification procedure, automatic video indexing and processing algorithms are needed. Many approaches in video retrieval propose an image retrieval process on video stills (see [ZLSW95] CZKA96] CSM 97] The authors report that good results in content analysis were found using a low level image analysis of video shots. Note that this observation refers to retrieval tasks only, mostly based on similarity queries. For browsing purposes, a conceptual access to video data ....

H.J. Zhang, C.Y. Low, S. W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of ACM MM, pages 15--24, 1995.

....engine and the client caching as the crucial parts of the browsing system architecture. For more information and related work of the HERMES consortium we refer to [1] 3 The Video Retrieval Engine Many approaches in video retrieval propose an image retrieval process based on video stills (cf [28], 4] 5] The authors report that good results in content analysis were found using a low level image analysis of video shots. Note that this result refers to retrieval tasks only, mostly based on similarity queries. For browsing purposes, additional functionality is needed, as wewill describe ....

H. Zhang, C. Low, S. W. Smoliar, and J. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of ACM MM, pages 15--24, 1995.

....building access structures for data. In our approach to automatic conceptual access for browsing, we attempt to employ indexing rules that capture the semantic content to a certain extent [32] Our approach enabling conceptual queries on videos is based on image retrieval applied to video stills [48], 7] 8] It can be divided into the following steps: first, a video is segmented into single shots by a shot detection algorithm. The video stills indexing (analysis) system then employs a number of feature detection algorithms on selected frames. The results of these algorithms called the ....

H. Zhang, C. Low, S. W. Smoliar, and J. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In ACM Multimedia, 1995.

....advanced algorithms outlined above hinge on the performance of the retrieval engine, we will now have a closer look at this part of our system. 4 The Video Retrieval Engine: Enabling Conceptual Queries Many approaches to video retrieval propose an image retrieval process based on video stills ( ZLSW95] CSM 97] CZKA96] The authors report that good results in content analysis were found using low level image analysis of video shots. Note that this result refers to retrieval tasks only, mostly based on similarity queries. For browsing purposes, additional functionality is needed, as we ....

H.J. Zhang, C.Y. Low, S. W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of ACM MM, pages 15--24, 1995.

....of 3D polyhedral objects independent of viewpoint. 1. Introduction Histograms have been used widely in image analysis and recognition. Swain and Ballard in [14] used them to identify 3D objects. Currently, they are an important tool for the retrieval of images and video from databases [3] 10] [15]. Some of the reasons for their wide applicability are that they can be computed easily and fast, they achieve significant data reduction, and they are robust to local image transformations. Furthermore, color properties can be related to functionality, and must be considered for a complete ....

H. Zhang, C. Low, W. Smoliar, and J. Wu. Video parsing, retrieval and browsing: An integrated and content--based solution. ACM Multimedia, pages 15--24, 1995.

.... more and more emphasis is placed on search by browsing e.g. Marchionini [6] Most especially in the case of databases with image and video data where subjectivity can be a big problem in indexing identification, search by browsing becomes an even more important way of locating wanted items [7]. It also seems to be a good idea to let the system make more use of the human user s efficient visual image recognition system [8] thus loading some burden off computer. This probably explains why some of the more advanced multimedia systems such as FRANK [9] and MediaKey [10] concentrate much ....

....video. Some of these are commercial products, others prototypes. These systems will be used later in our feature system matrix in section 4. We now present a brief pointer to each of them and more deatails of what features each has are included in section 4. The SWIM (Show What I Mean) system [7] at the National University of Singapore is cumulation of various tools developed for general video archiving purpose including broadcasting companies, film industries, security agencies, libraries and educational purposes. MediaKey [10] is a commercial product from Informedia project (URL: ....

[Article contains additional citation context not shown here]

ZHANG, H. J., LOW, C. Y., SMOLIAR, S. W., and WU, J. H. Video parsing, retrieval and browsing: an integrated and content-based solution. Proceedings of ACM International Conference on Multimedia '95, San Francisco, CA, November 7-9, 1995, 503-512.

....As more and more multimedia data are created and made available, segmentation algorithms can serve the important function of helping summarize this mass of material. There are several advantages of genetic algorithms over current methods for segmentation such as clustering (e.g. see [3] 11] [12]) First, the genetic mechanism is independent of the prescribed evaluation function and can be tailored to support a variety of characterizations based on heuristics depending on genre, domain, user type, etc. Second, evolutionary algorithms are naturally suited for doing incremental segmentation ....

....and are not aimed at segmentation of an image data stream or video. The GSA is fundamentally different from other video segmentation and summarization methods because it makes use of random processes. Other methods include uniform sampling, and clustering (e.g. see [1] 3] 7] 10] 11] [12]) Uniform sampling is simplest but suffers from undesirable repetition and poor access points for browsing. Clustering can produce good results but does not work with the wide range of characterizations that the GSA gets through evaluation functions, and does not support incremental ....

Zhang, H.J., Low, C.Y., Smoliar, S.W., and Wu, J.H. Video parsing, retrieval and browsing: An integrated and contentbased solution. Proceedings of ACM Multimedia '95. ACM Press, pp. 15-24.

....Universal Server, UDFs of NCR Teradata Object Relational database system, Cartridges of Oracle 8, or Extenders of IBM DB2. The second trend is to build special purpose systems from scratch to support content based queries for only a single media type, such as QBIC [13] and Virage[3] for images, [21, 26, 9, 8, 22] for video, and [1, 6] for audio to name a few. Finally, the third approach is to develop special purpose systems for a specific application which might support content based queries on multiple media types transparently from the user (e.g. Informedia [7] project at CMU) At the USC Integrated ....

H. Zhang, C. Y. Low, S. W. Smoliar, and D. Zhang. Video parsing;retrieval and browsing: An integrated and content-based solution. ACM Multimedia, pages 15--24, 1995. 10

....Data to Specify Browsing Interfaces Assembling these analyses of video data into one leads to a 3 dimensional space where each axis represents each dimension drawn above. In this space we can locate positions of several existing video browsing interfaces, for example SWIM hierarchical browser [28], DVB VCR [24] and AT TV [23] see Figure 4 below) Spatial Visualisation Temporal Visualisation Temporal vs. Spatial Visualisation Fig. 3. Temporal vs. Spatial Visualisation dimension and its two values Fig. 4. Dimension Space and three example interfaces positions Relative Time ....

....over one of the keyframes, another 6 keyframes(lower level) within selected keyframe are immediately displayed. Thus by using the mouse the user can browse the hierarchically broken down keyframe structure in a highly interactive way. This idea has been developed and implemented earlier [6] [28]. Some user comments: I d expect the structure precisely broken down by topic. 5.2 Further Improvements and New UI Additions The dimension space is useful for us to come up with new browsing interfaces. For example, in the space we can think of a new interface by dragging the Scroll Bar ....

Zhang, H., Low, C., Smoliar, S., Wu, J.: Video parsing, retrieval and browsing: an integrated and content-based solution. Proceedings of ACM Multimedia '95 (1995) 503-512.

....similar to human aided segmentation. Experimental results show that the method is reasonably better than existing methods, and has the potential to be used in other CBRS related applications. 1. INTRODUCTION Image content based retrieval systems (CBRS) have recently been gaining attention [1 3]. The low level image features currently used in CBRS include color, texture, and shape. Global image features such as the global color histogram, global texture features (coarseness, contrast, and directionality) and so on [1,4] are easily computed. However, local features are needed to support ....

H. J. Zhang, etal. Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, Submitted to ACM Multimedia, San Francisco, Nov. 5-9, 1995.

....where we have seen a shift from retrieval based on classification by means of indices towards full text retrieval [10] So, what is needed are retrieval techniques that act directly on the video content. These techniques are commonly denoted as content based retrieval techniques ( 9] 11] 20] [28]) A widely accepted first step towards content based retrieval is feature extraction. Features are interpretation independent characteristics. Examples are pitch and noise for audio, and color histogram and shape for images. Currently some impressive results have been shown on retrieval of ....

H.J. Zhang, C.Y. Low, S.W. Smoliar and J.H. Wu, Video parsing, retrieval and browsing: an integrated and content-based solution. In Proc. 3rd ACM Int. Multimedia Conf.,San Francisco, USA, `95, p. 15-24

....information, especially images, music, and video, is quickly gaining in importance for business and entertainment. Content based image retrieval (CBIR) is receiving widespread research interest [1] 4] 2] 3] 7] 8] 9] 10] 11] 12] 13] 14] 6] 15] 16] 17] 18] 19] [20]. It is motivated by the fast growth of image databases which, in turn, require ecient search schemes. A search typically consists of a query followed by repeated relevance feedback, where the user comments on the items which were retrieved. The user s query provides a description of the desired ....

H. J. Zhang, C. Y. Low, S. W. Smoliar, and J. H. Wu, \Video parsing, retrieval and browsing: An integrated and contentbased solution," in Proc. of ACM Multimedia'95, Nov 1995.

....in the knowledge base. Further research is needed in order to determine the behaviour of the system as the size of the knowledge base increases, especially since the confusion matrix of the experiments indicates that there might be a problem. 3.1. 3 Clustering and Classification Zhang et al. ZLSW95] present a solution for computer assisted video parsing and content based video retrieval and browsing. This is a different direction of research from the work we discussed above, yet, it has some very interesting and useful applications. The goal here is to temporally segment and abstract a ....

H. J. Zhang, C. Y. Low, S. W. Smoliar, and J. H. Wu. Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution. In ACM Multimedia, San Fransisco, CA, USA, 1995.

....more and more information is represented in various types of media. Databases for managing media data have been individually developed, including image databases, audio databases, and video databases [1] 4] 7] Content based retrieval of media data has become an important research issue. [17] provides strategies to detect shot changes in a video to extract the key frame of each shot for content based retrieval and browsing. The QBIC system [6] supports queryingby example on image and video databases. A logic based language considering spatial and temporal descriptions was designed to ....

H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, Proc. ACM Multimedia, (1995) pp.15-24. This article was processed using the L a T E X macro package with LLNCS style

....styles varied more in the current study, possibly reducing the chance for the participants to habituate to disadvantages of each abstraction. DISCUSSION AND RELATED WORK There has been considerable research on indexing, searching and browsing the rapidly expanding sources of digital video [1,2,5,9,10,12,14,17,18]. These approaches all focus on automatic techniques based on visual and aspects of media, primarily employing image recognition and image processing techniques. Some of them [10,14] use textual information from speech to text software or closed captions. Our study complements these systems by ....

Zhang, H.J., Low, C.Y., Smoliar, S.W. and Wu, J.H. Video parsing, retrieval and browsing: an integrated and contentbased solution. In Proceedings of ACM Multimedia, September 1995, pp. 15-24.

.... this is simplified by selecting the first frame in the shot, as used in an earlier version of Fschlr and also by others [SIMPSON YOUNG 96] CHANG 97] VideoLogger] or selecting the middle frame in the shot [DENG 98] More intelligent ways of selecting representative frames are also found in [ZHANG 95] SMITH 96] CHRISTEL98] 12 In the current version of the Fschlr system we calculate an average colour histogram from every frame in the shot, and compare this average histogram against each actual frame s histogram from the same shot, and we select the one that is closest to the average ....

.... formed and implemented, and will be reflected in the system s modified user interface and interaction mechanism to accommodate higher level segmentation, possibly a hierarchical display of representative frames showing scenelevel as well as shot level contents interactively, as in [MILLS 92] and [ZHANG 95] 5 Conclusion The development of a digital video recording, analysis and browsing tool which operates on real TV broadcasts from 8 TV channels and made available to real users who have real interest and motivation for using the system allows us to conduct real experiments in the area of ....

ZHANG H, LOW C, SMOLIAR S, WU J. Video parsing, retrieval and browsing: an integrated and content-based solution. Proceedings of ACM International Conference on Multimedia '95, San Francisco, CA, November 5-9, 1995, 15-24.

....about access patterns of previous users. We present results from user studies that compare these automatically generated summaries to author generated summaries. Several techniques have been proposed for summarizing and skimming multimedia content to enable quick browsing. In one class of schemes [1,2,18,25,26], a static storyboard of thumbnail images is built from the video channel using information such as amount of motion or the newness of visual context. These techniques are less applicable to informational presentations, in which the video channel consists primarily of a talking head and most of ....

....even with the early stage summarization technology presented in this paper, users may indeed find considerable satisfaction from the summaries. 7 DISCUSSION AND RELATED WORK There has been considerable research on indexing, searching and browsing the rapidly expanding sources of digital video [1,2,18,25,26]. These approaches all focus on visual aspects of media, primarily employing imagerecognition and image processing techniques. This study focuses on informational talks accompanied by slides. It has several characteristics: 1) audio carries most of the information; 2) slides provide a logical ....

Zhang, H.J., Low, C.Y., Smoliar, S.W. and Wu, J.H. Video parsing, retrieval and browsing: an integrated and contentbased solution. In Proceedings of ACM Multimedia, September 1995, pp. 15-24.

....shows and maintains an online database of stories organized by subject is introduced in [11] In this type of application, input data have to be manipulated or filtered by software rather than hardware because large extent of flexibility in design is required. Similar applications can be seen in [14] and [10] It is possible to make use of these techniques for traditional real time systems. For example, we may want to know if there is any intruder in an isolated area by filtering the data from the remote monitoring camera with a motion detection filter. The detection can be directly connected ....

H. J. Zhang, C. Y. Low, S. W. Smoliar and J. H. Wu, Video Parcing, Retrieval and Browsing: An Integrated and Content-Based Solution, Proceedings of ACM Multimedia, 1995, pp. 15-24.

....about indexing and retrieval of digital video sequences, each concentrating on different aspects. Some employ manual annotation [4] 2] others try to generate compute indices automatically. Automatic indexing generally uses indices based on color, texture, motion, luminance, objects or shape [20], and audio within the video or on external information such as story boards (scripts) and closed captions [7] Others systems are restricted to specific domains: newscasts [19] football or soccer [3] None of them try to extract and recognize automatically the text appearing in digital videos ....

H. J. Zhang, C. Y. Low, S. W. Smoliar, and J. H. Wu. Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution. Proc. ACM Multimedia 95, San Francisco, CA, pp. 15-24, Nov. 1995.

....to create video abstracts automatically and have implemented a prototype system named VAbstract. Other work on this topic is often based on textual abstracts generated from captions (e.g. dV96] or they extract still images only (see e.g. ADHC94] Ror93] TATH95] TATS94] YYWL95] ZSW95] [ZLSW95]) In contrast, we automatically produce a short movie abstract based on automatic content analysis of the video. The paper is structured as follows. Section 2 de nes a video abstract and explains some basic decisions for our abstracting research. Section 3 presents some simple abstracting ....

....usually extracting the images at equal distances (see e.g. MCW92] These images were then reduced in size and displayed in sequence. No attention was paid to semantic issues. There are now more advanced systems of this type (see e.g. ADHC94] Ror93] TATH95] TATS94] YYWL95] ZSW95] [ZLSW95]) Most newer systems rst perform a segmentation of the video based on the detection of scenes (shots) and then select a frame from each scene (shot) Sometimes not merely the rst frame is chosen, but the rst one of good quality or the rst one that satis es certain color or motion ....

H. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proc. of ACM International Conference on Multimedia, pages 1524, San Francisco, California, November 1995. ACM press.

....a similarity measure that can be computed in real time. We test our shape matching method on a set of Roman characters. Results indicate that our method is a feasible solution for real time shape comparison. 1 Introduction Content based retrieval (CBR) has gained considerable attention recently [1, 2, 3, 4, 5]. The most commonly researched image features used in retrieval are color, texture, and shape. Color and texture features are explored in [1, 2, 3, 4, 5] Although shape features have also been studied[1, 5] it is still difficult to obtain a good solution. To address the challenging issues ....

....is a feasible solution for real time shape comparison. 1 Introduction Content based retrieval (CBR) has gained considerable attention recently [1, 2, 3, 4, 5] The most commonly researched image features used in retrieval are color, texture, and shape. Color and texture features are explored in [1, 2, 3, 4, 5]. Although shape features have also been studied[1, 5] it is still difficult to obtain a good solution. To address the challenging issues involved in CBR, the Multimedia Analysis and Retrieval System (MARS) project was started at the University of Illinois [2, 6, 7, 8, 9, 10] MARS supports user ....

[Article contains additional citation context not shown here]

H.-J. Zhang, "Retrieval and browsing: An integrated and content-based solution," in ACM Multimedia'95.

....and motion. There can also be some features specific to a particular application domain and others based on knowledge (e.g. the features of center, left and right fields of a baseball field are stored, and a query such as hits to left fielder is performed based on searching left field ) [38, 18]. An active research area is the study of issues such as which features are best, how these features should be extracted, represented and compared, and what similarity measures are appropriate. One of the main issues that needs to be addressed for supporting content based queries is high ....

H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: an integrated and content-based solution. In Proceedings of ACM Multimedia, San Francisco, CA, pages 15-- 24, November 1995.

....NCR Teradata Object Relational database system, Cartridges of Oracle 8, or Extenders of IBM DB2. The second trend is to build special purpose systems from scratch to support content based queries for only a single media type, such as Figure 1: User Interface QBIC [12] and Virage[3] for images, [20, 27, 8, 7, 21] for video, and [1, 5] for audio to name a few. Finally, the third approach is to develop special purpose systems for a specific application which might support content based queries on multiple media types transparently from the user (e.g. Informedia [6] project at CMU) At our center (name is ....

H. Zhang, C. Y. Low, S. W. Smoliar, and D. Zhang. Video parsing;retrieval and browsing: An integrated and content-based solution. ACM Multimedia, pages 15--24, 1995.

....by Ahanger and Little [1] including a discussion of research trends in video indexing and requirements of future data delivery systems. Topics such as video data indexing, video data modeling, information extraction, and video scene segmentation are also presented. In other work, Zhang et al. [22] describe techniques for use in the pixel domain for dealing with the representation of shot content, as well as content based retrieval techniques using key frames and temporal properties of shots. They also present techniques for video parsing in the pixel domain followed by key frame ....

H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proc. of the ACM Multimedia Conference, pages 15--24, 1995.

....by the information retrieval engine in response to a query. When browsing the library, these news stories are the units of video presented in the results list and played back when selected by the user. This differs from other work that treats video segmentation as a problem of finding scene cuts [Zhang95, Hampapur94] in video. Generally there are multiple scene cuts that comprise a news story, and these cuts do not correspond in a direct way to topic boundaries. Work by Brown et al. [Brown95] clearly demonstrates the necessity of good segmentation. Using a British European version of closed captioning called ....

Zhang, H.J., Low, C.Y., Smoliar S.W., and Wu, J.H., Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, ACM Multimedia-95, p. 15 -- 24, San Francisco, CA 1995.

....extracted from each frame. Shot boundaries are then detected by comparing the edge difference [19] So far, the histogram based approach is the most popular approach. Several researchers claim that it achieves good trade off between accuracy and speed [23] Representatives of this approach are [12, 14, 23, 25, 24]. Two comprehensive comparisons of various shot boundary detection techniques are in [5, 6] ffl Key frame extraction: After the shot boundaries are detected, corresponding key frames can then be extracted. Simple approaches may just extract the first and last frames of each shot as the key frames ....

....Two comprehensive comparisons of various shot boundary detection techniques are in [5, 6] ffl Key frame extraction: After the shot boundaries are detected, corresponding key frames can then be extracted. Simple approaches may just extract the first and last frames of each shot as the key frames [24]. More sophisticated key frame extraction techniques are based on visual content complexity indicator[27] shot activity indicator [8] and shot motion indicator [15] After the shot boundaries are detected and key frames extracted, the sequence of key frames, together with their frame id s, are ....

HongJiang Zhang, C. Y. Low, Stephen W. Smoliar, and D. Zhong. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proc. ACM Conf. on Multimedia, 1995.

....frames. Consequently, neighboring shots portray different actions or events. These techniques for detection of shot boundaries that rely on simple temporal segmentation achieve satisfactory performance. Many such algorithms rely on detecting discontinuities in color and motion between frames [3]. Given the shot boundaries and data partitions, representative frames within each shot are selected to convey the content. These 2 D image frames, called keyframes allow users of retrieval systems to quickly browse over the entire video by viewing frames from selected time samples that ....

....for dealing with video content. Thus, the problem of video retrieval and browsing becomes manageable since it can rely on existing 2 D image database retrieval infrastructure to catalog the keyframes. These techniques are typically based on global analysis of color, texture, and simple motions [3, 4]. These global features are rather effective for general search applications because they describe the characteristics of the key frame as a whole. However, they sometimes do not provide sufficient resolution for object based retrievals and queries based on properties of objects. As a result, ....

[Article contains additional citation context not shown here]

H. J. Zhang, et al, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, Proc. of ACM Multimedia'95, San Francisco, Nov.7-9, 1995, pp.15-24.

.... most fruitful research approach may be to concentrate on facilitating tools,ting low level visual features and content information from audio and close caption data.Such8680 are clearly feasible and research in this direction should ultimately lead to anintelligent0 retrieval and browsing systems[ZLSW95] In this section, we review a variety of video parsing andanalysis39200 0 including temporal segmentation of video, camera motion analysis, video soundtrackanalysis,4028 abstraction approaches and shot similarity for shot comparison and clustering. 3.1 Temporal segmentation of video sequences ....

.... with B frames, as proposed by Zhang, et al. [ZLS95] Processing time could be further reduced while retaining high detection accuracy by using the difference between consecutive I frames as the first filter for potential shot boundaries and applying motion analysis to confirmandm 42 the detection [ZLSW95] It may be argued that shot analysis should be done before MPEG encoding, sinceite 46 possible a much better encoding. The fact that current hardware encodersdode apply it explains why 9 researchers have nonetheless work on shot change detection from MPEG218 It is hoped that MPEG 4 standard ....

[Article contains additional citation context not shown here]

H.J. Zhang, C.Y. Low, S.W. Smoliar and J.H. Wu, Video parsing, retrieval and browsing: an integrated and content-based solution, Proc. ACM Multimedia'95, San Francisco, Nov. 5-9, 1995, pp.15-24.

No context found.

H.J. Zhang, C.Y. Low, S.W. Smoliar and J.H. Wu, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, in Proc. of ACM MM 1995.

No context found.

H.J. Zhang, S.Y. Tan, S.W. Smoliar, and Y. Gong. Video parsing, retrieval and browsing: An integrated and contentbased solution. In ACM Multimedia '95, pages 15--24, 1995.

No context found.

H. Zhang, C. Low, S. Smoliar, J. Wu, Video parsing, retrieval and browsing: An integrated and content-based solution, in: The Third ACM International Conference on Multimedia, 1995, pp. 15--24.

No context found.

Zhang, H. J. Low, C. Y. Smoliar, S. W. Wu, J. H. Video parsing, retrieval and browsing: an integrated and content-based solution, Proceedings of the third ACM international conference on Multimedia. January 1995.

No context found.

H.J. Zhang, S.Y. Tan, S.W. Smoliar, and Y. Gong. Video parsing, retrieval and browsing: An integrated and contentbased solution. In ACM Multimedia '95, pages 15--24, 1995.

No context found.

H.J. Zhang, S.Y. Tan, S.W. Smoliar, and Y. Gong. Video parsing, retrieval and browsing: An integrated and contentbased solution. In ACM Multimedia '95, pages 15--24, 1995.

No context found.

H. Zhang, C. Low, S. Smoliar and J. Wu. Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution. ACM Multimedia Conference, San Francisco, Nov 1995.

No context found.

Zhang, H.; Low, C.; Smoliar, S.; Wu, J. (1995a). Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution. In: Proc. of ACM Multimedia'95.

No context found.

H. J. Zhang, etal. Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution, Submitted to ACM Multimedia, San Francisco, Nov. 5-9, 1995.

No context found.

H. J. Zhang, C. Y. Low, S. W. Smoliar and J. H. Wu, Video Parsing, Retrieval andBrowsing: An Integrated and ContentBased Solution, Proceedings of ACM Multimedia, 1995, pp. 15-24. 14

No context found.

H. J. Zhang, C. Y. Low, S. W. Smoliar, and J. H. Wu. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of The ACM International Multimedia Conference, pages 15---25, San Francisco, CA, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC