The problem of error control and concealment in video communication is becoming increasingly important because of the growing interest in video delivery over unreliable channels such as wireless networks and the Internet. This paper reviews the techniques that have been developed for error control and concealment in the past ten to fifteen years. These techniques are described in three categories according to the roles that the encoder and decoder play in the underlying approaches. Forward error concealment includes methods that add redundancy at the source end to enhance error resilience of the coded bit streams. Error concealment by postprocessing refers to operations at the decoder to recover the damaged areas based on characteristics of image and video signals. Finally, interactive error concealment covers techniques that are dependent on a dialog between the source and destination. Both current research activities and practice in international standards are covered.

In just a few years the "Internet Multicast Backbone", or MBone, has risen from a small, research curiosity to a large scale and widely used communications infrastructure. A driving force behind this growth was our development of multipoint audio, video, and shared whiteboard conferencing applications that are now used daily by the large and growing MBone community. Because these real-time media are transmitted at a uniform rate to all the receivers in the network, the source must either run below the bottleneck rate or overload portions of the multicast distribution tree. In this dissertation, we propose a solution to this problem by moving the burden of rate-adaptation from the source to the receivers with a scheme we call Receiver-driven Layered Multicast, or RLM. In RLM, a source distr...

Real-time multimedia streams like audio and video are now integral data types in modern programming environments. Although a great deal of research has investigated effective and efficient programming support for manipulating such streams and although the design of digital media "middleware" is fairly well understood, no widely available or commonly accepted programming model exists within the research community. We believe this lack of common practice impedes our collective progress because it prevents disparate research groups from easily leveraging each other's work. In this paper, we propose a solution to this problem that combines the best features of a number of existing multimedia toolkits --- Berkeley's Continuous Media Toolkit, MIT's VuSystem, and the LBL/UCB MBone tools --- into a fine-grained, extensible, and highperformance toolkit. We describe the convergence of these three toolkits into a common programming infrastructure and argue that the availability and acceptance of ...

This paper addresses the problem of processing motion-JPEG video data in the compressed domain. The operations covered are those where a pixel in the output image is an arbitrary linear combination of pixels in the input image, which includes convolution, scaling, rotation, translation, morphing, de-interlacing, image composition, and transcoding. This paper further develops an approximation technique called condensation to improve performance and evaluates condensations in terms of processing speed and image quality. Using condensation, motion-JPEG video can be processed at near real-time rates on current generation workstations. 1 Introduction Processing video data is problematic due to the high data rates involved. Television quality video requires approximately 100 GBytes for each hour, or about 27 MBytes for each second. Such data sizes and rates severely stress storage systems and networks and make even the most trivial real-time processing impossible without special purpose ha...

This paper presents techniques for processing motion-JPEG video data at near realtime rates on current generation workstations. The processing that is computable using the techniques developed in this paper belong to the class of operations called linear global digital video effects (LGDVEs), wherein a pixel in the output image is computed using a linear combination of pixels in the input image. Many important video processing problems, including convolution, scaling, rotation, translation, morphing, de-interlacing, image composition, and transcoding use LGDVEs. This paper shows how to compute LGDVEs on video without decompressing the data, improving performance over traditional methods by avoiding compression and decompression and by reducing the amount of data processed. An approximation technique called condensation is developed that further improves performance. 1 Introduction Many challenging and interesting problems faced by video applications are related to the volume of data ...

Maestro is a middleware support tool for distributed multimedia and collaborative computing applications. These applications share a common need for managing multiple subgroups while providing possibly different quality-of-service guarantees for each of these groups. Maestro’s functionality maps well into these requirements, and can significantly shorten the development time of such applications. In this paper, we report on Maestro, and demonstrate its utility in implementing several multimedia and collaborative computing applications. In particular, we provide a detailed description of the implementation of IMUX, a pseudo X-server (proxy) for collaborative computing applications that is based on Maestro. Copyright © 1999 John Wiley & Sons, Ltd. KEY WORDS: distributed multimedia; collaborative computing; middleware; group communication; quality of service

The MPEG format is widely used and therefore an attractive vehicle for the distribution of video and audio material over the Internet. However, the hierarchical structure of MPEG systems complicates the task of delivering continuous, synchronized streams of video and audio in a best-effort environment (today's Internet). If the network throws away packets on encountering congestion, the video and audio stream may loose synchronization for a number of frames. Therefore, adapting the resource demands of an MPEG system must be done by an entity that is knowledgeable of the MPEG system structure: an MPEG system filter. This paper describes the integration of such an MPEG system filter into a network. Our experience to date indicates that mid-range PCs can host such a filter, and that the filter succeeds in adapting the resource requirements of an MPEG system in response to changes in the network load. 1 Introduction Distribution of a movie via a network is a topic of great interest but al...

Internet video is emerging as an important multimedia application area. Although development and use of video applications is increasing, the ability to manipulate and process video is missing within this application area. Current video effects processing solutions are not well matched for the Internet video environment. A software-only solution, however, provides enough flexibility to match the constraints and needs of a particular video application. The key to a software solution is exploiting parallelism. This paper presents the design of a parallel software-only video effects processing system. Preliminary experimental results exploring the use of temporal parallelism are presented. 1 Introduction Internet packet video is emerging as an important multimedia application area. The Multicast Backbone (MBone) conferencing tool vic and NetMeeting from Microsoft are examples of Internet packet video conferencing tools. RealVideo from RealNetworks, NetShow from Microsoft, and IP/TV from...

We present a new medium composed of arbitrary commands. This command stream is a presentation medium that can be browsed at varying speeds, forwards and backwards, and with random access. Command streams can be synchronized with video, audio, and other media in multimedia presentations. We have used them to implement animation, timed user interaction, and device control. This paper discusses the design and implementation of the command stream. We address the intractability of inverting and skipping arbitrary code by constructing commands that are aware of their position in time and include logic for supporting VCR-style playback options. KEYWORDS Command stream, commands, TclStream, multimedia presentations, reversibility. INTRODUCTION Multimedia presentation systems are gaining widespread popularity. Information providers ranging from schools to news and wire services to product vendors are incorporating audio, video, and images into presentations that educate, inform, and entert...

This paper addresses the problem of sending an encoded video stream over a channel of limited bandwidth. When there is insufficient bandwidth available, some data must be dropped. For many video encodings, some data are more important that others, leading to a natural prioritization of the data. In this paper we give fast algorithms to determine a prioritization which optimizes the visual quality of the received data. By "optimized visual quality," we mean that the expected maximum interval of unplayable frames is minimized. Our results are obtained in a model of encoded video data that is applicable to many encoding technologies. The highlight of the model is an interesting relationship between the play order and dependence order of frames. The property allows fast determination of optimal send orders by dynamic programming and is satisfied by all MPEG sequences. 1 Introduction The transmission requirements for video data and ordinary data are quite different. Ordinary data need no...