Abstract—Due to the explosive growth of the Internet and increasing demand for multimedia information on the web, streaming video over the Internet has received tremendous attention from academia and industry. Transmission of real-time video typically has bandwidth, delay, and loss requirements. However, the current best-effort Internet does not offer any quality of service (QoS) guarantees to streaming video. Furthermore, for video multicast, it is difficult to achieve both efficiency and flexibility. Thus, Internet streaming video poses many challenges. To address these challenges, extensive research has been conducted. This special issue is aimed at dissemination of the contributions in the field of streaming video over the Internet. To introduce this special issue with the necessary background and provide an integral view on this field, we cover six key areas of streaming video. Specifically, we cover video compression, application-layer QoS control, continuous media distribution services, streaming servers, media synchronization mechanisms, and protocols for streaming media. For each area, we address the particular issues and review major approaches and mechanisms. We also discuss the tradeoffs of the approaches and point out future research directions. Index Terms—Application-layer QoS control, continuous media distribution services, Internet, protocol, streaming video,

This article explores the e#ciency of motion-compensated three-dimensional transform coding, a compression scheme that employs a motion-compensated transform for a group of pictures. We investigate this coding scheme experimentally and theoretically. The practical coding scheme employs in temporal direction a wavelet decomposition with motion-compensated lifting steps. Further, we compare the experimental results to that of a predictive video codec with single-hypothesis motion compensation and comparable computational complexity. The experiments show that the 5/3 wavelet kernel outperforms both the Haar kernel and, in many cases, the reference scheme utilizing single-hypothesis motion-compensated predictive coding. The theoretical investigation models this motion-compensated subband coding scheme for a group of K pictures with a signal model for K motion-compensated pictures that are decorrelated by a linear transform. We utilize the Karhunen-Loeve Transform to obtain theoretical performance bounds at high bit-rates and compare to both optimum intra-frame coding of individual motion-compensated pictures and single-hypothesis motion-compensated predictive coding. The investigation shows that motion-compensated three-dimensional transform coding can outperform predictive coding with single-hypothesis motion compensation by up to 0.5 bits/sample. Preprint submitted to Elsevier Science 30 April 2004 Key words: Video Coding, Motion Compensation, Adaptive Wavelets, Lifting, Three-Dimensional Subband Coding of Video 1

Hyperspectral image is a sequence of images generated by hundreds of detectors. Each detector is sensitive only to a narrow range of wavelengths. One can view such an image sequence as a three-dimensional array of intensity values (pixels) within a rectangular prism. This image prism reveals contiguous spectrum information about the composition of the area being viewed by the instrument. The price paid for these high resolution, contiguous spectrum images is an extremely large set of data. These data cause processing, storage and transmission problems. Therefore, some application-specific...

This paper presents an advanced motion threading technique for improved performance in 3D wavelet coding. First, we extend an original motion threading idea of ours to a lifting- based implementation. Methods for enabling fractional-pixel alignment in motion threading and for processing many-to-one pixel mapping and non-referred pixels are proposed to reduce the wavelet boundary effects. Second, we devise an advanced motion threading technique, in which one set of motion vectors is generated for each temporal layer of wavelet coefficients for temporal scalability. In order to reduce the motion overhead information, especially at lowbit rates, several correlated motion prediction modes at the macroblock level are defined to exploit the intra/inter layer correlation in motion vector coding. Finally, rate-distortion optimization is utilized in motion estimation to select the best motion prediction mode for each macroblock. With the newmotion threading technique, we are able to achieve 1.5–6.0 dB gain in average PSNR in 3D wavelet coding over our previous implementation of motion threading.

Hyperspectral images may be treated as a three-dimensional data set for the purposes of compression. Here we present some compression techniques based on a three-dimensional wavelet transform that produce compressed bit streams with many useful properties. These properties are progressive quality encoding and decoding, progressive lossyto-lossless encoding, and progressive resolution decoding. We feature an embedded, block-based, image coding algorithm of low complexity, called SPECK (Set Partitioning Embedded bloCK), that has been proposed originally for single images and is modified and extended to three dimensions. The resultant algorithm, Three-Dimensional Set Partitioning Embedded bloCK (3D-SPECK), efficiently encodes 3D volumetric image data by exploiting the dependencies in all dimensions. We describe the use of this coding algorithm in two implementations, first in a purely quality or rate scalable mode and secondly in a resolution scalable mode. We utilize both integer and floating point wavelet transforms, whereby the former one enables lossy and lossless decompression from the same bit stream, and the latter one achieves better performance in lossy compression. The structure of hyperspectral images reveals spectral responses that would seem ideal candidates for compression by 3D-SPECK. We demonstrate that 3D-SPECK, a wavelet domain algorithm, like other time domain algorithms, can preserve spectral profiles

Abstract—The conventional multiview video coding (MVC) schemes, utilizing both neighboring temporal frames and view frames as possible references, have only shown a slight gain over those using temporal frames alone in terms of coding efficiency. The reason for this is that the neighboring temporal frames exhibit stronger correlation with the current frame and the view frames often fail to be selected as references. This paper proposes an elegant MVC framework using high dimensional wavelet, which rightly matches the inherent high dimension property of multiview video. It also makes a better usage of both temporal and view correlations thanks to the hierarchical decomposition. Besides the proposed framework, this paper also investigates MVC coding from the following aspects. First, a disparity-compensated view filter (DCVF) with pixel alignment is proposed, which can accommodate both global and local view disparities among view frames. The proposed DCVF and the existing motion-compensated temporal filter (MCTF) unify the view and temporal decompositions as a generic lifting transform. Second, an adaptive decomposition structure based on the analysis of the temporal and view correlations is proposed. A Lagrangian cost function is derived to determine the optimum decomposition structure. Third, the major components of the proposed MVC coding are figured out, including macroblock type design, subband coefficient coding, and rate allocation. Extensive experiments are carried out on the MPEG 3DAV test sequences and the superior performance of the proposed MVC coding is demonstrated. In addition, the proposed MVC framework can easily support temporal, spatial, SNR, as well as view scalabilities. Index Terms—Global and local motion alignment, high dimensional wavelet, lifting, multiview video coding (MVC). I.

Recently, there have been a flurry of works on overcomplete motion compensated wavelet coding (OMCWC). In this paper, we address the importance of phase and focus on the design of scalable video coding algorithms within the OMCWC framework. Specifically, our new contributions consist of the following three components: 1) efficient block motion estimation techniques in the wavelet domain including hierarchical and fractional-pel block matching; 2) extend overcomplete motion compensated prediction (MCP) into overcomplete motion compensated temporal filtering (MCTF) to achieve temporal scalability; 3) context modeling strategies for embedded quantization and entropy coding of 3D wavelet coefficients. Experiment results are used to demonstrate that the class of overomplete MCP/MCTF coders are capable of achieving comparable performance to other competing interframe wavelet coders.

This paper makes a comparative study on the various spatial scalable coding frameworks. The frameworks with multiple image-domain motion aligned temporal filtering at various spatial resolutions, named as multi-T+2D, are mainly investigated. First we investigate a multi-T+2D scheme based on redundant frame representation. The cross spatial layer redundancy and prediction methods are discussed. The redundancy brings significant performance loss for schemes providing wide range SNR spatial scalability. To remove the redundancy produced in the multi-resolution temporal filtering while retaining the advantage of spatial-domain motion compensation, a novel non-redundant multi-T+2D scheme is proposed. Performance comparison is given among the discussed frameworks and it shows that the proposed non-redundant multi-T+2D framework has a good performance for fully scalable video coding. We also verify that the redundant multi-T+2D framework with cross spatial layer reconstruction feedback is practical in providing narrow range SNR scalability for each spatial layer. Keywords: spatial scalability, 3D wavelet video coding, motion aligned temporal filtering, multiple resolution MATF, in-scale MATF, cross-layer redundancy, subband coupling 1.

This paper proposes a framework of scalable motion estimation and coding with the structure of multi-layers for 3D wavelet video coding. The motion representation consists of multiple layers. The encoder uses motion of all layers to perform analysis, while the decoder may receive only part of motion for synthesis. Different from other schemes, each layer of motion is a point optimized at a certain range of bit-rate. We observe that the distortion introduced by motion mismatch is highly independent with the rate for texture in a wide range. Therefore, to make the best trade-off between motion and texture under the constraint of a given bit rate, a motion layer decision algorithm is used to find the appropriate number of motion layers to be included into the bit-stream. The proposed framework also supports the spatial and temporal scalabilities of motion. Experimental results show significant improvement at low bit-rates and nearly no loss at high bit-rates with layered motion coding and optimal motion decision. The performance is approaching to the convex hull of those with multiple sets of non-scalable motion. 1.

This paper presents a highly flexible video coding scheme, based on the use of a redundant dictionary of spatiotemporal three-dimensional (3-D) functions. Directionality and anisotropic scaling are key ingredients to the spatial components, which form a rich collection of two-dimensional (2-D) visual primitives. The temporal component is tuned to capture most of the energy in the temporal signal evolution, along motion trajectories in the video sequence. The video coding scheme (MP3D) first computes motion trajectories that are eventually entropy coded and sent as side information to the decoder. It then applies a spatio-temporal decomposition along motion trajectories, using an adaptive approximation algorithm based on matching pursuit (MP). Quantized coefficients and basis function parameters are entropy-coded in a embedded stream that is constructed to respect multiple rate constraints. The geometric properties of the 2-D primitive dictionary allow for flexible spatial resolution adaptation, so that the flexible MP3D stream enables decoding at different spatio-temporal resolutions, and multiple rates. The MP3D scheme is shown to provide rate-distortion performance that are comparable with state-of-the-art schemes, such as H.264, MPEG-4, at low and medium bit rate. However, the use of a redundant dictionary is penalizing at high coding rate, which makes the MP3D algorithm mostly interesting for low rate applications, or as a flexible base layer in hierarchical coding schemes.