Multimedia applications involving the transmission of video over communication networks are rapidly increasing in popularity. Such applications can greatly benefit from adapting video coding parameters to network conditions as well as adapting network parameters to better support the application requirements. These two dimensions can both be viewed as allocating source and network resources to improve video quality. In this paper, we highlight recent advances in optimal resource allocation for real-time video communications over unreliable and resource constrained communication channels. More specifically, we focus on point-to-point coding and delivery schemes in which the sequences are encoded on the fly. We present a high-level framework for resource-distortion optimization. The framework can be used for jointly considering factors across network layers, including source coding, channel resource allocation, and error concealment. For example, resources can take the form of transmission energy in a wireless channel, and transmission cost in a DiffServ-based Internet channel. This framework can be used to optimally trade off resource consumption with end-to-end video quality in packet-based video transmission. After giving an overview of this framework, we review recent work in two areas—energy efficient wireless video transmission and resource allocation for Internet-based applications. Keywords—Cross-layer design, energy efficient, error resilience, distortion estimation, internet video, wireless video. I.

Abstract — We study hybrid error control for real-time video transmission. The study is carried out using a proposed integrated joint source-channel coding framework, which jointly considers error resilient source coding, channel coding, and error concealment, in order to achieve the best video quality. We focus on the performance comparison of several error correction scenarios, such as forward error correction (FEC), retransmission, and the combination of both. Simulation results show that either FEC or retransmission can be optimal depending on the packet loss rates and network round trip time. The proposed hybrid FEC/retransmission scheme outperforms both. I.

A novel scheme for error-resilient digital video broadcasting, using Wyner-Ziv coding, is presented in this paper. We apply the general framework of systematic lossy source-channel coding to generate a supplementary bitstream that can correct transmission errors in the decoded video waveform up to a certain residual distortion. The systematic portion consists of a conventional MPEG-coded bitstream, which is transmitted over the error-prone channel without any forward error correction. The supplementary bitstream is a low rate representation of the transmitted video sequence generated using Wyner-Ziv encoding. We use the conventionally decoded error-concealed MPEG video sequence as side information to decode the Wyner-Ziv bits. The decoder combines the errorprone side information and the Wyner-Ziv description to yield an improved decoded video signal. We show how this scheme can be used to build a system which achieves graceful degradation without the need of a layered representation of the video waveform.

We consider an energy-efficient video streaming system where source coding, channel coding, and transmission power allocation are jointly designed to compensate for channel errors. Our focus is on streaming applications with relatively strict delay constraints; for such applications, forward error correction (FEC) is the preferred channel coding technique to recover from packet losses. We propose a framework of joint source-channel coding and power allocation (JSCCPA), where resources such as bandwidth and transmission power are optimally allocated to provide unequal error protection (UEP) for achieving the best video quality. An efficient algorithm based on Lagrangian relaxation and dynamic programming is proposed to solve the constrained optimization problem. Simulation results illustrate the advantage of the proposed framework.

Abstract—Efficient adaptation to channel bandwidth is broadly required for effective streaming video over the Internet. To address this requirement, a novel seamless switching scheme among scalable video bitstreams is proposed in this paper. It can significantly improve the performance of video streaming over a broad range of bit rates by fully taking advantage of both the high coding efficiency of nonscalable bitstreams and the flexibility of scalable bitstreams, where small channel bandwidth fluctuations are accommodated by the scalability of a single scalable bitstream, whereas large channel bandwidth fluctuations are tolerated by flexible switching between different scalable bitstreams. Two main techniques for switching between video bitstreams are proposed in this paper. Firstly, a novel coding scheme is proposed to enable drift-free switching at any frame from the current scalable bitstream to one operated at lower rates without sending any overhead bits. Secondly, an switching-frame coding scheme is proposed to greatly re-duce the number of extra bits needed for switching from the current scalable bitstream to one operated at higher rates. Compared with existing approaches, such as switching between nonscalable bitstreams and streaming with a single scalable bitstream, our experimental results clearly show that the proposed scheme brings higher efficiency and more flexibility in video streaming. Index Terms—Bitstream switching, fine granularity scalable video coding, scalable video coding, SP frame, video streaming. I.

Reliable transmission of video over wireless networks must address the limited bandwidth and the possibility of loss. When the bandwidth is insufficient on a single channel, the video can be partitioned over multiple channels with possibly unequal characteristics at the expense of more complex channel coding (error correction). This paper addresses the problem of efficiently partitioning forward error protected, pre-encoded video data for transmission over multiple channels. The assumption of pre-encoding precludes adjustment of source rates to the channels, since it is assumed that channel characteristics are not known until immediately prior to the start of transmission. The proposed partitioning exploits the structure of MPEG video, and frames in each groupof -picture are reordered based on their decoding dependence. To be spectrally efficient, the frames of different types are unequally error protected taking different channel reliabilities into account. A pruned tree search algorithm is implemented to efficiently solve the problem. Simulation results are presented.

Layered (LC) and multiple description coding (MDC) have been proposed as source coding techniques that are robust to channel errors for video transmission. LC and MDC have similar characteristics: they both generate multiple sub-bitstreams, and it is permissible to drop some portion of the data from the sub-bitstreams during transmission for both methods. However, they are different in the sense that the sub-bitstreams for LC have different levels of importance while all sub-bitstreams for MDC are equally important. Since these two encoding techniques have similar properties, some performance comparisons between LC and MDC have recently been reported. However, these studies are still not conclusive because several scenarios have not been carefully considered. Furthermore, they have been performed in different environments. In this paper, we further investigate the error-resilience capabilities of these two encoding techniques through extensive experimentation. Although some of our conclusions agree with those in the literature, we believe that this paper provides the most comprehensive performance comparison yet between LC and MDC.

In this work, our focus is on video streaming applications with relatively strict delay constraints; for such applications, forward error correction (FEC) is the preferred channel coding technique to recover from packet losses. Moreover, since video packets are usually of different importance, optimal bit allocation across video packets results in different packets receiving unequal error protection (UEP).

The need for efficient joint source-channel coding is growing as new multimedia services are introduced in commercial wireless communication systems. An important component of practical joint source-channel coding schemes is a distortion model to measure the quality of compressed digital multimedia such as images and videos. Unfortunately, models for estimating the distortion due to quantization and channel bit errors in a combined fashion do not appear to be available for practical image or video coding standards. This paper presents a statistical model for estimating the distortion introduced in progressive JPEG compressed images due to both quantization and channel bit errors. Important compression techniques such as Huffman coding, DPCM coding, and run-length coding are included in the model. Examples show that the distortion in terms of peak signal to noise ratio can be predicted within a 2 dB maximum error. 1.

During the past two decades, video coding technology has matured and stat-of-the-art coding standards have become very important part of the video industry. Standards such as MPEG-2 [16] and H.264/AVC [20] provide strong support for digital video transmission, storage and streaming applications. Wireless Communication and Mobile Computing