Multimedia applications involving the transmission of video over communication networks are rapidly increasing in popularity. Such applications can greatly benefit from adapting video coding parameters to network conditions as well as adapting network parameters to better support the application requirements. These two dimensions can both be viewed as allocating source and network resources to improve video quality. In this paper, we highlight recent advances in optimal resource allocation for real-time video communications over unreliable and resource constrained communication channels. More specifically, we focus on point-to-point coding and delivery schemes in which the sequences are encoded on the fly. We present a high-level framework for resource-distortion optimization. The framework can be used for jointly considering factors across network layers, including source coding, channel resource allocation, and error concealment. For example, resources can take the form of transmission energy in a wireless channel, and transmission cost in a DiffServ-based Internet channel. This framework can be used to optimally trade off resource consumption with end-to-end video quality in packet-based video transmission. After giving an overview of this framework, we review recent work in two areas—energy efficient wireless video transmission and resource allocation for Internet-based applications. Keywords—Cross-layer design, energy efficient, error resilience, distortion estimation, internet video, wireless video. I.

Differentiated Services (DiffServ) is one of the leading architectures for providing quality of service in the Internet. We propose a scheme for real-time video transmission over a DiffServ network that jointly considers video source coding, packet classification, and error concealment within a framework of cost-distortion optimization. The selection of encoding parameters and packet classification are both used to manage end-to-end delay variations and packet losses within the network. We present two dual formulations of the proposed scheme: the minimum distortion problem in which the objective is to minimize the end-to-end distortion subject to cost and delay constraints, and the minimum cost problem which minimizes the total cost subject to end-to-end distortion and delay constraints. A solution to these problems using Lagrangian relaxation and dynamic programming is given. Simulation results demonstrate the advantage of jointly adapting the source coding and packet classification in DiffServ networks.

We consider an energy-efficient video streaming system where source coding, channel coding, and transmission power allocation are jointly designed to compensate for channel errors. Our focus is on streaming applications with relatively strict delay constraints; for such applications, forward error correction (FEC) is the preferred channel coding technique to recover from packet losses. We propose a framework of joint source-channel coding and power allocation (JSCCPA), where resources such as bandwidth and transmission power are optimally allocated to provide unequal error protection (UEP) for achieving the best video quality. An efficient algorithm based on Lagrangian relaxation and dynamic programming is proposed to solve the constrained optimization problem. Simulation results illustrate the advantage of the proposed framework.

Providing quality of service (QoS) guarantees is an important objective in the design of the next-generation wireless networks. In this dissertation, we address the QoS provisioning problem from both the network and the end system perspectives. In the first

When designing an encoder for a real-time video application over a wireless channel, we must take into consideration the unpredictable fluctuation of the quality of the channel and its impact on the transmitted video data. This uncertainty motivates the development of an adaptive video encoding mechanism that can compensate for the infidelity caused either by data loss and/or by the post-processing (error concealment) at the decoder. In this paper, we first explore the major factors that cause quality degradation. We then propose an adaptive progressive replenishment algorithm for a packet loss rate (PLR) feedback enabled system. Assuming the availability of a feedback channel, we discuss a video quality assessment method, which allows the encoder to be aware of the decoder-side perceptual quality. Finally, we present a novel dual-feedback mechanism that guarantees an acceptable level of quality at the receiver side with modest increase in the complexity of the encoder.

In the context of joint source and channel coding, unequal loss protection is often used to make image data more robust to possible packet losses. The allocation of source and code symbols is customarily done so as to maximize the expected PSNR at the receiver. In this paper we propose a new objective function, attempting to maximize PSNR given a constraint on the system probability of failure, so that PSNR is constrained to be above a given threshold with a given probability. This leads to the definition of a hybrid loss protection scheme, and the related allocation algorithm, which is able to satisfy this constraint. Experimental results are reported, related to the transmission of JPEG2000-compressed images over the Internet. It is shown that the proposed hybrid approach outperforms existing algorithms in terms of PSNR, while requiring less computational resources.

Abstract—Characterizing the video quality seen by an end-user is a critical component of any video transmission system. In packetbased communication systems, such as wireless channels or the Internet, packet delivery is not guaranteed. Therefore, from the point-of-view of the transmitter, the distortion at the receiver is a random variable. Traditional approaches have primarily focused on minimizing the expected value of the end-to-end distortion. This paper explores the benefits of accounting for not only the mean, but also the variance of the end-to-end distortion when allocating limited source and channel resources. By accounting for the variance of the distortion, the proposed approach increases the reliability of the system by making it more likely that what the end-user sees, closely resembles the mean end-to-end distortion calculated at the transmitter. Experimental results demonstrate that varianceaware resource allocation can help limit error propagation and is more robust to channel-mismatch than approaches whose goal is to strictly minimize the expected distortion. Index Terms—Error analysis, multimedia communication, teleconferencing, video coding, videophone systems. I.

metrics for measuring the end-to-end distortion in

Although cross-layer has been thought as one of the most effective and efficient ways for multimedia communications over wireless networks and a plethora of research has been done in this area, there is still lacking of a rigorous mathematical model to gain in-depth understanding of cross-layer design tradeoffs, spanning from application layer to physical layer. As a result, many existing cross-layer designs enhance the performance of certain layers at the price of either introducing side effects to the overall system performance or violating the syntax and semantics of the layered network architecture. Therefore, lacking of a rigorous theoretical study makes existing cross-layer designs rely on heuristic approaches which are unable to guarantee sound results efficiently and consistently. In this paper, we attempt to fill this gap and develop a new methodological foundation for cross-layer design in wireless multimedia communications. We first introduce a delay-distortion-driven cross-layer optimization framework which can be solved as a large-scale dynamic programming problem. Then, we present new approximate dynamic programming based on significance measure and sensitivity analysis for high-dimensional nonlinear cross-layer optimization in support of real-time multimedia applications. The major contribution of this paper is to present the first rigorous theoretical modeling for integrated cross-layer control and optimization in wireless multimedia communications, providing design insights into multimedia communications over current wireless networks and throwing light on design optimization of the next-generation wireless multimedia systems and networks.

Abstract — Routing for video transmission over multi-hop wireless networks has gained increasing research interest in recent years. However, most existing works only focus on how to satisfy the network-oriented QoS, such as throughput, delay, and packet loss rate rather than application-oriented QoS such as the userperceived video quality. Although there are some research efforts which use application-centric video quality as the routing metric, the calculation of video quality is based on some predefined rate-distortion function or model without considering the impact of video coding and decoding (including error concealment) on the network path selection and the resulting perceived video quality. Moreover, unlike network-centric routing metrics such as hop count, average delay, or average success probability of packet transmission, end-to-end video distortion cannot be calculated either additively or multiplicatively in a hop-by-hop fashion due to the dependency among packets introduced by error concealment algorithms. As a result, most existing works use either exhaustive search or heuristic methods to find the optimal path, which leads to high computational complexity and/or suboptimal solutions to the routing problem in video transmission. In this paper, we propose an application-centric routing framework for real-time video transmission in multi-hop wireless networks, where expected video distortion is adopted as the routing metric. The major contributions of this work are: 1) the development of an efficient routing algorithm with the routing metric expressed in terms of the expected video distortion and being calculated on-the-fly, and 2) the development of a quality-driven cross-layer optimization framework to enhance the flexibility and robustness of routing by the joint optimization of routing path selection and video coding, thereby maximizing the user-perceived video quality under a given video playback delay constraint. Both theoretical and experimental results demonstrate that the proposed quality-driven applicationcentric routing approach can achieve a superior performance over existing network-centric routing approaches. I.