There has recently been a great deal of interest in the development of algorithms that objectively measure the integrity of video signals. Since video signals are being delivered to human end users in an increasingly wide array of applications and products, it is important that automatic methods of video quality assessment (VQA) be available that can assist in controlling the quality of video being delivered to this critical audience. Naturally, the quality of motion representation in videos plays an important role in the perception of video quality, yet existing VQA algorithms make little direct use of motion information, thus limiting their effectiveness. We seek to ameliorate this by developing a general, spatio-spectrally localized multiscale framework for evaluating dynamic video fidelity that integrates both spatial and temporal (and spatio-temporal) aspects of distortion assessment. Video quality is evaluated not only in space and time, but also in space-time, by evaluating motion quality along computed motion trajectories. Using this framework, we develop a full reference VQA algorithm for which we coin the term the MOtion-based Video Integrity Evaluation index, or MOVIE index. It is found that the MOVIE index delivers VQA scores that correlate quite closely with human subjective judgment, using the Video Quality Expert Group (VQEG) FRTV Phase 1 database as a test bed. Indeed, the MOVIE index is found to be quite competitive with, and even outperform, algorithms developed and submitted to the VQEG FRTV Phase 1 study, as well as more recent VQA algorithms tested on this database.

Modeling visual attention — particularly stimulus-driven, saliency-based attention — has been a very active research area over the past 25 years. Many different models of attention are now available, which aside from lending theoretical contributions to other fields, have demonstrated successful applications in computer vision, mobile robotics, and cognitive systems. Here we review, from a computational perspective, the basic concepts of attention implemented in these models. We present a taxonomy of nearly 65 models, which provides a critical comparison of approaches, their capabilities, and shortcomings. In particular, thirteen criteria derived from behavioral and computational studies are formulated for qualitative comparison of attention models. Furthermore, we address several challenging issues with models, including biological plausibility of the computations, correlation with eye movement datasets, bottom-up and top-down dissociation, and constructing meaningful performance measures. Finally, we highlight current research trends in attention modeling and provide insights for future.

Abstract—Recent image quality assessment (IQA) metrics achieve high correlation with human perception of image quality. Naturally, it is of interest to produce even better results. One promising method is to weight image quality measurements by visual importance. To this end, we describe two strategies—visual fixation-based weighting, and quality-based weighting. By contrast with some prior studies we find that these strategies can improve the correlations with subjective judgment significantly. We demonstrate improvements on the SSIM index in both its multiscale and single-scale versions, using the LIVE database as a test-bed. Index Terms—Image quality assessment (IQA), quality-based weighting, structural similarity, subjective quality assessment, visual fixations. I.

Abstract not found

Spatial pooling strategies used in recent Image Quality Assessment (IQA) algorithms have generally been that of simply averaging the values of the obtained scores across the image. Given that certain regions in an image are perceptually more important than others, it is not unreasonable to suspect that gains can be achieved by using an appropriate pooling strategy. In this paper, we explore two hypothesis that explore spatial pooling strategies for the popular SSIM metrics. 1,2 The first is visual attention and gaze direction- ‘where ’ a human looks. The second is that humans tend to perceive ‘poor ’ regions in an image with more severity than the ‘good ’ ones- and hence penalize images with even a small number of ‘poor ’ regions more heavily. The improvements in correlation between the objective metrics ’ score and human perception is demonstrated by evaluating the performance of these pooling strategies on the LIVE database 3 of images.

A decomposition is described, which parameterizes the geometry and appearance of contours and regions of gray-scale images with the goal of fast categorization. To express the contour geometry, a contour is transformed into a local/global space, from which parameters are derived classifying its global geometry (arc, inflexion or alternating) and describing its local aspects (degree of curvature, edginess, symmetry). Regions are parameterized based on their symmetric axes, which are evolved with a wave-propagation process enabling to generate the distance map for fragmented contour images. The methodology is evaluated on three image sets, the Caltech 101 set and two sets drawn from the Corel collection. The performance nearly reaches the one of other categorization systems for unsupervised learning.

We present information-theoretic underpinnings of a computation theory of low-level visual fixations in natural images. In continuation of our prior work on optimal contrast-based fixations [1], we develop an optimum texturebased fixation selection algorithm based on a recent theory of non-stationarity measurement in natural images [2]. Thereafter we propose a simple coupling of the optimal texture-based and contrast-based fixation features to produce a new algorithm called CONTEXT, which exhibits robust performance for fixation selection in natural images. The performance of the fixation algorithms are evaluated for natural images by comparison to randomized fixation strategies via actual human fixations performed on the images. The fixation patterns obtained outperform randomized, GAFFE-based [3], and Itti [4] fixation strategies in terms of matching human fixation patterns. These results also demonstrate the important role that contrast and textural information play in low-level visual processes in the Human Visual System (HVS).

Research on image quality assessment has shown the potential performance enhancement of adding visual attention in objective metrics. However, the use of attentionbased metrics in real-time applications is mainly limited by the complexity of modeling visual attention. Since most of the existing objective metrics are based on the luminance component of images only, we investigate whether also saliency can be modeled only with the luminance component. An eye-tracking experiment was carried out to subjectively assess the contribution of color in saliency. Additionally, two attention models well-known in literature were evaluated on predicting our eye-tracking data. Our experimental results show that human saliency is insensitive to color, but that this is not predicted by the attention models existing in literature. Index Terms — Visual attention, image quality assessment, objective metric, eye-tracking, saliency