How predictable are human eye movements during search in real world scenes? We recorded 14 observers ’ eye movements as they performed a search task (person detection) in 912 outdoor scenes. Observers were highly consistent in the regions fixated during search, even when the target was absent from the scene. These eye movements were used to evaluate computational models of search guidance from three sources: saliency, target features, and scene context. Each of these models independently outperformed a cross-image control in predicting human fixations. Models that combined sources of guidance ultimately predicted 94 % of human agreement, with the scene context component providing the most explanatory power. None of the models, however, could reach the precision and fidelity of an attentional map defined by human fixations. This work puts forth a benchmark for computational models of search in real world scenes. Further improvements in Please address all correspondence to Aude Oliva, Department of Brain and Cognitive

Modeling visual attention — particularly stimulus-driven, saliency-based attention — has been a very active research area over the past 25 years. Many different models of attention are now available, which aside from lending theoretical contributions to other fields, have demonstrated successful applications in computer vision, mobile robotics, and cognitive systems. Here we review, from a computational perspective, the basic concepts of attention implemented in these models. We present a taxonomy of nearly 65 models, which provides a critical comparison of approaches, their capabilities, and shortcomings. In particular, thirteen criteria derived from behavioral and computational studies are formulated for qualitative comparison of attention models. Furthermore, we address several challenging issues with models, including biological plausibility of the computations, correlation with eye movement datasets, bottom-up and top-down dissociation, and constructing meaningful performance measures. Finally, we highlight current research trends in attention modeling and provide insights for future.

Abstract. Many methods for 3D reconstruction in computer vision rely on probability models, for example, Bayesian reasoning. Here we introduce a probability model of surface visibilities in densely cluttered 3D scenes. The scenes consist of a large number of small surfaces distributed randomly in a 3D view volume. An example is the leaves or branches on a tree. We derive probabilities for surface visibility, instantaneous image velocity under egomotion, and binocular half–occlusions in these scenes. The probabilities depend on parameters such as scene depth, object size, 3D density, observer speed, and binocular baseline. We verify the correctness of our models using computer graphics simulations, and briefly discuss applications of the model to stereo and motion. 1

Most research on the readability of graph visualisation focuses on node-link diagrams of static graphs. But in many applications graphs are not static, but change over time, or graphs are too dense to be drawn as node-link diagrams. In this paper we look at dynamic graph visualisations: We translate the general goal of graph visualisation—to convey the underlying information of a graph—into aesthetic dimensions that are applicable in practice. These aesthetic dimensions help to design, compare, and evaluate dynamic graph visualisations. 1

We introduce a probability model of surface visibilities in densely cluttered 3D scenes. The model assumes the scene consists of a large number of small surfaces distributed randomly in a 3D view volume. An example is the leaves and/or branches on a tree. We derive probabilities for surface visibility, instantaneous image velocity under egomotion, and binocular half–occlusions in these scenes. The probabilies depend on parameters such as scene depth, object size, 3D density, observer speed, and binocular baseline. We verify the correctness of our models using computer graphics simulations and discuss possible applications of this model to various problems in computer vision.

Speakers and listeners in a dialogue establish mutual understanding by coordinating their linguistic responses. When a visual scene is present, scan patterns on that scene are also coordinated. However, it is an open question which linguistic and scene factors affect coordination. In this paper, we investigate the coordination of scan patterns during the comprehension and generation of scene descriptions. We manipulate the animacy of the subject and the number of visual referents associated with it. By using Cross Recurrence Analysis, we demonstrate that coordination emerges only during linguistic processing, and that it is especially pronounced for inanimate unambiguous subjects. When the subject is referentially ambiguous (more than one visual object associated with it), scan pattern variability increases to the extent that the animacy effect is neutralized.

Learning to rank has received great attention in recent years as it plays a crucial role in information retrieval. The existing concept of learning to rank assumes that each training sample is associated with an instance and a reliable label. However, in practice, this assumption does not necessarily hold true. This study focuses on the learning to rank when each training instance is labeled by multiple annotators that may be unreliable. In such a scenario, no accurate labels can be obtained. This study proposes two learning approaches. One is to simply estimate the ground truth first and then to learn a ranking model with it. The second approach is a maximum likelihood learning approach which estimates the ground truth and learns the ranking model iteratively. The two approaches have been tested on both synthetic and real-world data. The results reveal that the maximum likelihood approach outperforms the first approach significantly and is comparable of achieving results with the learning model considering reliable labels. Further more, both the approaches have been applied for ranking the Web visual clutter. 1

How do human observers perceive visual complexity in images? This long-studied problem is especially relevant for computer graphics, where a better understanding of complexity can aid in the development of more advanced perceptually based rendering algorithms. In this project, we describe a study of the dimensionality of visual complexity in computer graphics scenes. We present an experiment where subjects judged the relative complexity of 21 high-resolution building, room, and tabletop scenes, rendered with photorealistic methods. Scenes were gathered from web archives and varied in theme, number and layout of objects, material properties, and lighting. Multidimensional scaling of pooled subject responses embeds the stimulus images in a two-dimensional space, with axes of ”numerosity ” and ”material / lighting complexity”. In a follow-up analysis, we derive a onedimensional complexity ordering of the stimulus images and show discrepancies between this ordering and several computable complexity metrics, such as scene polygon count and JPEG compression size.

This paper proposes an automatic method for predicting the inter-observer visual congruency (IOVC). The IOVC reflects the congruence or the variability among different subjects looking at the same image. Predicting this congruence is of interest for image processing applications where the visual perception of a picture matters such as website design, advertisement, etc. This paper makes several new contributions. First, a computational model of the IOVC is proposed. This new model is a mixture of low-level visual features extracted from the input picture where model’s parameters are learned by using a large eye-tracking database. Once the parameters have been learned, it can be used for any new picture. Second, regarding low-level visual feature extraction, we propose a new scheme to compute the depth of field of a picture. Finally, once the training and the feature extraction have been carried out, a score ranging from 0 (minimal congruency) to 1 (maximal congruency) is computed. A value of 1 indicates that observers would focus on the same locations and suggests that the picture presents strong locations of interest. A second database of eye movements is used to assess the performance of the proposed model. Results show that our IOVC criterion outperforms the Feature Congestion measure [33]. To illustrate the interest of the proposed model, we have used it to automatically rank personalized photograph.

Problematic and context Measuring the IOVC by using eye data IOVC computational model Performance Application to image ranking Conclusion Problematic and context De nition (Inter-observer visual congruency) Inter-observer visual congruency (IOVC) re ects the visual dispersion between observers or the consistency of overt attention (eye movement) while observers watch the same visual scene. Do observers look at the scene similarly? 2 Table of Content Problematic and context Measuring the IOVC by using eye data IOVC computational model Performance Application to image ranking Conclusion Problematic and context