A comparison of algorithms for maximum entropy parameter estimation Conditional maximum entropy (ME) models provide a general purpose machine learning technique which has been successfully applied to fields as diverse as computer vision and econometrics, and which is used for a wide variety of classification problems in natural language processing. However, the flexibility of ME models is not without cost. While parameter estimation for ME models is conceptually straightforward, in practice ME models for typical natural language tasks are very large, and may well contain many thousands of free parameters. In this paper, we consider a number of algorithms for estimating the parameters of ME models, including iterative scaling, gradient ascent, conjugate gradient, and variable metric methods. Surprisingly, the standardly used iterative scaling algorithms perform quite poorly in comparison to the others, and for all of the test problems, a limited-memory variable metric algorithm outperformed the other choices.

Log-linear models provide a statistically sound framework for Stochastic "Unification-Based" Grammars (SUBGs) and stochastic versions of other kinds of grammars. We describe two computationally-tractable ways of estimating the parameters of such grammars from a training corpus of syntactic analyses, and apply these to estimate a stochastic version of LexicalFunctional Grammar.

Statistical analysis of images reveals two interesting properties: (i) invariance of image statistics to scaling of images, and (ii) non-Gaussian behavior of image statistics, i.e. high kurtosis, heavy tails, and sharp central cusps. In this paper we review some recent results in statistical modeling of natural images that attempt to explain these patterns. Two categories of results are considered: (i) studies of probability models of images or image decompositions (such as Fourier or wavelet decompositions), and (ii) discoveries of underlying image manifolds while restricting to natural images. Applications of these models in areas such as texture analysis, image classification, compression, and denoising are also considered.

The statistics of range images from natural environments is a largely unexplored eldofresearch. It closely relates to the statistical modeling of the scene geometry in natural environments, and the modeling of optical natural images. We have use d a 3D laser range- nder to collect range images from mixed forest scenes. The images are hereanalyzed with respect to di erent statistics. 1

processes involved in language learning, production, and comprehension. Computational linguists believe that the essence of these processes (in humans and machines) is a computational manipulation of information. Computational psycholinguistics studies psychological aspects of human

Abstract—We propose to model the statistics of natural images, thanks to the large class of stochastic processes called Infinitely Divisible Cascades (IDCs). IDCs were first introduced in one dimension to provide multifractal time series to model the so-called intermittency phenomenon in hydrodynamical turbulence. We have extended the definition of scalar IDCs from one to N dimensions and commented on the relevance of such a model in fully developed turbulence in [1]. In this paper, we focus on the particular 2D case. IDCs appear as good candidates to model the statistics of natural images. They share most of their usual properties and appear to be consistent with several independent theoretical and experimental approaches of the literature. We point out the interest of IDCs for applications to procedural texture synthesis. Index Terms—Stochastic processes, picture/image generation, fractals, image processing and computer vision, statistical, image models. 1

. We study a model for scale invariance of natural images based on the idea of images as collages of statistically independent objects. The model takes occlusions into account, and shows an excellent qualitative, and good quantitative agreement with empirical data from natural images. At this point, the random-collage model is the only model which comes close to duplicating the simplest, elementary statistics of natural images --- for example, the scale invariance property, the full co-occurrence statistics of two pixels, and the joint statistics of pairs of Haar wavelet responses. Keywords: natural images, stochastic model, image statistics, scaling, randomcollage model 1.

We present a model for scale invariance of natural images based on the ideas of images as collages of statistically independent objects. The model takes occlusions into account, and produces images that show translational invariance, and approximate scale invariance under block averaging and median filtering. We compare the statistics of the simulated images with data from natural scenes, and find good agreement for short-range and middle-range statistics. Furthermore, we discuss the implications of the model on a 3D description of the world. 1 Introduction One of the most remarkable properties of natural images is an invariance to scale. Scale invariance is interesting because it distinguishes natural scenes from random noise and many man-made images. Scale invariance is also a very robust property of natural images; it depends little on calibration [6], and has been observed in images from very different environments, e.g. scenes from the woods [5], or scenes of mountains, cities, ...

This paper summarizes our recent work in developing statistical models of language which are compatible with the kinds of linguistic structures posited by current linguistic theories. In a series of papers we have developed tools for estimating or \learning" such models from data (Johnson et al., 1999; Johnson and Riezler, 2000; Riezler et al., 2000) and this paper provides a high-level overview of both the general approach and the methods we developed