Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis which is based on a linear transformations between the latent space and the data space. In this paper we introduce a form of non-linear latent variable model called the Generative Topographic Mapping for which the parameters of the model can be determined using the EM algorithm. GTM provides a principled alternative to the widely used Self-Organizing Map (SOM) of Kohonen (1982), and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multi-phase oil pipeline. Copyright c○MIT Press (1998). 1

was supported by ONR grants N00014-96-1-0192 and N00014-96-1-0330. The authors are Clustering on principal curves combines parametric modeling of noise with nonparametric modeling of feature shape. This is useful for detecting curvilinear features in spatial point patterns, with or without background noise. Applications of this include the detection of curvilinear mine elds from reconnaissance images, some of the points in which represent false detections, and the detection of seismic faults from earthquake catalogs. Our algorithm for principal curve clustering is in two steps: the rst is hierarchical and agglomerative (HPCC), and the second consists of iterative relocation based on the Classi cation EM algorithm (CEM-PCC). HPCC is used to combine potential feature clusters, while CEM-PCC re nes the results and deals with background noise. It is importanttohave a good starting point for the algorithm: this can be found manually or automatically using, for example, nearest neighbor clutter removal or model-based clustering. We choose the number of features and the amount of smoothing simultaneously using approximate Bayes

We propose a new nonparametric regression method for high-dimensional data, nonlinear partial least squares (NLPLS). NLPLS is motivated by projection-based regression methods, e.g., partial least squares (PLS), projection pursuit (PPR), and feedforward neural networks. The model takes the form of a composition of two functions. The first function in the composition projects the predictor variables onto a lower-dimensional curve or surface yielding scores, and the second predicts the response variable from the scores. We implement NLPLS with feedforward neural networks. NLPLS will often produce a more parsimonious model (fewer score vectors) than projection-based methods, and the model is well suited for detecting outliers and future covariates requiring extrapolation. The scores are also shown to have useful interpretations. We also extend the model for multiple response variables and discuss situations when multiple response variab...

INTRODUCTION Consider a multivariate random variable X in R p with density function f and a random sample from X, namely X 1 , ..., X n . The first principal component can be viewed as the straight line which best fits the cloud of data (see, e.g., [17, pp. 386#387]). When the distribution of X is ellipsoidal the population first principal component is the main axis of the ellipsoids of equal concentration. In the past 40 years many works have appeared proposing extensions of principal components to distributions with nonlinear structure. We cite Shepard and Carroll [24], Gnanadesikan and Wilk [13], Srivastava [27], Etezadi-Amoli and McDonald [10], Yohai, Ackermann and Haigh [33], Koyak [19] and Gifi [12], among others. Some of them look for nonlinear transformations of the observable variables into spaces admitting a doi:10

Abstract. We present a method to simultaneously reconstruct the shape and motion of an unknown smooth convex object. The object is manipulated by planar palms covered with tactile elements. The shape and dynamics of the object can be expressed as a function of the sensor values and the motion of the palms. We present a brief review of previous results for the planar case. In this paper we show that the 3D case is fundamentally different from the planar case, due to increased tangent dimensionality. The main contribution of this paper is a shape-dynamics analysis in 3D, and the synthesis of shape approximation methods via reconstructed contact point curves. 1

Nonlinear principal components analysis (NLPCA) neural networks are feedforward autoassociative networks with five layers. The third layer has fewer nodes than the input or output layers. NLPCA has been shown to give better solutions to several feature extraction problems than existing methods, but very little is know about the theoretical properties of this method or its estimates. This paper studies NLPCA. It proposes a geometric interpretation by showing that NLPCA fits a lower-dimensional curve or surface through the training data. The first three layers project observations onto the curve or surface giving scores. The last three layers define the curve or surface. The first three layers are a continuous function, which I show has several implications: NLPCA "projections" are suboptimal producing larger approximation error, NLPCA is unable to model curves and surfaces that intersect themselves, and NLPCA cannot parameterize curves with parameterizations having discontinuous jumps. ...

Principal manifolds serve as useful tool for many practical applications. These manifolds are defined as lines or surfaces passing through “the middle” of data distribution. We propose an algorithm for fast construction of grid approximations of principal manifolds with given topology. It is based on analogy of principal manifold and elastic membrane. First advantage of this method is a form of the functional to be minimized which becomes quadratic at the step of the vertices position refinement. This makes the algorithm very effective, especially for parallel implementations. Another advantage is that the same algorithmic kernel is applied to construct principal manifolds of different dimensions and topologies. We demonstrate how flexibility of the approach allows numerous adaptive strategies like principal graph constructing, etc. The algorithm is implemented as a C++ package elmap and as a part of stand-alone data visualization tool VidaExpert, available on the web. We describe the approach and provide several examples of its application with speed performance characteristics.

Abstract not found

When beginning the analysis of a new data set of which very little is known a priori, the first step is to explore the data. This paper presents new methods for this preliminary data mining phase: for detecting, visualizing, and interpreting cluster or density structures of the data. The groundwork for the visualizations is a nonlinear ordered map display constructed using the Self-Organizing Map algorithm. The detected structures can be interpreted using linear local factors extracted from the nonlinear map. In a case study using a collection of patent abstracts the methods have, for instance, detected a cluster of neural networks patents not previously distinguished by the international patent classification system.

We present a novel multiscale clustering algorithm inspired by algebraic multigrid techniques. Our method begins with assembling data points according to local similarities. It uses an aggregation process to obtain reliable scale-dependent global properties, which arise from the local similarities. As the aggregation process proceeds, these global properties affect the formation of coherent clusters. The global features that can be utilized are for example density, shape, intrinsic dimensionality and orientation. The last three features are a part of the manifold identification process which is performed in parallel to the clustering process. The algorithm detects clusters that are distinguished by their multiscale nature, separates between clusters with different densities, and identifies and resolves intersections between clusters. The algorithm is tested on synthetic and real data sets, its running time complexity is linear in the size of the data set.