M. Collins, S. Dasgupta, and R.E. Schapire. A generalization of principal component analysis to the exponential family. In NIPS*13, 2001.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....sample covariance matrix or by peribrming a singular value decomposition on the matrix of mean centered data. While the centering operations and least squares criteria of PCA are naturally suited to real valued data, they are not generally appropriate for other data types. Recently Collins et al.[5] derived generalized criteria for dimensionality reduction by appealing to properties of distributions in the exponential family. In their framework, the conventional PCA of real valued data emerges naturally from assuming a Gaussian distribution over a set of observations, while generalized ....

....exploits the log odds as the natural parameter of the Bernoulli distribution and the logistic function as its canonical link. In this paper we will refer to the PCA model fbr binary data as logistic PCA, and to its counterpart fbr real valued data as linear (or conventional) PCA. Collins et al.[5] proposed an iterative algorithm for all of these generalizations of PCA, but the optimizations required at each iteration of their algorithm do not have a sirnple closed fbrm for the logistic PCA case. In this paper, we derive an alternating least squares method to estimate the basis vectors and ....

[Article contains additional citation context not shown here]

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal component analysis to the exponential family. In Proceedings of NIPS, 2001.

....v is tanh v and the derivative of log det U is U . Right multiplying (10) by (UV ) and substituting in (11) yields = tanh(V ) UV ) 12) Now since UV X as 0, 12) is equivalent to (9) in the limit of vanishing . 6. 2 Exponential family PCA To duplicate exponential family PCA [13], we can set the prediction link f arbitrarily and let the parameter links g and h be large multiples of the identity. Our Newton algorithm is applicable under the assumptions of [13] and (7) becomes D j UV j (X j f j (UV j ) 13) Equation (13) along with the corresponding modi ....

....is equivalent to (9) in the limit of vanishing . 6.2 Exponential family PCA To duplicate exponential family PCA [13] we can set the prediction link f arbitrarily and let the parameter links g and h be large multiples of the identity. Our Newton algorithm is applicable under the assumptions of [13], and (7) becomes D j UV j (X j f j (UV j ) 13) Equation (13) along with the corresponding modi cation of (8) should provide a much faster algorithm than the one proposed in [13] which updates only part of U or V at a time and keeps updating the same part until convergence before ....

[Article contains additional citation context not shown here]

Michael Collins, Sanjoy Dasgupta, and Robert Schapire. A generalization of principal component analysis to the exponential family. In NIPS, vol. 14. MIT Press, 2002.

....are rarely linear; in particular, sparse beliefs are usually very non linear. We therefore transform the data into a space where it does lie near a linear manifold; the algorithm which does so (while also correctly handling the transformed residual errors) is called Exponential Family PCA (E PCA) [3, 4]. E PCA will allow us to represent POMDPs with only a handful of dimensions, even for belief spaces with thousands of dimensions. We will demonstrate the use of this planning technique on the problem of how to find a person whose location is initially unknown. 2 Finding People The problem we ....

....not enforce positive probability predictions. We use exponential family PCA to address this problem. Other nonlinear dimensionality reduction techniques [9, 10] could also work for this purpose, but would have different domains of applicability. Exponential family Principal Component Analysis [3] (E PCA) varies from conventional PCA by adding a link function, in analogy to generalised linear models, and modifying the loss function appropriately. As long as we choose a link function that corresponds to an exponential family distribution log likelihood, and as long as the link and loss ....

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal components analysis to the exponential family. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14, Cambridge, MA, 2002. MIT Press.

....sample covariance matrix or by performing a singular value decomposition on the matrix of mean centered data. While the centering operations and least squares criteria of PCA are naturally suited to real valued data, they are not generally appropriate for other data types. Recently, Collins et al.[5] derived generalized criteria for dimensionality reduction by appealing to properties of distributions in the exponential family. In their framework, the conventional PCA of real valued data emerges naturally from assuming a Gaussian distribution over a set of observations, while generalized ....

....exploits the log odds as the natural parameter of the Bernoulli distribution and the logistic function as its canonical link. In this paper we will refer to the PCA model for binary data as logistic PCA, and to its counterpart for real valued data as linear (or conventional) PCA. Collins et al.[5] proposed an iterative algorithm for all of these generalizations of PCA, but the optimizations required at each iteration of their algorithm do not have a simple closed form for the logistic PCA case. In this paper, we derive an alternating least squares method to estimate the basis vectors and ....

[Article contains additional citation context not shown here]

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal component analysis to the exponential family. In Proceedings of NIPS, 2001.

....manifold of plausible beliefs embedded in the high dimensional belief space. We introduce a new method for solving large scale POMDPs by taking advantage of belief space sparsity. We reduce the dimensionality of the belief space by exponential family Principal Components Analysis [1], which allows us to turn the sparse, highdimensional belief space into a compact, low dimensional representation in terms of learned features of the belief state. We then plan directly on the low dimensional belief features. By planning in a low dimensional space, we can find policies for ....

....for E PCA may be more complicated than that for locally linear models, it requires many fewer samples of the belief space. For real world systems such as mobile robots, large sample sets may be difficult to acquire. 3. 1 Exponential family PCA Exponential family Principal Component Analysis [1] (E PCA) varies from conventional PCA by adding a link function, in analogy to generalised linear models, and modifying the loss function appropriately. As long as we choose the link and loss functions to match each other, there will exist efficient algorithms for finding U and V given X . By ....

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal components analysis to the exponential family. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14, Cambridge, MA, 2002. MIT Press.

....have proposed analogues to PCA intended to handle discrete or positive only data. Methods include non negative matrix factorization (NMF) 3] probabilistic latent semantic analysis (pLSI) 2] latent Dirichlet al..location (LDA) 4] and a general purpose extension of PCA itself to Bregman distances [5], which are a generalization of Kullback Leibler (KL) divergence. A good discussion of the motivation for these techniques can be found in [2] and an analysis of related reduced dimension models and some of the earlier statistical literature which used simpler algorithms can be found in [6] ....

.... some of the earlier statistical literature which used simpler algorithms can be found in [6] Related models using Dirichlets have been dubbed Dirichlet mixtures and applied extensively in molecular biology [7] A common problem with the earlier formulations of these discrete component analysis [3, 2, 5] is that they fail to make a full probability model of the target data in question, a model where hidden variables, observed data, and assumptions are all clearly exposed. Moreover, the relationship to LDA remains unclear. In this paper I present the problem as a multinomial analogue to PCA, ....

[Article contains additional citation context not shown here]

Collins, M., Dasgupta, S., Schapire, R.: A generalization of principal component analysis to the exponential family. In: NIPS*13. (2001)

No context found.

M. Collins, S. Dasgupta, and R.E. Schapire. A generalization of principal component analysis to the exponential family. In NIPS*13, 2001.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Advances in Neural Information Processing Systems 14, 2002.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Neural Information Processing Systems, volume 14, 2001. 163

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Neural Information Processing Systems,vol- ume 14, 2001.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Proc. 14th Ann. Conf. on NIPS, 2001.

No context found.

Collins, M., Dasgupta, S., Schapire, R.: A generalization of principal component analysis to the exponential family. In: NIPS*13. (2001)

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In NIPS, 2001.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Proc. 14th Ann. Conf. on NIPS, 2001.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Advances in Neural Information Processing Systems 13, 2001.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Advances in Neural Information Processing Systems (NIPS), volume 14, 2001.

No context found.

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal components analysis to the exponential family. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, MA, 2002. MIT Press.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to

No context found.

M. Collins, S. Dasgupta, and R. E. Schapire. A generalization of principal components analysis to the exponential family. In Advances in Neural Information Processing Systems 14, 2002.

No context found.

M. Collins, S. Dasgupta, and R. Schapire. A generalization of principal component analysis to the exponential family. In Advances in Neural Information Processing Systems 13, 2001.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC