Thomas Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....model in which a link is caused by latent factors or topics . They use the Expectation Maximization (EM) Algorithm of Dempster et al. 15] to compute the authority weights of the pages. Their work is based on the Probabilistic Latent Semantic Analysis framework introduced by Hofmann [24], who proposed a probabilistic alternative to Singular Value Decomposition. Hofmann [25] proposes an algorithm similar to PHITS which also takes into account the text of the documents. These algorithms require specifying in advance the number of factors. Furthermore, it is possible that the EM ....

Thomas Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

....and enhancements. Most of the subsequent work follows a similar algebraic approach, manipulating some matrix related to the web graph [4, 3, 13, 16, 2, 1, 15] Recently, there were some interesting attempts in applying statistical, and machine learning tools for computing authority weights [4, 6, 11]. 3. DYNAMICAL SYSTEMS A dynamical system describes a weight propagation scheme on the nodes of a graph. We construct the Base Set as described by Kleinberg [12] Let P denote a set of nodes, where each node corresponds to a page in the Base Set. We derive the graph G on the set P by placing a ....

Thomas Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

....class models (LCM) presented in [6] The underlying principle of the former approach is that in order to recommend new items to a given user, ratings judgments of people in the database with similar interests are used. The latter approach is based on Probabilistic Latent Semantic Analysis [5]. The main advantage of this approach is that it is able to automatically discover preference patterns in user profile data without su#ering the flaw of memory based approaches the inability to account for the fact that one person can be reliable recommender for another person on a subset of ....

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the 15th Conference on Uncertainty in AI, pages 289--296, 1999.

....and demonstrate its application to PCA based approximations of several data sets. 1 Introduction Many practical problems involve modeling large, high dimensional data sets to uncover similarities or latent structure. Linear low rank approximation techniques such as PCA [12] LSA [5] PLSA [6] and generative aspect models [1] are powerful tools for approaching these tasks. They identify (relatively) low dimensional hyperplanes that best approximate the data according to a given noise model. In doing so, they exploit and expose regularities in the data: the hyperplanes represent a ....

....further study. A second broad area for future work is the application of the techniques described here to richer low rank approximation models. While this paper considered the effect of informing PCA, it would be fruitful to examine both the process and effect of informing multinomialbased models [3, 6], fully generative models [1] and local linear embeddings [14] A third area for exploration is the study of potential applications for this approach, which include improved relevance modeling, directed web crawling, and personalized search and recommendation across a wide variety of media. ....

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

....to zero. The were updated with a higher (t) than the other parameters. The precise values were chosen based on preliminary experiments. ACM and SMM were trained to convergence by the EM algorithm, and then by deterministic annealing iterations until convergence, as recommended in [2]. The vMF M was optimized to convergence by the EM algorithm. The dispersion parameter of vMF M, DDC, and VDC, and the annealing parameters of ACM and SMM, were chosen by validation: the models were optimized for a validation set equal in size to the training set. To keep the optimization of ....

....ACM and SMM, were chosen by validation: the models were optimized for a validation set equal in size to the training set. To keep the optimization of the annealing parameter comparable to the cross validation of parameters in the other models, instead of varying it within a run as suggested in [2] it was kept constant and the EM algorithm was run until convergence. Data set Model Mean STD Random DDC 0.56 0.023 features VDC 0.47 0.022 vMF M 0.26 0.014 ACM 0.48 0.015 SMM 0.12 0.006 IDF picked DDC 0.58 0.060 features VDC 0.80 0.048 vMF M 0.18 0.023 ACM 0.23 0.023 SMM 0.08 0.015 ....

T. Hofmann, Probabilistic latent semantic analysis, in Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, Morgan Kaufmann Publishers, San Francisco, CA, pp. 289-296, 1999.

.... with levels l: n l l=1 p(ljc)p(M k jc; l)p(T k jc; l) 2) This is still a multinomial model, but by applying appropriate parameter constraints we can produce a tree like browsing structure [5] It is also easy to formulate the model in terms of aspects and clusters as suggested in [6, 7]. 2.1 Prior specification We follow a hierarchical Bayesian strategy, where the unknown parameters and the allocation variables z are regarded as being drawn from appropriate prior distributions. We acknowledge our uncertainty about the exact form of the prior by specifying it in terms of some ....

T Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, 1999.

.... probabilistic counterpart has been presented e.g. in [16] Another basic algorithm is Latent Semantic Indexing (LSI) 7] in which the data is projected onto a subspace spanned by the most important singular vectors of the data matrix; its probabilistic counterparts have been presented by Hofmann [9] and Papadimitriou [27] LSI is found to capture some of the underlying semantics of textual data, resolving problems of synonymy and polysemy. In recent years, the use of higher order statistics and information theoretic measures has gained popularity in the data analysis community. LSI uses ....

Hofmann, T.: Probabilistic Latent Semantic Analysis, In: Proc. 15th Annual Con?(. on UncertainO' in Artificial Intelligence ( UA I'99), Sweden: Stockhohn, 1999.

....variety of instantiations of each part. The constraints built into MCVQ limit its generality, but also lead to rapid learning and inference, and enable it to scale up to high dimensional data. Finally, MCVQ also closely relates to sparse matrix decomposition techniques, such as the aspect model [8], a latent variable model which associates an unobserved class variable, the aspect z, with each observation. Observations consist of co occurrence statistics, such as counts of how often a specific word occurs in a document. The latent Dirichlet al..location model [9] can be seen as a proper ....

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

....docIDs in the posting lists are bursty . We utilize statistical clustering techniques to group similar documents together based on their term occurrences. By assigning adjacent docIDs to similar documents, the posting lists are made burstier. We use Probabilistic Latent Semantic Analysis (PLSA) [13] to group all the MIT Web documents into 100 clusters. Documents within the same cluster are assigned contiguous docIDs. Clustering improves the compression ratio of adaptive set intersection with gap compression to 75. More than 4 rounds yield little further improvement. Technique ....

T. Hofmann. Probabilistic Latent Semantic Analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

....the SVD is used to map the new text into the latent space and to find out documents which are semantically related with it. Hence, a word distribution (or a scoring function) over the whole vocabulary is estimated, which better fits the observed text, and combined with the background LM. In [7, 6] a probabilistic LSA (PLSA) is instead proposed which maps document word conditional distributions into topic mixtures with hidden variables. In [10] non negative matrix factorization (NMF) 9] is applied, which has been recently proposed as an appealing alternative to SVD to represent semantic ....

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of the 15th Conference on Uncertainty in AI, pages 289--296, Stockholm, Sweden, 1999.

....avoid the O(N ) storage requirements. An AST represents the similarity of pixels within an image across an entire classspecific data set. Pairwise statistics have been used for segmentation previously [3] Recently, work centered on factoring joint distributions has gained increasing attention [7, 8, 9, 10]. Rather than estimating two sets of marginals (conditioned on a latent variable) that explain co occurrence data (e.g. word document pairs) we seek a single set of marginals conditioned on a latent variable (the ROR) that explain our co occurrence data (pixel position pairs) Hence, it is a ....

.... possible to estimate the priors for each region p(r) and the probability of each region producing each pixel p(p i r) The error function we minimize is the KL divergence between the empirically measured S and our parameterized estimate S, E = S i,j log S i,j S i,j # (6) as in [8]. Because our model S is symmetric, this case can be updated with only two rules: p i r) # p(r)p(p j , and (7) r) p(r) p i p(p j . 8) 50 100 150 200 250 300 350 400 450 500 50 100 150 200 250 300 350 400 450 500 Figure 3: The similarity template and the ....

Thomas Hofmann, "Probabilistic Latent Semantic Analysis," UAI (99), Morgan Kaufmann Publishers, Inc., San Francisco, 1999.

.... applications (e.g. analyzing contingency tables) under the names of structural equation models, modi ed path models, and log linear models with latent variables [11] Our methods build most closely on Hofmann and Puzicha s model based clustering de nitions [16, 17] Hofmann s folding in algorithm [15], and Cohn and Hofmann s event space combination models [6] Many recommender systems implicitly make a closed world assumption that all of the items that exist in the world are observed at least once during training. In the aspect model, this assumption manifests itself in the probability ....

....that generate or reconstruct the observed data, in a sense describing or explaining the data. Model parameters are learned using expectation maximization (EM) in an attempt to nd the model most likely to explain the data. Our models are mainly extensions and generalizations of aspect models [15, 16, 17]. Such models are useful for softpartitioning data, naturally incorporating the ability to, for example, soft cluster people by movies (collaborative ltering) or people by content attributes like actors (hybrid collaborative content based ltering) We divide the models into three categories: 1) ....

[Article contains additional citation context not shown here]

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, pages 289-296, 1999.

.... probabilistic counterpart has been presented e.g. in [3] Another basic algorithm is Latent Semantic Indexing (LSI) 4] in which the data is projected onto a subspace spanned by the most important singular vectors of the data matrix; its probabilistic counterparts have been presented by Hofmann [5] and Papadimitriou [6] LSI uses only second order moments of the data and neglects any higher order correlations, so independent component analysis (ICA) type algorithms are in this sense a possible step forward. First approaches of using ICA in the context of text data were presented by Isbell ....

T. Hofmann, "Probabilistic latent semantic analysis," in Proc. 15th Annual Conf. on Uncertainty in Artificial Intelligence (UAI'99), Stockholm, Sweden, 1999.

.... probabilistic counterpart has been presented e.g. in [3] Another basic algorithm is Latent Semantic Indexing (LSI) 4] in which the data is projected onto a subspace spanned by the most important singular vectors of the data matrix; its probabilistic counterparts have been presented by Hofmann [5] and Papadimitriou [6] LSI uses only second order moments of the data and neglects any higher order correlations, so independent component analysis (ICA) type algorithms are in this sense a possible step forward. First approaches of using ICA in the context of text data were presented by Isbell ....

T. Hofmann, "Probabilistic latent semantic analysis," in Proc. 15th Annual Conf. on Uncertainty in Artificial Intelligence (UAI'99), Stockholm, Sweden, 1999.

....P(w z) Figure 1: Graphical representation of the aspect model in A) asymmetric and B) symmetric formulations. 2 2 An Overview of Our Implementation Our implementation closely follows the model and algorithm exposition given in Hofmann s Probabilistic Latent Semantic Analysis (PLSA) paper [3]. 2.1 External Libraries Our implementation relies on three third party libraries, available under separate licenses. These libraries can be downloaded for free from their respective web sites. The rst is a (sparse) matrix package from Ops Research. We use sorting routines from the Cern Colt ....

....is trained using a training set and a validation set. We recommend keeping a third independent data set around for testing (assuming you want to test the model) Testing the performance of a trained aspect model is inherently application speci c. See Hofmann s work in comparing querying methods [3] or our own work in benchmarking recommender systems [7] 2.7 PennAspect Output Format Here is some sample output after training with PennAspect begins: Reading SparseMatrix from . data TRAINEVENTS Reading in sparse matrix with dimensions 944 x 2769 The sparse matrix contains 272903 non null ....

[Article contains additional citation context not shown here]

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, pages 289-296, 1999.

....X B 4 30 Q8: 18) 4 5 R X I B 4 5 Q8: 19) B C 5 5 # R X X Q 5 5 #. 8= Q8: 20) of the EM respectively. We term the method Hierarchical PLSA to stress the connection with Probabilistic Latent Semantic Analysis (PLSA) [4]. It follows the discussion in the Introduction that plain PLSA is a particular case of the more general HPLSA where the hierarchy has a single layer. 2.5. Hierarchical Binomial Symmetric Semantic Analysis (HBSSA) Having assumed the conditional distribution of document word pairs to be binomial ....

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--99), pages 289--296, San Francisco, CA, 1999. Morgan Kaufmann Publishers.

....there be no negative values in the matrix. The authors present a loosely defined probability model, assuming a Poisson noise distribution on the words. However, the authors are unclear as to what distribution they place upon the unconditional probability distributions for the latent variables. In [6] and 6 [7] a probabilistic model for latent semantic analysis and indexing is proposed involving discrete variable latent factors. In this work, a model is defined for a probability distribution on the co occurrence of documents and words given the state of a single multi state, latent class ....

Hofmann, T. "Probabilistic Latent Semantic Analysis", Proc. Uncertainty in AI, UAI 1999, Stockholm.

.... (PCA) has also been shown to have a probabilistic basis and the principal directions of multi variate data emerge as the maximum likelihood solutions of the associated generative model [2] Indeed a probabilistic generative model of Latent Semantic Analysis [7] has recently been introduced in [11]. Many generative models take the form of a mixture model such as that found in probability density estimation [1] 3] In this case the notion of a structural hierarchy of classes (or mixture components) has not been considered in great detail, chie y as the expressive power of a at mixture ....

....of web search results [4] This paper presents a probabilistic mixture model with hierarchic structure for the unsupervised organisation of a collection of documents. The mixtures are based on both standard multinomial event models [18] and probabilistic latent semantic analysers (PLSA) [11] [25] In addition to providing a hierarchic partitioned organization of a document collection the associated generative model allows the derivation of the Fisher kernel for the hierarchy. The Fisher kernel [14] engenders a similarity measure between documents based on the metric space induced by ....

[Article contains additional citation context not shown here]

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Annual Conference on Uncertainty in Articial Intelligence (UAI{99), pages 289-296, San Francisco, CA, 1999. Morgan Kaufmann Publishers. 18

....) 17) and M step p(djc l ) X w ndw p(c l jd; w) 18) p(wjc l ) X d ndw p(c l jd; w) 19) p(c l jc l 1 ) X d X w ndw p(c l jc l 1 ; d; w) 20) of the EM respectively. We term the method Hierarchical PLSA to stress the connection with Probabilistic Latent Semantic Analysis (PLSA) [4]. It follows the discussion in the Introduction that plain PLSA is a particular case of the more general HPLSA where the hierarchy has a single layer. 2.5. Hierarchical Binomial Symmetric Semantic Analysis (HBSSA) Having assumed the conditional distribution of document word pairs to be binomial ....

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--99), pages 289--296, San Francisco, CA, 1999. Morgan Kaufmann Publishers.

....product and community information to target products to consumers. Researchers have developed collaborative recommenders, content based recommenders, and a few hybrid systems. We propose a unified probabilistic framework for merging collaborative and content based recommendations. We extend Hofmann s (1999) aspect model to incorporate three way co occurrence data among users, items, and item content. The relative influence of collaboration data versus content data is not imposed as an exogenous parameter, but rather emerges naturally from the given data sources. However, global probabilistic ....

....each (Basu et al. 1998; Claypool et al. 1999; Good et al. 1999) In this paper, we propose a generative probabilistic model for combining collaborative and content based recommendations in a normative manner. The model builds on previous two way co occurrence models for information filtering (Hofmann, 1999) and collaborative filtering (Hofmann Puzicha, 1999) Our model incorporates three way cooccurrence data by presuming that users are interested in a set of latent topics which in turn generate both items and item content information. Model parameters are learned using expectation maximization ....

[Article contains additional citation context not shown here]

Hofmann, T. (1999). Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 289--296.

....Finally, in topic mixture models [11] a number of language models (e.g. n grams) are trained on documents of various topics and are then combined at runtime. Our approach is closely related to the latter class of topic mixtures in that the proposed model is based on a topic decomposition [9], P (wjh) X t P (wjt)P (tjh) 1) Here t is a latent class variable that is supposed to refer to different topics, P (wjt) are topic specific word probabilities or topic factors and P (tjh) are mixing proportions that depend on the history h; for notational convenience all parameters are ....

.... known as Latent Semantic Analysis (LSA) 7] has been proposed in [1] a detailed implementation is provided in [4] Yet, compared to the LSA approach that makes use of Singular Value Decomposition techniques, our method has the crucial advantage of a strict probabilistic interpretation (cf. [9]) a fact that will be further discussed in Section 4. The model we describe here does not make use of syntax and ignores the order in which words appear. In fact, implicit in (1) is the simplifying assumption that the influence of different topics on the statistical properties of language is ....

[Article contains additional citation context not shown here]

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the 15th Conference on Uncertainty in AI, 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

Hofmann, T. (1999). Probabilistic latent semantic analysis. Proc. of Uncertainty in Artificial Intelligence (UAI'99). Stockholm.

No context found.

Hofmann, T. (1999): Probabilistic Latent Semantic Analysis. Proc. of the 22nd Annual ACM Conference on Research and Development in Information Retrieval, Berkeley, California, 50-57, ACM Press.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, 1999.

No context found.

Thomas Hofmann. Probabilistic Latent Semantic Analysis. In Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--99), pages 289--296, San Francisco, CA, 1999. Morgan Kaufmann Publishers.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

T. Hofmann, "Probabilistic latent semantic analysis," Proceedings of the 9th ACM-SIAM Symposium on Discrete Algorithms, 1998.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, pages 289-296. Morgan Kaufmann Publishers, San Francisco, CA, 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pages 289--296. Morgan Kaufmann, 1999. http://www2.sis.pitt.edu/ dsl/UAI/uai99.html.

No context found.

Thomas Hofmann, "Probabilistic latent semantic analysis," in Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence. 1999, pp. 289--296, Morgan Kaufmann.

No context found.

T Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence,Stockholm, Sweden, July 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99), 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99), 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99), 1999.

No context found.

T. Hofmann, Probabilistic latent semantic analysis, in Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, Morgan Kaufmann Publishers, San Francisco, CA, pp. 289-296, 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Arti cial Intelligence (UAI'99), 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pages 289--296, 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99), 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99),San Francisco, 1999. Morgan Kaufmann Publishers, Inc.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence, 1999.

No context found.

T. Hofmann. Probabilistic latent semantic analysis. In Proc. of Uncertainty in Artificial Intelligence, UAI'99, Stockholm, 1999.

No context found.

Thomas Hofmann. Probabilistic latent semantic analysis. In Proceedings of the Fifteenth Conference on Uncertainty in Articial Intelligence, pages 289-296. Morgan Kaufmann Publishers, San Francisco, CA, 1999.

No context found.

T. Hofmann, "Probabilistic latent semantic analysis," in Proc. of Uncertainty in Artificial Intelligence, UAI'99, (Stockholm), 1999.

No context found.

Hofmann, T. (1998). Probabilistic latent semantic analysis. TR 98-042, International Computer Science Institute.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC