Dumais, S. T. (1993) LSI meets TREC: A status report. In: D. Harman (Ed.), The First Text REtrieval Conference (TREC1). National Institute of Standards and Technology Special Publication 500-207, (pp. 137-152).  Home/Search   Document Details and Download   Summary   Related Articles   Check

....patterns in the data. LSI has been applied to a wide variety of learning tasks, such as classification [21] and filtering [8, 9] LSI is a vector space approach for modeling documents, and many have claimed that the technique brings out the latent semantics in a collection of documents [5,7]. LSI is based on well known mathematical technique called Singular Value Decomposition (SVD) The algebraic foundation for Latent Semantic Indexing (LSI) was first described in [5] and has been further discussed by Berry, et al. in [2] 3] These papers describe the SVD process and interpret the ....

....co occurrence can be extended to third, fourth, or n order co occurrence. This work provides the theoretical foundation for understanding the use of limited transitivity in LSI. Eventually, the patterns we are detecting will be used to approximate the SVD algorithm, which is resource intensive [7,8,9], at much lower cost. In [13] we describe an unsupervised learning algorithm that develops clusters of terms, by applying an equivalence relation to the LSI term term matrix. The ultimate goal of the current line of work is a theoretically sound, effective and efficient unsupervised clustering ....

[Article contains additional citation context not shown here]

Dumais, Susan T. 1993. LSI meets TREC: A status report. The First Text REtrieval Conference (TREC1), D. Harman (Ed.), National Institute of Standards and Technology Special Publication 500-207 , pp. 137-152.

....p such that 1 = p = m. Substituting back into (4b) shows that y ij = 0 for all k. The argument thus far depends on our assumption that S 1 S 2 S 3 . S m . When using SVD it is customary to truncate the matrices by removing all dimensions whose singular value is below a given threshold [7]; however, for our discussion, we will merely assume that, if s 1 s 2 . s z 1 s z = s z 1 = s z 2 = s z w s z w 1 . s m for some z and some w = 1, the truncation will either remove all of the dimensions with the duplicate singular value, or keep all of the dimensions with ....

Dumais, Susan T. 1993. LSI meets TREC: A status report. The First Text REtrieval Conference (TREC1), D. Harman (Ed.), National Institute of Standards and Technology Special Publication 500-207 , pp. 137-152.

....vector space. Each position in a vector represents a term, with the value of a position i equal to 0 if the term does not appear in the document, and having a positive value otherwise. Based upon previous research we represent the positive values as the log of the total frequency in that document [Dumais, 1993] weighted by the entropy or noise. The corpus can therefore be looked at as a large term by document (t d) matrix. This matrix is very sparse, as most documents contain only a small percentage of the total number of terms in the matrix. This td matrix represents the relationships between terms ....

S. Dumais. LSI meets TREC: A status report. In D. Hartman, Ed. The first Text REtrieval Conference. NIST special publication 500-215, 105-116, 1993.

....position in a vector represents a term (typically a word) with the value of a position i equal to 0 if the term does not appear in the document, and having a positive value otherwise. Based upon previous research we represent the positive values as the log of the total frequency in that document [7] weighted by the entropy of the term. As a result, the corpus can be looked at as a large term by document (t d) matrix X , with each position x ij corresponding to the presence or absence of a term (a row i) in a document (a column j) This matrix is typically very sparse, as most documents ....

S. Dumais. LSI meets TREC: A status report. In D. Hartman, editor, The first Text REtrieval Conference: NIST special publication 500-215, pages 105--116, 1993.

....We do not have the resources to write new definitions, so we are investigating statistical techniques for finding dependencies between defining terms in the LDOCE definitions. Singular Value Decomposition (SVD) has been used recently for finding the latent semantics in document collections [2, 10]. These techniques are a variation on the vector space model of IR in which the co occurrence of words in documents positions each document in n dimensional space, where n is the number of unique terms in the document collection. A query text is placed in this space, and documents are presented to ....

Susan T. Dumais. LSI meets TREC: a status report. In Proc. Text Retrieval Conference (TREC), Washington, November 1992. National Institute of Standards and Technology Special Publication 500207.

....by NetNews describes an environment, and human authors operating within some culturally defined parameters are the scarce resource. At the core of our study sits a large text analysis software system based primarily on Latent Semantic Indexing (LSI) Furnas, et.al 1988; Deerwester, et.al. 1990; Dumais 1992, 1993) This system reads each post and computes the frequency with which each word appears. These word counts are then used in computing a vector representation for each text. A principal component analysis is performed on this collection of vectors to discover re occurring word sets; these are ....

....replicating term sets which act as memes. We will first overview the LSI technique and then discuss how it discovers memes. LSI was originally proposed and has been extensively studied by Susan Dumais of Bell Communications Research and her colleagues (Furnas, et.al. 1988; Deerwester, et.al. 1990; Dumais 1992, 1993) Peter Foltz investigated the use of LSI in clustering NetNews articles for information filtering (Foltz 1990) Michael Berry and co authors researched a variety of numerical approaches to efficiently perform SVD on large sparse matrices such as those found in text retrieval (Berry 1992; ....

Dumais, S.T. (1992). LSI meets TREC: A status report. In The First Text REtrieval Conference (TREC-1), ed. D. Harman.

....to include document databases, we expect to see relational databases being extended to include relations with schemas such as (DocId, Concept, Prob) saying that a given document addresses a given topic with probability p. This is the basis of the well known technique of latent semantic indexing [9]. This provides practical justification for the use of interval probabilities. Later in this paper, it is shown that in any case, point probabilities cannot be used anyway unless independence assumptions are made. In general, probabilistic reasoning is notoriously tricky. To see this, consider ....

S. Dumais. (1993) LSI Meets TREC: A Status Report, in: Proc. First Text Retrieval Conference (TREC1) , pps 137--152, NIST Special Publication 500-207.

No context found.

Dumais, S. T. (1993) LSI meets TREC: A status report. In: D. Harman (Ed.), The First Text REtrieval Conference (TREC1). National Institute of Standards and Technology Special Publication 500-207, (pp. 137-152).

No context found.

Dumais, S.T. (1993). LSI meets TREC: A status report. In: D. Harman (Ed.), The First Text REtrieval Conference (TREC1), National Institute of Standards and Technology Special Publication 500-207, pp. 137-152.

No context found.

Dumais, S. T. 1993. LSI meets TREC: A status report. The First Text REtrieval Conference (TREC1), D. Harman (Ed.), National Institute of Standards and Technology Special Publication 500-207, pp. 137-152.

No context found.

Susan T. Dumais. LSI meets TREC: A status report. In D. Harman, editor, The First Text REtrieval Conference (TREC-1), National Institute of Standards and Technology Special Publication 500-207, pages 137--152, 1992.

No context found.

Susan T. Dumais. LSI meets TREC: A status report. In D. Harman, editor, The First Text REtrieval Conference (TREC-1), National Institute of Standards and Technology Special Publication 500-207, pages 137--152, 1992.

No context found.

Susan T. Dumais. LSI meets TREC: A status report. In D. Harman, editor, The First Text REtrieval Conference (TREC-1), National Institute of Standards and Technology Special Publication 500-207, pages 137--152, 1992.

No context found.

Susan T. Dumais. LSI meets TREC: A status report. In D. Harman, editor, The First Text REtrieval Conference (TREC-1), National Institute of Standards and Technology Special Publication 500-207, pages 137--152, 1992.

No context found.

Dumais, S.T. LSI meets TREC: A status report. The First Text REtrieval Conference (TREC1), D. Harman (Ed.), National Institute of Standards and Technology Special Publication 500-207 , pp. 137-152. 1993.

No context found.

Dumais, S. T. 1993. LSI meets TREC: A status report. The First Text REtrieval Conference (TREC1), D. Harman (Ed.), National Institute of Standards and Technology Special Publication 500-207 , pp. 137-152.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC