Results 1  10
of
130
Solving illconditioned and singular linear systems: A tutorial on regularization
 SIAM Rev
, 1998
"... Abstract. It is shown that the basic regularization procedures for finding meaningful approximate solutions of illconditioned or singular linear systems can be phrased and analyzed in terms of classical linear algebra that can be taught in any numerical analysis course. Apart from rewriting many kn ..."
Abstract

Cited by 124 (4 self)
 Add to MetaCart
(Show Context)
Abstract. It is shown that the basic regularization procedures for finding meaningful approximate solutions of illconditioned or singular linear systems can be phrased and analyzed in terms of classical linear algebra that can be taught in any numerical analysis course. Apart from rewriting many known results in a more elegant form, we also derive a new twoparameter family of merit functions for the determination of the regularization parameter. The traditional merit functions from generalized cross validation (GCV) and generalized maximum likelihood (GML) are recovered as special cases.
On the Early History of the Singular Value Decomposition
, 1992
"... This paper surveys the contributions of five mathematicians  Eugenio Beltrami (18351899), Camille Jordan (18381921), James Joseph Sylvester (18141897), Erhard Schmidt (18761959), and Hermann Weyl (18851955)  who were responsible for establishing the existence of the singular value de ..."
Abstract

Cited by 122 (1 self)
 Add to MetaCart
This paper surveys the contributions of five mathematicians  Eugenio Beltrami (18351899), Camille Jordan (18381921), James Joseph Sylvester (18141897), Erhard Schmidt (18761959), and Hermann Weyl (18851955)  who were responsible for establishing the existence of the singular value decomposition and developing its theory.
Subspace Algorithms for the Stochastic Identification Problem
, 1993
"... In this paper, we derive a new subspace algorithm to consistently identify stochastic state space models from given output data without forming the covariance matrix and using only semiinfinite block Hankel matrices. The algorithm is based on the concept of principal angles and directions. We descr ..."
Abstract

Cited by 91 (14 self)
 Add to MetaCart
In this paper, we derive a new subspace algorithm to consistently identify stochastic state space models from given output data without forming the covariance matrix and using only semiinfinite block Hankel matrices. The algorithm is based on the concept of principal angles and directions. We describe how they can be calculated with QR and Quotient Singular Value Decomposition. We also provide an interpretation of the principal directions as states of a nonsteady state Kalman filter bank. Key Words Principal angles and directions, QR and quotient singular value decomposition, Kalman filter, Riccati difference equation, stochastic balancing, stochastic realization. 1 Introduction Let y k 2 ! l ; k = 0; 1; : : : ; K be a data sequence that is generated by the following system : x k+1 = Ax k + w k (1) y k = Cx k + v k (2) where x k 2 ! n is a state vector. The vector sequence w k 2 ! n is a process noise while the vector sequence v k 2 ! l is a measurement noise. They are bo...
Generalizing discriminant analysis using the generalized singular value decomposition
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 2004
"... Discriminant analysis has been used for decades to extract features that preserve class separability. It is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters. The requirement that one of these matrices be nonsingular limi ..."
Abstract

Cited by 58 (15 self)
 Add to MetaCart
(Show Context)
Discriminant analysis has been used for decades to extract features that preserve class separability. It is commonly defined as an optimization problem involving covariance matrices that represent the scatter within and between clusters. The requirement that one of these matrices be nonsingular limits its application to data sets with certain relative dimensions. We examine a number of optimization criteria, and extend their applicability by using the generalized singular value decomposition to circumvent the nonsingularity requirement. The result is a generalization of discriminant analysis that can be applied even when the sample size is smaller than the dimension of the sample data. We use classification results from the reduced representation to compare the effectiveness of this approach with some alternatives, and conclude with a discussion of their relative merits. 1
Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition
 SIAM Journal on Matrix Analysis and Applications
, 2003
"... Abstract. In today’s vector space information retrieval systems, dimension reduction is imperative for efficiently manipulating the massive quantity of data. To be useful, this lowerdimensional representation must be a good approximation of the full document set. To that end, we adapt and extend th ..."
Abstract

Cited by 53 (19 self)
 Add to MetaCart
(Show Context)
Abstract. In today’s vector space information retrieval systems, dimension reduction is imperative for efficiently manipulating the massive quantity of data. To be useful, this lowerdimensional representation must be a good approximation of the full document set. To that end, we adapt and extend the discriminant analysis projection used in pattern recognition. This projection preserves cluster structure by maximizing the scatter between clusters while minimizing the scatter within clusters. A common limitation of trace optimization in discriminant analysis is that one of the scatter matrices must be nonsingular, which restricts its application to document sets in which the number of terms does not exceed the number of documents. We show that by using the generalized singular value decomposition (GSVD), we can achieve the same goal regardless of the relative dimensions of the termdocument matrix. In addition, applying the GSVD allows us to avoid the explicit formation of the scatter matrices in favor of working directly with the data matrix, thus improving the numerical properties of the approach. Finally, we present experimental results that confirm the effectiveness of our approach.
An optimization criterion for generalized discriminant analysis on undersampled problems
 IEEE Trans. Pattern Analysis and Machine Intelligence
, 2004
"... Abstract—An optimization criterion is presented for discriminant analysis. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) through the use of the pseudoinverse when the scatter matrices are singular. It is applicable regardless of the relative size ..."
Abstract

Cited by 50 (9 self)
 Add to MetaCart
(Show Context)
Abstract—An optimization criterion is presented for discriminant analysis. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) through the use of the pseudoinverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of classical LDA. The optimization problem can be solved analytically by applying the Generalized Singular Value Decomposition (GSVD) technique. The pseudoinverse has been suggested and used for undersampled problems in the past, where the data dimension exceeds the number of data points. The criterion proposed in this paper provides a theoretical justification for this procedure. An approximation algorithm for the GSVDbased approach is also presented. It reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices to which the GSVD can be applied efficiently. Experiments on text data, with up to 7,000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm. Index Terms—Classification, clustering, dimension reduction, generalized singular value decomposition, linear discriminant analysis, text mining. 1
Dimension reduction in text classification with support vector machines
 Journal of Machine Learning Research
, 2005
"... Support vector machines (SVMs) have been recognized as one of the most successful classification methods for many applications including text classification. Even though the learning ability and computational complexity of training in support vector machines may be independent of the dimension of th ..."
Abstract

Cited by 41 (4 self)
 Add to MetaCart
Support vector machines (SVMs) have been recognized as one of the most successful classification methods for many applications including text classification. Even though the learning ability and computational complexity of training in support vector machines may be independent of the dimension of the feature space, reducing computational complexity is an essential issue to efficiently handle a large number of terms in practical applications of text classification. In this paper, we adopt novel dimension reduction methods to reduce the dimension of the document vectors dramatically. We also introduce decision functions for the centroidbased classification algorithm and support vector classifiers to handle the classification problem where a document may belong to multiple classes. Our substantial experimental results show that with several dimension reduction methods that are designed particularly for clustered data, higher efficiency for both training and testing can be achieved without sacrificing prediction accuracy of text classification even when the dimension of the input space is significantly reduced.
On the Gaussian MIMO wiretap channel
 in Proc. IEEE International Symposium on Information Theory (ISIT
, 2007
"... Wyner's wiretap channel is generalized to the case when the sender, the receiver and the eavesdropper have multiple antennas. We consider two cases: the deterministic case and the fading case. In the deterministic case, the channel matrices of the intended receiver and the eavesdropper are fixe ..."
Abstract

Cited by 39 (2 self)
 Add to MetaCart
(Show Context)
Wyner's wiretap channel is generalized to the case when the sender, the receiver and the eavesdropper have multiple antennas. We consider two cases: the deterministic case and the fading case. In the deterministic case, the channel matrices of the intended receiver and the eavesdropper are fixed and known to all the nodes. In the fading case, the channel matrices experience block fading and the sender has only the intended receiver's channel state information (CSI) and statistical knowledge of the eavesdropper's channel. For the deterministic case, a scheme based on the generalizedsingularvaluedecomposition (GSVD) of the channel matrices is proposed and shown to achieve the secrecy capacity in the high signaltonoiseratio (SNR) limit. When the intended receiver has only one antenna (MISO case) the secrecycapacity is characterized for any SNR. Next, a suboptimal "artificial noise " based scheme is considered. Its performance is characterized and observed to be nearly optimal in the high SNR regime for the MISO case. This scheme extends naturally to the fading case and results are reported for the MISO case. For the independent Rayleigh fading distribution as we simultaneously increase the number of antennas at the sender and the eavesdropper, the secrecy capacity approaches zero if and only if the ratio of the number of eavesdropper antennas to transmitter antennas is at least two. I.
The betaJacobi matrix model, the CS decomposition, and generalized singular value problems
 Foundations of Computational Mathematics
, 2007
"... Abstract. We provide a solution to the βJacobi matrix model problem posed by Dumitriu and the first author. The random matrix distribution introduced here, called a matrix model, is related to the model of Killip and Nenciu, but the development is quite different. We start by introducing a new matr ..."
Abstract

Cited by 26 (4 self)
 Add to MetaCart
(Show Context)
Abstract. We provide a solution to the βJacobi matrix model problem posed by Dumitriu and the first author. The random matrix distribution introduced here, called a matrix model, is related to the model of Killip and Nenciu, but the development is quite different. We start by introducing a new matrix decomposition and an algorithm for computing this decomposition. Then we run the algorithm on a Haardistributed random matrix to produce the βJacobi matrix model. The Jacobi ensemble on R n, parameterized by β> 0, a> −1, and b> −1, is the probability distribution whose density is proportional to Q β 2 i λ (a+1)−1