Background: The construction of literature-based networks of gene-gene interactions is one of the most important applications of text mining in bioinformatics. Extracting potential gene relationships from the biomedical literature may be helpful in building biological hypotheses that can be explored further experimentally. Recently, latent semantic indexing based on the singular value decomposition (LSI/SVD) has been applied to gene retrieval. However, the determination of the number of factors k used in the reduced rank matrix is still an open problem. Results: In this paper, we introduce a way to incorporate a priori knowledge of gene relationships into LSI/SVD to determine the number of factors. We also explore the utility of the non-negative matrix factorization (NMF) to extract unrecognized gene relationships from the biomedical literature by taking advantage of known gene relationships. A gene retrieval method based on NMF (GR/NMF) showed comparable performance with LSI/SVD. Conclusions: Using known gene relationships of a given gene, we can determine the number of factors used in the reduced rank matrix and retrieve unrecognized genes related with the given gene by LSI/SVD or GR/NMF. 1

Nonnegative tensor factorization (NTF) is a technique for computing a parts-based representation of high-dimensional data. NTF excels at exposing latent structures in datasets, and at finding good low-rank approximations to the data. We describe an approach for computing the NTF of a dataset that relies only on iterative linear-algebra techniques and that is comparable in cost to the nonnegative matrix factorization. (The better-known nonnegative matrix factorization is a special case of NTF and is also handled by our implementation.) Some important features of our implementation include mechanisms for encouraging sparse factors and for ensuring that they are equilibrated in norm. The complete Matlab software package is available under the GPL license. Keywords N-dimensional arrays, tensors, nonnegative tensor factorization, alternating least squares, block Gauss-Seidel, sparse solutions, regularization, nonnegative least-squares 1

In this work, we apply Dirichlet Process Mixture Models (DPMMs) to a learning task in natural language processing (NLP): lexical-semantic verb clustering. We thoroughly evaluate a method of guiding DP-MMs towards a particular clustering solution using pairwise constraints. The quantitative and qualitative evaluation performed highlights the benefits of both standard and constrained DPMMs compared to previously used approaches. In addition, it sheds light on the use of evaluation measures and their practical application. 1

Abstract: Nonnegative matrix approximation (NNMA) is a popular matrix decomposition technique that has proven to be useful across a diverse variety of fields with applications ranging from document analysis and image processing to bioinformatics and signal processing. Over the years, several algorithms for NNMA have been proposed, e.g. Lee and Seung’s multiplicative updates, alternating least squares (ALS), and gradient descent-based procedures. However, most of these procedures suffer from either slow convergence, numerical instability, or at worst, serious theoretical drawbacks. In this paper, we develop a new and improved algorithmic framework for the least-squares NNMA problem, which is not only theoretically well-founded, but also overcomes many deficiencies of other methods. Our framework readily admits powerful optimization techniques and as concrete realizations we present implementations based on the Newton, BFGS and conjugate gradient methods. Our algorithms provide numerical results superior to both Lee and Seung’s method as well as to the alternating least squares heuristic, which was reported to work well in some situations but has no theoretical guarantees [1]. Our approach extends naturally to include regularization and box-constraints without sacrificing convergence guarantees. We present experimental results on both synthetic and real-world datasets that demonstrate the superiority of our methods, both in terms of better approximations as well as

Abstract — Non-negative matrix factorization (NMF) and non-negative tensor factorization (NTF) have attracted much attention and have been successfully applied to numerous data analysis problems where the components of the data are necessarily non-negative such as chemical concentrations in experimental results or pixels in digital images. Especially, Andersson and Bro’s PARAFAC algorithm with nonnegativity constraints (AB-PARAFAC-NC) provided the stateof-the-art NTF algorithm, which uses Bro and de Jong’s nonnegativity-constrained least squares with single right hand side (NLS/S-RHS). However, solving an NLS with multiple right hand sides (NLS/M-RHS) problem by multiple NLS/S-RHS problems is not recommended due to hidden redundant computation. In this paper, we propose an NTF algorithm based on alternating large-scale non-negativity-constrained least squares (NTF/ANLS) using NLS/M-RHS. In addition, we introduce an algorithm for the regularized NTF based on ANLS (RNTF/ANLS). Our experiments illustrate that our NTF algorithms outperform AB-PARAFAC-NC in terms of computing speed on several data sets we tested. I.

This paper is motivated by the industrial research problem of designing a real-world recommender system for a large Information Technology (IT) company. Given historical records of client purchases, compactly represented as a sparse client-times-product “who-bought-what ” binary matrix, the goal is to build a model that provides recommendations for what products should be sold next to the existing client base. Such a problem may naturally be formulated as a collaborative filtering task. However, this is a one-class setting, that is, if a client has not bought a product yet, it does not imply that the client has a low propensity to potentially buy that product later. In the absence of explicitly labeled negative examples, one may resort to considering zero-valued client-product pairs as either missing data or as surrogate negative instances. In this paper, we outline an approach to explicitly deal with this kind of ambiguity by instead treating zero-valued pairs as optimization variables. These variables are optimized in conjunction with learning a weighted, low-rank non-negative matrix factorization (NMF) of the client-product matrix. The proposed algorithm alternates NMF optimization with deterministicannealing/continuation techniques designed for global minimization of combinatorial and non-convex objective functions. Experimental results show that our approach can give significantly better recommendations in comparison to various competing alternatives on a one-class collaborative filtering task.

Abstract—Dynamic system modeling plays a crucial role in the development of techniques for stationary and nonstationary signal processing. Due to the inherent physical characteristics of systems under investigation, nonnegativity is a desired constraint that can usually be imposed on the parameters to estimate. In this paper, we propose a general method for system identification under nonnegativity constraints. We derive the so-called nonnegative leastmean-square algorithm (NNLMS) based on stochastic gradient descent, and we analyze its convergence. Experiments are conducted to illustrate the performance of this approach and consistency with the analysis. Index Terms — Adaptive filters, adaptive signal processing, least mean square algorithms, nonnegative constraints, transient analysis. I.

Optimization problems with a nuclear norm regularization, such as e.g. low norm matrix factorizations, have seen many applications recently. We propose a new approximation algorithm building upon the recent sparse approximate SDP solver of (Hazan, 2008). The experimental efficiency of our method is demonstrated on large matrix completion problems such as the Netflix dataset. The algorithm comes with strong convergence guarantees, and can be interpreted as a first theoretically justified variant of Simon-Funk-type SVD heuristics. The method is free of tuning parameters, and very easy to parallelize. 1.

Blind source separation (BSS) and related methods, e.g., independent component analysis (ICA) are generally based on a wide class of unsupervised learning algorithms and they found potential applications in many areas from engineering to neuroscience. The recent trends in blind source separation and generalized component analysis (GCA) is to consider problems in the framework of matrix factorization or more general signals decomposition with probabilistic generative models and exploit a priori knowledge about true nature, morphology or structure of latent (hidden) variables or sources such as sparseness, spatio-temporal decorrelation, statistical independence, nonnegativity, smoothness or lowest possible complexity. The goal of BSS can be considered as estimation of true physical sources and parameters of a mixing system, while objective of GCA is finding a new reduced or hierarchical and structured representation for the observed (sensor) data that can be interpreted as physically meaningful coding or blind signal decompositions. The key issue is to find a such transformation or coding which has true physical meaning and interpretation. In this paper we discuss some promising applications of BSS/GCA for analyzing multi-modal, multi-sensory data, especially EEG/MEG data. Moreover, we propose to apply

We introduce a new speaker independent method for reducing wind noise in single-channel recordings of noisy speech. The method is based on non-negative sparse coding and relies on a wind noise dictionary which is estimated from an isolated noise recording. We estimate the parameters of the model and discuss their sensitivity. We then compare the algorithm with the classical spectral subtraction method and the Qualcomm-ICSI-OGI noise reduction method. We optimize the sound quality in terms of signal-to-noise ratio and provide results on a noisy speech recognition task. 1.