#### DMCA

## Projected gradient methods for Nonnegative Matrix Factorization (2007)

### Cached

### Download Links

- [www.csie.ntu.edu.tw]
- [www.csie.ntu.edu.tw]
- [ntur.lib.ntu.edu.tw]
- [www.csie.ntu.edu.tw]
- DBLP

### Other Repositories/Bibliography

Venue: | Neural Computation |

Citations: | 273 - 2 self |

### Citations

1639 |
Learning the parts of objects by non-negative matrix factorization
- Lee, Seung
- 1999
(Show Context)
Citation Context ...osed methods converges faster than the popular multiplicative update approach. A simple MATLAB code is also provided. 1 Introduction Non-negative matrix factorization (NMF) (Paatero and Tapper, 1994; =-=Lee and Seung, 1999-=-) is useful for finding representations of non-negative data. Given an n × m data matrix V with Vij ≥ 0 and a pre-specified positive integer r < min(n, m), NMF finds two non-negative matrices W ∈ R n×... |

1213 | Algorithms for non-negative matrix factorization - Lee, Seung |

1121 |
Nonlinear Programming. Athena Scientific, 2nd edition
- Bertsekas
- 1999
(Show Context)
Citation Context ...arts: ∇W f(W, H) = (W H − V )H T and ∇Hf(W, H) = W T (W H − V ), (2) which are, respectively, partial derivatives to elements in W and H. From the Karush-Kuhn-Tucker (KKT) optimality condition (e.g., =-=Bertsekas, 1999-=-), (W, H) is a stationary point of (1) if and only if Wia ≥ 0, Hbj ≥ 0, ∇W f(W, H)ia ≥ 0, ∇Hf(W, H)bj ≥ 0, Wia · ∇W f(W, H)ia = 0, and Hbj · ∇Hf(W, H)bj = 0, ∀i, a, b, j. Optimization methods for NMF ... |

890 |
Solving Least Squares Problems
- Lawson, Hanson
- 1974
(Show Context)
Citation Context ...ast square problems: From (10), H k+1 ’s jth column = min h≥0 �v − W k+1 h� 2 , (11) 6swhere v is the jth column of V and h is a vector variable. Chu et al. (2005) suggest projected Newton’s methods (=-=Lawson and Hanson, 1974-=-) to solve each problem (11). Clearly, solving sub-problems (9) and (10) per iteration could be more expensive than the simple update in Algorithm 1. Then Algorithm 2 may be slower even though we expe... |

640 | RCV1: A New Benchmark Collection for Text Categorization Research - Lewis, Yang, et al. |

515 |
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values
- Paatero, Tapper
- 1994
(Show Context)
Citation Context ...trate that one of the proposed methods converges faster than the popular multiplicative update approach. A simple MATLAB code is also provided. 1 Introduction Non-negative matrix factorization (NMF) (=-=Paatero and Tapper, 1994-=-; Lee and Seung, 1999) is useful for finding representations of non-negative data. Given an n × m data matrix V with Vij ≥ 0 and a pre-specified positive integer r < min(n, m), NMF finds two non-negat... |

318 |
Document clustering based on non-negative matrix factorization
- Xu, Liu, et al.
- 2003
(Show Context)
Citation Context ... object, NMF approximates it by a linear combination of r “basis” columns in W . NMF has been applied to many areas such as finding basis vectors of images (Lee and Seung, 1999), document clustering (=-=Xu et al., 2003-=-), molecular pattern discovery (Brunet et al., 2004), etc. Donoho and 1sStodden (2004) have addressed the theoretical issues associated with the NMF approach. The conventional approach to find W and H... |

190 | When does non-negative matrix factorization give a correct decomposition into parts - Donoho, Stodden - 2003 |

170 | Metagenes and molecular pattern discovery using matrix factorization - Brunet, Tamayo, et al. - 2004 |

165 | Non-negative sparse coding
- HOYER
- 2002
(Show Context)
Citation Context ...ification in this study. Among the existing bound-constrained optimization techniques, the projected gradient method is simple and effective. Though several researchers have used this method for NMF (=-=Hoyer, 2002-=-; Chu et al., 2005; Shepherd, 2004), there is neither a systematic study nor an easy implementation comparable to that of 2sthe multiplicative update method. This paper presents a comprehensive study ... |

115 | Projected gradient methods for linearly constrained problems - Calamai, More - 1987 |

107 | On the Goldstein-Levitin-Polyak gradient projection method - Bertsekas - 1976 |

80 |
On the convergence of the block nonlinear Gauss-Seidel method under convex constraints
- Grippo, Sciandrone
- 2000
(Show Context)
Citation Context ...the zero matrix, any W is optimal for (9). Fortunately, for the case of two blocks, Grippo and Sciandrone (2000) have shown that this uniqueness condition is not needed. Directly from Corollary 2 of (=-=Grippo and Sciandrone, 2000-=-), we have the following convergence result: Theorem 2 Any limit point of the sequence {W k , H k } generated by Algorithm 2 is a stationary point of (1). The remaining issue is whether the sequence {... |

44 | Optimality, computation, and interpretation of nonnegative matrix factorizations
- Chu, Diele, et al.
- 2004
(Show Context)
Citation Context ...this study. Among the existing bound-constrained optimization techniques, the projected gradient method is simple and effective. Though several researchers have used this method for NMF (Hoyer, 2002; =-=Chu et al., 2005-=-; Shepherd, 2004), there is neither a systematic study nor an easy implementation comparable to that of 2sthe multiplicative update method. This paper presents a comprehensive study on using projected... |

33 |
The multilinear engine—A table-driven, least squares program for solving multilinear problems, including the N-way parallel factor analysis model
- Paatero
- 1999
(Show Context)
Citation Context ...source codes used in this paper are available online at http://www.csie.ntu.edu.tw/~cjlin/ nmf. 2 Existing Methods and New Properties There are many existing methods for NMF. Some discussions are in (=-=Paatero, 1999-=-), but bound constraints are not rigorously handled. A more recent and complete survey is by Chu et al. (2005). This section briefly discusses some existing 3smethods and presents several previously u... |

31 |
Newton’s method for large-scale bound constrained problems
- Lin, Moré
- 1999
(Show Context)
Citation Context ...e discussions about the choice of β. Searching αk is the most time consuming operation in Algorithm 3, so one should check as few step sizes as possible. Since αk−1 and αk may be similar, a trick in (=-=Lin and Moré, 1999-=-) uses αk−1 as the initial guess and then either increases or decreases it in order to find the largest β tk satisfying (13). Moreover, with non-negative tk, Algorithm 4 may be too conservative by res... |

23 | Accelerating the Lee-Seung algorithm for nonnegative matrix factorization - Gonzales, Zhang - 2005 |

22 | The gradient projection method under mild differentiability conditions - McCormick, Tapia - 1972 |

6 |
On search directions for minimization
- Powell
- 1973
(Show Context)
Citation Context ...s we have, the convergence is guaranteed. However, this issue deserves some attention. Past convergence analysis for “block coordinate descent” methods requires sub-problems to have unique solutions (=-=Powell, 1973-=-; Bertsekas, 1999), but this property does not hold here: Sub-problems (9) and (10) are convex, but they are not strictly convex. Hence, these sub-problems may have multiple optimal solutions. For exa... |

1 |
Theory, numerical methods and applications of the nonnegative matrix factorization
- Chu, Diele, et al.
- 2004
(Show Context)
Citation Context ... consider this modification. Among existing bound-constrained optimization techniques, projected gradients are a simple and effective one. Though several papers used this method for NMF (Hoyer, 2002; =-=Chu et al., 2004-=-; Shepherd, 2004), there is neither a systematic study nor an easy implementation competitive with the multiplicative update. This paper gives a comprehensive study on using projected gradient methods... |