Multitask Learning

Multitask Learning (1997)

Cached

Download Links

by Rich Caruana

Venue:	MACHINE LEARNING
Citations:	328 - 6 self

BibTeX

@MISC{Caruana97multitasklearning,
    author = {Rich Caruana},
    title = {Multitask Learning},
    year = {1997}
}

Bookmark

OpenURL

Abstract

Multitask Learning is an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias. It does this by learning tasks in parallel while using a shared representation; what is learned for each task can help other tasks be learned better. This paper reviews prior work on MTL, presents new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals, and presents new results for MTL with k-nearest neighbor and kernel regression. In this paper we demonstrate multitask learning in three domains. We explain how multitask learning works, and show that there are many opportunities for multitask learning in real domains. We present an algorithm and results for multitask learning with case-based methods like k-nearest neighbor and kernel regression, and sketch an algorithm for multitask learning in decision trees. Because multitask learning works, can be applied to many different kinds of domains, and can be used with different learning algorithms, we conjecture there will be many opportunities for its use on real-world problems.

Citations

2888	Induction of Decision Trees - Quinlan - 1986
1998	Bagging predictors - Breiman - 1996
877	A Bayesian method for the induction of probabilistic networks from data - Cooper, Herskovits - 1992
771	Statistical Analysis with Missing Data - Little, Rubin - 2002
655	C4.5: Programs for - Quinlan - 1993
634	Hierarchical mixtures of experts and the EM algorithm - Jordan, Jacobs - 1994
554	The strenght of weak learnability - Schapire - 1990
448	Solving Multiclass Learning Problems via Error-Correcting Output Codes - Dietterich, Bakiri - 1995
358	Boosting a weak learning algorithm by majority - Freund - 1995
296	What size net gives valid generalization - Baum, Haussler - 1989
256	Backpropagation applied to handwritten zip code recognition - LeCun, Boser, et al. - 1989
228	Neural Network Perception for Mobile Robot Guidance - Pomerleau - 1992
218	Learning representations by back-propagating errors. Nature - Rumelhart, Hinton, et al. - 1986
193	D.: Experience with a learning personal assistant - Mitchell, Caruana, et al. - 1994
167	The Need for Biases in Learning Generalizations - Mitchell - 1990
157	Supervised learning from incomplete data via an EM approach - Ghahramani, Jordan - 1994
154	Learning distributed representations of concepts - Hinton - 1986
128	NETtalk: a parallel network that learns to read aloud - Sejnowski, Rosenberg - 1986
116	Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355 - Becker, GE - 1992
109	Is learning the n-th thing any easier than learning the first - Thrun - 1996
94	Explanation-Based Neural Network Learning: A Lifelong Learning Approach - Thrun - 1996
91	A Personal Learning Apprentice - Dent, Boticario, et al. - 1992
86	A probabilistic approach to feature selection - a filter solution - Liu, Setiono - 1996
82	Learning from Hints in Neural Networks - Abu–Mostafa - 1990
81	Learning internal representations - Baxter - 1995
81	Hierarchical mixtures of experts and - Jordan, Jacobs - 1994
69	Discovering structure in multiple learning tasks: The TC algorithm - Thrun, O’Sullivan - 1996
66	Newsweeder: Learning to filter news - Lang - 1995
65	Using sampling and queries to extract rules from trained neural networks - Craven, Shavlik - 1994
62	Adapting bias by gradient descent: An incremental version of deltabar-delta - Sutton - 1992
57	Learning one more thing - Thrun, Mitchell - 1995
56	Multitask learning: A knowledge-based source of inductive bias - Caruana - 1997
52	Learning many related tasks at the same time with backpropagation - Caruana - 1995
48	Predicting multivariate responses in multiple linear regression - Breiman, Friedman - 1997
45	A Comparative Study of ID3 and Backpropagation for English Text-to-speech - Dietterich, Hild, et al. - 1990
45	A.A.: Direct transfer of learned information among neural networks - Pratt, Mostow, et al. - 1991
42	Training Neural Networks with Deficient Data - Tresp, Ahmad, et al. - 1994
41	Using the future to “sort out” the present: Rankprop and multitask learning for medical risk evaluation - Caruana, Baluja, et al. - 1996
37	Using additive noise in back-propagation training - Holmström, Koistinen - 1992
36	A comparison of id3 and backpropagation for English text-tospeech mapping - Dietterich, Hild, et al. - 1990
29	Learning classification with unlabeled data - Sa - 1994
26	1989], ‘Modularity and scaling in large phonemic neural networks - Waibel, Sawai, et al.
25	Symbolic-neural systems and the use of hints for developing complex systems - Suddarth, Holden - 1991
22	Rule-injection Hints as a Means of Improving Network Performance and Learning Time - Suddarth, Kergosien - 1990
22	A Bayesian Method for the Induction - Cooper, Herskovits - 1992
20	Lifelong learning: A case study - Thrun - 1995
19	Conceptual Clustering, Learning from Examples, and Inference - Fisher - 1987
19	Learning representations by back-propagating errors". Nature 323 - Rumelhart, Hinton, et al. - 1986
18	Hints and the VC Dimension - Abu–Mostafa - 1993
18	Acquiring and Combining Overlapping Concepts - Martin, Billman - 1994