Abstract

Hinton [6] proposed that generalization in artificial neural nets should improve if nets learn to represent the domain's underlying regularities. Abu-Mustafa's hints work [1] shows that the outputs of a backprop net can be used as inputs through which domain-specific information can be given to the net. We extend these ideas by showing that a backprop net learning many related tasks at the same time can use these tasks as inductive bias for each other and thus learn better. We identify five mechanisms by which multitask backprop improves generalization and give empirical evidence that multitask backprop generalizes better in real domains.

Citations

228	Neural Network Perception for Mobile Robot Guidance - Pomerleau - 1992
154	Learning distributed representations of concepts - Hinton - 1986
128	NETtalk: a parallel network that learns to read aloud - Sejnowski, Rosenberg - 1986
91	A Personal Learning Apprentice - Dent, Boticario, et al. - 1992
82	Learning from Hints in Neural Networks - Abu–Mostafa - 1990
45	A Comparative Study of ID3 and Backpropagation for English Text-to-speech - Dietterich, Hild, et al. - 1990
45	A.A.: Direct transfer of learned information among neural networks - Pratt, Mostow, et al. - 1991
25	Symbolic-neural systems and the use of hints for developing complex systems - Suddarth, Holden - 1991
18	Hints and the VC Dimension - Abu–Mostafa - 1993
14	Multitask Connectionist Learning - Caruana - 1994