Lori Y. Pratt. Discriminability-based transfer between neural networks. Same volume.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....across multiple control learning tasks. 4. Learning representations. Representations, together with inductive bias, determine the way a function approximator generalizes from examples. Many researchers have focussed on learning appropriate representations in order to learn bias. For example, Pratt [Pratt, 1993] describes several approaches that allow to re use learned representations in hidden units of neural networks. Although she could empirically demonstrate that this transfer could significantly reduce the number of training epochs required for the convergence of the Back propagation algorithm, she ....

Lori Y. Pratt. Discriminability-based transfer between neural net- works. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

.... [7] Others proposed hierarchical approaches, in which the building blocks, once learned, can be applied to multiple tasks [3, 7, 21] A third way for the transfer of knowledge is concerned with the construction of better internal representations, which improve generalization across multiple tasks [1, 16, 19, 23]. While this list is clearly incomplete, it nevertheless illustrates the impor Figure 6: Learning navigation. Traces of three early and three late episodes are shown. Each diagram shows a two dimensional occupancy maps of the world, which have been constructed based on sonar information. Bright ....

Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

....Gordon and Subrama nian [16] as well as Schultz and Grefenstette [32] use prior knowledge to initialize a genetic algorithm. The former system uses high level advice for the initialization. KBANN [35] uses a domain theory in the form of propositional rules to initialize a neural network. Pratt [24] describes the cliscrimiabilitl bascl trasfcr method for incorporating knowledge acquired during previous learning tasks into a neural network. Conclusions The framework we have presented for treating bias selection as search has two essential features: bias is divided into representational ....

Pratt, L. Y. (1993). Discriminability-based transfer between neural networks. In Giles, C. L., Hanson, S. J., mhd Cowan, J. D., editors, Advances in Neural formation Processing Systems 5, pages 204 211. Morgan Kaufmamh Publishers, San Mateo, CA.

....The continual learning framework is conceptually similar to learning structures presented by other researchers, although the temporal continuity of their systems are often not stressed as highly. Pratt, for example, has given attention to the problem of knowledge transfer among di erent tasks [65, 66, 67]. Here the goal is to apply knowledge acquired in one learning task to another learning task in order to speed learning and possibly increase nal accuracy. Pratt speci cally investigates transfer among neural network structures by preserving decision hyperplanes across tasks. The arrangement of ....

L. Pratt. Discriminability-based transfer between neural networks. In S. J. Hanson, C. L. Giles, and J. D. Cowan, editors, Advances in Neural Information Processing Systems 5, pages 204-211. Morgan Kaufmann, 1993.

....tasks are becoming well understood, there is growing research interest directed to dynamic and adaptable systems. Studies have examined issues such as concept drift [55] learning bias from multiple tasks [7] continual learning [53, 57] multitask learning [9] knowledge transfer between tasks [46, 44], and lifelong learning [69] While progress has been made, this branch of study still contains many unresolved issues. Methods are needed to reliably detect when the underlying concept being modeled has changed since training began and to adapt the current domain model to the new conditions. It ....

....techniques that automatically discard training instances by age, for example, integrate the two phases with an assumption of drift. In general, the process of adapting current models to new information has been studied under the rubrics of multitask learning [9] knowledge transfer between tasks [46, 44], and lifelong learning [69] Chapter 2 Issues and Related Work In this chapter we will discuss the goals of the anomaly detection domain, related background work, and the issues raised by the proposed research. Although we make an effort to divide the issues into those most nearly learning ....

[Article contains additional citation context not shown here]

L. Pratt. Discriminability-based transfer between neural networks. In Hanson et al. [23], pages 204--211.

....of novel tasks for good generalization A natural alternative line of enquiry is how the runningtime or computational complexity of a learning algorithm may be improved by training on related tasks. Some early algorithms for neural networks in this vein are contained in Sharkey and Sharkey (1993) Pratt (1992). Reinforcement Learning. Many control tasks can appropriately be viewed as elements of sets of related tasks, such as learning to navigate to different goal states, or learning a set of complex motor control tasks. A number of papers in the reinforcement learning literature have proposed ....

Pratt, L. Y. (1992). Discriminability-based transfer between neural networks. In Hanson, S. J., Cowan, J. D., & Giles, C. L. (Eds.), Advances in Neural Information Processing Systems 5, pp. 204--211. Morgan Kaufmann.

....of novel tasks for good generalization A natural alternative line of enquiry is how the runningtime or computational complexity of a learning algorithm may be improved by training on related tasks. Some early algorithms for neural networks in this vein are contained in Sharkey and Sharkey (1993) Pratt (1992). E Reinforcement Learning. Many control tasks can appropriately be viewed as elements of sets of related tasks, such as learning to navigate to different goal states, or learning a set of complex motor control tasks. A number of papers in the reinforcement learning literature have proposed ....

Pratt, L. Y. (1992). Discriminability-based transfer between neural networks. In Hanson, S. J., Cowan, J. D., & Giles, C. L. (Eds.), Advances in Neural Information Processing Systems 5, pp. 204--211. Morgan Kaufmann.

....realization that complex environments will require learning to perform multiple tasks and the learning can be greatly saved or simplified if the tasks to be learned can share what they learn. Recently there are increasing interests in knowledge transfer of ANN learning across multiple tasks. Pratt [3] studied learning on a target network which may be sped up by using weights obtained from another network trained for a related source task. She presented a DiscriminabilityBased Transfer algorithm to estimate the utility of hyperplanes defined by source weights in the target network and rescale ....

L.Y. Pratt, Discriminability-based transfer between neural networks, in J.E. Moody, S.J. Hanson, and R.P. Lippmann, editors, Advances in Neural Information Processing Systems 5, CA, 1993, Morgan Kaufmann, pp.204-211.

....across multiple control learning tasks. 4. Learning representations. Representations, together with inductive bias, determine the way a function approximator generalizes from examples. Many researchers have focussed on learning appropriate representations in order to learn bias. For example, Pratt [Pratt, 1993] describes several approaches that allow to re use learned representations in hidden units of neural networks. Although she could empirically demonstrate that this transfer could significantly reduce the number of training epochs required for the convergence of the Back propagation algorithm, she ....

Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advancesin Neural Information ProcessingSystems5, San Mateo, CA, 1993. Morgan Kaufmann. (to appear).

.... other, similar speakers (e.g. see [Hild and Waibel, 1993] Other approaches that use related functions to change the bias of an inductive learner can be found in [Utgoff, 1986] Rendell et al. 1987] Suddarth and Kergosien, 1990] Moore et al. 1992] Sutton, 1992] Caruana, 1993] [Pratt, 1993] , and [Baxter, 1995] Table 1 summarizes the problem definitions of the standard and the lifelong supervised learning problem. In lifelong supervised learning, the learner is given a collection Y of support sets, in addition to the training set X and the hypothesis space H. This raises two ....

....show that EBNN manages to extract useful invariance information in this domain, even if these invariances defy simple interpretation. 3. 4 Using Support Sets as Hints A related family of methods for the transfer of knowledge across learning tasks are proposed in [Suddarth and Kergosien, 1990] [Pratt, 1993] , Caruana, 1993] In a nutshell, these approaches develop improved internal representations by considering multiple functions in F (sequentially, or simultaneously) Following these ideas, we trained a single classification network providingthe support data as hints for the development of ....

L.Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems5,San Mateo, CA, 1993. Morgan Kaufmann.

....to decrease the training time for a new task and or reduce the number of training examples necessary for acceptable generalization. This form of knowledge based inductive bias is referred to in the literature as the transfer of knowledge from one or more source tasks to a target or primary task [21, 25]. The transfer of task knowledge can be considered a major aspect of the problem of learning to learn [14] and has close ties to analogical reasoning [17] Medical decision making is an excellent field for the application of machine learning technology [27] There have been a number of successes ....

....form of transfer involves the direct or indirect assignment of known task representation (weight values) to a new task. We consider this to be an explicit form of knowledge transfer from a source task to a target task. Since 1990 numerous authors have discussed methods of representational transfer [15, 25, 26, 28, 29, 32, 36] which often results in substantially reduced training time with no loss in generalization performance. In contrast to representational transfer is a form we define as functional. Functional transfer does not involve the explicit assignment of prior task representation to a new task, rather it ....

[Article contains additional citation context not shown here]

Lorien Y. Pratt. Discriminability-based transfer between neural networks. Advances in Neural Information Processing Systems 5, 5:204--211, 1993. ed. C. L. Giles and S. J. Hanson and J.D. Cowan.

.... the training time for a new task and or reduce the number of training examples necessary for acceptable generalization [Silv95, Silv96a] This form of knowledge based inductive learning is referred to elsewhere as the transfer of knowledge from one or more source tasks to a target or primary task [Utgo86, Prat93]. The transfer of task knowledge can be considered a major aspect of the problem of learning to learn [Elli65] and has close ties to analogical reasoning [Hall89] In [Silv96b] we define the distinction between two forms of task knowledge transfer, representational and functional, and present a ....

....form of transfer involves the direct or indirect assignment of known task representation (weight values) to a new task. We consider this to be an explicit form of knowledge transfer from a source task to a target task. Since 1990 numerous authors have discussed methods of representational transfer [Fahl90, Prat93, Ring93, Shar92, Shav90, Sing92, Towe90] which often results in substantially reduced training time with no loss in generalization performance. In contrast to representational transfer is a form we define as functional. Functional transfer does not involve the explicit assignment of prior task representation to a new task, rather it ....

Lorien Y. Pratt, "Discriminability-Based transfer between neural networks", Advances in Neural Information Processing Systems 5, Morgan Kaufmann, Vol. 5, pp. 204--211, San Mateo, CA, 1993.

.... similar speakers (e.g. see [Hild and Waibel, 1993] Other approaches that use related functions to change the bias of an inductive learner can be found in [Utgoff, 1986] Rendell et al. 1987] Suddarth and Kergosien, 1990] Moore et al. 1992] Sutton, 1992] Caruana, 1993] and [Pratt, 1993] . Table 1 summarizes the problem definitions of standard supervised learning and the lifelong supervised learning problem. In lifelong supervised learning, the learner is given a collection Y of support sets, in addition to the training set X and the hypothesis space H. This raises two ....

L.Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

....Learning internal representations. Other researchers report techniques to develop more appropriate hidden layer representations from multiple tasks. For example, Pratt proposed a method which transfered information by using an internal representation that was developed in earlier learning tasks [Pratt, 1993] . A similar technique has been proposed in [Sharkey and Sharkey, 1992] A second example of learning internal representations using multiple target functions is Caruana s multi task learning algorithm. In his approach, multiple, related tasks are trained simultaneously in a single neural ....

Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

....of Gordon and Subramanian [16] as well as Schultz and Grefenstette [32] use prior knowledge to initialize a genetic algorithm. The former system uses high level advice for the initialization. KBANN [35] uses a domain theory in the form of propositional rules to initialize a neural network. Pratt [24] describes the discriminability based transfer method for incorporating knowledge acquired during previous learning tasks into a neural network. 7. Conclusions The framework we have presented for treating bias selection as search has two essential features: bias is divided into representational ....

Pratt, L. Y. (1993). Discriminability-based transfer between neural networks. In Giles, C. L., Hanson, S. J., and Cowan, J. D., editors, Advances in Neural Information Processing Systems 5, pages 204--211. Morgan Kaufmann Publishers, San Mateo, CA.

.... [7] Others proposed hierarchical approaches, in which the building blocks, once learned, can be applied to multiple tasks [3, 7, 21] A third way for the transfer of knowledge is concerned with the construction of better internal representations, which improve generalization across multiple tasks [1, 16, 19, 23]. While this list is clearly incomplete, it nevertheless illustrates the impor Sebastian Thrun A Lifelong Learning Perspective for Mobile Robot Control 7 episode 1 episode 2 episode 6 episode 18 episode 19 episode 20 Figure 6: Learning navigation. Traces of three early and three late episodes ....

Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

....and weights for each leaf of the decision tree, is trained on the second context. This is similar to the two tiered structure originally proposed [15] for dealing with flexible contexts. Knowledge from an existing network can be used to significantly increase the speed of learning in a new context [16, 17] by using weights from the existing network to initialise the new neural network. 8 CONCLUSION 29 Methods for the transfer of knowledge from one context to another could be used to adapt Splice local concepts on line in a manner analogous to that used by FLORA3. Context has thus far been defined ....

L. Y. Pratt. Discriminability-based transfer between neural networks. In S. J. Hanson, C. L. Giles, and J. D. Cowan, editors, Advances in Neural Information Processing Systems 5, pages 204--211. Morgan Kaufmann, 1993. REFERENCES 33

....across multiple control learning tasks. 4. Learning representations. Representations, together with inductive bias, determine the way a function approximator generalizes from examples. Many researchers have focussed on learning appropriate representations in order to learn bias. For example, Pratt [Pratt, 1993] describes several approaches that allow to re use learned representations in hidden units of neural networks. Although she could empirically demonstrate that this transfer could significantly reduce the number of training epochs required for the convergence of the Back propagation algorithm, she ....

Lori Y. Pratt. Discriminability-based transfer between neural networks. In J. E. Moody, S. J. Hanson, and R. P. Lippmann, editors, Advances in Neural Information Processing Systems 5, San Mateo, CA, 1993. Morgan Kaufmann.

....Henceforth, the terms task and function should be considered synonymous. Background: Learning to learn, an area of intense inquiry since 1990, can be subdivided along several themes: catastrophic interference and methods of knowledge transfer between a learned source task and a new target task [Shar94, Prat93], priming of neural networks with symbolic knowledge [Shav92, Towe91] compositional learning which advocates first learning sub tasks of a more complex function [Sing92] learning search information during the training of source tasks and the use of this information to reduce the training time on ....

Lorien Y. Pratt, Discriminability-Based transfer between neural networks, Advances in Neural Information Processing Systems 5, Morgan Kaufmann, Vol. 5, pp. 204--211, San Mateo, CA, 1993.

....Relaxing the hyperplane assumption in the analysis and modification of back propagation neural networks Reprinted from: Robert Trappl, ed. Cybernetics and Systems 94. World Scientific, Singapore, 1994. Pages 1711 1718 L. Y. Pratt Department of Mathematical and Computer Sciences Colorado School of Mines Golden, CO 80401 lpratt mines.colorado.edu A. N. Christensen Department of Geophysics Colorado School of Mines Golden, CO 80401 achriste mines.colorado.edu Abstract Several algorithms that operate on back propagation ....

.... this simplification is used include network visualization [ Munro, 1992 ] skeletonization [ Ramachandran and Pratt, 1992 ] unsupervised learning [ Schraudolph and Sejnowski, 1993 ] hybrid learning architectures [ Sankar and Mammone, 1993 ] and transfer between networks for related tasks [ Pratt, 1993, Sharkey and Sharkey, 1993 ] In this simplification, a threshold is considered to occur at the point where the logistic crosses 0:5. This simplification is often justified by noting that network weights tend to grow with Correspondence author 1 Or hyperbolic tangents, for networks with ....

[Article contains additional citation context not shown here]

L. Y. Pratt. Discriminability-based transfer between neural networks. In C.L. Giles, S. J. Hanson, and J. D. Cowan, editors, Advances in Neural Information Processing Systems 5, pages 204--211. Morgan Kaufmann Publishers, San Mateo, CA, 1993. Also available via anonymous ftp to franklinite.mines.colorado.edu: pub/pratt-papers/pratt-nips5.ps.Z.

....asymptotic classifier performance can potentially be improved. Several recent studies have evaluated approaches to different formulations of the transfer problem in neural network applications [ Thrun and Mitchell, 1993a, Singh, 1992, Naik et al. 1992, Agarwal et al. 1992 ] This paper extends [ Pratt, 1993a ] which described the Transfer in neural networks 3 Discriminability based transfer (DBT) algorithm. We describe a number of important distinctions in characterizing approaches to transfer in Section 2. We then describe DBT in detail in Section 3. Empirical results from evaluating DBT on ....

L. Y. Pratt. Discriminability-based transfer between neural networks. In C.L. Giles, S. J. Hanson, and J. D. Cowan, editors, Advances in Neural Information Processing Systems 5, pages 204--211. Morgan Kaufmann Publishers, San Mateo, CA, 1993.

No context found.

Lori Y. Pratt. Discriminability-based transfer between neural networks. Same volume.

No context found.

Pratt, L.Y. (1993). Discriminability-Based Transfer Between Neural Networks. In Advances in Neural Information Processing Systems 5, S.J. Hanson, C.L. Giles, and J.D. Cowan (eds.), Morgan Kaufmann, San Mateo

No context found.

Lori Y Pratt. Discriminability-based transfer between neural networks. In Stephen J Hanson, Jack D Cowan, and C Lee Giles, editors, Advances in Neural Information Processing Systems 5, pages 204--211, San Mateo, 1992. Morgan Kaufmann.

No context found.

Lori Y. Pratt. Discriminability-based transfer between neural networks. Same volume.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC