K. Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems, Vol. 8, pp. 295--301. MIT Press, 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....the fields of mathematical statistics [6, 24] machine learning [8] and computational learning theory [2] as well as in the field of neural networks input hidden output # 1 # L . f 0 (x) layer layer layer Figure 1: A three layer feedforward neural network. [9, 7]. However, most of the studies do not directly aim at the optimal generalization. In this paper, we give a new method of active learning. The presented method provides exactly the optimal generalization capability in the trigonometric polynomial space, where the concept of POBs is crucial. By ....

K. Fukumizu, Active learning in multilayer perceptrons, in "Advances in Neural Information Processing Systems 8" (David S. Touretzky, Michael C. Mozer, and Michael E. Hasselmo, Eds.), pp. 295--301, The MIT Press, 1996.

....many scientific experiments or learning of sensorimotor maps of multijoint robot arms. Learning can be performed more e#ciently if we can actively design input signals. The problem of designing input signals for optimal generalization is called active learning (Cohn, Ghahramani, Jordan, 1996; Fukumizu, 1996; Vijayakumar Ogawa, 1999) It is also referred to as optimal experiments (Kiefer, 1959; Fedorov, 1972; Cohn, 1994) or query construction (Sollich, 1994) Reinforcement learning (Kaelbling, 1996) which has been extensively studied recently in the field of machine learning, can be regarded as ....

....of Bayesian statistics, MacKay (1992) derived a criterion for selecting the most informative training data for specifying the parameters of neural networks. Cohn (1994, 1996) and Cohn, Ghahramani, and Jordan (1996) gave an active learning criterion for minimizing the variance of the estimator. Fukumizu (1996) proposed an active learning method in multi layer perceptrons using asymptotic approximation for estimating the generalization error. Essentially, the criteria derived in these papers are Incremental Active Learning for Optimal Generalization 3 equivalent to the A optimal design shown in Fedorov ....

[Article contains additional citation context not shown here]

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. S. Touretzky, M.

....8. 6 Conclusion and Outlook With active learning, we reviewed a learning technique that has become increasingly popular during the last few years. We concentrated mainly on active learning in classi cation tasks, thereby neglecting a number of approaches for active learning in regression, e.g. [57, 58, 59]. The aim of active learning is to minimize the cost of data acquisition. The basic assumption in active supervised learning is that unlabeled data are cheap but that it is expensive to obtain the labels of the data. In other words, in active learning the information that already is available ....

K. Fukumizu. Active learning in multilayer perceptrons. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8, pages 295-301, Cambridge, MA, 1996. MIT Press.

....learning is classified into two categories depending on the optimality. One is global optimal, where a set of all training examples is optimal (e.g. Fedorov [3] The other is greedy optimal, where the next training example to sample is optimal in each step (e.g. MacKay [5] Cohn [2] Fukumizu [4], and Sugiyama and Ogawa [10] In this paper, we focus on the global optimal case and give a new active learning method in trigonometric polynomial networks. The proposed method does not employ any approximations in its derivation, so that it provides exactly the optimal generalization ....

K. Fukumizu. Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8, pp. 295--301. The MIT Press, Cambridge, 1996.

....many scientific experiments or learning of sensorimotor maps of multijoint robot arms. Learning can be performed more e#ciently if we can actively design input signals. The problem of designing input signals for optimal generalization is called active learning (Cohn, Ghahramani, Jordan, 1996; Fukumizu, 1996; Vijayakumar Ogawa, 1999) It is also referred to as optimal experiments (Kiefer, 1959; Fedorov, 1972; Cohn, 1994) or query construction (Sollich, 1994) Reinforcement learning (Kaelbling, 1996) which has been extensively studied recently in the field of machine learning, can be regarded as ....

....of Bayesian statistics, MacKay (1992) derived a criterion for selecting the most informative training data for specifying the parameters of neural networks. Cohn (1994, 1996) and Cohn, Ghahramani, and Jordan (1996) gave an active learning criterion for minimizing the variance of the estimator. Fukumizu (1996) proposed an active learning method in multi layer perceptrons using asymptotic approximation for estimating the generalization error. Essentially, the criteria derived in these papers are Incremental Active Learning for Optimal Generalization 3 equivalent to the A optimal design shown in ....

[Article contains additional citation context not shown here]

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. S. Touretzky, M.

.... has been studied from two di#erent standpoints depending on the optimality: global optimal where a set of all sample points is optimal (e.g. Fedorov [2] Sugiyama and Ogawa [11] and greedy optimal where the next sample point to add is optimal in each step (e.g. MacKay [4] Cohn [1] Fukumizu [3]) Generally, the global optimal methods give better generalization capability than the greedy optimal methods. However, the global optimal results have been obtained only for restricted cases. In contrast, the greedy optimal methods have been derived under general conditions. Even so, the greedy ....

....(Am ) 0 has been attained at the end of Stage 1. This means that, in Stage 2, the constraint #m 1 # R(A # m ) does not have to be taken into account. In the statistical active learning methods devised so far, the bias of the estimator is assumed to be zero (MacKay [4] Cohn [1] Fukumizu [3]) The assumption of zero bias is equivalent to that f belongs to H and En fm agrees with f . In contrast, the condition assumed in our framework is only f # H . The di#erence between f and En fm is explicitly evaluated in Stage 1 in spite of the fact that the bias is unknown. Based on the ....

[Article contains additional citation context not shown here]

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

....in many scientific experiments or learning of sensorimotor maps of multijoint robot arms. Learning can be performed more e#ciently if we can actively design input signals. The problem of designing input signals for optimal generalization is called active learning (Cohn et al. 5] Fukumizu [8], Vijayakumar and Ogawa [23] It is also referred to as optimal experiments (Fedorov [7] Cohn [3] or query construction (Sollich [14] Reinforcement learning (Kaelbling [10] which has been extensively studied recently in the field of machine learning, can be regarded as another form of ....

....depending on the optimality. One is the global optimal, where a set of all training examples is optimal (e.g. Fedorov [7] Sugiyama and Ogawa [21] The other is the greedy optimal, where the next training example to sample is optimal in each step (e.g. MacKay [11] Cohn [3] 4] Fukumizu [8]) In this paper, we focus on the latter greedy case. Within the framework of Bayesian statistics, MacKay [11] derived a criterion for selecting the most informative training data for specifying the parameters of neural networks. Cohn [3] and Cohn et al. 5] gave an active learning criterion for ....

[Article contains additional citation context not shown here]

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

....to add, an option that is not permitted in this work. Here, we look at the learning problem from a functional analytic perspective and define an optimization measure which decides on the usefulness of the training data. Works based on the Shannon entropy and Fisher s 1 information criterion [2] already exist. We use the Averaged Projection criterion described in Section 3.1, a criterion which enforces a trade o# between expanding the approximation space and reducing the noise variance. The training data selection scheme developed here works in two phases. At first, a batch selection of ....

K. Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems, Vol. 8, pp. 295--301. MIT Press, 1996.

....learning is classified into two categories depending on the optimality. One is global optimal, where a set of all training examples is optimal (e.g. Fedorov [4] The other is greedy optimal, where the next training example to sample is optimal in each step (e.g. MacKay [6] Cohn [3] Fukumizu [5], and Sugiyama and Ogawa [14] In this paper, we focus on the global optimal case and give a new method of active learning in trigonometric polynomial networks. The proposed method does not employ any approximations in its derivation, so that it provides exactly the optimal generalization ....

K. Fukumizu. Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8, pp. 295--301. The MIT Press, Cambridge, 1996.

....in the fields of mathematical statistics [6, 24] machine learning [8] and computational learning theory [2] as well as in the field of neural networks 1 input hidden output # 1 # 2 # L . f 0 (x) layer layer layer Figure 1: A three layer feedforward neural network. [9, 7]. However, most of the studies do not directly aim at the optimal generalization. In this paper, we give a new method of active learning. The presented method provides exactly the optimal generalization capability in the trigonometric polynomial space, where the concept of POBs is crucial. By ....

K. Fukumizu, Active learning in multilayer perceptrons, in "Advances in Neural Information Processing Systems 8" (David S. Touretzky, Michael C. Mozer, and Michael E. Hasselmo, Eds.), pp. 295--301, The MIT Press, 1996.

....generalization ability with fewer training samples. In the query algorithms proposed so far, the queries are either chosen according to some heuristic [1, 2, 3] or in a principled way by optimizing an objective function such as the expected information gain of a query [4, 5] or model uncertainty [6, 7, 8, 9]. A common feature of these query algorithms is that they have been applied to global learning algorithms only, such as multi layer perceptrons, although not all of them are constrained to a certain architecture. In the heuristic approaches to active learning, the idea is that the best questions ....

K. Fukumizu, "Active learning in multilayer perceptrons", in D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (eds) Advances in Neural Information Processing Systems, Vol. 8, pp. 295--301, MIT Pr., Cambridge, 1996.

....example to add, an option that is not permitted in this work. Here, we look at the learning problem from a functional analytic perspective and de ne an optimization measure which decides on the usefulness of the training data. Works based on the Shannon entropy and Fisher s information criterion [2] already exist. We use the Averaged Projection criterion described in Section 3.1, a criterion which enforces a trade o between expanding the approximation space and reducing the noise variance. The training data selection scheme developed here works in two phases. At rst, a batch selection of m ....

K. Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems, Vol. 8, pp. 295-301. MIT Press, 1996.

....learning Passive learninig Fig. 3. Deterministic active learning Deterministic active learning 15 10 5 0 5 10 15 15 10 5 0 5 10 15 Passive learning 15 10 5 0 5 10 15 15 10 5 0 5 10 15 Fig. 4. Distributions of input data This is a slight refinement of the method proposed by Fukumizu ([17]) The other is the multi point search active learning, which generates a finite number of input points as candidates and selects the best one. In both methods, we introduce randomness which is expected to solve the problem of excessive localization. A.1 Parametric active learning Instead of ....

....problem in this section. This method is first introduced in Fukumizu FUKUMIZU: STATISTICAL ACTIVE LEARNING IN MULTILAYER PERCEPTRON 7 Parametric method 15 10 5 0 5 10 15 15 10 5 0 5 10 15 Multi point search method 15 10 5 0 5 10 15 15 10 5 0 5 10 15 Fig. 7. Distributions of input data ([17]) and we give its full description here. B. Pruning for regularity of a Fisher information matrix Our pruning technique is based on the following theorem. Theorem 1 ( 12] The Fisher information matrix of a three layer perceptron at a parameter = w 11 ; wMH ; j 1 ; jM ; u 11 ....

K. Fukumizu. Active learning in multilayer perceptrons. In D. S. Touretzky et al., editor, Advances in Neural Information Processing Systems 8, pages 295--301, Cambridge, 1996. MIT Press.

No context found.

K. Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems, Vol. 8, pp. 295--301. MIT Press, 1996.

No context found.

K. Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems, Vol. 8, pp. 295-301. MIT Press, 1996.

No context found.

Kenji Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems 8, pages 295--301. MIT Press, 1996.

No context found.

Kenji Fukumizu. Active learning in multilayer perceptrons. In Advances in Neural Information Processing Systems 8, pages 295--301. MIT Press, 1996. 60

No context found.

K. Fukumizu, "Active learning in multilayer perceptrons, " in Advances in Neural Information Processing Systems 8 (D. Touretzky, M. Mozer, and M. Hasselmo, eds.), pp. 295--301, Cambridge, MA: MIT Press, 1996.

No context found.

K. Fukumizu, "Active learning in multilayer perceptrons," in Advances in Neural Information Processing Systems 8, D. Touretzky, M. Mozer, and M. Hasselmo, Eds. Cambridge, MA: MIT Press, 1996, pp. 295-- 301.

No context found.

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

No context found.

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

No context found.

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

No context found.

Fukumizu, K. (1996). Active learning in multilayer perceptrons. In D. Touretzky et al. (Eds.), Advances in Neural Information Processing Systems 8 (pp. 295--301). Cambridge: MIT Press.

No context found.

K. Fukumizu, "Active learning in multilayer perceptrons," in Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, eds., vol. 8, pp. 295--301, The MIT Press, 1996.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC