A. W. Moore, C. Atkeson, and S. Schaal, Memory-based learning for control, Tech. Report CMU-RI-TR-95-18, Carnegie Mellon University, Pittsburg, Pennsylvania, 1995. 79  Home/Search   Document Details and Download   Summary   Related Articles   Check

....ecient use of the experience. If the learner incorporates experience merely by averaging in its current, awed state space granularity, it is bound to attribute experience to the wrong states. In the learning methods above, we can distinguish between representational tools and learning paradigms [23]. The representational tools in VRDP and parti game are kd trees. They are used to partition the state space, approximate the value function, and store learning instances. The learning paradigm de nes what the representation is used for, how training data is used to modify the representation, ....

A. W. Moore, C. Atkeson, and S. Schaal. Memory-based learning for control. Technical Report CMU-RI-TR-95-18, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, 1995.

....to be faced by an expert designer. From the human user point of view, a good way to define a behavior is to interact directly with the robot in the destination environment. Several methods have been proposed in this direction : Learning by demonstrations or from examples, Memory based learning [19, 1, 13] , Imitation [12, 11, 2, 14, 7] or Supervised Learning [21, 20] Those approaches focus on the learning of complex action sequences but they rely on simple predefined and constrained percepts most use well known shapes, centroids of simple color objects and minimal environments. In our ....

A. Moore, C. Atkeson, and S. Schaal, `Memory-based learning for control ', Technical Report CMU-RI-TR-95-18, CMU Robotics Institute, (1995).

....parallel arrangement, the bucket cylinder with the lower resistance draws a greater fraction of the total flow. FIGURE 9. Section of the response surface for the boom joint of the HEX The memory based learning was performed using a software tool Vizier developed by Schneider et al. 24] 19][20]) at Carnegie Mellon University. Vizier allows the use of locally weighted linear regression to learn data sets and make predictions. The best parameters for the regression can be determined using a blackbox 0 2 4 6 8 10 12 0 5 10 15 0.05 0 0.05 0.1 0.15 Boom Spool Command Bucket Spool Command ....

Moore, A., Atkeson, C., Schaal, S., "Memory-based Learning for Control", 1995, CMU Technical Report CMU-RI-TR-95-18.

....Function Approximators Another class of sparse coarse coded memory is memory based. Although, memory based function approximators have not been widely used in conjunction with reinforcement learning, they are common in other tasks such as classification (e.g. 8] and robot control (e.g. [2]) but see [16, 14, 11] In a memory based function approximator, each memory element represents some of the state action pairs or a case the agent has experienced before. A query is performed by first retrieving the nearest neighbors to the query point according to some similarity metric and ....

C. G. Atkeson. Memory-based learning control. In Proceedings of the 1991 American Control Conference, volume 3, pages 2131--2136, Boston, MA, 1991.

....Using learning techniques results in a more autonomous operation and is much preferable for mobile robots. Most of the existing learning systems for image recognition use some type of neural network (NN) The approach reported in this paper uses a different scheme memory based learning (MBL) [3]. While having similar learning power to that of NN, MBL has an important advantage over NN it does not require any training, as opposed to the sometimes prohibitively long training time of NN systems. Instead, the computational load is shifted to run time. This, however, is not necessarily a ....

....can be achieved by the MBL scheme described in the next section. Furthermore, as is typical of all MBL systems, it has zero training time. 3 Fast Visual MBL MBL, also known as lazy or delayed learning, is a general function approximation method that matches input vectors x to output vectors y [3]. It differs from NN in the way it treats the training examples. Instead of using them in order to adjust a set of parameters (weights and biases) the example pairs (x; y) are simply stored in memory until run time. When a query xnew is made, the learning algorithm computes the distances from ....

[Article contains additional citation context not shown here]

Moore, A.W., Atkeson, C.G., and Schaal, S. (1995). Memory-based Learning for Control, CMU Robotics Institute Technical Report CMU-RI-TR-95-18, April 1995.

....Function Approximators Another class of sparse coarse coded memory is memory based. Although, memory based function approximators have not been widely used in conjunction with reinforcement learning, they are common in other tasks such as classification (e.g. 8] and robot control (e.g. [2]) but see [16, 14, 11] In a memory based memory, each memory element represents some of the state action pairs or a case the agent has experienced before. A query is performed by first retrieving the nearest neighbors to the query point according to some similarity metric and then performing a ....

Atkeson, C.G. (1991). Memory-Based Learning Control. In Proceedings of the 1991 American Control Conference, Vol. 3, pp. 2131--6, Boston, Massachusetts, 1991.

....refer to [1, 14] instead we only express a general opinion that they are usually very expensive in terms of a number of network configurations to try before coming up with a satisfactory one. In contrast, new optimization techniques emerging from experimental design [2] and memory based learning [11], offer a very attractive way of using both the computational time and the empirical knowledge about the neural network s performance so far. In this paper we describe a concept and preliminary results obtained with memory based stochastic optimization [10] used to spot useful and statistically ....

....With the above described stochastic approach to validation we have managed to turn a deterministic machine learning algorithm into a random process relevant to the memory based optimization. Memory based stochastic optimization [10] is a new technique combining memory based learning [11] with experimental design methods [2] It differs form a conventional numerical optimization because it accepts noisy samples, operates on non linear or just locally linear approximations of the objective function surface, makes use of uncertainty of the maintained objective function model, and ....

[Article contains additional citation context not shown here]

Moore A.W., Atkeson C.G., Schaal S. Memory-based Learning for Control. Technical Report CMU-RI-TR-95-18, The Robotics Institute, Carnegie Mellon University, 1995.

....Approximators Another class of sparse coarse coded memory is memory based. Although memorybased function approximators have not been widely used in conjunction with reinforcement learning, they are common in other tasks such as classification (e.g. Kibler and Aha, 1989) and robot control (e.g. Atkeson, 1991) (but see Ram and Santamar ia, 1997; Peng, 1993; McCallum et al. 1995) In a memory based function approximator, each memory element represents some of the state action pairs or a case the agent has experienced before. A query is performed by first retrieving the nearest neighbors to the query ....

Atkeson, C. G. (1991). Memory-based learning control. In Proceedings of the 1991 American Control Conference, Vol. 3, pp 2131--2136, Boston, MA.

....Another class of sparse coarse coded memory is memory based. Although, memory based function approximators have not been widely used in conjunction with reinforcement learning, they are common in other tasks such as classification (e.g. Kibler and Aha, 1989] and robot control (e.g. [Atkeson, 1991]) but see [Ram and Santamar ia, 1997; Peng, 1993; McCallum et al. 1995] In a memory based function approximator, each memory element represents some of the state action pairs or a case the agent has experienced before. A query is performed by first retrieving the nearest neighbors to the query ....

Atkeson, C. G. (1991). Memory-based learning control. In Proceedings of the 1991 American Control Conference, volume 3, pages 2131--2136, Boston, MA.

....to the development of new algorithms that can learn optimal policies for the control of dynamic systems described in continuous state and action spaces. While there have been successful attempts to adapt discrete space algortihms to such systems by discretizing the continuous variables (Boyan and Moore, 1995; Atkeson, 1994) this approach does not scale up favorably because of the well known curse of dimensionality , accompanying algorithms based on dynamic programming (Bellman, 1957) Another existing approach is that of Baird and Klopf (1993) based on a special parametric representation ( control ....

....G(x) are Gaussians centered at the nodes x (t) G t (x) e Gamma (x Gammax (t) 2 h (t) G t (x) where h is the width of the kernel and (t) are weights. The approximated value function V (x) at an arbitrary point x in state space is calculated by means of kernel regression (Moore et al. 1995): V (x) P T Gamma1 t=0 (t) V (x (t) P T Gamma1 t=0 (t) 2) We can differentiate directly equation (2) to obtain the gradient of the value function rxV , instead of approximating it by the difference of two values of V (x) A single component of the gradient will be: V x ....

[Article contains additional citation context not shown here]

Moore, A.W., Atkeson, C.G, and Schaal, S. (1995). Memory-based learning for control. Technical Report CMU-RI-TR-95-18, CMU Robotics Institute, 1995.

....the wide variety of function approximation techniques for supervised learning that support noisy training examples. Popular techniques include various neuralnetwork methods #Rumelhart McClelland, 1986#, fuzzy logic #Berenji, 1991; Lee, 1991#. CMAC #Albus, 1981#, and local memory based methods #Moore, Atkeson, Schaal, 1995#, such as generalizations of nearest neighbor methods. Other mappings, especially the policy 259 Kaelbling, Littman, Moore mapping, typically need specialized algorithms because training sets of input output pairs are not available. 6.1 Generalization over Input A reinforcement learning ....

....reinforcement learning techniques to be applied in large state spaces is modeled on value iteration and Q learning. Here, a function approximator is used to represent the value function by mapping a state description to a value. Many reseachers have experimented with this approach: Boyan and Moore #1995# used local memory based methods in conjunction with value iteration; Lin #1991# used backpropagation networks for Q learning; Watkins #1989# used CMAC for Q learning; Tesauro #1992, 1995# used backpropagation for learning the value function in backgammon #described in Section 8.1#; Zhang and ....

[Article contains additional citation context not shown here]

Moore, A. W., Atkeson, C. G., & Schaal, S. #1995#. Memory-based learning for control. Tech. rep. CMU-RI-TR-95-18, CMU Robotics Institute.

....in high dimensions, where it may be more effective to head in a known positive gradient without waiting for all the experiments that would be needed for a precise estimate of steepest gradient. ffl Other pros and cons of locally weighted regression in the context of control can be found in [ Moore et al. 1995 ] 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 True expected value as function of (x,y) Contours from 0.0511 to 0.958 in increments of 0.0394 0.544 0.917 0.642 0.243 0.711 0.35 0.107 0.59 0.211 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 True expected value as function of (x,y) Contours from 0.0399 to 0.957 in ....

A. W. Moore, C. G. Atkeson, and S. Schaal. Memory-based Learning for Control. Technical report, CMU Robotics Institute, Technical Report CMU-RI-TR-95-18 (Submitted for Publication), 1995.

....using straightforward supervised learning, and can be handled using any of the wide variety of function approximation techniques for supervised learning that support noisy training examples. Popular methods include various neural network methods [76] CMAC [2] and local memory based methods [67], such as generalizations of nearest neighbor methods. Other mappings, especially the policy mapping, typically need specialized algorithms because training sets of input output pairs are not available. 6.1 Generalization over Input A reinforcement learning agent s current state plays a central ....

Andrew W. Moore, Christopher G. Atkeson, and S. Schaal. Memory-based learning for control. Technical Report CMU-RI-TR-95-18, CMU Robotics Institute, 1995.

....supervised learning, and can be handled using any of the wide variety of function approximation techniques for supervised learning that support noisy training examples. Popular techniques include various neural network methods [94] fuzzy logic [11, 58] CMAC [3] and local memory based methods [84], such as generalizations of nearest neighbor methods. Other mappings, especially the policy mapping, typically need specialized algorithms because training sets of input output pairs are not available. 6.1 Generalization over Input A reinforcement learning agent s current state plays a central ....

Andrew W. Moore, Christopher G. Atkeson, and S. Schaal. Memory-based learning for control. Technical Report CMU-RI-TR-95-18, CMU Robotics Institute, 1995.

....of the wide variety of function approximation techniques for supervised learning that support noisy training examples. Popular techniques include various neuralnetwork methods (Rumelhart McClelland, 1986) fuzzy logic (Berenji, 1991; Lee, 1991) CMAC (Albus, 1981) and local memory based methods (Moore, Atkeson, Schaal, 1995), such as generalizations of nearest neighbor methods. Other mappings, especially the policy Kaelbling, Littman, Moore mapping, typically need specialized algorithms because training sets of input output pairs are not available. 6.1 Generalization over Input A reinforcement learning agent s ....

....allow reinforcement learning techniques to be applied in large state spaces is modeled on value iteration and Q learning. Here, a function approximator is used to represent the value function by mapping a state description to a value. Many reseachers have experimented with this approach: Boyan and Moore (1995) used local memory based methods in conjunction with value iteration; Lin (1991) used backpropagation networks for Q learning; Watkins (1989) used CMAC for Q learning; Tesauro (1992, 1995) used backpropagation for learning the value function in backgammon (described in Section 8.1) Zhang and ....

[Article contains additional citation context not shown here]

Moore, A. W., Atkeson, C. G., & Schaal, S. (1995). Memory-based learning for control.

No context found.

A. W. Moore, C. Atkeson, and S. Schaal, Memory-based learning for control, Tech. Report CMU-RI-TR-95-18, Carnegie Mellon University, Pittsburg, Pennsylvania, 1995. 79

No context found.

A. W. Moore, C. G. Atkeson, and S. Schaal. Memory-based learning for control. Technical Report CMU-RI-TR-95-18, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA, 1995.

No context found.

A. W. Moore, C. Atkeson, and S. Schaal, Memory-based learning for control, Tech. Report CMU-RI-TR-95-18, Carnegie Mellon University, Pittsburg, Pennsylvania, 1995. 79

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC