Y. Kuniyoshi, M. Inaba, and H. Inoue : "Learning by watching: Extracting reusable task knowledge from visual observation of human performance," IEEE Trans. on Robotics and Automation, Vol. 10, No. 6, Dec. 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....to observe the human performing tasks in a natural setting. Kang and Ikeuchi use vision to observe human hand movement, while attempting to determine the segmentation of a task [33] Kuniyoshi et al. use vision 21 to extract a high level task strategy in their learning by watching technique [76]. Similarly, Tung and Kak use a dataglove and Polhemus sensor to monitor the human hand movement, and deduce an assembly plan [71] Assumptions are made about where and how the object is grasped. Finally, Tso and Liu provide a wrist device for the human to hold an object. The wrist has LED markers ....

Y.Kuniyoshi, M. Inaba, and H. Inoue, "Learning by watching: Extracting reusable task knowledge from visual observation of human performance," IEEE Transactions on Robotics and Automation, vol. 10, no. 6, pp. 799--822, December 1994.

....is to increase the reliability of our system by extracting more information that can be detected by vision or a data glove from human demonstration. 1. 1 State of the Art In recent years several robot programming systems were developed that follow the Programming by Demonstration (PbD) paradigm [2, 7, 14, 16, 19]. Most of these systems are focused on the task of reconstructing trajectories and manipulations a user demonstrates. Their goal is to reconstruct and repli cate demonstrations or at least a set of environmental states with the highest accuracy possible. Other systems try to abstract from the ....

....the task by himself, while the system uses sensors like data gloves, Figure 1: Classification features for PbD systems. cameras and haptic devices for tracking the environ ment and or the user interaction. Obviously, powerful sensor systems are required to gather as much data as available [27, 15, 7, 26, 28, 14, 10, 19, 18, 30]. Most of these systems regard effects in the environment, trajectories, operations and object positions. While observing effects in the environment requires high level cognitive functions observing trajectories of the user s hand and fingers is a well understood task. Finally the representation ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799 - 822, 1994.

.... of the International Conference on Fusion and Integration for intelligent Systems Washington December, 1996 Modelling Human Assembly Actions from Observation George V Paul, Yunde Jiar, Mark D Wheeler, and Katsushi Ikeuchi The Robotics Institute, Mellon University, Pittsburgh, PA 15213. Abstract Thispaper describes a system which can model an assembly task by a human. The actions are recorded in ....

.... of the International Conference on Fusion and Integration for intelligent Systems Washington December, 1996 Modelling Human Assembly Actions from Observation George V Paul, Yunde Jiar, Mark D Wheeler, and Katsushi Ikeuchi The Robotics Institute, Mellon University, Pittsburgh, PA 15213. Abstract Thispaper describes a system which can model an assembly task by a human. The actions are recorded in ....

[Article contains additional citation context not shown here]

Inaba, M, and Inoue, H, Learning by Watching: Extracting Reusable Task Knowledge from Visual Observa- tion of Human Performance,IEEE Trans on 10, No 6, 1994.

....is done through a Support Vector Machine, by using a time delay approach. 2 State of the art Realization of recognition and interpretation of continuous human action sequences is critical to PbD. Though, there are few publications regarding sensors including visual processing. Kuniyoshi et al. [18, 19] presented a system with a visual hand tracker module that is able to detect grips and drops of objects. However, only one type of grasping is classi ed and the hand is constrained to appear under a certain angle. Kang [15] used a data glove in combination with depth images computed from recorded ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning bywatching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10, 1994.

....Previous work has used one or a combination of two basic approaches. The first is to extract a policy or control strategy directly from the demonstration [2] 10] The second is to extract enough information about the goal and or task that a planner or learning algorithm can compute such a policy [9] [8] 26] 22] The two approaches can be combined. In this case, the planner somehow makes use of the detailed information observed in the human demonstration. For example, in learning a pendulum swingup task, Atkeson and Schaal [3] begin by tracking the hand trajectory of the human demonstrator. ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....the instruction is situated so that the agent can use the current state to help disambiguate advice. A closely related research area is that of imitation. Similar work has gone under many di#erent names: learning by demonstration [1, 2, 10] behavior cloning [25, 26] learning by watching [11], and behavior or agent imitation [6,17,18] In all the cases, the robot or agent is given an example or examples of some task being done successfully (often from a human demonstration) and the agent s goal is to perform the same task. The demonstration of the task can be seen as a set of advised ....

Y. Kuniyoshi., M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE. Trans. on Robotics and Automation, 10(6):799--822, 1994.

....is to increase the reliability of our system by extracting more information that can be detected by vision or a data glove from human demonstration. 1. 1 State of the Art In recent years several robot programming systems were developed that follow the Programming by Demonstration (PbD) paradigm [2, 7, 14, 16, 19]. Most of these systems are focused on the task of reconstructing trajectories and manipulations a user demonstrates. Their goal is to reconstruct and repli cate demonstrations or at least a set of environmental states with the highest accuracy possible. Other systems try to abstract from the ....

....the task by himself, while the system uses sensors like data gloves, cameras and haptic devices for tracking the environ ment and or the user interaction. Obviously, powerful Figure l: Classification features for PbD systems. sensor systems are required to gather as much data as available [27, 15, 7, 26, 28, 14, 10, 19, 18, 30]. Most of these systems regard effects in the environment, trajectories, operations and object positions. While observing effects in the environment requires high level cognitive functions observing trajectories of the user s hand and fingers is a well understood task. Finally the representation ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799 - 822, 1994.

....that have been successfully applied for teaching robots various tasks. However, the majority of robot teaching approaches to date has been focused on learning policies [6, 16] or temporally extended sensory motor skills [3] Techniques for learning complex task structures have also been presented [10], but they are highly sensitive to the structure of the environment and of the demonstration, and do not allow for further improvement or adaptation if the task is not learned correctly in the first trial. Our goal is to develop a flexible mechanism that allows a robot to learn and refine ....

....and if they do not, the corresponding situations are treated with appropriate feedback as described in our experiments. 8. RELATED WORK Successful approaches for acquiring high level task information from demonstration have been developed for robotic manipulators learning assembly problems [10], 7] Since they rely solely on passive observations of a teacher demonstration, these methods have to make use of complex computer vision techniques, in carefully structured environments, in order to infer all the information necessary for the task. In the mobile robot domain, the majority of ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. T-RA, 10:799--822, 1994.

....of force processing in section 5 which serves for better grasp analysis. 2 State of the art Realization of recognition and interpretation of continous human action sequences is critical to Pbd. Though, there are few publications regarding sensors including visual processing. Kuniyoshi et al. [14, 15] presented a system with a visual hand tracker module that is able to detect grips and drops of objects. However, only one type of grasping is classified and the hand is constrained to appear under a certain an gle. Kang [11] used a data glove in combination with depth images computed from ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge fiom visual observation of human performance. IEEE Transactions on Robotics and Automation, 10, 1994.

....it has achieved its goal In order for the robot to improve its performance, it must be able to measure to what degree its imitation is accurate and to recognize when there are errors. Imitation has been used as a mechanism for learning simple motor skills from observation, such as block stacking [89] or pendulum balancing [141] Imitation has also been applied to the learning of sensor motor associations [3] and for constructing task representations [116] 2.10. Intentionality Dennett [50] contends that humans use three strategies to understand and predict behavior. The physical stance ....

Y. Kuniyoshi, et al., Learning by watching: Extracting reusable task knowledge from visual observation of human performance, IEEE Transactions of the Robotics and Automation 10 (6) (1994).

....agents like me can give the learning agent additional information about its own capabilities and how these capabilities relate to its own objectives. A number of techniques have been developed to exploit this, including imitation [Demiris and Hayes, 1999; Mataric, 2002] learning by watching [Kuniyoshi et al. 1994] , teaching or programming by demonstration [Atkeson and Schaal, 1997] behavioral cloning [Sammut et al. 1992] andinverse reinforcement learning [Ng and Russell, 2000] Learning by observation of other agents has intuitive appeal; however, explicit communication about action capabilities ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....it has achieved its goal In order for the robot to improve its performance, it must be able to measure to what degree its imitation is accurate and to recognize when there are errors. Imitation has been used as a mechanism for learning simple motor skills from observation, such as block stacking[91] or pendulum balancing[144] Imitation has also been applied to the learning of sensor motor associations[3] and for constructing task representations[117] 2.2.8 Intentionality Dennett contends that humans use three strategies to understand and predict behavior[51] The physical stance ....

Y. Kuniyoshi et al., Learning by watch- ing: extracting reusable task knowledge from visual observation of human performance, IEEE Trans. on Rob. Autom. 10 (6) (1994).

....for the mapping of a demonstration to a robot system is the task representation and the task analysis. Often, the analysis of a demonstration takes place observing the changes in the scene undertaken by the user. These changes can be described using relational expressions or contact relations [14, 12, 16]. For generalising of a single demonstration mainly explanation based methods are used [15, 9] Those allow for an adequate generalisation taken from only one example (One Shot Learning) Approaches based on One shotlearning techniques are the only ones feasible for end users since giving many ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....protocol; in competitive situationswhere agents are unwilling to share information; and even when the other agents are unwilling to fulfil a teacher role. The ability of imitation to effect transfer between agents has been demonstrated by a number of researchers for a range of domains [6,8,1,3,9,17,11]Thesedomains,however,haveprimarily dealt with agents imitating other agents with essentially the same action set as themselves. One of the major aspirations of imitation research is the promise of learning from agents which are different in some way from the learner. In previous work [14] we ....

....Our work can be viewed (loosely) as falling within the formal imitation framework proposed by Nehaniv and Dautenhahn [13] who propose viewing imitation as the construction of mappings between the states, actions, and goals of different agents (see also the abstraction model of Kuniyoshi at al. [8]) However, key differences include the fact that we assume that state space mappings are given, that the mentor s actions are not directly observable, that the objectives (goals) of the mentor and learner may be different, and that our environments are stochastic. Furthermore, we do not require ....

Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue. Learning by watching: Extracting reusable task knowledge from visual observationof human performance. IEEETransactions on Robotics and Automation, 10(6):799--822, 1994.

.... Teaching a robot by demonstration requires establishing the important features of a task [8] for example task impedance [3] or observable behavior of a controlled object [4] Alternatively, tasks may be represented symbolically as a sequence of actions and desired state changes in the environment [17] [14] While it will always be necessary to understand something about the task in order to execute it properly, duplicating an action from demonstration may be more straightforward when the static and dynamic characteristics of the robot are similar to those of the person demonstrating the ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....the mentor s trajectory. Our work can be viewed loosely as falling within the framework proposed by Nehaniv and Dautenhahn [11] who view imitation as the process of constructing mappings between states, actions, and goals of different agents (see also the abstraction model of Kuniyoshi at al. [8]) Unlike their model, we assume that state space mappings are given, the mentor s actions are not directly observable, the goals of the mentor and learner may differ, and that environments are stochastic. Furthermore, we do not require that the learner explicitly duplicate the behavior of the ....

Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....imitation was focused on assembly task learning from observation. Typically, a series of arm trajectories of a human, performing object moving stacking tasks, were recorded either using a manipulandum, with the advantage of measuring directly the joint torques [4, 14, 27] or using video images [53, 32, 23]. Data were analyzed to remove inconsistencies and extract key features of movement.An industrial non human like robotic arm would then be trained to reproduce the trajectory which maximizes the data key features. These efforts constitute a significant body of 3 research in robotics, and ....

.... of neurons specialized to particular orientation of motion [44] and the observed performed mapping is based on Meltzoff s proposed innate visuo motor map [39] Following a similar research line, Kuniyoshi and collaborators achieved fine oculo motor control of a robot head for on line tracking [7, 32] and reproduction [12] of human torso motion by a humanoid robot. Schaal and Sternad explored the idea of creating complex human like movements from biologically motivated movement primitives [50, 51] Each degree of freedom of a robot s limb is assumed to have two independent abilities to create ....

M.I. Kuniyoshi and I. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, vol.10, no.6, pages 799--822, 1994.

....al. 1998, Dautenhahn, 1995) both use imitation to learn trajectories for mobile robots, where what is learned is to follow a specific path, such as a square or a star. In (Hayes and Demiris, 1994) a robot learns to navigate a maze by associating actions with different locations in the maze, and (Kuniyoshi et al. 1994) describes a robot that learns sequences of actions by imitation of a human performing an assembly task. Mor en, 1998) and (Sun and Peterson, 1998) both use techniques built on top of a Q learning system (Sutton and Barto, 1998) to learn sequences of actions. Mor en clusters commonly occurring ....

Kuniyoshi, Y., Inaba, M., and Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance.

....where agents are unwilling to share information; and even when other agents are unwilling to fulfill a teacher role. The ability of imitation to effect skill transfer between agents has been demonstrated in a range of domains (Atkeson Schaal, 1997; Billard Hayes, 1997; Hayes Demiris, 1994; Kuniyoshi et al. 1994; Mataric, 1998; Mitchell et al. 1985; Utgoff Clouse, 1991) These domains, however, have dealt with agents imitating other agents with essentially the same action set as themselves. Our goal is to extend the benefits of imitation to situations in which the capabilities of agents in the ....

.... Our work can be viewed loosely as falling within the formal imitation framework proposed by Nehaniv and Dautenhahn (1998) who propose viewing imitation as the model based process of constructing mappings between states, actions, and goals of different agents (see also the abstraction model of Kuniyoshi at al. 1994)) However, key differences include the fact that we assume that state space mappings are given, that the mentor s actions are not directly observable, that the objectives (goals) of the mentor and learner may be different, and that our environments are stochastic. Furthermore, we do not require ....

Kuniyoshi, Y., Inaba, M., & Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10, 799--822.

....our architecture using data obtained by the study of natural systems. This is an important difference between our work and work done in the fields of Robot Programming by Demonstration [Munch et al., 1994, Kaiser and Dillman, 1996] and Assembly Programming by Observation [Ikeuchi and Suehiro, 1992, Kuniyoshi, Inaba, and Inoue, 1994] which are largely interested in the industrial application of learning from human demonstrations. In the next section we will describe the hardware we are using in order to implement the biologicallyinspired architecture of [Demiris and Hayes, 1996] in the context of the deferred imitation of ....

Kuniyoshi Y., Inaba M. and Inoue H., "Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance", in IEEE Transactions on Robotics and Automation, Vol. 10, No. 6, Dec. 1994

....a means to enhance the learning of communication skills in autonomous robots. Here, the ability of imitating is built in. Imitation as a means to direct attention: Learning by imitation has been used in different experiments for teaching the robot new motor skills ( 24] 7] 21] 22] 25] [26], 19] While the robot replicates the demonstrator s movements, it learns to adjust its own motor parameters and perceptuo motor mapping to replicate at best the actions of the demonstrator. An important advantage of such an approach to roboticists is that it saves the tedious programmer s work ....

M.I. Kuniyoshi and I. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, vol.10, no.6, pages 799--822, 1994.

....by predictively matching observed sequences to known behaviors. Finally, a variety of research programs have aimed at training robots to perform single tasks by observing a human demonstrator. Schaal (1997) used a robot arm to learn a pendulum balancing task from constrained visual feedback, and Kuniyoshi, Inaba, and Inoue (1994) discussed a methodology for allowing a robot in a highly constrained environment to replicate a block stacking task performed by a human but in a different part of the workspace. Traditionally in robot social learning, the model is indifferent to the attempts of the observer to imitate it. In ....

....problem tractable in an indifferent environment, researchers have vastly simplified one or more aspects of the environment and the behaviors of the observer and the model. Many have simplified the problem by using only simple perceptions which are matched to relevant aspects of the task, such as Kuniyoshi, Inaba, and Inoues (1994) use of white objects on a black background without any distractors or Mataric, Williamson, Demiris, and Mohans (1998) placement of reflective markers on the humans joints and use of multiple calibrated infrared cameras. Others have assumed the presence of a single model which is always detectable ....

Y. Kuniyoshi, M. Inaba and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, vol. 10, no. 6, pages 799822, 1994.

....and Demiris, 1994) or over a hilly landscape (Dautenhahn, 1995) where the actions to be copied are restricted to a very small set such as move forward or turn right. Kuniyoshi et al. also used a small number of actions but tackled the problem of recognising a human teacher on an assembly task (Kuniyoshi et al. 1994). At present, the research does not satisfy the necessary developmental conditions from the Piagetian viewpoint. Firstly, there is no development through the sensory motor stage to allow the internalisation of the world. Secondly, the behaviours are always imitated explicitly rather than covertly. ....

Kuniyoshi, Y., Inaba, M., and Inoue, H. (1994). Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822.

....1995) Robot Programming by Demonstration (RPD, Heise, 1989) has been realized through a number of applications on different levels of both robot control and perception. ffl Demonstrations were proven to be suitable for the acquisition of new program schemata on task level (Segre, 1989) In (Kuniyoshi et al. 1994), sequences of video images were analyzed in order to generate assembly plans. Andreae, 1984) presented NODDY, a system which generates generalized programs by fusing several demonstrations. Single demonstrations and user intentions are the basis for the robot programs generated by the system ....

Kuniyoshi, Y., Inaba, M., and Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799-- 822.

....streams. Using demonstration for task specification can be viewed as a coarse type of imitation. It has been studied in the area of robotics, where a series of visual images of a human performing an object stacking task was recorded, segmented, interpreted, and then repeated by a robotic arm [19, 22, 18, 21]. The major focus on this work was on the perceptual part of the problem, that of segmenting the visual stream and interpreting the sub tasks. Demonstration has also been used for priming learning, so as to provide an initial policy and thus greatly simplify the learning process. 33] applied ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....allowing the robot to learn new skills (which would otherwise require complex programming) by the sole ability of observing another agent s performance. Imitation can be the direct means of learning the skill, as in the case of learning new motor skills (see, for instance, 13] 14] 15] 22] [36], 58] It can also be an indirect means of teaching. For instance, in our previous work, the robot s ability to imitate the teacher is used to lead the robot to make specific perceptual experiences upon which the robot grounds its understanding of a proto language [7, 10, 9] Section 5 discusses ....

....of theoretical models of animals imitation, which propose different decompositions of the underlying cognitive processes, have been proposed (see e.g. 25, 44] Computational models have also been proposed [59] among which the most relevant are those implemented in robots. Kuniyoshi et al. [36] did experiments in which a robot was able to reproduce a human demonstration of object manipulation. Recognition of movements was done by preprocessing visual input from fixed cameras placed above the scene. The robot s controller had a predefined set of possible hand and arm movements actions ....

M.I. Kuniyoshi and I. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, vol.10, no.6, pages 799--822, 1994.

....The first robotics work to address imitation was focused on assembly task learning from observation. Typically, a series of visual images of a human performing a simple object moving stacking tasks was recorded, segmented, interpreted, and then repeated by an industrial non human like robotic arm [19, 18, 26, 16, 21]. These efforts constitute a significant body of research in robotics, and contribute to video segmentation and understanding. However, they provide highly task specific solutions, with little flexibility for applying the same algorithm to imitation after types of movements and tasks. More recent ....

M.I. Kuniyoshi and I. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, vol.10, no.6, pages 799--822, 1994.

.... (subroutines; Thrun and Schwartz 1995) 4 Models in the latter category rely on domain speci c knowledge or procedures for segmentation and are thus not generic or autonomous; for example, Lin (1993) and Singh (1994) in reinforcement learning, Knoblock et al. (1994) in hierarchical planning, Kuniyoshi et al. (1991) in robotic learning, and so on. 5 In such situations, optimal actions at one point may be dependent on not only the current state but also states and actions occurring some time ago. See Sondik (1978) Monahan (1982) and Puterman (1994) for treatments of POMDPs (partially observable Markovian ....

Y. Kuniyoshi, M. Inaba, and H. Inoue, (1991). Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation.

....and thus, not suited for interactive learning or task learning where each case presented must be learned immediately, on line. The system must remember each training example presented and generalize to other location based on relatively few examples learned. For learning by watching, the work in [10] generates a sequence of robot control language as final outputs. Some efforts concentrate on learning task level [5] In [1] it is assumed that the target will maintain a regular and repetitive movement that can be analyzed and intercepted. In [6] the three dimensional movement of a manipulator ....

Yasuo Kuniyoshi, Masayuki Inaba, Hirockika Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Trans on Robotics and Automation, 10(6):799--822, Dec 1994.

....and of course in humans (e. g [9, 37, 55] Similarly, recent work showed that artificial agents can also benefit from mutual interactions [14] e.g. for improving performance in collaborative tasks (e.g. 27, 31] and from interactions with humans, e.g. for learning complex motor skills (e.g. [30, 52]) We view learning, communication 1 and imitation as important capabilities to possess by social robotic agents, and our previous work studied how these skills can be designed and used by physical autonomous agents. In this paper, we study grounding and use of communication among simulated ....

....disadvantage of our mutual following strategy is that it restricts the number and type of movements that can be taught to only those of motion. This limitation could however be overcome by using a more complex imitative strategy, as for instance the imitative scenario developed in [17] or in [30]. Then, it should be possible to teach concepts relative to movements of many more body actuators and also of more complex sequences of movements (compare with experiments in [3] We mentioned in the introduction that previous studies on the development and evolution of communication differ from ....

Y. Kuniyoshi, M.Inaba and H. Inoue, (1994), `Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance', IEEE Transactions on Robotics and Automation, vol.10, no.6, pp.799--822, Dec. 1994. 45

....In contrast, our work is aimed at computationally simpler gross motion primitives. 6. Extensions and Conclusion A key direction of our continuing work is in the application of the above primitives to the perceptual side of manipulation, in particular within an imitation learning framework (Kuniyoshi, Inaba Inoue 1994, Demiris, Rougeaux, Hayes, Berthouze Kuniyoshi 1997) Again inspired by biological data, which indicate that perceptual and motor systems may share some representational substrates, we have been working on adapting our motor primitives into sensori motor behaviors, capable of not only ....

Kuniyoshi, Y., Inaba, M. & Inoue, H. (1994), `Learning by watching: extracting reusable task knowledge from visual observation of human performance', IEEE Transactions on Robotics and Automation 10(6), 799--822.

....focused on assembly task learning from observation. Typically, a series of visual images of a human performing a simple object moving stacking tasks was recorded, segmented, interpreted, and then repeated by a robotic arm (Ikeuchi, Suehiro, Tanguy Wheeler 1990, Ikeuchi, Kawade Suehiro 1993, Kuniyoshi, Inaba Inoue 1994, Hovland, Sikka McCarragher 1996, Kaiser 1997) This work was aimed at repeating the final outcome of the observed behavior, not at imitating the process that brought the result about. Thus, the goal was to achieve task level imitation (Byrne Russon 1998) the main focus being on extracting ....

Kuniyoshi, Y., Inaba, M. & Inoue, H. (1994), `Learning by watching: extracting reusable task knowledge from visual observation of human performance', IEEE Transactions on Robotics and Automation 10(6), 799--822.

.... this elementary operative intelligence, which is used by humans unconsciously, in a computercontrolled robot [10] This observation motivates research in the field of Robot Skill Acquisition via Human Demonstration [1, 11, 7] which is an extension of Robot Programming by Human Demonstration [9] that deals with the aquisition of sensor based robot skills from human demonstrations (Fig. 1) Figure 1: Demonstration of a peg insertion skill for a Puma 260 manipulator. Most approaches to skill acquisition rely on application specific solutions [1, 11, 15, 16] They do not consider human ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....not exist. In the same way, evaluating the behavior of the system during execution of a given operation can be much more complex than assessing the status of a feeder based on binary sensors. The Programming by Human Demonstration paradigm (in this case RPD Robot Programming by Demonstration [2,3,4,11,15]) seems indicated to overcome this type of difficulties. According to this paradigm, complex systems are programmed by showing particular examples of their desired behavior. Usually, emphasis is put on robots learning from their own percetion of humans performing certain MODELS KB EXECUTION ....

Kuniyoshi, Y.; Inaba, M.; Inoue, H. (1994) Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance, IEEE Transactions on Robotics and Automation, vol. 10 (6), pp. 799-822.

....does not exist. In the same way, evaluating the behavior of the systemduring execution of a given operation can be much more complex than assessing the status of a feeder based on binary sensors. The paradigm of Programming by Human Demonstration (in this case Robot Programming by Demonstration [59, 68, 80, 91] seems indicated toovercome this type of difficulties. According to this paradigm, complex systems are programmed by showing particular examples of their desired behavior. In our approach, interaction with the human, seen as a tutor, is fundamental. Functions for training and learning are included ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994.

....the sequence of robotic actions indirectly by executing the task in front of a camera. The human actions are detected in the image sequence and translated into robotic actions. The details of the human action (e.g. how a part is taken by the hand) are used to guide the robotic action (e.g. Kuniyoshi et al. 1994 ] This coupling between the human action and the robot action asks for a tight integration of all system components which is difficult to achieve when systems become more complex. The strength of this approach are situations where unknown parts actions can occur. In this paper we will focus on ....

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Trans. on Robotics and Automation, 10(6):799--822, 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue : "Learning by watching: Extracting reusable task knowledge from visual observation of human performance," IEEE Trans. on Robotics and Automation, Vol. 10, No. 6, Dec. 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue : "Learning by watching: Extracting reusable task knowledge from visual observation of human performance," IEEE Trans. on Robotics and Automation, Vol. 10, No. 6, Dec. 1994.

No context found.

Kuniyoshi, Y., Inaba, M., and Inoue, H. (1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Trans. Robotics and Automation, 10:799--822.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10:799--822, 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Trans. on Robotics and Automation, 10(6), 1994.

No context found.

Yasuo Kuniyoshi, Masayuki Inaba, and Hirochika Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Transaction on Robotics and Automation, Vol. 10, No. 6, pp. 799--822, 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by Watching: Extracting Reusable Task Knowledge from Visual Observation of Human Performance. IEEE Trans. on Robotics and Automation, 10(6), 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799 -- 822, 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6):799--822, 1994. 111

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transaction on Robotics and Automation, 10(6):799--822, Dec 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue, "Learning by watching: Extracting reusable task knowledge from visual observations of human performance," IEEE Trans. Robot. Automat., vol. 10, pp. 799--822, Nov. 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6), 1994.

No context found.

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. In IEEE Transactions on Robotics and Automation, pages 799-822, 1994.

No context found.

Kuniyoshi, Y., Inaba, M. & Inoue, H. (1994), `Learning by watching: extracting reusable task knowledge from visual observation of human performance', IEEE Transactions on Robotics and Automation 10(6), 799-- 822.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC