This paper provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed. Since any visual servo system must be capable of tracking image features in a sequence of images, we include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control. 1 Introduction Today there are over 800,000 robots in the world, mostly working in factory environment...

This paper attempts to present a comprehensive summary of research results in the use of visual information to control robot manipulators and related mechanisms. An extensive bibliography is provided which also includes important papers from the elemental disciplines upon which visual servoing is based. The research results are discussed in terms of historical context, commonality of function, algorithmic approach and method of implementation. 1 Introduction This paper presents the history, and reviews current research into the use of visual information for the control of robot manipulators and mechanisms. Visual control of manipulators promises substantial advantages when working with targets whose position is unknown, or with manipulators which may be flexible or inaccurate. The reported use of visual information to guide robots, or more generally mechanisms, is quite extensive and encompasses manufacturing applications, teleoperation, missile tracking cameras, fruit picking as well...

In this paper we introduce visual compliance, a new vision-based control scheme that lends itself to task-level specification of manipulation goals. Visual compliance is effected by a hybrid vision/position control structure. Specifically, the two degrees of freedom parallel to the image plane of a supervisory camera are controlled using visual feedback, and the remaining degree of freedom (perpendicular to the camera image plane) is controlled using position feedback provided by the robot joint encoders. With visual compliance, the motion of the end effector is constrained so that the tool center of the end effector maintains "contact" with a specified projection ray of the imaging system. This type of constrained motion can be exploited for grasping, parts mating, and assembly. We begin by deriving the projection equations for the vision system. We then derive equations used to position the manipulator prior to the execution of visual compliant motion. Following this, we derive the h...

We are investigating how to program robots so that they learn from experience. Our goal is to develop principled methods of learning that can improve a robot's performance of a wide range of dynamic tasks. Our interest is in complex tasks such as throwing, catching, batting, yo-yoing, and juggling. We have developed one method of learning, task-level learning, that successfully improves a robot's performance of both a ball-throwing and a juggling task. With task-level

Abstract not found

The study of visual navigation problems requires the integration of visual processes with motor control processes. Most essential in approaching this integration is the study of appropriate spatio-temporal representations which the system computes from the imagery and which serve as interfaces to all cognitive and motor activities. Since representations resulting from exact quantitative reconstruction have turned out to be very hard to obtain, we argue here for the necessity of of representations which can be computed easily, reliably and in real time and which recover only the information about the 3D world which is really needed in order to solve the navigational problems at hand. In this paper we introduce a number of such representations capturing aspects of 3D motion and scene structure which are used for the solution of navigational problems implemented in visual servo systems. This research is supported by the National Science Foundation under Grant IRI-90-57934, the Office of N...

We describe a method for 3D visual manipulator control using a redundant camera system without explicit external or internal calibration. Under the assumption of a simple linear camera model a fusion equation is derived for which only three parameters have to be estimated regardless the number of cameras. In simulations as well as in real experiments the feasibility of our approach for a 3D positioning task of a six degree of freedom (DOF) Puma 200 to a target is demonstrated. It is shown that using redundant views increases positioning accuracy and fault tolerance. The achieved accuracy is sufficient to perform an additional insertion task. 1 Introduction Using a robot manipulator for an assembly task requires the ability to grasp and insert different parts. The common approach is to solve the 3D relationship between the robot and the environment based upon a 2D vision measurements. That in turn requires the internal and external camera parameters to be calibrated which is difficult...

We describ e a system that integrates real-time computer vision with a sensorless gripper to provide closed loop feedback control for grasping and manipulation tasks. Many hand-eye coordination skills can be thought of as sensory-control loops, where specialized reasoning has been embodied as a feedback or control path in the loop's construction. Our framework captures the essence of these hand-eye coordination skills in simple visual control primitives, which are a key component of the software integration. The primitives use a simple visual tracking and correspondence scheme to provide real-time feedback control in the presence of imprecise camera calibrations. Experimental results are shown for the positioning task of locating, picking up, and inserting a bolt into a nut under visual control. Results are alsopresented for the visual control of abolt tightening task. 1

. We describe a method for 3D visual manipulator control using a redundant camera system without explicit external or internal calibration. Under the assumption of a simple linear camera model, a fusion equation is derived for which only three parameters have to be estimated, regardless of the number of cameras. Distributed sensor-units provide the necessary measurements, which are fused together in a Kalman filter. In simulations, as well as in real experiments, the feasibility of our approach for a 3D positioning task of a six degree of freedom (DOF) Puma 200 to a target is demonstrated. It is shown that using redundant views increases positioning accuracy and fault tolerance. The achieved accuracy is sufficient to perform an insertion task. Key Words. uncalibrated vision, distributed sensing, sensor fusion, redundantcamera system 1 Introduction Using a robot manipulator for an assembly task requires the ability to grasp different parts. The common approach is to solve the 3D rela...

Sensors play an important role in providing feedback to robotic control systems. Vision can be an effective sensing modality due to its speed, low cost, and flexibility. It can also serve as an external sensor that can provide control information for devices that lack internal sensing. Many robot grippers do not have internal sensors. These grippers can be prone to error as they operate under open loop control and usually require a precise model of the environment to be effective. This paper describes 2 visual control primitives that can be used to provide position control for a sensorless robot system. The primitives use a simple visual tracking and correspondence scheme to provide real-time feedback control in the presence of imprecise camera calibrations. Experimental results are shown for the positioning task of locating, picking up, and inserting a bolt into a nut under visual control. Results are also presented for the visual control of a bolt tightening task. 1 Introduction In ...