Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of the object- and camerarelated parameters to robustly determine such object-related parameters as velocity and depth. Such determination of extrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking, grasping, and collision-free motion planning. For both applications, we use the Minnesota Robotic Visual Tracker (a single visual sensor mounted on the end-effecto...

Most early research in robotic visual tracking, especially prior to 1990, separated the vision processing and robot control aspects of the system. Recent attempts to solve the problem close the control loop by incorporating the output of the vision processing as an input to the control subsystem. The Controlled Active Vision framework describes one such approach wherein dynamic target, camera, and environmental factors are incorporated via adaptive controllers that utilize the Sum-of-Squared Differences (SSD) optical flow measurements as an input to the control loop. This paper describes recent work at the University of Minnesota's Artificial Intelligence, Robotics, and Vision Laboratory in developing the Minnesota Robotic Visual Tracker (MRVT), a Controlled Active Vision robotic testbed. In addition, enhancements to the basic SSD algorithm are presented that produce order-of-magnitude improvements over previously reported results.

Flexible operation of a robotic agent in an uncalibrated environment requires the ability to recover unknown or partially known parameters of the workspace through sensing. Of the sensors available to a robotic agent, visual sensors provide information that is richer and more complete than other sensors. In this paper we present robust techniques for the derivation of depth from feature points on a target's surface and for the accurate and high-speed tracking of moving targets. We use these techniques in a system that operates with little or no a priori knowledge of object- and camera-related parameters to robustly determine such objectrelated parameters as velocity and depth. Such determination of extrinsic environmental parameters is essential for performing higher level tasks such as inspection, exploration, tracking, grasping, and collision-free motion planning. For both applications, we use the Minnesota robotic visual tracker (MRVT) (a single visual sensor mounted on the end-effector of a robotic manipulator combined with a real-time vision system) to automatically select feature points on surfaces, to derive an estimate of the environmental parameter in question, and to supply a control vector based upon these estimates to guide the manipulator.