On large problems, reinforcement learning systems must use parameterized function approximators such as neural networks in order to generalize between similar situations and actions. In these cases there are no strong theoretical results on the accuracy of convergence, and computational results have been mixed. In particular, Boyan and Moore reported at last year's meeting a series of negative results in attempting to apply dynamic programming together with function approximation to simple control problems with continuous state spaces. In this paper, we present positive results for all the control tasks they attempted, and for one that is significantly larger. The most important differences are that we used sparse-coarse-coded function approximators (CMACs) whereas they used mostly global function approximators, and that we learned online whereas they learned offline. Boyan and Moore and others have suggested that the problems they encountered could be solved by using actual outcomes (...

We would like to express our thank you to Katja Viita and the Municipality of Asikkala for hosting us in a superb cabin during our stay. We are also very grateful to Eva Heiskanen for setting up the project and to Pasi Tainio from SYKE for providing us with helpful information on HINKU and setting us up with a biogas car- without these contributions the project would not have been possible. We are especially thankful to everyone agreeing to meet with us and answer our often abundant questions. A big thank you goes to the farmers of Asikkala, as well as to Pekka Grönlund from TEM. Finally, we would like to thank Åke Thidell for his support throughout the project. Mia, Raffaele, Simon and Riitta

for computing difference feature vectors, and the “nonface ” clusters we include in our distribution-based model, are both critical for the success of our system. Index Terms—Face detection, object detection, example-based learning, example selection, pattern recognition, view-based

• Explores tools and processes along with a few

In almost all computer applications, users must enter correct words for the desired objects or actions. For success without extensive training, or in first-tries for new targets, the system must recognize terms that will be chosen spontaneously. We studied spontaneous word choice for objects

techniques focus on different aspects and would make good complements. Insofar as the examples we used are representative, current VPLs are successful in achieving a good `closeness of match', but designers need to consider the `viscosity' (resistance to local change) and the `secondary notation

The technology for building knowledge-based systems by inductive inference from examples has been demonstrated successfully in several practical applications. This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one

the smallest l 1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP and BOB, including better sparsity, and super-resolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total

Abstract. Reinforcement learning policies face the exploration versus exploitation dilemma, i.e. the search for a balance between exploring the environment to find profitable actions while taking the empirically best action as often as possible. A popular measure of a policy’s success in addressing

. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power