This report describes a series of results using the exponentiated gradient descent (EG) method recently proposed by Kivinen and Warmuth. Prior work is extended by comparing speed of learning on a nonstationary problem and on an extension to backpropagation networks. Most significantly, we present an extension of the EG method to temporal-difference and reinforcement learning. This extension is compared to conventional reinforcement learning methods on two test problems using CMAC function approximators and replace traces. On the larger of the two problems, the average loss was approximately 25% smaller for the EG method. The relative computational complexity and parameter sensitivity of the two methods is also discussed. 1. Introduction This report presents the results of several experiments designed to compare two different methods for learning the weights of neural networks: gradient descent (GD) and exponentiated gradient descent (EG), in the framework of supervised and reinforce...

Planning Doina Precup, Richard S. Sutton University of Massachusetts Amherst, MA 01003 fdprecupjrichg@cs.umass.edu Abstract Planning and learning at multiple levels of temporal abstraction is a key problem for artificial intelligence. In this paper we summarize an approach to this problem

. Doina Precup and Kiri Wagstaff have been wonderful friends and supporters of my re-search. It is very helpful to have such smart women friends in CS. They provided support when I needed it and they pushed me when I needed that. I feel privileged to know Doina both as a mentor and as a friend. I thank

and support over this time. I would also like to thank Doina Precup for her invaluable contributions to my research and studies at McGill University. Special thanks goes to Yuri Grinberg, Amir-massoud Farahmand and Csaba Szepesvári for their contributions to my research and publications. I am also thankful

-structive guidance, to Doina Precup for providing invaluable advice throughout my BSc and MSc, to Mahdi Milani Fard for helping me to get the theory of CPSRs off the ground, and to Borja Balle for showing me a whole new perspec-tive on my own work. Special thanks to lab members Clement Gehring, Ouais Alsharif

to every member of the Reasoning and Learning lab, past and present, that I had the chance to inter-act with—this includes, among others, Doina Precup, Prakash Panangaden, Robert

Precup, for always encouraging ‘cool ideas’ and advising me in much of the work presented in this thesis. I am thankful to both Joelle and Doina for fostering the friendly lab atmosphere that was a key component in making my time in Montreal as enjoyable as it has been, and to all members of the Mc

would like to thank my program committee members Claude Crépeau and Doina Precup for their advice and support. I also thank my external examiner, Carlo Mereghetti, for many helpful corrections and comments. I wish to thank Patrick Hayden, Alexei Miasnokov, and John Liede for participating on my defence

was realized during Summer 2007 and funded by a NSERC USRA supervised by Dr. Doina Precup. 1

advisor, professor Doina Precup whose inspiring and thoughtful guidance, and supervi-sion made my thesis work possible. Discussions and regular meetings with Doina always helped shaping my thoughts and motivated me to work hard. During the past several years Doina has imparted innumerable lessons from