IEEE Newsletter

2014 Best oral presentation and paper at the IEEE

Systems, Man, and Cybernetics Society Conference

The winning paper is: Jeffrey S. Campbell, Sidney N. Givigi, and Howard M. Schwartz,

Multiple Model Q-Learning for Stochastic Reinforcement Delays, 2014 IEEE International

Conference on Systems, Man, and Cybernetics, October 5-8, 2014, San Diego, CA, USA

Abstract:

The main contribution of this work is a novel machine reinforcement learning algorithm for

problems where a Poissonian stochastic time delay is present in the agent's reinforcement

signal. Despite the presence of the reinforcement noise, the algorithm can craft a suitable

control policy for the agent's environment. The novel approach can deal with reinforcements

which may be received out of order in time or may even overlap, which was not previously

considered in the literature. The proposed algorithm is simulated and its performance is

compared to a standard Q-learning algorithm. Through simulation, the proposed method is

found to improve the performance of a learning agent in an environment with Poissonian-type

stochastically delayed rewards.

THE TEAM

The Autonomous Robotics Research Group (ARRG) includes researchers in three

institutions, the Royal Military College of Canada (RMCC), Carleton University and Queen's

University. The main interest of the group is in machine learning, especially Reinforcement

Learning, and multiple robotics, from ground robots to Unmanned Aerial Vehicles (UAVs) and

Autonomous Underwater Vehicles (AUVs). It counts with state-of-the-art lab facilities located

at RMCC with more than a dozen ground robots and ten UAVs.

Biographies:

Second Lieutenant Jeffrey S. Campbell is a pilot in the Royal Canadian Air Force. He

completed his undergraduate degree in 2012 at the Royal Military College of Canada in

Kingston, Canada. While there, he specialized in robotic control and worked on quadrotor

unmanned aerial vehicles. In 2014 he received his master's degree in electrical engineering

from Carleton University in Ottawa, Canada. His work there focused on unsupervised machine

learning with applications in mobile robotics. Jeff is currently posted at Defence Research and

Development Canada to work on over-the-horizon radar projects.

Sidney N. Givigi received his B.Sc. in Computer Science and an M.A.Sc. in Electrical

Engineering from the Federal University of Esp$\acute{i}$rito Santo, Brazil. He also received

his Ph.D. in Electrical and Computer Engineering from Carleton University, Canada. In 2009,

he joined the Department of Electrical and Computer Engineering of the Royal Military

College of Canada (RMCC) as an Assistant Professor. Sidney's research interests are mainly

focused on autonomous systems, especially the decentralized control of multiple vehicles,

learning and adaptation of autonomous robots and modeling of complex systems with Game

Theory.

Professor H.M. Schwartz received his B.Eng. degree from McGill University, Montreal,

Quebec and his M.S. degree and Ph.D. degree from M.I.T., Cambridge, Massachusetts.

His research interests include adaptive and intelligent control systems, robotics, system

modelling and system identification. His most recent research is in multi agent learning with

applications to teams of mobile robots.