Statistical Reinforcement Learning

Masashi Sugiyama received his bachelor, master, and doctor of engineering degrees in computer science from the Tokyo Institute of Technology, Japan. In 2001 he was appointed assistant professor at the Tokyo Institute of Technology and he was promoted to associate professor in 2003. He moved to the University of Tokyo as professor in 2014.He received an Alexander von Humboldt Foundation Research Fellowship and researched at Fraunhofer Institute, Berlin, Germany, from 2003 to 2004. In 2006, he received a European Commission Program Erasmus Mundus Scholarship and researched at the University of Edinburgh, Scotland. He received the Faculty Award from IBM in 2007 for his contribution to machine learning under non-stationarity, the Nagao Special Researcher Award from the Information Processing Society of Japan in 2011, and the Young Scientists' Prize from the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology for his contribution to the density-ratio paradigm of machine learning.His research interests include theories and algorithms of machine learning and data mining, and a wide range of applications such as signal processing, image processing, and robot control. He published Density Ratio Estimation in Machine Learning (Cambridge University Press, 2012) and Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation (MIT Press, 2012).

This book by Prof. Masashi Sugiyama covers the range of reinforcement learning algorithms from a fresh, modern perspective. With a focus on the statistical properties of estimating parameters for reinforcement learning, the book relates a number of different approaches across the gamut of learning scenarios... It is a contemporary and welcome addition to the rapidly growing machine learning literature. Both beginner students and experienced researchers will find it to be an important source for understanding the latest reinforcement learning techniques. -Daniel D. Lee, GRASP Laboratory, School of Engineering and Applied Science, University of Pennsylvania

• Introduction to Reinforcement Learning. Model-Free Policy Iteration. Policy Iteration with Value Function Approximation. Basis Design for Value Function Approximation. Sample Reuse in Policy Iteration. Active Learning in Policy Iteration. Robust Policy Iteration. Model-Free Policy Search. Direct Policy Search by Gradient Ascent. Direct Policy Search by Expectation-Maximization. Policy-Prior Search. Model-Based Reinforcement Learning. Transition Model Estimation. Dimensionality Reduction for Transition Model Estimation.