Introduction to Statistical Machine Learning

Masashi Sugiyama received the degrees of Bachelor of Engineering, Master of Engineering, and Doctor of Engineering in Computer Science from Tokyo Institute of Technology, Japan in 1997, 1999, and 2001, respectively. In 2001, he was appointed Assistant Professor in the same institute, and he was promoted to Associate Professor in 2003. He moved to the University of Tokyo as Professor in 2014. He received an Alexander von Humboldt Foundation Research Fellowship and researched at Fraunhofer Institute, Berlin, Germany, from 2003 to 2004. In 2006, he received a European Commission Program Erasmus Mundus Scholarship and researched at the University of Edinburgh, Edinburgh, UK. He received the Faculty Award from IBM in 2007 for his contribution to machine learning under non-stationarity, the Nagao Special Researcher Award from the Information Processing Society of Japan in 2011 and the Young Scientists' Prize from the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology Japan for his contribution to the density-ratio paradigm of machine learning. His research interests include theories and algorithms of machine learning and data mining, and a wide range of applications such as signal processing, image processing, and robot control. Dr. Takashi Ishida is a Lecturer at Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The University of Tokyo. He is also affiliated with Department of Computer Science, Graduate School of Information Science and Technology and Department of Information Science, Faculty of Science. Dr. Ishida received his PhD from the University of Tokyo in 2021, advised by Prof. Masashi Sugiyama. Prior to that, he received the MSc from the University of Tokyo in September 2017 and the Bachelor of Economics from Keio University in March 2013.

• Part 1. Introduction1. Statistical Machine LearningPart 2. Statistics and Probability2. Random Variables and Probability Distributions3. Examples of Discrete Probability Distributions4. Examples of Continuous Probability Distributions5. Multidimensional Probability Distributions6. Examples of Multidimensional Probability Distributions7. Sum of Independent Random Variables8. Probability Inequalities9. Statistical Estimation10. Hypothesis TestingPart 3. Generative Approach to Statistical Pattern Recognition11. Pattern Recognition via Generative Model Estimation12. Maximum Likelihood Estimation13. Properties of Maximum Likelihood Estimation14. Model Selection for Maximum Likelihood Estimation15. Maximum Likelihood Estimation for Gaussian Mixture Models16. Nonparametric Estimation17. Bayesian Inference18. Analytic Approximation of Marginal Likelihood19. Numerical Approximation of Predictive Distribution20. Bayesian Mixture ModelsPart 4. Discriminative Approach to Statistical Machine Learning21. Learning Models22. Least Squares Regression23. Constrained Least Squares Regression24. Sparse Regression25. Robust Regression26. Least Squares Classification27. Support Vector Classification28. Probabilistic Classification29. Structured ClassificationPart 5. Further Topics30. Ensemble Learning31. Online Learning32. Confidence of Prediction33. Weakly Supervised Learning34. Transfer Learning35. Multitask Learning36. Linear Dimensionality Reduction37. Nonlinear Dimensionality Reduction38. Clustering39. Outlier Detection40. Change DetectionPart 6. Deep Learning41. Feedforward Neural Networks42. Neural Networks with Image Data43. Neural Networks with Sequential Data44. Learning from Limited Data45. Representation Learning46. Deep Generative Modelling47. Multimodal Learning