Theory of neural information processing systems

The book provides an excellent class-tested material for graduate courses in artificial neural networks. It is completely self-contained and includes also thorough introduction to the discussed discipline-specific areas of mathematics...Therefore, this book represents a good reference source of applicable ideas for a wide audience including students, researchers and application specialists as well.

• I INTRODUCTION TO NEURAL NETWORKS ; 1. General introduction ; 2. Layered networks ; 3. Recurrent networks with binary neurons ; II ADVANCED NEURAL NETWORKS ; 4. Competitive unsupervised learning processes ; 5. Bayesian techniques in supervised learning ; 6. Gaussian processes ; 7. Support vector machines for binary classification ; III INFORMATION THEORY AND NEURAL NETWORKS ; 8. Measuring information ; 9. Identification of entropy as an information measure ; 10. Building blocks of Shannon's information theory ; 11. Information theory and statistical inference ; 12. Applications to neural networks ; IV MACROSCOPIC ANALYSIS OF DYNAMICS ; 13. Network operation: macroscopic dynamics ; 14. Dynamics of online learning in binary perceptrons ; 15. Dynamics of online gradient descent learning ; V EQUILIBRIUM STATISTICAL MECHANICS OF NEURAL NETWORKS ; 16. Basics of equilibrium statistical mechanics ; 17. Network operation: equilibrium analysis ; 18. Gardner theory of task realizability ; APPENDICES ; A. Historical and bibliographical notes ; B. Probability theory in a nutshell ; C. Conditions for central limit theorem to apply ; D. Some simple summation identities ; E. Gaussian integrals and probability distributions ; F. Matrix identities ; G. The delta-distribution ; H. Inequalities based on convexity ; I. Metrics for parametrized probability distributions ; J. Saddle-point integration ; REFERENCES