Deep Learning

Manel Martínez-Ramón, PhD, is King Felipe VI Endowed Chair and Professor in the Department of Electrical and Computer Engineering at the University of New Mexico in the United States. He earned his doctorate in Telecommunication Technologies at the Universidad Carlos III de Madrid in 1999. Meenu Ajith, PhD, is a Postdoctoral Research Associate in Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) at Georgia State University, Georgia Institute of Technology, and Emory University. She earned her doctorate degree in Electrical Engineering from the University of New Mexico in 2022. Her research interests include machine learning, computer vision, medical imaging, and image processing. Aswathy Rajendra Kurup, PhD, is a Data Scientist at Intel Corporation. She earned her doctorate degree in Electrical Engineering from the University of Mexico in 2022. Her research interests include image processing, signal processing, deep learning, computer vision, data analysis and data processing.

• About the Authors xvForeword xviiPreface xixAcknowledgment xxiAbout the Companion Website xxiii1 The Multilayer Perceptron 11.1 Introduction 11.2 The Concept of Neuron 21.3 Structure of a Neural Network 141.4 Activations 211.5 Training a Multilayer Perceptron 221.6 Conclusion 372 Training Practicalities 412.1 Introduction 412.2 Generalization and Overfitting 422.3 Regularization Techniques 452.4 Normalization Techniques 502.5 Optimizers 522.6 Conclusion 583 Deep Learning Tools 613.1 Python: An Overview 613.2 NumPy 723.3 Matplotlib 833.4 Scipy 973.5 Scikit-Learn 1073.6 Pandas 1163.7 Seaborn 1253.8 Python Libraries for NLP 1313.9 TensorFlow 1383.10 Keras 1413.11 Pytorch 1443.12 Conclusion 1494 Convolutional Neural Networks 1534.1 Introduction 1534.2 Elements of a Convolutional Neural Network 1534.3 Training a CNN 1604.4 Extensions of the CNN 1664.5 Conclusion 1845 Recurrent Neural Networks 1875.1 Introduction 1875.2 RNN Architecture 1885.3 Training an RNN 1915.4 Long-Term Dependencies: Vanishing and Exploding Gradients 1995.5 Deep RNN 2015.6 Bidirectional RNN 2035.7 Long Short-Term Memory Networks 2045.8 Gated Recurrent Units 2185.9 Conclusion 2216 Attention Networks and Transformers 2256.1 Introduction 2256.2 Attention Mechanisms 2276.3 Transformers 2426.4 BERT 2496.5 GPT-2 2566.6.1 Comparison between ViTs and CNNs 2646.7 Conclusion 2697 Deep Unsupervised Learning I 2737.1 Introduction 2737.2 Restricted Boltzmann Machines 2747.3 Deep Belief Networks 2787.4 Autoencoders 2797.5 Undercomplete Autoencoder 2847.6 Sparse Autoencoder 2857.7 Denoising Autoencoders 2877.8 Convolutional Autoencoder 2887.9 Variational Autoencoders 2917.10 Conclusion 2978 Deep Unsupervised Learning II 3018.1 Introduction 3018.2 Elements of GAN 3038.3 Training a GAN 3058.4 Wasserstein GAN 3098.5 DCGAN 3128.6 cGAN 3168.7 CycleGAN 3188.8 StyleGAN 3238.9 StackGAN 3288.10 Diffusion Models 3338.11 Conclusion 3389 Deep Bayesian Networks 3419.1 Introduction 3419.2 Bayesian Models 3429.3 Bayesian Inference Methods for Deep Learning 3449.4 Conclusion 352Problems 353List of Acronyms 355Notation 359Bibliography 365Index 387