Accelerators for Convolutional Neural Networks

ARSLAN MUNIR, PhD, is an Associate Professor in the Department of Computer Science of Kansas State University. He is also the Director of the Intelligent Systems, Computer Architecture, Analytics, and Security (ISCAAS) Laboratory at the university. JOONHO KONG, PhD, is an Associate Professor in the School of Electronics Engineering College of IT Engineering at Kyungpook National University, South Korea. MAHMOOD AZHAR QURESHI, PhD, is a Senior IP Logic Design Engineer at Intel Corporation in Santa Clara, California.

• About the Authors xiiiPreface xvPart I Overview 11 Introduction 31.1 History and Applications 51.2 Pitfalls of High-Accuracy DNNs/CNNs 61.2.1 Compute and Energy Bottleneck 61.2.2 Sparsity Considerations 91.3 Chapter Summary 112 Overview of Convolutional Neural Networks 132.1 Deep Neural Network Architecture 132.2 Convolutional Neural Network Architecture 152.3 Popular CNN Models 262.4 Popular CNN Datasets 302.5 CNN Processing Hardware 312.6 Chapter Summary 37Part II Compressive Coding for CNNs 393 Contemporary Advances in Compressive Coding for CNNs 413.1 Background of Compressive Coding 413.2 Compressive Coding for CNNs 433.3 Lossy Compression for CNNs 433.4 Lossless Compression for CNNs 443.5 Recent Advancements in Compressive Coding for CNNs 483.6 Chapter Summary 504 Lossless Input Feature Map Compression 514.1 Two-Step Input Feature Map Compression Technique 524.2 Evaluation 554.3 Chapter Summary 575 Arithmetic Coding and Decoding for 5-Bit CNN Weights 595.1 Architecture and Design Overview 605.2 Algorithm Overview 635.3 Weight Decoding Algorithm 675.4 Encoding and Decoding Examples 695.5 Evaluation Methodology 745.6 Evaluation Results 755.7 Chapter Summary 84Part III Dense CNN Accelerators 856 Contemporary Dense CNN Accelerators 876.1 Background on Dense CNN Accelerators 876.2 Representation of the CNNWeights and Feature Maps in Dense Format 876.3 Popular Architectures for Dense CNN Accelerators 896.4 Recent Advancements in Dense CNN Accelerators 926.5 Chapter Summary 937 iMAC: Image-to-Column and General Matrix Multiplication-Based Dense CNN Accelerator 957.1 Background and Motivation 957.2 Architecture 977.3 Implementation 997.4 Chapter Summary 1008 NeuroMAX: A Dense CNN Accelerator 1018.1 RelatedWork 1028.2 Log Mapping 1038.3 Hardware Architecture 1058.4 Data Flow and Processing 1088.5 Implementation and Results 1188.6 Chapter Summary 124Part IV Sparse CNN Accelerators 1259 Contemporary Sparse CNN Accelerators 1279.1 Background of Sparsity in CNN Models 1279.2 Background of Sparse CNN Accelerators 1289.3 Recent Advancements in Sparse CNN Accelerators 1319.4 Chapter Summary 13310 CNN Accelerator for In Situ Decompression and Convolution of Sparse Input Feature Maps 13510.1 Overview 13510.2 Hardware Design Overview 13510.3 Design Optimization Techniques Utilized in the Hardware Accelerator 14010.4 FPGA Implementation 14110.5 Evaluation Results 14310.6 Chapter Summary 14911 Sparse-PE: A Sparse CNN Accelerator 15111.1 RelatedWork 15511.2 Sparse-PE 15611.3 Implementation and Results 17411.4 Chapter Summary 18412 Phantom: A High-Performance Computational Core for Sparse CNNs 18512.1 RelatedWork 18912.2 Phantom 19012.3 Phantom-2D 20112.4 Experiments and Results 20912.5 Chapter Summary 218Part V HW/SW Co-Design and Co-Scheduling for CNN Acceleration 22113 State-of-the-Art in HW/SW Co-Design and Co-Scheduling for CNN Acceleration 22313.1 HW/SW Co-Design 22313.2 HW/SW Co-Scheduling 22813.3 Chapter Summary 23014 Hardware/Software Co-Design for CNN Acceleration 23114.1 Background of iMAC Accelerator 23114.2 Software Partition for iMAC Accelerator 23214.3 Experimental Evaluations 23514.4 Chapter Summary 23715 CPU-Accelerator Co-Scheduling for CNN Acceleration 23915.1 Background and Preliminaries 24015.2 CNN Acceleration with CPU-Accelerator Co-Scheduling 24215.3 Experimental Results 25115.4 Chapter Summary 25716 Conclusions 259References 265Index 285