Computational Models for Cognitive Vision

HIRANMAY GHOSH, PHD, was a Research Advisor to TATA Consultancy Services and an Adjunct Faculty Member with the National Institute of Technology Karnataka. During his long professional career, he has served several reputed organizations, including CMC, ECIL and C-DOT and TCS. He was an Adjunct Faculty Member with IIT Delhi, and with the National Institute of Technology Karnataka. He is a Senior Member of IEEE, Life Member of IUPRAI, and a Member of ACM.

• About the Author ixAcknowledgments xiPreface xiiiAcronyms xv1 Introduction 11.1 What Is Cognitive Vision 21.2 Computational Approaches for Cognitive Vision 31.3 A Brief Review of Human Vision System 41.4 Perception and Cognition 61.5 Organization of the Book 72 Early Vision92.1 Feature Integration Theory 92.2 Structure of Human Eye 102.3 Lateral Inhibition 132.4 Convolution: Detection of Edges and Orientations 142.5 Color and Texture Perception 172.6 Motion Perception 192.6.1 Intensity-Based Approach 192.6.2 Token-Based Approach 202.7 Peripheral Vision 212.8 Conclusion 243 Bayesian Reasoning for Perception and Cognition 253.1 Reasoning Paradigms 263.2 Natural Scene Statistics 273.3 Bayesian Framework of Reasoning 283.4 Bayesian Networks 323.5 Dynamic Bayesian Networks 343.6 Parameter Estimation 363.7 On Complexity of Models and Bayesian Inference 383.8 Hierarchical Bayesian Models 393.9 Inductive Reasoning with Bayesian Framework 413.9.1 Inductive Generalization 413.9.2 Taxonomy Learning 453.9.3 Feature Selection 463.10 Conclusion 474 Late Vision 514.1 Stereopsis and Depth Perception 514.2 Perception of Visual Quality 534.3 Perceptual Grouping 554.4 Foreground–Background Separation 594.5 Multi-stability 604.6 Object Recognition 614.6.1 In-Context Object Recognition 624.6.2 Synthesis of Bottom-Up and Top-Down Knowledge 644.6.3 Hierarchical Modeling 654.6.4 One-Shot Learning 664.7 Visual Aesthetics 674.8 Conclusion 695 Visual Attention 715.1 Modeling of Visual Attention 725.2 Models for Visual Attention 755.2.1 Cognitive Models 755.2.2 Information-Theoretic Models 775.2.3 Bayesian Models 785.2.4 Context-Based Models 795.2.5 Object-Based Models 815.3 Evaluation 825.4 Conclusion 846 Cognitive Architectures 876.1 Cognitive Modeling 886.1.1 Paradigms for Modeling Cognition 886.1.2 Levels of Abstraction 916.2 Desiderata for Cognitive Architectures 926.3 Memory Architecture 946.4 Taxonomies of Cognitive Architectures 976.5 Review of Cognitive Architectures 996.5.1 STAR: Selective Tuning Attentive Reference 1006.5.2 LIDA: Learning Intelligent Distribution Agent 1026.6 Biologically Inspired Cognitive Architectures 1056.7 Conclusions 1067 Knowledge Representation for Cognitive Vision 1097.1 Classicist Approach to Knowledge Representation 1097.1.1 First Order Logic 1117.1.2 Semantic Networks 1137.1.3 Frame-Based Representation 1147.2 Symbol Grounding Problem 1177.3 Perceptual Knowledge 1187.3.1 Representing Perceptual Knowledge 1197.3.2 Structural Description of Scenes 1207.3.3 Qualitative Spatial and Temporal Relations 1227.3.4 Inexact Spatiotemporal Relations 1247.4 Unifying Conceptual and Perceptual Knowledge 1277.5 Knowledge-Based Visual Data Processing 1287.6 Conclusion 1298 Deep Learning for Visual Cognition 1318.1 A Brief Introduction to Deep Neural Networks 1328.1.1 Fully Connected Networks 1328.1.2 Convolutional Neural Networks 1348.1.3 Recurrent Neural Networks 1378.1.4 Siamese Networks 1408.1.5 Graph Neural Networks 1408.2 Modes of Learning with DNN 1428.2.1 Supervised Learning 1428.2.1.1 Image Segmentation 1428.2.1.2 Object Detection 1448.2.2 Unsupervised Learning with Generative Networks 1448.2.3 Meta-Learning: Learning to Learn 1468.2.3.1 Reinforcement Learning 1488.2.3.2 One-Shot and Few-Shot Learning 1488.2.3.3 Zero-Shot Learning 1508.2.3.4 Incremental Learning 1508.2.4 Multi-task Learning 1528.3 Visual Attention 1548.3.1 Recurrent Attention Models 1558.3.2 Recurrent Attention Model for Video 1588.4 Bayesian Inferencing with Neural Networks 1598.5 Conclusion 1609 Applications of Visual Cognition 1639.1 Computational Photography 1639.1.1 Color Enhancement 1649.1.2 Intelligent Cropping 1669.1.3 Face Beautification 1679.2 Digital Heritage 1689.2.1 Digital Restoration of Images 1689.2.2 Curating Dance Archives 1709.3 Social Robots 1729.3.1 Dynamic and Shared Spaces 1739.3.2 Recognition of Visual Cues 1749.3.3 Attention to Socially Relevant Signals 1759.4 Content Re-purposing 1779.5 Conclusion 17910 Conclusion 18110.1 “What Is Cognitive Vision” Revisited 18110.2 Divergence of Approaches 18310.3 Convergence on the Anvil? 185References 187Index 215