Video Tracking

Dr Emilio Maggio, Vicon, UKDr Maggio is Computer Vision Scientist at Vicon, the motion capture worldwide market leader. From 2004 – 2008 he was a Ph.D. student at the Department of Electronic Engineering, Queen Mary, University of London. In 2005 and again in 2007 he was awarded the best student paper prize at ICASSP. Dr Maggio has acted as a reviewer for the IEEE Transactions on Circuits and Systems for Video Technology, the International Journal of Image and Graphics and ACM Multimedia. Dr Andrea Cavallaro, School of Electronic Engineering and Computer Science, Queen Mary, University of London, UKDr Cavallaro is Reader in Multimedia Signal Processing at Queen Mary, University of London. He is the author of more than 70 papers, including 5 book chapters. He is an elected member of the IEEE Signal Processing Society, Multimedia Signal Processing Committee. He has been a member of the organizing/ technical committee for several international conferences such as Technical Chair of EUSIPCO 08 and General Chair of the IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS 2007), with General Chair positions being held for forthcoming 2009 conferences such as BMVC 09. He has been guest editor of several special issues, including 'Multi-sensor object detection and tracking', Signal, Image and Video Processing (Springer). Dr Cavallaro was awarded the Royal Academy of Engineering teaching prize in 2007.

"The design methodologies discussed throughout the book provide guidelines for developers in the industry working on vision-based applications. The book may also serve as a reference for engineering and computer science graduate students involved in vision, robotics, human-computer interaction, smart environments and virtual reality programs." (Zentralblatt MATH, 2011) "While technical, the text is clearly written and supported by exceptional illustrations." (Booknews, 1 June 2011)

• Foreword xiAbout the authors xvPreface xviiAcknowledgements xixNotation xxiAcronyms xxiii1 What is video tracking? 11.1 Introduction 11.2 The design of a video tracker 21.2.1 Challenges 21.2.2 Main components 61.3 Problem formulation 71.3.1 Single-target tracking 71.3.2 Multi-target tracking 101.3.3 Definitions 111.4 Interactive versus automated tracking 121.5 Summary 132 Applications 152.1 Introduction 152.2 Media production and augmented reality 162.3 Medical applications and biological research 172.4 Surveillance and business intelligence 202.5 Robotics and unmanned vehicles 212.6 Tele-collaboration and interactive gaming 222.7 Art installations and performances 222.8 Summary 23References 243 Feature extraction 273.1 Introduction 273.2 From light to useful information 283.2.1 Measuring light 283.2.2 The appearance of targets 303.3 Low-level features 323.3.1 Colour 323.3.2 Photometric colour invariants 393.3.3 Gradient and derivatives 423.3.4 Laplacian 473.3.5 Motion 493.4 Mid-level features 503.4.1 Edges 503.4.2 Interest points and interest regions 513.4.3 Uniform regions 563.5 High-level features 613.5.1 Background models 623.5.2 Object models 633.6 Summary 65References 654 Target representation 714.1 Introduction 714.2 Shape representation 724.2.1 Basic models 724.2.2 Articulated models 734.2.3 Deformable models 744.3 Appearance representation 754.3.1 Template 764.3.2 Histograms 784.3.3 Coping with appearance changes 834.4 Summary 84References 855 Localisation 895.1 Introduction 895.2 Single-hypothesis methods 905.2.1 Gradient-based trackers 905.2.2 Bayes tracking and the Kalman filter 955.3 Multiple-hypothesis methods 985.3.1 Grid sampling 995.3.2 Particle filter 1015.3.3 Hybrid methods 1055.4 Summary 111References 1116 Fusion 1156.1 Introduction 1156.2 Fusion strategies 1166.2.1 Tracker-level fusion 1166.2.2 Measurement-level fusion 1186.3 Feature fusion in a Particle Filter 1196.3.1 Fusion of likelihoods 1196.3.2 Multi-feature resampling 1216.3.3 Feature reliability 1236.3.4 Temporal smoothing 1266.3.5 Example 1266.4 Summary 128References 1287 Multi-target management 1317.1 Introduction 1317.2 Measurement validation 1327.3 Data association 1347.3.1 Nearest neighbour 1347.3.2 Graph matching 1367.3.3 Multiple-hypothesis tracking 1397.4 Random Finite Sets for tracking 1437.5 Probabilistic Hypothesis Density filter 1457.6 The Particle PHD filter 1477.6.1 Dynamic and observation models 1497.6.2 Birth and clutter models 1517.6.3 Importance sampling 1517.6.4 Resampling 1527.6.5 Particle clustering 1567.6.6 Examples 1607.7 Summary 163References 1658 Context modeling 1698.1 Introduction 1698.2 Tracking with context modelling 1708.2.1 Contextual information 1708.2.2 Influence of the context 1718.3 Birth and clutter intensity estimation 1738.3.1 Birth density 1738.3.2 Clutter density 1798.3.3 Tracking with contextual feedback 1818.4 Summary 184References 1849 Performance evaluation 1859.1 Introduction 1859.2 Analytical versus empirical methods 1869.3 Ground truth 1879.4 Evaluation scores 1909.4.1 Localisation scores 1909.4.2 Classification scores 1939.5 Comparing trackers 1969.5.1 Target life-span 1979.5.2 Statistical significance 1989.5.3 Repeatibility 1989.6 Evaluation protocols 1999.6.1 Low-level protocols 1999.6.2 High-level protocols 2039.7 Datasets 2079.7.1 Surveillance 2079.7.2 Human-computer interaction 2129.7.3 Sport analysis 2159.8 Summary 220References 220Epilogue 223Further reading 225Appendix A Comparative results 229A.1 Single versus structural histogram 229A.1.1 Experimental setup 229A.1.2 Discussion 230A.2 Localisation algorithms 233A.2.1 Experimental setup 233A.2.2 Discussion 235A.3 Multi-feature fusion 238A.3.1 Experimental setup 238A.3.2 Reliability scores 240A.3.3 Adaptive versus non-adaptive tracker 242A.3.4 Computational complexity 248A.4 PHD filter 248A.4.1 Experimental setup 248A.4.2 Discussion 250A.4.3 Failure modalities 251A.4.4 Computational cost 255A.5 Context modelling 257A.5.1 Experimental setup 257A.5.2 Discussion 257References 261Index 263