Augmented Reality

Dieter Schmalstieg is professor and head of the Institute for Computer Graphics and Vision at Graz University of Technology (TUG), Austria. His current research interests are augmented reality, virtual reality, real-time graphics, 3D user interfaces, and visualization. He received Dipl.-Ing. (1993), Dr. techn. (1997) and Habilitation (2001) degrees from Vienna University of Technology. He is author and coauthor of more than two hundred peer-reviewed scientific publications, associate editor of IEEE Transactions on Visualization and Computer Graphics, member of the editorial advisory board of computers and graphics and of Springer Virtual Reality, member of the steering committee of the IEEE International Symposium on Mixed and Augmented Reality, chair of the EUROGRAPHICS working group on Virtual Environments (1999-2010), advisor of the K-Plus Competence Center for Virtual Reality and Visualization in Vienna, and member of the Austrian Academy of Science. In 2002, he received the START career award presented by the Austrian Science Fund. In 2012, he received the IEEE Virtual Reality Technical Achievement Award for seminal contributions to the field of augmented reality. Since 2008, he is also director of the Christian Doppler Laboratory for Handheld Augmented Reality.Tobias Höllerer is professor of computer science at the University of California, Santa Barbara, where he leads the Four Eyes Laboratory, conducting research in the four I's of Imaging, Interaction, and Innovative Interfaces. Dr. Höllerer holds a Diplom in informatics from the Technical University of Berlin, as well as an M.S. and Ph.D. in computer science from Columbia University. He is a recipient of the U.S. National Science Foundation's CAREER award for his work on "Anywhere Augmentation." enabling mobile computer users to place annotations in 3D space wherever they go. In 2013, he was named an ACM Distinguished Scientist. Dr. Höllerer is author of more than one hundred fifty peer-reviewed journal and conference publications in the areas of augmented and virtual reality, information visualization, 3D displays and interaction, mobile and wearable computing, and social computing. Several of these publications have been selected for best paper or honorable mention awards at such venues as the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), IEEE Virtual Reality (VR), ACM Virtual Reality Software and Technology, ACM User Interface Software and Technology, ACM MobileHCI, IEEE SocialCom, and IEEE CogSIMA. Dr. Höllerer is an associate editor of IEEE Transactions on Visualization and Computer Graphics. Among his many organizational roles for scientific conferences he served as program chair for IEEE VR 2015, ICAT 2013, IEEE ISMAR 2010 and 2009; as general chair of IEEE ISMAR 2006; and as member of the steering committee of IEEE ISMAR.

"This is an essential book for anyone interested in Augmented Reality, by two of the leading pioneers. It provides an outstanding foundation to the fast growing field of AR, both for those already in the field as well as those who just want to understand the technology more deeply." -Dr. Mark Billinghurst, Professor of Human Computer Interaction, University of South Australia, and creator of the ARToolKit "At first, I thought this book provided a very solid foundation for any Augmented Reality newbie who needed to learn all aspects of AR, but then I realized I couldn't stop digging and learning...it goes deeper than many of the AR books I reviewed in the past decade!" -Ori Inbar, Executive Director, Augmented World Expo (

• Preface xixAcknowledgments xxvAbout the Authors xxviiChapter 1: Introduction to Augmented Reality 1Definition and Scope 3A Brief History of Augmented Reality 4Examples 13Related Fields 28Summary 31Chapter 2: Displays 33Multimodal Displays 34Visual Perception 39Requirements and Characteristics 40Spatial Display Model 56Visual Displays 58Summary 84Chapter 3: Tracking 85Tracking, Calibration, and Registration 86Coordinate Systems 87Characteristics of Tracking Technology 90Stationary Tracking Systems 96Mobile Sensors 99Optical Tracking 105Sensor Fusion 117Summary 120Chapter 4: Computer Vision for Augmented Reality 121Marker Tracking 123Multiple-Camera Infrared Tracking 132Natural Feature Tracking by Detection 138Incremental Tracking 149Simultaneous Localization and Mapping 156Outdoor Tracking 164Summary 176Chapter 5: Calibration and Registration 179Camera Calibration 180Display Calibration 183Registration 190Summary 194Chapter 6: Visual Coherence 195Registration 196Occlusion 199Photometric Registration 205Common Illumination 216Diminished Reality 227Camera Simulation 231Stylized Augmented Reality 236Summary 237Chapter 7: Situated Visualization 239Challenges 241Visualization Registration 245Annotations and Labeling 248X-Ray Visualization 254Spatial Manipulation 260Information Filtering 265Summary 270Chapter 8: Interaction 271Output Modalities 272Input Modalities 279Tangible Interfaces 286Virtual User Interfaces on Real Surfaces 294Augmented Paper 295Multi-view Interfaces 297Haptic Interaction 304Multimodal Interaction 304Conversational Agents 306Summary 309Chapter 9: Modeling and Annotation 311Specifying Geometry 312Specifying Appearance 317Semi-automatic Reconstruction 319Free-Form Modeling 322Annotation 325Summary 328Chapter 10: Authoring 329Requirements of AR Authoring 330Elements of Authoring 333Stand-Alone Authoring Solutions 335Plug-In Approaches 339Web Technology 341Summary 342Chapter 11: Navigation 345Foundations of Human Navigation 346Exploration and Discovery 347Route Visualization 347Viewpoint Guidance 350Multiple Perspectives 354Summary 360Chapter 12: Collaboration 361Properties of Collaboration Systems 362Co-located Collaboration 364Remote Collaboration 370Summary 377Chapter 13: Software Architectures 379AR Application Requirements 380Software Engineering Requirements 382Distributed Object Systems 385Dataflow 389Scene Graphs 395Developer Support 400Summary 407Chapter 14: The Future 409What May Drive Business Cases 410An AR Developer’s Wish List 411Taking AR Outdoors 415Interfacing with Smart Objects 417Confluence of Virtual Reality and Augmented Reality 418Augmented Humans 419AR as a Dramatic Medium 420AR as a Social Computing Platform 421Summary 422References 423Index 473