Every Bit Counts

Prof. John L. Gustafson is a pioneer in high-performance computing, known for a breakthrough in parallel processing (Gustafson’s law) for which he received the inaugural Gordon Bell Prize. He is a former Director at Intel Labs and a former Senior Fellow and Chief Product Architect at AMD. He now leads efforts to replace legacy computer arithmetic with high-efficiency next-generation approaches.

“John Gustafson is a masterful storyteller. The Every Bit Counts book takes on a seemingly dull topic – representation of numbers in computers, and makes it fascinating. Amusing at times, clear and pointed, with examples and illustrations, Gustafson critiques the state of the art and sheds light on its flaws, while showing the reader how we got here. But the real beauty of the exposé is the proposed alternative. Full of fresh ideas, new to anyone outside the ‘numbers standard’ community, the author presents a new and superior solution that revolutionizes the stagnant state of how our computers do arithmetic. Not an academic exercise, The Posit Standard is gaining acceptance in the industry. A must-read for anyone interested in how we compute for HPC and AI.”-- David Barkai, Author of Unmatched: 50 Years of Supercomputing“This book is a landmark work and breaks ground in the critical area of the future numerics of computing. Of greatest importance is that it finally fully exposes the gross limitations of conventional IEEE 754 floating point which reflects the ad hoc nature of its creation from its beginning to its current realization. If nothing else were to be derived from this work, burying 754 would be a major accomplishment. Gustafson’s contribution will stand for time without end or until something derived from unums and posits etc. are fully embraced and assumed by all practitioners without further thought. A critical aspect of the 13 chapters is the many areas yet to be fully explored in different regimes of application as repeatedly discussed by the author. It is pleasing to note the many credits proscribed to the many contributors over the years that have been cited by John throughout the work. This book will become required reading by the thousands of students and engineers who will derive foundational benefit for future computing methods based on this book. It makes a rare first-order contribution to the expansive nature of the field.”-- Thomas Sterling, Ph.D, Senior Research Scientist at Texas Advanced Computing Center“In his new book Every Bit Counts: Posit Computing, John Gustafson offers a radical re-examination of the foundations of computer arithmetic over the finite-precision real numbers. He defines an entirely new number system called posits as an alternative to traditional floating point numbers. In an expository tour-de-force he explains the many problems with floats, and then shows how posits have much better formal properties while achieving higher precision in most calculations than floats can when using the same number of bits. Pedagogically the book is brilliant. While the ideas are very sophisticated even for professional computer scientists, anyone with a college sophomore-level background in computing can grasp the key points, aided by the many beautifully rendered, colorful illustrations on almost every page. To learn about the future of computer arithmetic over real numbers, read this book. It is destined to be a classic.”---David Jefferson, Lawrence Livermore National Laboratory (retired)

• Preface Acknowledgements How to Read this Book 1. The Posit Number Format 2. Inaccuracy Done Right 3. More Power! 4. How We Got Into This Mess 5. The Posit Standard, Annotated 6. Preaching About the Quire 7. A Bridge to Sane Floats 8. Logarithmic Systems, The Musical 9. The Fastest Arithmetic Ever? 10. The Minefield Method 11. Matrix Multiply, Visualized 12. Linear Solvers that Suck Less 13. The Future of Posit Computing Glossary Answers to Exercises Appendix A: Posit Code Listings Appendix B: Float Code Listings Appendix C: The hypot Function Appendix D: L Table for 8-Bit Posits Appendix E: Linear Solver Tests Appendix F: Linear Solvers Appendix G: Takum Format Index