Design and Implementation of the FreeBSD Operating System, The

Marshall Kirk McKusick writes books and articles, consults, and teaches classes on UNIX- and BSD-related subjects. While at the University of California at Berkeley, he implemented the 4.2BSD fast filesystem and was the Research Computer Scientist at the Berkeley Computer Systems Research Group (CSRG), overseeing the development and release of 4.3BSD and 4.4BSD. His particular areas of interest are the virtual-memory system and the filesystem. He earned his undergraduate degree in electrical engineering from Cornell University and did his graduate work at the University of California at Berkeley, where he received master’s degrees in computer science and business administration, and a doctoral degree in computer science. He has twice been president of the board of the Usenix Association, is currently a member of the FreeBSD Foundation Board of Directors, a member of the editorial board of ACM’s Queue magazine, a senior member of the IEEE, and a member of the Usenix Association, ACM, and AAAS. In his spare time, he enjoys swimming, scuba diving, and wine collecting. The wine is stored in a specially constructed wine cellar (accessible from the Web at http://www.McKusick.com/cgi-bin/readhouse) in the basement of the house that he shares with Eric Allman, his partner of 35-and-some-odd years and husband since 2013.George V. Neville-Neil hacks, writes, teaches, and consults in the areas of Security, Networking, and Operating Systems. Other areas of interest include embedded and real-time systems, network time protocols, and code spelunking. In 2007, he helped start the AsiaBSDCon series of conferences in Tokyo, Japan, and has served on the program committee every year since then. He is a member of the FreeBSD Foundation Board of Directors, and was a member of the FreeBSD Core Team for 4 years. Contributing broadly to open source, he is the lead developer on the Precision Time Protocol project (http://ptpd.sf.net) and the developer of the Packet Construction Set (http://pcs.sf.net). Since 2004, he has written a monthly column, ‘‘Kode Vicious,’’ that appears both in ACM’s Queue and Communications of the ACM. He serves on the editorial board of ACM’s Queue magazine, is vice-chair of ACM’s Practitioner Board, and is a member of the Usenix Association, ACM, IEEE, and AAAS. He earned his bachelor’s degree in computer science at Northeastern University in Boston, Massachusetts. He is an avid bicyclist, hiker, and traveler who has lived in Amsterdam, The Netherlands, and Tokyo, Japan. He is currently based in Brooklyn, New York, where he lives with his husband, Kaz Senju.Robert N.M. Watson is a University Lecturer in Systems, Security, and Architecture in the Security Research Group at the University of Cambridge Computer Laboratory. He supervises doctoral students and postdoctoral researchers in cross-layer research projects spanning computer architecture, compilers, program analysis, program transformation, operating systems, networking, and security. Dr. Watson is a member of the FreeBSD Foundation Board of Directors, was a member of the FreeBSD Core Team for 10 years, and has been a FreeBSD committer for 15 years. His open-source contributions include work on FreeBSD networking, security, and multiprocessing. Having grown up in Washington, D. C., he earned his undergraduate degree in Logic and Computation, with a double major in Computer Science, at Carnegie Mellon University in Pittsburgh, Pennsylvania, and then worked at a series of industrial research labs investigating computer security. He earned his doctoral degree at the University of Cambridge, where his graduate research was in extensible operating system access control. Dr. Watson and his wife Dr. Leigh Denault have lived in Cambridge, England, for 10 years.

• Preface xxiAbout the Authors xxixPart I: Over view 1Chapter 1: History and Goals 31.1 History of the UNIX System 31.2 BSD and Other Systems 71.3 The Transition of BSD to Open Source 91.4 The FreeBSD Development Model 14References 17Chapter 2: Design Overview of FreeBSD 212.1 FreeBSD Facilities and the Kernel 212.2 Kernel Organization 232.3 Kernel Services 262.4 Process Management 262.5 Security 292.6 Memory Management 362.7 I/O System Overview 392.8 Devices 442.9 The Fast Filesystem 452.10 The Zettabyte Filesystem 492.11 The Network Filesystem 502.12 Interprocess Communication 502.13 Network-Layer Protocols 512.14 Transport-Layer Protocols 522.15 System Startup and Shutdown 52Exercises 54References 54Chapter 3: Kernel Services 573.1 Kernel Organization 573.2 System Calls 623.3 Traps and Interrupts 643.4 Clock Interrupts 653.5 Memory-Management Services 693.6 Timing Services 733.7 Resource Services 753.8 Kernel Tracing Facilities 77Exercises 84References 85Part II: Processes 87Chapter 4: Process Management 894.1 Introduction to Process Management 894.2 Process State 924.3 Context Switching 994.4 Thread Scheduling 1144.5 Process Creation 1264.6 Process Termination 1284.7 Signals 1294.8 Process Groups and Sessions 1364.9 Process Debugging 142Exercises 144References 146Chapter 5: Security 1475.1 Operating-System Security 1485.2 Security Model 1495.3 Process Credentials 1515.4 Users and Groups 1545.5 Privilege Model 1575.6 Interprocess Access Control 1595.7 Discretionary Access Control 1615.8 Capsicum Capability Model 1745.9 Jails 1805.10 Mandatory Access-Control Framework 1845.11 Security Event Auditing 2005.12 Cryptographic Services 2065.13 GELI Full-Disk Encryption 212Exercises 217References 217Chapter 6: Memory Management 2216.1 Terminology 2216.2 Overview of the FreeBSD Virtual-Memory System 2276.3 Kernel Memory Management 2306.4 Per-Process Resources 2446.5 Shared Memory 2506.6 Creation of a New Process 2586.7 Execution of a File 2626.8 Process Manipulation of Its Address Space 2636.9 Termination of a Process 2666.10 The Pager Interface 2676.11 Paging 2766.12 Page Replacement 2896.13 Portability 298Exercises 308References 310Part III: I/OSystem 313Chapter 7: I/O System Overview 3157.1 Descriptor Management and Services 3167.2 Local Interprocess Communication 3337.3 The Virtual-Filesystem Interface 3397.4 Filesystem-Independent Services 3447.5 Stackable Filesystems 352Exercises 358References 359Chapter 8: Devices 3618.1 Device Overview 3618.2 I/O Mapping from User to Device 3678.3 Character Devices 3708.4 Disk Devices 3748.5 Network Devices 3788.6 Terminal Handling 3828.7 The GEOM Layer 3918.8 The CAM Layer 3998.9 Device Configuration 4028.10 Device Virtualization 414Exercises 428References 429Chapter 9: The Fast Filesystem 4319.1 Hierarchical Filesystem Management 4319.2 Structure of an Inode 4339.3 Naming 4439.4 Quotas 4519.5 File Locking 4549.6 Soft Updates 4599.7 Filesystem Snapshots 4809.8 Journaled Soft Updates 4879.9 The Local Filestore 4969.10 The Berkeley Fast Filesystem 501Exercises 517References 519Chapter 10: The Zettabyte Filesystem 52310.1 Introduction 52310.2 ZFS Organization 52710.3 ZFS Structure 53210.4 ZFS Operation 53510.5 ZFS Design Tradeoffs 547Exercises 549References 549Chapter 11: The Network Filesystem 55111.1 Overview 55111.2 Structure and Operation 55311.3 NFS Evolution 567Exercises 586References 587Part IV: Interprocess Communication 591Chapter 12: Interprocess Communication 59312.1 Interprocess-Communication Model 59312.2 Implementation Structure and Overview 59912.3 Memory Management 60112.4 IPC Data Structures 60612.5 Connection Setup 61212.6 Data Transfer 61512.7 Socket Shutdown 62012.8 Network-Communication Protocol Internal Structure 62112.9 Socket-to-Protocol Interface 62612.10 Protocol-to-Protocol Interface 63112.11 Protocol-to-Network Interface 63412.12 Buffering and Flow Control 64312.13 Network Virtualization 644Exercises 646References 648Chapter 13: Network-Layer Protocols 64913.1 Internet Protocol Version 4 65013.2 Internet Control Message Protocols (ICMP) 65713.3 Internet Protocol Version 6 65913.4 Internet Protocols Code Structure 67013.5 Routing 67513.6 Raw Sockets 68613.7 Security 68813.8 Packet-Processing Frameworks 700Exercises 715References 717Chapter 14: Transport-Layer Protocols 72114.1 Internet Ports and Associations 72114.2 User Datagram Protocol (UDP) 72314.3 Transmission Control Protocol (TCP) 72514.4 TCP Algorithms 73214.5 TCP Input Processing 74114.6 TCP Output Processing 74514.7 Stream Control Transmission Protocol (SCTP) 761Exercises 768References 770Part V: System Operation 773Chapter 15: System Startup and Shutdown 77515.1 Firmware and BIOSes 77615.2 Boot Loaders 77715.3 Kernel Boot 78215.4 User-Level Initialization 79815.5 System Operation 800Exercises 805References 806Glossary 807Index 847