Shellcoder's Handbook

Chris Anley is a founder and director of NGSSoftware, a security software, consultancy, and research company based in London, England. He is actively involved in vulnerability research and has discovered security flaws in a wide variety of platforms including Microsoft Windows, Oracle, SQL Server, IBM DB2, Sybase ASE, MySQL, and PGP. John Heasman is the Director of Research at NGSSoftware. He is a prolific security researcher and has published many security advisories in enterprise level software. He has a particular interest in rootkits and has authored papers on malware persistence via device firmware and the BIOS. He is also a co-author of The Database Hacker’s Handbook: Defending Database Servers (Wiley 2005).Felix “FX” Linder leads SABRE Labs GmbH, a Berlin-based professional consulting company specializing in security analysis, system design creation, and verification work. Felix looks back at 18 years of programming and over a decade of computer security consulting for enterprise, carrier, and software vendor clients. This experience allows him to rapidly dive into complex systems and evaluate them from a security and robustness point of view, even in atypical scenarios and on arcane platforms. In his spare time, FX works with his friends from the Phenoelit hacking group on different topics, which have included Cisco IOS, SAP, HP printers, and RIM BlackBerry in the past.Gerardo Richarte has been doing reverse engineering and exploit development for more than 15 years non-stop. In the past 10 years he helped build the technical arm of Core Security Technologies, where he works today. His current duties include developing exploits for Core IMPACT, researching new exploitation techniques and other low-level subjects, helping other exploit writers when things get hairy, and teaching internal and external classes on assembly and exploit writing. As result of his research and as a humble thank you to the community, he has published some technical papers and open source projects, presented in a few conferences, and released part of his training material. He really enjoys solving tough problems and reverse engineering any piece of code that falls in his reach just for the fun of doing it.

• About the Authors viiAcknowledgments xiIntroduction to the Second Edition xxiiiPart I Introduction to Exploitation: Linux on X 86Chapter 1 Before You Begin 3Basic Concepts 3Memory Management 4Assembly 6Recognizing C and C++ Code Constructs in Assembly 7Conclusion 10Chapter 2 Stack Overflows 11Buffers 12The Stack 13Functions and the Stack 15Overflowing Buffers on the Stack 18Controlling EIP 22An Interesting Diversion 23Using an Exploit to Get Root Privileges 25The Address Problem 27The NOP Method 33Defeating a Non-Executable Stack 35Return to libc 35Conclusion 39Chapter 3 Shellcode 41Understanding System Calls 42Writing Shellcode for the exit() Syscall 44Injectable Shellcode 48Spawning a Shell 50Conclusion 59Chapter 4 Introduction to Format String Bugs 61Prerequisites 61What Is a Format String? 61What Is a Format String Bug? 63Format String Exploits 68Crashing Services 69Information Leakage 70Controlling Execution for Exploitation 75Why Did This Happen? 84Format String Technique Roundup 85Conclusion 88Chapter 5 Introduction to Heap Overflows 89What Is a Heap? 90How a Heap Works 91Finding Heap Overflows 91Basic Heap Overflows 93Intermediate Heap Overflows 98Advanced Heap Overflow Exploitation 105Conclusion 107Part II other Platforms—windows, Solaris, OS/X, and CiscoChapter 6 The Wild World of Windows 111How Does Windows Differ from Linux? 111Win32 API and PE-COFF 112Heaps 114Threading 115The Genius and Idiocy of the Distributed Common Object Model and DCE-RPC 116Recon 118Exploitation 120Tokens and Impersonation 120Exception Handling under Win 32 122Debugging Windows 124Bugs in Win 32 124Writing Windows Shellcode 125A Hacker’s Guide to the Win32 API 126A Windows Family Tree from the Hacker’s Perspective 126Conclusion 127Chapter 7 Windows Shellcode 129Syntax and Filters 129Setting Up 131Parsing the PEB 132Heapoverflow.c Analysis 132Searching with Windows Exception Handling 148Popping a Shell 153Why You Should Never Pop a Shell on Windows 153Conclusion 154Chapter 8 Windows Overflows 155Stack-Based Buffer Overflows 156Frame-Based Exception Handlers 156Abusing Frame-Based Exception Handling on Windows 2003 Server 161A Final Note about Frame-Based Handler Overwrites 166Stack Protection and Windows 2003 Server 166Heap-Based Buffer Overflows 173The Process Heap 173Dynamic Heaps 173Working with the Heap 173How the Heap Works 174Exploiting Heap-Based Overflows 178Overwrite Pointer to RtlEnterCriticalSection in the PEB 178Overwrite Pointer to Unhandled Exception Filter 185Repairing the Heap 191Other Aspects of Heap-Based Overflows 193Wrapping Up the Heap 194Other Overflows 194.data Section Overflows 194TEB/PEB Overflows 196Exploiting Buffer Overflows and Non-Executable Stacks 197Conclusion 203Chapter 9 Overcoming Filters 205Writing Exploits for Use with an Alphanumeric Filter 205Writing Exploits for Use with a Unicode Filter 209What Is Unicode? 210Converting from ASCII to Unicode 210Exploiting Unicode-Based Vulnerabilities 211The Available Instruction Set in Unicode Exploits 212The Venetian Method 213An ASCII Venetian Implementation 214Decoder and Decoding 218The Decoder Code 219Getting a Fix on the Buffer Address 220Conclusion 221Chapter 10 Introduction to Solaris Exploitation 223Introduction to the SPARC Architecture 224Registers and Register Windows 224The Delay Slot 227Synthetic Instructions 228Solaris/SPARC Shellcode Basics 228Self-Location Determination and SPARC Shellcode 228Simple SPARC exec Shellcode 229Useful System Calls on Solaris 230NOP and Padding Instructions 231Solaris/SPARC Stack Frame Introduction 231Stack-Based Overflow Methodologies 232Arbitrary Size Overflow 232Register Windows and Stack Overflow Complications 233Other Complicating Factors 233Possible Solutions 234Off-By-One Stack Overflow Vulnerabilities 234Shellcode Locations 235Stack Overflow Exploitation In Action 236The Vulnerable Program 236The Exploit 238Heap-Based Overflows on Solaris/SPARC 241Solaris System V Heap Introduction 242Heap Tree Structure 242Basic Exploit Methodology (t_delete) 263Standard Heap Overflow Limitations 266Targets for Overwrite 267Other Heap-Related Vulnerabilities 270Off-by-One Overflows 270Double Free Vulnerabilities 270Arbitrary Free Vulnerabilities 271Heap Overflow Example 271The Vulnerable Program 272Other Solaris Exploitation Techniques 276Static Data Overflows 276Bypassing the Non-Executable Stack Protection 276Conclusion 277Chapter 11 Advanced Solaris Exploitation 279Single Stepping the Dynamic Linker 281Various Style Tricks for Solaris SPARC Heap Overflows 296Advanced Solaris/SPARC Shellcode 299Conclusion 311Chapter 12 OS X Shellcode 313OS X Is Just BSD, Right? 314Is OS X Open Source? 314OS X for the Unix-aware 315Password Cracking 316OS X PowerPC Shellcode 316OS X Intel Shellcode 324Example Shellcode 326ret2libc 327ret2str(l)cpy 329OS X Cross-Platform Shellcode 332OS X Heap Exploitation 333Bug Hunting on OS X 335Some Interesting Bugs 335Essential Reading for OS X Exploits 337Conclusion 338Chapter 13 Cisco IOS Exploitation 339An Overview of Cisco IOS 339Hardware Platforms 340Software Packages 340IOS System Architecture 343Vulnerabilities in Cisco IOS 346Protocol Parsing Code 347Services on the Router 347Security Features 348The Command-Line Interface 348Reverse Engineering IOS 349Taking the Images Apart 349Diffing IOS Images 350Runtime Analysis 351Exploiting Cisco IOS 357Stack Overflows 357Heap Overflows 359Shellcodes 364Conclusion 373Chapter 14 Protection Mechanisms 375Protections 375Non-Executable Stack 376W^X (Either Writable or Executable) Memory 381Stack Data Protection 388AAAS: ASCII Armored Address Space 394ASLR: Address Space Layout Randomization 396Heap Protections 399Windows SEH Protections 407Other Protections 411Implementation Differences 413Windows 413Linux 417OpenBSD 421Mac OS X 422Solaris 423Conclusion 425Part III Vulnerability DiscoveryChapter 15 Establishing a Working Environment 429What You Need for Reference 430What You Need for Code 430gcc 430gdb 430NASM 431WinDbg 431OllyDbg 431Visual C++ 431Python 432What You Need for Investigation 432Useful Custom Scripts/Tools 432All Platforms 434Unix 434Windows 435What You Need to Know 436Paper Archives 438Optimizing Shellcode Development 439Plan the Exploit 439Write the Shellcode in Inline Assembler 439Maintain a Shellcode Library 441Make It Continue Nicely 441Make the Exploit Stable 442Make It Steal the Connection 443Conclusion 443Chapter 16 Fault Injection 445Design Overview 447Input Generation 447Fault Injection 450Modification Engines 450Fault Delivery 455Nagel Algorithm 455Timing 455Heuristics 456Stateless versus State-Based Protocols 456Fault Monitoring 456Using a Debugger 457FaultMon 457Putting It Together 458Conclusion 459Chapter 17 The Art of Fuzzing 461General Theory of Fuzzing 461Static Analysis versus Fuzzing 466Fuzzing Is Scalable 466Weaknesses in Fuzzers 468Modeling Arbitrary Network Protocols 469Other Fuzzer Possibilities 469Bit Flipping 469Modifying Open Source Programs 470Fuzzing with Dynamic Analysis 470Spike 471What Is a Spike? 471Why Use the SPIKE Data Structure to Model Network Protocols? 472Other Fuzzers 480Conclusion 480Chapter 18 Source Code Auditing: Finding Vulnerabilities in C-Based Languages 481Tools 482Cscope 482Ctags 483Editors 483Cbrowser 484Automated Source Code Analysis Tools 484Methodology 485Top-Down (Specific) Approach 485Bottom-Up Approach 485Selective Approach 485Vulnerability Classes 486Generic Logic Errors 486(Almost) Extinct Bug Classes 487Format Strings 487Generic Incorrect Bounds-Checking 489Loop Constructs 490Off-by-One Vulnerabilities 490Non-Null Termination Issues 492Skipping Null-Termination Issues 493Signed Comparison Vulnerabilities 494Integer-Related Vulnerabilities 495Different-Sized Integer Conversions 497Double Free Vulnerabilities 498Out-of-Scope Memory Usage Vulnerabilities 499Uninitialized Variable Usage 499Use After Free Vulnerabilities 500Multithreaded Issues and Re-Entrant Safe Code 500Beyond Recognition: A Real Vulnerability versus a Bug 501Conclusion 501Chapter 19 Instrumented Investigation: A Manual Approach 503Philosophy 503Oracle extproc Overflow 504Common Architectural Failures 508Problems Happen at Boundaries 508Problems Happen When Data Is Translated 509Problems Cluster in Areas of Asymmetry 511Problems Occur When Authentication and Authorization Are Confused 512Problems Occur in the Dumbest Places 512Bypassing Input Validation and Attack Detection 513Stripping Bad Data 513Using Alternate Encodings 514Using File-Handling Features 515Evading Attack Signatures 517Defeating Length Limitations 517Windows 2000 SNMP DOS 520Finding DOS Attacks 521SQL-UDP 522Conclusion 523Chapter 20 Tracing for Vulnerabilities 525Overview 526A Vulnerable Program 527Component Design 529Building VulnTrace 538Using VulnTrace 543Advanced Techniques 546Conclusion 548Chapter 21 Binary Auditing: Hacking Closed Source Software 549Binary versus Source-Code Auditing: The Obvious Differences 550IDA Pro—The Tool of the Trade 550Features: A Quick Crash Course 551Debugging Symbols 552Binary Auditing Introduction 552Stack Frames 552Calling Conventions 554Compiler-Generated Code 556memcpy-Like Code Constructs 560strlen-Like Code Constructs 560C++ Code Constructs 561The this Pointer 561Reconstructing Class Definitions 562vtables 562Quick but Useful Tidbits 563Manual Binary Analysis 563Quick Examination of Library Calls 564Suspicious Loops and Write Instructions 564Higher-Level Understanding and Logic Bugs 565Graphical Analysis of Binaries 566Manual Decompilation 566Binary Vulnerability Examples 566Microsoft SQL Server Bugs 566LSD’s RPC-DCOM Vulnerability 567IIS WebDAV Vulnerability 568Conclusion 570Part IV Advanced MaterialsChapter 22 Alternative Payload Strategies 573Modifying the Program 574The SQL Server 3-Byte Patch 575The MySQL 1-Bit Patch 578OpenSSH RSA Authentication Patch 580Other Runtime Patching Ideas 581GPG 1.2.2 Randomness Patch 583Upload and Run (or Proglet Server) 584Syscall Proxies 584Problems with Syscall Proxies 587Conclusion 596Chapter 23 Writing Exploits that Work in the Wild 597Factors in Unreliability 597Magic Numbers 597Versioning 598Shellcode Problems 599Countermeasures 601Preparation 602Brute Forcing 602Local Exploits 603OS/Application Fingerprinting 603Information Leaks 605Conclusion 606Chapter 24 Attacking Database Software 607Network Layer Attacks 608Application Layer Attacks 618Running Operating System Commands 619Microsoft SQL Server 619Oracle 620IBM DB 2 621Exploiting Overruns at the SQL Level 623SQL Functions 623Conclusion 625Chapter 25 Unix Kernel Overflows 627Kernel Vulnerability Types 6270day Kernel Vulnerabilities 636OpenBSD exec_ibcs2_coff_prep_zmagic() Stack Overflow 636The Vulnerability 638Solaris vfs_getvfssw() Loadable Kernel Module Traversal Vulnerability 642The sysfs() System Call 644The mount() System Call 645Conclusion 646Chapter 26 Exploiting Unix Kernel Vulnerabilities 647The exec_ibcs2_coff_prep_zmagic() Vulnerability 647Calculating Offsets and Breakpoints 652Overwriting the Return Address and Redirecting Execution 654Locating the Process Descriptor (or the Proc Structure) 655Kernel Mode Payload Creation 658Returning Back from Kernel Payload 659Getting root (uid=0) 665Solaris vfs_getvfssw() Loadable Kernel Module Path Traversal Exploit 672Crafting the Exploit 673The Kernel Module to Load 674Getting root (uid=0) 678Conclusion 678Chapter 27 Hacking the Windows Kernel 681Windows Kernel Mode Flaws—An Increasingly Hunted Species 681Introduction to the Windows Kernel 682Common Kernel-Mode Programming Flaws 683Stack Overflows 684Heap Overflows 688Insufficient Validation of User-Mode Addresses 688Repurposing Attacks 689Shared Object Attacks 689Windows System Calls 690Understanding System Calls 690Attacking System Calls 692Communicating with Device Drivers 693I/O Control Code Components 693Finding Flaws in IOCTL Handlers 694Kernel-Mode Payloads 695Elevating a User-Mode Process 696Running an Arbitrary User-Mode Payload 699Subverting Kernel Security 701Installing a Rootkit 703Essential Reading for Kernel Shellcoders 703Conclusion 704Index 705