Effective Python

Ive been programming in Python for years and thought I knew it pretty well. Thanks to this treasure trove of tips and techniques, I realize theres so much more I could be doing with my Python code to make it faster (e.g., using built-in data structures), easier to read (e.g., enforcing keyword-only arguments), and much more Pythonic (e.g., using zip to iterate over lists in parallel).

Pamela Fox, educationeer, Khan Academy

If I had this book when I first switched from Java to Python, it would have saved me many months of repeated code rewrites, which happened each time I realized I was doing particular things non-Pythonically. This book collects the vast majority of basic Python must-knows into one place, eliminating the need to stumble upon them one-by-one over the course of months or years. The scope of the book is impressive, starting with the importance of PEP8 as well as that of major Python idioms, then reaching through function, method and class design, effective standard library use, quality API design, testing, and performance measurementthis book really has it all. A fantastic introduction to what it really means to be a Python programmer for both the novice and the experienced developer.

Mike Bayer, creator of SQLAlchemy

Effective Python will take your Python skills to the next level with clear guidelines for improving Python code style and function.

Leah Culver, developer advocate, Dropbox

This book is an exceptionally great resource for seasoned developers in other languages who are looking to quickly pick up Python and move beyond the basic language constructs into more Pythonic code. The organization of the book is clear, concise, and easy to digest, and each item and chapter can stand on its own as a meditation on a particular topic. The book covers the breadth of language constructs in pure Python without confusing the reader with the complexities of the broader Python ecosystem. For more seasoned developers the book provides in-depth examples of language constructs they may not have previously encountered, and provides examples of less commonly used language features. It is clear that the author is exceptionally facile with Python, and he uses his professional experience to alert the reader to common subtle bugs and common failure modes. Furthermore, the book does an excellent job of pointing out subtleties between Python 2.X and Python 3.X and could serve as a refresher course as one transitions between variants of Python.

Katherine Scott, software lead, Tempo Automation

This is a great book for both novice and experienced programmers. The code examples and explanations are well thought out and explained concisely and thoroughly.

C. Titus Brown, associate professor, UC Davis

This is an immensely useful resource for advanced...

Brett Slatkin is a senior staff software engineer at Google. He is the engineering lead and co-founder of Google Consumer Surveys. He formerly worked on Google App Engine's Python infrastructure. He is the co-creator of the PubSubHubbub protocol. Nine years ago he cut his teeth using Python to manage Google's enormous fleet of servers. In his spare time, he works on open source tools and writes about software, bicycles, and other topics on his personal website (http://onebigfluke.com). He earned his B.S. in computer engineering from Columbia University in New York City. He lives in San Francisco.

Preface xiii

Chapter 1: Pythonic Thinking 1

Item 1: Know Which Version of Python Youre Using 1

Item 2: Follow the PEP 8 Style Guide 2

Item 3: Know the Differences Between bytes, str, and unicode 5

Item 4: Write Helper Functions Instead of Complex Expressions 8

Item 5: Know How to Slice Sequences 10

Item 6: Avoid Using start, end, and stride in a Single Slice 13

Item 7: Use List Comprehensions Instead of map and filter 15

Item 8: Avoid More Than Two Expressions in List Comprehensions 16

Item 9: Consider Generator Expressions for Large Comprehensions 18

Item 10: Prefer enumerate Over range 20

Item 11: Use zip to Process Iterators in Parallel 21

Item 12: Avoid else Blocks After for and while Loops 23

Item 13: Take Advantage of Each Block in try/except/else/finally 26

Chapter 2: Functions 29

Item 14: Prefer Exceptions to Returning None 29

Item 15: Know How Closures Interact with Variable Scope 31

Item 16: Consider Generators Instead of Returning Lists 36

Item 17: Be Defensive When Iterating Over Arguments 38

Item 18: Reduce Visual Noise with Variable Positional Arguments 43

Item 19: Provide Optional Behavior with Keyword Arguments 45

Item 20: Use None and Docstrings to Specify Dynamic Default Arguments 48

Item 21: Enforce Clarity with Keyword-Only Arguments 51

Chapter 3: Classes and Inheritance 55

Item 22: Prefer Helper Classes Over Bookkeeping with Dictionaries and Tuples 55

Item 23: Accept Functions for Simple Interfaces Instead of Classes 61

Item 24: Use @classmethod Polymorphism to Construct Objects Generically 64

Item 25: Initialize Parent Classes with super 69

Item 26: Use Multiple Inheritance Only for Mix-in Utility Classes 73

Item 27: Prefer Public Attributes Over Private Ones 78

Item 28: Inherit from collections.abc for Custom Container Types 83

Chapter 4: Metaclasses and Attributes 87

Item 29: Use Plain Attributes Instead of Get and Set Methods 87

Item 30: Consider @property Instead of Refactoring Attributes 91

Item 31: Use Descriptors for Reusable @property Methods 95

Item 32: Use __getattr__, __getattribute__, and __setattr__ for Lazy Attributes 100

Item 33: Validate Subclasses with Metaclasses 105

Item 34: Register Class Existence with Metaclasses 108

Item 35: Annotate Class Attributes with Metaclasses 112

Chapter 5: Concurrency and Parallelism 117

Item 36: Use subprocess to Manage Child Processes 118

Item 37: Use Threads for Blocking I/O, Avoid for Parallelism 122

Item 38: Use Lock to Prevent Data Races in Threads 126

Item 39: Use Queue to Coordinate Work Between T...