Fluent Python: Clear, Concise and Effective Programming

766 Pages·2015·7.44 MB·English

by  Luciano Ramalho

Fluent
Python
CLEAR, CONCISE, AND EFFECTIVE PROGRAMMING
Luciano Ramalho
Fluent Python
“
Python’s simplicity lets you become productive quickly, but this often means I am proud to have been
you aren’t using everything it has to ofer. With this hands-on guide, you’ll learn a tech reviewer for this
how to write effective, idiomatic Python code by leveraging its best—and
excellent book—not only
possibly most neglected—features. Author Luciano Ramalho takes you
will it help many
through Python’s core language features and libraries, and shows you how to
make your code shorter, faster, and more readable at the same time. intermediate Python
Many experienced programmers try to bend Python to fit patterns they programmers on their
learned from other languages, and never discover Python features outside road towards mastery,
of their experience. With this book, those Python programmers will
but it has taught me quite
thoroughly learn how to become proficient in Python 3.
”
a few things, too!
This book covers: —Alex Martelli
Python Software Foundation Fellow
■ The Python data model: understand how special methods are
the key to the consistent behavior of objects “
Fluent Python is a
■ Data structures: take full advantage of built-in types, and treasure trove full of
understand the text versus bytes duality in the Unicode age
useful programming
■ Functions as objects: view Python functions as irst-class objects,
tricks for intermediate to
and understand how this afects popular design patterns
advanced Python coders
■ Object-oriented idioms: build classes by learning about
references, mutability, interfaces, operator overloading, and who want to push the
multiple inheritance boundaries of their
”
■ Control low: leverage context managers, generators, knowledge.
coroutines, and concurrency with the concurrent.futures and
—Daniel and Audrey Roy Greenfeld
asyncio packages
authors of Two Scoops of Django
■ Metaprogramming: understand how properties, attribute
descriptors, class decorators, and metaclasses work
Luciano Ramalho, a Python programmer since 1998, is a Python Software
Foundation fellow, co-owner of Python.pro.br—a training company in Brazil—
and cofounder of Garoa Hacker Clube, Brazil’s first hackerspace. He has led
software development teams and taught Python courses in Brazilian media,
banking, and government sectors.
PROGRAMMING/PYTHON Twitter: @oreillymedia
facebook.com/oreilly
US $49.99 CAN $57.99
ISBN: 978-1-491-9-46008
Fluent Python
Luciano Ramalho
Boston
Fluent Python
by Luciano Ramalho
Copyright © 2015 Luciano Gama de Sousa Ramalho. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or [email protected].
Editors: Meghan Blanchette and Rachel Roumeliotis Indexer: Judy McConville
Production Editor: Melanie Yarbrough Cover Designer: Ellie Volckhausen
Copyeditor: Kim Cofer Interior Designer: David Futato
Proofreader: Jasmine Kwityn Illustrator: Rebecca Demarest
August 2015: First Edition
Revision History for the First Edition:
2015-07-24: First release
2015-08-21: Second release
See http://oreilly.com/catalog/errata.csp?isbn=9781491946008 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Fluent Python, the cover image, and
related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and author have used good faith efforts to ensure that the information and instructions
contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or
omissions, including without limitation responsibility for damages resulting from the use of or reliance on
this work. Use of the information and instructions contained in this work is at your own risk. If any code
samples or other technology this work contains or describes is subject to open source licenses or the intel‐
lectual property rights of others, it is your responsibility to ensure that your use thereof complies with such
licenses and/or rights.
ISBN: 978-1-491-94600-8
[LSI]
Para Marta, com todo o meu amor.

Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Part I. Prologue
1. The Python Data Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
A Pythonic Card Deck 4
How Special Methods Are Used 8
Emulating Numeric Types 9
String Representation 11
Arithmetic Operators 12
Boolean Value of a Custom Type 12
Overview of Special Methods 13
Why len Is Not a Method 14
Chapter Summary 14
Further Reading 15
Part II. Data Structures
2. An Array of Sequences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Overview of Built-In Sequences 20
List Comprehensions and Generator Expressions 21
List Comprehensions and Readability 21
Listcomps Versus map and filter 23
Cartesian Products 23
Generator Expressions 25
Tuples Are Not Just Immutable Lists 26
Tuples as Records 26
Tuple Unpacking 27
v
Nested Tuple Unpacking 29
Named Tuples 30
Tuples as Immutable Lists 32
Slicing 33
Why Slices and Range Exclude the Last Item 33
Slice Objects 34
Multidimensional Slicing and Ellipsis 35
Assigning to Slices 36
Using + and * with Sequences 36
Building Lists of Lists 37
Augmented Assignment with Sequences 38
A += Assignment Puzzler 40
list.sort and the sorted Built-In Function 42
Managing Ordered Sequences with bisect 44
Searching with bisect 44
Inserting with bisect.insort 47
When a List Is Not the Answer 48
Arrays 48
Memory Views 51
NumPy and SciPy 52
Deques and Other Queues 55
Chapter Summary 57
Further Reading 59
3. Dictionaries and Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Generic Mapping Types 64
dict Comprehensions 66
Overview of Common Mapping Methods 66
Handling Missing Keys with setdefault 68
Mappings with Flexible Key Lookup 70
defaultdict: Another Take on Missing Keys 70
The __missing__ Method 72
Variations of dict 75
Subclassing UserDict 76
Immutable Mappings 77
Set Theory 79
set Literals 80
Set Comprehensions 81
Set Operations 82
dict and set Under the Hood 85
A Performance Experiment 85
Hash Tables in Dictionaries 87
vi | Table of Contents
Practical Consequences of How dict Works 90
How Sets Work—Practical Consequences 93
Chapter Summary 93
Further Reading 94
4. Text versus Bytes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Character Issues 98
Byte Essentials 99
Structs and Memory Views 102
Basic Encoders/Decoders 103
Understanding Encode/Decode Problems 105
Coping with UnicodeEncodeError 105
Coping with UnicodeDecodeError 106
SyntaxError When Loading Modules with Unexpected Encoding 108
How to Discover the Encoding of a Byte Sequence 109
BOM: A Useful Gremlin 110
Handling Text Files 111
Encoding Defaults: A Madhouse 114
Normalizing Unicode for Saner Comparisons 117
Case Folding 119
Utility Functions for Normalized Text Matching 120
Extreme “Normalization”: Taking Out Diacritics 121
Sorting Unicode Text 124
Sorting with the Unicode Collation Algorithm 126
The Unicode Database 127
Dual-Mode str and bytes APIs 129
str Versus bytes in Regular Expressions 129
str Versus bytes on os Functions 130
Chapter Summary 132
Further Reading 133
Part III. Functions as Objects
5. First-Class Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Treating a Function Like an Object 140
Higher-Order Functions 141
Modern Replacements for map, filter, and reduce 142
Anonymous Functions 143
The Seven Flavors of Callable Objects 144
User-Defined Callable Types 145
Function Introspection 146
Table of Contents | vii
From Positional to Keyword-Only Parameters 148
Retrieving Information About Parameters 150
Function Annotations 154
Packages for Functional Programming 156
The operator Module 156
Freezing Arguments with functools.partial 159
Chapter Summary 161
Further Reading 162
6. Design Patterns with First-Class Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Case Study: Refactoring Strategy 168
Classic Strategy 168
Function-Oriented Strategy 172
Choosing the Best Strategy: Simple Approach 175
Finding Strategies in a Module 176
Command 177
Chapter Summary 179
Further Reading 180
7. Function Decorators and Closures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Decorators 101 184
When Python Executes Decorators 185
Decorator-Enhanced Strategy Pattern 187
Variable Scope Rules 189
Closures 192
The nonlocal Declaration 195
Implementing a Simple Decorator 196
How It Works 198
Decorators in the Standard Library 199
Memoization with functools.lru_cache 200
Generic Functions with Single Dispatch 202
Stacked Decorators 205
Parameterized Decorators 206
A Parameterized Registration Decorator 206
The Parameterized Clock Decorator 209
Chapter Summary 211
Further Reading 212
viii | Table of Contents