Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services

165 Pages·2018·6.6 MB·English

by  Brendan Burns

Azure
Try these
hands-on labs
for designing
distributed
systems
In this GitHub repository, you’ll find hands-on labs to build
practical experience with the reusable patterns and components
covered in the e-book. Follow step-by-step examples to create
containerized and distributed apps in Kubernetes and Kubeless,
using Azure Container Services (AKS) and other services to put
them into production.
Learn about:
The single-node Patterns for Event-driven
ambassador replicating and distributed
pattern—your first containerized patterns by
step in creating applications and creating a
and extending augmenting containerized
a containerized functions in batch-processing
application. a serverless workflow.
environment.
Try the labs at
https://aka.ms/GetHandsOn
Designing Distributed Systems
Patterns and Paradigms for
Scalable, Reliable Services
Brendan Burns
BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo
Designing Distributed Systems
by Brendan Burns
Copyright © 2018 Brendan Burns. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/insti‐
tutional sales department: 800-998-9938 or [email protected].
Editor: Angela Rufino Indexer: WordCo Indexing Services, Inc.
Production Editor: Colleen Cole Interior Designer: David Futato
Copyeditor: Gillian McGarvey Cover Designer: Randy Comer
Proofreader: Christina Edwards Illustrator: Rebecca Demarest
January 2018: First Edition
Revision History for the First Edition
2018-01-19: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781491983645 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Designing Distributed Systems, the
cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the author disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-492-03177-2
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
A Brief History of Systems Development 1
A Brief History of Patterns in Software Development 2
Formalization of Algorithmic Programming 3
Patterns for Object-Oriented Programming 3
The Rise of Open Source Software 3
The Value of Patterns, Practices, and Components 4
Standing on the Shoulders of Giants 4
A Shared Language for Discussing Our Practice 5
Shared Components for Easy Reuse 5
Summary 6
Part I. Single-Node Patterns
Motivations 7
Summary 8
2. The Sidecar Pattern. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
An Example Sidecar: Adding HTTPS to a Legacy Service 11
Dynamic Configuration with Sidecars 12
Modular Application Containers 14
Hands On: Deploying the topz Container 14
Building a Simple PaaS with Sidecars 15
Designing Sidecars for Modularity and Reusability 16
Parameterized Containers 17
Define Each Container’s API 17
v
Documenting Your Containers 18
Summary 19
3. Ambassadors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Using an Ambassador to Shard a Service 22
Hands On: Implementing a Sharded Redis 23
Using an Ambassador for Service Brokering 25
Using an Ambassador to Do Experimentation or Request Splitting 26
Hands On: Implementing 10% Experiments 27
4. Adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Monitoring 32
Hands On: Using Prometheus for Monitoring 33
Logging 34
Hands On: Normalizing Different Logging Formats with Fluentd 35
Adding a Health Monitor 36
Hands On: Adding Rich Health Monitoring for MySQL 37
Part II. Serving Patterns
Introduction to Microservices 41
5. Replicated Load-Balanced Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Stateless Services 45
Readiness Probes for Load Balancing 46
Hands On: Creating a Replicated Service in Kubernetes 47
Session Tracked Services 48
Application-Layer Replicated Services 49
Introducing a Caching Layer 49
Deploying Your Cache 50
Hands On: Deploying the Caching Layer 51
Expanding the Caching Layer 53
Rate Limiting and Denial-of-Service Defense 54
SSL Termination 54
Hands On: Deploying nginx and SSL Termination 55
Summary 57
6. Sharded Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Sharded Caching 59
Why You Might Need a Sharded Cache 60
The Role of the Cache in System Performance 61
Replicated, Sharded Caches 62
vi | Table of Contents
Hands On: Deploying an Ambassador and Memcache for a Sharded Cache 63
An Examination of Sharding Functions 66
Selecting a Key 67
Consistent Hashing Functions 68
Hands On: Building a Consistent HTTP Sharding Proxy 69
Sharded, Replicated Serving 70
Hot Sharding Systems 70
7. Scatter/Gather. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Scatter/Gather with Root Distribution 74
Hands On: Distributed Document Search 75
Scatter/Gather with Leaf Sharding 76
Hands On: Sharded Document Search 77
Choosing the Right Number of Leaves 78
Scaling Scatter/Gather for Reliability and Scale 79
8. Functions and Event-Driven Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Determining When FaaS Makes Sense 82
The Benefits of FaaS 82
The Challenges of FaaS 82
The Need for Background Processing 83
The Need to Hold Data in Memory 83
The Costs of Sustained Request-Based Processing 84
Patterns for FaaS 84
The Decorator Pattern: Request or Response Transformation 85
Hands On: Adding Request Defaulting Prior to Request Processing 86
Handling Events 87
Hands On: Implementing Two-Factor Authentication 87
Event-Based Pipelines 89
Hands On: Implementing a Pipeline for New-User Signup 89
9. Ownership Election. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Determining If You Even Need Master Election 94
The Basics of Master Election 95
Hands On: Deploying etcd 97
Implementing Locks 98
Hands On: Implementing Locks in etcd 100
Implementing Ownership 101
Hands On: Implementing Leases in etcd 102
Handling Concurrent Data Manipulation 103
Table of Contents | vii
Part III. Batch Computational Patterns
10. Work Queue Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
A Generic Work Queue System 109
The Source Container Interface 110
The Worker Container Interface 112
The Shared Work Queue Infrastructure 113
Hands On: Implementing a Video Thumbnailer 115
Dynamic Scaling of the Workers 117
The Multi-Worker Pattern 118
11. Event-Driven Batch Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Patterns of Event-Driven Processing 122
Copier 122
Filter 123
Splitter 124
Sharder 125
Merger 127
Hands On: Building an Event-Driven Flow for New User Sign-Up 128
Publisher/Subscriber Infrastructure 129
Hands On: Deploying Kafka 130
12. Coordinated Batch Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Join (or Barrier Synchronization) 134
Reduce 135
Hands On: Count 136
Sum 137
Histogram 137
Hands On: An Image Tagging and Processing Pipeline 138
13. Conclusion: A New Beginning?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
viii | Table of Contents
Preface
Who Should Read This Book
At this point, nearly every developer is a developer or consumer (or both) of dis‐
tributed systems. Even relatively simple mobile applications are backed with cloud
APIs so that their data can be present on whatever device the customer happens to be
using. Whether you are new to developing distributed systems or an expert with scars
on your hands to prove it, the patterns and components described in this book can
transform your development of distributed systems from art to science. Reusable
components and patterns for distributed systems will enable you to focus on the core
details of your application. This book will help any developer become better, faster,
and more efficient at building distributed systems.
Why I Wrote This Book
Throughout my career as a developer of a variety of software systems from web
search to the cloud, I have built a large number of scalable, reliable distributed sys‐
tems. Each of these systems was, by and large, built from scratch. In general, this is
true of all distributed applications. Despite having many of the same concepts and
even at times nearly identical logic, the ability to apply patterns or reuse components
is often very, very challenging. This forced me to waste time reimplementing systems,
and each system ended up less polished than it might have otherwise been.
The recent introduction of containers and container orchestrators fundamentally
changed the landscape of distributed system development. Suddenly we have an
object and interface for expressing core distributed system patterns and building
reusable containerized components. I wrote this book to bring together all of the
practitioners of distributed systems, giving us a shared language and common stan‐
dard library so that we can all build better systems more quickly.
ix