Terraform: Up and Running: Writing Infrastructure as Code

460 Pages·2022·9.978 MB·English

by  Yevgeniy Brikman

T
E hi
di r
tid
o
n
Terraform
Up & Running
B
r
ik
m
a
n
Writing Infrastructure as Code
Yevgeniy Brikman
Terraform: Up and Running
Terraform has become a key player in the DevOps world for
defining, launching, and managing infrastructure as code “This book teaches you
(IaC) across a variety of cloud and virtualization platforms, everything you need to
including AWS, Google Cloud, Azure, and more. This hands-on
know about Terraform
third edition, expanded and thoroughly updated for version
to massively improve
1.0 and beyond, shows you the fastest way to get up and
infrastructure provisioning
running with Terraform.
efficiency and enjoyability
Gruntwork cofounder Yevgeniy (Jim) Brikman takes you
across any platform.”
through code examples that demonstrate Terraform’s
—Mitchell Hashimoto
simple, declarative programming language for deploying
Creator of Terraform and
and managing infrastructure with a few commands. Veteran Cofounder of HashiCorp
sysadmins, DevOps engineers, and novice developers will
“If you are a DevOps
quickly go from Terraform basics to running a full stack that
can support a massive amount of traffic and a large team of practitioner and want
developers. to get started with
infrastructure as code,
• Compare Terraform with Chef, Puppet, Ansible,
this book is the perfect
CloudFormation, and Pulumi
resource for you.”
• Deploy servers, load balancers, and databases
—Akash Mahajan
• Create reusable infrastructure with Terraform modules Cofounder and Director, Appsecco
• Test your Terraform modules with static analysis, unit
tests, and integration tests Yevgeniy (Jim) Brikman is the
cofounder of Gruntwork, a company
• Configure CI/CD pipelines for both your apps and
with the mission of making it 10 times
infrastructure code
easier to build software. The author of
• Use advanced Terraform syntax for loops, conditionals, Hello, Startup: A Programmer’s Guide
and zero-downtime deployment to Building Products, Technologies, and
Teams (O’Reilly), he previously served
• Get up to speed on Terraform 0.13 to 1.0 and beyond as a software engineer at LinkedIn,
• Work with multiple clouds and providers (including TripAdvisor, Cisco, and Thomson
Financial. For more information, check
Kubernetes!)
out ybrikman.com.
INFRASTRUCTURE AS CODE Twitter: @oreillymedia
linkedin.com/company/oreilly-media
US $59.99 CAN $74.99 youtube.com/oreillymedia
ISBN: 978-1-098-11674-3
THIRD EDITION
Terraform: Up & Running
Writing Infrastructure as Code
Yevgeniy Brikman
BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo
Terraform: Up & Running
by Yevgeniy Brikman
Copyright © 2022 Yevgeniy Brikman. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional
sales department: 800-998-9938 or [email protected].
Acquisitions Editor: John Devins Indexer: nSight, Inc.
Developmental Editor: Corbin Collins Interior Designer: David Futato
Production Editor: Kate Galloway Cover Designer: Karen Montgomery
Copyeditor: Piper Editorial Consulting, LLC Illustrator: Kate Dullea
Proofreader: Kim Cofer
March 2017: First Edition
September 2019: Second Edition
September 2022: Third Edition
Revision History for the Third Edition
2022-09-19: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781098116743 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Terraform: Up & Running, the cover
image, and related trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the author, and do not represent the publisher’s views.
While the publisher and the author have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the author disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained in this work is at your
own risk. If any code samples or other technology this work contains or describes is subject to open
source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-098-11674-3
[LSI]
To Mom, Dad, Lyalya, and Molly

Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1. Why Terraform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What Is DevOps? 1
What Is Infrastructure as Code? 3
Ad Hoc Scripts 4
Configuration Management Tools 5
Server Templating Tools 7
Orchestration Tools 12
Provisioning Tools 14
What Are the Benefits of Infrastructure as Code? 16
How Does Terraform Work? 17
How Does Terraform Compare to Other IaC Tools? 20
Configuration Management Versus Provisioning 21
Mutable Infrastructure Versus Immutable Infrastructure 21
Procedural Language Versus Declarative Language 22
General-Purpose Language Versus Domain-Specific Language 25
Master Versus Masterless 26
Agent Versus Agentless 28
Paid Versus Free Offering 30
Large Community Versus Small Community 31
Mature Versus Cutting Edge 33
Use of Multiple Tools Together 34
Conclusion 36
2. Getting Started with Terraform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Setting Up Your AWS Account 40
Installing Terraform 43
v
Deploying a Single Server 44
Deploying a Single Web Server 52
Deploying a Configurable Web Server 60
Deploying a Cluster of Web Servers 66
Deploying a Load Balancer 70
Cleanup 79
Conclusion 80
3. How to Manage Terraform State. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
What Is Terraform State? 82
Shared Storage for State Files 83
Limitations with Terraform’s Backends 91
State File Isolation 93
Isolation via Workspaces 94
Isolation via File Layout 100
The terraform_remote_state Data Source 105
Conclusion 113
4. How to Create Reusable Infrastructure with Terraform Modules. . . . . . . . . . . . . . . . . . 115
Module Basics 118
Module Inputs 121
Module Locals 125
Module Outputs 127
Module Gotchas 129
File Paths 129
Inline Blocks 130
Module Versioning 133
Conclusion 139
5. Terraform Tips and Tricks: Loops, If-Statements, Deployment, and Gotchas. . . . . . . . 141
Loops 142
Loops with the count Parameter 142
Loops with for_each Expressions 149
Loops with for Expressions 156
Loops with the for String Directive 158
Conditionals 160
Conditionals with the count Parameter 160
Conditionals with for_each and for Expressions 165
Conditionals with the if String Directive 167
Zero-Downtime Deployment 169
Terraform Gotchas 179
count and for_each Have Limitations 179
vi | Table of Contents
Zero-Downtime Deployment Has Limitations 181
Valid Plans Can Fail 184
Refactoring Can Be Tricky 186
Conclusion 189
6. Managing Secrets with Terraform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Secret Management Basics 192
Secret Management Tools 193
The Types of Secrets You Store 193
The Way You Store Secrets 194
The Interface You Use to Access Secrets 195
A Comparison of Secret Management Tools 195
Secret Management Tools with Terraform 196
Providers 196
Resources and Data Sources 206
State Files and Plan Files 217
Conclusion 219
7. Working with Multiple Providers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Working with One Provider 221
What Is a Provider? 222
How Do You Install Providers? 223
How Do You Use Providers? 225
Working with Multiple Copies of the Same Provider 226
Working with Multiple AWS Regions 226
Working with Multiple AWS Accounts 238
Creating Modules That Can Work with Multiple Providers 245
Working with Multiple Different Providers 248
A Crash Course on Docker 249
A Crash Course on Kubernetes 252
Deploying Docker Containers in AWS Using Elastic Kubernetes Service 264
Conclusion 272
8. Production-Grade Terraform Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Why It Takes So Long to Build Production-Grade Infrastructure 277
The Production-Grade Infrastructure Checklist 279
Production-Grade Infrastructure Modules 280
Small Modules 281
Composable Modules 285
Testable Modules 291
Versioned Modules 298
Beyond Terraform Modules 305
Table of Contents | vii
Conclusion 313
9. How to Test Terraform Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Manual Tests 316
Manual Testing Basics 317
Cleaning Up After Tests 319
Automated Tests 320
Unit Tests 321
Integration Tests 348
End-to-End Tests 362
Other Testing Approaches 364
Conclusion 372
10. How to Use Terraform as a Team. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Adopting IaC in Your Team 376
Convince Your Boss 376
Work Incrementally 379
Give Your Team the Time to Learn 380
A Workflow for Deploying Application Code 382
Use Version Control 382
Run the Code Locally 383
Make Code Changes 383
Submit Changes for Review 384
Run Automated Tests 385
Merge and Release 386
Deploy 387
A Workflow for Deploying Infrastructure Code 390
Use Version Control 391
Run the Code Locally 395
Make Code Changes 396
Submit Changes for Review 397
Run Automated Tests 399
Merge and Release 400
Deploy 401
Putting It All Together 412
Conclusion 414
Appendix. Recommended Reading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
viii | Table of Contents