Table Of ContentGo Cookbook
Build modular, readable and
testable applications in Go
Aaron Torres
BIRMINGHAM - MUMBAI
Go Cookbook
Copyright © 2017 Packt Publishing
First published: June 2017
Production reference: 1240617
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78328-683-6
www.packtpub.com
Contents
Preface
1
Chapter 1: I/O and File Systems
7
Introduction 7
Using the common I/O interfaces 8
Getting ready 8
How to do it... 9
How it works... 12
Using the bytes and strings packages 12
Getting ready 12
How to do it... 12
How it works... 16
Working with directories and files 16
Getting ready 17
How to do it... 17
How it works... 20
Working with the CSV format 21
Getting ready 21
How to do it... 21
How it works... 26
Working with temporary files 26
Getting ready 26
How to do it... 27
How it works... 28
Working with text/template and HTML/templates 28
Getting ready 29
How to do it... 29
How it works... 35
Chapter 2: Command-Line Tools
36
Introduction 36
Using command-line flags 37
Getting ready 37
How to do it... 37
How it works... 41
Using command-line arguments 41
Getting ready 41
How to do it... 41
How it works... 45
Reading and setting environment variables 45
Getting ready 45
How to do it... 45
How it works... 49
Configuration using TOML, YAML, and JSON 49
Getting ready 49
How to do it... 50
How it works... 55
Working with Unix pipes 56
Getting ready 56
How to do it... 56
How it works... 58
Catching and handling signals 58
Getting ready 58
How to do it... 59
How it works... 61
An ANSI coloring application 61
Getting ready 61
How to do it... 61
How it works... 64
Chapter 3: Data Conversion and Composition
65
Introduction 65
Converting data types and interface casting 66
Getting ready 66
How to do it... 66
How it works... 70
Working with numeric data types using math and math/big 70
Getting ready 70
How to do it... 71
How it works... 74
Currency conversions and float64 considerations 74
Getting ready 74
How to do it... 74
How it works... 78
Using pointers and SQL NullTypes for encoding and decoding 78
Getting ready 78
How to do it... 79
How it works... 84
Encoding and decoding Go data 84
Getting ready 84
How to do it... 84
How it works... 88
Struct tags and basic reflection in Go 88
Getting ready 89
How to do it... 89
How it works... 94
Implementing collections via closures 94
Getting ready 94
How to do it... 95
How it works... 98
Chapter 4: Error Handling in Go
99
Introduction 99
Handling errors and the Error interface 100
Getting ready 100
How to do it... 100
How it works... 103
Using the pkg/errors package and wrapping errors 103
Getting ready 103
How to do it... 103
How it works... 106
Using the log package and understanding when to log errors 107
Getting ready 107
How to do it... 107
How it works... 110
Structured logging with the apex and logrus packages 110
Getting ready 111
How to do it... 111
How it works... 114
Logging with the context package 114
Getting ready 114
How to do it... 114
How it works... 117
Using package-level global variables 118
Getting ready 118
How to do it... 118
How it works... 121
Catching panics for long running processes 121
Getting ready 121
How to do it... 121
How it works... 123
Chapter 5: All about Databases and Storage
124
Introduction 124
The database/sql package with MySQL 125
Getting ready 125
How to do it... 125
How it works... 129
Executing a database transaction interface 129
Getting ready 130
How to do it... 130
How it works... 134
Connection pooling, rate limiting, and timeouts for SQL 134
Getting ready 134
How to do it... 134
How it works... 137
Working with Redis 137
Getting ready 137
How to do it... 138
How it works... 141
Using NoSQL with MongoDB and mgo 141
Getting ready 142
How to do it... 142
How it works... 144
Creating storage interfaces for data portability 145
Getting ready 145
How to do it... 145
How it works... 148
Chapter 6: Web Clients and APIs
150
Introduction 150
Initializing, storing, and passing http.Client structs 151
Getting ready 151
How to do it... 151
How it works... 155
Writing a client for a REST API 155
Getting ready 155
How to do it... 156
How it works... 158
Executing parallel and async client requests 159
Getting ready 159
How to do it... 159
How it works... 161
Making use of OAuth2 clients 162
Getting ready 162
How to do it... 162
How it works... 165
Implementing an OAuth2 token storage interface 166
Getting ready 166
How to do it... 166
How it works... 171
Wrapping a client in added functionality and function composition 172
Getting ready 172
How to do it... 173
How it works... 176
Understanding GRPC clients 177
Getting ready 177
How to do it... 177
How it works... 181
Chapter 7: Microservices for Applications in Go
182
Introduction 182
Working with web handlers, requests, and ResponseWriters 183
Getting ready 183
How to do it... 184
How it works... 186
Using structs and closures for stateful handlers 187
Getting ready 187
How to do it... 187
How it works... 191
Validating input for Go structs and user inputs 192
Getting ready 192
How to do it... 192
How it works... 196
Rendering and content negotiation 196
Getting ready 196
How to do it... 197
How it works... 199
Implementing and using middleware 200
Getting ready 200
How to do it... 200
How it works... 204
Building a reverse proxy application 204
Getting ready 204
How to do it... 205
How it works... 208
Exporting GRPC as a JSON API 208
Getting ready 208
How to do it... 208
How it works... 214
Chapter 8: Testing
215
Introduction 215
Mocking using the standard library 216
Getting ready 216
How to do it... 216
How it works... 220
Using the Mockgen package 220
Getting ready 220
How to do it... 220
How it works... 224
Using table-driven tests to improve coverage 224
Getting ready 225
How to do it... 225
How it works... 227
Using third-party testing tools 227
Getting ready 227
How to do it... 228
How it works... 231
Practical fuzzing 232
Getting ready 232
How to do it... 232
How it works... 235
Behavior testing using Go 236
Getting ready 236
How to do it... 236
How it works... 240
Chapter 9: Parallelism and Concurrency
241
Introduction 241
Using channels and the select statement 242
Getting ready 242
How to do it... 242
How it works... 245
Performing async operations with sync.WaitGroup 245
Getting ready 245
How to do it... 246
How it works... 249
Using atomic operations and mutex 249
Getting ready 249
How to do it... 250
How it works... 253
Using the context package 253
Getting ready 254
How to do it... 254
How it works... 256
Executing state management for channels 257
Getting ready 257
How to do it... 257
How it works... 261
Using the worker pool design pattern 261
Getting ready 261
How to do it... 261
How it works... 265
Using workers to create pipelines 266
Getting ready 266
How to do it... 266
How it works... 271
Chapter 10: Distributed Systems
272
Introduction 272
Using service discovery with Consul 273
Getting ready 273
How to do it... 273
How it works... 276
Implementing basic consensus using Raft 277
Getting ready 277
How to do it... 277
How it works... 283
Using containerization with Docker 283
Getting ready 284
How to do it... 284
How it works... 287
Orchestration and deployment strategies 288
Getting ready 288
How to do it... 288
How it works... 290
Monitoring applications 291
Getting ready 291
How to do it... 291
How it works... 294
Collecting metrics 294
Getting ready 294
How to do it... 294
How it works... 297
Chapter 11: Reactive Programming and Data Streams
298
Introduction 298
Goflow for dataflow programming 299
Getting ready 299
How to do it... 299
How it works... 302
Reactive programming with RxGo 303
Getting ready 303
How to do it... 303
How it works... 306
Using Kafka with Sarama 306
Getting ready 306
How to do it... 307
How it works... 309
Using async producers with Kafka 310
Getting ready 310
How to do it... 310
How it works... 313
Connecting Kafka to Goflow 313
Getting ready 314
