Programming Erlang. Software for a Concurrent World 2nd ed.

513 Pages·2013·2.211 MB·english

by  Joe Armstrong

Programming Erlang,
Second Edition
Software for a Concurrent World
Joe Armstrong
The Pragmatic Bookshelf
Copyright © 2013 Pragmatic Programmers, LLC.
Printed in the United States of America.
ISBN-13: 978-1-937785-53-6
Encoded using the finest acid-free high-entropy binary digits.
Book version: P1.0—August 2013
Contents
Introduction . . . . . . . . . . . . xiii
Part I — Why Erlang?
1. Introducing Concurrency . . . . . . . . . . 3
1.1 Modeling Concurrency 3
1.2 Benefits of Concurrency 6
1.3 Concurrent Programs and Parallel Computers 8
1.4 Sequential vs. Concurrent Programming Languages 9
2. A Whirlwind Tour of Erlang . . . . . . . . . 11
2.1 The Shell 11
2.2 Processes, Modules, and Compilation 13
2.3 Hello, Concurrency 15
Part II — Sequential Programming
3. Basic Concepts . . . . . . . . . . . . 25
3.1 Starting and Stopping the Erlang Shell 25
3.2 Simple Integer Arithmetic 27
3.3 Variables 28
3.4 Floating-Point Numbers 32
3.5 Atoms 33
3.6 Tuples 34
3.7 Lists 37
3.8 Strings 39
3.9 Pattern Matching Again 41
4. Modules and Functions . . . . . . . . . . 43
4.1 Modules Are Where We Store Code 43
4.2 Back to Shopping 50
4.3 Funs: The Basic Unit of Abstraction 52
4.4 Simple List Processing 57
4.5 List Comprehensions 59
4.6 BIFs 63
4.7 Guards 64
4.8 case and if Expressions 68
4.9 Building Lists in Natural Order 70
4.10 Accumulators 71
5. Records and Maps . . . . . . . . . . . 75
5.1 When to Use Maps or Records 75
5.2 Naming Tuple Items with Records 76
5.3 Maps: Associative Key-Value Stores 79
6. Error Handling in Sequential Programs . . . . . . 87
6.1 Handling Errors in Sequential Code 88
6.2 Trapping an Exception with try…catch 89
6.3 Trapping an Exception with catch 92
6.4 Programming Style with Exceptions 93
6.5 Stack Traces 95
6.6 Fail Fast and Noisily, Fail Politely 96
7. Binaries and the Bit Syntax . . . . . . . . . 99
7.1 Binaries 99
7.2 The Bit Syntax 101
7.3 Bitstrings: Processing Bit-Level Data 110
8. The Rest of Sequential Erlang . . . . . . . . 113
8.1 apply 115
8.2 Arithmetic Expressions 116
8.3 Arity 116
8.4 Attributes 117
8.5 Block Expressions 120
8.6 Booleans 120
8.7 Boolean Expressions 121
8.8 Character Set 122
8.9 Comments 122
8.10 Dynamic Code Loading 122
8.11 Erlang Preprocessor 126
8.12 Escape Sequences 126
8.13 Expressions and Expression Sequences 127
8.14 Function References 128
8.15 Include Files 128
8.16 List Operations ++ and - - 129
8.17 Macros 129
8.18 Match Operator in Patterns 131
8.19 Numbers 132
8.20 Operator Precedence 133
8.21 The Process Dictionary 134
8.22 References 135
8.23 Short-Circuit Boolean Expressions 135
8.24 Term Comparisons 136
8.25 Tuple Modules 137
8.26 Underscore Variables 137
9. Types . . . . . . . . . . . . . . 141
9.1 Specifying Data and Function Types 141
9.2 Erlang Type Notation 143
9.3 A Session with the Dialyzer 148
9.4 Type Inference and Success Typing 152
9.5 Limitations of the Type System 155
10. Compiling and Running Your Program . . . . . . 159
10.1 Modifying the Development Environment 159
10.2 Different Ways to Run Your Program 161
10.3 Automating Compilation with Makefiles 166
10.4 When Things Go Wrong 169
10.5 Getting Help 172
10.6 Tweaking the Environment 173
Part III — Concurrent and Distributed Programs
11. Real-World Concurrency . . . . . . . . . 177
12. Concurrent Programming . . . . . . . . . 181
12.1 The Concurrency Primitives 182
12.2 Introducing Client-Server 184
12.3 Processes Are Cheap 189
12.4 Receive with a Timeout 191
12.5 Selective Receive 193
12.6 Registered Processes 194
12.7 A Word About Tail Recursion 196
12.8 Spawning with MFAs or Funs 197
13. Errors in Concurrent Programs . . . . . . . . 199
13.1 Error Handling Philosophy 199
13.2 Error Handling Semantics 202
13.3 Creating Links 203
13.4 Groups of Processes That All Die Together 204
13.5 Setting Up a Firewall 205
13.6 Monitors 205
13.7 Error Handling Primitives 206
13.8 Programming for Fault Tolerance 207
14. Distributed Programming . . . . . . . . . 211
14.1 Two Models for Distribution 212
14.2 Writing a Distributed Program 213
14.3 Building the Name Server 213
14.4 Libraries and BIFS for Distributed Programming 219
14.5 The Cookie Protection System 222
14.6 Socket-Based Distribution 224
Part IV — Programming Libraries and Frameworks
15. Interfacing Techniques . . . . . . . . . . 231
15.1 How Erlang Communicates with External Programs 232
15.2 Interfacing an External C Program with a Port 234
15.3 Calling a Shell Script from Erlang 240
15.4 Advanced Interfacing Techniques 240
16. Programming with Files . . . . . . . . . 243
16.1 Modules for Manipulating Files 243
16.2 Ways to Read a File 244
16.3 Ways to Write a File 251
16.4 Directory and File Operations 255
16.5 Bits and Pieces 258
16.6 A Find Utility 258
17. Programming with Sockets . . . . . . . . . 263
17.1 Using TCP 263
17.2 Active and Passive Sockets 272
17.3 Error Handling with Sockets 275
17.4 UDP 276
17.5 Broadcasting to Multiple Machines 280
17.6 A SHOUTcast Server 281
18. Browsing with Websockets and Erlang . . . . . . 287
18.1 Creating a Digital Clock 288
18.2 Basic Interaction 291
18.3 An Erlang Shell in the Browser 292
18.4 Creating a Chat Widget 293
18.5 IRC Lite 295
18.6 Graphics in the Browser 299
18.7 The Browser Server Protocol 301
19. Storing Data with ETS and DETS . . . . . . . 305
19.1 Types of Table 306
19.2 ETS Table Efficiency Considerations 308
19.3 Creating an ETS Table 309
19.4 Example Programs with ETS 310
19.5 Storing Tuples on Disk 315
19.6 What Haven’t We Talked About? 318
20. Mnesia: The Erlang Database . . . . . . . . 321
20.1 Creating the Initial Database 321
20.2 Database Queries 322
20.3 Adding and Removing Data in the Database 326
20.4 Mnesia Transactions 328
20.5 Storing Complex Data in Tables 332
20.6 Table Types and Location 333
20.7 The Table Viewer 336
20.8 Digging Deeper 337
21. Profiling, Debugging, and Tracing . . . . . . . 339
21.1 Tools for Profiling Erlang Code 340
21.2 Testing Code Coverage 341
21.3 Generating Cross-References 342
21.4 Compiler Diagnostics 343
21.5 Runtime Diagnostics 346
21.6 Debugging Techniques 347
21.7 The Erlang Debugger 350
21.8 Tracing Messages and Process Execution 352
21.9 Frameworks for Testing Erlang Code 355
22. Introducing OTP . . . . . . . . . . . 359
22.1 The Road to the Generic Server 360
22.2 Getting Started with gen_server 368
22.3 The gen_server Callback Structure 372
22.4 Filling in the gen_server Template 376
22.5 Digging Deeper 377
23. Making a System with OTP . . . . . . . . . 381
23.1 Generic Event Handling 382
23.2 The Error Logger 384
23.3 Alarm Management 392
23.4 The Application Servers 394
23.5 The Supervision Tree 396
23.6 Starting the System 400
23.7 The Application 403
23.8 File System Organization 405
23.9 The Application Monitor 406
23.10 How Did We Make That Prime? 407
23.11 Digging Deeper 409
Part V — Building Applications
24. Programming Idioms . . . . . . . . . . 413
24.1 Maintaining the Erlang View of the World 413
24.2 A Multipurpose Server 416
24.3 Stateful Modules 418
24.4 Adapter Patterns 419
24.5 Intentional Programming 422
25. Third-Party Programs . . . . . . . . . . 425
25.1 Making a Shareable Archive and Managing Your Code with
Rebar 425
25.2 Integrating External Programs with Our Code 428
25.3 Making a Local Copy of the Dependencies 430
25.4 Building Embedded Web Servers with Cowboy 431
26. Programming Multicore CPUs . . . . . . . . 439
26.1 Good News for Erlang Programmers 440
26.2 How to Make Programs Run Efficiently on a Multicore
CPU 441
26.3 Parallelizing Sequential Code 445
26.4 Small Messages, Big Computations 447
26.5 Parallelizing Computations with mapreduce 451
27. Sherlock’s Last Case . . . . . . . . . . 457
27.1 Finding Similarities in Data 458
27.2 A Session with Sherlock 458
27.3 The Importance of Partitioning the Data 463
27.4 Adding Keywords to the Postings 464
27.5 Overview of the Implementation 467
27.6 Exercises 469
27.7 Wrapping Up 470
A1. OTP Templates . . . . . . . . . . . . 471
A1.1 The Generic Server Template 471
A1.2 The Supervisor Template 474
A1.3 The Application Template 475
A2. A Socket Application . . . . . . . . . . 477
A2.1 An Example 477
A2.2 How lib_chan Works 479
A2.3 The lib_chan Code 483
A3. A Simple Execution Environment . . . . . . . 493
A3.1 How Erlang Starts 494
A3.2 Running Some Test Programs in SEE 496
A3.3 The SEE API 499
A3.4 SEE Implementation Details 500
A3.5 How Code Gets Loaded in Erlang 508
Index . . . . . . . . . . . . . . 511
Introduction
New hardware is increasingly parallel, so new programming languages must
support concurrency or they will die.
“The way the processor industry is going is to add more and more cores, but
nobody knows how to program those things. I mean, two, yeah; four, not really;
eight, forget it.” —Steve Jobs, Apple1
Well, Steve was wrong; we do know how to program multicores. We program
them in Erlang, and many of our programs just go faster as we add more
cores.
Erlang was designed from the bottom up to program concurrent, distributed,
fault-tolerant, scalable, soft, real-time systems. Soft real-time systems are
systems such as telephone exchanges, banking systems, and so on, where
rapid response times are important but it’s not a disaster if the odd timing
deadline is missed. Erlang systems have been deployed on a massive scale
and control significant parts of the world’s mobile communication networks.
If your problem is concurrent, if you are building a multiuser system, or if
you are building a system that evolves with time, then using Erlang might
save you a lot of work, since Erlang was explicitly designed for building such
systems.
“It’s the mutable state, stupid.” —Brian Goetz, Java Concurrency in Practice
Erlang belongs to the family of functional programming languages. Functional
programming forbids code with side effects. Side effects and concurrency
don’t mix. In Erlang it’s OK to mutate state within an individual process but
not for one process to tinker with the state of another process. Erlang has no
mutexes, no synchronized methods, and none of the paraphernalia of shared
memory programming.
1. http://bits.blogs.nytimes.com/2008/06/10/apple-in-parallel-turning-the-pc-world-upside-down/