Last update: Sat Mar 18 22:00:08 GMT 2017 (build 21) Programming in Lua, Fourth Edition Roberto Ierusalimschy Programming in Lua, Fourth Edition Roberto Ierusalimschy Copyright © 2016, 2003 Roberto Ierusalimschy Feisty Duck Digital Book Distribution www.feistyduck.com Licensed for the exclusive use of: Eric Taylor <[email protected]> Table of Contents About the Book ................................................................................................................. ix I. The Basics ...................................................................................................................... 1 1. Getting Started ........................................................................................................ 4 Chunks .............................................................................................................. 4 Some Lexical Conventions ................................................................................... 6 Global Variables ................................................................................................. 7 Types and Values ................................................................................................ 7 Nil ............................................................................................................ 8 Booleans .................................................................................................... 8 The Stand-Alone Interpreter .................................................................................. 9 2. Interlude: The Eight-Queen Puzzle ............................................................................ 12 3. Numbers ............................................................................................................... 15 Numerals .......................................................................................................... 15 Arithmetic Operators .......................................................................................... 16 Relational Operators .......................................................................................... 17 The Mathematical Library ................................................................................... 18 Random-number generator .......................................................................... 18 Rounding functions .................................................................................... 18 Representation Limits ......................................................................................... 19 Conversions ...................................................................................................... 21 Precedence ....................................................................................................... 22 Lua Before Integers ........................................................................................... 22 4. Strings ................................................................................................................. 24 Literal strings .................................................................................................... 24 Long strings ..................................................................................................... 25 Coercions ......................................................................................................... 26 The String Library ............................................................................................. 27 Unicode ........................................................................................................... 29 5. Tables .................................................................................................................. 33 Table Indices .................................................................................................... 33 Table Constructors ............................................................................................. 35 Arrays, Lists, and Sequences ............................................................................... 36 Table Traversal ................................................................................................. 38 Safe Navigation ................................................................................................. 38 The Table Library ............................................................................................. 39 6. Functions .............................................................................................................. 42 Multiple Results ................................................................................................ 43 Variadic Functions ............................................................................................. 45 The function table.unpack ............................................................................ 47 Proper Tail Calls ............................................................................................... 48 7. The External World ................................................................................................ 50 The Simple I/O Model ....................................................................................... 50 The Complete I/O Model .................................................................................... 53 Other Operations on Files ................................................................................... 54 Other System Calls ............................................................................................ 55 Running system commands ......................................................................... 55 8. Filling some Gaps .................................................................................................. 57 Local Variables and Blocks ................................................................................. 57 Control Structures .............................................................................................. 58 if then else ............................................................................................... 58 while ....................................................................................................... 59 iii Programming in Lua, Fourth Edition repeat ..................................................................................................... 59 Numerical for ........................................................................................... 60 Generic for .............................................................................................. 60 break, return, and goto ..................................................................................... 61 II. Real Programming ......................................................................................................... 65 9. Closures ............................................................................................................... 68 Functions as First-Class Values ............................................................................ 68 Non-Global Functions ........................................................................................ 69 Lexical Scoping ................................................................................................ 71 A Taste of Functional Programming ..................................................................... 74 10. Pattern Matching .................................................................................................. 77 The Pattern-Matching Functions ........................................................................... 77 The function string.find ...................................................................... 77 The function string.match .................................................................... 77 The function string.gsub ...................................................................... 78 The function string.gmatch .................................................................. 78 Patterns ............................................................................................................ 78 Captures ........................................................................................................... 82 Replacements .................................................................................................... 83 URL encoding .......................................................................................... 84 Tab expansion ........................................................................................... 86 Tricks of the Trade ............................................................................................ 86 11. Interlude: Most Frequent Words ............................................................................. 90 12. Date and Time ..................................................................................................... 92 The Function os.time ..................................................................................... 92 The Function os.date ..................................................................................... 93 Date–Time Manipulation .................................................................................... 95 13. Bits and Bytes ..................................................................................................... 97 Bitwise Operators .............................................................................................. 97 Unsigned Integers .............................................................................................. 97 Packing and Unpacking Binary Data ..................................................................... 99 Binary files ..................................................................................................... 101 14. Data Structures ................................................................................................... 104 Arrays ............................................................................................................ 104 Matrices and Multi-Dimensional Arrays ............................................................... 105 Linked Lists .................................................................................................... 107 Queues and Double-Ended Queues ..................................................................... 107 Reverse Tables ................................................................................................ 108 Sets and Bags ................................................................................................. 109 String Buffers .................................................................................................. 110 Graphs ........................................................................................................... 111 15. Data Files and Serialization .................................................................................. 114 Data Files ....................................................................................................... 114 Serialization .................................................................................................... 116 Saving tables without cycles ...................................................................... 118 Saving tables with cycles .......................................................................... 119 16. Compilation, Execution, and Errors ....................................................................... 122 Compilation .................................................................................................... 122 Precompiled Code ............................................................................................ 125 Errors ............................................................................................................. 126 Error Handling and Exceptions .......................................................................... 127 Error Messages and Tracebacks .......................................................................... 128 17. Modules and Packages ........................................................................................ 131 The Function require .................................................................................... 132 iv Programming in Lua, Fourth Edition Renaming a module ................................................................................. 133 Path searching ......................................................................................... 133 Searchers ................................................................................................ 135 The Basic Approach for Writing Modules in Lua .................................................. 135 Submodules and Packages ................................................................................. 137 III. Lua-isms ................................................................................................................... 139 18. Iterators and the Generic for ................................................................................ 142 Iterators and Closures ....................................................................................... 142 The Semantics of the Generic for ....................................................................... 143 Stateless Iterators ............................................................................................. 145 Traversing Tables in Order ................................................................................ 146 True Iterators .................................................................................................. 147 19. Interlude: Markov Chain Algorithm ....................................................................... 149 20. Metatables and Metamethods ................................................................................ 152 Arithmetic Metamethods ................................................................................... 152 Relational Metamethods .................................................................................... 155 Library-Defined Metamethods ............................................................................ 155 Table-Access Metamethods ............................................................................... 156 The __index metamethod ....................................................................... 156 The __newindex metamethod ................................................................. 157 Tables with default values ......................................................................... 158 Tracking table accesses ............................................................................. 159 Read-only tables ...................................................................................... 160 21. Object-Oriented Programming .............................................................................. 162 Classes ........................................................................................................... 163 Inheritance ...................................................................................................... 165 Multiple Inheritance ......................................................................................... 166 Privacy ........................................................................................................... 168 The Single-Method Approach ............................................................................ 170 Dual Representation ......................................................................................... 170 22. The Environment ................................................................................................ 173 Global Variables with Dynamic Names ............................................................... 173 Global-Variable Declarations ............................................................................. 174 Non-Global Environments ................................................................................. 176 Using _ENV .................................................................................................... 177 Environments and Modules ............................................................................... 180 _ENV and load .............................................................................................. 181 23. Garbage ............................................................................................................ 183 Weak Tables ................................................................................................... 183 Memorize Functions ......................................................................................... 184 Object Attributes ............................................................................................. 185 Revisiting Tables with Default Values ................................................................. 186 Ephemeron Tables ........................................................................................... 187 Finalizers ........................................................................................................ 188 The Garbage Collector ...................................................................................... 190 Controlling the Pace of Collection ...................................................................... 191 24. Coroutines ......................................................................................................... 194 Coroutine Basics .............................................................................................. 194 Who Is the Boss? ............................................................................................ 196 Coroutines as Iterators ...................................................................................... 198 Event-Driven Programming ............................................................................... 200 25. Reflection .......................................................................................................... 205 Introspective Facilities ...................................................................................... 205 Accessing local variables .......................................................................... 207 v Programming in Lua, Fourth Edition Accessing non-local variables .................................................................... 208 Accessing other coroutines ........................................................................ 209 Hooks ............................................................................................................ 210 Profiles .......................................................................................................... 211 Sandboxing ..................................................................................................... 212 26. Interlude: Multithreading with Coroutines ............................................................... 217 IV. The C API ................................................................................................................ 221 27. An Overview of the C API .................................................................................. 223 A First Example .............................................................................................. 223 The Stack ....................................................................................................... 225 Pushing elements ..................................................................................... 226 Querying elements ................................................................................... 227 Other stack operations .............................................................................. 229 Error Handling with the C API .......................................................................... 231 Error handling in application code .............................................................. 232 Error handling in library code .................................................................... 232 Memory Allocation .......................................................................................... 233 28. Extending Your Application ................................................................................. 236 The Basics ...................................................................................................... 236 Table Manipulation .......................................................................................... 237 Some short cuts ....................................................................................... 240 Calling Lua Functions ...................................................................................... 241 A Generic Call Function ................................................................................... 242 29. Calling C from Lua ............................................................................................ 247 C Functions .................................................................................................... 247 Continuations .................................................................................................. 249 C Modules ...................................................................................................... 251 30. Techniques for Writing C Functions ...................................................................... 254 Array Manipulation .......................................................................................... 254 String Manipulation .......................................................................................... 255 Storing State in C Functions .............................................................................. 258 The registry ............................................................................................ 258 Upvalues ................................................................................................ 260 Shared upvalues ....................................................................................... 263 31. User-Defined Types in C ..................................................................................... 265 Userdata ......................................................................................................... 265 Metatables ...................................................................................................... 268 Object-Oriented Access ..................................................................................... 270 Array Access .................................................................................................. 271 Light Userdata ................................................................................................. 272 32. Managing Resources ........................................................................................... 274 A Directory Iterator .......................................................................................... 274 An XML Parser ............................................................................................... 277 33. Threads and States .............................................................................................. 286 Multiple Threads ............................................................................................. 286 Lua States ...................................................................................................... 289 vi List of Figures 2.1. The eight-queen program .............................................................................................. 13 7.1. A program to sort a file ............................................................................................... 52 8.1. An example of a state machine with goto ........................................................................ 62 8.2. A maze game ............................................................................................................. 63 8.3. A strange (and invalid) use of a goto .............................................................................. 64 9.1. Union, intersection, and difference of regions ................................................................... 75 9.2. Drawing a region in a PBM file ..................................................................................... 75 11.1. Word-frequency program ............................................................................................ 91 12.1. Directives for function os.date ................................................................................. 94 13.1. Unsigned division ...................................................................................................... 98 13.2. Dumping the dump program ...................................................................................... 102 14.1. Multiplication of sparse matrices ................................................................................ 106 14.2. A double-ended queue .............................................................................................. 108 14.3. Reading a graph from a file ....................................................................................... 112 14.4. Finding a path between two nodes .............................................................................. 112 15.1. Quoting arbitrary literal strings ................................................................................... 117 15.2. Serializing tables without cycles ................................................................................. 118 15.3. Saving tables with cycles .......................................................................................... 120 16.1. Example of output from luac -l .............................................................................. 125 16.2. String repetition ....................................................................................................... 130 17.1. A homemade package.searchpath ...................................................................... 134 17.2. A simple module for complex numbers ........................................................................ 136 17.3. Module with export list ............................................................................................. 137 18.1. Iterator to traverse all words from the standard input ...................................................... 143 19.1. Auxiliary definitions for the Markov program ............................................................... 150 19.2. The Markov program ................................................................................................ 151 20.1. A simple module for sets .......................................................................................... 153 20.2. Tracking table accesses ............................................................................................. 159 21.1. the Account class .................................................................................................. 165 21.2. An implementation of multiple inheritance ................................................................... 167 21.3. Accounts using a dual representation ........................................................................... 171 22.1. The function setfield ........................................................................................... 174 22.2. Checking global-variable declaration ........................................................................... 176 23.1. Constant-function factory with memorization ................................................................ 187 23.2. Running a function at every GC cycle ......................................................................... 190 23.3. Finalizers and memory .............................................................................................. 192 24.1. Producer–consumer with filters ................................................................................... 198 24.2. A function to generate permutations ............................................................................ 199 24.3. An ugly implementation of the asynchronous I/O library ................................................. 201 24.4. Reversing a file in event-driven fashion ....................................................................... 202 24.5. Running synchronous code on top of the asynchronous library ......................................... 203 25.1. Getting the value of a variable ................................................................................... 208 25.2. Hook for counting number of calls .............................................................................. 211 25.3. Getting the name of a function ................................................................................... 212 25.4. A naive sandbox with hooks ...................................................................................... 213 25.5. Controlling memory use ............................................................................................ 214 25.6. Using hooks to bar calls to unauthorized functions ......................................................... 215 26.1. Function to download a Web page .............................................................................. 218 26.2. The dispatcher ......................................................................................................... 219 26.3. Dispatcher using select ......................................................................................... 220 27.1. A bare-bones stand-alone Lua interpreter ..................................................................... 224 vii Programming in Lua, Fourth Edition 27.2. Dumping the stack ................................................................................................... 229 27.3. Example of stack manipulation ................................................................................... 231 28.1. Getting user information from a configuration file ......................................................... 236 28.2. A particular getcolorfield implementation ............................................................ 238 28.3. Colors as strings or tables ......................................................................................... 240 28.4. Calling a Lua function from C ................................................................................... 242 28.5. A generic call function .............................................................................................. 243 28.6. Pushing arguments for the generic call function ............................................................. 244 28.7. Retrieving results for the generic call function ............................................................... 245 29.1. A function to read a directory .................................................................................... 249 29.2. Implementation of pcall with continuations ............................................................... 251 30.1. The function map in C ............................................................................................. 255 30.2. Splitting a string ...................................................................................................... 256 30.3. The function string.upper ................................................................................... 257 30.4. A simplified implementation for table.concat ........................................................ 258 30.5. An implementation of tuples ...................................................................................... 262 31.1. Manipulating a Boolean array .................................................................................... 266 31.2. Extra code for the Boolean array library ...................................................................... 267 31.3. New versions for setarray/getarray .................................................................... 269 31.4. New initialization code for the Bit Array library ............................................................ 272 32.1. The dir.open factory function ................................................................................ 275 32.2. Other functions for the dir library ............................................................................. 276 32.3. Function to create XML parser objects ........................................................................ 280 32.4. Function to parse an XML fragment ............................................................................ 281 32.5. Handler for character data ......................................................................................... 282 32.6. Handler for end elements .......................................................................................... 282 32.7. Handler for start elements ......................................................................................... 283 32.8. Method to close an XML parser ................................................................................. 283 32.9. Initialization code for the lxp library .......................................................................... 284 33.1. Function to search for a process waiting for a channel .................................................... 291 33.2. Function to add a process to a waiting list .................................................................... 291 33.3. Functions to send and receive messages ....................................................................... 292 33.4. Function to create new processes ................................................................................ 293 33.5. Body for new threads ............................................................................................... 294 33.6. Extra functions for the lproc module ........................................................................ 295 33.7. Registering libraries to be opened on demand ............................................................... 296 viii About the Book When Waldemar, Luiz, and I started the development of Lua, back in 1993, we could hardly imagine that it would spread as it did. Started as an in-house language for two specific projects, currently Lua is widely used in all areas that can benefit from a simple, extensible, portable, and efficient scripting language, such as embedded systems, mobile devices, the Internet of Things, and, of course, games. We designed Lua, from the beginning, to be integrated with software written in C/C++ and other con- ventional languages. This integration brings many benefits. Lua is a small and simple language, partly because it does not try to do what C is already good for, such as sheer performance and interface with third-party software. Lua relies on C for these tasks. What Lua does offer is what C is not good for: a good distance from the hardware, dynamic structures, no redundancies, and ease of testing and debugging. For these goals, Lua has a safe environment, automatic memory management, and good facilities for handling strings and other kinds of data with dynamic size. Part of the power of Lua comes from its libraries. This is not by chance. After all, one of the main strengths of Lua is its extensibility. Many features contribute to this strength. Dynamic typing allows a great degree of polymorphism. Automatic memory management simplifies interfaces, because there is no need to de- cide who is responsible for allocating and deallocating memory or how to handle overflows. First-class functions allow a high degree of parameterization, making functions more versatile. More than an extensible language, Lua is also a glue language. Lua supports a component-based approach to software development, where we create an application by gluing together existing high-level compo- nents. These components are written in a compiled, statically-typed language, such as C or C++; Lua is the glue that we use to compose and connect these components. Usually, the components (or objects) represent more concrete, low-level concepts (such as widgets and data structures) that are not subject to many changes during program development, and that take the bulk of the CPU time of the final program. Lua gives the final shape of the application, which will probably change a lot during the life cycle of the product. We can use Lua not only to glue components, but also to adapt and reshape them, and to create completely new components. Of course, Lua is not the only scripting language around. There are other languages that you can use for more or less the same purposes. Nevertheless, Lua offers a set of features that makes it your best choice for many tasks and gives it a unique profile: Extensibility: Lua's extensibility is so remarkable that many people regard Lua not as a language, but as a kit for building domain-specific languages. We designed Lua from scratch to be extended, both through Lua code and through external C code. As a proof of concept, Lua implements most of its own basic functionality through external libraries. It is really easy to interface Lua with external languages like C/C++, Java, C#, and Python. Simplicity: Lua is a simple and small language. It has few (but powerful) concepts. This sim- plicity makes Lua easy to learn and contributes to its small size. (Its Linux 64-bit executable, including all standard libraries, has 220 KB.) Efficiency: Lua has a quite efficient implementation. Independent benchmarks show Lua as one of the fastest languages in the realm of scripting languages. Portability: When we talk about portability, we are talking about running Lua on all platforms we have ever heard about: all flavors of UNIX (Linux, FreeBSD, etc.) Windows, Android, iOS, OS X, IBM mainframes, game consoles (PlayStation, Xbox, Wii, etc.), microcontrollers (Arduino, etc.), and many more. The source code for each of these platforms is virtually the same. Lua does not use conditional compilation ix