Make your code scriptable
In which we take a whirlwind tour of Lua’s C API, including but not limited to: writing a Lua interpreter in C and writing your own Lua mini-API from C. Kind of a sneak peek at my O’Reilly ebook, Creating Solid APIs with Lua.
Good languages tend to be great at some things and just-ok at others. Python is designed for read- and writability; C is designed for low-level portability; Go is designed to scale. This article is about Lua, which is designed to be portable like C yet flexible and powerful like… well, like JavaScript would be if it were more elegant from the get-go 😁. Lua also happens to be insanely easy to learn and — what I’ll focus on here—it’s the easiest-to-customize language I’ve met so far.
The When and the Why
You might ask me: “Tyler, in which situations can I integrate Lua into my app?” And the answer is “Yes.”
Any app that sits on top of C plays well with Lua. This directly applies to C (derp), C++ and Objective-C. It applies to languages that make C-calls easy, like Swift or C# . Some languages, like Go, Java, and Python, also support C integration, but with caveats. If you’re writing for the browser, then technically you could use some black magic like emscripten to essentially convert C to JavaScript but for the love of the flying spaghetti monster don’t do that (too often).
You can do fun things by adding Lua. (Adding Lua means that your existing code can call out to Lua, and Lua can call back.) You can make a game mod-able by setting up an API — for example, letting users define custom spells in an RPG. Adobe Lightroom empowers advanced photo-editing with user-written Lua scripts. Or you can split your dev work between hardware-intensive low-level code and easy-to-write high-level Lua.
Let’s write an interpreter because why not
Some folks start off with a “hello, world!” thingy, but sometimes you can quickly create a more impressive result. So let’s make an interpreter. Albeit a minimalistic one, weighing in at 25 lines of C.
Both allocation and cleanup of a Lua run-time state is simple. Allocation looks like this:
lua_State *L = luaL_newstate(); // Setup a new Lua run-time state.The result value L is a black box. We don’t ask questions about the actual fields of L, and no one gets hurt. This value is sent in as the first parameter to virtually every C function that works with Lua.
Finishing up your use of a Lua state works like so:
lua_close(L); // Deallocate all vars and memory used by L.That deallocation includes garbage collection of all values used from within Lua.
The full interpreter uses only 3 other functions from Lua’s C API. We’ll take a brief look at those after seeing the full interpreter code here:
Here are complete macOS steps to download this file along with Lua’s source, assuming you have some standard tools like curl and gcc already installed:
# Start in a fresh directory.
# This doesn't install anything outside the current directory.# Download, build, and move around the Lua source.curl -O https://www.lua.org/ftp/lua-5.3.4.tar.gz
tar xf lua-5.3.4.tar.gz && cd lua-5.3.4
make macosx test && cd ..
cp lua-5.3.4/src/* .# Download, build, and run our Lua interpreter.curl -L https://goo.gl/oJqCz2 > interp.c
gcc interp.c -o interp -llua
./interp# Try out "print('hi!')". Press ctrl-D to exit.
Let’s take a look at the 3 Lua C API functions we used in the interpreter but didn’t explain yet. Here’s the short version:
// Load Lua's standard libraries (table, math, etc) into state L.
lua_openlibs(L);// Parse the string `buff` as a Lua program; don't yet run the
// result, but hold on to it as a runnable function.
luaL_loadstring(L, buff);// Run the function loaded by luaL_loadstring().
lua_pcall(L, 0, 0, 0);
If you’re like me, you’re probably asking a few great questions right about now:
- Why are
luaL_loadstring()andlua_pcall()separate functions if they appear to work so well together? - What are all those zeros for in our call to
lua_pcall()? - What does the
pstand for inlua_pcall()?
String loading and calling are separate for a few reasons. The main one is that we get more power by splitting up these operations. We can parse code without committing to running it yet. We can run it multiple times without having to re-parse it. But we also get other benefits: It turns out that we can get more detailed error information based on either the parse results or, separately, the execution results (compile-time vs run-time errors). We can also use lua_pcall() to call functions from several other sources besides parsed strings, and we can use a couple of other calling mechanisms besides lua_pcall().
Speaking of lua_pcall(), its full signature looks like this:
int lua_pcall (lua_State *L, int nargs, int nresults, int errfunc);This C function calls a Lua function that has magically been passed to it. Not so magically, actually — all of Lua’s C API functions use a stack that lives inside L. Every Lua C API call performs precise manipulations to this stack, so that we can consistently be aware of its state (from C, not Lua). The luaL_loadstring() function pushed a Lua function onto the top of this stack, exactly where lua_pcall() can pick it up.
The nargs parameter to lua_pcall() indicates exactly how many input arguments to pass on over to the Lua function being called; these are pulled off the top of the stack, and are expected to live above (that is, later in the stack than) the Lua function being called. The nresults parameter tells lua_pcall() exactly how many return values you want, and these will be left on top of the stack when the function call is done. Lua is very un-C-like in that you can choose how many return values you want, and it’s never an error. Return values not provided by the called Lua function are compensated with nil values; return values not asked for are simply discarded. Finally, the errfunc parameter indicates the stack position (0 meaning null, 1 meaning the bottom = the 1st position, k meaning the kth position) of another Lua function; this other function is called in case a Lua exception is thrown before lua_pcall() completes its work. This mechanism acts similarly to a try-catch block in other languages. Correspondingly, the p in lua_pcall() stands for protected, as errors can be caught. If you called lua_call() instead, a Lua exception would abort the entire process. Quite melodramatic, really. [All of these lovely details are documented with succinct clarity here in the official docs.]
In our interpreter, we blithely threw a slew of zeros into lua_pcall() and wished for the best. More precisely, the first zero means we called the code provided by the user’s string without giving it any arguments (which makes sense; we don’t usually think of interpreter statements as getting function arguments), the second zero means we are ignoring any return values (which, if I were writing a longer article, I would explain is not always what we want, but sometimes is), and the final zero value for errfunc means we have no error handler. Without an error handler, lua_pcall() is kind enough to not abort the process, but rather just keeps around an error message on top of the stack for us if there is one. The current C code silently ignores such errors. I told you it was pretty bare-bones.
A disturbingly pleasant introduction to creating a Lua API in C
If you have not already been blow away by writing a 25-line Lua interpreter in C, then this next example is certain to overwhelm your impressive capacity for sheer awesome. Prepare your brain… for in our next example, we will create a lua module that is capable of drawing a cow who will say whatever you want. [Note: most of the heavy lifting here is performed by the cowsay command, which you can install on macOS via brew install cowsay].
There are multiple ways of sharing a Lua API with users. In this section, I’ll stick with a common setup, which is to simply create a Lua module written in C. The advantages of this approach are:
- Basically anything your hardware can do, you can do from C.
- By writing a Lua module in C, you can bring that power into the realm of a nice, high-level language.
- The Lua-C interface is relatively clean, easy to learn, and fun to use.
The general structure of a Lua C module looks like this:
#include "lua.h"
// Include any other needed headers.static int myfunction(lua_State *L) {
// Your inputs from Lua-land are sitting on top of the
// stack in L. Use functions like lua_tostring(L, stack_index)
// to retrieve their values. // Note that stack_index -1 indicates the top of the stack
// just as mylist[-1] indicates the end of a list in Python. // When your function completes, return (in C) the number of
// values on top of the Lua stack that you want to be visible
// as Lua return values. Remember that the stack "lives inside"
// L; you can push values onto it with functions like
// lua_pushnumber(L, number). return num_return_values;
}// Define any other Lua-callable C functions you like.int luaopen_mymodule(lua_State *L) { // The function name luaopen_NAME() in the file NAME.so or
// NAME.dll is special to Lua; this is the init function for
// your module. It's called when the user requires your module:
//
// [some user does this in Lua:]
// mymodule = require 'mymodule' -- BAM! This C fn is called. // Use luaL_newlib() or something equivalent to set up a table
// with keys = your module's function names, values = your
// modules functions. This would include `myfunction`. // A 1 here tells `require` to return the table atop the stack.
return 1;
}
That’s the general idea. Here’s a specific working example:
Compiling a C module is a bit platform-dependent. I’ll assume you’ll run this from the same directory as earlier, where we put all those scrumptious Lua header files. Here’s a way to build the C module on macOS:
gcc cow.c -o cow.so -undefined dynamic_lookup -I.[Compilation tips specific to Windows and Ubuntu are in the book, by the way.]
Once you’ve done that, you’ll have a shiny new file called cow.so. The so stands for shared object, as this is executable object code that is designed to be used by possibly different binaries; hence it’s shared (or at least shareable in principle). It’s essentially the same as a DLL file on Windows (in fact this same source, compiled on Windows, would result in cow.dll).
Now you can use the cow module from the Lua interpreter, or any Lua script that has the file cow.so in its search path [advanced: how search paths work in Lua].
Here’s an example use of our cow module from Lua’s official interpreter (which ought to be in the same directory right now if you’ve done all the above steps!):
$ ./lua
Lua 5.3.4 Copyright (C) 1994-2017 Lua.org, PUC-Rio
> cow = require 'cow'
> cow.say('I moo-a for Lua')
_________________
< I moo-a for Lua >
-----------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
>Writing a game engine and other examples from the upside down
I hope the above examples gave you a fun peek into the expressive power and ease of integration Lua offers. Creating Solid APIs with Lua is structured around a running example that I particularly like (completely unbiased here): Over the course of 6 chapters, we build a minimalistic game engine in C for games written in Lua, along with a somewhat mod-able ASCII-art game on top of that game engine; this game is called EatyGuy because it’s a guy who eats little dots all over the place:
All of the source for this example—and in fact for the entire book—is available from the APIsWithLua github repo.
Where to go from here
If you like this stuff, here are a couple other resources I’ve put together:
- Learn Lua in 15 Minutes; works as a quick reference, or a tutorial if you know other languages.
- The apidemo module; this is a Lua module designed to help you learn and experiment with the C API. Includes a quick reference that covers most of the API in a surprisingly small amount of text.
Finally, you can probably read Solid APIs with Lua for free 😯🙀🎉. Here’s how:
- If your employer provides you with a free membership to Safari Books Online, then you’re already there. You can just read the whole thing right now.
- Otherwise: (a) You can sign up for Safari for free and read the whole thing right now. The catch is that you only get access for 10 days. Also (b) tell your employer to get everyone memberships. It’s one of those surprisingly-affordable perks that provides a treasure trove of knowledge and cool ideas.
