How variables work in Lua
Trying to justify why a strange bit of code I had written works, I wrote this strange snippet:
local x = 1
local x = 2
print(firstX()) -- 1
print(secondX()) -- 2
print(x) -- 2
What is going on there? How does all this work?
First, it would do to explain how variables work in the first place.
Global variables are straightforward. Whenever you ask for a particular name, it lives in a particular place. In a compiled language like C, you can just stick them sequentially. Lua actually stores them in a table called the “environment” which can be accessed by getfenv.
x = 1
y = 2
x = x + y
y = 3
x and y are global in the above, so everywhere x is used, you mean the same x. This is really simple, but problematic — two different functions have to worry about accidentally using the same x, and recursion is impossible.
Local variables are more complicated. There is no longer just one place to look, since variables can be created all the time and are allowed to have the same names!
They are stored on a stack. Ideally, you always know where to look for your local variables, just like global variables. While they aren’t in as obvious a place as global variables, the trick is actually really simple:
Keep track of a list (actually a stack) of every local variable defined. A function’s local variables are thus always at the very end of the list, and thus can be identified by how many back from the end they are. Here’s an example:
local a = x + 1
local a = x * 2
local a = x + 1
local b = x + 2
local c = x + 3
At the point where print(x) happens, the stack looks like this (oldest first):
- x = 5 (first’s parameter)
- a = 6 (first’s local variable)
- x = 6 (second’s parameter)
- a = 12 (second’s local variable)
- x = 12 (third’s parameter)
- a = 13 (third’s local variable)
- b = 14 (third’s local variable)
- c = 15 (third’s local variable)
Thus the final print(b) thinks about b as THIRD + 3. Finding THIRD is easy, since the executing function is always the last (most recent) one, and it knows how many local variables it has defined. This easily extends to recursion.
The main problem the stack introduces is that variables “die” when the function (or other block, like if or while or for) ends. This is a problem if you want the object to live longer than the function — for example, if you need to return a table or a ROBLOX Instance, the value can’t be obliterated when the function ends.
For that purpose, variables that need to outlive the function are allocated on the heap. The variable will still exist on the stack, but it becomes a pointer — it just says where the value is, rather than the value itself.
That way, the underlying value survives, even when the variable dies.
Closures and Upvalues
There’s a big problem with what has been described so far. A local variable can be used in a closure — that’s when a function can get and set a local variable:
local i = 0
local function count()
i = i + 1
print(i) -- 0
local m = maker()
print(m()) -- 1
print(m()) -- 2
print(m()) -- 3
As described, this shouldn’t work. i ought to be on the stack — but then it wouldn’t survive after the end. But it is very much a local variable, and can be used normally.
Lua converts variables used into closures into upvalues. It saves the value onto the heap, and makes the local variable a pointer to that position in the heap. The function (count in this case) is wrapped up with its collection of upvalues, so that the m variable actually contains i’s pointer, as well as count. When you call m, it’s informed of where to look for its upvalue, and finds the value.
Like all good variables, it can get and set it because it knows where it is.
So how did that first snippet work?
When Lua creates the upvalues, it associates them with the appropriate functions. This creates two separate variables that are both fully fledged upvalues.
It should be noted that storing things on the stack and just pointing to it creates a big problem — just because you stop looking at something, doesn’t mean it will go away.
It takes up space forever — it becomes garbage.
Languages like Lua have garbage collectors which identify the forgotten pieces of heap-allocated memory to clear them out. This is a big reason why high-level languages are slower and more complicated than low-level languages — they create large amounts of garbage, and tracking down that garbage becomes difficult.