In-depth understanding of LUA closures: concepts and applications

In this paper, we first explain the concept of closure in Lua, then summarize the application of closure, and finally discuss the implementation principle of the closure in Lua.

The concept of closures

In Lua, closures (closure) consist of a function and a non-local variable (or upvalue) that the function accesses, where a non-local variable (non-local variable) is a variable that is not defined within the local scope. But at the same time not a global variable, mainly used in nested functions and anonymous functions, so if a closure does not have access to the non-local variables, then it is usually said function. In other words, in Lua, a function is a special case of closures. In addition, in Lua's C API, all of the core APIs for functions in Lua are named by closure (not function), which can also be seen as a continuation of this view. In Lua, a function is a first-type value (first-class value) that has a specific lexical domain (Lexical scoping).

The first type value indicates that the function has the same rights as other traditional types of values, such as numbers and string types. That is, a function can be stored in a variable or table, passed as an argument to another function, or as a return value for other functions, and can be created during run time. In Lua, functions are as anonymous as all other values, that is, they do not have a name. When discussing a function (for example, print), you are essentially discussing a variable that holds a function. Like what:

?

1	function foo(x) print(x) end

The essence is equivalent to

?

1	foo = function (x) print(x) end

So a function definition is essentially an assignment statement that creates a value of type "function" and assigns a value to a variable. An expression function (x) can be

End is treated as a function constructor, just like the constructor of table {}.

It is worth mentioning that the C language inside functions cannot be created at run time, therefore not the first class of values, but sometimes they are referred to as the second class of values, because they can be implemented through the function pointers to certain features, such as the often apparent callback function shadow.

A lexical domain is one in which a function can be nested within another function, and the internal function can access the variables of the external function. Like what:

?

12345678910111213

function f1(n)   --函数参数n也是局部变量   local function f2()      print(n)   --引用外部函数的局部变量   end   return f2endg1 = f1(2015)g1() -- 打印出2015g2 = f1(2016)g2() -- 打印出2016

Note that the G1 and G2 function bodies are the same (all function bodies of the F1 inline function F2), but the print values are different. This is because when both closures are created, they all have separate instances of local variable N. In fact, when Lua compiles a function, it generates a prototype for him (prototype), which contains the virtual machine instruction for the function body, the constant value (number, text string, and so on) used by the function, and some debugging information. At run time, whenever Lua executes an expression like function...end, he creates a new data object that contains a reference to the corresponding function prototype and an array of all the Upvalue references, which are called closures. Thus, the function is a compile-time concept, is static, and the closure is the concept of runtime, is dynamic. The value of G1 and G2 is strictly not a function but a closure, and two different closures, and each closure can retain its own upvalue value, so G1 and G2 will certainly not print the same result.

Here the function F2 can access the parameter n, and n is the local variable of the external function F1. In F2, the variable n is either a global variable or a local variable, which is referred to as a non-local variable (non-local variable) or upvalue. Upvalue actually refers to variables rather than values, which can be shared between internal functions, i.e. Upvalue provides a way to share data between closures, such as:

?

123456789101112131415161718

function Create(n)   local function foo1()      print(n)   end   local function foo2()      n = n + 10   end   return foo1,foo2endf1,f2 = Create(2015)f1() -- 打印2015f2()f1() -- 打印2025f2()f1() -- 打印2035

Note that in the above example, the closure F1 and F2 share the same upvalue, because when Lua discovers that two closures of Upvalue point to the same variable on the current stack, it intelligently generates only one copy, and then lets the two closures share the copy. Any closure that modifies the upvalue will be known to another.

It is also possible for a closure to be upvalue on the stack at the time it was created because the inline function can refer to the local variables of the outer outsourced function:

?

1234567891011121314151617181920212223242526

function Test(n)   local function foo()      local function inner1()         print(n)      end      local function inner2()         n = n + 10      end      return inner1,inner2   end   return fooendt = Test(2015)f1,f2 = t()f1()        -- 打印2015f2()f1()        -- 打印2025g1,g2 = t()g1()        -- 打印2025g2()g1()        -- 打印2035f1()        -- 打印2035

Note that the result of the above execution shows that the closure F1, F2, G1, and G2 all share the same upvalue, because when the creation of the Inner1,inner2, the two closures are created, no traces of n are found on the stack, but the upvalue of the closure foo is used directly. t = Test (2015), t this closure must have been properly preserved n, then F1, F2 if not found on the current stack n will automatically go to their outsourced closure of the Upvalue reference array to find, and copy the found reference value into their own Upvalue reference array. So the upvalue referenced by F1, F2, G1, and G2 are actually the same variable, and the search mechanism just described ensures that eventually their upvalue references will point to the same place.

Application of Closures
Closures are a valuable tool in many situations, mainly in the following areas:

I) as a parameter of a higher order function, such as a parameter like the Table.sort function.

II) The function that creates the other function, that is, the function returns a closure.

III) Closures are also useful for callback functions. The typical example is the callback function of the button on the interface, the function code logic may be identical, but the callback function parameter is different, that is, the value of Upvalue is different.

V) Create a safe operating environment, known as sandbox (sandbox). A secure runtime environment is required when some untrusted code is executed. For example, to restrict a program to access files, you only need to use closures to redefine the function Io.open:

?

1234567891011121314

do  local oldOpen = io.open  local accessOk = function(filename, mode)      <权限访问检查>        end  io.open = function (filename, mode)          if accessOk(filename, mode) then              return oldOpen(filename, mode)          else              return nil, access denied          end     endend

After being redefined, the original unsafe version is saved to the private variable of the closure, so that the external can no longer directly access the original version.

V) implements the iterator. A so-called iterator is a mechanism that iterates through a set of so-called elements. Each iterator needs to maintain some state between each successful call in order to know where it is and how to go to the next position. Closures are just right for this scenario. Like what:

?

12345678910111213

function values(t)    local i = 0    return function () i = i + 1 return t[i] endendt = {10, 20, 30}iter = values(t)while true do    local element = iter()    if element == nil then break end    print(element)end

Implementation principle of closure package

When Lua compiles a function, it generates a prototype (prototype) that includes the virtual machine instructions for the function, the constants in the function (numeric and string, and so on), and some debugging information. At any time as long as Lua executes a function: End is expressed, it creates a new closure (closure). Each closure has a reference to the corresponding function prototype and an array in which each element is a reference to the upvalue that can be used to access the external local variables (outer local variables). It is worth noting that before Lua 5.2, the closure also included a reference to the environment (environment), the environment is essentially a table, the function can index global variables in the table, starting from Lua 5.2, the closure of the environment, and introduce a variable _ Env to set the closure environment. Thus, the function is a compile-time concept, is static, and the closure is the concept of runtime, is dynamic.

Nested functions under the scope (generation period) rules how to implement memory functions to store local variables of external functions is a well-known conundrum (the combination of lexical scoping with first-class functions creates a Well-known difficulty for accessing outer local variables). For example:

?

12345678

function add (x)     returnfunction (y)         return x+y    endendadd2 = add(2)print(add2(5))

When ADD2 is called, its function body accesses the external local variable x (in Lua, the function parameter is also a local variable). However, when the ADD2 function is called, the Add function that created ADD2 has returned, and if X is created in the stack, when add returns, X is no longer present (that is, the storage space for x is recycled).

In order to solve the above problem, different languages have different methods, such as Python through scoping, Pascal restriction function nesting, and C language is not allowed. In Lua, closures are implemented using a structure called Upvalue. Any external local variables are accessed indirectly through Upvalue. The Upvalue initial value is pointed to the stack, which is the position of the variable in the stack. As on the left. When running, leaving the scope of the variable (that is, exceeding the variable life cycle), the variable is copied into the Upvalue structure (note that this operation is only performed at this time), such as the right. Because access to variables is done indirectly through pointers in the Upvalue structure, the copy operation has no effect on the code of any read or write variable. Unlike an intrinsic function (inner functions), the function that declares the local variable operates directly on the stack.

By creating a maximum of one upvalue for each variable and reusing the upvalue as needed, it ensures that local variables (pending VARs) in the pending state (not exceeding the life cycle) can be shared correctly between closures. To ensure this uniqueness, LUA maintains this linked list, where each node in the list corresponds to an open Upvalue (opend upvalue) structure, and the open Upvalue refers to the upvalue that is currently pointing to the stack local variable, such as the list of local variables for the pending state (the Pending VARs list). When LUA creates a new closure, LUA iterates through all the external local variables of the current function, and for each external local variable, if the variable is found in the linked list above, the open upvalue is reused, otherwise LUA creates a new open upvalue. and insert it into the list. When a local variable leaves the scope (that is, beyond the variable life cycle), the open upvalue becomes the closed upvalue (closed upvalue) and removes it from the linked list, as shown in the diagram on the right. Once a closed upvalue is no longer referenced by any closure, its storage space is recycled.

A function may have access to its outer function rather than to the local variable of the immediate outer function. In this case, when the closure is created, the local variable may not be in the stack. Lua uses flat closures (flat closures) to handle this situation. With flat closures, whenever a function accesses an external local variable and the variable is not in a direct external function, the variable enters the closure of the direct external function. When a function is instantiated, all the variables corresponding to the closure are either in the stack of the direct external function or in the closure of the direct external function. The last example in the first part is the case. The next article will analyze the source code implementation of the closure in Lua and the process of invoking it.

In-depth understanding of LUA closures: concepts and applications