C ++ lexical Closure

C ++ lexical Closure

Thomas M. breuel [1]

Translation: Wu Hui

Summary

We describe a function defined nesting in the C ++ programming language and provide extensions of lexical closures.

The main motivation for this extension is that it allows programmers to define the iterator for the Collection class as simply as a member function. Such an iterator accepts a function pointer or closure as a parameter; it provides lexical closure to make the expression state (for example, the accumulators) Natural and easy. This technology is commonplace in programming languages like scheme, T or Smalltalk-80, and in object-oriented programming languages, it may be the most concise and natural way to provide a general Iterative Structure. The ability to define nested functions also encourages a modular programming style.

We want to extend the C ++ language in this way, without introducing new data types for closures, and without affecting the efficiency of programs that do not use this feature. To do this, we recommend that you generate a small piece of code when creating a closure to load the static link pointer and jump to the function body. A closure is a pointer to this small piece of code. This technique allows processing closures like a pointer to a common C ++ function that does not reference any non-local or non-global variables.

We discuss compatibility with existing scope rules, syntaxes, allocation policies, portability, and efficiency.

1. What should we add ...... And why

We hope to nested function definitions in C ++ programs. In this section, we will discuss why nested function-defined capabilities are expected. Almost all other modern programming languages, such as scheme, T, Smalltalk-80, Common LISP, Pascal, Modula-2 and Ada, provide this feature.

To demonstrate the functions of this language feature, we must reach an agreement in terms of syntax. To represent the definition of a function in another function, we will simply move its definition to that function, but in other aspects it is the same as the fully-local function. For example, the following code snippet defines function2 (INT) in function1 (INT ):

Function1 (int x ){

//...

Function2 (inty ){

//...

}

//...

}

Unless function2 declares the identifier X, the X mentioned in function2 references the real parameter of function1.

To allow mutual recursion of functions at the internal lexical level, it is necessary to provide a certain method to declare a function that is not defined at the global level. We recommend that you add the keyword auto as the prefix to the function declaration. Currently, the use of the keyword auto before a function declaration is invalid. Therefore, this extension is compatible.

1.1 nesting and modularization

The ability to define nested functions encourages modular design of programs. It allows programmers to keep the functions and variables used by only one function in this function. In C, such modularization is only possible at the file level: the scope of a function identifier can be limited to a file by declaring it as static. The scope of a variable identifier shared between functions must contain at least one compilation unit, because it must be declared at a global level. C ++ supports limiting the scope of a function and variable identifier by making these functions and variables a member of a class.

However, it is not natural to introduce a new class just to limit the scope of an identifier. For example, consider the heap Sorting Algorithm wir79 ]. It contains a sorting function that repeatedly calls the sift function to insert elements into the heap. In C or C ++, We will represent the following:

Sift (int * V, int X, int L, int R ){

// Insert X into the heap

// Formed by elements l... r of V

}

// Sort an array of integers v

// V is n elements long

Heapsort (int * V, int N ){

// Codethat CILS Sift

}

However, this is not satisfactory, because the sift function is unlikely to be used elsewhere in the program. The sift function should be visible only in the heapsort function. In addition, we want to reference the variable V in sift without passing it as a parameter. Nesting allows us to rewrite:

// Sort an array of integers v

// V is n elements long

Heapsort (int * V, int N ){

Sift (intx, int L, int R ){

// Insert X into the heap

// Formed by elements l... r of V

}

// Codethat CILS Sift

}

Note that the use of the identifier V in the function sift encapsulates the real parameter V [2] of the function heapsort on the lexical basis.

For another example, assume that we have the function integrate to calculate the points of a given function between two boundaries, and we want to calculate the points of a parameterized function family. Similarly, the most natural expression is as follows:

Integrate_all (int n, double * as, double * BS, double * cs, double EPS ){

Doubleintegrate (double low, double high,

Doubleepsilon, double (* f) (double ));

Double A, B, C;

Doublef (Double X ){

Returna * x + B * x + C;

}

For (INTI = 0; I <n; I ++ ){

A = as [I]; B = BS [I]; C = cs [I];

Printf ("A: % G, B: % G, C: % G, I: % G \ n ",

A, B, C, integrate (0.0, 1.0, EPS, f ));

}

}

1.2 iterator

Nested function definition and the ability to reference variable declared by closed blocks, together with the iterator on the entire collection class, are particularly useful. Consider the following simple collection classes:

Class bagofints {

Public:

// Addan int to the Collection

Voidadd (INT );

// Testwhether an int

// Isin the collection

Intmember (INT );

// Apply a function to every

// Element in the collection

Voidwalk (void (*) (INT ));

};

For example, to print all elements in a package, we do this in Standard C ++:

Printit (int x ){

Printf ("integer in bag: % d \ n", X );

}

Main (){

Bagofintsabag;

...

Abag. Walk (printit );

}

Note that we are forced to define the function printit far away from its actual use. The reason why we make printit the first function is not because it is a useful abstraction of a process, nor because we will use it in several places. Only because the member function walk requires a function pointer as a parameter. Even more annoying, in standard C ++, a function passed as an iterator parameter can only work for static or global variables. If we want to use the iterator walk to summarize all the elements in a package, we must use a global variable:

Int xxx_counter;

Int xxx_count (int x ){

Xxx_counter + = X;

}

Fizzle (){

Bagofintsabag;

...

Xxx_counter = 0;

Abag. Walk (xxx_count );

INTSUM = xxx_counter;

...

}

If nested function definitions are allowed, we can simply express this:

Fizzle (){

Bagofintsabag;

...

INTSUM = 0;

Count (intx ){

Sum + = X;

}

Abag. Walk (count );

...

}

Now, all identifiers are declared and defined where they are used. The identifier that does not appear in the global scope should not be visible in the global scope [3]. In addition, the counter does not waste global data space. The space used for the counter only exists when the function fizzle is active.

This practice of writing iterator for user-defined classes is very common in Smalltalk and lisp-like languages. For example, in Smalltalk-80 [gr83], we can express the function fizzle as follows:

Fizzle

| Sum |

...

Sum <-0.

Abagdo: [x | sum <-sum + X].

...

In commonlisp [ste84], we can write:

(Define fizzle ()

(Let (abag ...)

(Sum 0 ))

...

(Mapnil # '(lambda (x) (incf sum X )))

...))

Readers may ask if there is any replacement method using the standard C ++ construction, and we will describe one of them soon. However, let's first remind ourselves of the iterator structures we require in any language.

An iterator basically accepts a piece of code, calls it repeatedly, and changes the construction of some bound values in the Code environment. According to the built-in iterators of C and C ++, we should be able to do the following.

1. In the source code where the iterator is to be executed, we can write down the code to be executed repeatedly.
2. This code that is repeatedly executed can reference identifiers with a limited scope in this encapsulated function.
3. The variable names modified by this iteration can be freely selected.

These attributes are a very useful feature of the built-in iterator, and it is a pity that the user-defined operators give up one of them.

One way to achieve these goals is to introduce a new class together with a macro, like the following:

Class bagofintsstepper {

Public:

Intmore ();

Intnext ();

};

# Define iteratebagfints (bag, VAR )\

For (bagofintsstepper stepper = bag. makestepper ();\

Stepper. More ();\

Var = stepper. Next ())

This construction will not undermine any of the above restrictions. However, it has several disadvantages. It requires us to introduce a new class for the purpose of expressing iteration, requires two calls to the member function in each iteration step, and requires us to define a macro. But most importantly, we do not like it because it is difficult to expand when we want to use the same base class [4] To iterate different collection classes. For example, we may have another class setofints. If we use the Iterative Structure in the form of walk, we can simply make bagofints and setofints a subclass of the class collectionofints, and declare the walk as a virtual function. Using a step (stepper) class to achieve the same effect is far less intuitive, and may require the additional overhead of calling two virtual functions in each iteration step.

2. How to Implement

To implement nesting and syntax closure in C ++, we must introduce a static link, which is a function closed in lexical form (for terminology, refer to [au79] and [wir77 ]) link each activity record to the current activity record. When we call a function, we must not only know its address, but also pass a pointer to the current activity record for the lexical closed function. However, there are two exceptions to this rule.

In the activity record of a fully-defined function, you do not need to keep the space of a static link pointer because its lexical environment is known to be a global environment and whether an identifier invokes a global variable, it is clear at the Compilation Time.

In addition, a function defining the full local area does not need to pass a pointer to the lexical closure environment, because this lexical closure environment is a global environment and is unique and has a known address.

These two exceptions, together with the fact that C ++ only allows fully-defined functions, enable lexical scopes in C ++ without the need to use an explicit or static chain, or pass a pointer to the environment.

It is not important to add space for static chains to activity records that do not define fully-local functions (they can be simply considered as additional automatic variables ). Because the compiler always knows which lexical level is corresponding to an activity record, it is not a problem for the activity record of the outermost lexical level function to be different from that of other lexical level functions.

The fact that an important problem is presented is that it is not only a pointer to the code, but also a pointer to the correct activity record of the function encapsulated on the lexical layer for calling a function at an internal lexical level.

If we want to freely pass pointers to functions at the internal lexical level, this information must be transmitted together with the pointer pointing to the function code, because this information cannot be deduced in any other way.

As long as the function is not used as a parameter of other functions, or the function is assigned to a variable, the compiler can silently ensure that the correct lexical environment information is transmitted to the caller. However, once we try to pass the function as data, we encounter a problem. In C ++, when defining and using a function pointer, you only need to reserve and pass enough space for a pointer. However, a closure, that is, a function with an environment, usually requires two pointers, as we just saw.

It is unacceptable to change the expression of a C ++ function pointer to two pointer Sizes Instead of a pointer. This means that, for example, a "function pointer" cannot be assigned to a variable of Type void * without losing information.

An alternative method may be to introduce a new data type, "closure", which must be used to express reference to functions and their environments if these functions are not defined in a fully-local area. Implicit type conversion from the function pointer to the closure will allow us to mix the function pointer that can be used in the closure position. However, the closure cannot be passed to the function of the expected function pointer. In addition, because of the two data types that essentially share the same concept, the language becomes unnecessary confusion.

Obviously, these practices are unacceptable. Fortunately, there is a simple and efficient solution. In fact, our solutions are fully compatible with C ++ and C. In particular:

1. Our closures can be used in a place where a C or C ++ function pointer can be used, or even code that uses closures is compiled by a compiler that does not know the closure.
2. When you add a closure to a compiler, the code generated for a function that does not contain nested functions remains unchanged.

Code snippets that generate a closure are implemented by generating a small segment of code that loads a static link to a known register and redirects to this function (the following list contains lines. At the beginning of a function, the content of this register is moved into the domain of the newly constructed activity record (row 0076) static chain. In an assembly language similar to 68000 assembly language [5], this will look like this:

0000; source code:

0001 ;;;

0002; function1 (){

0003; function2 (){

0004 ;;;...

0005 ;;;}

0006; void (* X )();

0007; X = function2;

0008 ;;;...

0009 ;;;}

0010

0011; machineregisters:

0012; FP: currentframe

0013; SL: A registerthat is used to hold the static link pointer

0014; temporarily

0015

0016; Instruction toload the SL register with constant

0017 inst_loadsl equ0x77777777

0018

0019; Instruction fordirect jump

0020 inst_jump equ0x888888

0021

0022 function1:

0023; Allocate space for the followingvariables:

0024; void * $ DL -- dynamic link chain

0025; void * $ sl -- Static link chain

0026; int $ stub [4] -- space for machine code forclosure

0027; int (* X) () -- A function pointer

0028; also sets up the dynamic link chain

0029

0030 $ DL equ-4

0031 $ SL equ-8

0032 $ stub equ-24

0033 function1.x equ-28

0034

0035 link FP, 28

0036

0037; set up the static link chain

0038

0039 move SL, $ SL (FP)

0040

0041; Create closure for function2

0042

0043 move # inst_loadsl, $ stub (FP)

0044 move FP, $ stub + 4 (FP)

0045 move # inst_jump, $ stub + 8 (FP)

0046 moveea function2, $ stub + 12 (FP)

0047

0048; X = function2

0049

0050 moveea stub (FP), function1.x (FP)

0051...

0052

0053; code using X

0054

0055 move function1.x (FP), r1

0056 call (R1)

0057...

0058

0059; finish up

0060

0061 unlinkfps

0062 return

0063

0064 function2:

0065; function entry, as above

0066

0067 link4

0068

0069 $ DL equ-4

0070 $ SL equ-8

0071

0072; set up the static link chain

0073; register SL was set up by the stub

0074; Code in function1

0075

0076 movesl, $ SL

0077...

0078

0079 unlinkfps

0080 return

The life cycle of the Code array $ stub is the same as that of the function1 activity record. This is reasonable, because this static chain, implicitly defined by $ stub, becomes invalid once function1 exits [6].

This is the basic idea. There are several possible optimizations during compilation, some of which we have mentioned. If function1 is defined throughout the local area, for example, it does not need to reserve space for the static chain in its activity records. If function2 does not reference any variables in the function1 activity record, the compiler can leave the activity record of function1 outside the static chain passed to function2.

If function2 does not reference any non-local or non-global variables, its definition can actually be moved to the full local area (except the scope of its name), and there is no need to generate a static chain domain in function1, you do not need to generate a code stub. If function2 is only used in function1 and no closure is passed, code that generates the stub code can be eliminated, because the compiler can generate code that loads SL inline before calling function2 [7].

3. How efficient is it?

To understand the efficiency of this solution, we must compare it with the alternative implementation. The two most direct implementations are to express a closure as a struct with two members, a code pointer and an environment pointer, or a pointer to such a struct.

The extra space required by our scheme only includes two machine commands contained in the stub code, and two commands in function1 are used to generate two machine commands in the stub code. Allocating and passing closures is as efficient as using a pointer to a struct.

However, the overhead of creating a closure is relatively unimportant. As we can see above, the compiler only needs to create a closure to pass a function pointer. In our experience, a passed function pointer is usually used repeatedly, so the overhead of calling a closure is far more important than the overhead of creating a closure.

Let's take a look at the sequence of commands executed in different implementations of the three closures. First, it is the sequence of commands directly represented as the closure of the struct [8]. This closure contains two machine words (pointers) at the offset X of the current active record [9]:

Move X (FR), SL

Move x + 4 (FR), r1

Call (R1)

In the case of a pointer pointing to a closure, the call sequence becomes somewhat complicated. Assume that PX is the offset of the pointer pointing to this closure in the current activity record [10]:

Move PX (FR), r1

Move (R1), SL

Move 4 (R1), r1

Call (R1)

For our suggestions, the command sequence we encounter is:

Move PX (FR), r1

Call (R1)

Move # staticlink, SL

JMP Function

We can see that when a function pointer pointing to a function that requires a static link pointer is called, The executed command sequence is not significantly different. The main difference in efficiency may come from the effect of cache and command prefetch. Some simple experiments show that the call/return pair that passes a static link pointer on the first 80386 requires about 1.5 times of a simple indirect or direct function call time. If we use our proposed sequence of commands, this number is increased to approximately 1.8 times.

4 portability Problems

To implement our proposal, a piece of code can generate a small piece of code at runtime and execute it. However, the generated code does not need to be placed on the stack.

In most modern computer architectures, such as the 68000 Series Microprocessor mot80, VAX [dig81b], and 80386 [int87], this is possible and simple. Even on a PDP-11 series processor with a separate instruction, data space (I and D space, stacks are usually mapped to the I, D space at the same time to allow the execution of commands on the stack (in the standard PDP-11 instruction call sequence, the mark command is executed outside the stack, refer to the PDP-11 processor manual [dig81a ]).

However, some architectures and/or operating systems prohibit programs from generating and executing code at runtime. We regard this restriction as arbitrary [11] and regard it as bad hardware or software design. Programming languages, such as forth, lisp, or smalltalk, can greatly benefit from quickly generating or modifying code at runtime.

We can use another technique, even in these architectures, to implement lexical closures. We pre-allocate this form of command sequence array in the instruction space:

Stub_n movelocation_n, r1

Move (R1), SL

Move location_n + 4, r1

JMP (R1)

We use this array as a stack for allocating and recycling the closure stubs. Saves the actual pointer to the Code and the closure static link in a corresponding array of the data space. These two new stacks are essentially the same as the runtime stacks. In particular, you must modify longjmp to correctly restore the two stack pointers of the stub stack and the position stack.

5. Future expansion

We can see that in some of the above examples, a function usually does not need to be explicitly named. This is especially true when we use an iterator that accepts function pointers as parameters. The example of the lisp and smalltalk code given above is an unnamed function in the design field. We recommend that you convert an unnamed function into a compound statement to a function pointer. For example, the value of the following expression is a function pointer or a closure (dependent on context ):

(INT (*) (int x) {returnx + 1 ;}

Alternatively, we can introduce a new keyword, unnamed, and write the same constructor:

(INT unnamed (int x) {return x + 1 ;})

We prefer the first form slightly, but the second form may be easier to parse and allow better syntax error checking and recovery.

We propose implementing closures to limit their lifecycles to the lifecycles recorded in their activities. To make the closure a data type at the same level as other data types in C ++, it is possible to allocate and recycle the closure. There is a good reason to provide closures that can be processed as data structures. Abelson and Sussman [as85] strongly support this feature. In scheme programming language rc86, they are generally used to build complex data types. However, classes and struct provide most of the functions of heap allocation closure.

6 conclusion

The definition and creation of nested functions with the minimum dynamic life cycle are an important part of many modern programming forms. Most modern programming languages provide it, and it can be added to C ++ (and c) programming languages without affecting execution efficiency or the meaning of the feature program. Therefore, we expect that the nested Function Definition and lexical closure will be added to the C ++ language definition. We are currently working on extending the gnu c [sta88a] [sta88b] and C ++ [tie88] compilers to provide nested function definitions and closures. There is a good, free compiler with a source code-level debugger available, will encourage more people to use this feature.

Thank you

I would like to thank Richard M. Stallman, Robert S. Thau, and many people who have provided useful comments on this proposal and paper.

References

[As85] Harold Abelson and geraldjay Sussman. Structure and interpretation of computer programs. mitpress, 1985.

[Au79] ALFRED v. Aho and je_reyd. Ullman. Principles of compiler design. Addison-Wesley, 1979.

[Dig81a] digital equipmentcorporation. PDP-11 processor handbook, 1981.

[Dig81b] digital equipmentcorporation. VAX architecture handbook, 1981.

[Gr83] Adele Goldberg and David Robson. Smalltalk-80: The Language and Its Implementation. Addison-Wesley, 1983.

[Int87] Intel Corporation. 801_programmer's reference manual, 1987.

[Mot80] Motorola semiconductorsproducts, Inc. mc68000 16-bit microprocessor user's manual, 1980.

[Rc86] Jonathan Rees and William amclinger. The revised3 report on the ALGOrithmic Language Scheme. Technical Report 848a, MIT arti_cial Intelligence Laboratory, September 1986.

[Sta88a] Richard M. Stallman. internalsof gnu cc, limit l 1988.

[Sta88b] Richard M. Stallman. The gnudebugger for gnu c ++ Free Software, limit l 1988.

[Ste84] guy l. Steele Jr. commonlisp: The Language. digitalpress, 1984.

[Tie88] Michael D. tiann. User's Guide to gnu c ++, may1988.

[Wir77] Niklaus Wirth. compilerbau. B. G. Teubner, Stuttgart, 1977.

[Wir79] Niklaus Wirth. algorithmen unddatenstrukturen. B. G. Teubner, Stuttgart, 1979.

[1] Author's address: MIT artificialintelligence laboratory, room 711,545 Technology Square, Cambridge,

Ma 02139, us. The author is funded by an association from the Fairchild Foundation.

[2] Unless sift declares or defines another identifier v

[3] We must create a name for the function count, which is still annoying. In the subsequent sections, we recommend that you define the syntax for anonymous functions. Here we use a named version to avoid Syntax problems.

[4] The base class of a collection is the type of the collection element.

[5] assume that all data and commands are 32-bit wide. The temporary label starts with a "$. The instruction moveea moves the actual address of its first operand to a location. The command link fills in the "$ DL" Field of the activity record.

[6] We will discuss possible extensions later, and extend the lifetime of a specific static chain beyond the dynamic lifetime recorded by the component activity.

[7] In fact, a restricted form of nesting, where we are not allowed to obtain the address defined in the internal lexical hierarchy function, can be implemented without the need to extend the compiler to generate code stubs.

[8] These are not assembly language program fragments, but traces of assembly language commands executed during the call of a closure.

[9] R1 is a general register.

[10] in the following example, the content of location (R1) and 4 (R1) can be viewed as cached.

[11] Such a system usually provides an operating system call to move data to the instruction space, for example, to assist the loader; however, the overhead of an operating system call is too high for building a closure.