Explore Erlang abstract form-dynamic module generation and Modification

Document directory

• Module
• Module Summary
• Description
• Exports
• Module
• Module Summary
• Description
• Exports

Reprinted: http://agileprogrammer.blogspot.com/2006/08/erlang-abstract-form-module.html

In the previous article, we briefly described the basic components of abstract form. Now let's take a look at how to use abstract form to dynamically generate and modify a module.

In the first article, we explored Erlang abstract form -- generation and acquisition. As we said, there are two methods to obtain abstract form: one is to read debug_info from the beam file, another method is to directly parse the source code.

Provide source code text

The most useful function for modifying a module is to add new functions. We can obtain the abstract form of the existing module from the beam file. However, if you need to dynamically Add a method, the most likely thing to come up with is to provide the source code text of the function.

The source code parsing usually requires two tools: a scanner and a parser. The basic scanner provided by Erlang is erl_scan and the parser is erl_parse. Let's take a look at their documents.

Moduleerl_scan module SummaryThe Erlang token parameter description

This module contains functions for tokenizing characters into Erlang tokens.

Exports

string(CharList,StartLine]) -> {ok, Tokens, EndLine} | Error
string(CharList) -> {ok, Tokens, EndLine} | Error

Types:

CharList = string()
StartLine = EndLine = Line = integer()
Tokens = [{atom(),Line}|{atom(),Line,term()}]
Error = {error, ErrorInfo, EndLine}

Takes the list of CharactersCharListAnd tries to scan (tokenize) them. Returns
{ok, Tokens, EndLine}, WhereTokensAre the Erlang tokens from
CharList.EndLineIs the last line where a token was found.

StartLineIndicates the initial line when scanning starts.
string/1Is equivalentstring(CharList,1).

{error, ErrorInfo, EndLine}Is returned if an error occurs.
EndLineIndicates where the error occurred.

Erl_scan: string method to scan string text. If no error occurs, all tokens are returned in the result tuple. Otherwise, the wrong row number is returned.

Eshell V5.5  (abort with ^G)1> c(simplest, [debug_info]).{ok,simplest}2> {ok, Tokens, EndLine} = erl_scan:string("test() -> ok.").{ok,[{atom,1,test},{'(',1},{')',1},{'->',1},{atom,1,ok},{dot,1}],1}3>

The simplest function test, erl_scan: returns tokens from string scan results, which contains five tokens. The names of the atom-type functions are test, left brace, right brace, end of the function header->, atom-type atom OK, and dot.

With this tokens, we can use erl_parse to parse and generate an abstract form. The erl_parse document says:

Moduleerl_parse module SummaryThe Erlang parser description

This module is the basic Erlang parser which converts tokens into the abstract form of either forms (I. E ., top-level constructs), expressions, or terms. the abstract format is described in the ERTs user's guide. note that a token list must end with
DotToken in order to be acceptable to the parse functions (see erl_scan ).

Exports

parse_form(Tokens) -> {ok, AbsForm} | {error, ErrorInfo}

Types:

Tokens = [Token]
Token = {Tag,Line} | {Tag,Line,term()}
Tag = atom()
AbsForm = term()
ErrorInfo = see section Error Information below.

This function parsesTokensAs if it were a form. It returns:

{ok, AbsForm}

The parsing was successful. AbsFormIs the abstract form of the parsed form.

{error, ErrorInfo}

An error occurred.

Erl_parse can parse many token types, including expressions, terms, and forms. What we need is to completely parse the form function parse_form. Similarly, if the parsing is successful, the returned tuple will contain the abstract form represented by tokens. Otherwise, a syntax error message will be returned.


3> erl_parse:parse_form(Tokens).
{ok,{function,1,test,0,[{clause,1,[],[],[{atom,1,ok}]}]}}

We can see that the returned result contains the Abtract form of the function.
Compile abstract form

With abstract form, we can compile and load it.

The compile module documentation says:

forms(Forms)

Is the sameforms(File, [verbose,report_errors,report_warnings]).

forms(Forms, Options) -> CompRet

Types:

Forms = [Form]
CompRet = BinRet | ErrRet
BinRet = {ok,ModuleName,BinaryOrCode} | {ok,ModuleName,BinaryOrCode,Warnings}
BinaryOrCode = binary() | term() ErrRet = error | {error,Errors,Warnings}

Analogousfile/1, But takes a list of forms (in the Erlang abstract format representation) as first argument. The option
binaryIs implicit; I. E ., no object code file is produced. options that wowould ordinarily produce a listing file, such as 'E', will instead cause the internal format for that compiler pass (an Erlang term; usually not a binary) to be returned instead
Of a binary.

Compile: forms this function takes the form to be compiled as a parameter and compiles it into binary object code data that can be executed by virtual machines. It is basically the same as compile: file, but only provides the form parameter, while compile: file is the file name to be compiled.

In fact, the compile: file method is always used. In ERL shell, input C (file, options) to compile the file:

c(File) -> {ok, Module} | error
c(File, Options) -> {ok, Module} | error

Types:

File = Filename | Module
Filename = string() | atom()
Options = [Opt] -- see compile:file/2
Module = atom()

c/1,2Compiles and then purges and loads the code for a file.
OptionsDefaults to []. Compilation is equivalent:

compile:file(File, Options ++ [report_errors, report_warnings])

Note that purging the code means that any processes lingering in old code for the module are killed without warning. See
code/3For more information.

C calls the file method of compile to compile the. erl file, removes the existing code from the memory, and loads the new code.

To compile abstract form, we must provide the complete form of the entire module. Therefore, we need to provide the module and export attributes.

1> c(simplest,[debug_info]).{ok,simplest}2> {ok, Tokens, EndLine} = erl_scan:string("test() -> ok.").{ok,[{atom,1,test},{'(',1},{')',1},{'->',1},{atom,1,ok},{dot,1}],1}3> {ok, Forms} = erl_parse:parse_form(Tokens).{ok,{function,1,test,0,[{clause,1,[],[],[{atom,1,ok}]}]}}4>4> NewForms = [{attribute, 1, module, simplest},{attribute, 2, export, [{test,0}]}, Forms].[{attribute,1,module,simplest},{attribute,2,export,[{test,0}]},{function,1,test,0,[{clause,1,[],[],[{atom,1,ok}]}]}]5> compile:forms(NewForms).{ok,simplest,<<70,79,82,49,0,0,1,188,66,69,65,77,65,116,111,109,0,0,0,55,0,0,0,6,7,...>>}

Load the newly compiled object code

Another advantage of reading this document is that we know how to load it after compilation. The Code module defines these functions:

purge(Module) -> true | false

Types:

Module = atom()

Purges the codeModule, That is, removes Code marked as old. If some processes still linger in the old Code, these processes are killed before the code is removed.

ReturnstrueIf successful and any process needed to be killed, otherwise
false.

The purge function removes existing code and marks it as an old version. If any process is using the old Code, these processes will be killed. Note that although purge is always successful, its return value returns true only when any process needs to be killed.

load_binary(Module, Filename, Binary) -> {module, Module} | {error, What}

Types:

Module = atom()
Filename = string()
What = sticky_directory | badarg | term()

This function can be used to load object code on remote Erlang nodes. it can also be used to load object code where the file name and module name differ. this, however, is a very unusual situation and not recommended. the Parameter
BinaryMust contain object codeModule.FilenameIs only used by the Code server to keep a record of from which file the object code
ModuleComes. Accordingly,FilenameIs not opened and read by the Code server.

Returns{module, Module}If successful, or{error, sticky_directory}If the object code resides in a sticky directory, or
{error, badarg}If any argument is invalid. Also if the loading fails, an error tuple is returned. See

Erlang: load_module/2 for possible valuesWhat.

Load_binary loads the compiled object code so that the module can be used by the program. If the object code exists in the sticky directory, it may not be replaced. The sticky directory is the runtime system of Erlang, including kernel, stdlib, and compiler. To ensure normal operation of Erlang, these directories are protected by default and are considered to be sticky.

Combination

Using the content we discussed earlier, we can perform a complete test.
Assume that the following program simplest. erl is available:

-module(simplest).-export([foo/0]).foo() -> io:format("foo~n").

We use ERL for step-by-step testing.

1> c(simplest,[debug_info]).{ok,simplest}2> simplest:foo().foook3> simplest:test().=ERROR REPORT==== 18-Aug-2006::15:06:17 ===Error in process <0.32.0> with exit value: {undef,[{simplest,test,[]},{erl_eval,do_apply,5},{shell,exprs,6},{shell,eval_loop,3}]}** exited: {undef,[{simplest,test,[]},               {erl_eval,do_apply,5},               {shell,exprs,6},               {shell,eval_loop,3}]} **4> {ok, Tokens, EndLine} = erl_scan:string("test() -> ok.").{ok,[{atom,1,test},{'(',1},{')',1},{'->',1},{atom,1,ok},{dot,1}],1}5> {ok, Forms} = erl_parse:parse_form(Tokens).{ok,{function,1,test,0,[{clause,1,[],[],[{atom,1,ok}]}]}}6> NewForms = [{attribute, 1, module, simplest},{attribute, 2, export, [{test,0}]}, Forms].[{attribute,1,module,simplest},{attribute,2,export,[{test,0}]},{function,1,test,0,[{clause,1,[],[],[{atom,1,ok}]}]}]7> {ok,simplest,Binary} = compile:forms(NewForms).{ok,simplest,<<70,79,82,49,0,0,1,188,66,69,65,77,65,116,111,109,0,0,0,56,0,0,0,6,8,...>>}8> code:purge(simplest).false9> code:load_binary(simplest,"simplest.erl",Binary).{module,simplest}10>  simplest:foo().=ERROR REPORT==== 18-Aug-2006::15:08:13 ===Error in process <0.40.0> with exit value: {undef,[{simplest,foo,[]},{erl_eval,do_apply,5},{shell,exprs,6},{shell,eval_loop,3}]}** exited: {undef,[{simplest,foo,[]},               {erl_eval,do_apply,5},               {shell,exprs,6},               {shell,eval_loop,3}]} **11> simplest:test().ok12>

The result is that an export test/0 function is added, and the original Foo/0 function does not exist.

Next step

Of course, our goal is not to completely overwrite the original module, but to add, delete, and replace the original function. Therefore, we need to read and save the old form through the beam_lib: chunks method, add and replace the form of the new function according to different operations, and compile it together.
This is what smerl does.