Analysis on the working principle of the Erlang database ETS

Reprinted: http://mryufeng.iteye.com/blog/113856

ETS is Erlang term strorage meaning document see erl5.5.5/lib/stdlib-1.14.5/doc/html/index.html. This is a core function in beam. ETS, dets, and mnesia constitute the Erlang database. Note that mnesia has no storage mechanism.
Its storage is ETS and dets.

You can use ETS: I ().
11 code set 254 11393 code_server
12 code_names set 48 5323 code_server
13 shell_records ordered_set 0 72 <0.25.0>
Ac_tab set 6 853 application_controller
File_io_servers set 0 279 file_server_2
Global_locks set 0 279 global _name_server
Global_names set 0 279 global_name_server
Global_names_ext set 0 279 global_name_server
Global_pid_ids bag 0 279 global_name_server
Global_pid_names bag 0 279 global_name_server
Inet_cache bag 0 279 inet_db
Inet_db set 21 528 inet_db
Inet_hosts set 1 310 inet_db

That is to say, the implementation of Erlang's kernel and stdlib libraries is dependent on this ets.

This module is an interface to the Erlang built-in term storage BIFS. ETS. erl itself is only an encapsulated module used to check parameters and so on. The actual work is done by BIF, so the efficiency is very good.
See otp_src_R11B-5 \ ERTs \ emulator \ Beam \ BIF. Tab

#
# BIFS in ETS module.
#

Bif ets: All/0
BIF 'erl. Lang. ETS ': All/0 ebif_ets_all_0
Bif ets: New/2
BIF 'erl. Lang. ETS ': New/2 ebif_ets_new_2
...
BIF 'erl. Lang. ETS ': match/1 ebif_ets_match_1
Bif ets: match/2
BIF 'erl. Lang. ETS ': match/2 ebif_ets_match_2
Bif ets: match/3

In emulator, erl_db.c (Interface) erl_db_hash.c (hash implementation) erl_db_tree.c (tree implementation) erl_db_util.c (match virtual machine, etc.) is related to ETS implementation) nearly 20,000 lines of code are complicated to implement. It is said that the next version will use the jarray Algorithm for higher efficiency.

The implementation of ETS is not multi-thread secure, and data is not involved in GC. Pay attention to it when using it.

When we want to traverse ETS, we can use first/next to traverse it, or use foldr foldl to see ETS as list. However, in this way, the efficiency problem is that the data needs to be moved from the ERTs to the process. When the ETS is large, the efficiency is low.

At this time, ETS: select match matchspec will help you. ETS internally implements a virtual machine to compile matchspec into opcode and then copy the required data to the process during Eval, greatly reducing the data volume. this method is similar to SQLite.

See db_match_set_compile to compile matchspec into opcode.
For details about the opcode operation of db_prog_match, see the code.

This is not enough. Considering that matchspec is hard to write, ETS provides a function fun2ms to convert standard Erlang fun into matchspec. For more information, see ms_transform.

With these functions, it is very convenient to use ets.