The storage mechanism of Erlang's massive data: ETS and Dets

1. ETS and Dets Introduction: ETS (Erlang term Storage) and Dets (Dist ETS) are system modules that Erlang uses to efficiently store large numbers of Erlang data entries. ETS vs. Dets: Same: Both ETS and Dets provide large "key-value" search tables. Different: ETS resides in memory and Dets resides on the hard disk. The ETS storage is temporary and the data store in the dets is persistent. ETS is very efficient, and in ETS, no matter how much data you store, the query speed is irrelevant (given a logarithmic relationship), provided that you have large enough memory. Dets is much less efficient, but saves more memory than ETS. ETS tables and Dets tables can be shared by multiple processes, enabling efficient data exchange between processes through these two modules. ETS is not implemented by Erlang itself, but by the underlying runtime system. ETS is a lot different than a typical Erlang object. For example, ETS will not be garbage collected. ETS or Dets is essentially a series of Erlang tuples.
2. Basic operation of the table: 1. Create a new table or open an existing table: Ets:new () or Dets:open_file () 2. Insert one or more tuples into the table: Insert (TABLENAME,X). X is a list of tuples or tuples. 3. Finding a tuple in a table: lookup (Tablename,key) returns a list of tuples matching keys4. Release table: Dets:close (TableId) or Ets:delete (TableId)
3. Types of Tables: (4 kinds) set ordered set bag duplicate bag type is set table, requires all tuples key values are different, ordered set table, key value size sorting. A table of type bag that allows multiple tuples to have the same key value, but does not allow the same tuple. A table of type duplicate bag that allows for multiple identical tuples.
4. ETS Example: Ets_test.erl-module (Ets_test). -export ([start/0]).
Start ()     Lists:foreach (fun test_ets/1                         [set,ordered_set,bag,duplicate_bag]) .      Test_ets (Mode)     TableId = Ets:new (Test_ets,[public,named_table,mode]),     Ets:insert (tableid,{a,1}),     ETS: Insert (tableid,{b,2}),     Ets:insert (tableid,{a,1}),     Ets:insert (tableid,{a,3}),     The named_table mode is used when the ETS table is created, so the ETS table can be used directly using the ETS table name     list = Ets:tab2list (test_ets),       & nbsp   Io:format ("~-13w =>~p~n", [Mode,list]),     Ets:delete (test_ets). Running Result: 1> Ets_test:start (). Set           =>[{b,2},{a,3}]            NOTE: Set each key value is allowed to occur only once, so {a,1} is eventually overwritten by {a,3}. Ordered_set   =>[{a,3},{b,2}]     bag           =>[{b,2},{a,1}, {a,3}] Duplicate_Bag =>[{b,2},{a,1},{a,1},{a,3}]
5. ETS Efficiency Considerations: 1. Internally, the ETS table is represented by a hash column (except that the ordered set is represented by a balanced binary tree) so set is a bit of a waste of space and ordered set a bit of a waste of time. The time it takes for the set table to insert data is constant, and the time it takes to insert the order set is a logarithmic relationship to the size of the table. Bag is more expensive to use than duplicate bag because the key value is compared for each insertion. 2. The ETS table is separated from the normal process storage space and is not garbage collected. 3. The ETS table is part of the process that created him, and when the process dies or calls Ets:delete, the table is deleted.     4. Sending a large number of binary data messages between processes, or inserting tuples containing binary data into an ETS table, is inexpensive. As a result, it is an efficient way to programmatically represent strings or chunks of untyped data with binary data.
6. Creation of ETS tables: ets:new (name,[option])Name is the name of the table and is an atom atom. option is the list of options, with the following values: Set | Ordered_set | Bag | Duplicate_bag Private | protected | Public named_table indicates that you can use name to manipulate table {Keypos,k} When opening an ETS table with no options, the default option is [ets,proteced,{keypos,1}]