Direct access to MySQL's bdb storage engine

We know that BDB is an embedded database with a higher access efficiency than mysql, but it is not convenient to manage mysql. synchronization is also a problem in distributed applications.

Mysql supports the use of BDB as the storage engine (which is not directly supported after 5.1, because BDB was acquired by oracle). Can mysql be used for data management and synchronization, what about direct access to BDB files through mysql in actual applications?

 

I tried:

First, obtain the mysql 5.0 source code and configure it according to the following parameters:

./Configure -- with-berkeley-db =./bdb 

Start mysql after compilation, and add engine = bdb when creating the table

The *. db file is displayed in the mysql data directory.

Run the file command to check whether the file format is Berkeley DB (Btree, version 9, native byte-order)

If you use the db_dump command to view it, you can find that it is different from a common db file:

1. in general, there is only one database in a single database file, while mysql generates two databases, main and status (if an index other than the primary key is added during table creation, an additional index library will be generated)

2. the db file generated by mysql uses the primary index of the table as the key and the whole row of the table as the value, especially for the column types of char and varchar, during mysql storage, the character length is added before the string. For example, "abc" is stored as "/x03abc"

 

For the first point, you can specify the database name during dbopen. For the db file generated by mysql, you can specify the database name as "main.

For the second point, you need to add a string length before the key to be queried during the query, find the result, and then strip the length information in the result.

 

There are still some problems, that is, the definition of the table structure is not in the db file, but in the. frm file generated by mysql. So first, you don't know how to split fields. Second, you don't know whether the table length is 1 byte or 2 byte or more. However, this can be solved through the agreed method. The two parties have agreed on a fixed table format, which can be used as described below.

 

But the more difficult problem arises. After making these changes, you can find the results when querying some key values, and some key values won't be able to find the results.

 

It turns out that the problem lies in the length of the byte before the string. In general, bdb applications use strings as keys, but a byte is added before strings during mysql storage, which leads to the problem of user-defined comparison functions.

As mentioned above, the file format uses the Btree. When a key is inserted, the size of the key will be compared with that of the root node, and the decision on which subtree to go is determined. However, the standard comparison size is byte-by-byte comparison, and after adding the Length Byte before the string, the string equivalent to the same length will be transferred to a subtree, mysql may think this affects the retrieval efficiency. Therefore, mysql removes the Length Byte before comparing strings through a custom comparison function. The standard DBD interface is compared according to the standard method and cannot be found in many cases.

 

No proper solution yet. Please try ......

--------------------------------------------

Solved the problem. Before bdb opens the file, set a custom comparison function (DB-> set_bt_compare) to skip the first byte in the comparison function, then call strncmp to compare the rest.

 

So far, the direct access to the mysql storage engine has been successfully solved.