Develop a personal lightweight Database Engine

Develop Personal lightweight Database Engine Delphi/Windows SDK/API
Http://www.delphi2007.net/DelphiDB/html/delphi_20061214173952300.html
I plan to develop a lightweight database engine for some special occasions.
At present, we have used the hash-based method to make a simple one. keyword query time of 0.5 records, the slowest less than seconds. I did this for the first time and had no experience. I hope to find some like-minded people to discuss and learn together.

Ah
I want to do it too.
Comrade (not that Comrade)
I have another question: How can I reduce this loss when dealing with external storage? Is there any good idea for the landlord?

Is it just a hash?
If the target is only 100 million records, how can LZ retrieve 4,294,967,296 (Cardinal) billion records? In my opinion, if you do not need to do this, you can get the maximum search range of 32 bits.

How is the interpreter prepared?
Is there any plan?
Can you tell me more?

I understand a simple structure (record) that supports retrieval, addition, modification, and deletion. The number of records should be around 1000 W-> 0.1 billion, other functions are not required. you can load/Save the structure at most.

If you are interested, you can get it done by yourself.

Yes. Try to reduce the number of disk reads and writes.
Currently, the table uses the record file format, and maintains a delete linked list. only Delete the deleted records and add them to the deleted linked list. the newly added record first selects the header node for deleting the linked list, which is similar to the stack structure.
Hash buckets are used to manage records. This method is most suitable for storing records on persistent storage devices such as disks.

At present, there is no problem in 0.1 billion W, and the bigger one is not tested yet. It is estimated that the problem is not big by, and it will be uncertain to go up again.

No need to parse functions such as SQL
Because it is only used for some occasions, because it requires a large amount of data, but does not want to rely on the database, of course, the speed should also be considered, so you can get a lightweight structure processing. if you want to write a database, what are the parsing SQL statements, and a bunch of functions, you will not be interested. You can directly look at the MySQL source code.

The interpreter's problem is the next step. The underlying functions are fully implemented first. The interpreter's problem is being considered.

In general. currently, these popular databases actually use less than 40% features. in addition, if you want to implement some confidentiality measures, although the current encryption function is provided, it is open after all. security is not as strong as defined by yourself.

My current problem is that it is a bit similar to access. as we all know, access database files will become larger and larger. unless you compress it once. this is because the disk should be read and written as much as possible to protect the disk.AlgorithmCause.

Currently, only deleting tags are available for deleted nodes. there is no problem with increasing the data volume, that is, if the data volume is greatly deleted, there will be a lot of storage space. unless it is the same as access, it is compressed once.

This should not be a problem, even if the problem is not solved, it is not big. At most, like the OS, it can be sorted out regularly for the file.
If you want to get it later, you can do it like this, but it seems that a pile of fragments will be generated:
Each piece (record) is stored in the file, and the length and flag are added. It is estimated that you have it. when adding a block, check the deleted block. If the added block is smaller than the deleted block, the added block is written to the deleted block and the remaining space is created and deleted, the size of the deleted block is reduced. however, it is possible that the block to be deleted will become more and more as fragments, as if the increased volume is relatively slow.

It is good to study the technology and get good experience.

In actual application, the most common and simplest method is used.

SQL query is not supported. Multi-table join cannot be checked.

Embedded RDBMS with Firebird.

Does FB support embedded databases? FB is the best lightweight dbms I have ever seen

This should not be a problem, even if the problem is not solved, it is not big. At most, like the OS, it can be sorted out regularly for the file.
If you want to get it later, you can do it like this, but it seems that a pile of fragments will be generated:
Each piece (record) is stored in the file, and the length and flag are added. It is estimated that you have it. when adding a block, check the deleted block. If the added block is smaller than the deleted block, the added block is written to the deleted block and the remaining space is created and deleted, the size of the deleted block is reduced. however, it is possible that the block to be deleted will become more and more as fragments, as if the increased volume is relatively slow.
Bytes ------------------------------------------------------------------------------------

Yes, I have basically implemented something similar to what you said. It seems that this fragment is inevitable. Fortunately, the hash method I used is to keep the recent increase as close as possible, and try to conform to the external read/write method.

As for SQL, it is not the most urgent at present. Multi-table join can also be implemented, but it is only troublesome.
What is the implementation method of Firebird RDBMS?

It is good to study the technology and get good experience.

In actual application, the most common and simplest method is used.

-----------------------------------------------------------
I plan to use it in practice. Because some data does not need to be understood by others. Custom databases are a good method.

Firebird's embedded RDBMS is not connected to the exe. It is a DLL and does not need to be configured.

Firebird's embedded RDBMS is not connected to the exe. It is a DLL and does not need to be configured.

---------------------------------------------------------------------

What is its engine?

You don't need to know the data, but you don't need to develop a database. There are too many encryption methods. Your head will be overwhelmed, and project maintenance is a big problem.

Isn't it necessary to develop an operating system for data with higher confidentiality requirements.

Technical Research can be avoided in actual projects, resulting in a lot of time on the top, and less time investment in business and design.

You don't need to know the data, but you don't need to develop a database. There are too many encryption methods. Your head will be overwhelmed, and project maintenance is a big problem.

Isn't it necessary to develop an operating system for data with higher confidentiality requirements.

Technical Research can be avoided in actual projects, resulting in a lot of time on the top, and less time investment in business and design.

------------------------------------------------------------------

This database engine varies from person to person. Of course, the time and efficiency issues are already under consideration.

It is a lightweight engine, and there is not much maintenance time. Now that you have done it, you have already considered the problem.

Therefore, I am not talking about how to do this, but about how to do it. I hope to give more technical comments. Thank you.

You can refer to absolute, dbisam, nexusdb, and so on. They are all written in Delphi, and nexusdb is better than oracle.

The advantage of nexusdb is that it can be directly connected to the EXE, but it is easy to use without the embedded version of Firebird.

Firebird is the best.

Almost all database functions
Table, foreign key, view, process, trigger, UDF

Easy to use and practical

A project management system I made for the company also uses it. It is also possible. It is small in size and has powerful functions.
Now we have the latest version 2.0.

You can refer to absolute, dbisam, nexusdb, and so on. They are all written in Delphi, and nexusdb is better than oracle.
-------------------------------------------------------

Thanks for the information. I don't know if they are open source? Where is the source code?

I don't know the Beijing monkey.

Mak

Firebird is open-source