Common Embedded Database __ Database

Overview of embedded database used in embedded database

Berkeley DB Overview

Http://www.cppblog.com/michaelgao/archive/2008/06/24/54435.html
Berkeley DB is a library of open source embedded databases (database library) developed by the US Sleepycat Software Corporation, which provides scalable, high-performance, transaction-protected data management services for applications. Berkeley DB provides a concise set of function call API interfaces for data access and management.

It is a classic c-library-mode toolkit that provides programmers with a wide range of function sets that provide industrial-level strength for application developers
Database service. The main features are as follows:

Embedded (Embedded): It is directly linked to the application and runs in the same address space as the application, so whether it is different on the network
Database operations do not require interprocess communication between computers or between different processes on the same computer.

Berkeley DB provides API interfaces for a variety of programming languages, including C, C + +, Java, Perl, TCL, Python, and PHP, all database operations
Occurs inside the library. Multiple processes, or multiple threads of the same process, can use the database at the same time, as if they were used separately, the underlying services such as lock,
transaction logs, shared buffer management, memory management, and so on are performed transparently by the library.

Lightweight and flexible (portable): It can run on almost all UNIX and Linux systems and their variants, Windows operating systems, and a variety of embedded real
Under the operating system. It works on both 32-and 64-bit systems and has been used by many high-end Internet servers, desktops, PDAs, set-top boxes, networks
Switches and other application areas. Once Berkeley DB is linked to an application, the end user generally does not feel that there is a database
System exists.

Scalable (Scalable): This is reflected in many ways. The Database library itself is very compact (less than 300KB of text space), but it
Ability to manage databases up to 256TB in size. It supports high concurrency, and thousands of users can manipulate the same database at the same time. Berkeley DB can be small enough
's footprint runs on tightly constrained embedded systems, and can also consume several gigabytes of memory and several terabytes of disk space on high-end servers.

Berkeley db is preferable to relational and object-oriented databases in embedded applications for the following two reasons:

(1) Database operations do not require interprocess communication because the database library runs in the same address space as the application. In a machine's
The overhead of process communication between different processes or between machines in a network is much larger than that of function calls.

(2) Because Berkeley DB uses a set of API interfaces for all operations, there is no need to parse a query language or generate an execution plan.
Greatly improve the operation efficiency.

BERKELEYDB System Structure

Berkeley DB is comprised of five major subsystems. Includes: Access management subsystem, memory pool management subsystem, transaction subsystem, lock subsystem and log subsystem.
The Access management subsystem acts as the internal core component of the Berkeley DB database process package, while the other subsystems exist outside the Berkeley DB database process package.

Each subsystem supports different levels of application.

1. Data Access Subsystem
The data access (Access Methods) subsystem provides multiple support for creating and accessing database files. Berkeley DB provides the following four methods of file storage:
Hash files, b-trees, fixed-length records (queues), and variable-length records (simple storage based on record numbers) from which an application can select the most appropriate file organization structure.
Programmers can use any structure when creating tables, and can mix files of different storage types in the same application.

In the absence of transaction management, modules in this subsystem can be used alone to provide fast and efficient data access services for applications.
The data access subsystem is suitable for applications that require no transaction only for fast format file access.

2. Memory Pool Management Subsystem
The memory pool (Memory Pool) subsystem manages the shared buffers used by Berkeley DB effectively. It allows simultaneous access to multiple processes of the database or
Multiple threads of a process share a cache that is responsible for writing the modified page back to the file and allocating memory space for the newly paged page.

It can also be used independently of the Berkeley DB system to allocate memory space for its own files and pages.
The Memory pool management subsystem is suitable for applications that require flexible, page-oriented, buffered, shared file access.

3. Transaction subsystem
The transaction (Transaction) subsystem provides transaction management functionality for Berkeley DB. It allows you to consider a set of modifications to a database as an atomic unit,
This set of operations is either done entirely or none at all. By default, the system will provide strict acid transaction properties, but the application can choose not to use the system
Isolation guarantee. The subsystem uses the two-segment lock technique and the first-write logging strategy to ensure the correctness and consistency of the database data.

It can also be used by the application alone to protect its own data updates. The transaction subsystem is suitable for applications that require modification of transaction-guaranteed data.

4. Lock Subsystem
The lock (locking) subsystem provides a lock mechanism for Berkeley DB, which provides multiuser reading and shared control for the single user to modify the same object.
The data access subsystem can use this subsystem to gain read and write access to the page or record; The transaction subsystem uses the lock mechanism to realize concurrency control of multiple transactions.

The subsystem can also be used separately by applications. The lock subsystem is suitable for a flexible, fast, and configurable lock manager.

5. Log Subsystem
The log (Logging) subsystem uses a first-write-log strategy to support transactional subsystems for data recovery to ensure data consistency.
It is not likely to be used by applications alone, but as a calling module for the transaction subsystem.

The above sections constitute the entire Berkeley DB database system. The relationship between the parts is shown in the following illustration:

In this model, the application directly calls the data access subsystem and the transaction management subsystem, which in turn calls the lower memory management subsystem,
Lock subsystem and log subsystem.

Because several subsystems are relatively independent, the application can initially specify which data management services will be used. Can be used all,
You can also use only a fraction of them. For example, if an application needs to support multi-user concurrency, but does not require transaction management, it can
Use only the lock subsystem instead of the transaction. Some applications may require a fast, single-user, and no transaction management capability for a B-tree storage structure, then the application
The lock subsystem and transaction child systems can be invalidated, which reduces overhead.

Overview of BerkeleyDB Storage Features

The logical organizational unit of data managed by Berkeley DB is a number of independent or relational databases (database),
Each database consists of several records, all of which are represented in the form of (Key,value).

If a group of related (key,value) pairs is also considered a table, then each database is allowed to hold only one table,
This is different from the general relational database. In fact, the reference to "database" in Berkeley DB is equivalent to a table in a general relational database system;
and "Key/data" corresponds to rows in the relational database system (rows); Berkeley DB does not provide direct access to the columns in the relational database.
Instead, the fields (columns) are encapsulated in the data item in the "Key/data" pair by a practical application.

In physical organization, each database can be created by an application based on its data characteristics to choose a suitable storage structure.
The four file storage structures that are available for selection are hash files, B-trees, fixed-length records (queues), and variable-length records (simple storage based on record numbers).

A physical file can only hold a single database, or it can hold several related or unrelated databases, and these databases
A database that is organized in queues can be stored separately in one file and cannot be mixed with other storage types, depending on the organization in any way except queues.

In addition to being constrained by the maximum file length and storage space, a file can theoretically store any number of databases.
So a system that locates a database typically requires two parameters-"filename" and "Database name", which is also Berkeley DB different from
General relational database of the place.

The Berkeley DB Storage System provides applications with a series of interface functions for managing and manipulating databases. These include:
(1) Database creation, open, close, delete, rename, etc., as well as the data retrieval and additions and deletions to the operation;
(2) Provide some additional functions, such as reading database status information, reading information about the file, reading the information of the database environment,
Empty the contents of the database, database synchronization backup, version upgrade, prompt error messages, etc.
(3) The system also provides a cursor mechanism for accessing and accessing group data, as well as associating and equivalent connection operations on two or more related databases;
(4) The system also gives some interface functions to optimize the configuration of the access policy, for example, the application can set its own sort comparison function of B-tree,
The minimum number of keys in each page, the fill factor for the hash bucket, the hash function, the maximum length of the hash table, the maximum length of the queue, the byte order in which the database is stored,
The size of the underlying storage page, the memory allocation function, the size of the cache, the size and padding of the fixed-length record, the separator used to change the length of the record, and so on.

Firebird DB Introduction

Firebird Embeded Database. As a single file small database, Firebird has many attractive features, such as supporting transactions, supporting stored procedures, triggers, and so on, and the embeded version of Firebird only needs to copy two files in. NET Development: A Fbembed.dll (Unmanaged but not registered dynamic-link libraries) and a ado.net Data Provider FirebirdSql.Data.Firebird.dll. These features are ideal for situations where you need to store some data on the client, but do not want to install the database (such as MSDE) software.

It is said that in foreign countries, the need to use the client database, there are about 30% of developers choose Access, 30% of developers choose MSDE 2000, 30% of the developers choose Embedded Firebird, the remaining 10% select other small databases, such as SQLite, MySQL and so on.

The Access,msde 2000,embedded Firebird,sqlite mentioned above are all available for free redistribution of the database. The obvious disadvantage of MSDE 2000 is that it requires installation, and the biggest advantage is that it is consistent with the server-side SQL Server programming model and facilitates development. The obvious disadvantage of access is that it is less functional and does not support common functions such as transactions, with the greatest advantage of simplicity, the familiarity of most developers, and the ease of deployment. SQLite support transactions, but also a single file database, relatively insufficient is. NET Data Provider is not very mature. Firebird is also: Single file, simple deployment without installation (only XCOPY two files), support transactions, stored procedures, triggers,. NET Data provider more stable and mature advantages.

The Firebird itself is divided into superserver and embedded versions, which can only be accessed natively and do not accept TCP connections. For developers, switching from the embedded database to SuperServer requires only changing the ServerType value in the database connection string.

SQLite DB

The source code for SQLite is C, which is completely open. The first alpha version of SQLite was born in May 2000. This May, SQLite again ushered in a new milestone of a solite 3.

SQLite has the following features: support for acid transactions; 0 Configuration One no need to install and manage configuration; a complete database stored in a single disk file; database files can be shared freely between machines in different byte order; Support database size to 2TB; small enough, all source code roughly 30,000 lines C 250KB; it is faster to manipulate data than most databases that are currently popular; provides support for transactional functions and concurrent processing, and application transaction ensures data integrity and speed, because multiple statements are submitted to the database faster than one piece of submission; independent , no additional dependencies. The SQL language largely implements the ANSI SQL92 standard, especially support for views, triggers, transactions, support for nested SQL

extremedb Memory-type real-time database

eXtremeDB is a product of the American McObject Company. The company is located in Washington State, with more than 30 years of experience in compilers, real-time programming, data management, and kernel-level drivers.
McObject products include extremedb memory databases, SQL feature support Extremesql, transaction log support Extremelog, high availability support Extremeha, and extremews embedded Web Server. The eXtremeDB is its basic product; Extremesql provides SQL support, Extremelog provides the transaction log support functionality, and Extremeha provides high availability for extremedb; Extremews and eXtremeDB Seamless integration, enabling users to access and use the database through the Web

The difference between eXtremeDB and similar products is that it is not based on any file system at all.
Real-time databases are not a particularly new concept, primarily for managing real-time data. Most of these real-time data management systems use the concept of enterprise database to optimize the real-time application, such as using RAM disk or Cache to reduce the access to the disk and improve the performance of the database. This optimization to some extent improve the speed of the database, but because it is not specially designed for real-time data management, still need to retain such as caching, file I/O operation mechanisms, the real-time performance of the increase is limited, can not fully meet the needs of the actual work, Most users still manage real-time data using their own database management program. Historically, in order to manage real-time data, mcobject companies often have to make corporate database cuts. As a result of such a heavy workload, it eventually prompted McObject to launch the eXtremeDB, a new product dedicated to real-time data management.

The difference between eXtremeDB and similar products is that the extremedb is an internal library structure, and most of the similar products are based on C/s structure. One disadvantage of the C/s structure management real time data is the uncertainty of communication between the tasks introduced. and the inline library eliminates this uncertainty.

As a real-time database, eXtremeDB is how to improve its real-time performance.
As a database system oriented to real-time data management, eXtremeDB improves its real time performance through the following measures:

Generate APIs based on user data characteristics; Data stored in memory is not based on any file system, the cost of file I/O operations is eliminated, the Cache overhead is omitted, and because it is not based on a file system, system optimization is no longer aimed at reducing file access, but is designed to reduce CPU overhead and therefore is a higher level of optimization As with real-time operating systems, transactions give priority to ensure the real-time nature of key transactions.

Yes, eXtremeDB is an embedded database. Its basic cost can be as small as 60K bytes, and the extra cost of managing the space for real-time data is only 20. Because APIs are generated based on user data characteristics, invoking these APIs allows you to manage real-time data using eXtremeDB, so extremedb is naturally integrated with user programs. So, eXtremeDB is an embedded database. Here, one thing to add is that some file-based databases can be cropped in size, so they work on Flash or RAM Disk at run time. Such a database can be said to be embeddable, and eXtremeDB does not rely on the file system, is fully embedded in the application, and therefore is truly embedded.

Embedded systems are usually compact systems, i.e. small systems. For a large number of real-time data needs to be managed, eXtremeDB is competent.

This point should be particularly pointed out: eXtremeDB First is a real-time database, and then the embedded database. Because real-time data management requires very high time space efficiency, fewer resources are required to complete the same work. Therefore, in the resource-compact system, the advantages of the eXtremeDB are shown. Therefore, extremedb overhead and embedding with the application's natural integration does not mean that eXtremeDB can only manage data in small systems. eXtremeDB is embedded, but the application system is not required to be embedded. Application system can be embedded system such as mobile phone, home network management, set-top box, digital TV, etc., can also be desktop system or server system such as core router, call center, financial trading system and so on. The real time data management of large system is the advantage of eXtremeDB, which has been proved in practical use at home and abroad.

msql Embedded Database

mSQL (mini SQL) is a single user database management system, free of charge for personal use and commercial use. Because of its short and short, so that its development of the application system is particularly favored by internet users. mSQL (mini SQL) is a small relational database that does not perform well and is not fully supported in SQL language, but is sufficient in some network database applications. Because the mSQL is simpler, running a simple SQL statement is faster than MySQL, while MySQL is on a thread and index, running complex SQL statements faster than Msql,postgresql. mSQL's website is: http://www.hughes.com.au, the latest version is May 8, 2005 released 3.7. The sign of msql is a deer.

Technical features of mSQL:

On the security side, msql the access rights of each user on each host through the ACL file, the default is all readable/written. mSQL lacks most of the features of ANSI SQL, and it only implements a minimal API with no transactional and referential integrity. mSQL and Lite (a scripting language like C, distributed together with distribution) can get a Web integration package called W3-msql, which is JDBC, ODBC, Perl, and PHP APIs.