Architecture of SQLite database (translated from sqlite.org)

$1 Introduction
This document describes the architecture of the SQLite library, which is useful to those who want to understand and modify the internal working mechanism of SQLite.
Shows the main components of SQLite and their relationships. The following describes each component. (For more information, see http://www.sqlite.org/arch2.gif)

This document describes SQLite 3.0, 2.8, or an earlier version, but the details are different.
$2 Interface
Although some functions are distributed in other files, the common interface functions of the main SQLite library are implemented in the main. C, legacy. C, and vdbeapi. C source code files. Sqlite3_get_table () function is implemented in table. C. sqlite3_mprintf () is implemented in printf. C. sqlite3_complete () is implemented in tokenize. C. The TCL interface is implemented in tclsqlite. C. For more complete information about the C interface of SQLite, see http://www.sqlite.org/capi3ref.html.
To avoid naming conflicts with other software, all external symbols in the SQLite library are prefixed with sqlite3. These symbols are used externally. In other words, all the symbols starting with sqlite3 _ form the SQLite API.
$3 lexical analyzer
When an SQL statement is executed, the interface first transmits the string containing the SQL statement to the lexical analyzer for processing. The Lexical analyzer is responsible for dividing strings into one lexical unit and passing the lexical unit to the syntax analyzer. the lexical analyzer is in tokenize. in the c file, the code is compiled by hand, instead of being generated using tools such as lex.
Note that in this design, the lexical analyzer calls the syntax analyzer. People familiar with yacc and BISON usually call the lexical analyzer in the syntax analyzer, the author of SQLite tried both methods and found that it is better to call the syntax analyzer in the lexical analyzer.
$4 syntax analyzer
The syntax analyzer can understand the meaning of the lexical analysis unit based on the context. SQLite syntax analyzer is generated using the LALR (1) tool generator of Lemon (http://www.hwaci.com/sw/lemon. Lemon and YACC/BISON tools are similar, but Lemon uses a different input syntax, which is more difficult to make mistakes. Lemon can generate a reentrant and thread-safe syntax analyzer. Lemon defines a non-Terminator that prevents Memory leakage when syntax errors occur. The input file of the Lemon analyzer is defined in parse. y.
Since Lemon is not a common program, its complete source code is only one C file in the SQLite tool subdirectory. The Lemon documentation is in the doc subdirectory.
$5 Code Generator
After the syntax analyzer analyzes the SQL statement, it calls the code generator to generate the code executed on the virtual machine. The code is executed according to the requirements of the SQL statement. The code generator is included in many files: attach. c, auth. c, build. c, delete. c, expr. c, insert. c, pragma. c, select. c, trigger. c. update. c, vacuum. c and where. c. These files are what happens to our magic. Expr. c. process the code generation of the expression, where. c processes the code generation of the WHERE clause in SELECT, UPDATE, and DELETE statements, attach. c, delete. c, insert. c, select. c, trigger. c update. c and vacuum. c processes the code generation of SQL statements with the same name. Each file in c calls expr when necessary. c and where. c. Other SQL statements are implemented in build. c. The auth. c file implements the sqlite3_set_authorizer () function.
$6 virtual machines
The program generated by the Code Generator runs on the VM. The VM information is shown in the document http://www.sqlite.org/opcode.html. In summary, a virtual machine implements an abstract computing engine used to operate database files. A virtual machine has a stack to store the intermediate state of computing. Each command contains an operation code and a maximum of three operands. Virtual machines are implemented in vdbe. c. The virtual machine has its own header file: The vdbe. h file defines the interfaces of the Virtual Machine and SQLite library, and the vdbeInt. h file defines the structure of the virtual machine. The vdbeaux. c file contains tools used by virtual machines and interface modules. The vdbeapi. c file contains Virtual Machine external interfaces, such as sqlite3_bind _... and other functions. The string, integer, floating point, and BLOB types are all stored in an internal object named Mem, which is implemented in the vdbemem. c file.
SQLite uses the C language function callback method to implement the SQL statement function. Even the built-in SQL function is implemented in this way. Most built-in SQL functions, such as coalesce (), count (), and substr (), are implemented in func. C. The date and time conversion functions are implemented in date. C.
$7 B tree
SQLite uses the B tree for database implementation, and the B tree is implemented in the btree. c file. Each table and index in the database uses a separate B-tree. All Tree B is stored in a disk file. The details of the database file format are described in the comment at the beginning of the btree. c file.
The interface of the B-tree subsystem is defined in the header file btree. h.
$8 page Cache
The B-tree module uses a fixed block size to request information from the disk. The default block size is 1024b, but it can be adjusted from 512 to 65536b. The page cache reads, writes, and caches these blocks. The page cache also provides functional abstraction and database file lock operations for rollback and atomic commit. The B-tree driver retrieves pages from the page cache and notifies the page cache program when to modify, submit, or roll back operations. The page cache processes all these troublesome details to ensure that the request is fast, secure and Efficient processing.
Code that implements the page cache mechanism is stored in pager. C of a single c file. The interface of the page cache subsystem is defined in pager. h.
$9 Operating System Interface
To improve portability in POSIX and Win32 systems, SQLite and the operating system interfaces use an abstraction layer. This abstraction layer interface is defined in OS. h. Each operating system, especially its own implementation: OS _unix.c is UNIX, OS _win.c is windows, and so on. The implementation of each operating system, especially its own header files: OS _unix.h, OS _win.h, and so on.
$10 tool programs
The memory allocation and case-insensitive string comparison functions are implemented in the util. c file. The symbol table used by the syntax analyzer is a hash table, which is implemented in hash. C. The UTF. c file contains Unicode conversion functions. SQLite has its own printf () function implementation, which is defined in printf. C and the random number function implementation, which is implemented in random. C.
$11 test code
More than half of the code is for testing. There are many assert () functions in the main code file. Test1.c to test5.c and md5.c are both used for testing. OS _test.c simulates the disaster recovery capability of the page cache mechanism after the power supply fails.