SQLite is an open source, embedded relational database. Rathern than running independently as a stand-alone process, it coexist insede the application it serves - within its process space.

One advantage of having a database server inside your program is that no network configuration or administation is required.

Examples of use of SQLite are:

  • Managing complex session information in a web environment
  • Stand-in relational database for development and testing
  • Serve as cache
  • Hold configuration data
  • Work as an application file format
  • Serve as a functional tool for general data processing, especially as an in-memory database

Architecture

SQLite consist of eight separate modules grouped within three major subsystems. These modules diviede the query processing into discrete tasks that work like an assembly line. The top of the stack compiles the query, the middle executes it, and the bottom handles storage and interfacing with the operating system.

imagen

The interface

The inteface is the top of the stack and consists of the SQLite C API. It is the means through which programs, scripting languages, and libraries alike interact with SQLite.

The compiler

The compilation process starts with the tokenizer and pareser. They work together to take a Structured Query Langage (SQL) statement in text form, validate its syntax, and then convert it to a hierarchical data structure that the lower layers can manipularte. SQLite’s tokenizer is hand-coded. Its parser is generated by SQLite custom parser generator, which is called Lemon.

The Lemon parser generator is designed for high performance and take special precautions to guard aginst memory leaks. Once the statement has been broken into tokens, evaluated, and recast in the form of a parse tree, the parser passes the tree down to the code generator.

The coder generator translates the parse tree into a kind of assembly language specific to SQLite. This assembly language consists of instructions that are executable by its virtual machine for processing.

The Virtual Machine

At the center of the stack is the virtual machine, also called the virtual database engine (VDBE). It is a register-based virtual machine that works on byte code, making it independent of the underlying operating system, CPU, or system architcture. The VDBE language consists of mor than 100 possible tasks known as opcode, which are all centered on the database operation. The VDBE is designed specifically for data processing. Every instruction either accomplishes a specific database operation or performs manipulations to prepare for such an operation. Every SQL statement in SQLite is first compiled to this virtual machine language, formin a standalone instruction set that defines how to perform the given command.

The Backend

The backend consist of the B-tree, page cache, and OS interface. The B-tree and page cache(pager) work together as informations brokers. Their currency is database pages, which are uniformly sized blocks of data, that, like containers, are made for transportation. Inside the pages are the bit of information such as records and columns and index entries. Neither the B-tree nor the pager has any knowledge of the contents. They only move and order pages; they don’t care what’s inside.

The B-tree’s job is order. It maintains many complex and intricate relationships between pages, which keeps everything connected and easy to locate. It organizes pages into tree-like structures, which are highly optimized for searching.

The pager serves the B-tree, feeding it pages. Its job its transportation and efficiency. The pager transfer pages to an from disk at the B-tree’s behest. The pager tries to speed this up by keeping frequently used pages cached in memory and thus minimizes the number of times it has to deal directly with the hard drive. It uses special techniques to predict which pages will be needed in the future and thus anticipate the needs of the B-tree, keeping pages flying as fast as possible. Also in the pager’s job description are transaction management, database locking, and crash recovery. Many of these jobs are mediated by the OS interface.

Thing such as file locking are often implemented differently in differente operating systems. The OS interface provides and abstraction layer that hides these deifferences from other SQLite modules. The end result is that the other modules see a single consistent interface with which to do things like locking.

Utilities and Test Code

Miscellaneous utilities and commond services such as memory allocation, string comparison, and Unicode conversion routines are kept in the utilities module. This is basically a catchall module for services that multiple modules need to use or share. The testing module contains a myriad of regression tests designed to examine very little corner of the database code.