// Copyright 2019 TiKV Project Authors. Licensed under Apache-2.0. // Modifications Copyright 2022 PingCAP, Inc. //! Shared storage-engine traits. //! //! The crate has no native storage dependency. Current production persistence //! is implemented by `kvengine` and `rfengine`. //! //! //! ## Capabilities of a TiKV engine //! //! TiKV engines store binary keys and values. //! //! Every pair lives in a [_column family_], which can be thought of as being //! independent data stores. //! //! [_column family_]: https://github.com/facebook/rocksdb/wiki/Column-Families //! //! Consistent read-only views of the database are accessed through _snapshots_. //! //! Multiple writes can be committed atomically with a _write batch_. //! //! //! # The TiKV engine API //! //! The API inherits its design from RocksDB. As support for other engines is //! added to TiKV, it is expected that this API will become more abstract, and //! less Rocks-specific. //! //! This crate is almost entirely traits, plus a few "plain-old-data" types that //! are shared between engines. //! //! Some key types include: //! //! - [`KvEngine`] - a key-value engine, and the primary type defined by this //! crate. Most code that uses generic engines will be bounded over a generic //! type implementing `KvEngine`. `KvEngine` itself is bounded by many other //! traits that provide collections of functionality, with the intent that as //! TiKV evolves it may be possible to use each trait individually, and to //! define classes of engine that do not implement all collections of //! features. //! //! - [`Snapshot`] - a view into the state of the database at a moment in time. //! For reading sets of consistent data. //! //! - [`Peekable`] - types that can read single values. This includes engines //! and snapshots. //! //! - [`Iterable`] - types that can iterate over the values of a range of keys, //! by creating instances of the TiKV-specific [`Iterator`] trait. This //! includes engines and snapshots. //! //! - [`WriteBatch`] - types that can commit multiple key/value pairs in //! batches. A `WriteBatchExt::WriteBatch` commits all pairs in one atomic //! transaction. A `WriteBatchExt::WriteBatchVec` does not (FIXME: is this //! correct?). //! //! The `KvEngine` instance generally acts as a factory for types that implement //! other traits in the crate. These factory methods, associated types, and //! other associated methods are defined in "extension" traits. For example, //! methods on engines related to batch writes are in the `WriteBatchExt` trait. //! //! //! # Design notes //! //! - `KvEngine` is the main engine trait. It requires many other traits, which //! have many other associated types that implement yet more traits. //! //! - Features should be grouped into their own modules with their own traits. A //! common pattern is to have an associated type that implements a trait, and //! an "extension" trait that associates that type with `KvEngine`, which is //! part of `KvEngine's trait requirements. //! //! - For now, for simplicity, all extension traits are required by `KvEngine`. //! In the future it may be feasible to separate them for engines with //! different feature sets. //! //! - Associated types generally have the same name as the trait they are //! required to implement. Engine extensions generally have the same name //! suffixed with `Ext`. Concrete implementations usually have the same name //! prefixed with the database name, i.e. `Rocks`. //! //! Example: //! //! ```ignore //! // in engine_traits //! //! trait WriteBatchExt { //! type WriteBatch: WriteBatch; //! } //! //! trait WriteBatch { } //! ``` //! //! ```ignore //! // in a concrete engine implementation //! //! impl WriteBatchExt for RocksEngine { //! type WriteBatch = RocksWriteBatch; //! } //! //! impl WriteBatch for RocksWriteBatch { } //! ``` //! //! - All engines use the same error type, defined in this crate. Thus //! engine-specific type information is boxed and hidden. //! //! - `KvEngine` is a factory type for some of its associated types, but not //! others. For now, use factory methods when RocksDB would require factory //! method (that is, when the DB is required to create the associated type - //! if the associated type can be created without context from the database, //! use a standard new method). If future engines require factory methods, the //! traits can be converted then. //! //! - Types that require a handle to the engine (or some other "parent" type) do //! so with either Rc or Arc. An example is EngineIterator. The reason for //! this is that associated types cannot contain lifetimes. That requires //! "generic associated types". See //! //! - //! - //! //! - Traits can't have mutually-recursive associated types. That is, if //! `KvEngine` has a `Snapshot` associated type, `Snapshot` can't then have a //! `KvEngine` associated type - the compiler will not be able to resolve both //! `KvEngine`s to the same type. In these cases, e.g. `Snapshot` needs to be //! parameterized over its engine type and `impl Snapshot for //! RocksSnapshot`. #![cfg_attr(test, feature(test))] #![feature(min_specialization)] #![feature(assert_matches)] // These modules contain traits that need to be implemented by engines, either // they are required by KvEngine or are an associated type of KvEngine. It is // recommended that engines follow the same module layout. // // Many of these define "extension" traits, that end in `Ext`. mod cf_names; pub use crate::cf_names::*; mod db_vector; pub use crate::db_vector::*; mod file_system; pub use crate::file_system::*; mod sst; pub use crate::sst::*; mod encryption; pub use crate::encryption::*; // These modules contain more general traits, some of which may be implemented // by multiple types. mod iterable; pub use crate::iterable::*; mod peekable; pub use crate::peekable::*; // These modules contain concrete types and support code that do not need to // be implemented by engines. mod cf_defs; pub use crate::cf_defs::*; mod errors; pub use crate::errors::*; mod options; pub use crate::options::*; pub mod range; pub use crate::range::*; // These modules need further scrutiny pub mod util; // FIXME: This should live somewhere else pub const DATA_KEY_PREFIX_LEN: usize = 1;