From 332ec75d5acd2f767a54552afb4a94df57d0bb17 Mon Sep 17 00:00:00 2001 From: Jordan Hrycaj Date: Fri, 8 Mar 2024 18:42:46 +0000 Subject: [PATCH] Database architecture diagram & module overview (#2065) --- nimbus/db/{aristo => }/.gitignore | 0 nimbus/db/README.md | 209 ++++++++++++++++++++++++++++++ nimbus/db/core_db/.gitignore | 1 - nimbus/db/ledger/.gitignore | 1 - 4 files changed, 209 insertions(+), 2 deletions(-) rename nimbus/db/{aristo => }/.gitignore (100%) create mode 100644 nimbus/db/README.md delete mode 100644 nimbus/db/core_db/.gitignore delete mode 100644 nimbus/db/ledger/.gitignore diff --git a/nimbus/db/aristo/.gitignore b/nimbus/db/.gitignore similarity index 100% rename from nimbus/db/aristo/.gitignore rename to nimbus/db/.gitignore diff --git a/nimbus/db/README.md b/nimbus/db/README.md new file mode 100644 index 000000000..65b937e8b --- /dev/null +++ b/nimbus/db/README.md @@ -0,0 +1,209 @@ +Nimbus-eth1 -- Ethereum execution layer database architecture +============================================================= +Last update: 2024-03-08 + +The following diagram gives a simplified view how components relate with +regards to the data storage management. + +An arrow between components **a** and **b** (as in *a->b*) is meant to be read +as **a** relies directly on **b**, or **a** is served by **b**. For classifying +the functional type of a component in the below diagram, the abstraction type +is enclosed in brackets after the name of a component. + +* *(application)*
+ This is a group of software modules at the top level of the hierarchy. In the + diagram below, the **EVM** is used as an example. Another application might + be the **RPC** service. + +* *(API)*
+ The *API* classification is used for a thin software layer hiding a set of + different drivers where only one driver is active for the same *API* + instance. It servers as sort of a logical switch. + +* *(concentrator)*
+ The *concentrator* merges several sub-module instances and provides their + collected services as a single unified instance. There is not much additional + logic implemented besides what the sub-modules provide. + +* *(driver)*
+ The *driver* instances are sort of the lower layer workhorses. The implement + logic for solving a particular problem, providing a typically well defined + service, etc. + +* *(engine)*
+ This is a bottom level *driver* in the below diagram. + + +-------------------+ + | EVM (application) | + +-------------------+ + | | + v | + +-------------------------+ | + | State DB (concentrator) | | + +-------------------------+ | + | | | | + v | | | + +----------------------------+ | | | + | Ledger (API) | | | | + +----------------------------+ | | | + | | | | | + v v | | | + +--------------+ +--------------+ | | | + | legacy cache | | ledger cache | | | | + | (driver) | | (driver) | | | | + +--------------+ +--------------+ | | | + | | v | | + | | +----------------+ | | + | | | Common | | | + | | | (concentrator) | | | + | | +----------------+ | | + | | | | | + v v v v v + +---------------------------------------------------------------------+ + | Core DB (API) | + +---------------------------------------------------------------------+ + | | + v v + +--------------------------+ +--------------------------------------+ + | legacy DB (concentrator) | | Aristo DB (driver,concentrator) | + +--------------------------+ +--------------------------------------+ + | | | | + v | v v + +--------------------+ | +--------------+ +---------------------+ + | Hexary DB (driver) | | | Kvt (driver) | | Aristo MPT (driver) | + +--------------------+ | +--------------+ +---------------------+ + | | | | + v v | | + +--------------------------+ | | + | Key-value table (driver) | | | + +--------------------------+ | | + | | | + v v v + +---------------------------------------------------------------------+ + | Rocks DB (engine) | + +---------------------------------------------------------------------+ + +Here is a list of path references for the components with some explanation. +The sources for the components are not always complete but indicate the main +locations where to start looking at. + +* *Aristo DB (driver)* + + Sources:
+ ./nimbus/db/core_db/backend/aristo_*
+ + + Synopsis:
+ Combines both, the *Kvt* and the *Aristo* driver sub-modules providing an + interface similar to the [legacy DB (concentrator)](#ldc) module. + +* *Aristo MPT (driver)* + + Sources:
+ ./nimbus/db/aristo* + + + Synopsis:
+ Revamped implementation of a hexary *Merkle Patricia Tree*. + +* *Common (concentrator)* + * Sources:
+ ./nimbus/common*
+ + * Synopsis:
+ Collected information for running block chain execution layer + applications. + +* *Core DB (API)* + * Sources:
+ ./nimbus/db/core_db* + + * Synopsis:
+ Database abstraction layer. Unless for legacy applications, there should + be no need to reach out to the layers below. + +* *EVM (application)* + + Sources:
+ ./nimbus/core/executor/* + ./nimbus/evm/* + + + Synopsis:
+ An implementation of the *Ethereum Virtual Machine*. + +* *Hexary DB (driver)* + + Sources:
+ ./vendor/nim-eth/eth/trie/hexary.nim
+ + + Synopsis:
+ Implementation of an MPT, see + [compact Merkle Patricia Tree](http://archive.is/TinyK). + +* *Key-value table (driver)* + + Sources:
+ ./vendor/nim-eth/eth/trie/db.nim
+ + + Synopsis:
+ Key value table interface to be used directly for key-value storage or + by the [Hexary DB (driver)](#hdd) module for storage. Some magic is applied + in order to treat hexary data accordingly (based on key length.) + +* *Kvt (driver)* + + Sources:
+ ./nimbus/db/kvt* + + + Synopsis:
+ Key value table interface for the [Aristo DB (driver)](#add) module. + Contrary to the [Key-value table (driver)](#kvtd), it is not used for + MPT data. + +* *Ledger (API)* + + Sources:
+ ./nimbus/db/ledger* + + + Synopsis:
+ Abstraction layer for either the [legacy cache (driver)](#lgcd) accounts + cache (which works with the [legacy DB (driver)](#ldd) backend only) or + the [ledger cache (driver)](#ldcd) re-write which is supposed to work + with all [Core DB (API)](#cda) backends. + +* *ledger cache (driver)* + + Sources:
+ ./nimbus/db/ledger/accounts_ledger.nim
+ ./nimbus/db/ledger/backend/accounts_ledger*
+ ./nimbus/db/ledger/distinct_ledgers.nim + + + Synopsis:
+ Management of accounts and storage data. This is a re-write of the + [legacy DB (driver)](#lgdd) which is supposed to work with all + [Core DB (API)](#cda) backends. + +* *legacy cache (driver)* + + Sources:
+ ./nimbus/db/distinct_tries.nim
+ ./nimbus/db/ledger/accounts_cache.nim
+ ./nimbus/db/ledger/backend/accounts_cache* + + + Synopsis:
+ Management of accounts and storage data. It works only for the legacy + driver of the [Core DB (API)](#cda) backend. + +* *legacy DB (concentrator)* + + Sources:
+ ./nimbus/db/core_db/backend/legacy_* + + + Synopsis:
+ Legacy database abstraction. It mostly forwards requests directly to the + to the [Key-value table (driver)](#kvtd) and/or the + [hexary DB (driver)](#hdd). + +* *Rocks DB (engine)* + + Sources:
+ ./vendor/nim-rocksdb/* + + + Synopsis:
+ Persistent storage engine. + +* *State DB (concentrator)* + + Sources:
+ ./nimbus/evm/state.nim
+ ./nimbus/evm/types.nim + + + Synopsis:
+ Integrated collection of modules and methods relevant for the EVM. + diff --git a/nimbus/db/core_db/.gitignore b/nimbus/db/core_db/.gitignore deleted file mode 100644 index 0b84df0f0..000000000 --- a/nimbus/db/core_db/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.html \ No newline at end of file diff --git a/nimbus/db/ledger/.gitignore b/nimbus/db/ledger/.gitignore deleted file mode 100644 index 0b84df0f0..000000000 --- a/nimbus/db/ledger/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.html \ No newline at end of file