nimbus-eth1/nimbus/db/README.md

Nimbus-eth1 -- Ethereum execution layer database architecture

Last update: 2024-03-08

The following diagram gives a simplified view how components relate with regards to the data storage management.

An arrow between components a and b (as in a->b) is meant to be read as a relies directly on b, or a is served by b. For classifying the functional type of a component in the below diagram, the abstraction type is enclosed in brackets after the name of a component.

• (application)
  This is a group of software modules at the top level of the hierarchy. In the diagram below, the EVM is used as an example. Another application might be the RPC service.

• (API)
  The API classification is used for a thin software layer hiding a set of different drivers where only one driver is active for the same API instance. It servers as sort of a logical switch.

• (concentrator)
  The concentrator merges several sub-module instances and provides their collected services as a single unified instance. There is not much additional logic implemented besides what the sub-modules provide.

• (driver)
  The driver instances are sort of the lower layer workhorses. The implement logic for solving a particular problem, providing a typically well defined service, etc.

• (engine)
  This is a bottom level driver in the below diagram.

                                        +-------------------+
                                        | EVM (application) |
                                        +-------------------+
                                             |          |
                                             v          |
                          +-------------------------+   |
                          | State DB (concentrator) |   |
                          +-------------------------+   |
                              |           |       |     |
                              v           |       |     |
     +----------------------------+       |       |     |
     |       Ledger (API)         |       |       |     |
     +----------------------------+       |       |     |
                  |      |                |       |     |
                  v      v                |       |     |
     +--------------+  +--------------+   |       |     |
     | legacy cache |  | ledger cache |   |       |     |
     |   (driver)   |  |   (driver)   |   |       |     |
     +--------------+  +--------------+   |       |     |
                  |      |                v       |     |
                  |      |   +----------------+   |     |
                  |      |   |   Common       |   |     |
                  |      |   | (concentrator) |   |     |
                  |      |   +----------------+   |     |
                  |      |         |              |     |
                  v      v         v              v     v
     +---------------------------------------------------------------------+
     |               Core DB (API)                                         |
     +---------------------------------------------------------------------+
                     |                  |
                     v                  v
     +--------------------------+   +--------------------------------------+
     | legacy DB (concentrator) |   |   Aristo DB (driver,concentrator)    |
     +--------------------------+   +--------------------------------------+
           |                 |                |          |
           v                 |                v          v
     +--------------------+  |     +--------------+  +---------------------+
     | Hexary DB (driver) |  |     | Kvt (driver) |  | Aristo MPT (driver) |
     +--------------------+  |     +--------------+  +---------------------+
           |                 |                |          |
           v                 v                |          |
     +--------------------------+             |          |
     | Key-value table (driver) |             |          |
     +--------------------------+             |          |
                  |                           |          |
                  v                           v          v
     +---------------------------------------------------------------------+
     |                Rocks DB (engine)                                    |
     +---------------------------------------------------------------------+
  

Here is a list of path references for the components with some explanation. The sources for the components are not always complete but indicate the main locations where to start looking at.

• Aristo DB (driver)

  • Sources:
    ./nimbus/db/core_db/backend/aristo_*
    

  • Synopsis:
    Combines both, the Kvt and the Aristo driver sub-modules providing an interface similar to the legacy DB (concentrator) module.

• Aristo MPT (driver)

  • Sources:
    ./nimbus/db/aristo*

  • Synopsis:
    Revamped implementation of a hexary Merkle Patricia Tree.

• Common (concentrator)

  • Sources:
    ./nimbus/common*
    

  • Synopsis:
    Collected information for running block chain execution layer applications.

• Core DB (API)

  • Sources:
    ./nimbus/db/core_db*

  • Synopsis:
    Database abstraction layer. Unless for legacy applications, there should be no need to reach out to the layers below.

• EVM (application)

  • Sources:
    ./nimbus/core/executor/* ./nimbus/evm/*

  • Synopsis:
    An implementation of the Ethereum Virtual Machine.

• Hexary DB (driver)

  • Sources:
    ./vendor/nim-eth/eth/trie/hexary.nim
    

  • Synopsis:
    Implementation of an MPT, see compact Merkle Patricia Tree.

• Key-value table (driver)

  • Sources:
    ./vendor/nim-eth/eth/trie/db.nim
    

  • Synopsis:
    Key value table interface to be used directly for key-value storage or by the Hexary DB (driver) module for storage. Some magic is applied in order to treat hexary data accordingly (based on key length.)

• Kvt (driver)

  • Sources:
    ./nimbus/db/kvt*

  • Synopsis:
    Key value table interface for the Aristo DB (driver) module. Contrary to the Key-value table (driver), it is not used for MPT data.

• Ledger (API)

  • Sources:
    ./nimbus/db/ledger*

  • Synopsis:
    Abstraction layer for either the legacy cache (driver) accounts cache (which works with the legacy DB (driver) backend only) or the ledger cache (driver) re-write which is supposed to work with all Core DB (API) backends.

• ledger cache (driver)

  • Sources:
    ./nimbus/db/ledger/accounts_ledger.nim
    ./nimbus/db/ledger/backend/accounts_ledger*
    ./nimbus/db/ledger/distinct_ledgers.nim

  • Synopsis:
    Management of accounts and storage data. This is a re-write of the legacy DB (driver) which is supposed to work with all Core DB (API) backends.

• legacy cache (driver)

  • Sources:
    ./nimbus/db/distinct_tries.nim
    ./nimbus/db/ledger/accounts_cache.nim
    ./nimbus/db/ledger/backend/accounts_cache*

  • Synopsis:
    Management of accounts and storage data. It works only for the legacy driver of the Core DB (API) backend.

• legacy DB (concentrator)

  • Sources:
    ./nimbus/db/core_db/backend/legacy_*

  • Synopsis:
    Legacy database abstraction. It mostly forwards requests directly to the to the Key-value table (driver) and/or the hexary DB (driver).

• Rocks DB (engine)

  • Sources:
    ./vendor/nim-rocksdb/*

  • Synopsis:
    Persistent storage engine.

• State DB (concentrator)

  • Sources:
    ./nimbus/evm/state.nim
    ./nimbus/evm/types.nim

  • Synopsis:
    Integrated collection of modules and methods relevant for the EVM.