nimbus-eth1/nimbus/db
Jordan Hrycaj 3a62250d04
Make test op memory work again (#2236)
* Remove crufty `pruneTrie` arguments

* Replaced legacy `distinct_trie` logic by new `ledger` functionality

why:
  The module `distinct_trie` is supported by `Aristo` in trivial cases.

* Activate `test_op_memory`
2024-05-28 14:24:10 +00:00
..
aristo Fix accounts_ledger savePoint conversion (#2226) 2024-05-26 17:42:02 +07:00
core_db restore a few tests, cleanup (#2234) 2024-05-28 14:49:35 +02:00
era1_db era: simplify, instant startup (#2218) 2024-05-26 08:24:13 +02:00
kvt fix warnings around unused imports of std/algorithm; proc -> func (#2220) 2024-05-25 21:01:28 +02:00
ledger Make test op memory work again (#2236) 2024-05-28 14:24:10 +00:00
state_db Culling legacy DB and accounts cache (#2197) 2024-05-20 10:17:51 +00:00
.gitignore Database architecture diagram & module overview (#2065) 2024-03-08 18:42:46 +00:00
README.md Aristo resume off line syncing on pre loaded database (#2203) 2024-05-22 13:41:14 +00:00
access_list.nim Implement RPC method eth_getAccessList (#2091) 2024-03-21 18:24:32 +07:00
aristo.nim Coredb use stackable api for aristo backend (#2060) 2024-02-29 21:10:24 +00:00
core_db.nim Culling legacy DB and accounts cache (#2197) 2024-05-20 10:17:51 +00:00
distinct_tries.nim Culling legacy DB and accounts cache (#2197) 2024-05-20 10:17:51 +00:00
era1_db.nim era: simplify, instant startup (#2218) 2024-05-26 08:24:13 +02:00
kvstore_rocksdb.nim Generalise prune mode option 4 different db models (#2139) 2024-04-17 18:09:55 +00:00
kvt.nim Coredb use stackable api for aristo backend (#2060) 2024-02-29 21:10:24 +00:00
ledger.nim Aristo resume off line syncing on pre loaded database (#2203) 2024-05-22 13:41:14 +00:00
state_db.nim Rename hasCodeOrNonce to contractCollision 2024-04-16 09:31:10 +07:00
storage_types.nim rm clique/mining remnants; rm unused code (#2232) 2024-05-28 07:10:10 +02:00
transient_storage.nim implement EIP-1153: Transient storage 2023-06-23 14:04:36 +07:00
trie_get_branch.nim Implementation of Nimbus eth_getProof RPC Endpoint. (#1960) 2024-01-09 10:05:52 +08:00
values_from_bytes.nim Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00

README.md

Nimbus-eth1 -- Ethereum execution layer database architecture

Last update: 2024-03-08

The following diagram gives a simplified view how components relate with regards to the data storage management.

An arrow between components a and b (as in a->b) is meant to be read as a relies directly on b, or a is served by b. For classifying the functional type of a component in the below diagram, the abstraction type is enclosed in brackets after the name of a component.

  • (application)
    This is a group of software modules at the top level of the hierarchy. In the diagram below, the EVM is used as an example. Another application might be the RPC service.

  • (API)
    The API classification is used for a thin software layer hiding a set of different drivers where only one driver is active for the same API instance. It servers as sort of a logical switch.

  • (concentrator)
    The concentrator merges several sub-module instances and provides their collected services as a single unified instance. There is not much additional logic implemented besides what the sub-modules provide.

  • (driver)
    The driver instances are sort of the lower layer workhorses. The implement logic for solving a particular problem, providing a typically well defined service, etc.

  • (engine)
    This is a bottom level driver in the below diagram.

                           +-------------------+
                           | EVM (application) |
                           +-------------------+
                                   |     |
                                   v     |
       +-----------------------------+   |
       |   State DB (concentrator)   |   |
       +-----------------------------+   |
           |                       |     |
           v                       |     |
       +------------------------+  |     |
       |      Ledger (API)      |  |     |
       +------------------------+  |     |
           |              |        |     |
           v              |        |     |
       +--------------+   |        |     |
       | ledger cache |   |        |     |
       |   (driver)   |   |        |     |
       +--------------+   |        |     |
           |              v        |     |
           |   +----------------+  |     |
           |   |   Common       |  |     |
           |   | (concentrator) |  |     |
           |   +----------------+  |     |
           |             |         |     |
           v             v         v     v
       +---------------------------------------+
       |               Core DB (API)           |
       +---------------------------------------+
                         |
                         v
       +---------------------------------------+
       |    Aristo DB (driver,concentrator)    |
       +---------------------------------------+
                 |             |
                 v             v
       +--------------+  +---------------------+
       | Kvt (driver) |  | Aristo MPT (driver) |
       +--------------+  +---------------------+
                 |             |
                 v             v
       +---------------------------------------+
       |         Rocks DB (engine)             |
       +---------------------------------------+
    

Here is a list of path references for the components with some explanation. The sources for the components are not always complete but indicate the main locations where to start looking at.

  • Aristo DB (driver)

    • Sources:
      ./nimbus/db/core_db/backend/aristo_*

    • Synopsis:
      Combines both, the Kvt and the Aristo driver sub-modules providing an interface similar to the legacy DB (concentrator) module.

  • Aristo MPT (driver)

    • Sources:
      ./nimbus/db/aristo*

    • Synopsis:
      Revamped implementation of a hexary Merkle Patricia Tree.

  • Common (concentrator)

    • Sources:
      ./nimbus/common*

    • Synopsis:
      Collected information for running block chain execution layer applications.

  • Core DB (API)

    • Sources:
      ./nimbus/db/core_db*

    • Synopsis:
      Database abstraction layer. Unless for legacy applications, there should be no need to reach out to the layers below.

  • EVM (application)

    • Sources:
      ./nimbus/core/executor/* ./nimbus/evm/*

    • Synopsis:
      An implementation of the Ethereum Virtual Machine.

  • Hexary DB (driver)

  • Key-value table (driver)

    • Sources:
      ./vendor/nim-eth/eth/trie/db.nim

    • Synopsis:
      Key value table interface to be used directly for key-value storage or by the Hexary DB (driver) module for storage. Some magic is applied in order to treat hexary data accordingly (based on key length.)

  • Kvt (driver)

  • Ledger (API)

  • ledger cache (driver)

    • Sources:
      ./nimbus/db/ledger/accounts_ledger.nim
      ./nimbus/db/ledger/backend/accounts_ledger*
      ./nimbus/db/ledger/distinct_ledgers.nim

    • Synopsis:
      Management of accounts and storage data. This is a re-write of the legacy DB (driver) which is supposed to work with all Core DB (API) backends.

  • legacy DB (concentrator)

  • Rocks DB (engine)

    • Sources:
      ./vendor/nim-rocksdb/*

    • Synopsis:
      Persistent storage engine.

  • State DB (concentrator)

    • Sources:
      ./nimbus/evm/state.nim
      ./nimbus/evm/types.nim

    • Synopsis:
      Integrated collection of modules and methods relevant for the EVM.