nimbus-eth1/nimbus/db/ledger
Jordan Hrycaj 13f51939f6
Core db aristo hasher profiling and timing improvement (#1938)
* Explicitly use shared `Kvt` table on `Ledger` and `Clique` lookup.

why:
  Speeds up lookup time with `Aristo` backend. For writing `Clique` data,
  the `Companion` model allows to write `Clique` data past the database
  locked by evm transactions.

* Implement `CoreDb` profiling with API tracking

why:
  Chasing time spent per APT procs ...

* Implement `Ledger` profiling with API tracking

why:
  Chasing time spent per APT procs ...

* Always hashify when commiting or storing

why:
  A dirty cache makes no sense when committing

* Make sure that a zero key is created when adding/updating vertices

why:
  This is an error fix mainly for edge cases. A typical error was
  that the root key got deleted when there were only a few vertices
  left on the DB.

* Need all created and changed vertices zero-keyed on the cache

why:
  A zero key (i.e. empty Merkle hash) indicates that a vertex key
  needs to be updated. This would not be needed immediately after
  a merge as there is an actual leaf path on the cache layer. But
  after subsequent merge and delete operations this information
  might get blurred.

* Re-org hashing algorithm

why:
  Apart from errors, the previous implementation was too slow for
  two reasons:
  + some control hashes were calculated for debugging (now all
    verification is done in `aristo_check` module)
  + the leaf paths stored on the cache are used to build the
    labelling (aka hashing) schedule; there paths were accumulated
    over successive hash sessions although it is clear that all
    keys were generated, already
2023-12-12 17:47:41 +00:00
..
backend Optional accounts cache module for creating genesis (#1897) 2023-11-20 11:51:43 +00:00
base Core db aristo hasher profiling and timing improvement (#1938) 2023-12-12 17:47:41 +00:00
.gitignore Add check copyright year linter to CI 2023-11-01 10:41:20 +07:00
README.md Ledger abstraction for accounts cache (#1824) 2023-10-18 20:27:22 +01:00
accounts_cache.nim Fix default pruning for ledger and update core db and ledger logging (#1861) 2023-10-25 15:03:09 +01:00
accounts_ledger.nim Core db aristo hasher profiling and timing improvement (#1938) 2023-12-12 17:47:41 +00:00
base.nim Core db aristo hasher profiling and timing improvement (#1938) 2023-12-12 17:47:41 +00:00
base_iterators.nim Core db aristo hasher profiling and timing improvement (#1938) 2023-12-12 17:47:41 +00:00
distinct_ledgers.nim Core db aristo hasher profiling and timing improvement (#1938) 2023-12-12 17:47:41 +00:00

README.md

The file accounts_cache.nim has been relocated

Background

The new LedgerRef module unifies different implementations of the accounts_cache. It is intended to be used as new base method for all of the AccountsCache implementations. Only constructors differ, depending on the implementation.

This was needed to accomodate for different CoreDb API paradigms. While the overloaded legacy AccountsCache implementation is just a closure based wrapper around the accounts_cache module, the overloaded AccountsLedgerRef is a closure based wrapper around the accounts_ledger module with the new CoreDb API returning Result[] values and saparating the meaning of trie root hash and trie root reference.

This allows to use the legacy hexary database (with the new CoreDb API) as well as the Aristo database (only supported on new API.)

Instructions

Legacy notation LedgerRef replacement Comment
import accounts_cache import ledger preferred method,
AccountsCache.init(..) AccountsCache.init(..) wraps AccountsCache
methods
or
import ledger/accounts_cache stay with legacy
AccountsCache.init(..) version of
AccountsCache
--
fn(ac: AccountsCache) fn(ac: LedgerRef) function example for
preferred wrapper
or method
fn(ac: AccountsCache) with legacy version,
no change here

The constructor decides which CoreDb API is to be used

Legacy API constructor new API Constructor
import ledger import ledger
let w = AccountsCache.init(..) let w = AccountsLedgerRef.init(..)