nimbus-eth1/nimbus/db/ledger
Jordan Hrycaj 0f430c70fd
Aristo avoid storage trie update race conditions (#2251)
* Update TDD suite logger output format choices

why:
  New format is not practical for TDD as it just dumps data across a wide
  range (considerably larder than 80 columns.)

  So the new format can be turned on by function argument.

* Update unit tests samples configuration

why:
  Slightly changed the way to find the `era1` directory

* Remove compiler warnings (fix deprecated expressions and phrases)

* Update `Aristo` debugging tools

* Always update the `storageID` field of account leaf vertices

why:
  Storage tries are weekly linked to an account leaf object in that
  the `storageID` field is updated by the application.

  Previously, `Aristo` verified that leaf objects make sense when passed
  to the database. As a consequence
  * the database was inconsistent for a short while
  * the burden for correctness was all on the application which led
    to delayed error handling which is hard to debug.

  So `Aristo` will internally update the account leaf objects so that
  there are no race conditions due to the storage trie handling

* Aristo: Let `stow()`/`persist()` bail out unless there is a `VertexID(1)`

why:
  The journal and filter logic depends on the hash of the `VertexID(1)`
  which is commonly known as the state root. This implies that all
  changes to the database are somehow related to that.

* Make sure that a `Ledger` account does not overwrite the storage trie reference

why:
  Due to the abstraction of a sub-trie (now referred to as column with a
  hash describing its state) there was a weakness in the `Aristo` handler
  where an account leaf could be overwritten though changing the validity
  of the database. This has been changed and the database will now reject
  such changes.

  This patch fixes the behaviour on the application layer. In particular,
  the column handle returned by the `CoreDb` needs to be updated by
  the `Aristo` database state. This mitigates the problem that a storage
  trie might have vanished or re-apperaed with a different vertex ID.

* Fix sub-trie deletion test

why:
  Was originally hinged on `VertexID(1)` which cannot be wholesale
  deleted anymore after the last Aristo update. Also, running with
  `VertexID(2)` needs an artificial `VertexID(1)` for making `stow()`
  or `persist()` work.

* Cosmetics

* Activate `test_generalstate_json`

* Temporarily `deactivate test_tracer_json`

* Fix copyright header

---------

Co-authored-by: jordan <jordan@dry.pudding>
Co-authored-by: Jacek Sieka <jacek@status.im>
2024-05-30 17:48:38 +00:00
..
base Simplify AccountsLedgerRef complexity (#2239) 2024-05-29 13:06:49 +02:00
README.md Aristo resume off line syncing on pre loaded database (#2203) 2024-05-22 13:41:14 +00:00
accounts_ledger.nim Aristo avoid storage trie update race conditions (#2251) 2024-05-30 17:48:38 +00:00
base.nim Simplify AccountsLedgerRef complexity (#2239) 2024-05-29 13:06:49 +02:00
base_iterators.nim Simplify AccountsLedgerRef complexity (#2239) 2024-05-29 13:06:49 +02:00
distinct_ledgers.nim Make test op memory work again (#2236) 2024-05-28 14:24:10 +00:00

README.md

The file accounts_cache.nim has been relocated

Background

The new LedgerRef module unifies different implementations of the legacy accounts_cache. It is intended to be used as new base method for all of the AccountsCache implementations. The legacy accounts_cache version has been removed.

This was needed to accomodate for different CoreDb API paradigms. While the overloaded legacy AccountsCache implementation is just a closure based wrapper around the accounts_cache module, the overloaded AccountsLedgerRef is a closure based wrapper around the accounts_ledger module with the new CoreDb API returning Result[] values and saparating the meaning of trie root hash and trie root reference.

This allows to use the legacy hexary database (with the new CoreDb API) as well as the Aristo database (only supported on new API.)

Instructions

Legacy notation LedgerRef replacement Comment
import accounts_cache import ledger preferred method,
AccountsCache.init(..) AccountsCache.init(..) wraps AccountsCache
methods
or
import ledger/accounts_cache stay with legacy
AccountsCache.init(..) version of
AccountsCache
--
fn(ac: AccountsCache) fn(ac: LedgerRef) function example for
preferred wrapper
or method
fn(ac: AccountsCache) with legacy version,
no change here

The constructor decides which CoreDb API is to be used

Legacy API constructor new API Constructor
import ledger import ledger
let w = AccountsCache.init(..) let w = AccountsLedgerRef.init(..)