status-im
/
nimbus-eth1
mirror of https://github.com/status-im/nimbus-eth1.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth1/nimbus/db/ledger.nim

37 lines
1.3 KiB
Nim
Raw Normal View History

Ledger abstraction for accounts cache (#1824) * Provide TDD/debug facility for inspecting `persistBlocks()` working detail: + Make sure that the last block of a test sample is the first batch item in `persistBlocks()`. + Additionally, allow `AccountsCache` API tracing by setting the flag `extraTraceMessages = true` in the file `accounts_cache.nim` * Overload AccountsCache by abstraction wrapper details: Can facilitate CoreDb API switch, details in `ledger/README.md`.
2023-10-18 19:27:22 +00:00
# Nimbus
Core db update storage root management for sub tries (#1964) * Aristo: Re-phrase `LayerDelta` and `LayerFinal` as object references why: Avoids copying in some cases * Fix copyright header * Aristo: Verify `leafTie.root` function argument for `merge()` proc why: Zero root will lead to inconsistent DB entry * Aristo: Update failure condition for hash labels compiler `hashify()` why: Node need not be rejected as long as links are on the schedule. In that case, `redo[]` is to become `wff.base[]` at a later stage. This amends an earlier fix, part of #1952 by also testing against the target nodes of the `wff.base[]` sets. * Aristo: Add storage root glue record to `hashify()` schedule why: An account leaf node might refer to a non-resolvable storage root ID. Storage root node chains will end up at the storage root. So the link `storage-root->account-leaf` needs an extra item in the schedule. * Aristo: fix error code returned by `fetchPayload()` details: Final error code is implied by the error code form the `hikeUp()` function. * CoreDb: Discard `createOk` argument in API `getRoot()` function why: Not needed for the legacy DB. For the `Arsto` DB, a lazy approach is implemented where a stprage root node is created on-the-fly. * CoreDb: Prevent `$$` logging in some cases why: Logging the function `$$` is not useful when it is used for internal use, i.e. retrieving an an error text for logging. * CoreDb: Add `tryHashFn()` to API for pretty printing why: Pretty printing must not change the hashification status for the `Aristo` DB. So there is an independent API wrapper for getting the node hash which never updated the hashes. * CoreDb: Discard `update` argument in API `hash()` function why: When calling the API function `hash()`, the latest state is always wanted. For a version that uses the current state as-is without checking, the function `tryHash()` was added to the backend. * CoreDb: Update opaque vertex ID objects for the `Aristo` backend why: For `Aristo`, vID objects encapsulate a numeric `VertexID` referencing a vertex (rather than a node hash as used on the legacy backend.) For storage sub-tries, there might be no initial vertex known when the descriptor is created. So opaque vertex ID objects are supported without a valid `VertexID` which will be initalised on-the-fly when the first item is merged. * CoreDb: Add pretty printer for opaque vertex ID objects * Cosmetics, printing profiling data * CoreDb: Fix segfault in `Aristo` backend when creating MPT descriptor why: Missing initialisation error * CoreDb: Allow MPT to inherit shared context on `Aristo` backend why: Creates descriptors with different storage roots for the same shared `Aristo` DB descriptor. * Cosmetics, update diagnostic message items for `Aristo` backend * Fix Copyright year
2024-01-11 19:11:38 +00:00
# Copyright (c) 2023-2024 Status Research & Development GmbH
Ledger abstraction for accounts cache (#1824) * Provide TDD/debug facility for inspecting `persistBlocks()` working detail: + Make sure that the last block of a test sample is the first batch item in `persistBlocks()`. + Additionally, allow `AccountsCache` API tracing by setting the flag `extraTraceMessages = true` in the file `accounts_cache.nim` * Overload AccountsCache by abstraction wrapper details: Can facilitate CoreDb API switch, details in `ledger/README.md`.
2023-10-18 19:27:22 +00:00
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## Unifies different ledger management APIs. All ledger objects are
## derived from the base objects
## ::
## LedgerSpRef => SavePoint, overloaded SavePoint etc
##
{.push raises: [].}
import
eth/common,
./core_db,
Separate config for core db and ledger (#2479) * Updates and corrections * Extract `CoreDb` configuration from `base.nim` into separate module why: This makes it easier to avoid circular imports, in particular when the capture journal (aka tracer) is revived. * Extract `Ledger` configuration from `base.nim` into separate module why: This makes it easier to avoid circular imports (if any.) also: Move `accounts_ledger.nim` file to sub-folder `backend`. That way the layout resembles that of the `core_db`.
2024-07-12 13:12:25 +00:00
./ledger/backend/accounts_ledger,
Separate constructor helpers for core db and ledger (#2480) * Extract `CoreDb` constructor helpers from `base.nim` into separate module why: This makes it easier to avoid circular imports. * Extract `Ledger` constructor helpers from `base.nim` into separate module why: Move `accounts_ledger.nim` file to sub-folder `backend`. That way the layout resembles that of the `core_db`.
2024-07-12 19:32:31 +00:00
./ledger/base/[base_config, base_desc, base_helpers],
Separate config for core db and ledger (#2479) * Updates and corrections * Extract `CoreDb` configuration from `base.nim` into separate module why: This makes it easier to avoid circular imports, in particular when the capture journal (aka tracer) is revived. * Extract `Ledger` configuration from `base.nim` into separate module why: This makes it easier to avoid circular imports (if any.) also: Move `accounts_ledger.nim` file to sub-folder `backend`. That way the layout resembles that of the `core_db`.
2024-07-12 13:12:25 +00:00
./ledger/[base, base_iterators]
Core db aristo hasher profiling and timing improvement (#1938) * Explicitly use shared `Kvt` table on `Ledger` and `Clique` lookup. why: Speeds up lookup time with `Aristo` backend. For writing `Clique` data, the `Companion` model allows to write `Clique` data past the database locked by evm transactions. * Implement `CoreDb` profiling with API tracking why: Chasing time spent per APT procs ... * Implement `Ledger` profiling with API tracking why: Chasing time spent per APT procs ... * Always hashify when commiting or storing why: A dirty cache makes no sense when committing * Make sure that a zero key is created when adding/updating vertices why: This is an error fix mainly for edge cases. A typical error was that the root key got deleted when there were only a few vertices left on the DB. * Need all created and changed vertices zero-keyed on the cache why: A zero key (i.e. empty Merkle hash) indicates that a vertex key needs to be updated. This would not be needed immediately after a merge as there is an actual leaf path on the cache layer. But after subsequent merge and delete operations this information might get blurred. * Re-org hashing algorithm why: Apart from errors, the previous implementation was too slow for two reasons: + some control hashes were calculated for debugging (now all verification is done in `aristo_check` module) + the leaf paths stored on the cache are used to build the labelling (aka hashing) schedule; there paths were accumulated over successive hash sessions although it is clear that all keys were generated, already
2023-12-12 17:47:41 +00:00
Don't write slot hashes on import (#2564) The reverse slot hash mechanism causes quite a bit of database traffic but is broadly not useful except for iterating the storage of an account, something that a validator never does (it's used by the tracers). This flag adds one more thing that is not stored in the database, to be explored more comprehensively when designing full, validator and archive modes with different pruning options in the future. `ldb` says this is 60gb of data (!): ``` ldb --db=. --ignore_unknown_options --column_family=KvtGen approxsize --hex --from=0x05 --to=0x05ffffffffffffffffffffffffffffffffffffffffffffff 66488353954 ```
2024-08-16 06:22:51 +00:00
export AccountsLedgerRef, base, base_config, base_iterators
Ledger abstraction for accounts cache (#1824) * Provide TDD/debug facility for inspecting `persistBlocks()` working detail: + Make sure that the last block of a test sample is the first batch item in `persistBlocks()`. + Additionally, allow `AccountsCache` API tracing by setting the flag `extraTraceMessages = true` in the file `accounts_cache.nim` * Overload AccountsCache by abstraction wrapper details: Can facilitate CoreDb API switch, details in `ledger/README.md`.
2023-10-18 19:27:22 +00:00
Fix default pruning for ledger and update core db and ledger logging (#1861) * Make sure that storage tries are not pruned (by default) on the new Ledger API why: Pruning might kill some unwanted entries from storage tries ending up with an unstable database leading to crashes. * Implement `CoreDb` and `LedgerRef` API tracing details: + Locally enabled at compile time via constants `ProvideCoreDbLegacyAPI` and `EnableApiTracking` in either `base.nim` source + If enabled it can be selectively turned on/off via public switches in the `CoreDb` descriptor. * Allow suppressing opportunistic `ifNecessaryGetXxx()` functions why: Better troubleshooting when the system crashes (assertions will then most probably happen outside an `async` function.)
2023-10-25 14:03:09 +00:00
# ------------------------------------------------------------------------------
# Public constructor
# ------------------------------------------------------------------------------
Use stateRoot/storageRoot more consistently (#2791) * prefer the spec-derived name where possible * don't pass stateRoot to LedgerRef and friends (it doesn't do anything) * add deprecation warning in graphql - it needs updating to use forkedchain instead
2024-10-27 18:56:28 +00:00
proc init*(_: type LedgerRef, db: CoreDbRef, storeSlotHash: bool = false): LedgerRef =
LedgerRef(ac: AccountsLedgerRef.init(db, storeSlotHash)).bless(db)
Fix default pruning for ledger and update core db and ledger logging (#1861) * Make sure that storage tries are not pruned (by default) on the new Ledger API why: Pruning might kill some unwanted entries from storage tries ending up with an unstable database leading to crashes. * Implement `CoreDb` and `LedgerRef` API tracing details: + Locally enabled at compile time via constants `ProvideCoreDbLegacyAPI` and `EnableApiTracking` in either `base.nim` source + If enabled it can be selectively turned on/off via public switches in the `CoreDb` descriptor. * Allow suppressing opportunistic `ifNecessaryGetXxx()` functions why: Better troubleshooting when the system crashes (assertions will then most probably happen outside an `async` function.)
2023-10-25 14:03:09 +00:00
Ledger abstraction for accounts cache (#1824) * Provide TDD/debug facility for inspecting `persistBlocks()` working detail: + Make sure that the last block of a test sample is the first batch item in `persistBlocks()`. + Additionally, allow `AccountsCache` API tracing by setting the flag `extraTraceMessages = true` in the file `accounts_cache.nim` * Overload AccountsCache by abstraction wrapper details: Can facilitate CoreDb API switch, details in `ledger/README.md`.
2023-10-18 19:27:22 +00:00
# ------------------------------------------------------------------------------
# End
# ------------------------------------------------------------------------------