status-im/nimbus-eth1
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth1.git synced 2025-03-02 21:00:49 +00:00
Code Issues Projects Releases Wiki Activity
nimbus-eth1/execution_chain/db/aristo/aristo_persist.nim

33 lines
1.0 KiB
Nim
Raw Normal View History

Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 18:45:23 +01:00
# nimbus-eth1
Simplify txFrame protocol, improve persist performance (#3077) * Simplify txFrame protocol, improve persist performance To prepare forked-layers for further surgery to avoid the nesting tax, the commit/rollback style of interacting must first be adjusted, since it does not provide a point in time where the frame is "done" and goes from being actively written to, to simply waiting to be persisted or discarded. A collateral benefit of this change is that the scheme removes some complexity from the process by moving the "last saved block number" into txframe along with the actual state changes thus reducing the risk that they go "out of sync" and removing the "commit" consolidation responsibility from ForkedChain. * commit/rollback become checkpoint/dispose - since these are pure in-memory constructs, there's less error handling and there's no real "rollback" involved - dispose better implies that the instance cannot be used and we can more aggressively clear the memory it uses * simplified block number handling that moves to become part of txFrame just like the data that the block number references * avoid reparenting step by replacing the base instead of keeping a singleton instance * persist builds the set of changes from the bottom which helps avoid moving changes in the top layers through each ancestor level of the frame stack * when using an in-memory database in tests, allow the instance to be passed around to enable testing persist and reload logic
2025-02-17 02:51:56 +01:00
# Copyright (c) 2023-2025 Status Research & Development GmbH
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 18:45:23 +01:00
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## Aristo DB -- Transaction interface
## ==================================
##
{.push raises: [].}
import
Simplify shared rocksdb instance / write batch handling (#3063) By introducing the "shared rocksdb instance" concept to the backend, we can remove the "piggybacking" mode , thus reducing the complexity of database initialisation and opening the possibility of extending how write batching works across kvt/aristo. The change makes explicit the hidden shared state that was previously hiding in closures and provides the first step towards simplifying the "commit/persist" interface of coredb, preparing it for optimizations to reduce the "layering tax" that `forked-layers` introduced.
2025-02-14 09:40:22 +01:00
./[aristo_desc, aristo_tx_frame]
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 18:45:23 +01:00
# ------------------------------------------------------------------------------
Core db+aristo update recovery journal management (#2156) * Aristo: Allow to define/set `FilterID` for journal filter records why: After some changes, the `FilterID` is isomorphic to the `BlockNumber` scalar (well, the first 2^64 entries of a `BlockNumber`.) The needed change for `FilterID` is that the `FilterID(0)` value is valid part of the `FilterID` scalar. A non-valid `FilterID` entry is represented by `none(FilterID)`. * Aristo: Split off function `persist()` as persistent version of `stow()` why: In production, `stow(persistent=false,..)` is currently unused. So, using `persist()` rather than `stow(persistent=true,..)` improves readability and is better to maintain. * CoreDb+Aristo: Store block numbers in journal records why: This makes journal records searchable by block numbers * Aristo: Rename some journal related functions why: The name *journal* is more appropriate to api functions than something with *fifo* or *filter*. * CoreDb+Aristo: Update last/oldest journal state retrieval * CoreDb+Aristo: Register block number with state root in journal why: No need anymore for extra lookup table `stRootToBlockNum` which maps a storage root -> block number. * Aristo: Remove unused function `getFilUbe()` from api * CoreDb: Remove now unused virtual table `stRootToBlockNum` why: Was used to map a state root to a block number. This functionality is now embedded into the recovery journal backend. * Turn of API tracking (will fail on `fluffy`)
2024-04-29 20:17:17 +00:00
# Public functions: save to database
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 18:45:23 +01:00
# ------------------------------------------------------------------------------
Core db+aristo update recovery journal management (#2156) * Aristo: Allow to define/set `FilterID` for journal filter records why: After some changes, the `FilterID` is isomorphic to the `BlockNumber` scalar (well, the first 2^64 entries of a `BlockNumber`.) The needed change for `FilterID` is that the `FilterID(0)` value is valid part of the `FilterID` scalar. A non-valid `FilterID` entry is represented by `none(FilterID)`. * Aristo: Split off function `persist()` as persistent version of `stow()` why: In production, `stow(persistent=false,..)` is currently unused. So, using `persist()` rather than `stow(persistent=true,..)` improves readability and is better to maintain. * CoreDb+Aristo: Store block numbers in journal records why: This makes journal records searchable by block numbers * Aristo: Rename some journal related functions why: The name *journal* is more appropriate to api functions than something with *fifo* or *filter*. * CoreDb+Aristo: Update last/oldest journal state retrieval * CoreDb+Aristo: Register block number with state root in journal why: No need anymore for extra lookup table `stRootToBlockNum` which maps a storage root -> block number. * Aristo: Remove unused function `getFilUbe()` from api * CoreDb: Remove now unused virtual table `stRootToBlockNum` why: Was used to map a state root to a block number. This functionality is now embedded into the recovery journal backend. * Turn of API tracking (will fail on `fluffy`)
2024-04-29 20:17:17 +00:00
proc persist*(
Simplify txFrame protocol, improve persist performance (#3077) * Simplify txFrame protocol, improve persist performance To prepare forked-layers for further surgery to avoid the nesting tax, the commit/rollback style of interacting must first be adjusted, since it does not provide a point in time where the frame is "done" and goes from being actively written to, to simply waiting to be persisted or discarded. A collateral benefit of this change is that the scheme removes some complexity from the process by moving the "last saved block number" into txframe along with the actual state changes thus reducing the risk that they go "out of sync" and removing the "commit" consolidation responsibility from ForkedChain. * commit/rollback become checkpoint/dispose - since these are pure in-memory constructs, there's less error handling and there's no real "rollback" involved - dispose better implies that the instance cannot be used and we can more aggressively clear the memory it uses * simplified block number handling that moves to become part of txFrame just like the data that the block number references * avoid reparenting step by replacing the base instead of keeping a singleton instance * persist builds the set of changes from the bottom which helps avoid moving changes in the top layers through each ancestor level of the frame stack * when using an in-memory database in tests, allow the instance to be passed around to enable testing persist and reload logic
2025-02-17 02:51:56 +01:00
db: AristoDbRef;
Simplify shared rocksdb instance / write batch handling (#3063) By introducing the "shared rocksdb instance" concept to the backend, we can remove the "piggybacking" mode , thus reducing the complexity of database initialisation and opening the possibility of extending how write batching works across kvt/aristo. The change makes explicit the hidden shared state that was previously hiding in closures and provides the first step towards simplifying the "commit/persist" interface of coredb, preparing it for optimizations to reduce the "layering tax" that `forked-layers` introduced.
2025-02-14 09:40:22 +01:00
batch: PutHdlRef;
Simplify txFrame protocol, improve persist performance (#3077) * Simplify txFrame protocol, improve persist performance To prepare forked-layers for further surgery to avoid the nesting tax, the commit/rollback style of interacting must first be adjusted, since it does not provide a point in time where the frame is "done" and goes from being actively written to, to simply waiting to be persisted or discarded. A collateral benefit of this change is that the scheme removes some complexity from the process by moving the "last saved block number" into txframe along with the actual state changes thus reducing the risk that they go "out of sync" and removing the "commit" consolidation responsibility from ForkedChain. * commit/rollback become checkpoint/dispose - since these are pure in-memory constructs, there's less error handling and there's no real "rollback" involved - dispose better implies that the instance cannot be used and we can more aggressively clear the memory it uses * simplified block number handling that moves to become part of txFrame just like the data that the block number references * avoid reparenting step by replacing the base instead of keeping a singleton instance * persist builds the set of changes from the bottom which helps avoid moving changes in the top layers through each ancestor level of the frame stack * when using an in-memory database in tests, allow the instance to be passed around to enable testing persist and reload logic
2025-02-17 02:51:56 +01:00
txFrame: AristoTxRef;
Simplify shared rocksdb instance / write batch handling (#3063) By introducing the "shared rocksdb instance" concept to the backend, we can remove the "piggybacking" mode , thus reducing the complexity of database initialisation and opening the possibility of extending how write batching works across kvt/aristo. The change makes explicit the hidden shared state that was previously hiding in closures and provides the first step towards simplifying the "commit/persist" interface of coredb, preparing it for optimizations to reduce the "layering tax" that `forked-layers` introduced.
2025-02-14 09:40:22 +01:00
) =
Simplify txFrame protocol, improve persist performance (#3077) * Simplify txFrame protocol, improve persist performance To prepare forked-layers for further surgery to avoid the nesting tax, the commit/rollback style of interacting must first be adjusted, since it does not provide a point in time where the frame is "done" and goes from being actively written to, to simply waiting to be persisted or discarded. A collateral benefit of this change is that the scheme removes some complexity from the process by moving the "last saved block number" into txframe along with the actual state changes thus reducing the risk that they go "out of sync" and removing the "commit" consolidation responsibility from ForkedChain. * commit/rollback become checkpoint/dispose - since these are pure in-memory constructs, there's less error handling and there's no real "rollback" involved - dispose better implies that the instance cannot be used and we can more aggressively clear the memory it uses * simplified block number handling that moves to become part of txFrame just like the data that the block number references * avoid reparenting step by replacing the base instead of keeping a singleton instance * persist builds the set of changes from the bottom which helps avoid moving changes in the top layers through each ancestor level of the frame stack * when using an in-memory database in tests, allow the instance to be passed around to enable testing persist and reload logic
2025-02-17 02:51:56 +01:00
db.txFramePersist(batch, txFrame)
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 18:45:23 +01:00
# ------------------------------------------------------------------------------
# End
# ------------------------------------------------------------------------------
Reference in New Issue Copy Permalink