status-im
/
nimbus-eth1
mirror of https://github.com/status-im/nimbus-eth1.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth1/nimbus/db/aristo.nim

52 lines
1.1 KiB
Nim
Raw Normal View History

Aristo db transaction based interface (#1628) * Provide transaction based interface for standard operations * Provide unit tests for new Aristo interface using transactions details: These new tests combine and replace several single-purpose tests. The now unused test sources will be kept for a while to be eventually removed.
2023-07-05 13:50:11 +00:00
# nimbus-eth1
# Copyright (c) 2021 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## Aristo DB -- Standard interface
## ===============================
##
{.push raises: [].}
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
import aristo/[
Aristo db integrate hashify into tx (#1679) * Renamed type `NoneBackendRef` => `VoidBackendRef` * Clarify names: `BE=filter+backend` and `UBE=backend (unfiltered)` why: Most functions used full names as `getVtxUnfilteredBackend()` or `getKeyBackend()`. After defining abbreviations (and its meaning) it seems easier to use `getVtxUBE()` and `getKeyBE()`. * Integrate `hashify()` process into transaction logic why: Is now transparent unless explicitly controlled. details: Cache changes imply setting a `dirty` flag which in turn triggers `hashify()` processing in transaction and `pack()` directives. * Removed `aristo_tx.exec()` directive why: Inconsistent implementation, functionality will be provided with a different paradigm.
2023-08-11 17:23:57 +00:00
aristo_constants, aristo_delete, aristo_fetch, aristo_init,
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
aristo_merge, aristo_nearby, aristo_tx, aristo_utils, aristo_walk]
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
export
Aristo db integrate hashify into tx (#1679) * Renamed type `NoneBackendRef` => `VoidBackendRef` * Clarify names: `BE=filter+backend` and `UBE=backend (unfiltered)` why: Most functions used full names as `getVtxUnfilteredBackend()` or `getKeyBackend()`. After defining abbreviations (and its meaning) it seems easier to use `getVtxUBE()` and `getKeyBE()`. * Integrate `hashify()` process into transaction logic why: Is now transparent unless explicitly controlled. details: Cache changes imply setting a `dirty` flag which in turn triggers `hashify()` processing in transaction and `pack()` directives. * Removed `aristo_tx.exec()` directive why: Inconsistent implementation, functionality will be provided with a different paradigm.
2023-08-11 17:23:57 +00:00
aristo_constants, aristo_delete, aristo_fetch, aristo_init,
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
aristo_merge, aristo_nearby, aristo_tx, aristo_utils, aristo_walk
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
import
aristo/aristo_transcode
export
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
append, read, serialise
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
Core db update providing additional results code interface (#1776) * Split `core_db/base.nim` into several sources * Rename `core_db/legacy.nim` => `core_db/legacy_db.nim` * Update `CoreDb` API, dual methods returning `Result[]` or plain value detail: Plain value methods implemet the legacy API, they defect on error results * Redesign `CoreDB` direct backend access why: Made the `backend` directive integral part of the API * Discontinue providing unused or otherwise available functions details: + setTransactionID() removed, not used and not easily replicable in Aristo + maybeGet() removed, available via direct backend access + newPhk() removed, never used & was experimental anyway * Update/reorg backend API why: + Added error print function `$$()` + General descriptor completion (and optional validation) via `bless()` * Update `Aristo`/`Kvt` exception handling why: Avoid `CatchableError` exceptions, rather pass them as error code where appropriate. * More `CoreDB` compliant `Aristo` and `Kvt` methods details: + Providing functions like `contains()`, `getVtxRc()` (returns `Result[]`). + Additional error code: `NotImplemented` * Rewrite/reorg of Aristo DB constructor why: Previously used global object `DefaultQidLayoutRef` as default initialiser. This object was created at compile time which lead to non-gc safe functions. * Update nimbus/db/core_db/legacy_db.nim Co-authored-by: Kim De Mey <kim.demey@gmail.com> * Update nimbus/db/aristo/aristo_transcode.nim Co-authored-by: Kim De Mey <kim.demey@gmail.com> * Update nimbus/db/core_db/legacy_db.nim Co-authored-by: Kim De Mey <kim.demey@gmail.com> --------- Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2023-09-26 09:21:13 +00:00
import
aristo/aristo_get
export
getKeyRc
Aristo db implement filter storage scheduler (#1713) * Rename FilterID => QueueID why: The current usage does not identify a particular filter but uses it as storage tag to manage it on the database (to be organised in a set of FIFOs or queues.) * Split `aristo_filter` source into sub-files why: Make space for filter management API * Store filter queue IDs in pairs on the backend why: Any pair will will describe a FIFO accessed by bottom/top IDs * Reorg some source file names why: The "aristo_" prefix for make local/private files is tedious to use, so removed. * Implement filter slot scheduler details: Filters will be stored on the database on cascaded FIFOs. When a FIFO queue is full, some filter items are bundled together and stored on the next FIFO.
2023-08-25 22:53:59 +00:00
import aristo/aristo_desc/[desc_identifiers, desc_structural]
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
export
Aristo db use filter betw backend and tx cache (#1678) * Provide deep copy for each transaction layer why: Localising changes. Selective deep copy was just overlooked. * Generalise vertex ID generator state reorg function `vidReorg()` why: makes it somewhat easier to handle when saving layers. * Provide dummy back end descriptor `NoneBackendRef` * Optional read-only filter between backend and transaction cache why: Some staging area for accumulating changes to the backend DB. This will eventually be an access layer for emulating a backend with multiple/historic state roots. * Re-factor `persistent()` with filter between backend/tx-cache => `stow()` why: The filter provides an abstraction from the physically stored data on disk. So, there can be several MPT instances using the same disk data with different state roots. Of course, all the MPT instances should not differ too much for practical reasons :). TODO: Filter administration tools need to be provided.
2023-08-10 20:01:28 +00:00
AristoAccount,
PayloadRef,
PayloadType,
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
desc_identifiers,
Aristo db use filter betw backend and tx cache (#1678) * Provide deep copy for each transaction layer why: Localising changes. Selective deep copy was just overlooked. * Generalise vertex ID generator state reorg function `vidReorg()` why: makes it somewhat easier to handle when saving layers. * Provide dummy back end descriptor `NoneBackendRef` * Optional read-only filter between backend and transaction cache why: Some staging area for accumulating changes to the backend DB. This will eventually be an access layer for emulating a backend with multiple/historic state roots. * Re-factor `persistent()` with filter between backend/tx-cache => `stow()` why: The filter provides an abstraction from the physically stored data on disk. So, there can be several MPT instances using the same disk data with different state roots. Of course, all the MPT instances should not differ too much for practical reasons :). TODO: Filter administration tools need to be provided.
2023-08-10 20:01:28 +00:00
`==`
Aristo db transaction based interface (#1628) * Provide transaction based interface for standard operations * Provide unit tests for new Aristo interface using transactions details: These new tests combine and replace several single-purpose tests. The now unused test sources will be kept for a while to be eventually removed.
2023-07-05 13:50:11 +00:00
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
import
aristo/aristo_desc
Aristo db transaction based interface (#1628) * Provide transaction based interface for standard operations * Provide unit tests for new Aristo interface using transactions details: These new tests combine and replace several single-purpose tests. The now unused test sources will be kept for a while to be eventually removed.
2023-07-05 13:50:11 +00:00
export
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
AristoDbAction,
Aristo db transaction based interface (#1628) * Provide transaction based interface for standard operations * Provide unit tests for new Aristo interface using transactions details: These new tests combine and replace several single-purpose tests. The now unused test sources will be kept for a while to be eventually removed.
2023-07-05 13:50:11 +00:00
AristoDbRef,
AristoError,
Core db aristo and kvt updates preparing for integration (#1760) * Kvt: Implemented multi-descriptor access on the same backend why: This behaviour mirrors the one of Aristo and can be used for simultaneous transactions on Aristo + Kvt * Kvt: Update database iterators why: Forgot to run on the top layer first * Kvt: Misc fixes * Aristo, use `openArray[byte]` rather than `Blob` in prototype * Aristo, by default hashify right after cloning descriptor why: Typically, a completed descriptor is expected after cloning. Hashing can be suppressed by argument flag. * Aristo provides `replicate()` iterator, similar to legacy `replicate()` * Aristo API fixes and updates * CoreDB: Rename `legacy_persistent` => `legacy_rocksdb` why: More systematic, will be in line with Aristo DB which might have more than one persistent backends * CoreDB: Prettify API sources why: Better to read and maintain details: Annotating with custom pragmas which cleans up the prototypes * CoreDB: Update MPT/put() prototype allowing `CatchableError` why: Will be needed for Aristo API (legacy is OK with `RlpError`)
2023-09-18 20:20:28 +00:00
AristoTxRef,
forget,
Aristo db refactor tx paradim (#1674) * Better error handling why: Bail out on some error as early as possible before any changes. * Implement `fetch()` as opposite of `merge()` rationale: In the `Aristo` realm, the action named `fetch()` and `merge()` indicate leaf value related actions on the MPT, while actions `get()` and `put()` handle vertex or hash key related operations that constitute the MPT. * Re-factor `merge()` prototypes why: The most used variant of `merge()` should have the simplest prototype. * Persistent DB constructor needs to import `aristo/aristo_init/persistent` why: Most applications use memory DB anyway. This avoids linking `-lrocksdb` or any other back end libraries by default. * Re-factor transaction module why: Got the paradigm wrong. The transaction descriptor did replace the database one but should be handled separately.
2023-08-07 17:45:23 +00:00
isValid
Aristo db transaction based interface (#1628) * Provide transaction based interface for standard operations * Provide unit tests for new Aristo interface using transactions details: These new tests combine and replace several single-purpose tests. The now unused test sources will be kept for a while to be eventually removed.
2023-07-05 13:50:11 +00:00
# End