status-im
/
nimbus-eth1
mirror of https://github.com/status-im/nimbus-eth1.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth1/nimbus/db/kvt/kvt_init/rocks_db/rdb_desc.nim

61 lines
1.8 KiB
Nim
Raw Normal View History

Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
# nimbus-eth1
Update Nimbus codebase to use the new nim-rocksdb API. (#2054) * Bump nim-rocksdb. * Update codebase to use latest nim-rocksdb API. * Update copyright notices. * Fix memory leak due to allocCStringArray without deallocCStringArray. * Improve kvstore_rocksdb code. * Refactor and cleanup RocksStoreRef. * Update nim-rocksdb submodule to latest.
2024-03-05 04:54:42 +00:00
# Copyright (c) 2023-2024 Status Research & Development GmbH
Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## Rocks DB internal driver descriptor
## ===================================
{.push raises: [].}
import
Core db fix legacy db root vertex fetcher (#1899) * Using different `tmp` directories for `Kvt` and `Aristo` why: Closing one database would leave the other set of directories incomplete. * Code cosmetics, silence compiler * Fix typo `EMPTY_ROOT_HASH` vs. `EMPTY_CODE_HASH` * Fix copyright years
2023-11-20 20:22:27 +00:00
std/os,
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
../../kvt_desc,
Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
rocksdb
type
RdbInst* = object
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
store*: KvtCfStore ## Rocks DB database handler
Aristo update rocksdb backend drivers (#2135) * Aristo+RocksDB: Update backend drivers why: RocksDB update allows use some of the newly provided methods which were previously implemented by using the very C backend (for the lack of NIM methods.) * Aristo+RocksDB: Simplify drivers wrapper * Kvt: Update backend drivers and wrappers similar to `Aristo` * Aristo+Kvm: Use column families for RocksDB * Aristo+MemoryDB: Code cosmetics * Aristo: Provide guest column family for export why: So `Kvt` can piggyback on `Aristo` so there avoiding to run a second DBMS system in parallel. * Kvt: Provide import mechanism for RoksDB guest column family why: So `Kvt` can piggyback on `Aristo` so there avoiding to run a second DBMS system in parallel. * CoreDb+Aristo: Run persistent `Kvt` DB piggybacked on `Aristo` why: Avoiding to run two DBMS systems in parallel. * Fix copyright year * Ditto
2024-04-16 20:39:11 +00:00
session*: WriteBatchRef ## For batched `put()`
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
basePath*: string ## Database directory
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
delayedPersist*: KvtDbRef ## Enable next proggyback write session
KvtCFs* = enum
## Column family symbols/handles and names used on the database
KvtGeneric = "KvtGen" ## Generic column family
Use RocksDb column family handles instead of name strings. (#2418) * Bump RocksDb to latest and update Nimbus database to pass column family handles to RocksDb API. * Bump RocksDb version.
2024-06-27 08:51:43 +00:00
KvtCfStore* = array[KvtCFs, ColFamilyReadWrite]
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
## List of column family handlers
Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
const
Aristo update rocksdb backend drivers (#2135) * Aristo+RocksDB: Update backend drivers why: RocksDB update allows use some of the newly provided methods which were previously implemented by using the very C backend (for the lack of NIM methods.) * Aristo+RocksDB: Simplify drivers wrapper * Kvt: Update backend drivers and wrappers similar to `Aristo` * Aristo+Kvm: Use column families for RocksDB * Aristo+MemoryDB: Code cosmetics * Aristo: Provide guest column family for export why: So `Kvt` can piggyback on `Aristo` so there avoiding to run a second DBMS system in parallel. * Kvt: Provide import mechanism for RoksDB guest column family why: So `Kvt` can piggyback on `Aristo` so there avoiding to run a second DBMS system in parallel. * CoreDb+Aristo: Run persistent `Kvt` DB piggybacked on `Aristo` why: Avoiding to run two DBMS systems in parallel. * Fix copyright year * Ditto
2024-04-16 20:39:11 +00:00
BaseFolder* = "nimbus" ## Same as for Legacy DB
DataFolder* = "kvt" ## Legacy DB has "data"
Core db fix legacy db root vertex fetcher (#1899) * Using different `tmp` directories for `Kvt` and `Aristo` why: Closing one database would leave the other set of directories incomplete. * Code cosmetics, silence compiler * Fix typo `EMPTY_ROOT_HASH` vs. `EMPTY_CODE_HASH` * Fix copyright years
2023-11-20 20:22:27 +00:00
# ------------------------------------------------------------------------------
# Public functions
# ------------------------------------------------------------------------------
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
template logTxt*(info: static[string]): static[string] =
"RocksDB/" & info
Core db fix legacy db root vertex fetcher (#1899) * Using different `tmp` directories for `Kvt` and `Aristo` why: Closing one database would leave the other set of directories incomplete. * Code cosmetics, silence compiler * Fix typo `EMPTY_ROOT_HASH` vs. `EMPTY_CODE_HASH` * Fix copyright years
2023-11-20 20:22:27 +00:00
func baseDir*(rdb: RdbInst): string =
rdb.basePath / BaseFolder
func dataDir*(rdb: RdbInst): string =
rdb.baseDir / DataFolder
Triggered write event for kvt (#2351) * bump rockdb * Rename `KVT` objects related to filters according to `Aristo` naming details: filter* => delta* roFilter => balancer * Compulsory error handling if `persistent()` fails * Add return code to `reCentre()` why: Might eventually fail if re-centring is blocked. Some logic will be added in subsequent patch sets. * Add column families from earlier session to rocksdb in opening procedure why: All previously used CFs must be declared when re-opening an existing database. * Update `init()` and add rocksdb `reinit()` methods for changing parameters why: Opening a set column families (with different open options) must span at least the ones that are already on disk. * Provide write-trigger-event interface into `Aristo` backend why: This allows to save data from a guest application (think `KVT`) to get synced with the write cycle so the guest and `Aristo` save all atomically. * Use `KVT` with new column family interface from `Aristo` * Remove obsolete guest interface * Implement `KVT` piggyback on `Aristo` backend * CoreDb: Add separate `KVT`/`Aristo` backend mode for debugging * Remove `rocks_db` import from `persist()` function why: Some systems (i.p `fluffy` and friends) use the `Aristo` memory backend emulation and do not link against rocksdb when building the application. So this should fix that problem.
2024-06-13 18:15:11 +00:00
template baseDb*(rdb: RdbInst): RocksDbReadWriteRef =
rdb.store[KvtGeneric].db
Simple stupid key-value table companion for Aristo DB (#1746) why: Additional tables needed for the `CoreDB` object with separate key-value table and MPT. details: + Stripped down copy of Aristo DB to have a similar look'n feel. Otherwise it is just a posh way for accessing `Table` objects or `RocksDB` data. + No unit tests yet, will be tested on the go.
2023-09-12 18:44:45 +00:00
# ------------------------------------------------------------------------------
# End
# ------------------------------------------------------------------------------