status-im
/
nimbus-eth1
mirror of https://github.com/status-im/nimbus-eth1.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth1/nimbus/db/aristo/aristo_init/aristo_rocksdb/rdb_desc.nim

61 lines
2.0 KiB
Nim
Raw Normal View History

Aristo db with storage backends (#1603) * Generalised Aristo DB constructor for any type of backend details: * Records to be deleted are represented as key-void (rather than key-value) pairs by the put-function arguments * Allow direct driver access, iterators as example implementation and for testing. * Provide backend storage interface details: Stores the top layer onto backend tables * Implemented Rocks DB backend details: Transaction based `put()` functionality Iterators (based on direct RocksDB access)
2023-06-20 13:26:25 +00:00
# nimbus-eth1
# Copyright (c) 2021 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## Rocks DB internal driver descriptor
## ===================================
{.push raises: [].}
import
std/tables,
eth/common,
rocksdb,
stint,
../../aristo_desc,
../aristo_init_common.nim
type
RdbInst* = object
store*: RocksDBInstance ## Rocks DB database handler
basePath*: string ## Database directory
# Low level Rocks DB access for bulk store
envOpt*: rocksdb_envoptions_t
impOpt*: rocksdb_ingestexternalfileoptions_t
RdbKey* = array[1 + sizeof VertexID, byte]
## Sub-table key, <pfx> + VertexID
Aristo db implement filter serialisation for storage (#1695) * Remove concept of empty/blind filters why: Not needed. A non-existent filter is is coded as a nil reference. * Slightly generalised backend iterators why: * VertexID as key for the ID generator state makes no sense * there will be more tables addressed by non-VertexID keys * Store serialised/blobified vertices on memory backend why: This is more in line with the RocksDB backend so more appropriate for testing when comparing behaviour. For a speedy memory database, a backend-less variant should be used. * Drop the `Aristo` prefix from names `AristoLayerRef`, etc. * Suppress compiler warning why: duplicate imports * Add filter serialisation transcoder why: Will be used as storage format
2023-08-18 19:46:55 +00:00
RdbTabs* = array[StorageType,Table[VertexID,Blob]]
Aristo db with storage backends (#1603) * Generalised Aristo DB constructor for any type of backend details: * Records to be deleted are represented as key-void (rather than key-value) pairs by the put-function arguments * Allow direct driver access, iterators as example implementation and for testing. * Provide backend storage interface details: Stores the top layer onto backend tables * Implemented Rocks DB backend details: Transaction based `put()` functionality Iterators (based on direct RocksDB access)
2023-06-20 13:26:25 +00:00
## Combined table for caching data to be stored/updated
const
BaseFolder* = "nimbus" # Same as for Legacy DB has "backups"
DataFolder* = "aristo" # Legacy DB has "data"
BackupFolder* = "history" # Legacy DB has "backups"
TempFolder* = "tmp" # Not used with legacy DB
SstCache* = "bulkput" # Rocks DB bulk load file name in temp folder
# ------------------------------------------------------------------------------
# Public functions
# ------------------------------------------------------------------------------
Aristo db implement filter serialisation for storage (#1695) * Remove concept of empty/blind filters why: Not needed. A non-existent filter is is coded as a nil reference. * Slightly generalised backend iterators why: * VertexID as key for the ID generator state makes no sense * there will be more tables addressed by non-VertexID keys * Store serialised/blobified vertices on memory backend why: This is more in line with the RocksDB backend so more appropriate for testing when comparing behaviour. For a speedy memory database, a backend-less variant should be used. * Drop the `Aristo` prefix from names `AristoLayerRef`, etc. * Suppress compiler warning why: duplicate imports * Add filter serialisation transcoder why: Will be used as storage format
2023-08-18 19:46:55 +00:00
proc toRdbKey*(vid: VertexID; pfx: StorageType): Rdbkey =
Aristo db with storage backends (#1603) * Generalised Aristo DB constructor for any type of backend details: * Records to be deleted are represented as key-void (rather than key-value) pairs by the put-function arguments * Allow direct driver access, iterators as example implementation and for testing. * Provide backend storage interface details: Stores the top layer onto backend tables * Implemented Rocks DB backend details: Transaction based `put()` functionality Iterators (based on direct RocksDB access)
2023-06-20 13:26:25 +00:00
let vidKey = vid.uint64.toBytesBE
result[0] = pfx.ord.byte
copyMem(addr result[1], unsafeAddr vidKey, sizeof vidKey)
Aristo db implement filter serialisation for storage (#1695) * Remove concept of empty/blind filters why: Not needed. A non-existent filter is is coded as a nil reference. * Slightly generalised backend iterators why: * VertexID as key for the ID generator state makes no sense * there will be more tables addressed by non-VertexID keys * Store serialised/blobified vertices on memory backend why: This is more in line with the RocksDB backend so more appropriate for testing when comparing behaviour. For a speedy memory database, a backend-less variant should be used. * Drop the `Aristo` prefix from names `AristoLayerRef`, etc. * Suppress compiler warning why: duplicate imports * Add filter serialisation transcoder why: Will be used as storage format
2023-08-18 19:46:55 +00:00
template toOpenArray*(vid: VertexID; pfx: StorageType): openArray[byte] =
Aristo db with storage backends (#1603) * Generalised Aristo DB constructor for any type of backend details: * Records to be deleted are represented as key-void (rather than key-value) pairs by the put-function arguments * Allow direct driver access, iterators as example implementation and for testing. * Provide backend storage interface details: Stores the top layer onto backend tables * Implemented Rocks DB backend details: Transaction based `put()` functionality Iterators (based on direct RocksDB access)
2023-06-20 13:26:25 +00:00
vid.toRdbKey(pfx).toOpenArray(0, sizeof VertexID)
# ------------------------------------------------------------------------------
# End
# ------------------------------------------------------------------------------