Commit Graph

8 Commits

Author SHA1 Message Date
Jacek Sieka eb041abba7
avoid unnecessary memory allocations and lookups (#2334)
* use `withValue` instead of `hasKey` + `[]`
* avoid `@` et al
* parse database data inside `onData` instead of making seq then parsing
2024-06-11 11:38:58 +02:00
Jordan Hrycaj a347291413
Aristo use rocksdb cf instead of key pfx (#2332)
* Use RocksDb column families instead of a prefixed single column

why:
  Better performance

* Use structural objects `VertexRef` and `HashKey` in LRU cache for RocksDb

why:
  Avoids repeated de/serialisation
2024-06-10 12:04:22 +00:00
Jacek Sieka 0a49833d69
avoid a few more copies (#2215) 2024-05-24 11:27:17 +02:00
Jordan Hrycaj b9187e0493
Aristo selective read cashing for rocksdb backend (#2145)
* Aristo+Kvt: Better RocksDB profiling

why:
  Providing more detailed information, mainly for `Aristo`

* Aristo: Renamed journal `stats()` to `capacity()`

why:
  `Stats()` was a misnomer

* Aristo: Provide backend read caches for key and vertex IDs

why:
  Dedicated LRU caching for particular types gives a throughput advantage.
  The sizes of the LRU queues used for caching are currently constant
  but might be adjusted at a later time.

* Fix copyright year
2024-04-22 19:02:22 +00:00
Jordan Hrycaj d6a4205324
Aristo update rocksdb backend drivers (#2135)
* Aristo+RocksDB: Update backend drivers

why:
  RocksDB update allows use some of the newly provided methods which
  were previously implemented by using the very C backend (for the lack
  of NIM methods.)

* Aristo+RocksDB: Simplify drivers wrapper

* Kvt: Update backend drivers and wrappers similar to `Aristo`

* Aristo+Kvm: Use column families for RocksDB

* Aristo+MemoryDB: Code cosmetics

* Aristo: Provide guest column family for export

why:
  So `Kvt` can piggyback on `Aristo` so there avoiding to run a second
  DBMS system in parallel.

* Kvt: Provide import mechanism for RoksDB guest column family

why:
  So `Kvt` can piggyback on `Aristo` so there avoiding to run a second
   DBMS system in parallel.

* CoreDb+Aristo: Run persistent `Kvt` DB piggybacked on `Aristo`

why:
  Avoiding to run two DBMS systems in parallel.

* Fix copyright year

* Ditto
2024-04-16 20:39:11 +00:00
Jordan Hrycaj 6e0397e276
Aristo and ledger small updates (#1888)
* Fix debug noise in `hashify()` for perfectly normal situation

why:
  Was previously considered a fixable error

* Fix test sample file names

why:
  The larger test file `goerli68161.txt.gz` is already in the local
  archive. So there is no need to use the smaller one from the external
  repo.

* Activate `accounts_cache` module from `db/ledger`

why:
  A copy of the original `accounts_cache.nim` source to be integrated
  into the `Ledger` module wrapper which allows to switch between
  different `accounts_cache` implementations unser tha same API.

details:
  At a later state, the `db/accounts_cache.nim` wrapper will be
  removed so that there is only one access to that module via
  `db/ledger/accounts_cache.nim`.

* Fix copyright headers in source code
2023-11-08 16:52:25 +00:00
Jordan Hrycaj 8e00143313
Aristo db code massage n cosmetics (#1745)
* Rewrite remaining `AristoError` return code into `Result[void,AristoError]`

why:
  Better code maintenance

* Update import sections

* Update Aristo DB paths

why:
 More systematic so directory can be shared with other DB types

* More cosmetcs

* Update unit tests runners

why:
  Proper handling of persistent and mem-only DB. The latter can be
  consistently triggered by an empty DB path.
2023-09-12 19:45:12 +01:00
Jordan Hrycaj 465d694834
Aristo db implement filter storage scheduler (#1713)
* Rename FilterID => QueueID

why:
  The current usage does not identify a particular filter but uses it as
  storage tag to manage it on the database (to be organised in a set of
  FIFOs or queues.)

* Split `aristo_filter` source into sub-files

why:
  Make space for filter management API

* Store filter queue IDs in pairs on the backend

why:
  Any pair will will describe a FIFO accessed by bottom/top IDs

* Reorg some source file names

why:
  The "aristo_" prefix for make local/private files is tedious to
  use, so removed.

* Implement filter slot scheduler

details:
  Filters will be stored on the database on cascaded FIFOs. When a FIFO
  queue is full, some filter items are bundled together and stored on the
  next FIFO.
2023-08-25 23:53:59 +01:00