Commit Graph

9 Commits

Author SHA1 Message Date
Jordan Hrycaj 124ac064c6
Aristo db store filters on backend (#1703)
* Simplify RocksDB sub-tables iterator

* Implement `filter` storage on backend db

details:
  Unit tests working
2023-08-22 19:44:54 +01:00
Jordan Hrycaj b9a4fd3137
Aristo db update serialisation (#1700)
* Remove unused unit test sources

* Redefine and document serialised data records for Aristo backend

why:
  Unique record types determined by marker byte, i.e. the last byte of a
  serialisation record. This just needed some tweaking after adding new
  record types.
2023-08-21 19:18:06 +01:00
Jordan Hrycaj 445fa75251
Aristo db consolidate and clean up (#1699)
* Removed dedicated transcoder tests

why:
  will implicitely be provided by other tests:
  + encode/write -> hashify -> test_tx
  + decode/read -> merge raw nodes -> test_tx
  + de/blobfiy -> backend operations, taext_tx, test_backend, test_filter

* Clarify how the vertex ID generator state is accessed from the backend

why:
  This state is a list of unused vertex IDs. It was just stored somewhere
  on the backend which details were exposed when iterating over some
  sub-table(s).

  As there will be more such single information records, an admin
  sub-tables has been defined (formerly ID generator table) with dedicated
  access keys and type. Also, the iterator over the single ID generator
  state item has been removed. It must be accessed via the `get()`
  interface.

* Remove trailing space from file name

why:
  fixes windows bail out
2023-08-21 15:58:30 +01:00
Jordan Hrycaj 4c9141ffac
Aristo db implement filter serialisation for storage (#1695)
* Remove concept of empty/blind filters

why:
  Not needed. A non-existent filter is is coded as a nil reference.

* Slightly generalised backend iterators

why:
 * VertexID as key for the ID generator state makes no sense
 * there will be more tables addressed by non-VertexID keys

* Store serialised/blobified vertices on memory backend

why:
  This is more in line with the RocksDB backend so more appropriate
  for testing when comparing behaviour. For a speedy memory database,
  a backend-less variant should be used.

* Drop the `Aristo` prefix from names `AristoLayerRef`, etc.

* Suppress compiler warning

why:
  duplicate imports

* Add filter serialisation transcoder

why:
  Will be used as storage format
2023-08-18 20:46:55 +01:00
Jordan Hrycaj 3078c207ca
Aristo db implement distributed backend access (#1688)
* Fix hashing algorithm

why:
  Particular case where a sub-tree is on the backend, linked by an
  Extension vertex to the top level.

* Update backend verification to report `dirty` top layer

* Implement distributed merge of backend filters

* Implement distributed backend access management

details:
  Implemented and tested as described in chapter 5 of the `README.md`
  file.
2023-08-17 14:42:01 +01:00
Jordan Hrycaj 8f21cf48a8
Distributed access architecture specs (#1680)
why:
  Manage access to different MPTs via the same database (makes sense
  only if the MPTs are not too different.)
2023-08-14 19:39:10 +01:00
Jordan Hrycaj 93a72025a1
Extended data Payload specs for the backend. (#1630)
why:
  For the main tree with root vertex ID 1, the leaf nodes hold the
  account data. These accounts may link to sub trees the storage root
  node ID of which must be registered here. There is no reverse key
  lookup on the backend.

note:
  These definitions are experimental. Also, there are some tests missing
  for validating Payload data conversions.
2023-07-05 21:27:48 +01:00
Jordan Hrycaj ff0fc98fdf
Multi layer architecture 4 aristo db (#1581)
* Cosmetics, renamed fields (eVtx, bVtx) -> (eVid, bVid)

* Multilayered delta architecture for Aristo DB

details:
  Any VertexID or data retrieval needs to go down the rabbit hole and
  fetch/get/manipulate the bottom layer -- even without explicit
  backend.

* Direct reference to backend from top-level layer

why:
  Some services as the vid management needs to be synchronised among all
  layers. So access is optimised.
2023-05-14 18:43:01 +01:00
Jordan Hrycaj 605739ef4c
Experimental MP-trie (#1573)
* Experimental MP-trie

why:
  Deleting records is a infeasible with the current structure

* Added vertex ID recycling management

Todo:
  Provide some unit tests

* DB layout update

why:
  Main news is the separation of `Merkel` hashes into an extra table.

details:
  The code fragments cover conversion between compact MPT records and
  Aristo DB records as well as some rudimentary cache handling for
  the `Merkel` hashes (i.e. the extra table entries.)

todo:
  Add some simple unit test for the descriptor record (currently used
  for vertex ID management, only.)

* Updated vertex ID recycling management

details:
  added simple unit tests (mainly testing ABI)

* docu update
2023-05-11 15:25:29 +01:00