Commit Graph

5 Commits

Author SHA1 Message Date
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Jordan Hrycaj 9facab91cb
Prepare snap client for continuing with full sync (#1534)
* Somewhat tighten error handling

why:
  Zombie state is invoked when the current peer turns out to be useless
  for further communication. While there is a chance to further talk
  to a peer about another topic (aka healing) after some protocol failure,
  it makes no sense to do so after a network problem.

  The latter state is explained bu the `peerDegraded` flag that goes
  together with the `zombie` state flag. A degraded peer is dropped
  immediately.

* Remove `--sync-mode=snapCtx` option, always start snap in recovery mode

why:
  No need for a snap sync option without recovery mode, can be achieved
  by deleting the database.

* Code cosmetics, typos, prettify logging, debugging helper, etc.

* Split off snap sync sub-mode handler into separate modules

details:
  The original `worker.nim` source has become a multiplexer for several
  snap sync sub-modes `full` and `snap`. The source modules of the
  incarnations of a particular sync sub-mode are places into the
  `worker/play` directory.

* Update ticker for snap and full sync logging
2023-04-06 20:42:07 +01:00
Jordan Hrycaj df1217b7ca
Silence compiler gossip after nim upgrade cont3 (#1466)
* Removed some Windows specific unit test annoyances

details:
+ Short put()/get() cycles on persistent database have a race condition
  with vendor rocksdb. On a specific (and slow) qemu/win7 a 50ms `sleep()`
  in between will mostly do the job (i.e. unless heavy CPU load.) This
  issue was not observed on github/ci.
+ Removed annoyances when qemu/Win7 keeps the rocksdb database files
  locked even after closing the db. The problem is solved by strictly
  using fresh names for each test. No assumption made to be able to
  properly clean up. This issue was not observed on github/ci.

* Silence some compiler gossip -- part 7, misc/non(sync or graphql)

details:
  Adding some missing exception annotation
2023-02-14 20:27:17 +00:00
Jordan Hrycaj 197d2b16dd
Snap sync interval range extractor (#1449)
* Update comments and test noise

* Fix boundary proofs

why:
  Where neither used in production, nor unit tested. For production, other
  methods apply to test leaf range integrity directly based of the proof
  nodes.

* Added `hexary_range()`: interval range + proof extractor

details:
+ Will be used for `snap/1` protocol handler
+ Unit tests added (also for testing left boundary proof)

todo:
  Need to verify completeness of proof nodes

* Reduce some nim 1.6 compiler noise

* Stop unit test gossip for ci tests
2023-01-30 17:50:58 +00:00
Jordan Hrycaj e093fa452d
Declutter snap sync unit tests (#1444)
* Extracted RocksDB timing unit tests into separate file

why:
  make space for more in main module :)

* Extracted `inspectionRunner()` unit tests into separate file

why:
  make space for more in main module :)

* Extracted `storagesRunner()` unit tests into separate file

why:
  make space for more in main module :)

* Extracted pivot checkpoint store/retrieval unit tests into separate file

why:
  make space for more in main module :)

* Extract helper functions into separate source file

* Extracted account import unit tests into separate file

why:
  make space for more in main module :)

* Rename `test_decompose()` => `test_NodeRangeDecompose()`

why:
  There will be more functions with `test_NodeRange` prefix.
2023-01-23 16:09:12 +00:00