Commit Graph

6 Commits

Author SHA1 Message Date
Jordan Hrycaj 13f51939f6
Core db aristo hasher profiling and timing improvement (#1938)
* Explicitly use shared `Kvt` table on `Ledger` and `Clique` lookup.

why:
  Speeds up lookup time with `Aristo` backend. For writing `Clique` data,
  the `Companion` model allows to write `Clique` data past the database
  locked by evm transactions.

* Implement `CoreDb` profiling with API tracking

why:
  Chasing time spent per APT procs ...

* Implement `Ledger` profiling with API tracking

why:
  Chasing time spent per APT procs ...

* Always hashify when commiting or storing

why:
  A dirty cache makes no sense when committing

* Make sure that a zero key is created when adding/updating vertices

why:
  This is an error fix mainly for edge cases. A typical error was
  that the root key got deleted when there were only a few vertices
  left on the DB.

* Need all created and changed vertices zero-keyed on the cache

why:
  A zero key (i.e. empty Merkle hash) indicates that a vertex key
  needs to be updated. This would not be needed immediately after
  a merge as there is an actual leaf path on the cache layer. But
  after subsequent merge and delete operations this information
  might get blurred.

* Re-org hashing algorithm

why:
  Apart from errors, the previous implementation was too slow for
  two reasons:
  + some control hashes were calculated for debugging (now all
    verification is done in `aristo_check` module)
  + the leaf paths stored on the cache are used to build the
    labelling (aka hashing) schedule; there paths were accumulated
    over successive hash sessions although it is clear that all
    keys were generated, already
2023-12-12 17:47:41 +00:00
jangko 7a1fe5707c
Refactor engine api and cleanup web3 types conversion 2023-08-30 10:42:46 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Jordan Hrycaj 89ae9621c4
Silence compiler gossip after nim upgrade (#1454)
* Silence some compiler gossip -- part 1, tx_pool

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 2, clique

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 3, misc core

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Silence some compiler gossip -- part 4, sync

details:
  Mostly removing redundant imports and `Defect` tracer after switch
  to nim 1.6

* Clique update

why:
  Missing exception annotation
2023-01-30 22:10:23 +00:00
jangko b81511fcfc
move poa and pow from ChainRef to CommonRef 2022-12-06 16:42:55 +07:00
jangko ac2cb82a2b saner source code grouping 2022-12-02 13:51:42 +07:00