nimbus-eth1/nimbus/sync/snap/worker/db
Jordan Hrycaj e14fd4b96c
Prep for full sync after snap make 6 (#1291)
* Update log ticker, using time interval rather than ticker count

why:
  Counting and logging ticker occurrences is inherently imprecise. So
  time intervals are used.

* Use separate storage tables for snap sync data

* Left boundary proof update

why:
  Was not properly implemented, yet.

* Capture pivot in peer worker (aka buddy) tasks

why:
  The pivot environment is linked to the `buddy` descriptor. While
  there is a task switch, the pivot may change. So it is passed on as
  function argument `env` rather than retrieved from the buddy at
  the start of a sub-function.

* Split queues `fetchStorage` into `fetchStorageFull` and `fetchStoragePart`

* Remove obsolete account range returned from `GetAccountRange` message

why:
  Handler returned the wrong right value of the range. This range was
  for convenience, only.

* Prioritise storage slots if the queue becomes large

why:
  Currently, accounts processing is prioritised up until all accounts
  are downloaded. The new prioritisation has two thresholds for
  + start processing storage slots with a new worker
  + stop account processing and switch to storage processing

also:
  Provide api for `SnapTodoRanges` pair of range sets in `worker_desc.nim`

* Generalise left boundary proof for accounts or storage slots.

why:
  Detailed explanation how this works is documented with
  `snapdb_accounts.importAccounts()`.

  Instead of enforcing a left boundary proof (which is still the default),
  the importer functions return a list of `holes` (aka node paths) found in
  the argument ranges of leaf nodes. This in turn is used by the book
   keeping software for data download.

* Forgot to pass on variable in function wrapper

also:
  + Start healing not before 99% accounts covered (previously 95%)
  + Logging updated/prettified
2022-11-08 18:56:04 +00:00
..
hexary_desc.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
hexary_error.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
hexary_import.nim Prep for full sync after snap make 4 (#1282) 2022-10-27 14:49:28 +01:00
hexary_inspect.nim Fix trie inspector for healing (#1284) 2022-10-28 08:26:17 +01:00
hexary_interpolate.nim Remodel persistent snapdb access (#1274) 2022-10-20 17:59:54 +01:00
hexary_paths.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
rocky_bulk_load.nim Snap sync accounts db code reorg (#1189) 2022-08-15 16:51:50 +01:00
snapdb_accounts.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
snapdb_check.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
snapdb_desc.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
snapdb_persistent.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00
snapdb_storage_slots.nim Prep for full sync after snap make 6 (#1291) 2022-11-08 18:56:04 +00:00