nimbus-eth1/nimbus/sync/snap
Jordan Hrycaj 5e865edec0
Update snap client storage slots download and healing (#1529)
* Fix fringe condition for `GetStorageRanges` message handler

why:
  Receiving a proved empty range was not considered at all. This lead to
  inconsistencies of the return value which led to subsequent errors.

* Update storage range bulk download

details;
  Mainly re-org of storage queue processing in `storage_queue_helper.nim`

* Update logging variables/messages

* Update storage slots healing

details:
  Mainly clean up after improved helper functions from the sources
  `find_missing_nodes.nim` and `storage_queue_helper.nim`.

* Simplify account fetch

why:
  To much fuss made tolerating some errors. There will be an overall
  strategy implemented where the concert of download and healing function
  is orchestrated.

* Add error resilience to the concert of download and healing.

why:
  The idea is that a peer might stop serving snap/1 accounts and storage
  slot downloads while still able to support fetching nodes for healing.
2023-04-04 14:36:18 +01:00
..
worker Update snap client storage slots download and healing (#1529) 2023-04-04 14:36:18 +01:00
README.txt Prepare snap server client test scenario cont4 (#1507) 2023-03-17 14:46:50 +00:00
constants.nim Update snap client storage slots download and healing (#1529) 2023-04-04 14:36:18 +01:00
range_desc.nim Update snap client storage slots download and healing (#1529) 2023-04-04 14:36:18 +01:00
update_beacon_header.nim Prepare snap server client test scenario cont2 (#1487) 2023-03-07 14:23:22 +00:00
worker.nim Update snap client storage slots download and healing (#1529) 2023-04-04 14:36:18 +01:00
worker_desc.nim Update snap client storage slots download and healing (#1529) 2023-04-04 14:36:18 +01:00

README.txt

Snap sync test & debugging scenario
===================================


Start snap/1 server
-------------------

  # Enter nimbus directory for snap/1 protocol server.
  cd server

  # Tell nimbus to stop full sync after 2 mio blocks.
  echo 2000000 > full-limit.txt

  # Tell nimbus to use this predefined key ID
  echo 123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0 > full-id.key

  ./build/nimbus \
    --tcp-port:30319 --nat=None --sync-mode=full \
	--protocols=snap --discovery=none \
	--net-key=./full-id.key \
	--sync-ctrl-file=./full-limit.txt \
	--log-level:TRACE

  # Wait for several hours until enough blocks have been downloaded so that
  # snap sync data are available. The full 2 mio blocks are available if the
  # log ticker shows something like
  #
  # INF 2023-03-17 [..] Sync statistics (suspended) topics="full-tick" [..] persistent=#2000080 [..]
  #
  # where the persistent=#2000080 field might vary


Start snap/1 client
-------------------

  # Note: When the snap/1 server has enough blocks, the client can be started.

  # Enter nimbus directory for snap/1 protocol server
  cd client

  # Tell nimbus to use this pivot block number. This number must be smaller
  # than the 2000000 written into the file full-limit.txt above.
  echo 600000 > snap/snap-update.txt.

  # Tell nimbus to use this hard coded peer enode.
  echo enode://192d7e7a302bd4ff27f48d7852621e0d3cb863a6dd67dd44e0314a25a3aa866837f0d2460b4444dc66e7b7a2cd56a2de1c31b2a2ba4e23549bf3ba3b0c4f2eb5@127.0.0.1:30319 > snap/full-servers.txt

  ./build/nimbus \
    --tcp-port:30102 --nat=None --sync-mode=snap \
	--protocols=none --discovery=none \
	--static-peers-file=./full-servers.txt \
	--sync-ctrl-file=./snap-update.txt \
	--log-level:TRACE


Modifications while the programs are syncing
--------------------------------------------

  # Increasing the number in the files full/full-limit.txt or
  # snap/snap-update.txt will be recognised while running. Decreasing
  # or removing will be ignored.