History

Jacek Sieka b98f46c04d Avoid global in p2p macro (fixes #4578 ) (#5719 ) * Avoid global in p2p macro (fixes #4578) * copy p2p macro to this repo and start de-crufting it * make protocol registration dynamic, removing light client hacks et al * split out light client protocol into its own file * cleanups * Option -> Opt * remove more cruft * further split beacon_sync this allows the light client to respond to peer metadata messages without exposing the block sync protocol * better protocol init * "constant" protocol index * avoid casts * copyright * move some discovery code to discovery * avoid extraneous data copy when sending chunks * remove redundant forkdigest field * document how to connect to a specific peer		2024-01-13 11:54:24 +02:00
..
README.md	docs: fix typos (#5571 )	2023-11-06 03:56:07 +00:00
light_client_manager.nim	Avoid global in p2p macro (fixes #4578 ) (#5719 )	2024-01-13 11:54:24 +02:00
light_client_protocol.nim	Avoid global in p2p macro (fixes #4578 ) (#5719 )	2024-01-13 11:54:24 +02:00
light_client_sync_helpers.nim	verify `genesis_time` more strictly (fixes #1667 ) (#5694 )	2024-01-06 15:26:56 +01:00
request_manager.nim	verify `genesis_time` more strictly (fixes #1667 ) (#5694 )	2024-01-06 15:26:56 +01:00
sync_manager.nim	Avoid global in p2p macro (fixes #4578 ) (#5719 )	2024-01-13 11:54:24 +02:00
sync_protocol.nim	Avoid global in p2p macro (fixes #4578 ) (#5719 )	2024-01-13 11:54:24 +02:00
sync_queue.nim	verify `genesis_time` more strictly (fixes #1667 ) (#5694 )	2024-01-06 15:26:56 +01:00

README.md

Block syncing

This folder holds all modules related to block syncing

Block syncing uses ETH2 RPC protocol.

Reference diagram

Eth2 RPC in

Blocks are requested during sync by the SyncManager.

Blocks are received by batch:

syncStep(SyncManager, index, peer)
in case of success:
- push(SyncQueue, SyncRequest, seq[SignedBeaconBlock]) is called to handle a successful sync step. It calls validate(SyncQueue, SignedBeaconBlock)` on each block retrieved one-by-one
- validate only enqueues the block in the SharedBlockQueue AsyncQueue[BlockEntry] but does no extra validation only the GossipSub case
in case of failure:
- push(SyncQueue, SyncRequest) is called to reschedule the sync request.

Every second when sync is not in progress, the beacon node will ask the RequestManager to download all missing blocks currently in quarantine.

via handleMissingBlocks
which calls fetchAncestorBlocks
which asynchronously enqueue the request in the SharedBlockQueue AsyncQueue[BlockEntry].

The RequestManager runs an event loop:

that calls fetchAncestorBlocksFromNetwork
which RPC calls peers with beaconBlocksByRoot
and calls validate(RequestManager, SignedBeaconBlock) on each block retrieved one-by-one
validate only enqueues the block in the AsyncQueue[BlockEntry] but does no extra validation only the GossipSub case

Weak subjectivity sync

Not implemented!

Comments

The validate procedure name for SyncManager and RequestManager as no P2P validation actually occurs.

Sync vs Steady State

During sync:

The RequestManager is deactivated
The syncManager is working full speed ahead
Gossip is deactivated

Bottlenecks during sync

During sync:

The bottleneck is clearing the SharedBlockQueue AsyncQueue[BlockEntry] via storeBlock which requires full verification (state transition + cryptography)

Backpressure

The SyncManager handles backpressure by ensuring that current_queue_slot <= request.slot <= current_queue_slot + sq.queueSize * sq.chunkSize.

queueSize is -1, unbounded, by default according to comment but all init paths uses 1 (?)
chunkSize is SLOTS_PER_EPOCH = 32

However the shared AsyncQueue[BlockEntry] itself is unbounded. Concretely:

The shared AsyncQueue[BlockEntry] is bounded for sync
The shared AsyncQueue[BlockEntry] is unbounded for validated gossip blocks

RequestManager and Gossip are deactivated during sync and so do not contribute to pressure.