8.8 KiB
Syncing
Syncing blocks is performed in two partially overlapping phases
- loading the header chains into separate database tables
- removing headers from the headers chain, fetching the rest of the block the header belongs to and executing it
Header chains
The header chains are the triple of
- a consecutively linked chain of headers starting starting at Genesis
- followed by a sequence of missing headers
- followed by a consecutively linked chain of headers ending up at a finalised block header received from the consensus layer
A sequence @[h(1),h(2),..] of block headers is called a consecutively linked chain if
- block numbers join without gaps, i.e. h(n).number+1 == h(n+1).number
- parent hashes match, i.e. h(n).hash == h(n+1).parentHash
General header chains layout diagram
G B L F (1)
o----------------o---------------------o----------------o--->
| <-- linked --> | <-- unprocessed --> | <-- linked --> |
Here, the single upper letter symbols G, B, L, F denote block numbers. For convenience, these letters are also identified with its associated block header or the full block. Saying "the header G" is short for "the header with block number G".
Meaning of G, B, L, F:
- G -- Genesis block number #0
- B -- base, maximal block number of linked chain starting at G
- L -- least, minimal block number of linked chain ending at F with B <= L
- F -- final, some finalised block
This definition implies G <= B <= L <= F and the header chains can uniquely be described by the triple of block numbers (B,L,F).
Storage of header chains:
Some block numbers from the set {w|G<=w<=B} may correspond to finalised blocks which may be stored anywhere. If some block numbers do not correspond to finalised blocks, then the headers must reside in the beaconHeader database table. Of course, due to being finalised such block numbers constitute a sub-chain starting at G.
The block numbers from the set {w|L<=w<=F} must reside in the beaconHeader database table. They do not correspond to finalised blocks.
Header chains initialisation:
Minimal layout on a pristine system
G (2)
B
L
F
o--->
When first initialised, the header chains are set to (G,G,G).
Updating header chains:
A header chain with an non empty open interval (B,L) can be updated only by increasing B or decreasing L by adding headers so that the linked chain condition is not violated.
Only when the open interval (B,L) vanishes the right end F can be increased by Z say. Then
- B==L beacuse interval (B,L) is empty
- B==F because B is maximal
and the header chains (F,F,F) (depicted in (3)) can be set to (B,Z,Z) (as depicted in (4).)
Layout before updating of F
B (3)
L
G F Z
o----------------o---------------------o---->
| <-- linked --> |
New layout with Z
L' (4)
G B F'
o----------------o---------------------o---->
| <-- linked --> | <-- unprocessed --> |
with L'=Z and F'=Z.
Note that diagram (3) is a generalisation of (2).
Complete header chain:
The header chain is relatively complete if it satisfies clause (3) above for G < B. It is fully complete if F==Z. It should be obvious that the latter condition is temporary only on a live system (as Z is permanently updated.)
If a relatively complete header chain is reached for the first time, the execution layer can start running an importer in the background compiling or executing blocks (starting from block number #1.) So the ledger database state will be updated incrementally.
Imported block chain
The following imported block chain diagram amends the layout (1):
G T B L F (5)
o------------------o-------o---------------------o----------------o-->
| <-- imported --> | | | |
| <------- linked ------> | <-- unprocessed --> | <-- linked --> |
where T is the number of the last imported and executed block. Coincidentally, T also refers to the global state of the ledger database.
The headers corresponding to the half open interval (T,B] can be completed by
fetching block bodies and then imported/executed.
Running the sync process for MainNet
For syncing, a beacon node is needed that regularly informs via RPC of a recently finalised block header.
The beacon node program used here is the nimbus_beacon_node binary from the nimbus-eth2 project (any other will do.) Nimbus_beacon_node is started as
./run-mainnet-beacon-node.sh \
--web3-url=http://127.0.0.1:8551 \
--jwt-secret=/tmp/jwtsecret
where http://127.0.0.1:8551 is the URL of the sync process that receives the
finalised block header (here on the same physical machine) and /tmp/jwtsecret
is the shared secret file needed for mutual communication authentication.
It will take a while for nimbus_beacon_node to catch up (see the Nimbus Guide for details.)
Starting nimbus for syncing
As the sync process is quite slow, it makes sense to pre-load the database
with data from an Era1 archive (if available) before starting the real
sync process. The command would be something like
./build/nimbus import \
--era1-dir:/path/to/main-era1/repo \
...
which will take a while for the full MainNet era1 repository (but way faster than the sync.)
On a system with memory considerably larger than 8GiB the nimbus binary is started on the same machine where the beacon node runs as
./build/nimbus \
--network=mainnet \
--engine-api=true \
--engine-api-port=8551 \
--engine-api-ws=true \
--jwt-secret=/tmp/jwtsecret \
...
Note that --engine-api-port=8551 and --jwt-secret=/tmp/jwtsecret match the corresponding options from the nimbus-eth2 beacon source example.
Syncing on a low memory machine
On a system with memory with 8GiB the following additional options proved useful for nimbus to reduce the memory footprint.
For the Era1 pre-load (if any) the following extra options apply to "nimbus import":
--chunk-size=1024
--debug-rocksdb-row-cache-size=512000
--debug-rocksdb-block-cache-size=1500000
To start syncing, the following additional options apply to nimbus:
--debug-beacon-chunk-size=384
--debug-rocksdb-max-open-files=384
--debug-rocksdb-write-buffer-size=50331648
--debug-rocksdb-block-cache-size=1073741824
--debug-rdb-key-cache-size=67108864
--debug-rdb-vtx-cache-size=268435456
Also, to reduce the backlog for nimbus-eth2 stored on disk, the following changes might be considered. For file nimbus-eth2/vendor/mainnet/metadata/config.yaml change setting constants:
MIN_EPOCHS_FOR_BLOCK_REQUESTS: 33024
MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 4096
to
MIN_EPOCHS_FOR_BLOCK_REQUESTS: 8
MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 8
Caveat: These changes are not useful when running nimbus_beacon_node as a production system.
Metrics
The following metrics are defined in worker/update/metrics.nim which will be available if nimbus is compiled with the additional make flags NIMFLAGS="-d:metrics --threads:on":
| Variable | Logic type | Short description |
|---|---|---|
| beacon_state_block_number | block height | T, increasing |
| beacon_base_block_number | block height | B, increasing |
| beacon_least_block_number | block height | L |
| beacon_final_block_number | block height | F, increasing |
| beacon_beacon_block_number | block height | Z, increasing |
| beacon_headers_staged_queue_len | size | # of staged header list records |
| beacon_headers_unprocessed | size | # of accumulated header block numbers |
| beacon_blocks_staged_queue_len | size | # of staged block list records |
| beacon_blocks_unprocessed | size | # of accumulated body block numbers |
| beacon_number_of_buddies | size | # of working peers |