238 lines
8.8 KiB
Markdown
238 lines
8.8 KiB
Markdown
Syncing
|
|
=======
|
|
|
|
Syncing blocks is performed in two partially overlapping phases
|
|
|
|
* loading the header chains into separate database tables
|
|
* removing headers from the headers chain, fetching the rest of the
|
|
block the header belongs to and executing it
|
|
|
|
Header chains
|
|
-------------
|
|
|
|
The header chains are the triple of
|
|
|
|
* a consecutively linked chain of headers starting starting at Genesis
|
|
* followed by a sequence of missing headers
|
|
* followed by a consecutively linked chain of headers ending up at a
|
|
finalised block header received from the consensus layer
|
|
|
|
A sequence *@[h(1),h(2),..]* of block headers is called a consecutively
|
|
linked chain if
|
|
|
|
* block numbers join without gaps, i.e. *h(n).number+1 == h(n+1).number*
|
|
* parent hashes match, i.e. *h(n).hash == h(n+1).parentHash*
|
|
|
|
General header chains layout diagram
|
|
|
|
G B L F (1)
|
|
o----------------o---------------------o----------------o--->
|
|
| <-- linked --> | <-- unprocessed --> | <-- linked --> |
|
|
|
|
Here, the single upper letter symbols *G*, *B*, *L*, *F* denote block numbers.
|
|
For convenience, these letters are also identified with its associated block
|
|
header or the full block. Saying *"the header G"* is short for *"the header
|
|
with block number G"*.
|
|
|
|
Meaning of *G*, *B*, *L*, *F*:
|
|
|
|
* *G* -- Genesis block number *#0*
|
|
* *B* -- base, maximal block number of linked chain starting at *G*
|
|
* *L* -- least, minimal block number of linked chain ending at *F* with *B <= L*
|
|
* *F* -- final, some finalised block
|
|
|
|
This definition implies *G <= B <= L <= F* and the header chains can uniquely
|
|
be described by the triple of block numbers *(B,L,F)*.
|
|
|
|
### Storage of header chains:
|
|
|
|
Some block numbers from the set *{w|G<=w<=B}* may correspond to finalised
|
|
blocks which may be stored anywhere. If some block numbers do not correspond
|
|
to finalised blocks, then the headers must reside in the *beaconHeader*
|
|
database table. Of course, due to being finalised such block numbers constitute
|
|
a sub-chain starting at *G*.
|
|
|
|
The block numbers from the set *{w|L<=w<=F}* must reside in the *beaconHeader*
|
|
database table. They do not correspond to finalised blocks.
|
|
|
|
### Header chains initialisation:
|
|
|
|
Minimal layout on a pristine system
|
|
|
|
G (2)
|
|
B
|
|
L
|
|
F
|
|
o--->
|
|
|
|
When first initialised, the header chains are set to *(G,G,G)*.
|
|
|
|
### Updating header chains:
|
|
|
|
A header chain with an non empty open interval *(B,L)* can be updated only by
|
|
increasing *B* or decreasing *L* by adding headers so that the linked chain
|
|
condition is not violated.
|
|
|
|
Only when the open interval *(B,L)* vanishes the right end *F* can be increased
|
|
by *Z* say. Then
|
|
|
|
* *B==L* beacuse interval *(B,L)* is empty
|
|
* *B==F* because *B* is maximal
|
|
|
|
and the header chains *(F,F,F)* (depicted in *(3)*) can be set to *(B,Z,Z)*
|
|
(as depicted in *(4)*.)
|
|
|
|
Layout before updating of *F*
|
|
|
|
B (3)
|
|
L
|
|
G F Z
|
|
o----------------o---------------------o---->
|
|
| <-- linked --> |
|
|
|
|
New layout with *Z*
|
|
|
|
L' (4)
|
|
G B F'
|
|
o----------------o---------------------o---->
|
|
| <-- linked --> | <-- unprocessed --> |
|
|
|
|
with *L'=Z* and *F'=Z*.
|
|
|
|
Note that diagram *(3)* is a generalisation of *(2)*.
|
|
|
|
|
|
### Complete header chain:
|
|
|
|
The header chain is *relatively complete* if it satisfies clause *(3)* above
|
|
for *G < B*. It is *fully complete* if *F==Z*. It should be obvious that the
|
|
latter condition is temporary only on a live system (as *Z* is permanently
|
|
updated.)
|
|
|
|
If a *relatively complete* header chain is reached for the first time, the
|
|
execution layer can start running an importer in the background compiling
|
|
or executing blocks (starting from block number *#1*.) So the ledger database
|
|
state will be updated incrementally.
|
|
|
|
Imported block chain
|
|
--------------------
|
|
|
|
The following imported block chain diagram amends the layout *(1)*:
|
|
|
|
G T B L F (5)
|
|
o------------------o-------o---------------------o----------------o-->
|
|
| <-- imported --> | | | |
|
|
| <------- linked ------> | <-- unprocessed --> | <-- linked --> |
|
|
|
|
|
|
where *T* is the number of the last imported and executed block. Coincidentally,
|
|
*T* also refers to the global state of the ledger database.
|
|
|
|
The headers corresponding to the half open interval `(T,B]` can be completed by
|
|
fetching block bodies and then imported/executed.
|
|
|
|
Running the sync process for *MainNet*
|
|
--------------------------------------
|
|
|
|
For syncing, a beacon node is needed that regularly informs via *RPC* of a
|
|
recently finalised block header.
|
|
|
|
The beacon node program used here is the *nimbus_beacon_node* binary from the
|
|
*nimbus-eth2* project (any other will do.) *Nimbus_beacon_node* is started as
|
|
|
|
./run-mainnet-beacon-node.sh \
|
|
--web3-url=http://127.0.0.1:8551 \
|
|
--jwt-secret=/tmp/jwtsecret
|
|
|
|
where *http://127.0.0.1:8551* is the URL of the sync process that receives the
|
|
finalised block header (here on the same physical machine) and `/tmp/jwtsecret`
|
|
is the shared secret file needed for mutual communication authentication.
|
|
|
|
It will take a while for *nimbus_beacon_node* to catch up (see the
|
|
[Nimbus Guide](https://nimbus.guide/quick-start.html) for details.)
|
|
|
|
### Starting `nimbus` for syncing
|
|
|
|
As the sync process is quite slow, it makes sense to pre-load the database
|
|
with data from an `Era1` archive (if available) before starting the real
|
|
sync process. The command would be something like
|
|
|
|
./build/nimbus import \
|
|
--era1-dir:/path/to/main-era1/repo \
|
|
...
|
|
|
|
which will take a while for the full *MainNet* era1 repository (but way faster
|
|
than the sync.)
|
|
|
|
On a system with memory considerably larger than *8GiB* the *nimbus*
|
|
binary is started on the same machine where the beacon node runs as
|
|
|
|
./build/nimbus \
|
|
--network=mainnet \
|
|
--engine-api=true \
|
|
--engine-api-port=8551 \
|
|
--engine-api-ws=true \
|
|
--jwt-secret=/tmp/jwtsecret \
|
|
...
|
|
|
|
Note that *--engine-api-port=8551* and *--jwt-secret=/tmp/jwtsecret* match
|
|
the corresponding options from the *nimbus-eth2* beacon source example.
|
|
|
|
### Syncing on a low memory machine
|
|
|
|
On a system with memory with *8GiB* the following additional options proved
|
|
useful for *nimbus* to reduce the memory footprint.
|
|
|
|
For the *Era1* pre-load (if any) the following extra options apply to
|
|
"*nimbus import*":
|
|
|
|
--chunk-size=1024
|
|
--debug-rocksdb-row-cache-size=512000
|
|
--debug-rocksdb-block-cache-size=1500000
|
|
|
|
To start syncing, the following additional options apply to *nimbus*:
|
|
|
|
--debug-beacon-chunk-size=384
|
|
--debug-rocksdb-max-open-files=384
|
|
--debug-rocksdb-write-buffer-size=50331648
|
|
--debug-rocksdb-block-cache-size=1073741824
|
|
--debug-rdb-key-cache-size=67108864
|
|
--debug-rdb-vtx-cache-size=268435456
|
|
|
|
Also, to reduce the backlog for *nimbus-eth2* stored on disk, the following
|
|
changes might be considered. For file
|
|
*nimbus-eth2/vendor/mainnet/metadata/config.yaml* change setting constants:
|
|
|
|
MIN_EPOCHS_FOR_BLOCK_REQUESTS: 33024
|
|
MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 4096
|
|
to
|
|
|
|
MIN_EPOCHS_FOR_BLOCK_REQUESTS: 8
|
|
MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS: 8
|
|
|
|
Caveat: These changes are not useful when running *nimbus_beacon_node* as a
|
|
production system.
|
|
|
|
Metrics
|
|
-------
|
|
|
|
The following metrics are defined in *worker/update/metrics.nim* which will
|
|
be available if *nimbus* is compiled with the additional make flags
|
|
*NIMFLAGS="-d:metrics \-\-threads:on"*:
|
|
|
|
| *Variable* | *Logic type* | *Short description* |
|
|
|:-------------------------------|:------------:|:--------------------|
|
|
| | | |
|
|
| beacon_state_block_number | block height | **T**, *increasing* |
|
|
| beacon_base_block_number | block height | **B**, *increasing* |
|
|
| beacon_least_block_number | block height | **L** |
|
|
| beacon_final_block_number | block height | **F**, *increasing* |
|
|
| beacon_beacon_block_number | block height | **Z**, *increasing* |
|
|
| | | |
|
|
| beacon_headers_staged_queue_len| size | # of staged header list records |
|
|
| beacon_headers_unprocessed | size | # of accumulated header block numbers|
|
|
| beacon_blocks_staged_queue_len | size | # of staged block list records |
|
|
| beacon_blocks_unprocessed | size | # of accumulated body block numbers |
|
|
| | | |
|
|
| beacon_number_of_buddies | size | # of working peers |
|