nimbus-eth2/beacon_chain/validators
Jacek Sieka 61342c2449
limit by-root requests to non-finalized blocks (#3293)
* limit by-root requests to non-finalized blocks

Presently, we keep a mapping from block root to `BlockRef` in memory -
this has simplified reasoning about the dag, but is not sustainable with
the chain growing.

We can distinguish between two cases where by-root access is useful:

* unfinalized blocks - this is where the beacon chain is operating
generally, by validating incoming data as interesting for future fork
choice decisions - bounded by the length of the unfinalized period
* finalized blocks - historical access in the REST API etc - no bounds,
really

In this PR, we limit the by-root block index to the first use case:
finalized chain data can more efficiently be addressed by slot number.

Future work includes:

* limiting the `BlockRef` horizon in general - each instance is 40
bytes+overhead which adds up - this needs further refactoring to deal
with the tail vs state problem
* persisting the finalized slot-to-hash index - this one also keeps
growing unbounded (albeit slowly)

Anyway, this PR easily shaves ~128mb of memory usage at the time of
writing.

* No longer honor `BeaconBlocksByRoot` requests outside of the
non-finalized period - previously, Nimbus would generously return any
block through this libp2p request - per the spec, finalized blocks
should be fetched via `BeaconBlocksByRange` instead.
* return `Opt[BlockRef]` instead of `nil` when blocks can't be found -
this becomes a lot more common now and thus deserves more attention
* `dag.blocks` -> `dag.forkBlocks` - this index only carries unfinalized
blocks from now - `finalizedBlocks` covers the other `BlockRef`
instances
* in backfill, verify that the last backfilled block leads back to
genesis, or panic
* add backfill timings to log
* fix missing check that `BlockRef` block can be fetched with
`getForkedBlock` reliably
* shortcut doppelganger check when feature is not enabled
* in REST/JSON-RPC, fetch blocks without involving `BlockRef`

* fix dag.blocks ref
2022-01-21 13:33:16 +02:00
..
README.md Consolidate modules by areas [part 1] (#2365) 2021-03-02 11:27:45 +01:00
action_tracker.nim time: spring cleaning (#3262) 2022-01-11 11:01:54 +01:00
keystore_management.nim Implementation of the Keymanager API (BETA) 2022-01-04 18:51:45 +02:00
slashing_protection.nim Slashing prot interchange tests v5.2.1 (#3277) 2022-01-20 17:14:06 +01:00
slashing_protection_common.nim Slashing prot interchange tests v5.2.1 (#3277) 2022-01-20 17:14:06 +01:00
slashing_protection_v2.nim Slashing prot interchange tests v5.2.1 (#3277) 2022-01-20 17:14:06 +01:00
validator_duties.nim limit by-root requests to non-finalized blocks (#3293) 2022-01-21 13:33:16 +02:00
validator_monitor.nim fix non-totalized validator metric (#3297) 2022-01-19 15:20:48 +01:00
validator_pool.nim time: spring cleaning (#3262) 2022-01-11 11:01:54 +01:00

README.md

Validators

This folder holds all modules related to a Beacon Chain Validator besides the binaries they interact directly with (nimbus_validator_cliant and nimbus_signing_process):

  • Validator keystore
  • Validator slashing protection
  • Validator duties
  • Validator pool