status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/beacon_node.nim

150 lines
5.3 KiB
Nim
Raw Normal View History

Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
# beacon_chain
make LC data fork aware (#4493) In a future fork, light client data will be extended with execution info to support more use cases. To anticipate such an upgrade, introduce `Forky` and `Forked` types, and ready the database schema. Because the mapping of sync committee periods to fork versions is not necessarily unique (fork schedule not in sync with period boundaries), an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 17:11:38 +00:00
# Copyright (c) 2018-2023 Status Research & Development GmbH
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
import cleanup (#2997) * import cleanup ...and remove some unused types * add random imports * more imports
2021-10-19 14:09:26 +00:00
# Everything needed to run a full Beacon Node
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
import
handle fork choice selecting a head unknown to chaindag (#1907) * move head slot metric to head selection * avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
std/osproc,
first batch of work towards the VC/BN split: - we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot - the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make - factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim - the VC loads its associated private keys from the datadir for its BN - most of the validator API calls have been implemented as a stub. - the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-22 17:04:52 +00:00
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
# Nimble packages
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
chronos, json_rpc/servers/httpserver, presto, bearssl/rand,
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
# Local modules
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
"."/[beacon_clock, beacon_chain_db, conf, light_client],
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
./gossip_processing/[eth2_processor, block_processor, optimistic_processor],
Reorg (5/5) (#2377) * Reorg things left into networking and gossip_processing * time -> beacon_clock * fix builds
2021-03-05 13:12:00 +00:00
./networking/eth2_network,
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944)
2023-05-15 05:05:12 +00:00
./el/el_manager,
clean up fork enum and field names * single naming strategy * simplify some fork code * simplify forked block production
2021-10-18 16:37:27 +00:00
./consensus_object_pools/[
RequestManager: add support for fetching Blobs (#4844) * RequestManager: add support for fetching Blobs * Review feedback * Lint * Change peekSortedBlobless -> peekBlobless
2023-04-28 12:57:35 +00:00
blockchain_dag, blob_quarantine, block_quarantine, consensus_manager,
exit_pool, attestation_pool, sync_committee_msg_pool],
adopt LC REST API with v0 suffix (without proofs) (#3775) * adopt LC REST API with v0 suffix (without proofs) Adopts the light client data REST API used by Lodestar as defined in https://github.com/ethereum/beacon-APIs/pull/181 with a v0 suffix. Requests: - `/eth/v0/beacon/light_client/bootstrap/{block_root}` - `/eth/v0/beacon/light_client/updates?start_period={start_period}&count={count}` - `/eth/v0/beacon/light_client/finality_update` - `/eth/v0/beacon/light_client/optimistic_update` HTTP Server-Sent Events (SSE): - `light_client_finality_update_v0` - `light_client_optimistic_update_v0` More work is needed to adopt the proofs endpoint, it is not included. * initialize event queues * register event topics
2022-06-19 05:57:52 +00:00
./spec/datatypes/[base, altair],
Implement the /eth/v1/validator/prepare_beacon_proposer end-point (#3901)
2022-07-25 20:12:53 +00:00
./spec/eth2_apis/dynamic_fee_recipients,
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
./sync/[sync_manager, request_manager],
Introduce message router (#3829) Whether new blocks/attestations/etc are produced internally or received via REST, their journey through the node is the same - to ensure that they get the same treatment (logging, metrics, processing), this PR moves the routing to a dedicated module and fixes several small differences that existed before. * `xxxValidator` -> `processMessageName` - the processor also was adding messages to pools, so we want the name to reflect that action * add missing "sent" metrics for some messages * document ignore policy better - already-seen messages are not actaully rebroadcast by libp2p * skip redundant signature checks for internal validators consistently
2022-07-06 16:11:44 +00:00
./validators/[
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
action_tracker, message_router, validator_monitor, validator_pool,
keystore_management],
Implement #3129 (Optimized history traversals in the REST API) (#3219) * Fix REST some rest call signatures and implement a simple API benchmark tool * Implement #3129 (Optimized history traversals in the REST API) Other notable changes: The `updateStateData` procedure in the `blockchain_dag.nim` module is optimized to not rewind down to the last snapshot state saved in the database if the supplied input state can be used as a starting point instead. * Disallow await in withStateForBlockSlot
2022-01-05 14:49:10 +00:00
./rpc/state_ttl_cache
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
handle fork choice selecting a head unknown to chaindag (#1907) * move head slot metric to head selection * avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
export
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
osproc, chronos, httpserver, presto, action_tracker,
beacon_clock, beacon_chain_db, conf, light_client,
attestation_pool, sync_committee_msg_pool, validator_pool,
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944)
2023-05-15 05:05:12 +00:00
eth2_network, el_manager, request_manager, sync_manager,
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
eth2_processor, optimistic_processor, blockchain_dag, block_quarantine,
base, exit_pool, message_router, validator_monitor,
Implement the /eth/v1/validator/prepare_beacon_proposer end-point (#3901)
2022-07-25 20:12:53 +00:00
consensus_manager, dynamic_fee_recipients
first batch of work towards the VC/BN split: - we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot - the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make - factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim - the VC loads its associated private keys from the datadir for its BN - most of the validator API calls have been implemented as a stub. - the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-22 17:04:52 +00:00
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
type
AsyncEventBus missing attestations fix. (#3664)
2022-06-17 15:27:28 +00:00
EventBus* = object
Add the `execution_optimistic` flag to REST API responses. (#3780) * Initial commit * Make `events` API spec compliant. * Add `Eth-Consensus-Version` in responses. * Bump chronos to get redirect with headers working. * Add `is_optimistic` field and handling to syncing RestSyncInfo.
2022-06-20 05:53:39 +00:00
blocksQueue*: AsyncEventQueue[EventBeaconBlockObject]
AsyncEventBus missing attestations fix. (#3664)
2022-06-17 15:27:28 +00:00
headQueue*: AsyncEventQueue[HeadChangeInfoObject]
reorgQueue*: AsyncEventQueue[ReorgInfoObject]
make LC data fork aware (#4493) In a future fork, light client data will be extended with execution info to support more use cases. To anticipate such an upgrade, introduce `Forky` and `Forked` types, and ready the database schema. Because the mapping of sync committee periods to fork versions is not necessarily unique (fork schedule not in sync with period boundaries), an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 17:11:38 +00:00
finUpdateQueue*: AsyncEventQueue[
RestVersioned[ForkedLightClientFinalityUpdate]]
optUpdateQueue*: AsyncEventQueue[
RestVersioned[ForkedLightClientOptimisticUpdate]]
AsyncEventBus missing attestations fix. (#3664)
2022-06-17 15:27:28 +00:00
attestQueue*: AsyncEventQueue[Attestation]
contribQueue*: AsyncEventQueue[SignedContributionAndProof]
exitQueue*: AsyncEventQueue[SignedVoluntaryExit]
finalQueue*: AsyncEventQueue[FinalizationInfoObject]
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
BeaconNode* = ref object
nickname*: string
Add an option for graffiti customization
2020-06-29 17:30:19 +00:00
graffitiBytes*: GraffitiBytes
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
network*: Eth2Node
Bump modules to take advantage of the new Json format flavors support Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had to be worked-around: * The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors due to conflicts with `nativesockets.Domain`. Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`. * A new more robust version of `hexToByteArray` was added to stew
2021-03-19 02:22:45 +00:00
netKeys*: NetKeyPair
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
db*: BeaconChainDB
config*: BeaconNodeConf
startup cleanup * fix several memory leaks due to temporaries not being reset during init * avoid massive main() function with lots of stuff in it * disable nim-prompt (unused) * reuse validator pool instance in eth2_processor * style cleanup
2021-02-22 16:17:48 +00:00
attachedValidators*: ref ValidatorPool
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
optimisticProcessor*: OptimisticProcessor
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
lightClient*: LightClient
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
dag*: ChainDAGRef
move quarantine outside of chaindag (#3124) * move quarantine outside of chaindag The quarantine has been part of the ChainDAG for the longest time, but this design has a few issues: * the function in which blocks are verified and added to the dag becomes reentrant and therefore difficult to reason about - we're currently using a stateful flag to work around it * quarantined blocks bypass the processing queue leading to a processing stampede * the quarantine flow is unsuitable for orphaned attestations - these should also should be quarantined eventually Instead of processing the quarantine inside ChainDAG, this PR moves re-queueing to `block_processor` which already is responsible for dealing with follow-up work when a block is added to the dag This sets the stage for keeping attestations in the quarantine as well. Also: * make `BlockError` `{.pure.}` * avoid use of `ValidationResult` in block clearance (that's for gossip)
2021-12-06 09:49:01 +00:00
quarantine*: ref Quarantine
RequestManager: add support for fetching Blobs (#4844) * RequestManager: add support for fetching Blobs * Review feedback * Lint * Change peekSortedBlobless -> peekBlobless
2023-04-28 12:57:35 +00:00
blobQuarantine*: ref BlobQuarantine
avoid double deserialization When blocks and attestations arrive, they are SSZ-decoded twice: once for validation and once for processing. This branch enqueues the decoded block directly for processing, avoiding the second, slow deserialization. * move processing of blocks and attestations to queue * ...and out from beacon_node * split attestation processing into attestations and aggregates * also updates metrics * clean up logging to better follow the lifetime of gossip: arrival, validation and processing * drop attestations and aggregates if there are too many * try to prioritise blocks and aggregates before single-validator attestations
2020-08-20 16:30:47 +00:00
attestationPool*: ref AttestationPool
introduce SyncCommitteeMsgPool to eth2_processor and nimbus_beacon_node (#2831)
2021-08-28 22:27:51 +00:00
syncCommitteeMsgPool*: ref SyncCommitteeMsgPool
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
lightClientPool*: ref LightClientPool
exit/validatorchange pool includes BLS to execution messages; REST support for new pool (#4519) * exit/validatorchange pool includes BLS to execution messages; REST support for new pool * catch failed individual futures * increase BLS changes bound and keep BLS seen consistent with subpool * deque capacities should be powers of 2
2023-01-19 22:00:40 +00:00
validatorChangePool*: ref ValidatorChangePool
Support for driving multiple EL nodes from a single Nimbus BN (#4465) * Support for driving multiple EL nodes from a single Nimbus BN Full list of changes: * Eth1Monitor has been renamed to ELManager to match its current responsibilities better. * The ELManager is no longer optional in the code (it won't have a nil value under any circumstances). * The support for subscribing for headers was removed as it only worked with WebSockets and contributed significant complexity while bringing only a very minor advantage. * The `--web3-url` parameter has been deprecated in favor of a new `--el` parameter. The new parameter has a reasonable default value and supports specifying a different JWT for each connection. Each connection can also be configured with a different set of responsibilities (e.g. download deposits, validate blocks and/or produce blocks). On the command-line, these properties can be configured through URL properties stored in the #anchor part of the URL. In TOML files, they come with a very natural syntax (althrough the URL scheme is also supported). * The previously scattered EL-related state and logic is now moved to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim` in a follow-up commit). State is assigned properly either to the `ELManager` or the to individual `ELConnection` objects where appropriate. The ELManager executes all Engine API requests against all attached EL nodes, in parallel. It compares their results and if there is a disagreement regarding the validity of a certain payload, this is detected and the beacon node is protected from publishing a block with a potential execution layer consensus bug in it. The BN provides metrics per EL node for the number of successful or failed requests for each type Engine API requests. If an EL node goes offline and connectivity is resoted later, we report the problem and the remedy in edge-triggered fashion. * More progress towards implementing Deneb block production in the VC and comparing the value of blocks produced by the EL and the builder API. * Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
elManager*: ELManager
Rest API initial implementation.
2021-03-17 18:46:45 +00:00
restServer*: RestServerRef
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
keymanagerHost*: ref KeymanagerHost
Implementation of the Keymanager API (BETA) https://github.com/ethereum/keymanager-APIs
2021-12-22 12:37:31 +00:00
keymanagerServer*: RestServerRef
Keystore cache implementation. (#4372)
2023-02-16 17:25:48 +00:00
keystoreCache*: KeystoreCacheRef
AsyncEventBus missing attestations fix. (#3664)
2022-06-17 15:27:28 +00:00
eventBus*: EventBus
use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
2020-09-01 13:44:40 +00:00
vcProcess*: Process
Use one single async queue and loop for processing blocks. (#1487) * Initial commit * Fix compilation problem. * Address review comments.
2020-08-12 09:29:11 +00:00
requestManager*: RequestManager
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
syncManager*: SyncManager[Peer, PeerId]
backfiller*: SyncManager[Peer, PeerId]
Embed the Altona metadata in the NBC executable
2020-07-02 15:14:11 +00:00
genesisSnapshotContent*: string
avoid double deserialization When blocks and attestations arrive, they are SSZ-decoded twice: once for validation and once for processing. This branch enqueues the decoded block directly for processing, avoiding the second, slow deserialization. * move processing of blocks and attestations to queue * ...and out from beacon_node * split attestation processing into attestations and aggregates * also updates metrics * clean up logging to better follow the lifetime of gossip: arrival, validation and processing * drop attestations and aggregates if there are too many * try to prioritise blocks and aggregates before single-validator attestations
2020-08-20 16:30:47 +00:00
processor*: ref Eth2Processor
clean up block processing (#2610) * gossip_to_consensus -> block_processor (it's processing only blocks, but not only from gossip) * measure queue and validation time for blocks * measure assignment and state loading times for updateStateData * avoid some unnecessary block copies in block sync * warn that database is corrupt if we hit tail without a state
2021-05-28 16:34:00 +00:00
blockProcessor*: ref BlockProcessor
Split Eth2Processor in prep for batching (#2396) * Split Eth2Processor in gossip and consensus part and materialize the shared block queue * Update initialization in test_sync_manager
2021-03-11 10:10:57 +00:00
consensusManager*: ref ConsensusManager
Don't rely on a metric value for the ETH display in the status bar
2020-11-29 21:35:39 +00:00
attachedValidatorBalanceTotal*: uint64
altair topic switching (#2767) * altair topic switching * remove validate{Committee,Validator}IndexOr unused within branch
2021-08-09 12:54:45 +00:00
gossipState*: GossipState
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
blocksGossipState*: GossipState
move BeaconClock from ChainDAG to BeaconNode (#2796)
2021-08-20 08:58:15 +00:00
beaconClock*: BeaconClock
Number of REST fixes. (#2987) * Initial commit. * Fix path. * Add validator keys to indices cache mechanism. Move syncComitteeParticipants to common place. * Fix sync participants order issue. * Fix error code when state could not be found. Refactor `state/validators` to use keysToIndices mechanism. * Fix RestValidatorIndex to ValidatorIndex conversion TODOs. * Address review comments. * Fix REST test rules.
2021-10-14 10:38:38 +00:00
restKeysCache*: Table[ValidatorPubKey, ValidatorIndex]
Validator monitoring (#2925) Validator monitoring based on and mostly compatible with the implementation in Lighthouse - tracks additional logs and metrics for specified validators so as to stay on top on performance. The implementation works more or less the following way: * Validator pubkeys are singled out for monitoring - these can be running on the node or not * For every action that the validator takes, we record steps in the process such as messages being seen on the network or published in the API * When the dust settles at the end of an epoch, we report the information from one epoch before that, which coincides with the balances being updated - this is a tradeoff between being correct (waiting for finalization) and providing relevant information in a timely manner)
2021-12-20 19:20:31 +00:00
validatorMonitor*: ref ValidatorMonitor
Implement #3129 (Optimized history traversals in the REST API) (#3219) * Fix REST some rest call signatures and implement a simple API benchmark tool * Implement #3129 (Optimized history traversals in the REST API) Other notable changes: The `updateStateData` procedure in the `blockchain_dag.nim` module is optimized to not rewind down to the last snapshot state saved in the database if the supplied input state can be used as a starting point instead. * Disallow await in withStateForBlockSlot
2022-01-05 14:49:10 +00:00
stateTtlCache*: StateTtlCache
Introduce message router (#3829) Whether new blocks/attestations/etc are produced internally or received via REST, their journey through the node is the same - to ensure that they get the same treatment (logging, metrics, processing), this PR moves the routing to a dedicated module and fixes several small differences that existed before. * `xxxValidator` -> `processMessageName` - the processor also was adding messages to pools, so we want the name to reflect that action * add missing "sent" metrics for some messages * document ignore policy better - already-seen messages are not actaully rebroadcast by libp2p * skip redundant signature checks for internal validators consistently
2022-07-06 16:11:44 +00:00
router*: ref MessageRouter
allow execution clients several seconds to construct blocks (#4012)
2022-08-23 16:19:52 +00:00
dynamicFeeRecipientsStore*: ref DynamicFeeRecipientsStore
implement /eth/v1/validator/register_validator (#4115)
2022-09-13 11:52:26 +00:00
externalBuilderRegistrations*:
Table[ValidatorPubKey, SignedValidatorRegistrationV1]
fix slow checking of unknown validators (#4590) We do a linear scan of all pubkeys for each validator and slot - this becomes expensive with large validator counts. * normalise BN/VC validator startup logging * fix crash when host cannot be resolved while adding remote validator * silence repeated log spam for unknown validators * print pubkey/index/activation mapping on startup/validator identification
2023-02-07 14:53:36 +00:00
dutyValidatorCount*: int
## Number of validators that we've checked for activation
VC: Remote BN clock offset monitoring. (#4846) * Initial commit. * Add algorithm in comment. Remove delays. Fix logging statement issues. Change update from epoch to slot. * Obtain timestamp earlier. * Add processing delays into algorithm. * Fix time offset logging to produce integers instead of strings. * Address review comments. * Fix copyright year. Fix updateStatus(). * Remove fields from Slot start log statement. Fix issues when BN do not support Nimbus Extensions. Rename metric name and type change. * Add beacon role to disable time offset check manually.
2023-06-28 13:33:07 +00:00
processingDelay*: Opt[Duration]
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
first batch of work towards the VC/BN split: - we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot - the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make - factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim - the VC loads its associated private keys from the datadir for its BN - most of the validator API calls have been implemented as a stub. - the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-22 17:04:52 +00:00
const
MaxEmptySlotCount* = uint64(10*60) div SECONDS_PER_SLOT
Separate validator duties from the Beacon Node (#976) * Separate validator duties from the Beacon Node * remove duplicate MaxEmptySlotCOunt * imports * have beacnde properly compile
2020-05-06 13:23:45 +00:00
stop discarding future epochs; remove a StateCache() construction (#1610) * stop discarding non-existent future epochs during epoch state transitions; remove a pointless StateCache() construction in advance_slots() * update nbench to pass StateCache to process_slots()
2020-09-07 15:04:33 +00:00
# TODO stew/sequtils2
openarray -> openArray
2020-10-28 18:35:31 +00:00
template findIt*(s: openArray, predicate: untyped): int =
more work on the BN/VC split - fixed comments from the last review - getting more data VIA RPC, moved some code back into the BN only - attestation duties being requested as well
2020-05-27 17:06:28 +00:00
var res = -1
for i, it {.inject.} in s:
if predicate:
res = i
break
res
move rpc api to own folder, mimic upstream structure (#1905) also implements a few more endpoints
2020-10-27 09:00:57 +00:00
Keymanager API for the validator client (#3976) * Keymanager API for the validator client * Properly treat the 'description' field as optional when loading Keystores * Spec-compliant serialization of the slashing data in Keymanager's DeleteKeys response () Fixes #3940 Fixes #3964 Closes #3884 by adding test
2022-08-19 10:30:07 +00:00
template rng*(node: BeaconNode): ref HmacDrbgContext =
node.network.rng
move rpc api to own folder, mimic upstream structure (#1905) also implements a few more endpoints
2020-10-27 09:00:57 +00:00
proc currentSlot*(node: BeaconNode): Slot =
node.beaconClock.now.slotOrZero
refactor payload builder REST client usage (#4973) * refactor payload builder REST client usage * change HTTP response code
2023-05-25 15:38:56 +00:00
per-validator payload builder configuration (#5062)
2023-06-25 12:00:17 +00:00
func getPayloadBuilderAddress*(config: BeaconNodeConf): Opt[string] =
if config.payloadBuilderEnable:
Opt.some config.payloadBuilderUrl
else:
Opt.none(string)
allow payload builder client to be function of validator/proposer (#5015) * allow payload builder client to be function of validator/proposer * fileExists has side effects on Windows and only Windows * another not-always-func
2023-06-02 11:06:33 +00:00
proc getPayloadBuilderClient*(
node: BeaconNode, validator_index: uint64): RestResult[RestClientRef] =
per-validator payload builder configuration (#5062)
2023-06-25 12:00:17 +00:00
if not node.config.payloadBuilderEnable:
return err "Payload builder globally disabled"
let
defaultPayloadBuilderAddress = node.config.getPayloadBuilderAddress
pubkey = withState(node.dag.headState):
if validator_index >= forkyState.data.validators.lenu64:
return err "Validator index too high"
forkyState.data.validators.item(validator_index).pubkey
payloadBuilderAddress =
if node.keyManagerHost.isNil:
defaultPayloadBuilderAddress
else:
node.keyManagerHost[].getBuilderConfig(pubkey).valueOr:
defaultPayloadBuilderAddress
if payloadBuilderAddress.isNone:
return err "Payload builder disabled"
let res = RestClientRef.new(payloadBuilderAddress.get)
if res.isOk and res.get.isNil:
err "Got nil payload builder REST client reference"
refactor payload builder REST client usage (#4973) * refactor payload builder REST client usage * change HTTP response code
2023-05-25 15:38:56 +00:00
else:
per-validator payload builder configuration (#5062)
2023-06-25 12:00:17 +00:00
res