status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/light_client.nim

373 lines
13 KiB
Nim
Raw Normal View History

add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
# beacon_chain
# Copyright (c) 2022 Status Research & Development GmbH
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
compatibility with Nim 1.4+ (#3888)
2022-07-29 10:53:42 +00:00
when (NimMajor, NimMinor) < (1, 4):
{.push raises: [Defect].}
else:
{.push raises: [].}
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
import
chronicles,
eth/keys,
./gossip_processing/light_client_processor,
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
./networking/[eth2_network, topic_params],
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
./spec/datatypes/altair,
./spec/helpers,
./sync/light_client_manager,
"."/[beacon_clock, conf_light_client]
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
export LightClientFinalizationMode, eth2_network, conf_light_client
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
logScope: topics = "lightcl"
type
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
LightClientHeaderCallback* =
proc(lightClient: LightClient, header: BeaconBlockHeader) {.
gcsafe, raises: [Defect].}
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
LightClient* = ref object
network: Eth2Node
cfg: RuntimeConfig
forkDigests: ref ForkDigests
getBeaconTime: GetBeaconTimeFn
store: ref Option[LightClientStore]
processor: ref LightClientProcessor
manager: LightClientManager
gossipState: GossipState
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
onFinalizedHeader*, onOptimisticHeader*: LightClientHeaderCallback
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
trustedBlockRoot*: Option[Eth2Digest]
func finalizedHeader*(lightClient: LightClient): Opt[BeaconBlockHeader] =
if lightClient.store[].isSome:
ok lightClient.store[].get.finalized_header
else:
err()
func optimisticHeader*(lightClient: LightClient): Opt[BeaconBlockHeader] =
if lightClient.store[].isSome:
ok lightClient.store[].get.optimistic_header
else:
err()
persist LC sync progress across restarts (#4371) Persist the latest finalized header and sync committee across restarts of `nimbus_light_client` to avoid redoing time-consuming bootstrap step.
2022-11-30 03:45:03 +00:00
func finalizedSyncCommittee*(
lightClient: LightClient): Opt[altair.SyncCommittee] =
if lightClient.store[].isSome:
ok lightClient.store[].get.current_sync_committee
else:
err()
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc createLightClient(
network: Eth2Node,
bearssl: split abi (#3755)
2022-06-21 08:29:16 +00:00
rng: ref HmacDrbgContext,
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
dumpEnabled: bool,
dumpDirInvalid, dumpDirIncoming: string,
cfg: RuntimeConfig,
forkDigests: ref ForkDigests,
getBeaconTime: GetBeaconTimeFn,
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
genesis_validators_root: Eth2Digest,
finalizationMode: LightClientFinalizationMode
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
): LightClient =
let lightClient = LightClient(
network: network,
cfg: cfg,
forkDigests: forkDigests,
getBeaconTime: getBeaconTime,
store: (ref Option[LightClientStore])())
func getTrustedBlockRoot(): Option[Eth2Digest] =
lightClient.trustedBlockRoot
proc onStoreInitialized() =
discard
proc onFinalizedHeader() =
if lightClient.onFinalizedHeader != nil:
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
lightClient.onFinalizedHeader(
lightClient, lightClient.finalizedHeader.get)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc onOptimisticHeader() =
if lightClient.onOptimisticHeader != nil:
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
lightClient.onOptimisticHeader(
lightClient, lightClient.optimisticHeader.get)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.processor = LightClientProcessor.new(
dumpEnabled, dumpDirInvalid, dumpDirIncoming,
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
cfg, genesis_validators_root, finalizationMode,
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.store, getBeaconTime, getTrustedBlockRoot,
onStoreInitialized, onFinalizedHeader, onOptimisticHeader)
proc lightClientVerifier(obj: SomeLightClientObject):
rename `BlockError` -> `VerifierError` (#4310) We currently use `BlockError` for both beacon blocks and LC objects. In light of EIP4844, we will likely also use it for blob sidecars. To avoid confusion, renaming it to a more generic `VerifierError`, and update its documentation to be more generic. To avoid long lines as a followup, also renaming the `block_processor`'s `BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and `BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
Future[Result[void, VerifierError]] =
let resfut = newFuture[Result[void, VerifierError]]("lightClientVerifier")
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.processor[].addObject(MsgSource.gossip, obj, resfut)
resfut
proc bootstrapVerifier(obj: altair.LightClientBootstrap): auto =
lightClientVerifier(obj)
proc updateVerifier(obj: altair.LightClientUpdate): auto =
lightClientVerifier(obj)
proc finalityVerifier(obj: altair.LightClientFinalityUpdate): auto =
lightClientVerifier(obj)
proc optimisticVerifier(obj: altair.LightClientOptimisticUpdate): auto =
lightClientVerifier(obj)
func isLightClientStoreInitialized(): bool =
lightClient.store[].isSome
func isNextSyncCommitteeKnown(): bool =
if lightClient.store[].isSome:
lightClient.store[].get.is_next_sync_committee_known
else:
false
func getFinalizedPeriod(): SyncCommitteePeriod =
if lightClient.store[].isSome:
lightClient.store[].get.finalized_header.slot.sync_committee_period
else:
GENESIS_SLOT.sync_committee_period
func getOptimisticPeriod(): SyncCommitteePeriod =
if lightClient.store[].isSome:
lightClient.store[].get.optimistic_header.slot.sync_committee_period
else:
GENESIS_SLOT.sync_committee_period
lightClient.manager = LightClientManager.init(
lightClient.network, rng, getTrustedBlockRoot,
bootstrapVerifier, updateVerifier, finalityVerifier, optimisticVerifier,
isLightClientStoreInitialized, isNextSyncCommitteeKnown,
getFinalizedPeriod, getOptimisticPeriod, getBeaconTime)
lightClient.gossipState = {}
lightClient
proc createLightClient*(
network: Eth2Node,
bearssl: split abi (#3755)
2022-06-21 08:29:16 +00:00
rng: ref HmacDrbgContext,
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
config: BeaconNodeConf,
cfg: RuntimeConfig,
forkDigests: ref ForkDigests,
getBeaconTime: GetBeaconTimeFn,
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
genesis_validators_root: Eth2Digest,
finalizationMode: LightClientFinalizationMode
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
): LightClient =
createLightClient(
network, rng,
config.dumpEnabled, config.dumpDirInvalid, config.dumpDirIncoming,
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
cfg, forkDigests, getBeaconTime, genesis_validators_root, finalizationMode)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc createLightClient*(
network: Eth2Node,
bearssl: split abi (#3755)
2022-06-21 08:29:16 +00:00
rng: ref HmacDrbgContext,
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
config: LightClientConf,
cfg: RuntimeConfig,
forkDigests: ref ForkDigests,
getBeaconTime: GetBeaconTimeFn,
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
genesis_validators_root: Eth2Digest,
finalizationMode: LightClientFinalizationMode
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
): LightClient =
createLightClient(
network, rng,
dumpEnabled = false, dumpDirInvalid = ".", dumpDirIncoming = ".",
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
cfg, forkDigests, getBeaconTime, genesis_validators_root, finalizationMode)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc start*(lightClient: LightClient) =
notice "Starting light client",
trusted_block_root = lightClient.trustedBlockRoot
lightClient.manager.start()
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
proc resetToFinalizedHeader*(
lightClient: LightClient,
header: BeaconBlockHeader,
current_sync_committee: SyncCommittee) =
lightClient.processor[].resetToFinalizedHeader(header, current_sync_committee)
import metrics
from
./gossip_processing/eth2_processor
import
Introduce message router (#3829) Whether new blocks/attestations/etc are produced internally or received via REST, their journey through the node is the same - to ensure that they get the same treatment (logging, metrics, processing), this PR moves the routing to a dedicated module and fixes several small differences that existed before. * `xxxValidator` -> `processMessageName` - the processor also was adding messages to pools, so we want the name to reflect that action * add missing "sent" metrics for some messages * document ignore policy better - already-seen messages are not actaully rebroadcast by libp2p * skip redundant signature checks for internal validators consistently
2022-07-06 16:11:44 +00:00
processLightClientFinalityUpdate, processLightClientOptimisticUpdate
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
declareCounter beacon_light_client_finality_updates_received,
"Number of valid LC finality updates processed by this node"
declareCounter beacon_light_client_finality_updates_dropped,
"Number of invalid LC finality updates dropped by this node", labels = ["reason"]
declareCounter beacon_light_client_optimistic_updates_received,
"Number of valid LC optimistic updates processed by this node"
declareCounter beacon_light_client_optimistic_updates_dropped,
"Number of invalid LC optimistic updates dropped by this node", labels = ["reason"]
template logReceived(
msg: altair.LightClientFinalityUpdate) =
debug "LC finality update received", finality_update = msg
template logValidated(
msg: altair.LightClientFinalityUpdate) =
trace "LC finality update validated", finality_update = msg
beacon_light_client_finality_updates_received.inc()
proc logDropped(
msg: altair.LightClientFinalityUpdate, es: varargs[ValidationError]) =
for e in es:
debug "Dropping LC finality update", finality_update = msg, error = e
beacon_light_client_finality_updates_dropped.inc(1, [$es[0][0]])
template logReceived(
msg: altair.LightClientOptimisticUpdate) =
debug "LC optimistic update received", optimistic_update = msg
template logValidated(
msg: altair.LightClientOptimisticUpdate) =
trace "LC optimistic update validated", optimistic_update = msg
beacon_light_client_optimistic_updates_received.inc()
proc logDropped(
msg: altair.LightClientOptimisticUpdate, es: varargs[ValidationError]) =
for e in es:
debug "Dropping LC optimistic update", optimistic_update = msg, error = e
beacon_light_client_optimistic_updates_dropped.inc(1, [$es[0][0]])
proc installMessageValidators*(
lightClient: LightClient, eth2Processor: ref Eth2Processor = nil) =
# When registering multiple message validators, IGNORE results take precedence
# over ACCEPT results. However, because the opposite behaviour is needed here,
# we handle both full node and light client validation in this module
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
template getLocalWallPeriod(): auto =
lightClient.getBeaconTime().slotOrZero().sync_committee_period
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
template validate[T: SomeLightClientObject](
msg: T, validatorProcName: untyped): ValidationResult =
msg.logReceived()
var
ignoreErrors {.noinit.}: array[2, ValidationError]
numIgnoreErrors = 0
let res1 =
if eth2Processor != nil:
let
v = eth2Processor[].`validatorProcName`(MsgSource.gossip, msg)
res = v.toValidationResult()
if res == ValidationResult.Reject:
msg.logDropped(v.error)
return res
if res == ValidationResult.Ignore:
ignoreErrors[numIgnoreErrors] = v.error
inc numIgnoreErrors
res
else:
ValidationResult.Ignore
let res2 =
if lightClient.manager.isGossipSupported(getLocalWallPeriod()):
let
v = lightClient.processor[].`validatorProcName`(MsgSource.gossip, msg)
res = v.toValidationResult()
if res == ValidationResult.Reject:
msg.logDropped(v.error)
return res
if res == ValidationResult.Ignore:
ignoreErrors[numIgnoreErrors] = v.error
inc numIgnoreErrors
res
else:
ValidationResult.Ignore
if res1 == ValidationResult.Accept or res2 == ValidationResult.Accept:
msg.logValidated()
return ValidationResult.Accept
doAssert res1 == ValidationResult.Ignore and res2 == ValidationResult.Ignore
if numIgnoreErrors == 0:
ignoreErrors[numIgnoreErrors] = static:
(ValidationResult.Ignore, cstring T.name & ": irrelevant")
inc numIgnoreErrors
msg.logDropped(ignoreErrors.toOpenArray(0, numIgnoreErrors - 1))
ValidationResult.Ignore
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
let forkDigests = lightClient.forkDigests
for digest in [forkDigests.altair, forkDigests.bellatrix]:
lightClient.network.addValidator(
getLightClientFinalityUpdateTopic(digest),
proc(msg: altair.LightClientFinalityUpdate): ValidationResult =
Introduce message router (#3829) Whether new blocks/attestations/etc are produced internally or received via REST, their journey through the node is the same - to ensure that they get the same treatment (logging, metrics, processing), this PR moves the routing to a dedicated module and fixes several small differences that existed before. * `xxxValidator` -> `processMessageName` - the processor also was adding messages to pools, so we want the name to reflect that action * add missing "sent" metrics for some messages * document ignore policy better - already-seen messages are not actaully rebroadcast by libp2p * skip redundant signature checks for internal validators consistently
2022-07-06 16:11:44 +00:00
validate(msg, processLightClientFinalityUpdate))
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.network.addValidator(
getLightClientOptimisticUpdateTopic(digest),
proc(msg: altair.LightClientOptimisticUpdate): ValidationResult =
Introduce message router (#3829) Whether new blocks/attestations/etc are produced internally or received via REST, their journey through the node is the same - to ensure that they get the same treatment (logging, metrics, processing), this PR moves the routing to a dedicated module and fixes several small differences that existed before. * `xxxValidator` -> `processMessageName` - the processor also was adding messages to pools, so we want the name to reflect that action * add missing "sent" metrics for some messages * document ignore policy better - already-seen messages are not actaully rebroadcast by libp2p * skip redundant signature checks for internal validators consistently
2022-07-06 16:11:44 +00:00
validate(msg, processLightClientOptimisticUpdate))
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc updateGossipStatus*(
lightClient: LightClient, slot: Slot, dagIsBehind = default(Option[bool])) =
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
template cfg(): auto = lightClient.cfg
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
let
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
epoch = slot.epoch
lcBehind =
not lightClient.manager.isGossipSupported(slot.sync_committee_period)
dagBehind =
# While separate message validators can be installed for both
# full node and light client (both are called unless one rejects msg),
# only a single subscription is supported per topic.
# The full node subscription is also handled in this module, even though
# it does not directly relate to the client side of the LC sync protocol
dagIsBehind.get(true)
isBehind = lcBehind and dagBehind
currentEpochTargetGossipState = getTargetGossipState(
epoch, cfg.ALTAIR_FORK_EPOCH, cfg.BELLATRIX_FORK_EPOCH, isBehind)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
targetGossipState =
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
if lcBehind or epoch < 1:
currentEpochTargetGossipState
else:
# The fork digest for light client topics depends on the attested slot,
# which is in the past relative to the signature slot (current slot).
# Therefore, LC topic subscriptions are kept for 1 extra epoch.
let previousEpochTargetGossipState = getTargetGossipState(
epoch - 1, cfg.ALTAIR_FORK_EPOCH, cfg.BELLATRIX_FORK_EPOCH, isBehind)
currentEpochTargetGossipState + previousEpochTargetGossipState
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
template currentGossipState(): auto = lightClient.gossipState
if currentGossipState == targetGossipState:
return
if currentGossipState.card == 0 and targetGossipState.card > 0:
debug "Enabling LC topic subscriptions",
wallSlot = slot, targetGossipState
elif currentGossipState.card > 0 and targetGossipState.card == 0:
debug "Disabling LC topic subscriptions",
wallSlot = slot
else:
# Individual forks added / removed
discard
let
newGossipForks = targetGossipState - currentGossipState
oldGossipForks = currentGossipState - targetGossipState
for gossipFork in oldGossipForks:
if gossipFork >= BeaconStateFork.Altair:
let forkDigest = lightClient.forkDigests[].atStateFork(gossipFork)
lightClient.network.unsubscribe(
getLightClientFinalityUpdateTopic(forkDigest))
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
lightClient.network.unsubscribe(
getLightClientOptimisticUpdateTopic(forkDigest))
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
for gossipFork in newGossipForks:
if gossipFork >= BeaconStateFork.Altair:
let forkDigest = lightClient.forkDigests[].atStateFork(gossipFork)
integrate light client into beacon node (#3557) Adds a `LightClient` instance to the beacon node as preparation to accelerate syncing in the future (optimistic sync). - `--light-client-enable` turns on the feature - `--light-client-trusted-block-root` configures block to start from If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 17:01:11 +00:00
lightClient.network.subscribe(
getLightClientFinalityUpdateTopic(forkDigest),
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
basicParams)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.network.subscribe(
getLightClientOptimisticUpdateTopic(forkDigest),
reduce LC optsync latency (#4002) The optimistic sync spec was updated since the LC based optsync module was introduced. It is no longer necessary to wait for the justified checkpoint to have execution enabled; instead, any block is okay to be optimistically imported to the EL client, as long as its parent block has execution enabled. Complex syncing logic has been removed, and the LC optsync module will now follow gossip directly, reducing the latency when using this module. Note that because this is now based on gossip instead of using sync manager / request manager, that individual blocks may be missed. However, EL clients should recover from this by fetching missing blocks themselves.
2022-08-25 03:53:59 +00:00
basicParams)
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
lightClient.gossipState = targetGossipState