status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/tests/test_light_client_processor...

306 lines
12 KiB
Nim
Raw Normal View History

introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
# beacon_chain
# Copyright (c) 2022 Status Research & Development GmbH
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.used.}
import
# Status libraries
chronos, eth/keys,
# Beacon chain internals
../beacon_chain/consensus_object_pools/
[block_clearance, block_quarantine, blockchain_dag],
../beacon_chain/gossip_processing/light_client_processor,
../beacon_chain/spec/[beacon_time, light_client_sync, state_transition],
# Test utilities
./testutil, ./testdbutil
suite "Light client processor" & preset():
let
cfg = block:
var res = defaultRuntimeConfig
res.ALTAIR_FORK_EPOCH = GENESIS_EPOCH + 1
res
const numValidators = SLOTS_PER_EPOCH
let
validatorMonitor = newClone(ValidatorMonitor.init())
dag = ChainDAGRef.init(
cfg, makeTestDB(numValidators), validatorMonitor, {},
encapsulate LC config into one type (#3817) Separate LC initialization options from the main ChainDAGRef options to allow ChainDAGRef to treat them as opaque and reduce risk for conflicts when extending those options in the future.
2022-06-28 20:52:29 +00:00
lcDataConfig = LightClientDataConfig(
serve: true,
importMode: LightClientDataImportMode.OnlyNew))
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
quarantine = newClone(Quarantine.init())
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
taskpool = Taskpool.new()
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
var verifier = BatchVerifier(rng: keys.newRng(), taskpool: taskpool)
var cache: StateCache
proc addBlocks(blocks: uint64, syncCommitteeRatio: float) =
for blck in makeTestBlocks(dag.headState, cache, blocks.int,
attested = true, syncCommitteeRatio, cfg):
let added =
case blck.kind
of BeaconBlockFork.Phase0:
const nilCallback = OnPhase0BlockAdded(nil)
dag.addHeadBlock(verifier, blck.phase0Data, nilCallback)
of BeaconBlockFork.Altair:
const nilCallback = OnAltairBlockAdded(nil)
dag.addHeadBlock(verifier, blck.altairData, nilCallback)
of BeaconBlockFork.Bellatrix:
const nilCallback = OnBellatrixBlockAdded(nil)
dag.addHeadBlock(verifier, blck.bellatrixData, nilCallback)
add Capella support to Forked* (#4276) * add Capella support to Forked* * remove cruft * add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
of BeaconBlockFork.Capella:
raiseAssert $capellaImplementationMissing
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
doAssert added.isOk()
dag.updateHead(added[], quarantine[])
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
addBlocks(SLOTS_PER_EPOCH, 0.82)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
let
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
genesis_validators_root = dag.genesis_validators_root
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
trustedBlockRoot = dag.head.root
add light client (standalone) (#3653) Introduces a new library for syncing using libp2p based light client sync protocol, and adds a new `nimbus_light_client` executable that uses this library for syncing. The new executable emits log messages when new beacon block headers are received, and is integrated into testing.
2022-05-31 10:45:37 +00:00
proc getTrustedBlockRoot(): Option[Eth2Digest] =
some trustedBlockRoot
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
const
lowPeriod = 0.SyncCommitteePeriod
lastPeriodWithSupermajority = 3.SyncCommitteePeriod
highPeriod = 5.SyncCommitteePeriod
for period in lowPeriod .. highPeriod:
const numFilledEpochsPerPeriod = 3
let slot = ((period + 1).start_epoch - numFilledEpochsPerPeriod).start_slot
var info: ForkedEpochInfo
doAssert process_slots(cfg, dag.headState, slot,
cache, info, flags = {}).isOk()
let syncCommitteeRatio =
if period > lastPeriodWithSupermajority:
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
0.62
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
else:
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
0.82
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
addBlocks(numFilledEpochsPerPeriod * SLOTS_PER_EPOCH, syncCommitteeRatio)
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
for finalizationMode in LightClientFinalizationMode:
let testNameSuffix = " (" & $finalizationMode & ")" & preset()
setup:
var time = chronos.seconds(0)
proc getBeaconTime(): BeaconTime =
BeaconTime(ns_since_genesis: time.nanoseconds)
func setTimeToSlot(slot: Slot) =
time = chronos.seconds((slot * SECONDS_PER_SLOT).int64)
var numOnStoreInitializedCalls = 0
func onStoreInitialized() = inc numOnStoreInitializedCalls
let store = (ref Option[LightClientStore])()
var
processor = LightClientProcessor.new(
false, "", "", cfg, genesis_validators_root, finalizationMode,
store, getBeaconTime, getTrustedBlockRoot, onStoreInitialized)
res: Result[bool, BlockError]
test "Sync" & testNameSuffix:
let bootstrap = dag.getLightClientBootstrap(trustedBlockRoot)
check bootstrap.isOk
setTimeToSlot(bootstrap.get.header.slot)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
res = processor[].storeObject(
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
MsgSource.gossip, getBeaconTime(), bootstrap.get)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
check:
res.isOk
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
numOnStoreInitializedCalls == 1
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
# Reduce stack size by making this a `proc`
proc applyPeriodWithSupermajority(period: SyncCommitteePeriod) =
let update = dag.getLightClientUpdateForPeriod(period)
check update.isSome
setTimeToSlot(update.get.signature_slot)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), update.get)
check:
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
res.isOk
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
store[].isSome
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
if update.get.finalized_header.slot > bootstrap.get.header.slot:
store[].get.finalized_header == update.get.finalized_header
else:
store[].get.finalized_header == bootstrap.get.header
store[].get.optimistic_header == update.get.attested_header
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
for period in lowPeriod .. lastPeriodWithSupermajority:
applyPeriodWithSupermajority(period)
# Reduce stack size by making this a `proc`
proc applyPeriodWithoutSupermajority(period: SyncCommitteePeriod) =
let update = dag.getLightClientUpdateForPeriod(period)
check update.isSome
setTimeToSlot(update.get.signature_slot)
for i in 0 ..< 2:
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), update.get)
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
if finalizationMode == LightClientFinalizationMode.Optimistic or
period == lastPeriodWithSupermajority + 1:
if finalizationMode == LightClientFinalizationMode.Optimistic or
i == 0:
check:
res.isOk
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get == update.get
else:
check:
res.isErr
res.error == BlockError.Duplicate
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get == update.get
else:
check:
res.isErr
res.error == BlockError.MissingParent
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get != update.get
proc applyDuplicate() = # Reduce stack size by making this a `proc`
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), update.get)
if finalizationMode == LightClientFinalizationMode.Optimistic or
period == lastPeriodWithSupermajority + 1:
check:
res.isErr
res.error == BlockError.Duplicate
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get == update.get
else:
check:
res.isErr
res.error == BlockError.MissingParent
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get != update.get
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
applyDuplicate()
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
time += chronos.minutes(15)
for _ in 0 ..< 150:
applyDuplicate()
time += chronos.seconds(5)
time += chronos.minutes(15)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), update.get)
if finalizationMode == LightClientFinalizationMode.Optimistic:
check:
res.isErr
res.error == BlockError.Duplicate
store[].isSome
store[].get.best_valid_update.isNone
if store[].get.finalized_header == update.get.attested_header:
break
check store[].get.finalized_header == update.get.finalized_header
elif period == lastPeriodWithSupermajority + 1:
check:
res.isErr
res.error == BlockError.Duplicate
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get == update.get
else:
check:
res.isErr
res.error == BlockError.MissingParent
store[].isSome
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get != update.get
if finalizationMode == LightClientFinalizationMode.Optimistic:
check store[].get.finalized_header == update.get.attested_header
else:
check store[].get.finalized_header != update.get.attested_header
for period in lastPeriodWithSupermajority + 1 .. highPeriod:
applyPeriodWithoutSupermajority(period)
let
previousFinalized = store[].get.finalized_header
finalityUpdate = dag.getLightClientFinalityUpdate()
check finalityUpdate.isSome
setTimeToSlot(finalityUpdate.get.signature_slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), finalityUpdate.get)
if res.isOk:
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
check:
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
finalizationMode == LightClientFinalizationMode.Optimistic
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
store[].isSome
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
store[].get.finalized_header == previousFinalized
store[].get.best_valid_update.isSome
store[].get.best_valid_update.get.matches(finalityUpdate.get)
store[].get.optimistic_header == finalityUpdate.get.attested_header
elif finalizationMode == LightClientFinalizationMode.Optimistic:
check res.error == BlockError.Duplicate
else:
check res.error == BlockError.MissingParent
check numOnStoreInitializedCalls == 1
test "Invalid bootstrap" & testNameSuffix:
var bootstrap = dag.getLightClientBootstrap(trustedBlockRoot)
check bootstrap.isOk
bootstrap.get.header.slot.inc()
setTimeToSlot(bootstrap.get.header.slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), bootstrap.get)
check:
res.isErr
res.error == BlockError.Invalid
numOnStoreInitializedCalls == 0
test "Duplicate bootstrap" & testNameSuffix:
let bootstrap = dag.getLightClientBootstrap(trustedBlockRoot)
check bootstrap.isOk
setTimeToSlot(bootstrap.get.header.slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), bootstrap.get)
check:
res.isOk
numOnStoreInitializedCalls == 1
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), bootstrap.get)
check:
res.isErr
res.error == BlockError.Duplicate
numOnStoreInitializedCalls == 1
test "Missing bootstrap (update)" & testNameSuffix:
let update = dag.getLightClientUpdateForPeriod(lowPeriod)
check update.isSome
setTimeToSlot(update.get.signature_slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), update.get)
check:
res.isErr
res.error == BlockError.MissingParent
numOnStoreInitializedCalls == 0
test "Missing bootstrap (finality update)" & testNameSuffix:
let finalityUpdate = dag.getLightClientFinalityUpdate()
check finalityUpdate.isSome
setTimeToSlot(finalityUpdate.get.signature_slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), finalityUpdate.get)
check:
res.isErr
res.error == BlockError.MissingParent
numOnStoreInitializedCalls == 0
test "Missing bootstrap (optimistic update)" & testNameSuffix:
let optimisticUpdate = dag.getLightClientOptimisticUpdate()
check optimisticUpdate.isSome
setTimeToSlot(optimisticUpdate.get.signature_slot)
res = processor[].storeObject(
MsgSource.gossip, getBeaconTime(), optimisticUpdate.get)
introduce light client processor (#3509) Adds `LightClientProcessor` as the pendant to `BlockProcessor` while operating in light client mode. Note that a similar mechanism based on async futures is used for interoperability with existing infrastructure, despite light client object validation being done synchronously.
2022-03-17 22:26:56 +00:00
check:
add strict mode to light client processor (#3894) The light client sync protocol employs heuristics to ensure it does not become stuck during non-finality or low sync committee participation. These can enable use cases that prefer availability of recent data over security. For our syncing use case, though, security is preferred. An option is added to light client processor to configure this tradeoff.
2022-07-21 09:16:10 +00:00
res.isErr
res.error == BlockError.MissingParent
numOnStoreInitializedCalls == 0