status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/consensus_object_pools/spec_cache.nim

200 lines
8.0 KiB
Nim
Raw Normal View History

removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
# beacon_chain
use IntSet rather than HashSet[ValidatorIndex] (#2267) * use IntSet rather than HashSet[ValidatorIndex] * add bounds check before uint64 -> int conversion * use intsets in block transitions * remove superfluous Nim issue explanation/reference
2021-01-26 11:52:00 +00:00
# Copyright (c) 2018-2021 Status Research & Development GmbH
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
fix broken metrics during replay (#2090) * move metrics out of state transition * add validator count metric * remove expensive beacon_current_validators, beacon_previous_validators metrics (they should be reimplemented with cache), add cheap beacon_active_validators to approximate * remove unused validator count metrics * tidy imports/defects
2020-11-27 22:16:13 +00:00
{.push raises: [Defect].}
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
import
refactor executionPayload tests; reduce HashSet creation (#3003)
2021-10-20 11:36:38 +00:00
std/sequtils,
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
chronicles,
remove one cache, add another (#1449) * remove one cache, add another This cache removes the need for rewinding in most attestation validation flow since the attestations come from one of two epochs and must be targetting a viable block. Additionally, it also removes all state caches which are less likely to be used over-all - more metrics are needed to track the rewinding. On risk is that when chains don't finalize, we'll have lots of epochrefs in memory meaning lots of validator key databases, most being exactly the same. This can be addressed in any number of ways. Some of the memory usage is mitigated by the fact that we previously had lots of big state caches and now we're keeping only keys instead. * cleanups * doc
2020-08-06 19:48:47 +00:00
../extras,
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
../spec/[helpers, network, signatures, validator],
../spec/datatypes/base,
./block_pools_types, blockchain_dag
export
fixes (#2901) * export kvstore from beacon_chain_db * fix rest HashList deserialization * fix asTrusted
2021-09-27 09:24:58 +00:00
base, extras, block_pools_types
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
# Spec functions implemented based on cached values instead of the full state
func count_active_validators*(epochInfo: EpochRef): uint64 =
epochInfo.shuffled_active_validator_indices.lenu64
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_committee_count_per_slot
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
func get_committee_count_per_slot*(epochInfo: EpochRef): uint64 =
get_committee_count_per_slot(count_active_validators(epochInfo))
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
iterator get_committee_indices*(epochRef: EpochRef): CommitteeIndex =
for i in 0'u64..<get_committee_count_per_slot(epochRef):
yield CommitteeIndex(i)
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_beacon_committee
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
iterator get_beacon_committee*(
epochRef: EpochRef, slot: Slot, index: CommitteeIndex): ValidatorIndex =
update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990)
2021-10-14 06:30:21 +00:00
## Return the beacon committee at ``slot`` for ``index``.
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
let
committees_per_slot = get_committee_count_per_slot(epochRef)
for idx in compute_committee(
epochRef.shuffled_active_validator_indices,
(slot mod SLOTS_PER_EPOCH) * committees_per_slot +
index.uint64,
committees_per_slot * SLOTS_PER_EPOCH
): yield idx
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_beacon_committee
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
func get_beacon_committee*(
epochRef: EpochRef, slot: Slot, index: CommitteeIndex): seq[ValidatorIndex] =
update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990)
2021-10-14 06:30:21 +00:00
## Return the beacon committee at ``slot`` for ``index``.
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
let
committees_per_slot = get_committee_count_per_slot(epochRef)
compute_committee(
epochRef.shuffled_active_validator_indices,
(slot mod SLOTS_PER_EPOCH) * committees_per_slot +
index.uint64,
committees_per_slot * SLOTS_PER_EPOCH
)
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_beacon_committee
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
func get_beacon_committee_len*(
epochRef: EpochRef, slot: Slot, index: CommitteeIndex): uint64 =
update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990)
2021-10-14 06:30:21 +00:00
## Return the number of members in the beacon committee at ``slot`` for ``index``.
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
let
committees_per_slot = get_committee_count_per_slot(epochRef)
compute_committee_len(
count_active_validators(epochRef),
(slot mod SLOTS_PER_EPOCH) * committees_per_slot +
index.uint64,
committees_per_slot * SLOTS_PER_EPOCH
)
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_attesting_indices
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
iterator get_attesting_indices*(epochRef: EpochRef,
data: AttestationData,
bits: CommitteeValidatorsBits):
ValidatorIndex =
if bits.lenu64 != get_beacon_committee_len(epochRef, data.slot, data.index.CommitteeIndex):
trace "get_attesting_indices: inconsistent aggregation and committee length"
else:
var i = 0
for index in get_beacon_committee(epochRef, data.slot, data.index.CommitteeIndex):
if bits[i]:
yield index
inc i
remove attestation/aggregate queue (#2519) With the introduction of batching and lazy attestation aggregation, it no longer makes sense to enqueue attestations between the signature check and adding them to the attestation pool - this only takes up valuable CPU without any real benefit. * add successfully validated attestations to attestion pool directly * avoid copying participant list around for single-vote attestations, pass single validator index instead * release decompressed gossip memory earlier, specially during async message validation * use cooked signatures in a few more places to avoid reloads and errors * remove some Defect-raising versions of signature-loading * release decompressed data memory before validating message
2021-04-26 20:39:44 +00:00
func get_attesting_indices_one*(epochRef: EpochRef,
data: AttestationData,
bits: CommitteeValidatorsBits):
Option[ValidatorIndex] =
# A variation on get_attesting_indices that returns the validator index only
# if only one validator index is set
if bits.lenu64 != get_beacon_committee_len(epochRef, data.slot, data.index.CommitteeIndex):
trace "get_attesting_indices: inconsistent aggregation and committee length"
none(ValidatorIndex)
else:
var res = none(ValidatorIndex)
var i = 0
for index in get_beacon_committee(epochRef, data.slot, data.index.CommitteeIndex):
if bits[i]:
if res.isNone():
res = some(index)
else:
return none(ValidatorIndex)
inc i
res
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#get_attesting_indices
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
func get_attesting_indices*(epochRef: EpochRef,
data: AttestationData,
bits: CommitteeValidatorsBits):
remove several IntSet usages in lieu of seq[ValidatorIndex] (#2288) * remove several IntSet usages in lieu of seq[ValidatorIndex] * convert smaller types to larger types * larger type, again
2021-02-08 07:27:30 +00:00
seq[ValidatorIndex] =
# TODO sequtils2 mapIt
misc memory and perf fixes (#1899) * misc memory and perf fixes * use EpochRef for attestation aggregation * compress effective balances in memory (medalla unfinalized: 4gb -> 1gb) * avoid hitting db when rewinding to head or clearance state * avoid hitting db when blocks can be applied to in-memory state - speeds up startup considerably * avoid storing epochref in fork choice * simplify and speed up beacon block creation flow - avoids state reload thanks to head rewind optimization * iterator-based committee and attestation participation help avoid lots of small memory allocations throughout epoch transition (40% speedup on epoch processing, for example during startup) * add constant for threshold
2020-10-22 10:53:33 +00:00
for idx in get_attesting_indices(epochRef, data, bits):
remove several IntSet usages in lieu of seq[ValidatorIndex] (#2288) * remove several IntSet usages in lieu of seq[ValidatorIndex] * convert smaller types to larger types * larger type, again
2021-02-08 07:27:30 +00:00
result.add(idx)
removed the BlockPool type and all of the proxy functions around it (#1401) * removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type * fixed the build after the rebase
2020-07-30 19:18:17 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/beacon-chain.md#is_valid_indexed_attestation
implement clock disparity for attestation validation (#1568) This implements disparity, resolving a part of https://github.com/status-im/nim-beacon-chain/issues/1367 * make BeaconTime a duration for fractional seconds * factor out attestation/aggregate validation * simplify recording of queued attestations * simplify attestation signature check * fix blocks_received metric * add some trivial validation tests * remove unresolved attestation table - attestations for unknown blocks are dropped instead (cannot verify their signature)
2020-08-27 07:34:12 +00:00
proc is_valid_indexed_attestation*(
fork: Fork, genesis_validators_root: Eth2Digest,
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
epochRef: EpochRef,
implement clock disparity for attestation validation (#1568) This implements disparity, resolving a part of https://github.com/status-im/nim-beacon-chain/issues/1367 * make BeaconTime a duration for fractional seconds * factor out attestation/aggregate validation * simplify recording of queued attestations * simplify attestation signature check * fix blocks_received metric * add some trivial validation tests * remove unresolved attestation table - attestations for unknown blocks are dropped instead (cannot verify their signature)
2020-08-27 07:34:12 +00:00
attestation: SomeAttestation, flags: UpdateFlags): Result[void, cstring] =
# This is a variation on `is_valid_indexed_attestation` that works directly
# with an attestation instead of first constructing an `IndexedAttestation`
# and then validating it - for the purpose of validating the signature, the
# order doesn't matter and we can proceed straight to validating the
# signature instead
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
let sigs = attestation.aggregation_bits.countOnes()
if sigs == 0:
return err("is_valid_indexed_attestation: no attesting indices")
implement clock disparity for attestation validation (#1568) This implements disparity, resolving a part of https://github.com/status-im/nim-beacon-chain/issues/1367 * make BeaconTime a duration for fractional seconds * factor out attestation/aggregate validation * simplify recording of queued attestations * simplify attestation signature check * fix blocks_received metric * add some trivial validation tests * remove unresolved attestation table - attestations for unknown blocks are dropped instead (cannot verify their signature)
2020-08-27 07:34:12 +00:00
# Verify aggregate signature
avoid copying keys for trusted signatures this removes `is_valid_indexed_attestion` from performance benchmarks when replaying blocks
2021-01-25 16:09:31 +00:00
if not (skipBLSValidation in flags or attestation.signature is TrustedSig):
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
var
singe validator key cache Instead of keeping a validator key list per EpochRef, this PR introduces a single shared validator key list in ChainDAG, and cleans up some other ChainDAG and key-related issues. The PR does not introduce the validator key list in the state transition - this is because we batch-check all signatures before entering the spec code, thus the spec code never hits the cache. A future refactor should _probably_ remove the threadvar altogether. There's a few other small fixes in here that make the flow easier to read: * fix `var ChainDAGRef` -> `ChainDAGRef` * fix `var QuarantineRef` -> `QuarantineRef` * consistent `dag` variable name * avoid using threadvar pubkey cache in most cases * better error messages in batch signature checking
2021-06-01 11:13:40 +00:00
pubkeys = newSeqOfCap[CookedPubKey](sigs)
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
for index in get_attesting_indices(
epochRef, attestation.data, attestation.aggregation_bits):
write uncompressed validator keys to database (#2639) * write uncompressed validator keys to database Loading 150k+ validator keys on startup in compressed format takes a lot of time - better store them in uncompressed format which makes behaviour just after startup faster / more predictable. * refactor cached validator key access * fix isomorphic cast to work with non-var instances * remove cooked pubkey cache - directly use database cache in chaindag as well (one less cache to keep in sync) * bump blscurve, introduce loadValid for known-to-be-valid keys
2021-06-10 07:37:02 +00:00
pubkeys.add(epochRef.validatorKey(index).get())
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
implement clock disparity for attestation validation (#1568) This implements disparity, resolving a part of https://github.com/status-im/nim-beacon-chain/issues/1367 * make BeaconTime a duration for fractional seconds * factor out attestation/aggregate validation * simplify recording of queued attestations * simplify attestation signature check * fix blocks_received metric * add some trivial validation tests * remove unresolved attestation table - attestations for unknown blocks are dropped instead (cannot verify their signature)
2020-08-27 07:34:12 +00:00
if not verify_attestation_signature(
fork, genesis_validators_root, attestation.data,
pubkeys, attestation.signature):
attestation processing speedups * avoid creating indexed attestation just to check signatures - above all, don't create it when not checking signatures ;) * avoid pointer op when adding attestation to pool * better iterator for yielding attestations * add metric / log for attestation packing time
2021-04-14 14:43:29 +00:00
return err("is_valid_indexed_attestation: signature verification failure")
remove one cache, add another (#1449) * remove one cache, add another This cache removes the need for rewinding in most attestation validation flow since the attestations come from one of two epochs and must be targetting a viable block. Additionally, it also removes all state caches which are less likely to be used over-all - more metrics are needed to track the rewinding. On risk is that when chains don't finalize, we'll have lots of epochrefs in memory meaning lots of validator key databases, most being exactly the same. This can be addressed in any number of ways. Some of the memory usage is mitigated by the fact that we previously had lots of big state caches and now we're keeping only keys instead. * cleanups * doc
2020-08-06 19:48:47 +00:00
small spec cleanups (#1501) * clean up logging a bit * return error on indexed attestation check
2020-08-13 13:47:06 +00:00
ok()
use epochcache for attesting (#1478)
2020-08-10 13:21:31 +00:00
func makeAttestationData*(
epochRef: EpochRef, bs: BlockSlot,
Datatype cleanup (#1953) * clear up spec todo * test fix * remove unnecessary toSszType * type * one more
2020-11-04 21:52:47 +00:00
committee_index: CommitteeIndex): AttestationData =
use epochcache for attesting (#1478)
2020-08-10 13:21:31 +00:00
## Create an attestation / vote for the block `bs` using the
## data in `epochRef` to fill in the rest of the fields.
## `epochRef` is the epoch information corresponding to the `bs` advanced to
## the slot we're attesting to.
let
slot = bs.slot
current_epoch = slot.compute_epoch_at_slot()
epoch_boundary_slot = compute_start_slot_at_epoch(current_epoch)
epoch_boundary_block = bs.blck.atSlot(epoch_boundary_slot)
doAssert current_epoch == epochRef.epoch
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#attestation-data
use epochcache for attesting (#1478)
2020-08-10 13:21:31 +00:00
AttestationData(
slot: slot,
Datatype cleanup (#1953) * clear up spec todo * test fix * remove unnecessary toSszType * type * one more
2020-11-04 21:52:47 +00:00
index: committee_index.uint64,
use epochcache for attesting (#1478)
2020-08-10 13:21:31 +00:00
beacon_block_root: bs.blck.root,
source: epochRef.current_justified_checkpoint,
target: Checkpoint(
epoch: current_epoch,
root: epoch_boundary_block.blck.root
)
)
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067)
2021-11-09 07:40:41 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/validator.md#validator-assignments
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
iterator get_committee_assignments*(
refactor executionPayload tests; reduce HashSet creation (#3003)
2021-10-20 11:36:38 +00:00
epochRef: EpochRef, validator_indices: HashSet[ValidatorIndex]):
Better REST/RPC error messages (#3046) * Better REST/RPC error messages * homogenise block logging (root first) * homegenise message verification pipeline (verify in `gossip_verification`, act in `eth2_processor`) * use `subcommitteeIdx` consistently * log each sent contribution * fix block_sim * fix block topic * don't recalc root on gossip block validation * move position loop into sync pool
2021-11-05 15:39:47 +00:00
tuple[subcommitteeIdx: CommitteeIndex,
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
subnet_id: SubnetId, slot: Slot] =
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
let
committees_per_slot = get_committee_count_per_slot(epochRef)
register vc duties with subnet tracker (#2949) * register vc duties with subnet tracker * fix activation logging during startup * cache slot signature to avoid duplicate signature work * schedule aggregation duties one slot at a time to avoid CPU spike at each epoch * lower aggregation subnet pre-subscription time to 4 slots (lowers bandwidth and CPU usage) * update stability subnets in ENR on startup * log gossip state * perform gossip subscriptions just before the next slot starts * document stuff * add random include * don't overwrite subscription state when not subscribed * log target gossip state * updating gossip status once is enough * add test * remove syncQueueLen - this one is not updated at the end of the sync and may cause gossip to disconnect itself completely - use a simple head distance instead * fix gossip disconnection - if in hysteresis, node.gossipState will be set to disabled even though we don't disable topic subscriptions * fix extra duty registration call
2021-10-18 09:11:44 +00:00
epoch = epochRef.epoch
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
start_slot = compute_start_slot_at_epoch(epoch)
for slot in start_slot ..< start_slot + SLOTS_PER_EPOCH:
for index in 0'u64 ..< committees_per_slot:
let
idx = index.CommitteeIndex
refactor executionPayload tests; reduce HashSet creation (#3003)
2021-10-20 11:36:38 +00:00
if anyIt(
get_beacon_committee(epochRef, slot, idx), it in validator_indices):
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
yield (
refactor executionPayload tests; reduce HashSet creation (#3003)
2021-10-20 11:36:38 +00:00
idx,
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
compute_subnet_for_attestation(committees_per_slot, slot, idx),
use epochref for get_committee_assignments() to avoid repeated shuffling (#2463) * use epochref for get_committee_assignments() to avoid repeated shuffling * remove unnecessary imports and StateCache() construction
2021-03-30 15:01:47 +00:00
slot)
cleanups (#2809) * cleanups * use ForkedTrustedSignedBeaconBlock.ionit where appropriate * move `is_aggregator` to `spec/` * use `errReject` in a few more places * update enr fork id when time is auspicious * use network broadcast functions * Return Ignore for aggregate signature validation timeouts ...consistently between aggregates and attestations. * clean up some more reject/ignore rules * shorten texts a bit * errReject->checkedReject, use err helpers throughout * get rid of quarantine in exitpool as well
2021-08-24 19:49:51 +00:00
func is_aggregator*(epochRef: EpochRef, slot: Slot, index: CommitteeIndex,
slot_signature: ValidatorSig): bool =
let
committee_len = get_beacon_committee_len(epochRef, slot, index)
return is_aggregator(committee_len, slot_signature)