status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/spec/network.nim

225 lines
9.9 KiB
Nim
Raw Normal View History

Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# beacon_chain
spec ref URL & copyright year updates (#4467)
2023-01-06 16:28:46 +00:00
# Copyright (c) 2018-2023 Status Research & Development GmbH
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# Licensed and distributed under either of
replace links to Apache and MIT licenses with HTTPS versions (#592)
2019-11-25 15:30:02 +00:00
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# at your option. This file may not be copied, modified, or distributed except according to those terms.
remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528)
2023-01-20 14:14:37 +00:00
{.push raises: [].}
spec: regulate exceptions (#913) * spec: regulate exceptions * a few more simple raises
2020-04-22 05:53:02 +00:00
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
import
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
"."/[helpers, forks],
remove false OnBlockAdded dependency on phase0 HashedBeaconState (#2661) * remove false OnBlockAdded dependency on phase.HashedBeaconState * introduce altair data types into block_clearance; update some alpha.6 spec refs to alpha.7; add get_active_validator_indices_len ForkedHashedBeaconState wrapper * switch many modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 and remove rm'd G2_POINT_AT_INFINITY * switch more modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 * remove unnecessary phase0-only wrapper of get_attesting_indices(); allow signatures_batch to process either fork; remove O(n^2) nested loop in process_inactivity_updates(); add altair support to getAttestationsforTestBlock() * add Altair versions of asSigVerified(), asTrusted(), and makeBeaconBlock() * fix spec URL to be Altair for Altair makeBeaconBlock()
2021-06-21 08:35:24 +00:00
"."/datatypes/base
split attestation beacon topic per honest validator spec
2020-02-21 12:16:58 +00:00
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
export base
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
const
automated consensus spec URL updating to v1.4.0-beta.1 (#5280)
2023-08-09 03:58:47 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.1/specs/phase0/p2p-interface.md#topics-and-messages
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/capella/p2p-interface.md#topics-and-messages
odds and ends (#3015) * `allSyncCommittees` => `allSyncSubcommittees` * simplify `_snappy` topic generation (avoid pointless string copies) * simplify gossip id generator (avoid pointless string copies) * avoid redundant syncnet ENR updates * simplify topic validation (allow only validated topics)
2021-10-21 13:09:19 +00:00
topicBeaconBlocksSuffix* = "beacon_block/ssz_snappy"
topicVoluntaryExitsSuffix* = "voluntary_exit/ssz_snappy"
topicProposerSlashingsSuffix* = "proposer_slashing/ssz_snappy"
topicAttesterSlashingsSuffix* = "attester_slashing/ssz_snappy"
topicAggregateAndProofsSuffix* = "beacon_aggregate_and_proof/ssz_snappy"
more capella (#4350)
2022-11-24 14:38:07 +00:00
topicBlsToExecutionChangeSuffix* = "bls_to_execution_change/ssz_snappy"
split attestation beacon topic per honest validator spec
2020-02-21 12:16:58 +00:00
remove 4 more missing constants in preset handling warning (#5428)
2023-09-14 12:37:35 +00:00
const
# The spec now includes this as a bare uint64 as `RESP_TIMEOUT`
RESP_TIMEOUT_DUR* = RESP_TIMEOUT.int64.seconds
[skip ci] Use GOSSIP_MAX_SIZE when snappy decoding in the inspector as well; Bumps
2020-08-19 11:33:52 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/light-client/p2p-interface.md#configuration
move spec constants to spec (#4825) * move spec constants to spec quite funny, this removes >_1000_ transitive imports when building `nimbus_validator_client`. `genDepend`: ```sh [arnetheduck@praeceps beacon_chain]$ diff -u nimbus_validator_client.dot.pre nimbus_validator_client.dot | wc 1167 3507 103822 ``` * imports * better constants * oops * too many blobs * reorder spec constants in fork order * cleanup
2023-04-17 17:18:54 +00:00
MAX_REQUEST_LIGHT_CLIENT_UPDATES* = 128
consensus spec v1.4.0-beta.5 URL updates (#5655)
2023-12-06 22:16:55 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/deneb/p2p-interface.md#configuration
remove 4 more missing constants in preset handling warning (#5428)
2023-09-14 12:37:35 +00:00
MAX_REQUEST_BLOB_SIDECARS*: uint64 =
MAX_REQUEST_BLOCKS_DENEB * MAX_BLOBS_PER_BLOCK
move spec constants to spec (#4825) * move spec constants to spec quite funny, this removes >_1000_ transitive imports when building `nimbus_validator_client`. `genDepend`: ```sh [arnetheduck@praeceps beacon_chain]$ diff -u nimbus_validator_client.dot.pre nimbus_validator_client.dot | wc 1167 3507 103822 ``` * imports * better constants * oops * too many blobs * reorder spec constants in fork order * cleanup
2023-04-17 17:18:54 +00:00
Initial implementation of a JSON-RPC service
2020-03-16 22:28:54 +00:00
defaultEth2TcpPort* = 9000
cleanup magic strings (#3891) Replaces hardcoded IP address strings with constants. Adjusts some LC log levels. Removes no longer used `RpcServer` type.
2022-07-20 18:17:21 +00:00
defaultEth2TcpPortDesc* = $defaultEth2TcpPort
Initial implementation of a JSON-RPC service
2020-03-16 22:28:54 +00:00
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
# This is not part of the spec! But it's port which Lighthouse uses
cleanup magic strings (#3891) Replaces hardcoded IP address strings with constants. Adjusts some LC log levels. Removes no longer used `RpcServer` type.
2022-07-20 18:17:21 +00:00
defaultEth2RestPort* = 5052
defaultEth2RestPortDesc* = $defaultEth2RestPort
Add TODO comments for missing implementations. Change default REST port to use 5052 (Lighthouse). Add missing checks for maximum amount of validator ids.
2021-03-29 10:59:39 +00:00
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
enrAttestationSubnetsField* = "attnets"
enrSyncSubnetsField* = "syncnets"
enrForkIdField* = "eth2"
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
template eth2Prefix(forkDigest: ForkDigest): string =
"/eth2/" & $forkDigest & "/"
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getBeaconBlocksTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicBeaconBlocksSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
Fix the inspector build
2020-04-15 03:11:45 +00:00
func getVoluntaryExitsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicVoluntaryExitsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getProposerSlashingsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicProposerSlashingsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getAttesterSlashingsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicAttesterSlashingsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getAggregateAndProofsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicAggregateAndProofsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/capella/p2p-interface.md#topics-and-messages
more capella (#4350)
2022-11-24 14:38:07 +00:00
func getBlsToExecutionChangeTopic*(forkDigest: ForkDigest): string =
eth2Prefix(forkDigest) & topicBlsToExecutionChangeSuffix
update some consensus-spec URLs to v1.4.0-beta.1 (#5429)
2023-09-14 18:23:59 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.1/specs/phase0/validator.md#broadcast-attestation
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
func compute_subnet_for_attestation*(
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346)
2020-07-22 08:04:21 +00:00
committees_per_slot: uint64, slot: Slot, committee_index: CommitteeIndex):
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
SubnetId =
update consensus layer spec ref URLs to v1.2.0-rc.3 (#4109)
2022-09-10 17:16:38 +00:00
## Compute the correct subnet for an attestation for Phase 0.
# Note, this mimics expected future behavior where attestations will be
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
# mapped to their shard subnet.
let
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
slots_since_epoch_start = slot.since_epoch_start()
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
committees_since_epoch_start =
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346)
2020-07-22 08:04:21 +00:00
committees_per_slot * slots_since_epoch_start
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
SubnetId(
time: spring cleaning (#3262) Time in the beacon chain is expressed relative to the genesis time - this PR creates a `beacon_time` module that collects helpers and utilities for dealing the time units - the new module does not deal with actual wall time (that's remains in `beacon_clock`). Collecting the time related stuff in one place makes it easier to find, avoids some circular imports and allows more easily identifying the code actually needs wall time to operate. * move genesis-time-related functionality into `spec/beacon_time` * avoid using `chronos.Duration` for time differences - it does not support negative values (such as when something happens earlier than it should) * saturate conversions between `FAR_FUTURE_XXX`, so as to avoid overflows * fix delay reporting in validator client so it uses the expected deadline of the slot, not "closest wall slot" * simplify looping over the slots of an epoch * `compute_start_slot_at_epoch` -> `start_slot` * `compute_epoch_at_slot` -> `epoch` A follow-up PR will (likely) introduce saturating arithmetic for the time units - this is merely code moves, renames and fixing of small bugs.
2022-01-11 10:01:54 +00:00
(committees_since_epoch_start + committee_index.asUInt64) mod
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
ATTESTATION_SUBNET_COUNT)
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
update some consensus spec URLs to v1.4.0-beta.1 (#5287)
2023-08-12 10:38:06 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.1/specs/phase0/validator.md#broadcast-attestation
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
func getAttestationTopic*(forkDigest: ForkDigest,
subnetId: SubnetId): string =
refactor topic (un)subscribing/validating to collate each (#1510) * refactor topic (un)subscribing/validating to collate each * fix comment * tweak comment
2020-08-17 12:07:29 +00:00
## For subscribing and unsubscribing to/from a subnet.
odds and ends (#3015) * `allSyncCommittees` => `allSyncSubcommittees` * simplify `_snappy` topic generation (avoid pointless string copies) * simplify gossip id generator (avoid pointless string copies) * avoid redundant syncnet ENR updates * simplify topic validation (allow only validated topics)
2021-10-21 13:09:19 +00:00
eth2Prefix(forkDigest) & "beacon_attestation_" & $(subnetId) & "/ssz_snappy"
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/p2p-interface.md#topics-and-messages
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
func getSyncCommitteeTopic*(forkDigest: ForkDigest,
Better REST/RPC error messages (#3046) * Better REST/RPC error messages * homogenise block logging (root first) * homegenise message verification pipeline (verify in `gossip_verification`, act in `eth2_processor`) * use `subcommitteeIdx` consistently * log each sent contribution * fix block_sim * fix block topic * don't recalc root on gossip block validation * move position loop into sync pool
2021-11-05 15:39:47 +00:00
subcommitteeIdx: SyncSubcommitteeIndex): string =
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
## For subscribing and unsubscribing to/from a subnet.
Harden CommitteeIndex, SubnetId, SyncSubcommitteeIndex (#3259) * Harden CommitteeIndex, SubnetId, SyncSubcommitteeIndex Harden the use of `CommitteeIndex` et al to prevent future issues by using a distinct type, then validating before use in several cases - datatypes in spec are kept simple though so that invalid data still can be read. * fix invalid epoch used in REST `/eth/v1/beacon/states/{state_id}/committees` committee length (could return invalid data) * normalize some variable names * normalize committee index loops * fix `RestAttesterDuty` to use `uint64` for `validator_committee_index` * validate `CommitteeIndex` on ingress in REST API * update rest rules with stricter parsing * better REST serializers * save lots of memory by not using `zip` ...at least a few bytes!
2022-01-08 23:28:49 +00:00
eth2Prefix(forkDigest) & "sync_committee_" & $subcommitteeIdx & "/ssz_snappy"
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/p2p-interface.md#topics-and-messages
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
func getSyncCommitteeContributionAndProofTopic*(forkDigest: ForkDigest): string =
## For subscribing and unsubscribing to/from a subnet.
odds and ends (#3015) * `allSyncCommittees` => `allSyncSubcommittees` * simplify `_snappy` topic generation (avoid pointless string copies) * simplify gossip id generator (avoid pointless string copies) * avoid redundant syncnet ENR updates * simplify topic validation (allow only validated topics)
2021-10-21 13:09:19 +00:00
eth2Prefix(forkDigest) & "sync_committee_contribution_and_proof/ssz_snappy"
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
consensus spec v1.4.0-beta.5 URL updates (#5655)
2023-12-06 22:16:55 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/deneb/p2p-interface.md#blob_sidecar_subnet_id
Add blob sidecar gossip validation (#4785) * Add blob gossip validation * lint * Add test for getBlobSidecarTopic * Fix closure variable capture issue * Update beacon_chain/nimbus_beacon_node.nim Co-authored-by: tersec <tersec@users.noreply.github.com> --------- Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-04-04 15:12:34 +00:00
func getBlobSidecarTopic*(forkDigest: ForkDigest,
introduce `BlobId` type for tracking blob subnets (#5560) Instead of mixing up `SubnetId` (attestation subnet type) for blobs, introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
subnet_id: BlobId): string =
Add blob handling to message router (#5106) * Add blob handling to message router * address review feedback * Fix typos
2023-06-28 17:55:31 +00:00
eth2Prefix(forkDigest) & "blob_sidecar_" & $subnet_id & "/ssz_snappy"
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/deneb/validator.md#sidecar
introduce `BlobId` type for tracking blob subnets (#5560) Instead of mixing up `SubnetId` (attestation subnet type) for blobs, introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
func compute_subnet_for_blob_sidecar*(blob_index: BlobIndex): BlobId =
BlobId(blob_index mod BLOB_SIDECAR_SUBNET_COUNT)
Add blob sidecar gossip validation (#4785) * Add blob gossip validation * lint * Add test for getBlobSidecarTopic * Fix closure variable capture issue * Update beacon_chain/nimbus_beacon_node.nim Co-authored-by: tersec <tersec@users.noreply.github.com> --------- Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-04-04 15:12:34 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/light-client/p2p-interface.md#light_client_finality_update
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
func getLightClientFinalityUpdateTopic*(forkDigest: ForkDigest): string =
## For broadcasting or obtaining the latest `LightClientFinalityUpdate`.
update light client protocol version (#3550) * Use final `v1` version for light client protocols * Unhide LC data collection options * Default enable LC data serving * rm unneeded import * Connect to EL on startup * Add docs for LC based EL sync
2022-07-29 08:45:39 +00:00
eth2Prefix(forkDigest) & "light_client_finality_update/ssz_snappy"
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/light-client/p2p-interface.md#light_client_optimistic_update
update to latest light client libp2p protocol (#3623) Incorporates the latest changes to the light client sync protocol based on Devconnect AMS feedback. Note that this breaks compatibility with the previous prototype, due to changes to data structures and endpoints. See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 12:02:54 +00:00
func getLightClientOptimisticUpdateTopic*(forkDigest: ForkDigest): string =
## For broadcasting or obtaining the latest `LightClientOptimisticUpdate`.
update light client protocol version (#3550) * Use final `v1` version for light client protocols * Unhide LC data collection options * Default enable LC data serving * rm unneeded import * Connect to EL on startup * Add docs for LC based EL sync
2022-07-29 08:45:39 +00:00
eth2Prefix(forkDigest) & "light_client_optimistic_update/ssz_snappy"
libp2p light client gossip validation (#3486) When `--serve-light-client-data` is specified, provides stability on the `optimistic_light_client_update` GossipSub topic.
2022-03-14 13:05:38 +00:00
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
func getENRForkID*(cfg: RuntimeConfig,
epoch: Epoch,
Implement the new Altair req/resp protocols (#2676) * Implement the new Altair req/resp protocols Also fixes the altair message-id computation by providing the correct forkdigest prefix in `isAltairTopic`. Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 09:09:47 +00:00
genesis_validators_root: Eth2Digest): ENRForkID =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
let
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
current_fork_version = cfg.forkVersionAtEpoch(epoch)
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
next_fork_version = if cfg.nextForkEpochAtEpoch(epoch) == FAR_FUTURE_EPOCH:
Fix the Prater ENR fork id (#2798)
2021-08-20 13:04:29 +00:00
current_fork_version
else:
merge hardfork database support (#2911) * merge hardfork database support * working block_sim * recreate state transition changes
2021-09-30 01:07:24 +00:00
cfg.forkVersionAtEpoch(cfg.nextForkEpochAtEpoch(epoch))
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
fork_digest = compute_fork_digest(current_fork_version,
genesis_validators_root)
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
ENRForkID(
fork_digest: fork_digest,
Fix the Prater ENR fork id (#2798)
2021-08-20 13:04:29 +00:00
next_fork_version: next_fork_version,
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
next_fork_epoch: cfg.nextForkEpochAtEpoch(epoch))
Improve discovery during fork (#2913)
2021-09-29 11:06:16 +00:00
func getDiscoveryForkID*(cfg: RuntimeConfig,
support connecting to peers without bellatrix (#4011) * support connecting to peers without bellatrix Make discovery fork ID aware of scheduled Bellatrix fork to enable connections to peers that don't have Bellatrix scheduled yet. Without this, has peering issues with peers on older SW version. * expand tests with compatibility checks * more exhaustive compatibility checks
2022-08-21 17:36:46 +00:00
epoch: Epoch,
genesis_validators_root: Eth2Digest): ENRForkID =
# Until 1 epoch from fork, return pre-fork value.
if cfg.nextForkEpochAtEpoch(epoch) - epoch <= 1:
Improve discovery during fork (#2913)
2021-09-29 11:06:16 +00:00
getENRForkID(cfg, epoch, genesis_validators_root)
else:
let
current_fork_version = cfg.forkVersionAtEpoch(epoch)
fork_digest = compute_fork_digest(current_fork_version,
genesis_validators_root)
ENRForkID(
fork_digest: fork_digest,
next_fork_version: current_fork_version,
next_fork_epoch: FAR_FUTURE_EPOCH)
Merge fork gossip support (#3213) * Merge fork gossip support * index directly by BeaconStateFork and remove debugging log statement
2021-12-21 14:24:23 +00:00
automated consensus spec URL updating to v1.4.0-beta.1 (#5280)
2023-08-09 03:58:47 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.1/specs/altair/p2p-interface.md#transitioning-the-gossip
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560) * `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` * revert unrelated change * revert unrelated changes * update test summaries
2023-01-28 19:53:41 +00:00
type GossipState* = set[ConsensusFork]
Merge fork gossip support (#3213) * Merge fork gossip support * index directly by BeaconStateFork and remove debugging log statement
2021-12-21 14:24:23 +00:00
func getTargetGossipState*(
eip4844 fork and epoch transition tests; some eip4844 gossip (#4393)
2022-12-06 16:43:11 +00:00
epoch, ALTAIR_FORK_EPOCH, BELLATRIX_FORK_EPOCH, CAPELLA_FORK_EPOCH,
Capella metadata for Sepolia (#4615) Other changes: Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as this matches the latest spec and it's already used in the official Sepolia config.
2023-02-15 14:44:09 +00:00
DENEB_FORK_EPOCH: Epoch, isBehind: bool): GossipState =
Merge fork gossip support (#3213) * Merge fork gossip support * index directly by BeaconStateFork and remove debugging log statement
2021-12-21 14:24:23 +00:00
if isBehind:
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
return {}
doAssert BELLATRIX_FORK_EPOCH >= ALTAIR_FORK_EPOCH
doAssert CAPELLA_FORK_EPOCH >= BELLATRIX_FORK_EPOCH
Capella metadata for Sepolia (#4615) Other changes: Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as this matches the latest spec and it's already used in the official Sepolia config.
2023-02-15 14:44:09 +00:00
doAssert DENEB_FORK_EPOCH >= CAPELLA_FORK_EPOCH
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
# https://github.com/ethereum/consensus-specs/issues/2902
# Don't care whether ALTAIR_FORK_EPOCH == BELLATRIX_FORK_EPOCH or
# BELLATRIX_FORK_EPOCH == CAPELLA_FORK_EPOCH works, because those
# theoretically possible networks are ill-defined regardless, and
# consequently prohibited by checkForkConsistency(). Therefore, a
# transitional epoch always exists, for every fork.
var targetForks: GossipState
template maybeIncludeFork(
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560) * `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` * revert unrelated change * revert unrelated changes * update test summaries
2023-01-28 19:53:41 +00:00
targetFork: ConsensusFork, targetForkEpoch: Epoch,
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
successiveForkEpoch: Epoch) =
# Subscribe one epoch ahead
if epoch + 1 >= targetForkEpoch and epoch < successiveForkEpoch:
targetForks.incl targetFork
maybeIncludeFork(
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560) * `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` * revert unrelated change * revert unrelated changes * update test summaries
2023-01-28 19:53:41 +00:00
ConsensusFork.Phase0, GENESIS_EPOCH, ALTAIR_FORK_EPOCH)
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
maybeIncludeFork(
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560) * `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` * revert unrelated change * revert unrelated changes * update test summaries
2023-01-28 19:53:41 +00:00
ConsensusFork.Altair, ALTAIR_FORK_EPOCH, BELLATRIX_FORK_EPOCH)
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
maybeIncludeFork(
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560) * `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` * revert unrelated change * revert unrelated changes * update test summaries
2023-01-28 19:53:41 +00:00
ConsensusFork.Bellatrix, BELLATRIX_FORK_EPOCH, CAPELLA_FORK_EPOCH)
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
maybeIncludeFork(
Capella metadata for Sepolia (#4615) Other changes: Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as this matches the latest spec and it's already used in the official Sepolia config.
2023-02-15 14:44:09 +00:00
ConsensusFork.Capella, CAPELLA_FORK_EPOCH, DENEB_FORK_EPOCH)
eip4844 fork and epoch transition tests; some eip4844 gossip (#4393)
2022-12-06 16:43:11 +00:00
maybeIncludeFork(
rename ConsensusFork.EIP4844 to ConsensusFork.Deneb (#4692)
2023-03-04 13:35:39 +00:00
ConsensusFork.Deneb, DENEB_FORK_EPOCH, FAR_FUTURE_EPOCH)
capella gossip support (#4386)
2022-12-04 07:42:03 +00:00
doAssert len(targetForks) <= 2
targetForks
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
spec: Option -> Opt (#4488)
2023-01-11 12:29:21 +00:00
func nearSyncCommitteePeriod*(epoch: Epoch): Opt[uint64] =
consensus spec v1.4.0-beta.5 URL updates (#5655)
2023-12-06 22:16:55 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/validator.md#sync-committee-subnet-stability
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
if epoch.is_sync_committee_period():
spec: Option -> Opt (#4488)
2023-01-11 12:29:21 +00:00
return Opt.some 0'u64
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
let epochsBefore =
EPOCHS_PER_SYNC_COMMITTEE_PERIOD - epoch.since_sync_committee_period_start()
if epoch.is_sync_committee_period() or epochsBefore <= SYNC_COMMITTEE_SUBNET_COUNT:
spec: Option -> Opt (#4488)
2023-01-11 12:29:21 +00:00
return Opt.some epochsBefore
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
spec: Option -> Opt (#4488)
2023-01-11 12:29:21 +00:00
Opt.none(uint64)
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
func getSyncSubnets*(
nodeHasPubkey: proc(pubkey: ValidatorPubKey):
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352) In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 09:29:07 +00:00
bool {.noSideEffect, raises: [].},
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
syncCommittee: SyncCommittee): SyncnetBits =
var res: SyncnetBits
for i, pubkey in syncCommittee.pubkeys:
enable `styleCheck:usages` (#3573) Some upstream repos still need fixes, but this gets us close enough that style hints can be enabled by default. In general, "canonical" spellings are preferred even if they violate nep-1 - this applies in particular to spec-related stuff like `genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
if not nodeHasPubkey(pubkey):
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
continue
automated consensus spec URL updating to v1.4.0-beta.5 (#5647)
2023-12-05 02:34:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.4.0-beta.5/specs/altair/validator.md#broadcast-sync-committee-message
dynamic sync committee subscriptions (#3308) * dynamic sync committee subscriptions * fast-path trivial case rather than rely on RNG with probability 1 outcome Co-authored-by: zah <zahary@gmail.com> * use func instead of template; avoid calling async function unnecessarily * avoid unnecessary sync committee topic computation; use correct epoch lookahead; enforce exception/effect tracking * don't over-optimistically update ENR syncnets; non-looping version of nearSyncCommitteePeriod * allow separately setting --allow-all-{sub,att,sync}nets * remove unnecessary async Co-authored-by: zah <zahary@gmail.com>
2022-01-24 20:40:59 +00:00
# The first quarter of the pubkeys map to subnet 0, the second quarter to
# subnet 1, the third quarter to subnet 2 and the final quarter to subnet
# 3.
res.setBit(i div (SYNC_COMMITTEE_SIZE div SYNC_COMMITTEE_SUBNET_COUNT))
res
subscribe to Deneb blob subnets (#5342)
2023-08-23 19:31:41 +00:00
iterator blobSidecarTopics*(forkDigest: ForkDigest): string =
introduce `BlobId` type for tracking blob subnets (#5560) Instead of mixing up `SubnetId` (attestation subnet type) for blobs, introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
for subnet_id in BlobId:
yield getBlobSidecarTopic(forkDigest, subnet_id)