status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/spec/network.nim

108 lines
4.4 KiB
Nim
Raw Normal View History

Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# beacon_chain
only initially subscribe to relevant attestation subnets (#2231)
2021-01-14 08:43:21 +00:00
# Copyright (c) 2018-2021 Status Research & Development GmbH
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# Licensed and distributed under either of
replace links to Apache and MIT licenses with HTTPS versions (#592)
2019-11-25 15:30:02 +00:00
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
# at your option. This file may not be copied, modified, or distributed except according to those terms.
spec: regulate exceptions (#913) * spec: regulate exceptions * a few more simple raises
2020-04-22 05:53:02 +00:00
{.push raises: [Defect].}
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
import
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
"."/[helpers, forks],
remove false OnBlockAdded dependency on phase0 HashedBeaconState (#2661) * remove false OnBlockAdded dependency on phase.HashedBeaconState * introduce altair data types into block_clearance; update some alpha.6 spec refs to alpha.7; add get_active_validator_indices_len ForkedHashedBeaconState wrapper * switch many modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 and remove rm'd G2_POINT_AT_INFINITY * switch more modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 * remove unnecessary phase0-only wrapper of get_attesting_indices(); allow signatures_batch to process either fork; remove O(n^2) nested loop in process_inactivity_updates(); add altair support to getAttestationsforTestBlock() * add Altair versions of asSigVerified(), asTrusted(), and makeBeaconBlock() * fix spec URL to be Altair for Altair makeBeaconBlock()
2021-06-21 08:35:24 +00:00
"."/datatypes/base
split attestation beacon topic per honest validator spec
2020-02-21 12:16:58 +00:00
rework spec imports (#2779) The spec imports are a mess to work with, so this branch cleans them up a bit to ensure that we avoid generic sandwitches and that importing stuff generally becomes easier. * reexport crypto/digest/presets because these are part of the public symbol set of the rest of the spec types * don't export `merge` types from `base` - this causes circular deps * fix circular deps in `ssz/spec_types` - this is the first step in disentangling ssz from spec * be explicit about phase0 vs altair - longer term, `altair` will become the "natural" type set, then merge and so on, so no point in giving `phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
export base
Add gossipsub messages monitoring tool. (#458) * Add gossipsub messages monitoring tool. * Add support of bootstrap nodes file. * Move topic constants to spec/network.nim. * Add ability to monitor custom topics.
2019-10-01 13:52:28 +00:00
const
eth2.0-specs -> consensus-specs repo rename (#2801)
2021-08-20 23:37:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#topics-and-messages
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
topicBeaconBlocksSuffix* = "beacon_block/ssz"
topicVoluntaryExitsSuffix* = "voluntary_exit/ssz"
topicProposerSlashingsSuffix* = "proposer_slashing/ssz"
topicAttesterSlashingsSuffix* = "attester_slashing/ssz"
topicAggregateAndProofsSuffix* = "beacon_aggregate_and_proof/ssz"
split attestation beacon topic per honest validator spec
2020-02-21 12:16:58 +00:00
eth2.0-specs -> consensus-specs repo rename (#2801)
2021-08-20 23:37:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#eth2-network-interaction-domains
[skip ci] Use GOSSIP_MAX_SIZE when snappy decoding in the inspector as well; Bumps
2020-08-19 11:33:52 +00:00
MAX_CHUNK_SIZE* = 1 * 1024 * 1024 # bytes
GOSSIP_MAX_SIZE* = 1 * 1024 * 1024 # bytes
TTFB_TIMEOUT* = 5.seconds
RESP_TIMEOUT* = 10.seconds
Initial implementation of a JSON-RPC service
2020-03-16 22:28:54 +00:00
defaultEth2TcpPort* = 9000
rpc: setLogLevel (#1991) * rpc: setLogLevel also change default rpc port to match makefile and manual * set topic-level formats too and make feature less secret
2020-11-12 10:46:02 +00:00
# This is not part of the spec yet! Keep in sync with BASE_RPC_PORT
defaultEth2RpcPort* = 9190
Initial implementation of a JSON-RPC service
2020-03-16 22:28:54 +00:00
Add TODO comments for missing implementations. Change default REST port to use 5052 (Lighthouse). Add missing checks for maximum amount of validator ids.
2021-03-29 10:59:39 +00:00
# This is not part of the spec! But its port which uses Lighthouse
DefaultEth2RestPort* = 5052
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
enrAttestationSubnetsField* = "attnets"
enrSyncSubnetsField* = "syncnets"
enrForkIdField* = "eth2"
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
template eth2Prefix(forkDigest: ForkDigest): string =
"/eth2/" & $forkDigest & "/"
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getBeaconBlocksTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicBeaconBlocksSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
Fix the inspector build
2020-04-15 03:11:45 +00:00
func getVoluntaryExitsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicVoluntaryExitsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getProposerSlashingsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicProposerSlashingsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getAttesterSlashingsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicAttesterSlashingsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
func getAggregateAndProofsTopic*(forkDigest: ForkDigest): string =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2Prefix(forkDigest) & topicAggregateAndProofsSuffix
Implement most of the v0.11.1 networking changes
2020-04-15 02:41:22 +00:00
eth2.0-specs -> consensus-specs repo rename (#2801)
2021-08-20 23:37:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/validator.md#broadcast-attestation
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
func compute_subnet_for_attestation*(
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346)
2020-07-22 08:04:21 +00:00
committees_per_slot: uint64, slot: Slot, committee_index: CommitteeIndex):
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
SubnetId =
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
# Compute the correct subnet for an attestation for Phase 0.
# Note, this mimics expected Phase 1 behavior where attestations will be
# mapped to their shard subnet.
let
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346)
2020-07-22 08:04:21 +00:00
slots_since_epoch_start = slot mod SLOTS_PER_EPOCH
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
committees_since_epoch_start =
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346)
2020-07-22 08:04:21 +00:00
committees_per_slot * slots_since_epoch_start
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Perform attestation check before broadcast (#2550) Currently, we have a bit of a convoluted flow where when sending attestations, we start broadcasting them over gossip then pass them to the attestation validation to include them in the local attestation pool - it should be the other way around: we should be checking attestations _before_ gossipping them - this serves as an additional safety net to ensure that we don't publish junk - this becomes more important when publishing attestations from the API. Also, the REST API was performing its own validation meaning attestations coming from REST would be validated twice - finally, the JSON RPC wasn't pre-validating and would happily broadcast invalid attestations. * Unified attestation production pipeline with the same flow for gossip, locally and API-produced attestations: all are now validated and entered into the pool, then broadcast/republished * Refactor subnet handling with specific SubnetId alias, streamlining where subnets are computed, avoiding the need to pass around the number of active validators * Move some of the subnet handling code to eth2_network * Use BitArray throughout for subnet handling
2021-05-10 07:13:36 +00:00
SubnetId(
(committees_since_epoch_start + committee_index.uint64) mod
ATTESTATION_SUBNET_COUNT)
remove v0.11.3 support; add block_sim to CI (#1253) * remove v0.11.3 support; add block_sim to CI * rm stray PERSISTENT_COMMITTEE_PERIOD * remove TopicFilter.InteropAttestations * bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
eth2.0-specs -> consensus-specs repo rename (#2801)
2021-08-20 23:37:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/validator.md#broadcast-attestation
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
func getAttestationTopic*(forkDigest: ForkDigest,
subnetId: SubnetId): string =
refactor topic (un)subscribing/validating to collate each (#1510) * refactor topic (un)subscribing/validating to collate each * fix comment * tweak comment
2020-08-17 12:07:29 +00:00
## For subscribing and unsubscribing to/from a subnet.
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
eth2Prefix(forkDigest) & "beacon_attestation_" & $(subnetId) & "/ssz"
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
eth2.0-specs -> consensus-specs repo rename (#2801)
2021-08-20 23:37:45 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.0-alpha.8/specs/altair/p2p-interface.md#topics-and-messages
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
func getSyncCommitteeTopic*(forkDigest: ForkDigest,
committeeIdx: SyncCommitteeIndex): string =
## For subscribing and unsubscribing to/from a subnet.
eth2Prefix(forkDigest) & "sync_committee_" & $(committeeIdx.asUInt8) & "/ssz"
mass update from beta.3 spec refs to beta.4 spec refs (#2862)
2021-09-10 18:56:03 +00:00
# https://github.com/ethereum/consensus-specs/blob/v1.1.0-beta.4/specs/altair/p2p-interface.md#topics-and-messages
sync committee topic names; borrow -> template/distinctbase (#2781)
2021-08-15 14:50:31 +00:00
func getSyncCommitteeContributionAndProofTopic*(forkDigest: ForkDigest): string =
## For subscribing and unsubscribing to/from a subnet.
eth2Prefix(forkDigest) & "sync_committee_contribution_and_proof" & "/ssz"
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
func getENRForkID*(cfg: RuntimeConfig,
epoch: Epoch,
Implement the new Altair req/resp protocols (#2676) * Implement the new Altair req/resp protocols Also fixes the altair message-id computation by providing the correct forkdigest prefix in `isAltairTopic`. Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 09:09:47 +00:00
genesis_validators_root: Eth2Digest): ENRForkID =
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
let
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
current_fork_version = cfg.forkVersionAtEpoch(epoch)
Fix the Prater ENR fork id (#2798)
2021-08-20 13:04:29 +00:00
next_fork_version = if cfg.ALTAIR_FORK_EPOCH == FAR_FUTURE_EPOCH:
current_fork_version
else:
cfg.ALTAIR_FORK_VERSION
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
fork_digest = compute_fork_digest(current_fork_version,
genesis_validators_root)
move ENRForkID into `spec` (#2538) * move ENRForkID into `spec` also get rid of strformat in topic formation and fix some case discrepancies * also move `Eth2Metadata`
2021-05-04 15:28:48 +00:00
ENRForkID(
fork_digest: fork_digest,
Fix the Prater ENR fork id (#2798)
2021-08-20 13:04:29 +00:00
next_fork_version: next_fork_version,
altair enr switching (#2770) * altair enr switching * altair transitions only happen on epoch boundaries * determine ENR period by wall time, not chain head
2021-08-10 06:19:13 +00:00
next_fork_epoch: cfg.nextForkEpochAtEpoch(epoch))