status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

update gossip_processing and attestation docs (#2860) 

The README file explaining gossip_processing, and the attestation_flow
docs were no longer accurate, as attestations and aggregates no longer
go through a queue (pending batching). This patch updates the docs
accordingly. It also improves some grammar and fixes some typos.
This commit is contained in:
Etan Kissling 2021-09-27 15:11:10 +02:00 committed by GitHub
parent 5bfff43785
commit cc30bf63b4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 60 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
beacon_chain/gossip_processing/README.md
View File

@ -1,34 +1,34 @@
# Gossip Processing
This folders hold a collection of modules to:
This folder holds a collection of modules to:
- validate raw gossip data before
- rebroadcasting them (potentially aggregated)
- sending it to one of the consensus object pool
- rebroadcasting it (potentially aggregated)
- sending it to one of the consensus object pools
## Validation
Gossip Validation is different from consensus verification in particular for blocks.
Gossip validation is different from consensus verification in particular for blocks.
- Blocks: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_block
- Attestations (aggregate): https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- Attestations (single): https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#attestation-subnets
- Exits: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#voluntary_exit
- Attestations (aggregated): https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- Attestations (unaggregated): https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#attestation-subnets
- Voluntary exits: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#voluntary_exit
- Proposer slashings: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#proposer_slashing
- Attester slashing: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#attester_slashing
There are 2 consumers of validated consensus objects:
There are multiple consumers of validated consensus objects:
- a `ValidationResult.Accept` output triggers rebroadcasting in libp2p
- method `validate(PubSub, message)` in libp2p/protocols/pubsub/pubsub.nim in the
- We jump into method `validate(PubSub, Message)` in libp2p/protocols/pubsub/pubsub.nim
- which was called by `rpcHandler(GossipSub, PubSubPeer, RPCMsg)`
- a `xyzValidator` message enqueues the validated object in one of the processing queue in eth2_processor
- `blocksQueue: AsyncQueue[BlockEntry]`, (shared with request_manager and sync_manager)
- `attestationsQueue: AsyncQueue[AttestationEntry]`
- `aggregatesQueue: AsyncQueue[AggregateEntry]`
Those queues are then regularly processed to be made available to the consensus object pools.
- a `blockValidator` message enqueues the validated object to the processing queue in block_processor
- `blocksQueue: AsyncQueue[BlockEntry]` (shared with request_manager and sync_manager)
- This queue is then regularly processed to be made available to the consensus object pools.
- a `xyzValidator` message adds the validated object to a pool in eth2_processor
- Attestations (unaggregated and aggregated) get collected into batches.
- Once a threshold is exceeded or after a timeout, they get validated together using BatchCrypto.
## Security concerns
As the first line of defense in Nimbus, modules must be able to handle burst of data that may come:
As the first line of defense in Nimbus, modules must be able to handle bursts of data that may come:
- from malicious nodes trying to DOS us
- from long periods of non-finality, creating lots of forks, attestations, forks
- from long periods of non-finality, creating lots of forks, attestations

102
docs/attestation_flow.md
View File

@ -4,73 +4,63 @@ This is a WIP document to explain the attestation flows.
## Validation & Verification flow
Important distinction:
- We distinguish attestation `validation` which is defined in the P2P specs:
- single: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_attestation_subnet_id
- aggregated: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
A validated attestation or aggregate can be forwarded on gossipsub.
- and we distinguish `verification` which is defined in consensus specs:
https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/beacon-chain.md#attestations
An attestation needs to be verified to enter fork choice, or be included in a block.
It is important to distinguish attestation `validation` from attestation `verification`.
- Attestation `validation` is defined in the P2P specs. Validated attestations can be forwarded on GossipSub.
- Aggregated: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- Unaggregated: https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_attestation_subnet_id
- Attestation `verification` is defined in the consensus specs. Verified attestations can affect fork choice and may be included in a block.
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/beacon-chain.md#attestations
To be clarified: from the specs it seems like gossip attestation validation is a superset of consensus attestation verification.
From the specs it seems like gossip attestation `validation` is a superset of consensus attestation `verification`.
### Inputs
Attestations can be received from the following sources:
- Gossipsub
- Aggregate: `/eth2/{$forkDigest}/beacon_aggregate_and_proof/ssz` stored `topicAggregateAndProofs` field of the beacon node
- Unaggregated `/eth2/{$forkDigest}/beacon_attestation_{subnetIndex}/ssz`
- Included in blocks received
- GossipSub
- Aggregated: `/eth2/{$forkDigest}/beacon_aggregate_and_proof/ssz`
- Unaggregated: `/eth2/{$forkDigest}/beacon_attestation_{subnetIndex}/ssz`
- within received blocks
- the NBC database (within a block)
- a local validator vote
- Devtools: test suite, ncli, fuzzing
- dev tools: test suite, ncli, fuzzing
The related base types are
- Attestation
- IndexedAttestation
- `Attestation`
- `IndexedAttestation`
The base types are defined in the Eth2 specs.
On top, Nimbus builds new types to represent the level of trust and validation we have with regards to each BeaconBlock.
Those types allow the Nim compiler to help us ensure proper usage at compile-time and zero runtime cost.
The base types are defined in the Eth2 specs. On top of these, Nimbus adds further types to represent the level of trust and validation / verification of attestations with regards to each BeaconBlock. Those additional types allow the Nim compiler to help ensuring proper usage at compile time without introducing runtime cost.
#### TrustedAttestation & TrustedIndexedAttestation
An attestation or indexed_attestation that was verified as per the consensus spec or that was retrieved from the database or any source of trusted blocks is considered trusted. In practice we assume that its signature was already verified.
An `Attestation` or `IndexedAttestation` that was successfully verified as per the consensus spec, or that was retrieved from a trusted source of blocks, e.g., the database, is considered trusted. The signature is assumed to have been already verified.
_TODO Note: it seems like P2P validation is a superset of consensus verification in terms of check and that we might use TrustedAttestation earlier in the attestation flow._
_TODO Note: As gossip attestation validation seems to be a superset of consensus attestation verification we might use `TrustedAttestation` earlier in the attestation flow._
## Attestation processing architecture
How the various modules interact with block is described in a diagram:
This diagram visualizes how the various modules interact with a block:
![./attestation_flow.png](./attestation_flow.png)
Note: The Eth2Processor as 2 queues for attestations (before and after P2P validation) and 2 queues for aggregates. The diagram highlights that with separated `AsyncQueue[AttestationEntry]` and `AsyncQueue[AggregateEntry]`
_TODO Note: Image is outdated, attestation queues were removed from the Eth2Processor._
### Gossip flow in
### Inbound gossip flow
Attestatios are listened to via the gossipsub topics
- Aggregate: `/eth2/{$forkDigest}/beacon_aggregate_and_proof/ssz` stored `topicAggregateAndProofs` field of the beacon node
- Unaggregated `/eth2/{$forkDigest}/beacon_attestation_{subnetIndex}/ssz`
These GossipSub topics are used to listen for attestations:
- Aggregated: `/eth2/{$forkDigest}/beacon_aggregate_and_proof/ssz`
- Unaggregated: `/eth2/{$forkDigest}/beacon_attestation_{subnetIndex}/ssz`
They are then
- validated by `validateAttestation()` or `validateAggregate()` in either `attestationValidator()` or `aggregateValidator()`
according to spec
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#attestation-subnets
- It seems like P2P validation is a superset of consensus verification as in `process_attestation()`:
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/beacon-chain.md#attestations
- Enqueued in
- `attestationsQueue: AsyncQueue[AttestationEntry]`
- `aggregatesQueue: AsyncQueue[AggregateEntry]`
- dropped in case of error
The attestations are then validated by `validateAttestation()` or `validateAggregate()` in either `attestationValidator()` or `aggregateValidator()` according to the P2P specs.
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- https://github.com/ethereum/consensus-specs/blob/v1.0.1/specs/phase0/p2p-interface.md#attestation-subnets
### Gossip flow out
Finally, valid attestations are added to the local `attestationPool`.
Attestations are dropped in case of an error.
- After validation in `attestationValidator()` or `aggregateValidator()` in the Eth2Processor
- Important: P2P validation is different from verification at the consensus level.
- We jump into libp2p/protocols/pubsub/pubsub.nim in the method `validate(PubSub, message)`
### Outbound gossip flow
After validation in `attestationValidator()` or `aggregateValidator()` in eth2_processor
- We jump into method `validate(PubSub, Message)` in libp2p/protocols/pubsub/pubsub.nim
- which was called by `rpcHandler(GossipSub, PubSubPeer, RPCMsg)`
### Eth2 RPC in
@ -80,32 +70,26 @@ There is no RPC for attestations but attestations might be included in synced bl
### Sync vs Steady State
During sync the only attestation we receive are within synced blocks.
Afterwards attestations come from GossipSub
During sync the only attestations we receive are included in synced blocks.
Afterwards attestations come from GossipSub.
#### Bottlenecks during sync
During sync, attestations are not a bottleneck, they are a small part of the large block processing.
##### Backpressure
The `attestationsQueue` to store P2P validated attestations has a max size of `TARGET_COMMITTEE_SIZE * MAX_COMMITTEES_PER_SLOT` (128 * 64 = 8192)
The `aggregatesQueue` has a max size of `TARGET_AGGREGATORS_PER_COMMITTEE * MAX_COMMITTEES_PER_SLOT` (16 * 64 = 1024)
Verifying attestations only consumes a small share of block processing time. Attestation processing is not a bottleneck during sync.
##### Latency & Throughput sensitiveness
We distinguish an aggregated attestation:
- aggregation is done according to eth2-spec rules, usually implies diffusion of aggregate signature and the aggregate public key is computed on-the-fly.
- and a batched validation (batching done client side), implies aggregation of signatures and public keys are done on-the-fly
We distinguish aggregation of attestations from batched validation:
- aggregation of attestations is defined by the Eth2 specs. When multiple validators sign the same attestation, their individual signatures may be combined into an aggregate signature to save storage space and to reduce validation time. The aggregate signature is exchanged over the P2P network and included in blocks, but the aggregate public key is computed on-the-fly based on the individual validator public keys according to a participation bitfield.
- batched validation is an opportunistic client side optimization. Multiple attestations of the same underlying data are collected and processed as a batch once a threshold of them has been received or after a time limit is exceeded. As part of this batched validation, both the aggregate signature as well as the aggregate public key are computed on-the-fly.
During sync attestations are included in blocks and so do not require gossip validation,
they are also aggregated per validators and can be batched with other signatures within a block.
Attestations are included in blocks during sync and do not require gossip validation.
They are also aggregated per validator and can be batched with other signatures within a block.
After sync, attestations need to be rebroadcasted fast to:
After sync, attestations need to be rebroadcasted quickly to:
- maintain the quality of the GossipSub mesh
- not be booted by peers
Attestations need to be validated or aggregated fast to avoid delaying other networking operations, in particular they are bottleneck by cryptography.
Attestations need to be validated or aggregated quickly to avoid delaying other networking operations. Their computation is bottlenecked by cryptographic operations.
The number of attestation to process grows with the number of peers. An aggregated attestation is as cheap to process as a non-aggregated one. Batching is worth it even with only 2 attestations to batch.
The number of attestations to process grows with the number of peers. An aggregated attestation is as cheap to process as an unaggregated one. Batching is worth it even with only 2 attestations to batch.