diff --git a/specs/sharding/beacon-chain.md b/specs/sharding/beacon-chain.md index 4bf5f7a03..6378389a7 100644 --- a/specs/sharding/beacon-chain.md +++ b/specs/sharding/beacon-chain.md @@ -17,7 +17,7 @@ - [Shard Work Status](#shard-work-status) - [Preset](#preset) - [Misc](#misc-1) - - [Shard block samples](#shard-block-samples) + - [Shard blob samples](#shard-blob-samples) - [Precomputed size verification points](#precomputed-size-verification-points) - [Gwei values](#gwei-values) - [Configuration](#configuration) diff --git a/specs/sharding/p2p-interface.md b/specs/sharding/p2p-interface.md index e9b561312..4eb3f6f2e 100644 --- a/specs/sharding/p2p-interface.md +++ b/specs/sharding/p2p-interface.md @@ -13,10 +13,11 @@ - [Misc](#misc) - [Gossip domain](#gossip-domain) - [Topics and messages](#topics-and-messages) - - [Shard block subnets](#shard-block-subnets) - - [`shard_block_{subnet_id}`](#shard_block_subnet_id) + - [Shard blob subnets](#shard-blob-subnets) + - [`shard_blob_{subnet_id}`](#shard_blob_subnet_id) - [Global topics](#global-topics) - - [`shard_block_header`](#shard_block_header) + - [`shard_blob_header`](#shard_blob_header) + - [`shard_blob_tx`](#shard_blob_tx) - [`shard_proposer_slashing`](#shard_proposer_slashing) @@ -34,7 +35,7 @@ The adjustments and additions for Shards are outlined in this document. | Name | Value | Description | | ---- | ----- | ----------- | -| `SHARD_BLOCK_SUBNET_COUNT` | `64` | The number of `shard_block_{subnet_id}` subnets used in the gossipsub protocol. | +| `SHARD_BLOB_SUBNET_COUNT` | `64` | The number of `shard_blob_{subnet_id}` subnets used in the gossipsub protocol. | ## Gossip domain @@ -42,24 +43,25 @@ The adjustments and additions for Shards are outlined in this document. Following the same scheme as the [Phase0 gossip topics](../phase0/p2p-interface.md#topics-and-messages), names and payload types are: -| Name | Message Type | -|----------------------------------|---------------------------| -| `shard_block_{subnet_id}` | `SignedShardBlock` | -| `shard_block_header` | `SignedShardBlockHeader` | -| `shard_proposer_slashing` | `ShardProposerSlashing` | +| Name | Message Type | +|---------------------------------|--------------------------| +| `shard_blob_{subnet_id}` | `SignedShardBlob` | +| `shard_blob_header` | `SignedShardBlobHeader` | +| `shard_blob_tx` | `SignedShardBlobHeader` | +| `shard_proposer_slashing` | `ShardProposerSlashing` | The [DAS network specification](./das-p2p.md) defines additional topics. -#### Shard block subnets +#### Shard blob subnets -Shard block subnets are used by builders to make their blobs available after selection by shard proposers. +Shard blob subnets are used by builders to make their blobs available after selection by shard proposers. -##### `shard_block_{subnet_id}` +##### `shard_blob_{subnet_id}` -Shard block data, in the form of a `SignedShardBlock` is published to the `shard_block_{subnet_id}` subnets. +Shard blob data, in the form of a `SignedShardBlob` is published to the `shard_blob_{subnet_id}` subnets. ```python -def compute_subnet_for_shard_block(state: BeaconState, slot: Slot, shard: Shard) -> uint64: +def compute_subnet_for_shard_blob(state: BeaconState, slot: Slot, shard: Shard) -> uint64: """ Compute the correct subnet for a shard blob publication. Note, this mimics compute_subnet_for_attestation(). @@ -69,75 +71,97 @@ def compute_subnet_for_shard_block(state: BeaconState, slot: Slot, shard: Shard) slots_since_epoch_start = Slot(slot % SLOTS_PER_EPOCH) committees_since_epoch_start = committees_per_slot * slots_since_epoch_start - return uint64((committees_since_epoch_start + committee_index) % SHARD_BLOCK_SUBNET_COUNT) + return uint64((committees_since_epoch_start + committee_index) % SHARD_BLOB_SUBNET_COUNT) ``` -The following validations MUST pass before forwarding the `signed_block` on the horizontal subnet or creating samples for it. +The following validations MUST pass before forwarding the `signed_blob`, +on the horizontal subnet or creating samples for it. Alias `blob = signed_blob.message`. -We define some aliases to the nested contents of `signed_block`: -```python -block: ShardBlock = signed_block.message -signed_blob: SignedShardBlob = block.signed_blob -blob: ShardBlob = signed_blob.message -``` - -- _[IGNORE]_ The `block` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- +- _[IGNORE]_ The `blob` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `blob.slot <= current_slot` (a client MAY queue future blobs for processing at the appropriate slot). -- _[IGNORE]_ The `blob` is new enough to be still be processed -- +- _[IGNORE]_ The `blob` is new enough to still be processed -- i.e. validate that `compute_epoch_at_slot(blob.slot) >= get_previous_epoch(state)` -- _[REJECT]_ The shard should have a committee at slot -- +- _[REJECT]_ The shard blob is for an active shard -- + i.e. `blob.shard < get_active_shard_count(state, compute_epoch_at_slot(blob.slot))` +- _[REJECT]_ The `blob.shard` MUST have a committee at the `blob.slot` -- i.e. validate that `compute_committee_index_from_shard(state, blob.slot, blob.shard)` doesn't raise an error - _[REJECT]_ The shard blob is for the correct subnet -- - i.e. `compute_subnet_for_shard_block(state, blob.slot, blob.shard) == subnet_id` -- _[IGNORE]_ The block is the first block with valid signature received for the `(block.proposer_index, blob.slot, blob.shard)` combination. + i.e. `compute_subnet_for_shard_blob(state, blob.slot, blob.shard) == subnet_id` +- _[IGNORE]_ The blob is the first blob with valid signature received for the `(blob.proposer_index, blob.slot, blob.shard)` combination. - _[REJECT]_ The blob is not too large, the data MUST NOT be larger than the SSZ list-limit, and a client MAY be more strict. - _[REJECT]_ The `blob.body.data` MUST NOT contain any point `p >= MODULUS`. Although it is a `uint256`, not the full 256 bit range is valid. -- _[REJECT]_ The block proposer signature, `signed_block.signature`, is valid with respect to the `proposer_index` pubkey. - _[REJECT]_ The blob builder exists and has sufficient balance to back the fee payment. -- _[REJECT]_ The blob builder signature, `signed_blob.signature`, is valid with respect to the `builder_index` pubkey. -- _[REJECT]_ The block is proposed by the expected `proposer_index` for the block's slot +- _[REJECT]_ The blob signature is valid for the aggregate of proposer and builder, `signed_blob.signature`, + i.e. `bls.FastAggregateVerify([builder_pubkey, proposer_pubkey], blob_signing_root, signed_blob.signature)`. +- _[REJECT]_ The blob is proposed by the expected `proposer_index` for the blob's `slot` and `shard`, in the context of the current shuffling (defined by `blob.body.beacon_block_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, - the block MAY be queued for later processing while proposers for the block's branch are calculated -- + the blob MAY be queued for later processing while proposers for the blob's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. #### Global topics -There are two additional global topics for Sharding. +There are three additional global topics for Sharding. -One is used to propagate shard block headers (`shard_block_header`) to all nodes on the network. -Another one is used to propagate shard proposer slashings (`shard_proposer_slashing`). +- `shard_blob_header`: co-signed headers, to be included on-chain, and signaling builders to publish full data. +- `shard_blob_tx`: builder-signed headers, also known as "data transaction". +- `shard_proposer_slashing`: slashings of duplicate shard proposals -##### `shard_block_header` +##### `shard_blob_header` -Shard header data, in the form of a `SignedShardBlockHeader` is published to the global `shard_block_header` subnet. -Shard block headers select shard blob bids by builders, and should be timely to ensure builders can publish the full shard block timely. +Shard header data, in the form of a `SignedShardBlobHeader` is published to the global `shard_blob_header` subnet. +Shard blob headers select shard blob bids by builders, +and should be timely to ensure builders can publish the full shard blob before subsequent attestations. -The following validations MUST pass before forwarding the `signed_block_header` (with inner `message` as `header`) on the network. +The following validations MUST pass before forwarding the `signed_blob_header` on the network. Alias `header = signed_blob_header.message` -We define some aliases to the nested contents of `signed_block_header`: -```python -block_header: ShardBlockHeader = signed_block_header.message -signed_blob_header: SignedShardBlobHeader = header.signed_blob_header -blob_header: ShardBlobHeader = signed_blob_header.message -``` - -- _[IGNORE]_ The header is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- - i.e. validate that `blob_header.slot <= current_slot` +- _[IGNORE]_ The `header` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- + i.e. validate that `header.slot <= current_slot` (a client MAY queue future headers for processing at the appropriate slot). -- _[IGNORE]_ The header is new enough to be still be processed -- - i.e. validate that `compute_epoch_at_slot(blob_header.slot) >= get_previous_epoch(state)` -- _[IGNORE]_ The header is the first header with valid signature received for the `(block_header.proposer_index, blob_header.slot, blob_header.shard)` combination. -- _[REJECT]_ The `shard` MUST have a committee at the `slot` -- - i.e. validate that `compute_committee_index_from_shard(state, blob_header.slot, blob_header.shard)` doesn't raise an error -- _[REJECT]_ The proposer signature, `signed_shard_block_header.signature`, is valid with respect to the `block_header.proposer_index` pubkey. -- _[REJECT]_ The header is proposed by the expected `proposer_index` for the block's slot +- _[IGNORE]_ The header is new enough to still be processed -- + i.e. validate that `compute_epoch_at_slot(header.slot) >= get_previous_epoch(state)` +- _[REJECT]_ The shard header is for an active shard -- + i.e. `header.shard < get_active_shard_count(state, compute_epoch_at_slot(header.slot))` +- _[REJECT]_ The `header.shard` MUST have a committee at the `header.slot` -- + i.e. validate that `compute_committee_index_from_shard(state, header.slot, header.shard)` doesn't raise an error +- _[IGNORE]_ The header is the first header with valid signature received for the `(header.proposer_index, header.slot, header.shard)` combination. +- _[REJECT]_ The blob builder exists and has sufficient balance to back the fee payment. +- _[REJECT]_ The header signature is valid for the aggregate of proposer and builder, `signed_blob_header.signature`, + i.e. `bls.FastAggregateVerify([builder_pubkey, proposer_pubkey], blob_signing_root, signed_blob_header.signature)`. +- _[REJECT]_ The header is proposed by the expected `proposer_index` for the blob's `header.slot` and `header.shard` in the context of the current shuffling (defined by `header.body_summary.beacon_block_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, - the block MAY be queued for later processing while proposers for the block's branch are calculated -- + the blob MAY be queued for later processing while proposers for the blob's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. +##### `shard_blob_tx` + +Shard data-transactions, in the form of a `SignedShardBlobHeader` is published to the global `shard_blob_tx` subnet. +These shard blob headers are signed solely by the blob-builder. + +The following validations MUST pass before forwarding the `signed_blob_header` on the network. Alias `header = signed_blob_header.message` + +- _[IGNORE]_ The `header` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- + i.e. validate that `header.slot <= current_slot` + (a client MAY queue future headers for processing at the appropriate slot). +- _[IGNORE]_ The header is new enough to still be processed -- + i.e. validate that `compute_epoch_at_slot(header.slot) >= get_previous_epoch(state)` +- _[REJECT]_ The shard header is for an active shard -- + i.e. `header.shard < get_active_shard_count(state, compute_epoch_at_slot(header.slot))` +- _[REJECT]_ The `header.shard` MUST have a committee at the `header.slot` -- + i.e. validate that `compute_committee_index_from_shard(state, header.slot, header.shard)` doesn't raise an error +- _[IGNORE]_ The header is the first header with valid signature received for the `(header.builder_index, header.slot, header.shard)` combination. +- _[REJECT]_ The blob builder exists and has sufficient balance to back the fee payment. +- _[IGNORE]_ The header fee SHOULD be higher than previously seen headers for `(header.slot, header.shard)`, from any builder. + Propagating nodes MAY increase fee increments in case of spam. +- _[REJECT]_ The header signature is valid for ONLY the builder, `signed_blob_header.signature`, + i.e. `bls.Verify(builder_pubkey, blob_signing_root, signed_blob_header.signature)`. The signature is not an aggregate with the proposer. +- _[REJECT]_ The header is designated for proposal by the expected `proposer_index` for the blob's `header.slot` and `header.shard` + in the context of the current shuffling (defined by `header.body_summary.beacon_block_root`/`slot`). + If the `proposer_index` cannot immediately be verified against the expected shuffling, + the blob MAY be queued for later processing while proposers for the blob's branch are calculated -- + in such a case _do not_ `REJECT`, instead `IGNORE` this message. ##### `shard_proposer_slashing` @@ -145,6 +169,6 @@ Shard proposer slashings, in the form of `ShardProposerSlashing`, are published The following validations MUST pass before forwarding the `shard_proposer_slashing` on to the network. - _[IGNORE]_ The shard proposer slashing is the first valid shard proposer slashing received - for the proposer with index `proposer_slashing.signed_reference_1.message.proposer_index`. - The `slot` and `shard` are ignored, there are no per-shard slashings. + for the proposer with index `proposer_slashing.proposer_index`. + The `proposer_slashing.slot` and `proposer_slashing.shard` are ignored, there are no repeated or per-shard slashings. - _[REJECT]_ All of the conditions within `process_shard_proposer_slashing` pass validation.