diff --git a/configs/mainnet/sharding.yaml b/configs/mainnet/sharding.yaml index d44c7f550..0e59a674b 100644 --- a/configs/mainnet/sharding.yaml +++ b/configs/mainnet/sharding.yaml @@ -18,6 +18,8 @@ MAX_SHARDS: 1024 INITIAL_ACTIVE_SHARDS: 64 # 2**3 (= 8) GASPRICE_ADJUSTMENT_COEFFICIENT: 8 +# 2**4 (= 16) +MAX_SHARD_PROPOSER_SLASHINGS: 16 # Shard block configs # --------------------------------------------------------------- @@ -41,5 +43,5 @@ SHARD_COMMITTEE_PERIOD: 256 # Signature domains # --------------------------------------------------------------- -DOMAIN_SHARD_PROPOSAL: 0x80000000 +DOMAIN_SHARD_PROPOSER: 0x80000000 DOMAIN_SHARD_COMMITTEE: 0x81000000 diff --git a/configs/minimal/sharding.yaml b/configs/minimal/sharding.yaml index 07b40181b..ca1cc1d6b 100644 --- a/configs/minimal/sharding.yaml +++ b/configs/minimal/sharding.yaml @@ -18,6 +18,8 @@ MAX_SHARDS: 8 INITIAL_ACTIVE_SHARDS: 2 # 2**3 (= 8) GASPRICE_ADJUSTMENT_COEFFICIENT: 8 +# [customized] reduced for testing +MAX_SHARD_PROPOSER_SLASHINGS: 4 # Shard block configs # --------------------------------------------------------------- @@ -41,6 +43,5 @@ SHARD_COMMITTEE_PERIOD: 256 # Signature domains # --------------------------------------------------------------- -DOMAIN_SHARD_PROPOSAL: 0x80000000 +DOMAIN_SHARD_PROPOSER: 0x80000000 DOMAIN_SHARD_COMMITTEE: 0x81000000 -DOMAIN_LIGHT_CLIENT: 0x82000000 diff --git a/specs/sharding/beacon-chain.md b/specs/sharding/beacon-chain.md index 6e1930786..e02229838 100644 --- a/specs/sharding/beacon-chain.md +++ b/specs/sharding/beacon-chain.md @@ -24,9 +24,13 @@ - [`BeaconState`](#beaconstate) - [New containers](#new-containers) - [`DataCommitment`](#datacommitment) - - [`ShardHeader`](#shardheader) - - [`SignedShardHeader`](#signedshardheader) + - [`ShardBlobBodySummary`](#shardblobbodysummary) + - [`ShardBlobHeader`](#shardblobheader) + - [`SignedShardBlobHeader`](#signedshardblobheader) - [`PendingShardHeader`](#pendingshardheader) + - [`ShardBlobReference`](#shardblobreference) + - [`SignedShardBlobReference`](#signedshardblobreference) + - [`ShardProposerSlashing`](#shardproposerslashing) - [Helper functions](#helper-functions) - [Misc](#misc-1) - [`next_power_of_two`](#next_power_of_two) @@ -48,6 +52,7 @@ - [Updated `process_attestation`](#updated-process_attestation) - [`update_pending_votes`](#update_pending_votes) - [`process_shard_header`](#process_shard_header) + - [Shard Proposer slashings](#shard-proposer-slashings) - [Epoch transition](#epoch-transition) - [Pending headers](#pending-headers) - [Shard epoch increment](#shard-epoch-increment) @@ -94,6 +99,7 @@ The following values are (non-configurable) constants used throughout the specif | `INITIAL_ACTIVE_SHARDS` | `uint64(2**6)` (= 64) | Initial shard count | | `GASPRICE_ADJUSTMENT_COEFFICIENT` | `uint64(2**3)` (= 8) | Gasprice may decrease/increase by at most exp(1 / this value) *per epoch* | | `MAX_SHARD_HEADERS_PER_SHARD` | `4` | | +| `MAX_SHARD_PROPOSER_SLASHINGS` | `2**4` (= 16) | Maximum amount of shard proposer slashing operations per block | ### Shard block configs @@ -127,7 +133,7 @@ The following values are (non-configurable) constants used throughout the specif | Name | Value | | - | - | -| `DOMAIN_SHARD_HEADER` | `DomainType('0x80000000')` | +| `DOMAIN_SHARD_PROPOSER` | `DomainType('0x80000000')` | | `DOMAIN_SHARD_COMMITTEE` | `DomainType('0x81000000')` | ## Updated containers @@ -153,7 +159,8 @@ class AttestationData(Container): ```python class BeaconBlockBody(merge.BeaconBlockBody): # [extends The Merge block body] - shard_headers: List[SignedShardHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD] + shard_proposer_slashings: List[ShardProposerSlashing, MAX_SHARD_PROPOSER_SLASHINGS] + shard_headers: List[SignedShardBlobHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD] ``` ### `BeaconState` @@ -186,26 +193,37 @@ class DataCommitment(Container): length: uint64 ``` -### `ShardHeader` +### `ShardBlobBodySummary` ```python -class ShardHeader(Container): - # Slot and shard that this header is intended for - slot: Slot - shard: Shard +class ShardBlobBodySummary(Container): # The actual data commitment commitment: DataCommitment # Proof that the degree < commitment.length degree_proof: BLSCommitment + # Hash-tree-root as summary of the data field + data_root: Root + # Latest block root of the Beacon Chain, before shard_blob.slot + beacon_block_root: Root ``` -TODO: add shard-proposer-index to shard headers, similar to optimization done with beacon-blocks. - -### `SignedShardHeader` +### `ShardBlobHeader` ```python -class SignedShardHeader(Container): - message: ShardHeader +class ShardBlobHeader(Container): + # Slot and shard that this header is intended for + slot: Slot + shard: Shard + body_summary: ShardBlobBodySummary + # Proposer of the shard-blob + proposer_index: ValidatorIndex +``` + +### `SignedShardBlobHeader` + +```python +class SignedShardBlobHeader(Container): + message: ShardBlobHeader signature: BLSSignature ``` @@ -226,6 +244,35 @@ class PendingShardHeader(Container): confirmed: boolean ``` +### `ShardBlobReference` + +```python +class ShardBlobReference(Container): + # Slot and shard that this reference is intended for + slot: Slot + shard: Shard + # Hash-tree-root of commitment data + body_root: Root + # Proposer of the shard-blob + proposer_index: ValidatorIndex +``` + +### `SignedShardBlobReference` + +```python +class SignedShardBlobReference(Container): + message: ShardBlobReference + signature: BLSSignature +``` + +### `ShardProposerSlashing` + +```python +class ShardProposerSlashing(Container): + signed_reference_1: SignedShardBlobReference + signed_reference_2: SignedShardBlobReference +``` + ## Helper functions ### Misc @@ -435,6 +482,8 @@ def process_operations(state: BeaconState, body: BeaconBlockBody) -> None: for_ops(body.proposer_slashings, process_proposer_slashing) for_ops(body.attester_slashings, process_attester_slashing) + # New shard proposer slashing processing + for_ops(body.shard_proposer_slashings, process_shard_proposer_slashing) # Limit is dynamic based on active shard count assert len(body.shard_headers) <= MAX_SHARD_HEADERS_PER_SHARD * get_active_shard_count(state, get_current_epoch(state)) for_ops(body.shard_headers, process_shard_header) @@ -499,30 +548,40 @@ def update_pending_votes(state: BeaconState, attestation: Attestation) -> None: ```python def process_shard_header(state: BeaconState, - signed_header: SignedShardHeader) -> None: + signed_header: SignedShardBlobHeader) -> None: header = signed_header.message - header_root = hash_tree_root(header) - assert compute_epoch_at_slot(header.slot) in [get_previous_epoch(state), get_current_epoch(state)] - + # Verify the header is not 0, and not from the future. + assert Slot(0) < header.slot <= state.slot + header_epoch = compute_epoch_at_slot(header.slot) + # Verify that the header is within the processing time window + assert header_epoch in [get_previous_epoch(state), get_current_epoch(state)] + # Verify that the shard is active + assert header.shard < get_active_shard_count(state, header_epoch) + # Verify that the block root matches, + # to ensure the header will only be included in this specific Beacon Chain sub-tree. + assert header.beacon_block_root == get_block_root_at_slot(state, header.slot - 1) + # Verify proposer + assert header.proposer_index == get_shard_proposer_index(state, header.slot, header.shard) # Verify signature - signer_index = get_shard_proposer_index(state, header.slot, header.shard) signing_root = compute_signing_root(header, get_domain(state, DOMAIN_SHARD_HEADER)) - assert bls.Verify(state.validators[signer_index].pubkey, signing_root, signed_header.signature) + assert bls.Verify(state.validators[header.proposer_index].pubkey, signing_root, signed_header.signature) # Verify the length by verifying the degree. - if header.commitment.length == 0: - assert header.degree_proof == G1_SETUP[0] + body_summary = header.body_summary + if body_summary.commitment.length == 0: + assert body_summary.degree_proof == G1_SETUP[0] assert ( - bls.Pairing(header.degree_proof, G2_SETUP[0]) - == bls.Pairing(header.commitment.point, G2_SETUP[-header.commitment.length]) + bls.Pairing(body_summary.degree_proof, G2_SETUP[0]) + == bls.Pairing(body_summary.commitment.point, G2_SETUP[-body_summary.commitment.length]) ) # Get the correct pending header list - if compute_epoch_at_slot(header.slot) == get_current_epoch(state): + if header_epoch == get_current_epoch(state): pending_headers = state.current_epoch_pending_shard_headers else: pending_headers = state.previous_epoch_pending_shard_headers + header_root = hash_tree_root(header) # Check that this header is not yet in the pending list assert header_root not in [pending_header.root for pending_header in pending_headers] @@ -532,7 +591,7 @@ def process_shard_header(state: BeaconState, pending_headers.append(PendingShardHeader( slot=header.slot, shard=header.shard, - commitment=header.commitment, + commitment=body_summary.commitment, root=header_root, votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length), confirmed=False, @@ -544,6 +603,33 @@ the length proof is the commitment to the polynomial `B(X) * X**(MAX_DEGREE + 1 where `MAX_DEGREE` is the maximum power of `s` available in the setup, which is `MAX_DEGREE = len(G2_SETUP) - 1`. The goal is to ensure that a proof can only be constructed if `deg(B) < l` (there are not hidden higher-order terms in the polynomial, which would thwart reconstruction). +##### Shard Proposer slashings + +```python +def process_shard_proposer_slashing(state: BeaconState, proposer_slashing: ShardProposerSlashing) -> None: + reference_1 = proposer_slashing.signed_reference_1.message + reference_2 = proposer_slashing.signed_reference_2.message + + # Verify header slots match + assert reference_1.slot == reference_2.slot + # Verify header shards match + assert reference_1.shard == reference_2.shard + # Verify header proposer indices match + assert reference_1.proposer_index == reference_2.proposer_index + # Verify the headers are different (i.e. different body) + assert reference_1 != reference_2 + # Verify the proposer is slashable + proposer = state.validators[reference_1.proposer_index] + assert is_slashable_validator(proposer, get_current_epoch(state)) + # Verify signatures + for signed_header in (proposer_slashing.signed_reference_1, proposer_slashing.signed_reference_2): + domain = get_domain(state, DOMAIN_SHARD_PROPOSER, compute_epoch_at_slot(signed_header.message.slot)) + signing_root = compute_signing_root(signed_header.message, domain) + assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature) + + slash_validator(state, reference_1.proposer_index) +``` + ### Epoch transition This epoch transition overrides the Merge epoch transition: diff --git a/specs/sharding/p2p-interface.md b/specs/sharding/p2p-interface.md index 7e8c40dcf..42984dfe3 100644 --- a/specs/sharding/p2p-interface.md +++ b/specs/sharding/p2p-interface.md @@ -10,12 +10,14 @@ - [Introduction](#introduction) - [New containers](#new-containers) + - [ShardBlobBody](#shardblobbody) - [ShardBlob](#shardblob) - [SignedShardBlob](#signedshardblob) - [Gossip domain](#gossip-domain) - [Topics and messages](#topics-and-messages) - [Shard blobs: `shard_blob_{shard}`](#shard-blobs-shard_blob_shard) - [Shard header: `shard_header`](#shard-header-shard_header) + - [Shard proposer slashing: `shard_proposer_slashing`](#shard-proposer-slashing-shard_proposer_slashing) @@ -29,30 +31,41 @@ The adjustments and additions for Shards are outlined in this document. ## New containers -### ShardBlob +### ShardBlobBody -Network-only. +```python +class ShardBlobBody(Container): + # The actual data commitment + commitment: DataCommitment + # Proof that the degree < commitment.length + degree_proof: BLSCommitment + # The actual data. Should match the commitment and degree proof. + data: List[BLSPoint, POINTS_PER_SAMPLE * MAX_SAMPLES_PER_BLOCK] + # Latest block root of the Beacon Chain, before shard_blob.slot + beacon_block_root: Root +``` + +The user MUST always verify the commitments in the `body` are valid for the `data` in the `body`. + +### ShardBlob ```python class ShardBlob(Container): # Slot and shard that this blob is intended for slot: Slot shard: Shard - # The actual data. Represented in header as data commitment and degree proof - data: List[BLSPoint, POINTS_PER_SAMPLE * MAX_SAMPLES_PER_BLOCK] + body: ShardBlobBody + # Proposer of the shard-blob + proposer_index: ValidatorIndex ``` -Note that the hash-tree-root of the `ShardBlob` does not match the `ShardHeader`, -since the blob deals with full data, whereas the header includes the KZG commitment and degree proof instead. +This is the expanded form of the `ShardBlobHeader` type. ### SignedShardBlob -Network-only. - ```python class SignedShardBlob(Container): message: ShardBlob - # The signature, the message is the commitment on the blob signature: BLSSignature ``` @@ -66,6 +79,7 @@ Following the same scheme as the [Phase0 gossip topics](../phase0/p2p-interface. |----------------------------------|---------------------------| | `shard_blob_{shard}` | `SignedShardBlob` | | `shard_header` | `SignedShardHeader` | +| `shard_proposer_slashing` | `ShardProposerSlashing` | The [DAS network specification](./das-p2p.md) defines additional topics. @@ -73,22 +87,49 @@ The [DAS network specification](./das-p2p.md) defines additional topics. Shard block data, in the form of a `SignedShardBlob` is published to the `shard_blob_{shard}` subnets. -The following validations MUST pass before forwarding the `signed_blob` (with inner `blob`) on the horizontal subnet or creating samples for it. +The following validations MUST pass before forwarding the `signed_blob` (with inner `message` as `blob`) on the horizontal subnet or creating samples for it. - _[REJECT]_ `blob.shard` MUST match the topic `{shard}` parameter. (And thus within valid shard index range) - _[IGNORE]_ The `blob` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `blob.slot <= current_slot` (a client MAY queue future blobs for processing at the appropriate slot). -- _[IGNORE]_ The blob is the first blob with valid signature received for the proposer for the `(slot, shard)` combination. +- _[IGNORE]_ The `blob` is new enough to be still be processed -- + i.e. validate that `compute_epoch_at_slot(blob.slot) >= get_previous_epoch(state)` +- _[IGNORE]_ The blob is the first blob with valid signature received for the `(blob.proposer_index, blob.slot, blob.shard)` combination. - _[REJECT]_ As already limited by the SSZ list-limit, it is important the blob is well-formatted and not too large. -- _[REJECT]_ The `blob.data` MUST NOT contain any point `p >= MODULUS`. Although it is a `uint256`, not the full 256 bit range is valid. -- _[REJECT]_ The proposer signature, `signed_blob.signature`, is valid with respect to the `proposer_index` pubkey, signed over the SSZ output of `commit_to_data(blob.data)`. -- _[REJECT]_ The blob is proposed by the expected `proposer_index` for the blob's slot. +- _[REJECT]_ The `blob.body.data` MUST NOT contain any point `p >= MODULUS`. Although it is a `uint256`, not the full 256 bit range is valid. +- _[REJECT]_ The proposer signature, `signed_blob.signature`, is valid with respect to the `proposer_index` pubkey. +- _[REJECT]_ The blob is proposed by the expected `proposer_index` for the blob's slot + in the context of the current shuffling (defined by `blob.body.beacon_block_root`/`slot`). + If the `proposer_index` cannot immediately be verified against the expected shuffling, + the block MAY be queued for later processing while proposers for the blob's branch are calculated -- + in such a case _do not_ `REJECT`, instead `IGNORE` this message. -TODO: make double blob proposals slashable? #### Shard header: `shard_header` -Shard header data, in the form of a `SignedShardHeader` is published to the global `shard_header` subnet. +Shard header data, in the form of a `SignedShardBlobHeader` is published to the global `shard_header` subnet. -TODO: validation conditions. +The following validations MUST pass before forwarding the `signed_shard_header` (with inner `message` as `header`) on the network. +- _[IGNORE]_ The `header` is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- + i.e. validate that `header.slot <= current_slot` + (a client MAY queue future headers for processing at the appropriate slot). +- _[IGNORE]_ The `header` is new enough to be still be processed -- + i.e. validate that `compute_epoch_at_slot(header.slot) >= get_previous_epoch(state)` +- _[IGNORE]_ The header is the first header with valid signature received for the `(header.proposer_index, header.slot, header.shard)` combination. +- _[REJECT]_ The proposer signature, `signed_shard_header.signature`, is valid with respect to the `proposer_index` pubkey. +- _[REJECT]_ The header is proposed by the expected `proposer_index` for the block's slot + in the context of the current shuffling (defined by `header.body_summary.beacon_block_root`/`slot`). + If the `proposer_index` cannot immediately be verified against the expected shuffling, + the block MAY be queued for later processing while proposers for the block's branch are calculated -- + in such a case _do not_ `REJECT`, instead `IGNORE` this message. + +#### Shard proposer slashing: `shard_proposer_slashing` + +Shard proposer slashings, in the form of `ShardProposerSlashing`, are published to the global `shard_proposer_slashing` topic. + +The following validations MUST pass before forwarding the `shard_proposer_slashing` on to the network. +- _[IGNORE]_ The shard proposer slashing is the first valid shard proposer slashing received + for the proposer with index `proposer_slashing.signed_header_1.message.proposer_index`. + The `slot` and `shard` are ignored, there are no per-shard slashings. +- _[REJECT]_ All of the conditions within `process_shard_proposer_slashing` pass validation.