From f72a26cac6168fa85b3cc5bd2596c2f95ade073d Mon Sep 17 00:00:00 2001 From: omahs <73983677+omahs@users.noreply.github.com> Date: Thu, 16 Feb 2023 11:02:58 +0100 Subject: [PATCH 01/34] Fix: typos --- sync/optimistic.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sync/optimistic.md b/sync/optimistic.md index 79a5271c2..14eb99fb1 100644 --- a/sync/optimistic.md +++ b/sync/optimistic.md @@ -375,7 +375,7 @@ Given all of this, we can say two things: justify an honest chain. 2. **BNs which are syncing can optimistically import transition blocks.** In this case a justified chain already exists blocks. The poison block would be - quickly reverted and would have no affect on liveness. + quickly reverted and would have no effect on liveness. Astute readers will notice that (2) contains a glaring assumption about network liveness. This is necessary because a node cannot feasibly ascertain that the @@ -408,13 +408,13 @@ Such a scenario requires manual intervention. An alternative to optimistic sync is to run a light client inside/alongside beacon nodes that mitigates the need for optimistic sync by providing -tip-of-chain blocks to the execution engine. However, light clients comes with +tip-of-chain blocks to the execution engine. However, light clients come with their own set of complexities. Relying on light clients may also restrict nodes from syncing from genesis, if they so desire. A notable thing about optimistic sync is that it's *optional*. Should an implementation decide to go the light-client route, then they can just ignore -optimistic sync all together. +optimistic sync altogether. ### What if `TERMINAL_BLOCK_HASH` is used? From b3db3ec83ad16e50f1d97c370a7fa354b44c391a Mon Sep 17 00:00:00 2001 From: omahs <73983677+omahs@users.noreply.github.com> Date: Sat, 25 Mar 2023 12:23:03 +0100 Subject: [PATCH 02/34] Fix: typo --- fork_choice/safe-block.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fork_choice/safe-block.md b/fork_choice/safe-block.md index 490d24538..b76285b3a 100644 --- a/fork_choice/safe-block.md +++ b/fork_choice/safe-block.md @@ -15,7 +15,7 @@ ## Introduction Under honest majority and certain network synchronicity assumptions -there exist a block that is safe from re-orgs. Normally this block is +there exists a block that is safe from re-orgs. Normally this block is pretty close to the head of canonical chain which makes it valuable to expose a safe block to users. From 3115d1140b23dd4c9c23fbd9e2428186cf816bde Mon Sep 17 00:00:00 2001 From: omahs <73983677+omahs@users.noreply.github.com> Date: Sat, 25 Mar 2023 13:14:57 +0100 Subject: [PATCH 03/34] Fix: typos --- specs/phase0/beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index 3794cd6be..b77e017ab 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -269,7 +269,7 @@ Additional preset configurations can be found in the [`configs`](../../configs) - The `INACTIVITY_PENALTY_QUOTIENT` equals `INVERSE_SQRT_E_DROP_TIME**2` where `INVERSE_SQRT_E_DROP_TIME := 2**13` epochs (about 36 days) is the time it takes the inactivity penalty to reduce the balance of non-participating validators to about `1/sqrt(e) ~= 60.6%`. Indeed, the balance retained by offline validators after `n` epochs is about `(1 - 1/INACTIVITY_PENALTY_QUOTIENT)**(n**2/2)`; so after `INVERSE_SQRT_E_DROP_TIME` epochs, it is roughly `(1 - 1/INACTIVITY_PENALTY_QUOTIENT)**(INACTIVITY_PENALTY_QUOTIENT/2) ~= 1/sqrt(e)`. Note this value will be upgraded to `2**24` after Phase 0 mainnet stabilizes to provide a faster recovery in the event of an inactivity leak. -- The `PROPORTIONAL_SLASHING_MULTIPLIER` is set to `1` at initial mainnet launch, resulting in one-third of the minimum accountable safety margin in the event of a finality attack. After Phase 0 mainnet stablizes, this value will be upgraded to `3` to provide the maximal minimum accountable safety margin. +- The `PROPORTIONAL_SLASHING_MULTIPLIER` is set to `1` at initial mainnet launch, resulting in one-third of the minimum accountable safety margin in the event of a finality attack. After Phase 0 mainnet stabilizes, this value will be upgraded to `3` to provide the maximal minimum accountable safety margin. ### Max operations per block @@ -1036,7 +1036,7 @@ def get_total_balance(state: BeaconState, indices: Set[ValidatorIndex]) -> Gwei: """ Return the combined effective balance of the ``indices``. ``EFFECTIVE_BALANCE_INCREMENT`` Gwei minimum to avoid divisions by zero. - Math safe up to ~10B ETH, afterwhich this overflows uint64. + Math safe up to ~10B ETH, after which this overflows uint64. """ return Gwei(max(EFFECTIVE_BALANCE_INCREMENT, sum([state.validators[index].effective_balance for index in indices]))) ``` From f9b359be09df44e12732c90e87d9c281542dc859 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 28 Mar 2023 12:03:14 +0900 Subject: [PATCH 04/34] Reuse indexes with full sweep --- specs/_features/reuse_indexes/beacon-chain.md | 45 +++++++++++++++++++ specs/altair/beacon-chain.md | 22 ++++++--- 2 files changed, 62 insertions(+), 5 deletions(-) create mode 100644 specs/_features/reuse_indexes/beacon-chain.md diff --git a/specs/_features/reuse_indexes/beacon-chain.md b/specs/_features/reuse_indexes/beacon-chain.md new file mode 100644 index 000000000..fcf1de896 --- /dev/null +++ b/specs/_features/reuse_indexes/beacon-chain.md @@ -0,0 +1,45 @@ +# Reuse indexes -- The Beacon Chain + +## Table of contents + + + + + +- [Introduction](#introduction) +- [Preset](#preset) + - [Time parameters](#time-parameters) +- [Beacon chain state transition function](#beacon-chain-state-transition-function) + - [Block processing](#block-processing) + - [Modified `assign_index_to_deposit`](#modified-assign_index_to_deposit) + + + + +## Introduction + +This is the beacon chain specification to assign new deposits to existing validator records that have withdrawn long ago. + +*Note:* This specification is built upon [Capella](../../capella/beacon_chain.md) and is under active development. + +## Preset + +### Time parameters + +| Name | Value | Unit | Duration | +| - | - | - | +| `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~1 year | + +## Beacon chain state transition function + +### Block processing + +#### Modified `assign_index_to_deposit` + +```python +def assign_index_to_deposit(state: BeaconState) -> int: + for index, validator in enumerate(state.validators): + if validator.withdrawable_epoch < get_current_epoch(state) - REUSE_VALIDATOR_INDEX_DELAY: + return index + return len(state.validators) +``` diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index 58dfad608..d5b89a387 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -511,16 +511,28 @@ def apply_deposit(state: BeaconState, signing_root = compute_signing_root(deposit_message, domain) # Initialize validator if the deposit signature is valid if bls.Verify(pubkey, signing_root, signature): - state.validators.append(get_validator_from_deposit(pubkey, withdrawal_credentials, amount)) - state.balances.append(amount) + index = assign_index_to_deposit(state) + update_list(state.validators, index, get_validator_from_deposit(pubkey, withdrawal_credentials, amount)) + update_list(state.balances, index, amount) # [New in Altair] - state.previous_epoch_participation.append(ParticipationFlags(0b0000_0000)) - state.current_epoch_participation.append(ParticipationFlags(0b0000_0000)) - state.inactivity_scores.append(uint64(0)) + update_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) + update_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) + update_list(state.inactivity_scores, index, uint64(0)) else: # Increase balance by deposit amount index = ValidatorIndex(validator_pubkeys.index(pubkey)) increase_balance(state, index, amount) + + +def assign_index_to_deposit(state: BeaconState) -> int: + return len(state.validators) + + +def update_list(list: List, index: int, value: Any) -> None: + if index == len(list): + list.append(value) + else: + list[index] = value ``` #### Sync aggregate processing From 201f113b5096978c2f2afca3f293870b2398f81a Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 28 Mar 2023 15:27:13 +1100 Subject: [PATCH 05/34] Introduce get_epoch_boundary_block --- specs/phase0/fork-choice.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 6e281d5c3..044d85053 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -192,6 +192,17 @@ def get_ancestor(store: Store, root: Root, slot: Slot) -> Root: return root ``` +#### `get_epoch_boundary_block` + +```python +def get_epoch_boundary_block(store: Store, root: Root, epoch: Epoch) -> Root: + """ + Compute the epoch boundary block for epoch ``epoch`` in the chain of block ``root`` + """ + epoch_first_slot = compute_start_slot_at_epoch(epoch) + return get_ancestor(store, root, epoch_first_slot) +``` + #### `get_weight` ```python @@ -278,10 +289,9 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB voting_source.epoch + 2 >= current_epoch ) - finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) correct_finalized = ( store.finalized_checkpoint.epoch == GENESIS_EPOCH - or store.finalized_checkpoint.root == get_ancestor(store, block_root, finalized_slot) + or store.finalized_checkpoint.root == get_epoch_boundary_block(store, block_root, store.finalized_checkpoint.epoch) ) # If expected finalized/justified, add to viable block-tree and signal viability to parent. @@ -442,8 +452,7 @@ def validate_on_attestation(store: Store, attestation: Attestation, is_from_bloc assert store.blocks[attestation.data.beacon_block_root].slot <= attestation.data.slot # LMD vote must be consistent with FFG vote target - target_slot = compute_start_slot_at_epoch(target.epoch) - assert target.root == get_ancestor(store, attestation.data.beacon_block_root, target_slot) + assert target.root == get_epoch_boundary_block(store, attestation.data.beacon_block_root, target.epoch) # Attestations can only affect the fork choice of subsequent slots. # Delay consideration in the fork choice until their slot is in the past. @@ -506,7 +515,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root + assert get_epoch_boundary_block(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root # Check the block is valid and compute the post-state state = pre_state.copy() From ddbd82e1be7fc38bcdcf672865bb036ab80837a7 Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 28 Mar 2023 15:51:34 +1100 Subject: [PATCH 06/34] Add toc --- specs/phase0/fork-choice.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 044d85053..058089a62 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -18,6 +18,7 @@ - [`get_current_slot`](#get_current_slot) - [`compute_slots_since_epoch_start`](#compute_slots_since_epoch_start) - [`get_ancestor`](#get_ancestor) + - [`get_epoch_boundary_block`](#get_epoch_boundary_block) - [`get_weight`](#get_weight) - [`get_voting_source`](#get_voting_source) - [`filter_block_tree`](#filter_block_tree) From ee3e1ac63e6e675e58d22cf055431b70a6f39696 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 28 Mar 2023 15:34:07 +0900 Subject: [PATCH 07/34] Apply review comments --- specs/_features/reuse_indexes/beacon-chain.md | 28 ++++++++++++++++--- specs/altair/beacon-chain.md | 16 +++++------ 2 files changed, 32 insertions(+), 12 deletions(-) diff --git a/specs/_features/reuse_indexes/beacon-chain.md b/specs/_features/reuse_indexes/beacon-chain.md index fcf1de896..8677631e5 100644 --- a/specs/_features/reuse_indexes/beacon-chain.md +++ b/specs/_features/reuse_indexes/beacon-chain.md @@ -9,9 +9,12 @@ - [Introduction](#introduction) - [Preset](#preset) - [Time parameters](#time-parameters) +- [Helpers](#helpers) + - [Predicates](#predicates) + - [`is_reusable_validator`](#is_reusable_validator) - [Beacon chain state transition function](#beacon-chain-state-transition-function) - [Block processing](#block-processing) - - [Modified `assign_index_to_deposit`](#modified-assign_index_to_deposit) + - [Modified `get_index_for_new_validator`](#modified-get_index_for_new_validator) @@ -30,16 +33,33 @@ This is the beacon chain specification to assign new deposits to existing valida | - | - | - | | `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~1 year | +## Helper functions + +### Predicates + +#### `is_reusable_validator` + +```python +def is_reusable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> bool: + """ + Check if ``validator`` index can be re-assigned to a new deposit. + """ + return ( + validator.withdrawable_epoch < epoch - REUSE_VALIDATOR_INDEX_DELAY + and balance == 0 + ) +``` + ## Beacon chain state transition function ### Block processing -#### Modified `assign_index_to_deposit` +#### Modified `get_index_for_new_validator` ```python -def assign_index_to_deposit(state: BeaconState) -> int: +def get_index_for_new_validator(state: BeaconState) -> int: for index, validator in enumerate(state.validators): - if validator.withdrawable_epoch < get_current_epoch(state) - REUSE_VALIDATOR_INDEX_DELAY: + if is_reusable_validator(validator, state.balances[index], get_current_epoch(state)): return index return len(state.validators) ``` diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index d5b89a387..5e24df83e 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -511,24 +511,24 @@ def apply_deposit(state: BeaconState, signing_root = compute_signing_root(deposit_message, domain) # Initialize validator if the deposit signature is valid if bls.Verify(pubkey, signing_root, signature): - index = assign_index_to_deposit(state) - update_list(state.validators, index, get_validator_from_deposit(pubkey, withdrawal_credentials, amount)) - update_list(state.balances, index, amount) + index = get_index_for_new_validator(state) + update_or_append_to_list(state.validators, index, get_validator_from_deposit(pubkey, withdrawal_credentials, amount)) + update_or_append_to_list(state.balances, index, amount) # [New in Altair] - update_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) - update_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) - update_list(state.inactivity_scores, index, uint64(0)) + update_or_append_to_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) + update_or_append_to_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) + update_or_append_to_list(state.inactivity_scores, index, uint64(0)) else: # Increase balance by deposit amount index = ValidatorIndex(validator_pubkeys.index(pubkey)) increase_balance(state, index, amount) -def assign_index_to_deposit(state: BeaconState) -> int: +def get_index_for_new_validator(state: BeaconState) -> int: return len(state.validators) -def update_list(list: List, index: int, value: Any) -> None: +def update_or_append_to_list(list: List, index: int, value: Any) -> None: if index == len(list): list.append(value) else: From c1273bbfa405f7c335d36561c92d169552ef7432 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 28 Mar 2023 15:46:01 +0900 Subject: [PATCH 08/34] Add epoch > REUSE_VALIDATOR_INDEX_DELAY condition --- specs/_features/reuse_indexes/beacon-chain.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specs/_features/reuse_indexes/beacon-chain.md b/specs/_features/reuse_indexes/beacon-chain.md index 8677631e5..3efa0c869 100644 --- a/specs/_features/reuse_indexes/beacon-chain.md +++ b/specs/_features/reuse_indexes/beacon-chain.md @@ -45,8 +45,9 @@ def is_reusable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> Check if ``validator`` index can be re-assigned to a new deposit. """ return ( - validator.withdrawable_epoch < epoch - REUSE_VALIDATOR_INDEX_DELAY - and balance == 0 + epoch > REUSE_VALIDATOR_INDEX_DELAY + and validator.withdrawable_epoch < epoch - REUSE_VALIDATOR_INDEX_DELAY + and balance == 0 ) ``` From 314b040fff2d5a5aa2ab8e7524c76200f3a64d79 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 28 Mar 2023 17:45:52 +0900 Subject: [PATCH 09/34] Reduce line len --- specs/altair/beacon-chain.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index 5e24df83e..9d7c8180e 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -512,7 +512,8 @@ def apply_deposit(state: BeaconState, # Initialize validator if the deposit signature is valid if bls.Verify(pubkey, signing_root, signature): index = get_index_for_new_validator(state) - update_or_append_to_list(state.validators, index, get_validator_from_deposit(pubkey, withdrawal_credentials, amount)) + validator = get_validator_from_deposit(pubkey, withdrawal_credentials, amount) + update_or_append_to_list(state.validators, index, validator) update_or_append_to_list(state.balances, index, amount) # [New in Altair] update_or_append_to_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) From c7029ce19e3f25d2d0e3ea301650561d1ac67d12 Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Wed, 29 Mar 2023 12:40:58 +1100 Subject: [PATCH 10/34] Rename get_epoch_boundary_block to get_ancestor_at_epoch_boundary --- specs/phase0/fork-choice.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 058089a62..2ee6ecb92 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -18,7 +18,7 @@ - [`get_current_slot`](#get_current_slot) - [`compute_slots_since_epoch_start`](#compute_slots_since_epoch_start) - [`get_ancestor`](#get_ancestor) - - [`get_epoch_boundary_block`](#get_epoch_boundary_block) + - [`get_ancestor_at_epoch_boundary`](#get_ancestor_at_epoch_boundary) - [`get_weight`](#get_weight) - [`get_voting_source`](#get_voting_source) - [`filter_block_tree`](#filter_block_tree) @@ -193,10 +193,10 @@ def get_ancestor(store: Store, root: Root, slot: Slot) -> Root: return root ``` -#### `get_epoch_boundary_block` +#### `get_ancestor_at_epoch_boundary` ```python -def get_epoch_boundary_block(store: Store, root: Root, epoch: Epoch) -> Root: +def get_ancestor_at_epoch_boundary(store: Store, root: Root, epoch: Epoch) -> Root: """ Compute the epoch boundary block for epoch ``epoch`` in the chain of block ``root`` """ @@ -292,7 +292,7 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB correct_finalized = ( store.finalized_checkpoint.epoch == GENESIS_EPOCH - or store.finalized_checkpoint.root == get_epoch_boundary_block(store, block_root, store.finalized_checkpoint.epoch) + or store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary(store, block_root, store.finalized_checkpoint.epoch) ) # If expected finalized/justified, add to viable block-tree and signal viability to parent. @@ -453,7 +453,7 @@ def validate_on_attestation(store: Store, attestation: Attestation, is_from_bloc assert store.blocks[attestation.data.beacon_block_root].slot <= attestation.data.slot # LMD vote must be consistent with FFG vote target - assert target.root == get_epoch_boundary_block(store, attestation.data.beacon_block_root, target.epoch) + assert target.root == get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, target.epoch) # Attestations can only affect the fork choice of subsequent slots. # Delay consideration in the fork choice until their slot is in the past. @@ -516,7 +516,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_epoch_boundary_block(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root + assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root # Check the block is valid and compute the post-state state = pre_state.copy() From f696b30608a1b6402d32c3c87ab01be99e2096bd Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 30 Mar 2023 09:11:36 +0900 Subject: [PATCH 11/34] Address PR review --- specs/_features/reuse_indexes/beacon-chain.md | 13 ++++++------- specs/altair/beacon-chain.md | 6 +++--- 2 files changed, 9 insertions(+), 10 deletions(-) diff --git a/specs/_features/reuse_indexes/beacon-chain.md b/specs/_features/reuse_indexes/beacon-chain.md index 3efa0c869..5ec08ed19 100644 --- a/specs/_features/reuse_indexes/beacon-chain.md +++ b/specs/_features/reuse_indexes/beacon-chain.md @@ -1,4 +1,4 @@ -# Reuse indexes -- The Beacon Chain +# Reuse indices -- The Beacon Chain ## Table of contents @@ -31,7 +31,7 @@ This is the beacon chain specification to assign new deposits to existing valida | Name | Value | Unit | Duration | | - | - | - | -| `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~1 year | +| `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~0.8 year | ## Helper functions @@ -45,8 +45,7 @@ def is_reusable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> Check if ``validator`` index can be re-assigned to a new deposit. """ return ( - epoch > REUSE_VALIDATOR_INDEX_DELAY - and validator.withdrawable_epoch < epoch - REUSE_VALIDATOR_INDEX_DELAY + epoch > validator.withdrawable_epoch + REUSE_VALIDATOR_INDEX_DELAY and balance == 0 ) ``` @@ -58,9 +57,9 @@ def is_reusable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> #### Modified `get_index_for_new_validator` ```python -def get_index_for_new_validator(state: BeaconState) -> int: +def get_index_for_new_validator(state: BeaconState) -> ValidatorIndex: for index, validator in enumerate(state.validators): if is_reusable_validator(validator, state.balances[index], get_current_epoch(state)): - return index - return len(state.validators) + return ValidatorIndex(index) + return ValidatorIndex(len(state.validators)) ``` diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index 9d7c8180e..df65d984c 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -525,11 +525,11 @@ def apply_deposit(state: BeaconState, increase_balance(state, index, amount) -def get_index_for_new_validator(state: BeaconState) -> int: - return len(state.validators) +def get_index_for_new_validator(state: BeaconState) -> ValidatorIndex: + return ValidatorIndex(len(state.validators)) -def update_or_append_to_list(list: List, index: int, value: Any) -> None: +def update_or_append_to_list(list: List, index: ValidatorIndex, value: Any) -> None: if index == len(list): list.append(value) else: From 3d9c87b27ccc9e8b96c9749abd7c727452bbb44a Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 30 Mar 2023 09:44:34 +0900 Subject: [PATCH 12/34] Move to misc helpers --- specs/altair/beacon-chain.md | 40 +++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index df65d984c..4c7bb3f9a 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -30,6 +30,8 @@ - [Misc](#misc-1) - [`add_flag`](#add_flag) - [`has_flag`](#has_flag) + - [`get_index_for_new_validator`](#get_index_for_new_validator) + - [`set_or_append_list`](#set_or_append_list) - [Beacon state accessors](#beacon-state-accessors) - [`get_next_sync_committee_indices`](#get_next_sync_committee_indices) - [`get_next_sync_committee`](#get_next_sync_committee) @@ -248,6 +250,23 @@ def has_flag(flags: ParticipationFlags, flag_index: int) -> bool: return flags & flag == flag ``` +#### `get_index_for_new_validator` + +```python +def get_index_for_new_validator(state: BeaconState) -> ValidatorIndex: + return ValidatorIndex(len(state.validators)) +``` + +#### `set_or_append_list` + +```python +def set_or_append_list(list: List[Any], index: ValidatorIndex, value: Any) -> None: + if index == len(list): + list.append(value) + else: + list[index] = value +``` + ### Beacon state accessors #### `get_next_sync_committee_indices` @@ -513,27 +532,16 @@ def apply_deposit(state: BeaconState, if bls.Verify(pubkey, signing_root, signature): index = get_index_for_new_validator(state) validator = get_validator_from_deposit(pubkey, withdrawal_credentials, amount) - update_or_append_to_list(state.validators, index, validator) - update_or_append_to_list(state.balances, index, amount) + set_or_append_list(state.validators, index, validator) + set_or_append_list(state.balances, index, amount) # [New in Altair] - update_or_append_to_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) - update_or_append_to_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) - update_or_append_to_list(state.inactivity_scores, index, uint64(0)) + set_or_append_list(state.previous_epoch_participation, index, ParticipationFlags(0b0000_0000)) + set_or_append_list(state.current_epoch_participation, index, ParticipationFlags(0b0000_0000)) + set_or_append_list(state.inactivity_scores, index, uint64(0)) else: # Increase balance by deposit amount index = ValidatorIndex(validator_pubkeys.index(pubkey)) increase_balance(state, index, amount) - - -def get_index_for_new_validator(state: BeaconState) -> ValidatorIndex: - return ValidatorIndex(len(state.validators)) - - -def update_or_append_to_list(list: List, index: ValidatorIndex, value: Any) -> None: - if index == len(list): - list.append(value) - else: - list[index] = value ``` #### Sync aggregate processing From e255d09840855e16e62f0e531f2ca50fa45e2724 Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Fri, 31 Mar 2023 10:52:52 +1100 Subject: [PATCH 13/34] Apply changes to Bellatrix and Daneb --- specs/bellatrix/fork-choice.md | 2 +- specs/deneb/fork-choice.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/bellatrix/fork-choice.md b/specs/bellatrix/fork-choice.md index ed7d60a93..aac26f3dd 100644 --- a/specs/bellatrix/fork-choice.md +++ b/specs/bellatrix/fork-choice.md @@ -170,7 +170,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root + assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root # Check the block is valid and compute the post-state state = pre_state.copy() diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 61714cf1a..83cdb9972 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -82,7 +82,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root + assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root # [New in Deneb] # Check if blob data is available From 25ea243859329cb7d7131a03051ed78e0abd03c3 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 4 Apr 2023 11:45:08 +0900 Subject: [PATCH 14/34] PR comments --- .../_features/{reuse_indexes => reuse_indices}/beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename specs/_features/{reuse_indexes => reuse_indices}/beacon-chain.md (99%) diff --git a/specs/_features/reuse_indexes/beacon-chain.md b/specs/_features/reuse_indices/beacon-chain.md similarity index 99% rename from specs/_features/reuse_indexes/beacon-chain.md rename to specs/_features/reuse_indices/beacon-chain.md index 5ec08ed19..ecac7fb30 100644 --- a/specs/_features/reuse_indexes/beacon-chain.md +++ b/specs/_features/reuse_indices/beacon-chain.md @@ -30,7 +30,7 @@ This is the beacon chain specification to assign new deposits to existing valida ### Time parameters | Name | Value | Unit | Duration | -| - | - | - | +| - | - | - | - | | `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~0.8 year | ## Helper functions From 622c9b97bac1a586a8b6692fe1998d3e621a8c30 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Tue, 4 Apr 2023 12:00:09 +0900 Subject: [PATCH 15/34] Fix CI --- specs/_features/reuse_indices/beacon-chain.md | 2 +- specs/altair/beacon-chain.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/_features/reuse_indices/beacon-chain.md b/specs/_features/reuse_indices/beacon-chain.md index ecac7fb30..6dd71e36f 100644 --- a/specs/_features/reuse_indices/beacon-chain.md +++ b/specs/_features/reuse_indices/beacon-chain.md @@ -9,7 +9,7 @@ - [Introduction](#introduction) - [Preset](#preset) - [Time parameters](#time-parameters) -- [Helpers](#helpers) +- [Helper functions](#helper-functions) - [Predicates](#predicates) - [`is_reusable_validator`](#is_reusable_validator) - [Beacon chain state transition function](#beacon-chain-state-transition-function) diff --git a/specs/altair/beacon-chain.md b/specs/altair/beacon-chain.md index 4c7bb3f9a..8c3a8877e 100644 --- a/specs/altair/beacon-chain.md +++ b/specs/altair/beacon-chain.md @@ -260,7 +260,7 @@ def get_index_for_new_validator(state: BeaconState) -> ValidatorIndex: #### `set_or_append_list` ```python -def set_or_append_list(list: List[Any], index: ValidatorIndex, value: Any) -> None: +def set_or_append_list(list: List, index: ValidatorIndex, value: Any) -> None: if index == len(list): list.append(value) else: From 912c9b37a13f42de334e676f1de68352706b8f18 Mon Sep 17 00:00:00 2001 From: Suphanat Chunhapanya Date: Fri, 31 Mar 2023 21:06:17 +0700 Subject: [PATCH 16/34] Fix typos --- specs/deneb/p2p-interface.md | 8 ++++---- specs/phase0/validator.md | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 9be028620..0b6381e20 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -16,7 +16,7 @@ The specification of these changes continues in the same format as the network s - [`SignedBlobSidecar`](#signedblobsidecar) - [`BlobIdentifier`](#blobidentifier) - [Helpers](#helpers) - - [`verify_sidecar_signature`](#verify_sidecar_signature) + - [`verify_blob_sidecar_signature`](#verify_blob_sidecar_signature) - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - [Topics and messages](#topics-and-messages) - [Global topics](#global-topics) @@ -77,7 +77,7 @@ class BlobIdentifier(Container): ### Helpers -#### `verify_sidecar_signature` +#### `verify_blob_sidecar_signature` ```python def verify_blob_sidecar_signature(state: BeaconState, signed_blob_sidecar: SignedBlobSidecar) -> bool: @@ -118,7 +118,7 @@ The *type* of the payload of this topic changes to the (modified) `SignedBeaconB This topic is used to propagate signed blob sidecars, one for each sidecar index. The number of indices is defined by `MAX_BLOBS_PER_BLOCK`. -The following validations MUST pass before forwarding the `sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: +The following validations MUST pass before forwarding the `signed_blob_sidecar` on the network, assuming the alias `sidecar = signed_blob_sidecar.message`: - _[REJECT]_ The sidecar is for the correct topic -- i.e. `sidecar.index` matches the topic `{index}`. - _[IGNORE]_ The sidecar is not from a future slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. validate that `sidecar.slot <= current_slot` (a client MAY queue future sidecars for processing at the appropriate slot). @@ -126,7 +126,7 @@ The following validations MUST pass before forwarding the `sidecar` on the netwo - _[IGNORE]_ The sidecar's block's parent (defined by `sidecar.block_parent_root`) has been seen (via both gossip and non-gossip sources) (a client MAY queue sidecars for processing once the parent block is retrieved). - _[REJECT]_ The sidecar's block's parent (defined by `sidecar.block_parent_root`) passes validation. - _[REJECT]_ The sidecar is from a higher slot than the sidecar's block's parent (defined by `sidecar.block_parent_root`). -- _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid as verified by `verify_sidecar_signature`. +- _[REJECT]_ The proposer signature, `signed_blob_sidecar.signature`, is valid as verified by `verify_blob_sidecar_signature`. - _[IGNORE]_ The sidecar is the only sidecar with valid signature received for the tuple `(sidecar.block_root, sidecar.index)`. - _[REJECT]_ The sidecar is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `block_parent_root`/`slot`). If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. diff --git a/specs/phase0/validator.md b/specs/phase0/validator.md index 54b344791..2a4d5b920 100644 --- a/specs/phase0/validator.md +++ b/specs/phase0/validator.md @@ -162,7 +162,7 @@ The `withdrawal_credentials` field must be such that: * `withdrawal_credentials[1:12] == b'\x00' * 11` * `withdrawal_credentials[12:] == eth1_withdrawal_address` -After the merge of the current Ethereum application layer into the Beacon Chain, +After the merge of the current Ethereum execution layer into the Beacon Chain, withdrawals to `eth1_withdrawal_address` will simply be increases to the account's ETH balance that do **NOT** trigger any EVM execution. ### Submit deposit From 68e7766b085b33618880d6529cce577097e333c9 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 4 Apr 2023 22:04:48 +0800 Subject: [PATCH 17/34] Remove gitter link --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9f2528263..d0d6b222d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Ethereum Proof-of-Stake Consensus Specifications -[![Join the chat at https://discord.gg/qGpsxSA](https://img.shields.io/badge/chat-on%20discord-blue.svg)](https://discord.gg/qGpsxSA) [![Join the chat at https://gitter.im/ethereum/sharding](https://badges.gitter.im/ethereum/sharding.svg)](https://gitter.im/ethereum/sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +[![Join the chat at https://discord.gg/qGpsxSA](https://img.shields.io/badge/chat-on%20discord-blue.svg)](https://discord.gg/qGpsxSA) To learn more about proof-of-stake and sharding, see the [PoS documentation](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/), [sharding documentation](https://ethereum.org/en/upgrades/sharding/) and the [research compendium](https://notes.ethereum.org/s/H1PGqDhpm). From 4cac76181827562a010c9c45dba1bb9f80f4f6cd Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 5 Apr 2023 11:38:20 +0800 Subject: [PATCH 18/34] make linter happy --- specs/bellatrix/fork-choice.md | 6 +++++- specs/deneb/fork-choice.md | 6 +++++- specs/phase0/fork-choice.md | 12 ++++++++++-- 3 files changed, 20 insertions(+), 4 deletions(-) diff --git a/specs/bellatrix/fork-choice.md b/specs/bellatrix/fork-choice.md index aac26f3dd..d22436c9d 100644 --- a/specs/bellatrix/fork-choice.md +++ b/specs/bellatrix/fork-choice.md @@ -170,7 +170,11 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root + assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + store, + block.parent_root, + store.finalized_checkpoint.epoch, + ) # Check the block is valid and compute the post-state state = pre_state.copy() diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 83cdb9972..e76e159c4 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -82,7 +82,11 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root + assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + store, + block.parent_root, + store.finalized_checkpoint.epoch, + ) # [New in Deneb] # Check if blob data is available diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 2ee6ecb92..478dd2142 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -292,7 +292,11 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB correct_finalized = ( store.finalized_checkpoint.epoch == GENESIS_EPOCH - or store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary(store, block_root, store.finalized_checkpoint.epoch) + or store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + store, + block_root, + store.finalized_checkpoint.epoch, + ) ) # If expected finalized/justified, add to viable block-tree and signal viability to parent. @@ -516,7 +520,11 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root + assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + store, + block.parent_root, + store.finalized_checkpoint.epoch, + ) # Check the block is valid and compute the post-state state = pre_state.copy() From 8fafc6c695253f6c64bb8e17fdfa2d825ca23ede Mon Sep 17 00:00:00 2001 From: Marius van der Wijden Date: Thu, 6 Apr 2023 11:20:07 +0200 Subject: [PATCH 19/34] deneb: switch blob tx type to 0x03 --- specs/deneb/beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/deneb/beacon-chain.md b/specs/deneb/beacon-chain.md index a0ac783b7..df1da8e2a 100644 --- a/specs/deneb/beacon-chain.md +++ b/specs/deneb/beacon-chain.md @@ -59,7 +59,7 @@ This upgrade adds blobs to the beacon chain as part of Deneb. This is an extensi | Name | Value | | - | - | -| `BLOB_TX_TYPE` | `uint8(0x05)` | +| `BLOB_TX_TYPE` | `uint8(0x03)` | | `VERSIONED_HASH_VERSION_KZG` | `Bytes1('0x01')` | ## Preset From 41386092b761d61792c2ad0d2c07a76c4511a823 Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Sat, 8 Apr 2023 14:00:01 +1000 Subject: [PATCH 20/34] Apply changes to p2p-interface.md --- specs/phase0/p2p-interface.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index f52752931..56c1b8cfb 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -317,7 +317,7 @@ The following validations MUST pass before forwarding the `signed_beacon_block` - _[REJECT]_ The block's parent (defined by `block.parent_root`) passes validation. - _[REJECT]_ The block is from a higher slot than its parent. - _[REJECT]_ The current `finalized_checkpoint` is an ancestor of `block` -- i.e. - `get_ancestor(store, block.parent_root, compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)) + `get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root` - _[REJECT]_ The block is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `parent_root`/`slot`). @@ -356,7 +356,7 @@ The following validations MUST pass before forwarding the `signed_aggregate_and_ (a client MAY queue aggregates for processing once block is retrieved). - _[REJECT]_ The block being voted for (`aggregate.data.beacon_block_root`) passes validation. - _[IGNORE]_ The current `finalized_checkpoint` is an ancestor of the `block` defined by `aggregate.data.beacon_block_root` -- i.e. - `get_ancestor(store, aggregate.data.beacon_block_root, compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)) + `get_ancestor_at_epoch_boundary(store, aggregate.data.beacon_block_root, finalized_checkpoint.epoch) == store.finalized_checkpoint.root` @@ -425,9 +425,9 @@ The following validations MUST pass before forwarding the `attestation` on the s (a client MAY queue attestations for processing once block is retrieved). - _[REJECT]_ The block being voted for (`attestation.data.beacon_block_root`) passes validation. - _[REJECT]_ The attestation's target block is an ancestor of the block named in the LMD vote -- i.e. - `get_ancestor(store, attestation.data.beacon_block_root, compute_start_slot_at_epoch(attestation.data.target.epoch)) == attestation.data.target.root` + `get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, attestation.data.target.epoch) == attestation.data.target.root` - _[IGNORE]_ The current `finalized_checkpoint` is an ancestor of the `block` defined by `attestation.data.beacon_block_root` -- i.e. - `get_ancestor(store, attestation.data.beacon_block_root, compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)) + `get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root` From 1c5b9fddaf5069e642a16180aac73364e289754a Mon Sep 17 00:00:00 2001 From: Alex Stokes Date: Tue, 11 Apr 2023 13:34:42 +0900 Subject: [PATCH 21/34] add `EIP-4788` feature --- specs/_features/eip4788/beacon-chain.md | 72 +++++++++++++++++ specs/_features/eip4788/validator.md | 103 ++++++++++++++++++++++++ 2 files changed, 175 insertions(+) create mode 100644 specs/_features/eip4788/beacon-chain.md create mode 100644 specs/_features/eip4788/validator.md diff --git a/specs/_features/eip4788/beacon-chain.md b/specs/_features/eip4788/beacon-chain.md new file mode 100644 index 000000000..6cd876de9 --- /dev/null +++ b/specs/_features/eip4788/beacon-chain.md @@ -0,0 +1,72 @@ +# EIP-4788 -- The Beacon Chain + +## Table of contents + + + + + +- [Introduction](#introduction) +- [Containers](#containers) + - [Extended Containers](#extended-containers) + - [`ExecutionPayload`](#executionpayload) + - [`ExecutionPayloadHeader`](#executionpayloadheader) + + + + +## Introduction + +TODO + +## Containers + +### Extended Containers + +#### `ExecutionPayload` + +```python +class ExecutionPayload(Container): + # Execution block header fields + parent_hash: Hash32 + fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 # 'difficulty' in the yellow paper + block_number: uint64 # 'number' in the yellow paper + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + # Extra payload fields + block_hash: Hash32 # Hash of execution block + transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_beacon_block_root: Root # [New in EIP-4788] +``` + +#### `ExecutionPayloadHeader` + +```python +class ExecutionPayloadHeader(Container): + # Execution block header fields + parent_hash: Hash32 + fee_recipient: ExecutionAddress + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + # Extra payload fields + block_hash: Hash32 # Hash of execution block + transactions_root: Root + withdrawals_root: Root + parent_beacon_block_root: Root # [New in EIP-4788] +``` diff --git a/specs/_features/eip4788/validator.md b/specs/_features/eip4788/validator.md new file mode 100644 index 000000000..421e297ce --- /dev/null +++ b/specs/_features/eip4788/validator.md @@ -0,0 +1,103 @@ +# EIP-4788 -- Honest Validator + +**Notice**: This document is a work-in-progress for researchers and implementers. + +## Table of contents + + + + + +- [Introduction](#introduction) +- [Prerequisites](#prerequisites) +- [Helpers](#helpers) +- [Protocols](#protocols) + - [`ExecutionEngine`](#executionengine) + - [`get_payload`](#get_payload) +- [Beacon chain responsibilities](#beacon-chain-responsibilities) + - [Block proposal](#block-proposal) + - [Constructing the `BeaconBlockBody`](#constructing-the-beaconblockbody) + - [ExecutionPayload](#executionpayload) + + + + +## Introduction + +This document represents the changes to be made in the code of an "honest validator" to implement the EIP-4788 feature. + +## Prerequisites + +This document is an extension of the [Capella -- Honest Validator](../capella/validator.md) guide. +All behaviors and definitions defined in this document, and documents it extends, carry over unless explicitly noted or overridden. + +All terminology, constants, functions, and protocol mechanics defined in the updated Beacon Chain doc of [Capella](../capella/beacon-chain.md) are requisite for this document and used throughout. +Please see related Beacon Chain doc before continuing and use them as a reference throughout. + +## Helpers + +## Protocols + +### `ExecutionEngine` + +#### `get_payload` + +`get_payload` returns the upgraded EIP-4788 `ExecutionPayload` type. + +## Beacon chain responsibilities + +All validator responsibilities remain unchanged other than those noted below. + +### Block proposal + +#### Constructing the `BeaconBlockBody` + +##### ExecutionPayload + +`ExecutionPayload`s are constructed as they were in Capella, except that the parent beacon block root is also supplied. + +*Note*: In this section, `state` is the state of the slot for the block proposal _without_ the block yet applied. +That is, `state` is the `previous_state` processed through any empty slots up to the assigned slot using `process_slots(previous_state, slot)`. + +*Note*: The only change made to `prepare_execution_payload` is to add the parent beacon block root as an additional +parameter to the `PayloadAttributes`. + +```python +def prepare_execution_payload(state: BeaconState, + pow_chain: Dict[Hash32, PowBlock], + safe_block_hash: Hash32, + finalized_block_hash: Hash32, + suggested_fee_recipient: ExecutionAddress, + execution_engine: ExecutionEngine) -> Optional[PayloadId]: + if not is_merge_transition_complete(state): + is_terminal_block_hash_set = TERMINAL_BLOCK_HASH != Hash32() + is_activation_epoch_reached = get_current_epoch(state) >= TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH + if is_terminal_block_hash_set and not is_activation_epoch_reached: + # Terminal block hash is set but activation epoch is not yet reached, no prepare payload call is needed + return None + + terminal_pow_block = get_terminal_pow_block(pow_chain) + if terminal_pow_block is None: + # Pre-merge, no prepare payload call is needed + return None + # Signify merge via producing on top of the terminal PoW block + parent_hash = terminal_pow_block.block_hash + else: + # Post-merge, normal payload + parent_hash = state.latest_execution_payload_header.block_hash + + # Set the forkchoice head and initiate the payload build process + payload_attributes = PayloadAttributes( + timestamp=compute_timestamp_at_slot(state, state.slot), + prev_randao=get_randao_mix(state, get_current_epoch(state)), + suggested_fee_recipient=suggested_fee_recipient, + withdrawals=get_expected_withdrawals(state), + parent_beacon_block_root=hash_tree_root(state.latest_block_header), # [New in EIP-4788] + ) + return execution_engine.notify_forkchoice_updated( + head_block_hash=parent_hash, + safe_block_hash=safe_block_hash, + finalized_block_hash=finalized_block_hash, + payload_attributes=payload_attributes, + ) +``` From 334114d9d373d3a71ab49720ba4831d0b1fce6dd Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 13:14:53 +1000 Subject: [PATCH 22/34] Rename get_ancestor_at_epoch_boundary to get_checkpoint_block --- specs/bellatrix/fork-choice.md | 2 +- specs/deneb/fork-choice.md | 2 +- specs/phase0/fork-choice.md | 12 ++++++------ specs/phase0/p2p-interface.md | 8 ++++---- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/specs/bellatrix/fork-choice.md b/specs/bellatrix/fork-choice.md index d22436c9d..6c7a31508 100644 --- a/specs/bellatrix/fork-choice.md +++ b/specs/bellatrix/fork-choice.md @@ -170,7 +170,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + assert store.finalized_checkpoint.root == get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index e76e159c4..8a33fecc5 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -82,7 +82,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + assert store.finalized_checkpoint.root == get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 478dd2142..a2bbb8f62 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -18,7 +18,7 @@ - [`get_current_slot`](#get_current_slot) - [`compute_slots_since_epoch_start`](#compute_slots_since_epoch_start) - [`get_ancestor`](#get_ancestor) - - [`get_ancestor_at_epoch_boundary`](#get_ancestor_at_epoch_boundary) + - [`get_checkpoint_block`](#get_checkpoint_block) - [`get_weight`](#get_weight) - [`get_voting_source`](#get_voting_source) - [`filter_block_tree`](#filter_block_tree) @@ -193,10 +193,10 @@ def get_ancestor(store: Store, root: Root, slot: Slot) -> Root: return root ``` -#### `get_ancestor_at_epoch_boundary` +#### `get_checkpoint_block` ```python -def get_ancestor_at_epoch_boundary(store: Store, root: Root, epoch: Epoch) -> Root: +def get_checkpoint_block(store: Store, root: Root, epoch: Epoch) -> Root: """ Compute the epoch boundary block for epoch ``epoch`` in the chain of block ``root`` """ @@ -292,7 +292,7 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB correct_finalized = ( store.finalized_checkpoint.epoch == GENESIS_EPOCH - or store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + or store.finalized_checkpoint.root == get_checkpoint_block( store, block_root, store.finalized_checkpoint.epoch, @@ -457,7 +457,7 @@ def validate_on_attestation(store: Store, attestation: Attestation, is_from_bloc assert store.blocks[attestation.data.beacon_block_root].slot <= attestation.data.slot # LMD vote must be consistent with FFG vote target - assert target.root == get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, target.epoch) + assert target.root == get_checkpoint_block(store, attestation.data.beacon_block_root, target.epoch) # Attestations can only affect the fork choice of subsequent slots. # Delay consideration in the fork choice until their slot is in the past. @@ -520,7 +520,7 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_ancestor_at_epoch_boundary( + assert store.finalized_checkpoint.root == get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 56c1b8cfb..5401a15da 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -317,7 +317,7 @@ The following validations MUST pass before forwarding the `signed_beacon_block` - _[REJECT]_ The block's parent (defined by `block.parent_root`) passes validation. - _[REJECT]_ The block is from a higher slot than its parent. - _[REJECT]_ The current `finalized_checkpoint` is an ancestor of `block` -- i.e. - `get_ancestor_at_epoch_boundary(store, block.parent_root, store.finalized_checkpoint.epoch) + `get_checkpoint_block(store, block.parent_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root` - _[REJECT]_ The block is proposed by the expected `proposer_index` for the block's slot in the context of the current shuffling (defined by `parent_root`/`slot`). @@ -356,7 +356,7 @@ The following validations MUST pass before forwarding the `signed_aggregate_and_ (a client MAY queue aggregates for processing once block is retrieved). - _[REJECT]_ The block being voted for (`aggregate.data.beacon_block_root`) passes validation. - _[IGNORE]_ The current `finalized_checkpoint` is an ancestor of the `block` defined by `aggregate.data.beacon_block_root` -- i.e. - `get_ancestor_at_epoch_boundary(store, aggregate.data.beacon_block_root, finalized_checkpoint.epoch) + `get_checkpoint_block(store, aggregate.data.beacon_block_root, finalized_checkpoint.epoch) == store.finalized_checkpoint.root` @@ -425,9 +425,9 @@ The following validations MUST pass before forwarding the `attestation` on the s (a client MAY queue attestations for processing once block is retrieved). - _[REJECT]_ The block being voted for (`attestation.data.beacon_block_root`) passes validation. - _[REJECT]_ The attestation's target block is an ancestor of the block named in the LMD vote -- i.e. - `get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, attestation.data.target.epoch) == attestation.data.target.root` + `get_checkpoint_block(store, attestation.data.beacon_block_root, attestation.data.target.epoch) == attestation.data.target.root` - _[IGNORE]_ The current `finalized_checkpoint` is an ancestor of the `block` defined by `attestation.data.beacon_block_root` -- i.e. - `get_ancestor_at_epoch_boundary(store, attestation.data.beacon_block_root, store.finalized_checkpoint.epoch) + `get_checkpoint_block(store, attestation.data.beacon_block_root, store.finalized_checkpoint.epoch) == store.finalized_checkpoint.root` From 36fcb81b88c87f279cd4b46d094eb58514e9be8b Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 13:26:16 +1000 Subject: [PATCH 23/34] Break long statement into two statements --- specs/bellatrix/fork-choice.md | 3 ++- specs/deneb/fork-choice.md | 3 ++- specs/phase0/fork-choice.md | 15 +++++++++------ 3 files changed, 13 insertions(+), 8 deletions(-) diff --git a/specs/bellatrix/fork-choice.md b/specs/bellatrix/fork-choice.md index 6c7a31508..68519ff90 100644 --- a/specs/bellatrix/fork-choice.md +++ b/specs/bellatrix/fork-choice.md @@ -170,11 +170,12 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_checkpoint_block( + finalized_checkpoint_block = get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, ) + assert store.finalized_checkpoint.root == finalized_checkpoint_block # Check the block is valid and compute the post-state state = pre_state.copy() diff --git a/specs/deneb/fork-choice.md b/specs/deneb/fork-choice.md index 8a33fecc5..9faa11077 100644 --- a/specs/deneb/fork-choice.md +++ b/specs/deneb/fork-choice.md @@ -82,11 +82,12 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_checkpoint_block( + finalized_checkpoint_block = get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, ) + assert store.finalized_checkpoint.root == finalized_checkpoint_block # [New in Deneb] # Check if blob data is available diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index a2bbb8f62..8582547fd 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -290,13 +290,15 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB voting_source.epoch + 2 >= current_epoch ) + finalized_checkpoint_block = get_checkpoint_block( + store, + block.parent_root, + store.finalized_checkpoint.epoch, + ) + correct_finalized = ( store.finalized_checkpoint.epoch == GENESIS_EPOCH - or store.finalized_checkpoint.root == get_checkpoint_block( - store, - block_root, - store.finalized_checkpoint.epoch, - ) + or store.finalized_checkpoint.root == finalized_checkpoint_block ) # If expected finalized/justified, add to viable block-tree and signal viability to parent. @@ -520,11 +522,12 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None: finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) assert block.slot > finalized_slot # Check block is a descendant of the finalized block at the checkpoint finalized slot - assert store.finalized_checkpoint.root == get_checkpoint_block( + finalized_checkpoint_block = get_checkpoint_block( store, block.parent_root, store.finalized_checkpoint.epoch, ) + assert store.finalized_checkpoint.root == finalized_checkpoint_block # Check the block is valid and compute the post-state state = pre_state.copy() From c98560597351529f6f782fd434a2937d0f6c296e Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 13:49:08 +1000 Subject: [PATCH 24/34] Fix copy and past error --- specs/phase0/fork-choice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 8582547fd..0d5bfb4d7 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -292,7 +292,7 @@ def filter_block_tree(store: Store, block_root: Root, blocks: Dict[Root, BeaconB finalized_checkpoint_block = get_checkpoint_block( store, - block.parent_root, + block_root, store.finalized_checkpoint.epoch, ) From b5bd90dd5f6028d59de1fb5c97c5da95ac53b3aa Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 13:51:13 +1000 Subject: [PATCH 25/34] Applied changes to tests --- .../test/phase0/fork_choice/test_get_head.py | 22 +++++++---- .../test/phase0/fork_choice/test_on_block.py | 38 +++++++++++++------ 2 files changed, 40 insertions(+), 20 deletions(-) diff --git a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py index f5960ff70..f5c3aae15 100644 --- a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py +++ b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py @@ -479,7 +479,7 @@ def test_voting_source_within_two_epoch(spec, state): - store.voting_source[block_root].epoch != store.justified_checkpoint.epoch, and - store.unrealized_justifications[block_root].epoch >= store.justified_checkpoint.epoch, and - store.voting_source[block_root].epoch + 2 >= current_epoch, and - - store.finalized_checkpoint.root == get_ancestor(store, block_root, finalized_slot) + - store.finalized_checkpoint.root == get_checkpoint_block(store, block_root, store.finalized_checkpoint.epoch) """ test_steps = [] # Initialization @@ -536,8 +536,11 @@ def test_voting_source_within_two_epoch(spec, state): assert store.unrealized_justifications[last_fork_block_root].epoch >= store.justified_checkpoint.epoch # assert store.voting_source[last_fork_block_root].epoch + 2 >= \ # spec.compute_epoch_at_slot(spec.get_current_slot(store)) - finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - assert store.finalized_checkpoint.root == spec.get_ancestor(store, last_fork_block_root, finalized_slot) + assert store.finalized_checkpoint.root == spec.get_checkpoint_block( + store, + last_fork_block_root, + store.finalized_checkpoint.epoch + ) assert spec.get_head(store) == last_fork_block_root yield 'steps', test_steps @@ -552,7 +555,7 @@ def test_voting_source_beyond_two_epoch(spec, state): - store.voting_source[block_root].epoch != store.justified_checkpoint.epoch, and - store.unrealized_justifications[block_root].epoch >= store.justified_checkpoint.epoch, and - store.voting_source[block_root].epoch + 2 < current_epoch, and - - store.finalized_checkpoint.root == get_ancestor(store, block_root, finalized_slot) + - store.finalized_checkpoint.root == get_checkpoint_block(store, block_root, store.finalized_checkpoint.epoch) """ test_steps = [] # Initialization @@ -617,8 +620,11 @@ def test_voting_source_beyond_two_epoch(spec, state): assert store.unrealized_justifications[last_fork_block_root].epoch >= store.justified_checkpoint.epoch # assert store.voting_source[last_fork_block_root].epoch + 2 < \ # spec.compute_epoch_at_slot(spec.get_current_slot(store)) - finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - assert store.finalized_checkpoint.root == spec.get_ancestor(store, last_fork_block_root, finalized_slot) + assert store.finalized_checkpoint.root == spec.get_checkpoint_block( + store, + last_fork_block_root, + store.finalized_checkpoint.epoch + ) assert spec.get_head(store) == correct_head yield 'steps', test_steps @@ -641,7 +647,7 @@ def test_incorrect_finalized(spec, state): # Check that the store doesn't allow for a head block that has: # - store.voting_source[block_root].epoch == store.justified_checkpoint.epoch, and # - store.finalized_checkpoint.epoch != GENESIS_EPOCH, and - # - store.finalized_checkpoint.root != get_ancestor(store, block_root, finalized_slot) + # - store.finalized_checkpoint.root != get_checkpoint_block(store, block_root, store.finalized_checkpoint.epoch) test_steps = [] # Initialization store, anchor_block = get_genesis_forkchoice_store_and_block(spec, state) @@ -718,7 +724,7 @@ def test_incorrect_finalized(spec, state): assert store.voting_source[last_fork_block_root].epoch == store.justified_checkpoint.epoch assert store.finalized_checkpoint.epoch != spec.GENESIS_EPOCH finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - assert store.finalized_checkpoint.root != spec.get_ancestor(store, last_fork_block_root, finalized_slot) + assert store.finalized_checkpoint.root != spec.get_checkpoint_block(store, block_root, store.finalized_checkpoint.epoch) assert spec.get_head(store) != last_fork_block_root assert spec.get_head(store) == head_root diff --git a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py index 0af775339..a3f09c7c9 100644 --- a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py +++ b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py @@ -352,8 +352,7 @@ def test_new_finalized_slot_is_not_justified_checkpoint_ancestor(spec, state): # NOTE: Do not call `on_tick` here yield from add_block(spec, store, block, test_steps) - finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - ancestor_at_finalized_slot = spec.get_ancestor(store, pre_store_justified_checkpoint_root, finalized_slot) + ancestor_at_finalized_slot = spec.get_checkpoint_block(store, pre_store_justified_checkpoint_root, store.finalized_checkpoint.epoch) assert ancestor_at_finalized_slot != store.finalized_checkpoint.root assert store.finalized_checkpoint == another_state.finalized_checkpoint @@ -428,8 +427,7 @@ def test_new_finalized_slot_is_justified_checkpoint_ancestor(spec, state): for block in all_blocks: yield from tick_and_add_block(spec, store, block, test_steps) - finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - ancestor_at_finalized_slot = spec.get_ancestor(store, pre_store_justified_checkpoint_root, finalized_slot) + ancestor_at_finalized_slot = spec.get_checkpoint_block(store, pre_store_justified_checkpoint_root, store.finalized_checkpoint.epoch) assert ancestor_at_finalized_slot == store.finalized_checkpoint.root assert store.finalized_checkpoint == another_state.finalized_checkpoint @@ -857,10 +855,18 @@ def test_incompatible_justification_update_start_of_epoch(spec, state): # Now add the blocks & check that justification update was triggered for signed_block in signed_blocks: yield from tick_and_add_block(spec, store, signed_block, test_steps) - finalized_slot = spec.compute_start_slot_at_epoch(state.finalized_checkpoint.epoch) - assert spec.get_ancestor(store, last_block_root, finalized_slot) == state.finalized_checkpoint.root - justified_slot = spec.compute_start_slot_at_epoch(state.current_justified_checkpoint.epoch) - assert spec.get_ancestor(store, last_block_root, justified_slot) != state.current_justified_checkpoint.root + finalized_checkpoint_block = spec.get_checkpoint_block( + store, + last_block_root, + state.finalized_checkpoint.epoch, + ) + assert finalized_checkpoint_block == state.finalized_checkpoint.root + justified_checkpoint_block = spec.get_checkpoint_block( + store, + last_block_root, + state.current_justified_checkpoint.epoch, + ) + assert justified_checkpoint_block != state.current_justified_checkpoint.root assert store.finalized_checkpoint.epoch == 4 assert store.justified_checkpoint.epoch == 6 @@ -934,10 +940,18 @@ def test_incompatible_justification_update_end_of_epoch(spec, state): # Now add the blocks & check that justification update was triggered for signed_block in signed_blocks: yield from tick_and_add_block(spec, store, signed_block, test_steps) - finalized_slot = spec.compute_start_slot_at_epoch(state.finalized_checkpoint.epoch) - assert spec.get_ancestor(store, last_block_root, finalized_slot) == state.finalized_checkpoint.root - justified_slot = spec.compute_start_slot_at_epoch(state.current_justified_checkpoint.epoch) - assert spec.get_ancestor(store, last_block_root, justified_slot) != state.current_justified_checkpoint.root + finalized_checkpoint_block = spec.get_checkpoint_block( + store, + last_block_root, + state.finalized_checkpoint.epoch, + ) + assert finalized_checkpoint_block == state.finalized_checkpoint.root + justified_checkpoint_block = spec.get_checkpoint_block( + store, + last_block_root, + state.current_justified_checkpoint.epoch, + ) + assert justified_checkpoint_block != state.current_justified_checkpoint.root assert store.finalized_checkpoint.epoch == 4 assert store.justified_checkpoint.epoch == 6 From 313439a04b121a1c53585396dbea6df02026404f Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 13:54:31 +1000 Subject: [PATCH 26/34] Fix lint erorrs --- .../test/phase0/fork_choice/test_get_head.py | 6 +++++- .../test/phase0/fork_choice/test_on_block.py | 12 ++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py index f5c3aae15..30f94b854 100644 --- a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py +++ b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_get_head.py @@ -724,7 +724,11 @@ def test_incorrect_finalized(spec, state): assert store.voting_source[last_fork_block_root].epoch == store.justified_checkpoint.epoch assert store.finalized_checkpoint.epoch != spec.GENESIS_EPOCH finalized_slot = spec.compute_start_slot_at_epoch(store.finalized_checkpoint.epoch) - assert store.finalized_checkpoint.root != spec.get_checkpoint_block(store, block_root, store.finalized_checkpoint.epoch) + assert store.finalized_checkpoint.root != spec.get_checkpoint_block( + store, + block_root, + store.finalized_checkpoint.epoch + ) assert spec.get_head(store) != last_fork_block_root assert spec.get_head(store) == head_root diff --git a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py index a3f09c7c9..840413a36 100644 --- a/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py +++ b/tests/core/pyspec/eth2spec/test/phase0/fork_choice/test_on_block.py @@ -352,7 +352,11 @@ def test_new_finalized_slot_is_not_justified_checkpoint_ancestor(spec, state): # NOTE: Do not call `on_tick` here yield from add_block(spec, store, block, test_steps) - ancestor_at_finalized_slot = spec.get_checkpoint_block(store, pre_store_justified_checkpoint_root, store.finalized_checkpoint.epoch) + ancestor_at_finalized_slot = spec.get_checkpoint_block( + store, + pre_store_justified_checkpoint_root, + store.finalized_checkpoint.epoch + ) assert ancestor_at_finalized_slot != store.finalized_checkpoint.root assert store.finalized_checkpoint == another_state.finalized_checkpoint @@ -427,7 +431,11 @@ def test_new_finalized_slot_is_justified_checkpoint_ancestor(spec, state): for block in all_blocks: yield from tick_and_add_block(spec, store, block, test_steps) - ancestor_at_finalized_slot = spec.get_checkpoint_block(store, pre_store_justified_checkpoint_root, store.finalized_checkpoint.epoch) + ancestor_at_finalized_slot = spec.get_checkpoint_block( + store, + pre_store_justified_checkpoint_root, + store.finalized_checkpoint.epoch + ) assert ancestor_at_finalized_slot == store.finalized_checkpoint.root assert store.finalized_checkpoint == another_state.finalized_checkpoint From ffb84598cf5f7d29bd6220e977f76d47c599bb4f Mon Sep 17 00:00:00 2001 From: Roberto Saltini Date: Tue, 18 Apr 2023 16:03:10 +1000 Subject: [PATCH 27/34] Fixed doc in get_checkpoint_block --- specs/phase0/fork-choice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 0d5bfb4d7..e25ae6e90 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -198,7 +198,7 @@ def get_ancestor(store: Store, root: Root, slot: Slot) -> Root: ```python def get_checkpoint_block(store: Store, root: Root, epoch: Epoch) -> Root: """ - Compute the epoch boundary block for epoch ``epoch`` in the chain of block ``root`` + Compute the checkpoint block for epoch ``epoch`` in the chain of block ``root`` """ epoch_first_slot = compute_start_slot_at_epoch(epoch) return get_ancestor(store, root, epoch_first_slot) From 21d4370fd5a36112f035880d528e59a3a2c523ff Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 18 Apr 2023 18:31:34 +0800 Subject: [PATCH 28/34] Add docs about how to add a new feature proposal in consensus-specs and online viewer support (#3239) * Add docs * update link to template * Add more info * Try mkdocs * Create docs.yml * Update docs.yml * update * update * update * Try mkdocs * Add "B: Make it executable for pytest and test generator" section * Use mkdocs-material * Use `mkdocs-awesome-pages-plugin` to create custom specs order * Add toc permalink * Update site_url * Add .pages files for navigations. Update highlight style * Dark theme * Fix list indent --- .github/workflows/docs.yml | 24 +++ .gitignore | 8 + Makefile | 25 +++ README.md | 4 + docs/.pages | 5 + docs/README.md | 70 ++++++++ docs/docs/new-feature.md | 163 +++++++++++++++++++ docs/docs/templates/beacon-chain-template.md | 84 ++++++++++ docs/light-client/.pages | 5 + docs/light-client/index.md | 1 + docs/stylesheets/extra.css | 34 ++++ fork_choice/.pages | 7 + mkdocs.yml | 40 +++++ setup.py | 1 + specs/.pages | 4 + specs/_features/eip6110/beacon-chain.md | 2 +- specs/_features/sharding/p2p-interface.md | 2 +- 17 files changed, 477 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/docs.yml create mode 100644 docs/.pages create mode 100644 docs/README.md create mode 100644 docs/docs/new-feature.md create mode 100644 docs/docs/templates/beacon-chain-template.md create mode 100644 docs/light-client/.pages create mode 100644 docs/light-client/index.md create mode 100644 docs/stylesheets/extra.css create mode 100644 fork_choice/.pages create mode 100644 mkdocs.yml create mode 100644 specs/.pages diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 000000000..eab3bba17 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,24 @@ + +name: Publish docs +on: + push: + branches: + - master +permissions: + contents: write +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - name: Build docs + run: make copy_docs + - uses: actions/setup-python@v4 + with: + python-version: 3.x + - uses: actions/cache@v2 + with: + key: ${{ github.ref }} + path: .cache + - run: pip install -e .[docs] + - run: mkdocs gh-deploy --force diff --git a/.gitignore b/.gitignore index c56a658ce..82026c27b 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,11 @@ tests/core/pyspec/eth2spec/test_results.xml # TOC tool outputs temporary files *.tmp + +# docs reader build +docs/specs +docs/sync +docs/ssz +docs/fork_choice +docs/README.md +site diff --git a/Makefile b/Makefile index 1ec399e3a..ab5521663 100644 --- a/Makefile +++ b/Makefile @@ -19,6 +19,11 @@ GENERATORS = $(sort $(dir $(wildcard $(GENERATOR_DIR)/*/.))) # Map this list of generator paths to "gen_{generator name}" entries GENERATOR_TARGETS = $(patsubst $(GENERATOR_DIR)/%/, gen_%, $(GENERATORS)) GENERATOR_VENVS = $(patsubst $(GENERATOR_DIR)/%, $(GENERATOR_DIR)/%venv, $(GENERATORS)) +# Documents +DOCS_DIR = ./docs +SSZ_DIR = ./ssz +SYNC_DIR = ./sync +FORK_CHOICE_DIR = ./fork_choice # To check generator matching: #$(info $$GENERATOR_TARGETS is [${GENERATOR_TARGETS}]) @@ -214,3 +219,23 @@ detect_generator_incomplete: $(TEST_VECTOR_DIR) detect_generator_error_log: $(TEST_VECTOR_DIR) [ -f $(GENERATOR_ERROR_LOG_FILE) ] && echo "[ERROR] $(GENERATOR_ERROR_LOG_FILE) file exists" || echo "[PASSED] error log file does not exist" + + +# For docs reader +install_docs: + python3 -m venv venv; . venv/bin/activate; python3 -m pip install -e .[docs]; + +copy_docs: + cp -r $(SPEC_DIR) $(DOCS_DIR); + cp -r $(SYNC_DIR) $(DOCS_DIR); + cp -r $(SSZ_DIR) $(DOCS_DIR); + cp -r $(FORK_CHOICE_DIR) $(DOCS_DIR); + cp $(CURRENT_DIR)/README.md $(DOCS_DIR)/README.md + +build_docs: copy_docs + . venv/bin/activate; + mkdocs build + +serve_docs: + . venv/bin/activate; + mkdocs serve diff --git a/README.md b/README.md index d0d6b222d..28d61ad40 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,10 @@ Documentation on the different components used during spec writing can be found * [YAML Test Generators](tests/generators/README.md) * [Executable Python Spec, with Py-tests](tests/core/pyspec/README.md) +## Online viewer of the latest release (latest `master` branch) + +[Ethereum Consensus Specs](https://ethereum.github.io/consensus-specs/) + ## Consensus spec tests Conformance tests built from the executable python spec are available in the [Ethereum Proof-of-Stake Consensus Spec Tests](https://github.com/ethereum/consensus-spec-tests) repo. Compressed tarballs are available in [releases](https://github.com/ethereum/consensus-spec-tests/releases). diff --git a/docs/.pages b/docs/.pages new file mode 100644 index 000000000..d9e382ede --- /dev/null +++ b/docs/.pages @@ -0,0 +1,5 @@ +nav: + - Home: + - README.md + - specs + - ... diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..9f2528263 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,70 @@ +# Ethereum Proof-of-Stake Consensus Specifications + +[![Join the chat at https://discord.gg/qGpsxSA](https://img.shields.io/badge/chat-on%20discord-blue.svg)](https://discord.gg/qGpsxSA) [![Join the chat at https://gitter.im/ethereum/sharding](https://badges.gitter.im/ethereum/sharding.svg)](https://gitter.im/ethereum/sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) + +To learn more about proof-of-stake and sharding, see the [PoS documentation](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/), [sharding documentation](https://ethereum.org/en/upgrades/sharding/) and the [research compendium](https://notes.ethereum.org/s/H1PGqDhpm). + +This repository hosts the current Ethereum proof-of-stake specifications. Discussions about design rationale and proposed changes can be brought up and discussed as issues. Solidified, agreed-upon changes to the spec can be made through pull requests. + +## Specs + +[![GitHub release](https://img.shields.io/github/v/release/ethereum/eth2.0-specs)](https://github.com/ethereum/eth2.0-specs/releases/) [![PyPI version](https://badge.fury.io/py/eth2spec.svg)](https://badge.fury.io/py/eth2spec) + +Core specifications for Ethereum proof-of-stake clients can be found in [specs](specs/). These are divided into features. +Features are researched and developed in parallel, and then consolidated into sequential upgrades when ready. + +### Stable Specifications + +| Seq. | Code Name | Fork Epoch | Specs | +| - | - | - | - | +| 0 | **Phase0** |`0` |
  • Core
    • [The beacon chain](specs/phase0/beacon-chain.md)
    • [Deposit contract](specs/phase0/deposit-contract.md)
    • [Beacon chain fork choice](specs/phase0/fork-choice.md)
  • Additions
    • [Honest validator guide](specs/phase0/validator.md)
    • [P2P networking](specs/phase0/p2p-interface.md)
    • [Weak subjectivity](specs/phase0/weak-subjectivity.md)
| +| 1 | **Altair** | `74240` |
  • Core
    • [Beacon chain changes](specs/altair/beacon-chain.md)
    • [Altair fork](specs/altair/fork.md)
  • Additions
    • [Light client sync protocol](specs/altair/light-client/sync-protocol.md) ([full node](specs/altair/light-client/full-node.md), [light client](specs/altair/light-client/light-client.md), [networking](specs/altair/light-client/p2p-interface.md))
    • [Honest validator guide changes](specs/altair/validator.md)
    • [P2P networking](specs/altair/p2p-interface.md)
| +| 2 | **Bellatrix**
(["The Merge"](https://ethereum.org/en/upgrades/merge/)) | `144896` |
  • Core
    • [Beacon Chain changes](specs/bellatrix/beacon-chain.md)
    • [Bellatrix fork](specs/bellatrix/fork.md)
    • [Fork choice changes](specs/bellatrix/fork-choice.md)
  • Additions
    • [Honest validator guide changes](specs/bellatrix/validator.md)
    • [P2P networking](specs/bellatrix/p2p-interface.md)
| +| 3 | **Capella** | `194048` |
  • Core
    • [Beacon chain changes](specs/capella/beacon-chain.md)
    • [Capella fork](specs/capella/fork.md)
  • Additions
    • [Light client sync protocol changes](specs/capella/light-client/sync-protocol.md) ([fork](specs/capella/light-client/fork.md), [full node](specs/capella/light-client/full-node.md), [networking](specs/capella/light-client/p2p-interface.md))
    • [Validator additions](specs/capella/validator.md)
    • [P2P networking](specs/capella/p2p-interface.md)
| + +### In-development Specifications +| Code Name or Topic | Specs | Notes | +| - | - | - | +| Deneb (tentative) |
  • Core
    • [Beacon Chain changes](specs/deneb/beacon-chain.md)
    • [Deneb fork](specs/deneb/fork.md)
    • [Polynomial commitments](specs/deneb/polynomial-commitments.md)
    • [Fork choice changes](specs/deneb/fork-choice.md)
  • Additions
    • [Light client sync protocol changes](specs/deneb/light-client/sync-protocol.md) ([fork](specs/deneb/light-client/fork.md), [full node](specs/deneb/light-client/full-node.md), [networking](specs/deneb/light-client/p2p-interface.md))
    • [Honest validator guide changes](specs/deneb/validator.md)
    • [P2P networking](specs/deneb/p2p-interface.md)
| +| Sharding (outdated) |
  • Core
    • [Beacon Chain changes](specs/_features/sharding/beacon-chain.md)
  • Additions
    • [P2P networking](specs/_features/sharding/p2p-interface.md)
| +| Custody Game (outdated) |
  • Core
    • [Beacon Chain changes](specs/_features/custody_game/beacon-chain.md)
  • Additions
    • [Honest validator guide changes](specs/_features/custody_game/validator.md)
| Dependent on sharding | +| Data Availability Sampling (outdated) |
  • Core
    • [Core types and functions](specs/_features/das/das-core.md)
    • [Fork choice changes](specs/_features/das/fork-choice.md)
  • Additions
    • [P2P Networking](specs/_features/das/p2p-interface.md)
    • [Sampling process](specs/_features/das/sampling.md)
|
  • Dependent on sharding
  • [Technical explainer](https://hackmd.io/@HWeNw8hNRimMm2m2GH56Cw/B1YJPGkpD)
| +| EIP-6110 |
  • Core
    • [Beacon Chain changes](specs/_features/eip6110//beacon-chain.md)
    • [EIP-6110 fork](specs/_features/eip6110/fork.md)
  • Additions
    • [Honest validator guide changes](specs/_features/eip6110/validator.md)
| + +### Accompanying documents can be found in [specs](specs) and include: + +* [SimpleSerialize (SSZ) spec](ssz/simple-serialize.md) +* [Merkle proof formats](ssz/merkle-proofs.md) +* [General test format](tests/formats/README.md) + +## Additional specifications for client implementers + +Additional specifications and standards outside of requisite client functionality can be found in the following repos: + +* [Beacon APIs](https://github.com/ethereum/beacon-apis) +* [Beacon Metrics](https://github.com/ethereum/beacon-metrics/) + +## Design goals + +The following are the broad design goals for the Ethereum proof-of-stake consensus specifications: +* to minimize complexity, even at the cost of some losses in efficiency +* to remain live through major network partitions and when very large portions of nodes go offline +* to select all components such that they are either quantum secure or can be easily swapped out for quantum secure counterparts when available +* to utilize crypto and design techniques that allow for a large participation of validators in total and per unit time +* to allow for a typical consumer laptop with `O(C)` resources to process/validate `O(1)` shards (including any system level validation such as the beacon chain) + +## Useful external resources + +* [Design Rationale](https://notes.ethereum.org/s/rkhCgQteN#) +* [Phase 0 Onboarding Document](https://notes.ethereum.org/s/Bkn3zpwxB) +* [Combining GHOST and Casper paper](https://arxiv.org/abs/2003.03052) + +## For spec contributors + +Documentation on the different components used during spec writing can be found here: +* [YAML Test Generators](tests/generators/README.md) +* [Executable Python Spec, with Py-tests](tests/core/pyspec/README.md) + +## Consensus spec tests + +Conformance tests built from the executable python spec are available in the [Ethereum Proof-of-Stake Consensus Spec Tests](https://github.com/ethereum/consensus-spec-tests) repo. Compressed tarballs are available in [releases](https://github.com/ethereum/consensus-spec-tests/releases). diff --git a/docs/docs/new-feature.md b/docs/docs/new-feature.md new file mode 100644 index 000000000..5e6180329 --- /dev/null +++ b/docs/docs/new-feature.md @@ -0,0 +1,163 @@ +# How to add a new feature proposal in consensus-specs + + + +## Table of Contents + +- [A. Make it executable for linter checks](#a-make-it-executable-for-linter-checks) + - [1. Create a folder under `./specs/_features`](#1-create-a-folder-under-specs_features) + - [2. Choose the "previous fork" to extend: usually, use the scheduled or the latest mainnet fork version.](#2-choose-the-previous-fork-to-extend-usually-use-the-scheduled-or-the-latest-mainnet-fork-version) + - [3. Write down your proposed `beacon-chain.md` change](#3-write-down-your-proposed-beacon-chainmd-change) + - [4. Add `fork.md`](#4-add-forkmd) + - [5. Make it executable](#5-make-it-executable) +- [B: Make it executable for pytest and test generator](#b-make-it-executable-for-pytest-and-test-generator) + - [1. Add `light-client/*` docs if you updated the content of `BeaconBlock`](#1-add-light-client-docs-if-you-updated-the-content-of-beaconblock) + - [2. Add the mainnet and minimal presets and update the configs](#2-add-the-mainnet-and-minimal-presets-and-update-the-configs) + - [3. Update `context.py`](#3-update-contextpy) + - [4. Update `constants.py`](#4-update-constantspy) + - [5. Update `genesis.py`:](#5-update-genesispy) + - [6. To add fork transition tests, update fork_transition.py](#6-to-add-fork-transition-tests-update-fork_transitionpy) + - [7. Update CI configurations](#7-update-ci-configurations) +- [Others](#others) + - [Bonus](#bonus) + - [Need help?](#need-help) + + + + +## A. Make it executable for linter checks + +### 1. Create a folder under `./specs/_features` + +For example, if it's an `EIP-9999` CL spec, you can create a `./specs/_features/eip9999` folder. + +### 2. Choose the "previous fork" to extend: usually, use the scheduled or the latest mainnet fork version. + +For example, if the latest fork is Capella, use `./specs/capella` content as your "previous fork". + +### 3. Write down your proposed `beacon-chain.md` change +- You can either use [Beacon Chain Spec Template](./templates/beacon-chain-template.md), or make a copy of the latest fork content and then edit it. +- Tips: + - We use [`doctoc`](https://www.npmjs.com/package/doctoc) tool to generate the table of content. + ``` + cd consensus-specs + doctoc specs + ``` + - The differences between "Constants", "Configurations", and "Presets": + - Constants: The constant that should never be changed. + - Configurations: The settings that we may change for different networks. + - Presets: The settings that we may change for testing. + - Readability and simplicity are more important than efficiency and optimization. + - Use simple Python rather than the fancy Python dark magic. + +### 4. Add `fork.md` +You can refer to the previous fork's `fork.md` file. +### 5. Make it executable +- Update [`constants.py`](https://github.com/ethereum/consensus-specs/blob/dev/tests/core/pyspec/eth2spec/test/helpers/constants.py) with the new feature name. +- Update [`setup.py`](https://github.com/ethereum/consensus-specs/blob/dev/setup.py): + - Add a new `SpecBuilder` with the new feature name constant. e.g., `EIP9999SpecBuilder` + - Add the new `SpecBuilder` to `spec_builders` list. + - Add the path of the new markdown files in `finalize_options` function. + +## B: Make it executable for pytest and test generator + +### 1. Add `light-client/*` docs if you updated the content of `BeaconBlock` +- You can refer to the previous fork's `light-client/*` file. +- Add the path of the new markdown files in `setup.py`'s `finalize_options` function. + +### 2. Add the mainnet and minimal presets and update the configs +- Add presets: `presets/mainnet/.yaml` and `presets/minimal/.yaml` +- Update configs: `configs/mainnet.yaml` and `configs/minimal.yaml` + +### 3. Update [`context.py`](https://github.com/ethereum/consensus-specs/blob/dev/tests/core/pyspec/eth2spec/test/context.py) +- Update `spec_targets` by adding `` + +```python +from eth2spec.eip9999 import mainnet as spec_eip9999_mainnet, minimal as spec_eip9999_minimal + +... + +spec_targets: Dict[PresetBaseName, Dict[SpecForkName, Spec]] = { + MINIMAL: { + ... + EIP9999: spec_eip9999_minimal, + }, + MAINNET: { + ... + EIP9999: spec_eip9999_mainnet + }, +} +``` + +### 4. Update [`constants.py`](https://github.com/ethereum/consensus-specs/blob/dev/tests/core/pyspec/eth2spec/test/helpers/constants.py) +- Add `` to `ALL_PHASES` and `TESTGEN_FORKS` + +### 5. Update [`genesis.py`](https://github.com/ethereum/consensus-specs/blob/dev/tests/core/pyspec/eth2spec/test/helpers/genesis.py): + +We use `create_genesis_state` to create the default `state` in tests. + +- Update `create_genesis_state` by adding `fork_version` setting: + +```python +def create_genesis_state(spec, validator_balances, activation_threshold): + ... + if spec.fork == ALTAIR: + current_version = spec.config.ALTAIR_FORK_VERSION + ... + elif spec.fork == EIP9999: + # Add the previous fork version of given fork + previous_version = spec.config. + current_version = spec.config.EIP9999_FORK_VERSION +``` + +- If the given feature changes `BeaconState` fields, you have to set the initial values by adding: + +```python +def create_genesis_state(spec, validator_balances, activation_threshold): + ... + if is_post_eip9999(spec): + state. = + + return state +``` + +- If the given feature changes `ExecutionPayload` fields, you have to set the initial values by updating `get_sample_genesis_execution_payload_header` helper. + +### 6. To add fork transition tests, update [fork_transition.py](https://github.com/ethereum/consensus-specs/blob/dev/tests/core/pyspec/eth2spec/test/helpers/fork_transition.py) + +```python +def do_fork(state, spec, post_spec, fork_epoch, with_block=True, sync_aggregate=None, operation_dict=None): + ... + + if post_spec.fork == ALTAIR: + state = post_spec.upgrade_to_altair(state) + ... + elif post_spec.fork == EIP9999: + state = post_spec.upgrade_to_eip9999(state) + + ... + + if post_spec.fork == ALTAIR: + assert state.fork.previous_version == post_spec.config.GENESIS_FORK_VERSION + assert state.fork.current_version == post_spec.config.ALTAIR_FORK_VERSION + ... + elif post_spec.fork == EIP9999: + assert state.fork.previous_version == post_spec.config. + assert state.fork.current_version == post_spec.config.EIP9999_FORK_VERSION + + ... +``` + +### 7. Update CI configurations +- Update [GitHub Actions config](https://github.com/ethereum/consensus-specs/blob/dev/.github/workflows/run-tests.yml) + - Update `pyspec-tests.strategy.matrix.version` list by adding new feature to it +- Update [CircleCI config](https://github.com/ethereum/consensus-specs/blob/dev/.circleci/config.yml) + - Add new job to the `workflows.test_spec.jobs` + +## Others + +### Bonus +- Add `validator.md` if honest validator behavior changes with the new feature. + +### Need help? +You can tag spec elves for cleaning up your PR. 🧚 diff --git a/docs/docs/templates/beacon-chain-template.md b/docs/docs/templates/beacon-chain-template.md new file mode 100644 index 000000000..4d22d3908 --- /dev/null +++ b/docs/docs/templates/beacon-chain-template.md @@ -0,0 +1,84 @@ +# `beacon-chain.md` Template + +# -- The Beacon Chain + +## Table of contents + + + + + + + + + +## Introduction + +## Notation + +## Custom types + +## Constants + +### [CATEGORY OF CONSTANTS] + +| Name | Value | +| - | - | +| `` | ``` | + +## Preset + + +### [CATEGORY OF PRESETS] + +| Name | Value | +| - | - | +| `` | `` | + +## Configuration + +### [CATEGORY OF CONFIGURATIONS] + +| Name | Value | +| - | - | +| `` | `` | + +## Containers + +### [CATEGORY OF CONTAINERS] + +#### `CONTAINER_NAME` + +```python +class CONTAINER_NAME(Container): + FILED_NAME: SSZ_TYPE +``` + +## Helper functions + +### [CATEGORY OF HELPERS] + +```python + +``` + +### Epoch processing + + +### Block processing + + + + +## Testing + +*Note*: The function `initialize_beacon_state_from_eth1` is modified for pure testing only. + +```python +def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32, + eth1_timestamp: uint64, + deposits: Sequence[Deposit], + execution_payload_header: ExecutionPayloadHeader=ExecutionPayloadHeader() + ) -> BeaconState: + ... +``` diff --git a/docs/light-client/.pages b/docs/light-client/.pages new file mode 100644 index 000000000..a372a5d2e --- /dev/null +++ b/docs/light-client/.pages @@ -0,0 +1,5 @@ +nav: + - 'Index': index.md + - 'Altair': specs/altair/light-client/sync-protocol + - 'Capella': specs/capella/light-client/sync-protocol + - 'Deneb': specs/deneb/light-client/sync-protocol diff --git a/docs/light-client/index.md b/docs/light-client/index.md new file mode 100644 index 000000000..32155b185 --- /dev/null +++ b/docs/light-client/index.md @@ -0,0 +1 @@ +# Light client specifications diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css new file mode 100644 index 000000000..384976248 --- /dev/null +++ b/docs/stylesheets/extra.css @@ -0,0 +1,34 @@ +/* Reference: https://zenn.dev/mebiusbox/articles/81d977a72cee01 */ + +[data-md-color-scheme=default] { + --md-default-fg-color--light: #222 !important; +} +[data-md-color-scheme=slate] { + --md-default-fg-color--light: #fefefe !important; + --md-typeset-a-color: #fc0 !important; +} + +.md-typeset pre { + color: #f8f8f2; +} +.md-typeset .highlighttable { + margin-left:-20px; + margin-right: -20px; + border-radius: 0; +} +.md-typeset .highlighttable > * { + --md-code-bg-color: #222 !important; + --md-code-fg-color: #fefefe !important; +} +.md-typeset .highlighttable .linenos .linenodiv pre span { + background-color: #222 !important; + color: #fefefe !important; +} +.md-typeset .highlighttable .md-clipboard:before, +.md-typeset .highlighttable .md-clipboard:after { + color: rgba(240,240,240,.8); +} +.md-typeset .highlighttable .md-clipboard:hover:before, +.md-typeset .highlighttable .md-clipboard:hover:after { + color: rgba(102,217,224,1); +} diff --git a/fork_choice/.pages b/fork_choice/.pages new file mode 100644 index 000000000..a5e6ccc90 --- /dev/null +++ b/fork_choice/.pages @@ -0,0 +1,7 @@ +nav: + - ... + - Fork Choice -- Core: + - phase0: specs/phase0/fork-choice + - bellatrix: specs/bellatrix/fork-choice + - capella: specs/capella/fork-choice + - deneb: specs/deneb/fork-choice diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 000000000..dc6b352ba --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,40 @@ +site_name: Ethereum Consensus Specs +site_url: https://ethereum.github.io/consensus-specs/ +repo_name: ethereum/consensus-specs +theme: + name: material + palette: + - scheme: default + primary: black + toggle: + icon: material/brightness-7 + name: Switch to dark mode + - scheme: slate + primary: black + toggle: + icon: material/brightness-4 + name: Switch to light mode + features: + - navigation.tabs + - search +markdown_extensions: + - toc: + permalink: true + - pymdownx.superfences + - pymdownx.highlight: + use_pygments: true + noclasses: true + pygments_style: monokai + linenums: true + anchor_linenums: true + - mdx_truly_sane_lists: + nested_indent: 4 +plugins: + - search + - awesome-pages +extra_css: + - stylesheets/extra.css +extra: + social: + - icon: fontawesome/brands/github + link: https://github.com/ethereum/consensus-specs diff --git a/setup.py b/setup.py index 52bad2b71..fc3acb806 100644 --- a/setup.py +++ b/setup.py @@ -1181,6 +1181,7 @@ setup( "test": ["pytest>=4.4", "pytest-cov", "pytest-xdist"], "lint": ["flake8==5.0.4", "mypy==0.981", "pylint==2.15.3"], "generator": ["python-snappy==0.6.1", "filelock"], + "docs": ["mkdocs==1.4.2", "mkdocs-material==9.1.5", "mdx-truly-sane-lists==1.3", "mkdocs-awesome-pages-plugin==2.8.0"] }, install_requires=[ "eth-utils>=2.0.0,<3", diff --git a/specs/.pages b/specs/.pages new file mode 100644 index 000000000..7e47dc5f7 --- /dev/null +++ b/specs/.pages @@ -0,0 +1,4 @@ +nav: + - phase0 + - ... + - _features diff --git a/specs/_features/eip6110/beacon-chain.md b/specs/_features/eip6110/beacon-chain.md index 70a72a5f4..01999a929 100644 --- a/specs/_features/eip6110/beacon-chain.md +++ b/specs/_features/eip6110/beacon-chain.md @@ -33,7 +33,7 @@ This is the beacon chain specification of in-protocol deposits processing mechanism. This mechanism relies on the changes proposed by [EIP-6110](http://eips.ethereum.org/EIPS/eip-6110). -*Note:* This specification is built upon [Capella](../../capella/beacon_chain.md) and is under active development. +*Note:* This specification is built upon [Capella](../../capella/beacon-chain.md) and is under active development. ## Constants diff --git a/specs/_features/sharding/p2p-interface.md b/specs/_features/sharding/p2p-interface.md index c29146fe9..553444eff 100644 --- a/specs/_features/sharding/p2p-interface.md +++ b/specs/_features/sharding/p2p-interface.md @@ -47,7 +47,7 @@ Following the same scheme as the [Phase0 gossip topics](../../phase0/p2p-interfa | `shard_column_{subnet_id}` | `SignedShardSample` | | `builder_block_bid` | `BuilderBlockBid` | -The [DAS network specification](./das-p2p.md) defines additional topics. +The [DAS network specification](../das/das-core.md) defines additional topics. #### Builder block bid From 85c8daf08530545c50fd2f5b6998342bcac85977 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 18 Apr 2023 18:34:12 +0800 Subject: [PATCH 29/34] bump version.txt to 1.3.0 --- tests/core/pyspec/eth2spec/VERSION.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/core/pyspec/eth2spec/VERSION.txt b/tests/core/pyspec/eth2spec/VERSION.txt index d4f06976f..f0bb29e76 100644 --- a/tests/core/pyspec/eth2spec/VERSION.txt +++ b/tests/core/pyspec/eth2spec/VERSION.txt @@ -1 +1 @@ -1.3.0-rc.5 +1.3.0 From 87d42919b9f80e18ff2eba5354642d6b6c4829c6 Mon Sep 17 00:00:00 2001 From: Justin Traglia Date: Tue, 18 Apr 2023 14:00:11 -0500 Subject: [PATCH 30/34] Add "commitment" to test names for consistency --- tests/generators/kzg_4844/main.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/generators/kzg_4844/main.py b/tests/generators/kzg_4844/main.py index 699d1f369..b1391f678 100644 --- a/tests/generators/kzg_4844/main.py +++ b/tests/generators/kzg_4844/main.py @@ -708,7 +708,7 @@ def case06_verify_blob_kzg_proof_batch(): # Edge case: Invalid commitment, too few bytes commitments_invalid_tooFewBytes = commitments[:3] + [commitments[3][:-1]] + commitments[4:] expect_exception(spec.verify_blob_kzg_proof_batch, VALID_BLOBS, commitments, commitments_invalid_tooFewBytes) - yield 'verify_blob_kzg_proof_batch_case_too_few_bytes', { + yield 'verify_blob_kzg_proof_batch_case_commitment_too_few_bytes', { 'input': { 'blobs': encode_hex_list(VALID_BLOBS), 'commitments': encode_hex_list(commitments_invalid_tooFewBytes), @@ -720,7 +720,7 @@ def case06_verify_blob_kzg_proof_batch(): # Edge case: Invalid commitment, too many bytes commitments_invalid_tooManyBytes = commitments[:3] + [commitments[3] + b"\x00"] + commitments[4:] expect_exception(spec.verify_blob_kzg_proof_batch, VALID_BLOBS, commitments, commitments_invalid_tooManyBytes) - yield 'verify_blob_kzg_proof_batch_case_too_many_bytes', { + yield 'verify_blob_kzg_proof_batch_case_commitment_too_many_bytes', { 'input': { 'blobs': encode_hex_list(VALID_BLOBS), 'commitments': encode_hex_list(commitments_invalid_tooManyBytes), From 03a3e4082a4cfcc984bf4b46d1c17cc08e61b576 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 19 Apr 2023 19:10:46 +0800 Subject: [PATCH 31/34] Fix p2p-interface.md ToC --- docs/README.md | 70 ------------ specs/altair/p2p-interface.md | 42 +++---- specs/bellatrix/p2p-interface.md | 64 +++++------ specs/capella/p2p-interface.md | 24 ++-- specs/deneb/p2p-interface.md | 79 ++++++------- specs/phase0/p2p-interface.md | 186 +++++++++++++++---------------- 6 files changed, 199 insertions(+), 266 deletions(-) delete mode 100644 docs/README.md diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 9f2528263..000000000 --- a/docs/README.md +++ /dev/null @@ -1,70 +0,0 @@ -# Ethereum Proof-of-Stake Consensus Specifications - -[![Join the chat at https://discord.gg/qGpsxSA](https://img.shields.io/badge/chat-on%20discord-blue.svg)](https://discord.gg/qGpsxSA) [![Join the chat at https://gitter.im/ethereum/sharding](https://badges.gitter.im/ethereum/sharding.svg)](https://gitter.im/ethereum/sharding?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) - -To learn more about proof-of-stake and sharding, see the [PoS documentation](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/), [sharding documentation](https://ethereum.org/en/upgrades/sharding/) and the [research compendium](https://notes.ethereum.org/s/H1PGqDhpm). - -This repository hosts the current Ethereum proof-of-stake specifications. Discussions about design rationale and proposed changes can be brought up and discussed as issues. Solidified, agreed-upon changes to the spec can be made through pull requests. - -## Specs - -[![GitHub release](https://img.shields.io/github/v/release/ethereum/eth2.0-specs)](https://github.com/ethereum/eth2.0-specs/releases/) [![PyPI version](https://badge.fury.io/py/eth2spec.svg)](https://badge.fury.io/py/eth2spec) - -Core specifications for Ethereum proof-of-stake clients can be found in [specs](specs/). These are divided into features. -Features are researched and developed in parallel, and then consolidated into sequential upgrades when ready. - -### Stable Specifications - -| Seq. | Code Name | Fork Epoch | Specs | -| - | - | - | - | -| 0 | **Phase0** |`0` |
  • Core
    • [The beacon chain](specs/phase0/beacon-chain.md)
    • [Deposit contract](specs/phase0/deposit-contract.md)
    • [Beacon chain fork choice](specs/phase0/fork-choice.md)
  • Additions
    • [Honest validator guide](specs/phase0/validator.md)
    • [P2P networking](specs/phase0/p2p-interface.md)
    • [Weak subjectivity](specs/phase0/weak-subjectivity.md)
| -| 1 | **Altair** | `74240` |
  • Core
    • [Beacon chain changes](specs/altair/beacon-chain.md)
    • [Altair fork](specs/altair/fork.md)
  • Additions
    • [Light client sync protocol](specs/altair/light-client/sync-protocol.md) ([full node](specs/altair/light-client/full-node.md), [light client](specs/altair/light-client/light-client.md), [networking](specs/altair/light-client/p2p-interface.md))
    • [Honest validator guide changes](specs/altair/validator.md)
    • [P2P networking](specs/altair/p2p-interface.md)
| -| 2 | **Bellatrix**
(["The Merge"](https://ethereum.org/en/upgrades/merge/)) | `144896` |
  • Core
    • [Beacon Chain changes](specs/bellatrix/beacon-chain.md)
    • [Bellatrix fork](specs/bellatrix/fork.md)
    • [Fork choice changes](specs/bellatrix/fork-choice.md)
  • Additions
    • [Honest validator guide changes](specs/bellatrix/validator.md)
    • [P2P networking](specs/bellatrix/p2p-interface.md)
| -| 3 | **Capella** | `194048` |
  • Core
    • [Beacon chain changes](specs/capella/beacon-chain.md)
    • [Capella fork](specs/capella/fork.md)
  • Additions
    • [Light client sync protocol changes](specs/capella/light-client/sync-protocol.md) ([fork](specs/capella/light-client/fork.md), [full node](specs/capella/light-client/full-node.md), [networking](specs/capella/light-client/p2p-interface.md))
    • [Validator additions](specs/capella/validator.md)
    • [P2P networking](specs/capella/p2p-interface.md)
| - -### In-development Specifications -| Code Name or Topic | Specs | Notes | -| - | - | - | -| Deneb (tentative) |
  • Core
    • [Beacon Chain changes](specs/deneb/beacon-chain.md)
    • [Deneb fork](specs/deneb/fork.md)
    • [Polynomial commitments](specs/deneb/polynomial-commitments.md)
    • [Fork choice changes](specs/deneb/fork-choice.md)
  • Additions
    • [Light client sync protocol changes](specs/deneb/light-client/sync-protocol.md) ([fork](specs/deneb/light-client/fork.md), [full node](specs/deneb/light-client/full-node.md), [networking](specs/deneb/light-client/p2p-interface.md))
    • [Honest validator guide changes](specs/deneb/validator.md)
    • [P2P networking](specs/deneb/p2p-interface.md)
| -| Sharding (outdated) |
  • Core
    • [Beacon Chain changes](specs/_features/sharding/beacon-chain.md)
  • Additions
    • [P2P networking](specs/_features/sharding/p2p-interface.md)
| -| Custody Game (outdated) |
  • Core
    • [Beacon Chain changes](specs/_features/custody_game/beacon-chain.md)
  • Additions
    • [Honest validator guide changes](specs/_features/custody_game/validator.md)
| Dependent on sharding | -| Data Availability Sampling (outdated) |
  • Core
    • [Core types and functions](specs/_features/das/das-core.md)
    • [Fork choice changes](specs/_features/das/fork-choice.md)
  • Additions
    • [P2P Networking](specs/_features/das/p2p-interface.md)
    • [Sampling process](specs/_features/das/sampling.md)
|
  • Dependent on sharding
  • [Technical explainer](https://hackmd.io/@HWeNw8hNRimMm2m2GH56Cw/B1YJPGkpD)
| -| EIP-6110 |
  • Core
    • [Beacon Chain changes](specs/_features/eip6110//beacon-chain.md)
    • [EIP-6110 fork](specs/_features/eip6110/fork.md)
  • Additions
    • [Honest validator guide changes](specs/_features/eip6110/validator.md)
| - -### Accompanying documents can be found in [specs](specs) and include: - -* [SimpleSerialize (SSZ) spec](ssz/simple-serialize.md) -* [Merkle proof formats](ssz/merkle-proofs.md) -* [General test format](tests/formats/README.md) - -## Additional specifications for client implementers - -Additional specifications and standards outside of requisite client functionality can be found in the following repos: - -* [Beacon APIs](https://github.com/ethereum/beacon-apis) -* [Beacon Metrics](https://github.com/ethereum/beacon-metrics/) - -## Design goals - -The following are the broad design goals for the Ethereum proof-of-stake consensus specifications: -* to minimize complexity, even at the cost of some losses in efficiency -* to remain live through major network partitions and when very large portions of nodes go offline -* to select all components such that they are either quantum secure or can be easily swapped out for quantum secure counterparts when available -* to utilize crypto and design techniques that allow for a large participation of validators in total and per unit time -* to allow for a typical consumer laptop with `O(C)` resources to process/validate `O(1)` shards (including any system level validation such as the beacon chain) - -## Useful external resources - -* [Design Rationale](https://notes.ethereum.org/s/rkhCgQteN#) -* [Phase 0 Onboarding Document](https://notes.ethereum.org/s/Bkn3zpwxB) -* [Combining GHOST and Casper paper](https://arxiv.org/abs/2003.03052) - -## For spec contributors - -Documentation on the different components used during spec writing can be found here: -* [YAML Test Generators](tests/generators/README.md) -* [Executable Python Spec, with Py-tests](tests/core/pyspec/README.md) - -## Consensus spec tests - -Conformance tests built from the executable python spec are available in the [Ethereum Proof-of-Stake Consensus Spec Tests](https://github.com/ethereum/consensus-spec-tests) repo. Compressed tarballs are available in [releases](https://github.com/ethereum/consensus-spec-tests/releases). diff --git a/specs/altair/p2p-interface.md b/specs/altair/p2p-interface.md index 8d6b1c433..0f278b08c 100644 --- a/specs/altair/p2p-interface.md +++ b/specs/altair/p2p-interface.md @@ -13,7 +13,7 @@ Altair adds new messages, topics and data to the Req-Resp, Gossip and Discovery - - [Warning](#warning) +- [Warning](#warning) - [Modifications in Altair](#modifications-in-altair) - [MetaData](#metadata) - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) @@ -43,9 +43,9 @@ Altair adds new messages, topics and data to the Req-Resp, Gossip and Discovery This document is currently illustrative for early Altair testnets and some parts are subject to change. Refer to the note in the [validator guide](./validator.md) for further details. -# Modifications in Altair +## Modifications in Altair -## MetaData +### MetaData The `MetaData` stored locally by clients is updated with an additional field to communicate the sync committee subnet subscriptions: @@ -62,12 +62,12 @@ Where - `seq_number` and `attnets` have the same meaning defined in the Phase 0 document. - `syncnets` is a `Bitvector` representing the node's sync committee subnet subscriptions. This field should mirror the data in the node's ENR as outlined in the [validator guide](./validator.md#sync-committee-subnet-stability). -## The gossip domain: gossipsub +### The gossip domain: gossipsub Gossip meshes are added in Altair to support the consensus activities of the sync committees. Validators use an aggregation scheme to balance the processing and networking load across all of the relevant actors. -### Topics and messages +#### Topics and messages Topics follow the same specification as in the Phase 0 document. New topics are added in Altair to support the sync committees and the beacon block topic is updated with the modified type. @@ -103,11 +103,11 @@ Definitions of these new types can be found in the [Altair validator guide](./va Note that the `ForkDigestValue` path segment of the topic separates the old and the new `beacon_block` topics. -#### Global topics +##### Global topics Altair changes the type of the global beacon block topic and adds one global topic to propagate partially aggregated sync committee messages to all potential proposers of beacon blocks. -##### `beacon_block` +###### `beacon_block` The existing specification for this topic does not change from the Phase 0 document, but the type of the payload does change to the (modified) `SignedBeaconBlock`. @@ -115,7 +115,7 @@ This type changes due to the inclusion of the inner `BeaconBlockBody` that is mo See the [state transition document](./beacon-chain.md#beaconblockbody) for Altair for further details. -##### `sync_committee_contribution_and_proof` +###### `sync_committee_contribution_and_proof` This topic is used to propagate partially aggregated sync committee messages to be included in future blocks. @@ -152,11 +152,11 @@ def get_sync_subcommittee_pubkeys(state: BeaconState, subcommittee_index: uint64 - _[REJECT]_ The aggregator signature, `signed_contribution_and_proof.signature`, is valid. - _[REJECT]_ The aggregate signature is valid for the message `beacon_block_root` and aggregate pubkey derived from the participation info in `aggregation_bits` for the subcommittee specified by the `contribution.subcommittee_index`. -#### Sync committee subnets +##### Sync committee subnets Sync committee subnets are used to propagate unaggregated sync committee messages to subsections of the network. -##### `sync_committee_{subnet_id}` +###### `sync_committee_{subnet_id}` The `sync_committee_{subnet_id}` topics are used to propagate unaggregated sync committee messages to the subnet `subnet_id` to be aggregated before being gossiped to the global `sync_committee_contribution_and_proof` topic. @@ -170,7 +170,7 @@ The following validations MUST pass before forwarding the `sync_committee_messag Note this validation is _per topic_ so that for a given `slot`, multiple messages could be forwarded with the same `validator_index` as long as the `subnet_id`s are distinct. - _[REJECT]_ The `signature` is valid for the message `beacon_block_root` for the validator referenced by `validator_index`. -#### Sync committees and aggregation +##### Sync committees and aggregation The aggregation scheme closely follows the design of the attestation aggregation scheme. Sync committee messages are broadcast into "subnets" defined by a topic. @@ -182,7 +182,7 @@ Unaggregated messages (along with metadata) are sent as `SyncCommitteeMessage`s Aggregated sync committee messages are packaged into (signed) `SyncCommitteeContribution` along with proofs and gossiped to the `sync_committee_contribution_and_proof` topic. -### Transitioning the gossip +#### Transitioning the gossip With any fork, the fork version, and thus the `ForkDigestValue`, change. Message types are unique per topic, and so for a smooth transition a node must temporarily subscribe to both the old and new topics. @@ -205,9 +205,9 @@ Post-fork: E.g. an attestation on the both the old and new topic is ignored like any duplicate. - Two epochs after the fork, pre-fork topics SHOULD be unsubscribed from. This is well after the configured `seen_ttl`. -## The Req/Resp domain +### The Req/Resp domain -### Req-Resp interaction +#### Req-Resp interaction An additional `` field is introduced to the `response_chunk` as defined in the Phase 0 document: @@ -221,7 +221,7 @@ On a non-zero `` with `ErrorMessage` payload, the `` is a In Altair and later forks, `` functions as a short meta-data, defined per req-resp method, and can parametrize the payload decoder. -#### `ForkDigest`-context +##### `ForkDigest`-context Starting with Altair, and in future forks, SSZ type definitions may change. For this common case, we define the `ForkDigest`-context: @@ -229,9 +229,9 @@ For this common case, we define the `ForkDigest`-context: A fixed-width 4 byte ``, set to the `ForkDigest` matching the chunk: `compute_fork_digest(fork_version, genesis_validators_root)`. -### Messages +#### Messages -#### BeaconBlocksByRange v2 +##### BeaconBlocksByRange v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/2/` @@ -246,7 +246,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `GENESIS_FORK_VERSION` | `phase0.SignedBeaconBlock` | | `ALTAIR_FORK_VERSION` | `altair.SignedBeaconBlock` | -#### BeaconBlocksByRoot v2 +##### BeaconBlocksByRoot v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` @@ -261,7 +261,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `GENESIS_FORK_VERSION` | `phase0.SignedBeaconBlock` | | `ALTAIR_FORK_VERSION` | `altair.SignedBeaconBlock` | -#### GetMetaData v2 +##### GetMetaData v2 **Protocol ID:** `/eth2/beacon_chain/req/metadata/2/` @@ -279,7 +279,7 @@ Requests the MetaData of a peer, using the new `MetaData` definition given above that is extended from phase 0 in Altair. Other conditions for the `GetMetaData` protocol are unchanged from the phase 0 p2p networking document. -### Transitioning from v1 to v2 +#### Transitioning from v1 to v2 In advance of the fork, implementations can opt in to both run the v1 and v2 for a smooth transition. This is non-breaking, and is recommended as soon as the fork specification is stable. @@ -291,7 +291,7 @@ The v1 method MAY be unregistered at the fork boundary. In the event of a request on v1 for an Altair specific payload, the responder MUST return the **InvalidRequest** response code. -## The discovery domain: discv5 +### The discovery domain: discv5 The `attnets` key of the ENR is used as defined in the Phase 0 document. diff --git a/specs/bellatrix/p2p-interface.md b/specs/bellatrix/p2p-interface.md index 4d4044689..b8b3a11d6 100644 --- a/specs/bellatrix/p2p-interface.md +++ b/specs/bellatrix/p2p-interface.md @@ -13,23 +13,23 @@ Readers should understand the Phase 0 and Altair documents and use them as a bas - [Warning](#warning) -- [Modifications in Bellatrix](#modifications-in-bellatrix) - - [Configuration](#configuration) - - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - - [Topics and messages](#topics-and-messages) - - [Global topics](#global-topics) - - [`beacon_block`](#beacon_block) - - [Transitioning the gossip](#transitioning-the-gossip) - - [The Req/Resp domain](#the-reqresp-domain) - - [Messages](#messages) - - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) - - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) + - [Modifications in Bellatrix](#modifications-in-bellatrix) + - [Configuration](#configuration) + - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) + - [Topics and messages](#topics-and-messages) + - [Global topics](#global-topics) + - [`beacon_block`](#beacon_block) + - [Transitioning the gossip](#transitioning-the-gossip) + - [The Req/Resp domain](#the-reqresp-domain) + - [Messages](#messages) + - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) + - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) - [Design decision rationale](#design-decision-rationale) - - [Gossipsub](#gossipsub) - - [Why was the max gossip message size increased at Bellatrix?](#why-was-the-max-gossip-message-size-increased-at-bellatrix) - - [Req/Resp](#reqresp) - - [Why was the max chunk response size increased at Bellatrix?](#why-was-the-max-chunk-response-size-increased-at-bellatrix) - - [Why allow invalid payloads on the P2P network?](#why-allow-invalid-payloads-on-the-p2p-network) + - [Gossipsub](#gossipsub) + - [Why was the max gossip message size increased at Bellatrix?](#why-was-the-max-gossip-message-size-increased-at-bellatrix) + - [Req/Resp](#reqresp) + - [Why was the max chunk response size increased at Bellatrix?](#why-was-the-max-chunk-response-size-increased-at-bellatrix) + - [Why allow invalid payloads on the P2P network?](#why-allow-invalid-payloads-on-the-p2p-network) @@ -39,9 +39,9 @@ Readers should understand the Phase 0 and Altair documents and use them as a bas This document is currently illustrative for early Bellatrix testnets and some parts are subject to change. Refer to the note in the [validator guide](./validator.md) for further details. -# Modifications in Bellatrix +## Modifications in Bellatrix -## Configuration +### Configuration This section outlines modifications constants that are used in this spec. @@ -50,11 +50,11 @@ This section outlines modifications constants that are used in this spec. | `GOSSIP_MAX_SIZE_BELLATRIX` | `10 * 2**20` (= 10,485,760, 10 MiB) | The maximum allowed size of uncompressed gossip messages starting at Bellatrix upgrade. | | `MAX_CHUNK_SIZE_BELLATRIX` | `10 * 2**20` (= 10,485,760, 10 MiB) | The maximum allowed size of uncompressed req/resp chunked responses starting at Bellatrix upgrade. | -## The gossip domain: gossipsub +### The gossip domain: gossipsub Some gossip meshes are upgraded in Bellatrix to support upgraded types. -### Topics and messages +#### Topics and messages Topics follow the same specification as in prior upgrades. All topics remain stable except the beacon block topic which is updated with the modified type. @@ -76,11 +76,11 @@ The new topics along with the type of the `data` field of a gossipsub message ar Note that the `ForkDigestValue` path segment of the topic separates the old and the new `beacon_block` topics. -#### Global topics +##### Global topics Bellatrix changes the type of the global beacon block topic. -##### `beacon_block` +###### `beacon_block` The *type* of the payload of this topic changes to the (modified) `SignedBeaconBlock` found in Bellatrix. Specifically, this type changes with the addition of `execution_payload` to the inner `BeaconBlockBody`. @@ -107,12 +107,12 @@ Alias `block = signed_beacon_block.message`, `execution_payload = block.body.exe The following gossip validation from prior specifications MUST NOT be applied if the execution is enabled for the block -- i.e. `is_execution_enabled(state, block.body)`: - [REJECT] The block's parent (defined by `block.parent_root`) passes validation. -### Transitioning the gossip +#### Transitioning the gossip See gossip transition details found in the [Altair document](../altair/p2p-interface.md#transitioning-the-gossip) for details on how to handle transitioning gossip topics. -## The Req/Resp domain +### The Req/Resp domain Non-faulty, [optimistic](/sync/optimistic.md) nodes may send blocks which result in an INVALID response from an execution engine. To prevent network @@ -122,9 +122,9 @@ down-scored or disconnected. Transmission of a block which is invalid due to any consensus layer rules (i.e., *not* execution layer rules) MAY result in down-scoring or disconnection. -### Messages +#### Messages -#### BeaconBlocksByRange v2 +##### BeaconBlocksByRange v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/2/` @@ -146,7 +146,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `ALTAIR_FORK_VERSION` | `altair.SignedBeaconBlock` | | `BELLATRIX_FORK_VERSION` | `bellatrix.SignedBeaconBlock` | -#### BeaconBlocksByRoot v2 +##### BeaconBlocksByRoot v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` @@ -165,9 +165,9 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: # Design decision rationale -## Gossipsub +### Gossipsub -### Why was the max gossip message size increased at Bellatrix? +#### Why was the max gossip message size increased at Bellatrix? With the addition of `ExecutionPayload` to `BeaconBlock`s, there is a dynamic field -- `transactions` -- which can validly exceed the `GOSSIP_MAX_SIZE` limit (1 MiB) put in @@ -190,9 +190,9 @@ order of 128 KiB in the worst case and the current gas limit (~30M) bounds max b than 2 MiB today, this marginal difference in theoretical bounds will have zero impact on network functionality and security. -## Req/Resp +### Req/Resp -### Why was the max chunk response size increased at Bellatrix? +#### Why was the max chunk response size increased at Bellatrix? Similar to the discussion about the maximum gossip size increase, the `ExecutionPayload` type can cause `BeaconBlock`s to exceed the 1 MiB bounds put @@ -204,7 +204,7 @@ valid block sizes in the range of gas limits expected in the medium term. As with both gossip and req/rsp maximum values, type-specific limits should always by simultaneously respected. -### Why allow invalid payloads on the P2P network? +#### Why allow invalid payloads on the P2P network? The specification allows blocks with invalid execution payloads to propagate across gossip and via RPC calls. The reasoning for this is as follows: diff --git a/specs/capella/p2p-interface.md b/specs/capella/p2p-interface.md index 834fd44d8..a71b6479f 100644 --- a/specs/capella/p2p-interface.md +++ b/specs/capella/p2p-interface.md @@ -4,7 +4,7 @@ This document contains the networking specification for Capella. The specification of these changes continues in the same format as the network specifications of previous upgrades, and assumes them as pre-requisite. -## Table of contents +### Table of contents @@ -26,13 +26,13 @@ The specification of these changes continues in the same format as the network s -# Modifications in Capella +## Modifications in Capella -## The gossip domain: gossipsub +### The gossip domain: gossipsub A new topic is added to support the gossip of withdrawal credential change messages. And an existing topic is upgraded for updated types in Capella. -### Topics and messages +#### Topics and messages Topics follow the same specification as in prior upgrades. All existing topics remain stable except the beacon block topic which is updated with the modified type. @@ -45,17 +45,17 @@ The new topics along with the type of the `data` field of a gossipsub message ar Note that the `ForkDigestValue` path segment of the topic separates the old and the new `beacon_block` topics. -#### Global topics +##### Global topics Capella changes the type of the global beacon block topic and adds one global topic to propagate withdrawal credential change messages to all potential proposers of beacon blocks. -##### `beacon_block` +###### `beacon_block` The *type* of the payload of this topic changes to the (modified) `SignedBeaconBlock` found in Capella. Specifically, this type changes with the addition of `bls_to_execution_changes` to the inner `BeaconBlockBody`. See Capella [state transition document](./beacon-chain.md#beaconblockbody) for further details. -##### `bls_to_execution_change` +###### `bls_to_execution_change` This topic is used to propagate signed bls to execution change messages to be included in future blocks. @@ -67,16 +67,16 @@ The following validations MUST pass before forwarding the `signed_bls_to_executi for the validator with index `signed_bls_to_execution_change.message.validator_index`. - _[REJECT]_ All of the conditions within `process_bls_to_execution_change` pass validation. -### Transitioning the gossip +#### Transitioning the gossip See gossip transition details found in the [Altair document](../altair/p2p-interface.md#transitioning-the-gossip) for details on how to handle transitioning gossip topics for Capella. -## The Req/Resp domain +### The Req/Resp domain -### Messages +#### Messages -#### BeaconBlocksByRange v2 +##### BeaconBlocksByRange v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/2/` @@ -93,7 +93,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: | `BELLATRIX_FORK_VERSION` | `bellatrix.SignedBeaconBlock` | | `CAPELLA_FORK_VERSION` | `capella.SignedBeaconBlock` | -#### BeaconBlocksByRoot v2 +##### BeaconBlocksByRoot v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` diff --git a/specs/deneb/p2p-interface.md b/specs/deneb/p2p-interface.md index 0b6381e20..3c6f3c88a 100644 --- a/specs/deneb/p2p-interface.md +++ b/specs/deneb/p2p-interface.md @@ -10,32 +10,35 @@ The specification of these changes continues in the same format as the network s -- [Configuration](#configuration) -- [Containers](#containers) - - [`BlobSidecar`](#blobsidecar) - - [`SignedBlobSidecar`](#signedblobsidecar) - - [`BlobIdentifier`](#blobidentifier) - - [Helpers](#helpers) - - [`verify_blob_sidecar_signature`](#verify_blob_sidecar_signature) -- [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) - - [Topics and messages](#topics-and-messages) - - [Global topics](#global-topics) - - [`beacon_block`](#beacon_block) - - [`blob_sidecar_{index}`](#blob_sidecar_index) - - [Transitioning the gossip](#transitioning-the-gossip) -- [The Req/Resp domain](#the-reqresp-domain) - - [Messages](#messages) - - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) - - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) - - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) - - [BlobSidecarsByRange v1](#blobsidecarsbyrange-v1) +- [Modifications in Deneb](#modifications-in-deneb) + - [Configuration](#configuration) + - [Containers](#containers) + - [`BlobSidecar`](#blobsidecar) + - [`SignedBlobSidecar`](#signedblobsidecar) + - [`BlobIdentifier`](#blobidentifier) + - [Helpers](#helpers) + - [`verify_blob_sidecar_signature`](#verify_blob_sidecar_signature) + - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) + - [Topics and messages](#topics-and-messages) + - [Global topics](#global-topics) + - [`beacon_block`](#beacon_block) + - [`blob_sidecar_{index}`](#blob_sidecar_index) + - [Transitioning the gossip](#transitioning-the-gossip) + - [The Req/Resp domain](#the-reqresp-domain) + - [Messages](#messages) + - [BeaconBlocksByRange v2](#beaconblocksbyrange-v2) + - [BeaconBlocksByRoot v2](#beaconblocksbyroot-v2) + - [BlobSidecarsByRoot v1](#blobsidecarsbyroot-v1) + - [BlobSidecarsByRange v1](#blobsidecarsbyrange-v1) - [Design decision rationale](#design-decision-rationale) - [Why are blobs relayed as a sidecar, separate from beacon blocks?](#why-are-blobs-relayed-as-a-sidecar-separate-from-beacon-blocks) -## Configuration +## Modifications in Deneb + +### Configuration | Name | Value | Description | |------------------------------------------|-----------------------------------|---------------------------------------------------------------------| @@ -43,9 +46,9 @@ The specification of these changes continues in the same format as the network s | `MAX_REQUEST_BLOB_SIDECARS` | `MAX_REQUEST_BLOCKS_DENEB * MAX_BLOBS_PER_BLOCK` | Maximum number of blob sidecars in a single request | | `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` | `2**12` (= 4096 epochs, ~18 days) | The minimum epoch range over which a node must serve blob sidecars | -## Containers +### Containers -### `BlobSidecar` +#### `BlobSidecar` ```python class BlobSidecar(Container): @@ -59,7 +62,7 @@ class BlobSidecar(Container): kzg_proof: KZGProof # Allows for quick verification of kzg_commitment ``` -### `SignedBlobSidecar` +#### `SignedBlobSidecar` ```python class SignedBlobSidecar(Container): @@ -67,7 +70,7 @@ class SignedBlobSidecar(Container): signature: BLSSignature ``` -### `BlobIdentifier` +#### `BlobIdentifier` ```python class BlobIdentifier(Container): @@ -75,9 +78,9 @@ class BlobIdentifier(Container): index: BlobIndex ``` -### Helpers +#### Helpers -#### `verify_blob_sidecar_signature` +##### `verify_blob_sidecar_signature` ```python def verify_blob_sidecar_signature(state: BeaconState, signed_blob_sidecar: SignedBlobSidecar) -> bool: @@ -86,11 +89,11 @@ def verify_blob_sidecar_signature(state: BeaconState, signed_blob_sidecar: Signe return bls.Verify(proposer.pubkey, signing_root, signed_blob_sidecar.signature) ``` -## The gossip domain: gossipsub +### The gossip domain: gossipsub Some gossip meshes are upgraded in the fork of Deneb to support upgraded types. -### Topics and messages +#### Topics and messages Topics follow the same specification as in prior upgrades. @@ -106,15 +109,15 @@ The new topics along with the type of the `data` field of a gossipsub message ar | - | - | | `blob_sidecar_{index}` | `SignedBlobSidecar` (new) | -#### Global topics +##### Global topics Deneb introduces new global topics for blob sidecars. -##### `beacon_block` +###### `beacon_block` The *type* of the payload of this topic changes to the (modified) `SignedBeaconBlock` found in deneb. -##### `blob_sidecar_{index}` +###### `blob_sidecar_{index}` This topic is used to propagate signed blob sidecars, one for each sidecar index. The number of indices is defined by `MAX_BLOBS_PER_BLOCK`. @@ -132,16 +135,16 @@ The following validations MUST pass before forwarding the `signed_blob_sidecar` If the `proposer_index` cannot immediately be verified against the expected shuffling, the sidecar MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. -### Transitioning the gossip +#### Transitioning the gossip See gossip transition details found in the [Altair document](../altair/p2p-interface.md#transitioning-the-gossip) for details on how to handle transitioning gossip topics for this upgrade. -## The Req/Resp domain +### The Req/Resp domain -### Messages +#### Messages -#### BeaconBlocksByRange v2 +##### BeaconBlocksByRange v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/2/` @@ -161,7 +164,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. -#### BeaconBlocksByRoot v2 +##### BeaconBlocksByRoot v2 **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/2/` @@ -179,7 +182,7 @@ Per `context = compute_fork_digest(fork_version, genesis_validators_root)`: No more than `MAX_REQUEST_BLOCKS_DENEB` may be requested at a time. -#### BlobSidecarsByRoot v1 +##### BlobSidecarsByRoot v1 **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_root/1/` @@ -228,7 +231,7 @@ Clients MUST support requesting sidecars since `minimum_request_epoch`, where `m Clients MUST respond with at least one sidecar, if they have it. Clients MAY limit the number of blocks and sidecars in the response. -#### BlobSidecarsByRange v1 +##### BlobSidecarsByRange v1 **Protocol ID:** `/eth2/beacon_chain/req/blob_sidecars_by_range/1/` diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index f52752931..f38601ed8 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -111,11 +111,11 @@ It consists of four main sections: -# Network fundamentals +## Network fundamentals This section outlines the specification for the networking stack in Ethereum consensus-layer clients. -## Transport +### Transport Even though libp2p is a multi-transport stack (designed to listen on multiple simultaneous transports and endpoints transparently), we hereby define a profile for basic interoperability. @@ -133,14 +133,14 @@ All listening endpoints must be publicly dialable, and thus not rely on libp2p c Nodes operating behind a NAT, or otherwise undialable by default (e.g. container runtime, firewall, etc.), MUST have their infrastructure configured to enable inbound traffic on the announced public listening endpoint. -## Encryption and identification +### Encryption and identification The [Libp2p-noise](https://github.com/libp2p/specs/tree/master/noise) secure channel handshake with `secp256k1` identities will be used for encryption. As specified in the libp2p specification, clients MUST support the `XX` handshake pattern. -## Protocol Negotiation +### Protocol Negotiation Clients MUST use exact equality when negotiating protocol versions to use and MAY use the version to give priority to higher version numbers. @@ -148,7 +148,7 @@ Clients MUST support [multistream-select 1.0](https://github.com/multiformats/mu and MAY support [multiselect 2.0](https://github.com/libp2p/specs/pull/95) when the spec solidifies. Once all clients have implementations for multiselect 2.0, multistream-select 1.0 MAY be phased out. -## Multiplexing +### Multiplexing During connection bootstrapping, libp2p dynamically negotiates a mutually supported multiplexing method to conduct parallel conversations. This applies to transports that are natively incapable of multiplexing (e.g. TCP, WebSockets, WebRTC), @@ -163,9 +163,9 @@ and MAY support [yamux](https://github.com/hashicorp/yamux/blob/master/spec.md). If both are supported by the client, yamux MUST take precedence during negotiation. See the [Rationale](#design-decision-rationale) section below for tradeoffs. -# Consensus-layer network interaction domains +## Consensus-layer network interaction domains -## Configuration +### Configuration This section outlines constants that are used in this spec. @@ -182,7 +182,7 @@ This section outlines constants that are used in this spec. | `MESSAGE_DOMAIN_INVALID_SNAPPY` | `0x00000000` | 4-byte domain for gossip message-id isolation of *invalid* snappy messages | | `MESSAGE_DOMAIN_VALID_SNAPPY` | `0x01000000` | 4-byte domain for gossip message-id isolation of *valid* snappy messages | -## MetaData +### MetaData Clients MUST locally store the following `MetaData`: @@ -203,7 +203,7 @@ Where is entirely independent of the ENR sequence number, and will in most cases be out of sync with the ENR sequence number. -## The gossip domain: gossipsub +### The gossip domain: gossipsub Clients MUST support the [gossipsub v1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.0.md) libp2p Protocol including the [gossipsub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md) extension. @@ -229,7 +229,7 @@ The following gossipsub [parameters](https://github.com/libp2p/specs/blob/master for peer scoring and other attack mitigations. These are currently under investigation and will be spec'd and released to mainnet when they are ready. -### Topics and messages +#### Topics and messages Topics are plain UTF-8 strings and are encoded on the wire as determined by protobuf (gossipsub messages are enveloped in protobuf messages). Topic strings have form: `/eth2/ForkDigestValue/Name/Encoding`. @@ -289,7 +289,7 @@ We utilize `ACCEPT`, `REJECT`, and `IGNORE`. For each gossipsub topic, there are If all validations pass, return `ACCEPT`. If one or more validations fail while processing the items in order, return either `REJECT` or `IGNORE` as specified in the prefix of the particular condition. -#### Global topics +##### Global topics There are two primary global topics used to propagate beacon blocks (`beacon_block`) and aggregate attestations (`beacon_aggregate_and_proof`) to all nodes on the network. @@ -297,7 +297,7 @@ and aggregate attestations (`beacon_aggregate_and_proof`) to all nodes on the ne There are three additional global topics that are used to propagate lower frequency validator messages (`voluntary_exit`, `proposer_slashing`, and `attester_slashing`). -##### `beacon_block` +###### `beacon_block` The `beacon_block` topic is used solely for propagating new signed beacon blocks to all nodes on the networks. Signed blocks are sent in their entirety. @@ -325,7 +325,7 @@ The following validations MUST pass before forwarding the `signed_beacon_block` the block MAY be queued for later processing while proposers for the block's branch are calculated -- in such a case _do not_ `REJECT`, instead `IGNORE` this message. -##### `beacon_aggregate_and_proof` +###### `beacon_aggregate_and_proof` The `beacon_aggregate_and_proof` topic is used to propagate aggregated attestations (as `SignedAggregateAndProof`s) to subscribing nodes (typically validators) to be included in future blocks. @@ -360,7 +360,7 @@ The following validations MUST pass before forwarding the `signed_aggregate_and_ == store.finalized_checkpoint.root` -##### `voluntary_exit` +###### `voluntary_exit` The `voluntary_exit` topic is used solely for propagating signed voluntary validator exits to proposers on the network. Signed voluntary exits are sent in their entirety. @@ -370,7 +370,7 @@ The following validations MUST pass before forwarding the `signed_voluntary_exit for the validator with index `signed_voluntary_exit.message.validator_index`. - _[REJECT]_ All of the conditions within `process_voluntary_exit` pass validation. -##### `proposer_slashing` +###### `proposer_slashing` The `proposer_slashing` topic is used solely for propagating proposer slashings to proposers on the network. Proposer slashings are sent in their entirety. @@ -380,7 +380,7 @@ The following validations MUST pass before forwarding the `proposer_slashing` on for the proposer with index `proposer_slashing.signed_header_1.message.proposer_index`. - _[REJECT]_ All of the conditions within `process_proposer_slashing` pass validation. -##### `attester_slashing` +###### `attester_slashing` The `attester_slashing` topic is used solely for propagating attester slashings to proposers on the network. Attester slashings are sent in their entirety. @@ -392,11 +392,11 @@ Clients who receive an attester slashing on this topic MUST validate the conditi verify if `any(attester_slashed_indices.difference(prior_seen_attester_slashed_indices))`). - _[REJECT]_ All of the conditions within `process_attester_slashing` pass validation. -#### Attestation subnets +##### Attestation subnets Attestation subnets are used to propagate unaggregated attestations to subsections of the network. -##### `beacon_attestation_{subnet_id}` +###### `beacon_attestation_{subnet_id}` The `beacon_attestation_{subnet_id}` topics are used to propagate unaggregated attestations to the subnet `subnet_id` (typically beacon and persistent committees) to be aggregated before being gossiped to `beacon_aggregate_and_proof`. @@ -432,7 +432,7 @@ The following validations MUST pass before forwarding the `attestation` on the s -#### Attestations and Aggregation +##### Attestations and Aggregation Attestation broadcasting is grouped into subnets defined by a topic. The number of subnets is defined via `ATTESTATION_SUBNET_COUNT`. @@ -445,7 +445,7 @@ Unaggregated attestations are sent as `Attestation`s to the subnet topic, Aggregated attestations are sent to the `beacon_aggregate_and_proof` topic as `AggregateAndProof`s. -### Encodings +#### Encodings Topics are post-fixed with an encoding. Encodings define how the payload of a gossipsub message is encoded. @@ -461,9 +461,9 @@ so [basic snappy block compression](https://github.com/google/snappy/blob/master Implementations MUST use a single encoding for gossip. Changing an encoding will require coordination between participating implementations. -## The Req/Resp domain +### The Req/Resp domain -### Protocol identification +#### Protocol identification Each message type is segregated into its own libp2p protocol ID, which is a case-sensitive UTF-8 string of the form: @@ -485,7 +485,7 @@ With: This protocol segregation allows libp2p `multistream-select 1.0` / `multiselect 2.0` to handle the request type, version, and encoding negotiation before establishing the underlying streams. -### Req/Resp interaction +#### Req/Resp interaction We use ONE stream PER request/response interaction. Streams are closed when the interaction finishes, whether in success or in error. @@ -515,7 +515,7 @@ Regardless of these type specific bounds, a global maximum uncompressed byte siz Clients MUST ensure that lengths are within these bounds; if not, they SHOULD reset the stream immediately. Clients tracking peer reputation MAY decrement the score of the misbehaving peer under this circumstance. -#### Requesting side +##### Requesting side Once a new stream with the protocol ID for the request type has been negotiated, the full request message SHOULD be sent immediately. The request MUST be encoded according to the encoding strategy. @@ -537,7 +537,7 @@ A requester SHOULD read from the stream until either: For requests consisting of a single valid `response_chunk`, the requester SHOULD read the chunk fully, as defined by the `encoding-dependent-header`, before closing the stream. -#### Responding side +##### Responding side Once a new stream with the protocol ID for the request type has been negotiated, the responder SHOULD process the incoming request and MUST validate it before processing it. @@ -588,7 +588,7 @@ The `ErrorMessage` schema is: *Note*: By convention, the `error_message` is a sequence of bytes that MAY be interpreted as a UTF-8 string (for debugging purposes). Clients MUST treat as valid any byte sequences. -### Encoding strategies +#### Encoding strategies The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Only one value is possible at this time: @@ -599,7 +599,7 @@ Only one value is possible at this time: For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Root`'s. This encoding type MUST be supported by all clients. -#### SSZ-snappy encoding strategy +##### SSZ-snappy encoding strategy The [SimpleSerialize (SSZ) specification](../../ssz/simple-serialize.md) outlines how objects are SSZ-encoded. @@ -646,9 +646,9 @@ constituents individually as `response_chunk`s. For example, the `List[SignedBeaconBlock, ...]` response type sends zero or more `response_chunk`s. Each _successful_ `response_chunk` contains a single `SignedBeaconBlock` payload. -### Messages +#### Messages -#### Status +##### Status **Protocol ID:** ``/eth2/beacon_chain/req/status/1/`` @@ -694,7 +694,7 @@ SHOULD request beacon blocks from its counterparty via the `BeaconBlocksByRange` the client might need to send `Status` request again to learn if the peer has a higher head. Implementers are free to implement such behavior in their own way. -#### Goodbye +##### Goodbye **Protocol ID:** ``/eth2/beacon_chain/req/goodbye/1/`` @@ -718,7 +718,7 @@ The request/response MUST be encoded as a single SSZ-field. The response MUST consist of a single `response_chunk`. -#### BeaconBlocksByRange +##### BeaconBlocksByRange **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_range/1/` @@ -795,7 +795,7 @@ In particular when `step == 1`, each `parent_root` MUST match the `hash_tree_roo After the initial block, clients MAY stop in the process of responding if their fork choice changes the view of the chain in the context of the request. -#### BeaconBlocksByRoot +##### BeaconBlocksByRoot **Protocol ID:** `/eth2/beacon_chain/req/beacon_blocks_by_root/1/` @@ -835,7 +835,7 @@ Clients MAY limit the number of blocks in the response. `/eth2/beacon_chain/req/beacon_blocks_by_root/1/` is deprecated. Clients MAY respond with an empty list during the deprecation transition period. -#### Ping +##### Ping **Protocol ID:** `/eth2/beacon_chain/req/ping/1/` @@ -867,7 +867,7 @@ The request MUST be encoded as an SSZ-field. The response MUST consist of a single `response_chunk`. -#### GetMetaData +##### GetMetaData **Protocol ID:** `/eth2/beacon_chain/req/metadata/1/` @@ -890,14 +890,14 @@ The response MUST be encoded as an SSZ-container. The response MUST consist of a single `response_chunk`. -## The discovery domain: discv5 +### The discovery domain: discv5 Discovery Version 5 ([discv5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)) (Protocol version v5.1) is used for peer discovery. `discv5` is a standalone protocol, running on UDP on a dedicated port, meant for peer discovery only. `discv5` supports self-certified, flexible peer records (ENRs) and topic-based advertisement, both of which are (or will be) requirements in this context. -### Integration into libp2p stacks +#### Integration into libp2p stacks `discv5` SHOULD be integrated into the client’s libp2p stack by implementing an adaptor to make it conform to the [service discovery](https://github.com/libp2p/go-libp2p-core/blob/master/discovery/discovery.go) @@ -908,7 +908,7 @@ and the outputs will be multiaddrs converted from the ENR records returned by th This integration enables the libp2p stack to subsequently form connections and streams with discovered peers. -### ENR structure +#### ENR structure The Ethereum Node Record (ENR) for an Ethereum consensus client MUST contain the following entries (exclusive of the sequence number and signature, which MUST be present in an ENR): @@ -923,7 +923,7 @@ The ENR MAY contain the following entries: Specifications of these parameters can be found in the [ENR Specification](http://eips.ethereum.org/EIPS/eip-778). -#### Attestation subnet bitfield +##### Attestation subnet bitfield The ENR `attnets` entry signifies the attestation subnet bitfield with the following form to more easily discover peers participating in particular attestation gossip subnets. @@ -936,7 +936,7 @@ If a node's `MetaData.attnets` has any non-zero bit, the ENR MUST include the `a If a node's `MetaData.attnets` is composed of all zeros, the ENR MAY optionally include the `attnets` entry or leave it out entirely. -#### `eth2` field +##### `eth2` field ENRs MUST carry a generic `eth2` key with an 16-byte value of the node's current fork digest, next fork version, and next fork epoch to ensure connections are made with peers on the intended Ethereum network. @@ -979,11 +979,11 @@ Clients MAY connect to peers with the same `fork_digest` but a different `next_f Unless `ENRForkID` is manually updated to matching prior to the earlier `next_fork_epoch` of the two clients, these connecting clients will be unable to successfully interact starting at the earlier `next_fork_epoch`. -# Design decision rationale +## Design decision rationale -## Transport +### Transport -### Why are we defining specific transports? +#### Why are we defining specific transports? libp2p peers can listen on multiple transports concurrently, and these can change over time. Multiaddrs encode not only the address but also the transport to be used to dial. @@ -992,7 +992,7 @@ Due to this dynamic nature, agreeing on specific transports like TCP, QUIC, or W However, it is useful to define a minimum baseline for interoperability purposes. -### Can clients support other transports/handshakes than the ones mandated by the spec? +#### Can clients support other transports/handshakes than the ones mandated by the spec? Clients may support other transports such as libp2p QUIC, WebSockets, and WebRTC transports, if available in the language of choice. While interoperability shall not be harmed by lack of such support, the advantages are desirable: @@ -1007,7 +1007,7 @@ and the accompanying [QUIC-TLS document](https://tools.ietf.org/html/draft-ietf- The usage of one handshake procedure or the other shall be transparent to the application layer, once the libp2p Host/Node object has been configured appropriately. -### What are the advantages of using TCP/QUIC/Websockets? +#### What are the advantages of using TCP/QUIC/Websockets? TCP is a reliable, ordered, full-duplex, congestion-controlled network protocol that powers much of the Internet as we know it today. HTTP/1.1 and HTTP/2 run atop TCP. @@ -1027,7 +1027,7 @@ and we may only become subject to standard IP-based firewall filtering—somethi WebSockets and/or WebRTC transports are necessary for interaction with browsers, and will become increasingly important as we incorporate browser-based light clients to the Ethereum network. -### Why do we not just support a single transport? +#### Why do we not just support a single transport? Networks evolve. Hardcoding design decisions leads to ossification, preventing the evolution of networks alongside the state of the art. @@ -1039,7 +1039,7 @@ Clients can adopt new transports without breaking old ones, and the multi-transp (e.g. browsers, embedded devices) to interact with the network as first-class citizens via suitable/native transports (e.g. WSS), without the need for proxying or trust delegation to servers. -### Why are we not using QUIC from the start? +#### Why are we not using QUIC from the start? The QUIC standard is still not finalized (at working draft 22 at the time of writing), and not all mainstream runtimes/languages have mature, standard, and/or fully-interoperable [QUIC support](https://github.com/quicwg/base-drafts/wiki/Implementations). @@ -1052,9 +1052,9 @@ On the other hand, TLS 1.3 is the newest, simplified iteration of TLS. Old, insecure, obsolete ciphers and algorithms have been removed, adopting Ed25519 as the sole ECDH key agreement function. Handshakes are faster, 1-RTT data is supported, and session resumption is a reality, amongst other features. -## Multiplexing +### Multiplexing -### Why are we using mplex/yamux? +#### Why are we using mplex/yamux? [Yamux](https://github.com/hashicorp/yamux/blob/master/spec.md) is a multiplexer invented by Hashicorp that supports stream-level congestion control. Implementations exist in a limited set of languages, and it’s not a trivial piece to develop. @@ -1066,9 +1066,9 @@ It does not support stream-level congestion control and is subject to head-of-li Overlay multiplexers are not necessary with QUIC since the protocol provides native multiplexing, but they need to be layered atop TCP, WebSockets, and other transports that lack such support. -## Protocol Negotiation +### Protocol Negotiation -### When is multiselect 2.0 due and why do we plan to migrate to it? +#### When is multiselect 2.0 due and why do we plan to migrate to it? multiselect 2.0 is currently being conceptualized. The debate started [on this issue](https://github.com/libp2p/specs/pull/95), @@ -1084,7 +1084,7 @@ We plan to eventually migrate to multiselect 2.0 because it will: 3. Leverage *push data* mechanisms of underlying protocols to expedite negotiation. 4. Provide the building blocks for enhanced censorship resistance. -### What is the difference between connection-level and stream-level protocol negotiation? +#### What is the difference between connection-level and stream-level protocol negotiation? All libp2p connections must be authenticated, encrypted, and multiplexed. Connections using network transports unsupportive of native authentication/encryption and multiplexing (e.g. TCP) need to undergo protocol negotiation to agree on a mutually supported: @@ -1101,9 +1101,9 @@ When opening streams, peers pin a protocol to that stream, by conducting *stream At present, multistream-select 1.0 is used for both types of negotiation, but multiselect 2.0 will use dedicated mechanisms for connection bootstrapping process and stream protocol negotiation. -## Encryption +### Encryption -### Why are we not supporting SecIO? +#### Why are we not supporting SecIO? SecIO has been the default encryption layer for libp2p for years. It is used in IPFS and Filecoin. And although it will be superseded shortly, it is proven to work at scale. @@ -1114,7 +1114,7 @@ a mechanism that multiselect 2.0 will leverage to reduce round trips during conn SecIO is not considered secure for the purposes of this spec. -### Why are we using Noise? +#### Why are we using Noise? Copied from the Noise Protocol Framework [website](http://www.noiseprotocol.org): @@ -1129,7 +1129,7 @@ and are used in major cryptographic-centric projects like WireGuard, I2P, and Li [Various](https://www.wireguard.com/papers/kobeissi-bhargavan-noise-explorer-2018.pdf) [studies](https://eprint.iacr.org/2019/436.pdf) have assessed the stated security goals of several Noise handshakes with positive results. -### Why are we using encryption at all? +#### Why are we using encryption at all? Transport level encryption secures message exchange and provides properties that are useful for privacy, safety, and censorship resistance. These properties are derived from the following security guarantees that apply to the entire communication between two peers: @@ -1146,9 +1146,9 @@ Note that transport-level encryption is not exclusive of application-level encry Transport-level encryption secures the communication itself, while application-level cryptography is necessary for the application’s use cases (e.g. signatures, randomness, etc.). -## Gossipsub +### Gossipsub -### Why are we using a pub/sub algorithm for block and attestation propagation? +#### Why are we using a pub/sub algorithm for block and attestation propagation? Pubsub is a technique to broadcast/disseminate data across a network rapidly. Such data is packaged in fire-and-forget messages that do not require a response from every recipient. @@ -1156,18 +1156,18 @@ Peers subscribed to a topic participate in the propagation of messages in that t The alternative is to maintain a fully connected mesh (all peers connected to each other 1:1), which scales poorly (O(n^2)). -### Why are we using topics to segregate encodings, yet only support one encoding? +#### Why are we using topics to segregate encodings, yet only support one encoding? For future extensibility with almost zero overhead now (besides the extra bytes in the topic name). -### How do we upgrade gossip channels (e.g. changes in encoding, compression)? +#### How do we upgrade gossip channels (e.g. changes in encoding, compression)? Changing gossipsub/broadcasts requires a coordinated upgrade where all clients start publishing to the new topic together, during a hard fork. When a node is preparing for upcoming tasks (e.g. validator duty lookahead) on a gossipsub topic, the node should join the topic of the future epoch in which the task is to occur in addition to listening to the topics for the current epoch. -### Why must all clients use the same gossip topic instead of one negotiated between each peer pair? +#### Why must all clients use the same gossip topic instead of one negotiated between each peer pair? Supporting multiple topics/encodings would require the presence of relayers to translate between encodings and topics so as to avoid network fragmentation where participants have diverging views on the gossiped state, @@ -1182,7 +1182,7 @@ but the price here is pretty high in terms of overhead -- both computational and It is permitted for clients to publish data on alternative topics as long as they also publish on the network-wide mandatory topic. -### Why are the topics strings and not hashes? +#### Why are the topics strings and not hashes? Topic names have a hierarchical structure. In the future, gossipsub may support wildcard subscriptions @@ -1195,14 +1195,14 @@ since the domain is finite anyway, and calculating a digest's preimage would be Furthermore, the topic names are shorter than their digest equivalents (assuming SHA-256 hash), so hashing topics would bloat messages unnecessarily. -### Why are we using the `StrictNoSign` signature policy? +#### Why are we using the `StrictNoSign` signature policy? The policy omits the `from` (1), `seqno` (3), `signature` (5) and `key` (6) fields. These fields would: - Expose origin of sender (`from`), type of sender (based on `seqno`) - Add extra unused data to the gossip, since message IDs are based on `data`, not on the `from` and `seqno`. - Introduce more message validation than necessary, e.g. no `signature`. -### Why are we overriding the default libp2p pubsub `message-id`? +#### Why are we overriding the default libp2p pubsub `message-id`? For our current purposes, there is no need to address messages based on source peer, or track a message `seqno`. By overriding the default `message-id` to use content-addressing we can filter unnecessary duplicates before hitting the application layer. @@ -1214,7 +1214,7 @@ Some examples of where messages could be duplicated: Partial aggregates could be duplicated * Clients re-publishing seen messages -### Why are these specific gossip parameters chosen? +#### Why are these specific gossip parameters chosen? - `D`, `D_low`, `D_high`, `D_lazy`: recommended defaults. - `heartbeat_interval`: 0.7 seconds, recommended for the beacon chain in the [GossipSub evaluation report by Protocol Labs](https://gateway.ipfs.io/ipfs/QmRAFP5DBnvNjdYSbWhEhVRJJDFCLpPyvew5GwCCB4VxM4). @@ -1233,7 +1233,7 @@ Some examples of where messages could be duplicated: Attestation gossip validity is bounded by an epoch, so this is the safe max bound. -### Why is there `MAXIMUM_GOSSIP_CLOCK_DISPARITY` when validating slot ranges of messages in gossip subnets? +#### Why is there `MAXIMUM_GOSSIP_CLOCK_DISPARITY` when validating slot ranges of messages in gossip subnets? For some gossip channels (e.g. those for Attestations and BeaconBlocks), there are designated ranges of slots during which particular messages can be sent, @@ -1247,14 +1247,14 @@ For minimum and maximum allowable slot broadcast times, Although messages can at times be eagerly gossiped to the network, the node's fork choice prevents integration of these messages into the actual consensus until the _actual local start_ of the designated slot. -### Why are there `ATTESTATION_SUBNET_COUNT` attestation subnets? +#### Why are there `ATTESTATION_SUBNET_COUNT` attestation subnets? Depending on the number of validators, it may be more efficient to group shard subnets and might provide better stability for the gossipsub channel. The exact grouping will be dependent on more involved network tests. This constant allows for more flexibility in setting up the network topology for attestation aggregation (as aggregation should happen on each subnet). The value is currently set to be equal to `MAX_COMMITTEES_PER_SLOT` if/until network tests indicate otherwise. -### Why are attestations limited to be broadcast on gossip channels within `SLOTS_PER_EPOCH` slots? +#### Why are attestations limited to be broadcast on gossip channels within `SLOTS_PER_EPOCH` slots? Attestations can only be included on chain within an epoch's worth of slots so this is the natural cutoff. There is no utility to the chain to broadcast attestations older than one epoch, @@ -1265,7 +1265,7 @@ In addition to this, relaying attestations requires validating the attestation i Thus, validating arbitrarily old attestations would put additional requirements on which states need to be readily available to the node. This would result in a higher resource burden and could serve as a DoS vector. -### Why are aggregate attestations broadcast to the global topic as `AggregateAndProof`s rather than just as `Attestation`s? +#### Why are aggregate attestations broadcast to the global topic as `AggregateAndProof`s rather than just as `Attestation`s? The dominant strategy for an individual validator is to always broadcast an aggregate containing their own attestation to the global channel to ensure that proposers see their attestation for inclusion. @@ -1275,19 +1275,19 @@ the gossiped aggregate ensures that this dominant strategy will not flood the gl Also, an attacker can create any number of honest-looking aggregates and broadcast them to the global pubsub channel. Thus without some sort of proof of selection as an aggregator, the global channel can trivially be spammed. -### Why are we sending entire objects in the pubsub and not just hashes? +#### Why are we sending entire objects in the pubsub and not just hashes? Entire objects should be sent to get the greatest propagation speeds. If only hashes are sent, then block and attestation propagation is dependent on recursive requests from each peer. In a hash-only scenario, peers could receive hashes without knowing who to download the actual contents from. Sending entire objects ensures that they get propagated through the entire network. -### Should clients gossip blocks if they *cannot* validate the proposer signature due to not yet being synced, not knowing the head block, etc? +#### Should clients gossip blocks if they *cannot* validate the proposer signature due to not yet being synced, not knowing the head block, etc? The prohibition of unverified-block-gossiping extends to nodes that cannot verify a signature due to not being fully synced to ensure that such (amplified) DOS attacks are not possible. -### How are we going to discover peers in a gossipsub topic? +#### How are we going to discover peers in a gossipsub topic? In Phase 0, peers for attestation subnets will be found using the `attnets` entry in the ENR. @@ -1295,7 +1295,7 @@ Although this method will be sufficient for early upgrade of the beacon chain, w ENRs should ultimately not be used for this purpose. They are best suited to store identity, location, and capability information, rather than more volatile advertisements. -### How should fork version be used in practice? +#### How should fork version be used in practice? Fork versions are to be manually updated (likely via incrementing) at each hard fork. This is to provide native domain separation for signatures as well as to aid in usefulness for identitying peers (via ENRs) @@ -1308,9 +1308,9 @@ In these cases, extra care should be taken to isolate fork versions (e.g. flip a A node locally stores all previous and future planned fork versions along with the each fork epoch. This allows for handling sync and processing messages starting from past forks/epochs. -## Req/Resp +### Req/Resp -### Why segregate requests into dedicated protocol IDs? +#### Why segregate requests into dedicated protocol IDs? Requests are segregated by protocol ID to: @@ -1343,7 +1343,7 @@ Multiselect 2.0 will eventually remove this overhead by memoizing previously sel Fortunately, this req/resp protocol is not the expected network bottleneck in the protocol so the additional overhead is not expected to significantly hinder this domain. -### Why are messages length-prefixed with a protobuf varint in the SSZ-encoding? +#### Why are messages length-prefixed with a protobuf varint in the SSZ-encoding? We are using single-use streams where each stream is closed at the end of the message. Thus, libp2p transparently handles message delimiting in the underlying stream. @@ -1361,7 +1361,7 @@ Nevertheless, in the case of `ssz_snappy`, messages are still length-prefixed wi [Protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints) is an efficient technique to encode variable-length (unsigned here) ints. Instead of reserving a fixed-size field of as many bytes as necessary to convey the maximum possible value, this field is elastic in exchange for 1-bit overhead per byte. -### Why do we version protocol strings with ordinals instead of semver? +#### Why do we version protocol strings with ordinals instead of semver? Using semver for network protocols is confusing. It is never clear what a change in a field, even if backwards compatible on deserialization, actually implies. @@ -1382,11 +1382,11 @@ because it's unclear if "backwards compatibility" and "breaking change" apply on For this reason, we remove and replace semver with ordinals that require explicit agreement and do not mandate a specific policy for changes. -### Why is it called Req/Resp and not RPC? +#### Why is it called Req/Resp and not RPC? Req/Resp is used to avoid confusion with JSON-RPC and similar user-client interaction mechanisms. -### Why do we allow empty responses in block requests? +#### Why do we allow empty responses in block requests? When requesting blocks by range or root, it may happen that there are no blocks in the selected range or the responding node does not have the requested blocks. @@ -1413,7 +1413,7 @@ Failing to provide blocks that nodes "should" have is reason to trust a peer les -- for example, if a particular peer gossips a block, it should have access to its parent. If a request for the parent fails, it's indicative of poor peer quality since peers should validate blocks before gossiping them. -### Why does `BeaconBlocksByRange` let the server choose which branch to send blocks from? +#### Why does `BeaconBlocksByRange` let the server choose which branch to send blocks from? When connecting, the `Status` message gives an idea about the sync status of a particular peer, but this changes over time. By the time a subsequent `BeaconBlockByRange` request is processed, the information may be stale, @@ -1423,7 +1423,7 @@ To avoid this race condition, we allow the responding side to choose which branc The requesting client then goes on to validate the blocks and incorporate them in their own database -- because they follow the same rules, they should at this point arrive at the same canonical chain. -### Why are `BlocksByRange` requests only required to be served for the latest `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs? +#### Why are `BlocksByRange` requests only required to be served for the latest `MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs? Due to economic finality and weak subjectivity requirements of a proof-of-stake blockchain, for a new node to safely join the network the node must provide a recent checkpoint found out-of-band. This checkpoint can be in the form of a `root` & `epoch` or it can be the entire @@ -1447,7 +1447,7 @@ MIN_EPOCHS_FOR_BLOCK_REQUESTS = ( Where `MAX_SAFETY_DECAY = 100` and thus `MIN_EPOCHS_FOR_BLOCK_REQUESTS = 33024` (~5 months). -### Why must the proposer signature be checked when backfilling blocks in the database? +#### Why must the proposer signature be checked when backfilling blocks in the database? When backfilling blocks in a database from a know safe block/state (e.g. when starting from a weak subjectivity state), the node not only must ensure the `BeaconBlock`s form a chain to the known safe block, @@ -1462,7 +1462,7 @@ Although in this particular use case this does not represent a decay in safety would represent invalid historic data and could be unwittingly transmitted to additional nodes. -### What's the effect of empty slots on the sync algorithm? +#### What's the effect of empty slots on the sync algorithm? When syncing one can only tell that a slot has been skipped on a particular branch by examining subsequent blocks and analyzing the graph formed by the parent root. @@ -1472,9 +1472,9 @@ For example, if a peer responds with blocks [2, 3] when asked for [2, 3, 4], cli -- it merely means that the responding peer did not send it (they may not have it yet or may maliciously be trying to hide it) and successive blocks will be needed to determine if there exists a block at slot 4 in this particular branch. -## Discovery +### Discovery -### Why are we using discv5 and not libp2p Kademlia DHT? +#### Why are we using discv5 and not libp2p Kademlia DHT? discv5 is a standalone protocol, running on UDP on a dedicated port, meant for peer and service discovery only. discv5 supports self-certified, flexible peer records (ENRs) and topic-based advertisement, both of which are, or will be, requirements in this context. @@ -1490,7 +1490,7 @@ It should also help light clients of both networks find nodes with specific capa discv5 is in the process of being audited. -### What is the difference between an ENR and a multiaddr, and why are we using ENRs? +#### What is the difference between an ENR and a multiaddr, and why are we using ENRs? Ethereum Node Records are self-certified node records. Nodes craft and disseminate ENRs for themselves, proving authorship via a cryptographic signature. @@ -1510,7 +1510,7 @@ discv5 uses ENRs and we will presumably need to: 2. Define a bi-directional conversion function between multiaddrs and the corresponding denormalized fields in an ENR (ip, ip6, tcp, tcp6, etc.), for compatibility with nodes that do not support multiaddr natively (e.g. Ethereum execution-layer nodes). -### Why do we not form ENRs and find peers until genesis block/state is known? +#### Why do we not form ENRs and find peers until genesis block/state is known? Although client software might very well be running locally prior to the solidification of the beacon chain genesis state and block, clients cannot form valid ENRs prior to this point. @@ -1521,9 +1521,9 @@ Once genesis data is known, we can then form ENRs and safely find peers. When using a proof-of-work deposit contract for deposits, `fork_digest` will be known `GENESIS_DELAY` (7 days in mainnet configuration) before `genesis_time`, providing ample time to find peers and form initial connections and gossip subnets prior to genesis. -## Compression/Encoding +### Compression/Encoding -### Why are we using SSZ for encoding? +#### Why are we using SSZ for encoding? SSZ is used at the consensus layer, and all implementations should have support for SSZ-encoding/decoding, requiring no further dependencies to be added to client implementations. @@ -1533,7 +1533,7 @@ The actual data in most protocols will be further compressed for efficiency. SSZ has well-defined schemas for consensus objects (typically sent across the wire) reducing any serialization schema data that needs to be sent. It also has defined all required types that are required for this network specification. -### Why are we compressing, and at which layers? +#### Why are we compressing, and at which layers? We compress on the wire to achieve smaller payloads per-message, which, in aggregate, result in higher efficiency, better utilization of available bandwidth, and overall reduction in network-wide traffic overhead. @@ -1563,13 +1563,13 @@ This looks different depending on the interaction layer: implementers are encouraged to encapsulate the encoding and compression logic behind MessageReader and MessageWriter components/strategies that can be layered on top of the raw byte streams. -### Why are we using Snappy for compression? +#### Why are we using Snappy for compression? Snappy is used in Ethereum 1.0. It is well maintained by Google, has good benchmarks, and can calculate the size of the uncompressed object without inflating it in memory. This prevents DOS vectors where large uncompressed data is sent. -### Can I get access to unencrypted bytes on the wire for debugging purposes? +#### Can I get access to unencrypted bytes on the wire for debugging purposes? Yes, you can add loggers in your libp2p protocol handlers to log incoming and outgoing messages. It is recommended to use programming design patterns to encapsulate the logging logic cleanly. @@ -1580,7 +1580,7 @@ you can use logging facilities in those frameworks/runtimes to enable message tr For specific ad-hoc testing scenarios, you can use the [plaintext/2.0.0 secure channel](https://github.com/libp2p/specs/blob/master/plaintext/README.md) (which is essentially no-op encryption or message authentication), in combination with tcpdump or Wireshark to inspect the wire. -### What are SSZ type size bounds? +#### What are SSZ type size bounds? The SSZ encoding outputs of each type have size bounds: each dynamic type, such as a list, has a "limit", which can be used to compute the maximum valid output size. Note that for some more complex dynamic-length objects, element offsets (4 bytes each) may need to be included. @@ -1589,7 +1589,7 @@ Other types are static, they have a fixed size: no dynamic-length content is inv For reference, the type bounds can be computed ahead of time, [as per this example](https://gist.github.com/protolambda/db75c7faa1e94f2464787a480e5d613e). It is advisable to derive these lengths from the SSZ type definitions in use, to ensure that version changes do not cause out-of-sync type bounds. -# libp2p implementations matrix +## libp2p implementations matrix This section will soon contain a matrix showing the maturity/state of the libp2p features required by this spec across the languages in which clients are being developed. From accf99fba3b05e6a7c505e4f5bb07168e0921cbb Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 20 Apr 2023 08:47:49 +0900 Subject: [PATCH 32/34] rename to eip6914 --- specs/_features/{reuse_indices => eip6914}/beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename specs/_features/{reuse_indices => eip6914}/beacon-chain.md (93%) diff --git a/specs/_features/reuse_indices/beacon-chain.md b/specs/_features/eip6914/beacon-chain.md similarity index 93% rename from specs/_features/reuse_indices/beacon-chain.md rename to specs/_features/eip6914/beacon-chain.md index 6dd71e36f..5f7589594 100644 --- a/specs/_features/reuse_indices/beacon-chain.md +++ b/specs/_features/eip6914/beacon-chain.md @@ -1,4 +1,4 @@ -# Reuse indices -- The Beacon Chain +EIP-6914 -- The Beacon Chain ## Table of contents @@ -21,7 +21,7 @@ ## Introduction -This is the beacon chain specification to assign new deposits to existing validator records that have withdrawn long ago. +This is the beacon chain specification to assign new deposits to existing validator records. Refers to [EIP-6914](https://github.com/ethereum/EIPs/pull/6914). *Note:* This specification is built upon [Capella](../../capella/beacon_chain.md) and is under active development. From 498fbd04a23de5a21e4544e203cb7157679c5282 Mon Sep 17 00:00:00 2001 From: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu, 20 Apr 2023 08:52:25 +0900 Subject: [PATCH 33/34] rename SAFE_EPOCHS_TO_REUSE_INDEX --- specs/_features/eip6914/beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/_features/eip6914/beacon-chain.md b/specs/_features/eip6914/beacon-chain.md index 5f7589594..2c60c9bdb 100644 --- a/specs/_features/eip6914/beacon-chain.md +++ b/specs/_features/eip6914/beacon-chain.md @@ -31,7 +31,7 @@ This is the beacon chain specification to assign new deposits to existing valida | Name | Value | Unit | Duration | | - | - | - | - | -| `REUSE_VALIDATOR_INDEX_DELAY` | `uint64(2**16)` (= 65,536) | epochs | ~0.8 year | +| `SAFE_EPOCHS_TO_REUSE_INDEX` | `uint64(2**16)` (= 65,536) | epochs | ~0.8 year | ## Helper functions @@ -45,7 +45,7 @@ def is_reusable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> Check if ``validator`` index can be re-assigned to a new deposit. """ return ( - epoch > validator.withdrawable_epoch + REUSE_VALIDATOR_INDEX_DELAY + epoch > validator.withdrawable_epoch + SAFE_EPOCHS_TO_REUSE_INDEX and balance == 0 ) ``` From b7f3d37a528b8b8e3c2ce250eded3855d0b04552 Mon Sep 17 00:00:00 2001 From: Ben Edgington Date: Fri, 21 Apr 2023 09:16:51 +0100 Subject: [PATCH 34/34] Move is_previous_epoch_justified --- specs/phase0/fork-choice.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/specs/phase0/fork-choice.md b/specs/phase0/fork-choice.md index 6e281d5c3..f39a9234c 100644 --- a/specs/phase0/fork-choice.md +++ b/specs/phase0/fork-choice.md @@ -11,8 +11,8 @@ - [Configuration](#configuration) - [Helpers](#helpers) - [`LatestMessage`](#latestmessage) - - [`is_previous_epoch_justified`](#is_previous_epoch_justified) - [`Store`](#store) + - [`is_previous_epoch_justified`](#is_previous_epoch_justified) - [`get_forkchoice_store`](#get_forkchoice_store) - [`get_slots_since_genesis`](#get_slots_since_genesis) - [`get_current_slot`](#get_current_slot) @@ -92,17 +92,6 @@ class LatestMessage(object): root: Root ``` - -### `is_previous_epoch_justified` - -```python -def is_previous_epoch_justified(store: Store) -> bool: - current_slot = get_current_slot(store) - current_epoch = compute_epoch_at_slot(current_slot) - return store.justified_checkpoint.epoch + 1 == current_epoch -``` - - #### `Store` The `Store` is responsible for tracking information required for the fork choice algorithm. The important fields being tracked are described below: @@ -130,6 +119,15 @@ class Store(object): unrealized_justifications: Dict[Root, Checkpoint] = field(default_factory=dict) ``` +#### `is_previous_epoch_justified` + +```python +def is_previous_epoch_justified(store: Store) -> bool: + current_slot = get_current_slot(store) + current_epoch = compute_epoch_at_slot(current_slot) + return store.justified_checkpoint.epoch + 1 == current_epoch +``` + #### `get_forkchoice_store` The provided anchor-state will be regarded as a trusted state, to not roll back beyond.