From 351dcd4bb8ec963b7c38cbe1769c45eea2bfb47d Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 29 Oct 2019 08:48:16 +0100 Subject: [PATCH 01/33] p2p-interface: clarify that signing_root is used for block requests --- specs/networking/p2p-interface.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 5bccd25a9..662a9be6b 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -320,7 +320,7 @@ Here, `result` represents the 1-byte response code. The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time: -- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `HashTreeRoots`'s. +- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Hash`'s. - `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet. #### SSZ-encoding strategy (with or without Snappy) @@ -403,7 +403,7 @@ The response MUST consist of a single `response_chunk`. Request Content: ``` ( - head_block_root: HashTreeRoot + head_block_root: Hash start_slot: uint64 count: uint64 step: uint64 @@ -417,7 +417,7 @@ Response Content: ) ``` -Requests count beacon blocks from the peer starting from `start_slot` on the chain defined by `head_block_root`. The response MUST contain no more than count blocks. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`. +Requests count beacon blocks from the peer starting from `start_slot` on the chain defined by `head_block_root` (= `signing_root(BeaconBlock)`). The response MUST contain no more than count blocks. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`. The request MUST be encoded as an SSZ-container. @@ -441,7 +441,7 @@ Request Content: ``` ( - []HashTreeRoot + []Hash ) ``` @@ -453,7 +453,7 @@ Response Content: ) ``` -Requests blocks by their block roots. The response is a list of `BeaconBlock` whose length is less than or equal to the number of requested blocks. It may be less in the case that the responding peer is missing blocks. +Requests blocks by block root (= `signing_root(BeaconBlock)`). The response is a list of `BeaconBlock` whose length is less than or equal to the number of requested blocks. It may be less in the case that the responding peer is missing blocks. `BeaconBlocksByRoot` is primarily used to recover recent blocks (e.g. when receiving a block or attestation whose parent is unknown). From 1dc09e8567c9a1103d619fabb338c53fef384812 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 31 Oct 2019 17:32:08 +0100 Subject: [PATCH 02/33] hash cleanups * one more hash tree root gone for blocks - block hashes are always signing roots! * use simple serialize data types consistently --- specs/networking/p2p-interface.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 662a9be6b..9444ce267 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -320,7 +320,7 @@ Here, `result` represents the 1-byte response code. The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time: -- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Hash`'s. +- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Bytes32`'s. - `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet. #### SSZ-encoding strategy (with or without Snappy) @@ -344,20 +344,20 @@ constituents individually as `response_chunk`s. For example, the Request, Response Content: ``` ( - head_fork_version: bytes4 - finalized_root: bytes32 + head_fork_version: Bytes4 + finalized_root: Bytes32 finalized_epoch: uint64 - head_root: bytes32 + head_root: Bytes32 head_slot: uint64 ) ``` -The fields are: +The fields are, as seen by the client at the time of sending the message: - `head_fork_version`: The beacon_state `Fork` version. -- `finalized_root`: The latest finalized root the node knows about. -- `finalized_epoch`: The latest finalized epoch the node knows about. -- `head_root`: The block hash tree root corresponding to the head of the chain as seen by the sending node. -- `head_slot`: The slot corresponding to the `head_root`. +- `finalized_root`: The signing root of the latest finalized block. +- `finalized_epoch`: The epoch of the block corresponding to `finalized_root`. +- `head_root`: The signing root of the current head block. +- `head_slot`: The slot of the block corresponding to the `head_root`. The dialing client MUST send a `Status` request upon connection. @@ -403,7 +403,7 @@ The response MUST consist of a single `response_chunk`. Request Content: ``` ( - head_block_root: Hash + head_block_root: Bytes32 start_slot: uint64 count: uint64 step: uint64 @@ -441,7 +441,7 @@ Request Content: ``` ( - []Hash + []Bytes32 ) ``` From af5238ad9aea3fce7157357d1df61280f1b02f34 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Fri, 1 Nov 2019 08:42:10 +0100 Subject: [PATCH 03/33] Describe which finalized root/epoch to use --- specs/networking/p2p-interface.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 9444ce267..30ca43fd3 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -354,8 +354,8 @@ Request, Response Content: The fields are, as seen by the client at the time of sending the message: - `head_fork_version`: The beacon_state `Fork` version. -- `finalized_root`: The signing root of the latest finalized block. -- `finalized_epoch`: The epoch of the block corresponding to `finalized_root`. +- `finalized_root`: `state.finalized_checkpoint.root` for the state corresponding to the head block. +- `finalized_epoch`: `state.finalized_checkpoint.epoch` for the state corresponding to the head block. - `head_root`: The signing root of the current head block. - `head_slot`: The slot of the block corresponding to the `head_root`. From 65b615a4d4cf75a50b29d25c53f1bc5422770ae5 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 1 Nov 2019 21:02:53 -0600 Subject: [PATCH 04/33] remove custody_bits from attestation --- specs/core/0_beacon-chain.md | 33 +++-------- specs/core/0_fork-choice.md | 2 +- .../test/fork_choice/test_on_attestation.py | 13 ++-- .../eth2spec/test/helpers/attestations.py | 4 +- .../test_process_attestation.py | 58 ------------------ .../test_process_attester_slashing.py | 59 +++++-------------- .../test_process_rewards_and_penalties.py | 2 +- .../eth2spec/test/sanity/test_blocks.py | 3 +- 8 files changed, 35 insertions(+), 139 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9f0f21435..632c4963a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -320,8 +320,7 @@ class AttestationDataAndCustodyBit(Container): ```python class IndexedAttestation(Container): - custody_bit_0_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] # Indices with custody bit equal to 0 - custody_bit_1_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] # Indices with custody bit equal to 1 + attesting_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] data: AttestationData signature: BLSSignature ``` @@ -399,7 +398,6 @@ class AttesterSlashing(Container): class Attestation(Container): aggregation_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE] data: AttestationData - custody_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE] signature: BLSSignature ``` @@ -605,30 +603,21 @@ def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: Indexe """ Check if ``indexed_attestation`` has valid indices and signature. """ - bit_0_indices = indexed_attestation.custody_bit_0_indices - bit_1_indices = indexed_attestation.custody_bit_1_indices + indices = indexed_attestation.attesting_indices - # Verify no index has custody bit equal to 1 [to be removed in phase 1] - if not len(bit_1_indices) == 0: # [to be removed in phase 1] - return False # [to be removed in phase 1] # Verify max number of indices - if not len(bit_0_indices) + len(bit_1_indices) <= MAX_VALIDATORS_PER_COMMITTEE: - return False - # Verify index sets are disjoint - if not len(set(bit_0_indices).intersection(bit_1_indices)) == 0: + if not len(indices) <= MAX_VALIDATORS_PER_COMMITTEE: return False # Verify indices are sorted - if not (bit_0_indices == sorted(bit_0_indices) and bit_1_indices == sorted(bit_1_indices)): + if not indices == sorted(indices): return False # Verify aggregate signature if not bls_verify_multiple( pubkeys=[ - bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_0_indices]), - bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_1_indices]), + bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), ], message_hashes=[ hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b0)), - hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b1)), ], signature=indexed_attestation.signature, domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch), @@ -922,13 +911,9 @@ def get_indexed_attestation(state: BeaconState, attestation: Attestation) -> Ind Return the indexed attestation corresponding to ``attestation``. """ attesting_indices = get_attesting_indices(state, attestation.data, attestation.aggregation_bits) - custody_bit_1_indices = get_attesting_indices(state, attestation.data, attestation.custody_bits) - assert custody_bit_1_indices.issubset(attesting_indices) - custody_bit_0_indices = attesting_indices.difference(custody_bit_1_indices) return IndexedAttestation( - custody_bit_0_indices=sorted(custody_bit_0_indices), - custody_bit_1_indices=sorted(custody_bit_1_indices), + attesting_indices=sorted(attesting_indices), data=attestation.data, signature=attestation.signature, ) @@ -1460,8 +1445,8 @@ def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSla assert is_valid_indexed_attestation(state, attestation_2) slashed_any = False - attesting_indices_1 = attestation_1.custody_bit_0_indices + attestation_1.custody_bit_1_indices - attesting_indices_2 = attestation_2.custody_bit_0_indices + attestation_2.custody_bit_1_indices + attesting_indices_1 = attestation_1.attesting_indices + attesting_indices_2 = attestation_2.attesting_indices for index in sorted(set(attesting_indices_1).intersection(attesting_indices_2)): if is_slashable_validator(state.validators[index], get_current_epoch(state)): slash_validator(state, index) @@ -1479,7 +1464,7 @@ def process_attestation(state: BeaconState, attestation: Attestation) -> None: assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH committee = get_beacon_committee(state, data.slot, data.index) - assert len(attestation.aggregation_bits) == len(attestation.custody_bits) == len(committee) + assert len(attestation.aggregation_bits) == len(committee) pending_attestation = PendingAttestation( data=data, diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index b909ce732..bf9688db7 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -199,7 +199,7 @@ def on_attestation(store: Store, attestation: Attestation) -> None: assert is_valid_indexed_attestation(target_state, indexed_attestation) # Update latest messages - for i in indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices: + for i in indexed_attestation.attesting_indices: if i not in store.latest_messages or target.epoch > store.latest_messages[i].epoch: store.latest_messages[i] = LatestMessage(epoch=target.epoch, root=attestation.data.beacon_block_root) ``` diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py index 70375ef27..39c179b59 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py @@ -1,7 +1,5 @@ -from eth2spec.test.context import with_all_phases, spec_state_test, with_phases - - +from eth2spec.test.context import with_all_phases, spec_state_test from eth2spec.test.helpers.block import build_empty_block_for_next_slot from eth2spec.test.helpers.attestations import get_valid_attestation from eth2spec.test.helpers.state import state_transition_and_sign_block @@ -19,7 +17,7 @@ def run_on_attestation(spec, state, store, attestation, valid=True): indexed_attestation = spec.get_indexed_attestation(state, attestation) spec.on_attestation(store, attestation) assert ( - store.latest_messages[indexed_attestation.custody_bit_0_indices[0]] == + store.latest_messages[indexed_attestation.attesting_indices[0]] == spec.LatestMessage( epoch=attestation.data.target.epoch, root=attestation.data.beacon_block_root, @@ -100,7 +98,7 @@ def test_on_attestation_same_slot(spec, state): run_on_attestation(spec, state, store, attestation, False) -@with_phases(['phase0']) +@with_all_phases @spec_state_test def test_on_attestation_invalid_attestation(spec, state): store = spec.get_genesis_store(state) @@ -113,6 +111,7 @@ def test_on_attestation_invalid_attestation(spec, state): spec.on_block(store, block) attestation = get_valid_attestation(spec, state, slot=block.slot) - # make attestation invalid by setting a phase1-only custody bit - attestation.custody_bits[0] = 1 + # make invalid by using an invalid committee index + attestation.data.index = spec.MAX_COMMITTEES_PER_SLOT * spec.SLOTS_PER_EPOCH + run_on_attestation(spec, state, store, attestation, False) diff --git a/test_libs/pyspec/eth2spec/test/helpers/attestations.py b/test_libs/pyspec/eth2spec/test/helpers/attestations.py index d36e62ace..d00a4cc96 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/attestations.py +++ b/test_libs/pyspec/eth2spec/test/helpers/attestations.py @@ -54,11 +54,9 @@ def get_valid_attestation(spec, state, slot=None, index=None, signed=False): committee_size = len(beacon_committee) aggregation_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE](*([0] * committee_size)) - custody_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE](*([0] * committee_size)) attestation = spec.Attestation( aggregation_bits=aggregation_bits, data=attestation_data, - custody_bits=custody_bits, ) fill_aggregate_attestation(spec, state, attestation) if signed: @@ -83,7 +81,7 @@ def sign_aggregate_attestation(spec, state, attestation_data, participants: List def sign_indexed_attestation(spec, state, indexed_attestation): - participants = indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices + participants = indexed_attestation.attesting_indices indexed_attestation.signature = sign_aggregate_attestation(spec, state, indexed_attestation.data, participants) diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py index 04fa880e0..554d77a00 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py @@ -274,35 +274,6 @@ def test_bad_source_root(spec, state): yield from run_attestation_processing(spec, state, attestation, False) -@with_all_phases -@spec_state_test -def test_inconsistent_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - custody_bits = attestation.custody_bits[:] - custody_bits.append(False) - - attestation.custody_bits = custody_bits - - sign_attestation(spec, state, attestation) - - yield from run_attestation_processing(spec, state, attestation, False) - - -@with_phases(['phase0']) -@spec_state_test -def test_non_empty_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - attestation.custody_bits = attestation.aggregation_bits[:] - - sign_attestation(spec, state, attestation) - - yield from run_attestation_processing(spec, state, attestation, False) - - @with_all_phases @spec_state_test def test_empty_aggregation_bits(spec, state): @@ -344,32 +315,3 @@ def test_too_few_aggregation_bits(spec, state): attestation.aggregation_bits = attestation.aggregation_bits[:-1] yield from run_attestation_processing(spec, state, attestation, False) - - -@with_all_phases -@spec_state_test -def test_too_many_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state, signed=True) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - # one too many bits - attestation.custody_bits.append(0b0) - - yield from run_attestation_processing(spec, state, attestation, False) - - -@with_all_phases -@spec_state_test -def test_too_few_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - attestation.custody_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE]( - *([0b1] + [0b0] * (len(attestation.custody_bits) - 1))) - - sign_attestation(spec, state, attestation) - - # one too few bits - attestation.custody_bits = attestation.custody_bits[:-1] - - yield from run_attestation_processing(spec, state, attestation, False) diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py index 20a510648..85e807ec0 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py @@ -25,10 +25,7 @@ def run_attester_slashing_processing(spec, state, attester_slashing, valid=True) yield 'post', None return - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices proposer_index = spec.get_beacon_proposer_index(state) pre_proposer_balance = get_balance(state, proposer_index) @@ -112,10 +109,7 @@ def test_success_surround(spec, state): @always_bls def test_success_already_exited_recent(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices for index in slashed_indices: spec.initiate_validator_exit(state, index) @@ -127,10 +121,7 @@ def test_success_already_exited_recent(spec, state): @always_bls def test_success_already_exited_long_ago(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices for index in slashed_indices: spec.initiate_validator_exit(state, index) state.validators[index].withdrawable_epoch = spec.get_current_epoch(state) + 2 @@ -190,38 +181,23 @@ def test_participants_already_slashed(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) # set all indices to slashed - attestation_1 = attester_slashing.attestation_1 - validator_indices = attestation_1.custody_bit_0_indices + attestation_1.custody_bit_1_indices + validator_indices = attester_slashing.attestation_1.attesting_indices for index in validator_indices: state.validators[index].slashed = True yield from run_attester_slashing_processing(spec, state, attester_slashing, False) -@with_all_phases -@spec_state_test -def test_custody_bit_0_and_1_intersect(spec, state): - attester_slashing = get_valid_attester_slashing(spec, state, signed_1=False, signed_2=True) - - attester_slashing.attestation_1.custody_bit_1_indices.append( - attester_slashing.attestation_1.custody_bit_0_indices[0] - ) - - sign_indexed_attestation(spec, state, attester_slashing.attestation_1) - - yield from run_attester_slashing_processing(spec, state, attester_slashing, False) - - @with_all_phases @spec_state_test @always_bls def test_att1_bad_extra_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices.append(options[len(options) // 2]) # add random index, not previously in attestation. - attester_slashing.attestation_1.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_1.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad extra index is spotted, and slashing is aborted. @@ -234,10 +210,10 @@ def test_att1_bad_extra_index(spec, state): def test_att1_bad_replaced_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices[3] = options[len(options) // 2] # replace with random index, not previously in attestation. - attester_slashing.attestation_1.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_1.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad replaced index is spotted, and slashing is aborted. @@ -250,10 +226,10 @@ def test_att1_bad_replaced_index(spec, state): def test_att2_bad_extra_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices.append(options[len(options) // 2]) # add random index, not previously in attestation. - attester_slashing.attestation_2.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_2.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad extra index is spotted, and slashing is aborted. @@ -266,10 +242,10 @@ def test_att2_bad_extra_index(spec, state): def test_att2_bad_replaced_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices[3] = options[len(options) // 2] # replace with random index, not previously in attestation. - attester_slashing.attestation_2.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_2.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad replaced index is spotted, and slashing is aborted. @@ -278,10 +254,10 @@ def test_att2_bad_replaced_index(spec, state): @with_all_phases @spec_state_test -def test_unsorted_att_1_bit0(spec, state): +def test_unsorted_att_1(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=False, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices assert len(indices) >= 3 indices[1], indices[2] = indices[2], indices[1] # unsort second and third index sign_indexed_attestation(spec, state, attester_slashing.attestation_1) @@ -291,15 +267,12 @@ def test_unsorted_att_1_bit0(spec, state): @with_all_phases @spec_state_test -def test_unsorted_att_2_bit0(spec, state): +def test_unsorted_att_2(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=False) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices assert len(indices) >= 3 indices[1], indices[2] = indices[2], indices[1] # unsort second and third index sign_indexed_attestation(spec, state, attester_slashing.attestation_2) yield from run_attester_slashing_processing(spec, state, attester_slashing, False) - - -# note: unsorted indices for custody bit 0 are to be introduced in phase 1 testing. diff --git a/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py b/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py index 99ba9d284..7d844b63b 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py @@ -141,7 +141,7 @@ def test_duplicate_attestation(spec, state): attestation = get_valid_attestation(spec, state, signed=True) indexed_attestation = spec.get_indexed_attestation(state, attestation) - participants = indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices + participants = indexed_attestation.attesting_indices assert len(participants) > 0 diff --git a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py index bffb1c1fc..bbc4bba3a 100644 --- a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py +++ b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py @@ -188,8 +188,7 @@ def test_attester_slashing(spec, state): pre_state = deepcopy(state) attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - validator_index = (attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices)[0] + validator_index = attester_slashing.attestation_1.attesting_indices[0] assert not state.validators[validator_index].slashed From 600265a3110b647418836db59768e8fa453a3c1d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 1 Nov 2019 21:12:32 -0600 Subject: [PATCH 05/33] remove AttestationDataAndCustodyBit --- scripts/build_spec.py | 3 +-- specs/core/0_beacon-chain.md | 19 +++-------------- specs/validator/0_beacon-chain-validator.md | 21 +------------------ .../pyspec/eth2spec/fuzzing/test_decoder.py | 2 +- .../eth2spec/test/helpers/attestations.py | 9 ++------ .../test_process_attestation.py | 2 +- 6 files changed, 9 insertions(+), 47 deletions(-) diff --git a/scripts/build_spec.py b/scripts/build_spec.py index e05907014..9ed9d3ae4 100644 --- a/scripts/build_spec.py +++ b/scripts/build_spec.py @@ -24,14 +24,13 @@ from eth2spec.utils.ssz.ssz_impl import ( signing_root, ) from eth2spec.utils.ssz.ssz_typing import ( - bit, boolean, Container, List, Vector, uint64, + boolean, Container, List, Vector, uint64, Bytes1, Bytes4, Bytes8, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, ) from eth2spec.utils.bls import ( bls_aggregate_signatures, bls_aggregate_pubkeys, bls_verify, - bls_verify_multiple, bls_sign, ) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 632c4963a..037e1672f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -26,7 +26,6 @@ - [`Checkpoint`](#checkpoint) - [`Validator`](#validator) - [`AttestationData`](#attestationdata) - - [`AttestationDataAndCustodyBit`](#attestationdataandcustodybit) - [`IndexedAttestation`](#indexedattestation) - [`PendingAttestation`](#pendingattestation) - [`Eth1Data`](#eth1data) @@ -308,14 +307,6 @@ class AttestationData(Container): target: Checkpoint ``` -#### `AttestationDataAndCustodyBit` - -```python -class AttestationDataAndCustodyBit(Container): - data: AttestationData - custody_bit: bit # Challengeable bit (SSZ-bool, 1 byte) for the custody of shard data -``` - #### `IndexedAttestation` ```python @@ -612,13 +603,9 @@ def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: Indexe if not indices == sorted(indices): return False # Verify aggregate signature - if not bls_verify_multiple( - pubkeys=[ - bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), - ], - message_hashes=[ - hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b0)), - ], + if not bls_verify( + pubkey=bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), + message_hash=hash_tree_root(indexed_attestation.data), signature=indexed_attestation.signature, domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch), ): diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 166534031..236d12a88 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -45,7 +45,6 @@ - [Construct attestation](#construct-attestation) - [Data](#data) - [Aggregation bits](#aggregation-bits) - - [Custody bits](#custody-bits) - [Aggregate signature](#aggregate-signature) - [Broadcast attestation](#broadcast-attestation) - [Attestation aggregation](#attestation-aggregation) @@ -53,7 +52,6 @@ - [Construct aggregate](#construct-aggregate) - [Data](#data-1) - [Aggregation bits](#aggregation-bits-1) - - [Custody bits](#custody-bits-1) - [Aggregate signature](#aggregate-signature-1) - [Broadcast aggregate](#broadcast-aggregate) - [`AggregateAndProof`](#aggregateandproof) @@ -331,25 +329,14 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes *Note*: Calling `get_attesting_indices(state, attestation.data, attestation.aggregation_bits)` should return a list of length equal to 1, containing `validator_index`. -##### Custody bits - -- Let `attestation.custody_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` filled with zeros of length `len(committee)`. - -*Note*: This is a stub for Phase 0. - ##### Aggregate signature Set `attestation.signature = signed_attestation_data` where `signed_attestation_data` is obtained from: ```python def get_signed_attestation_data(state: BeaconState, attestation: IndexedAttestation, privkey: int) -> BLSSignature: - attestation_data_and_custody_bit = AttestationDataAndCustodyBit( - data=attestation.data, - custody_bit=0b0, - ) - domain = get_domain(state, DOMAIN_BEACON_ATTESTER, attestation.data.target.epoch) - return bls_sign(privkey, hash_tree_root(attestation_data_and_custody_bit), domain) + return bls_sign(privkey, hash_tree_root(attestation.data), domain) ``` #### Broadcast attestation @@ -391,12 +378,6 @@ Set `aggregate_attestation.data = attestation_data` where `attestation_data` is Let `aggregate_attestation.aggregation_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` of length `len(committee)`, where each bit set from each individual attestation is set to `0b1`. -##### Custody bits - -- Let `aggregate_attestation.custody_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` filled with zeros of length `len(committee)`. - -*Note*: This is a stub for Phase 0. - ##### Aggregate signature Set `aggregate_attestation.signature = aggregate_signature` where `aggregate_signature` is obtained from: diff --git a/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py b/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py index 77b52e7a2..9cabefb13 100644 --- a/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py +++ b/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py @@ -9,7 +9,7 @@ def test_decoder(): rng = Random(123) # check these types only, Block covers a lot of operation types already. - for typ in [spec.AttestationDataAndCustodyBit, spec.BeaconState, spec.BeaconBlock]: + for typ in [spec.Attestation, spec.BeaconState, spec.BeaconBlock]: # create a random pyspec value original = random_value.get_random_ssz_object(rng, typ, 100, 10, mode=random_value.RandomizationMode.mode_random, diff --git a/test_libs/pyspec/eth2spec/test/helpers/attestations.py b/test_libs/pyspec/eth2spec/test/helpers/attestations.py index d00a4cc96..299ca32ff 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/attestations.py +++ b/test_libs/pyspec/eth2spec/test/helpers/attestations.py @@ -95,14 +95,9 @@ def sign_attestation(spec, state, attestation): attestation.signature = sign_aggregate_attestation(spec, state, attestation.data, participants) -def get_attestation_signature(spec, state, attestation_data, privkey, custody_bit=0b0): - message_hash = spec.AttestationDataAndCustodyBit( - data=attestation_data, - custody_bit=custody_bit, - ).hash_tree_root() - +def get_attestation_signature(spec, state, attestation_data, privkey): return bls_sign( - message_hash=message_hash, + message_hash=attestation_data.hash_tree_root(), privkey=privkey, domain=spec.get_domain( state=state, diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py index 554d77a00..f19bc66d3 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py @@ -2,7 +2,7 @@ from eth2spec.test.context import ( spec_state_test, expect_assertion_error, always_bls, never_bls, - with_all_phases, with_phases, + with_all_phases, spec_test, low_balances, with_custom_state, From 1c99ab2c96be00cdd50fdf1bec56fd3481d6a5b9 Mon Sep 17 00:00:00 2001 From: Paul Hauner Date: Fri, 1 Nov 2019 10:51:21 +1100 Subject: [PATCH 06/33] Specify inclusive range for genesis deposits --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9f0f21435..f00be9a88 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1026,7 +1026,7 @@ Before the Ethereum 2.0 genesis has been triggered, and for every Ethereum 1.0 b - `eth1_block_hash` is the hash of the Ethereum 1.0 block - `eth1_timestamp` is the Unix timestamp corresponding to `eth1_block_hash` -- `deposits` is the sequence of all deposits, ordered chronologically, up to the block with hash `eth1_block_hash` +- `deposits` is the sequence of all deposits, ordered chronologically, up to (and including) the block with hash `eth1_block_hash` ```python def initialize_beacon_state_from_eth1(eth1_block_hash: Hash, From 405e2185981a689ac98871ed3e430435ddb26a04 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 08:55:34 -0700 Subject: [PATCH 07/33] add initial fork choice bounce prevention and tests --- specs/core/0_fork-choice.md | 50 ++++++++++++- .../eth2spec/test/fork_choice/test_on_tick.py | 72 +++++++++++++++++++ 2 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index b909ce732..722695f23 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -43,6 +43,12 @@ The head block root associated with a `store` is defined as `get_head(store)`. A 4) **Manual forks**: Manual forks may arbitrarily change the fork choice rule but are expected to be enacted at epoch transitions, with the fork details reflected in `state.fork`. 5) **Implementation**: The implementation found in this specification is constructed for ease of understanding rather than for optimization in computation, space, or any other resource. A number of optimized alternatives can be found [here](https://github.com/protolambda/lmd-ghost). +### Configuration + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `9` | slots | 108 seconds | + ### Helpers #### `LatestMessage` @@ -60,8 +66,10 @@ class LatestMessage(object): @dataclass class Store(object): time: uint64 + genesis_time: uint64 justified_checkpoint: Checkpoint finalized_checkpoint: Checkpoint + queued_justified_checkpoints: List[Checkpoint, 2**40] = field(default_factory=list) blocks: Dict[Hash, BeaconBlock] = field(default_factory=dict) block_states: Dict[Hash, BeaconState] = field(default_factory=dict) checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) @@ -78,6 +86,7 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: finalized_checkpoint = Checkpoint(epoch=GENESIS_EPOCH, root=root) return Store( time=genesis_state.genesis_time, + genesis_time=genesis_state.genesis_time, justified_checkpoint=justified_checkpoint, finalized_checkpoint=finalized_checkpoint, blocks={root: genesis_block}, @@ -86,6 +95,11 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: ) ``` +```python +def get_current_slot(store: Store) -> Slot: + return Slot((store.time - store.genesis_time) // SECONDS_PER_SLOT) +``` + #### `get_ancestor` ```python @@ -130,13 +144,44 @@ def get_head(store: Store) -> Hash: head = max(children, key=lambda root: (get_latest_attesting_balance(store, root), root)) ``` +#### `should_update_justified_checkpoint` + +```python +def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: + current_epoch = compute_epoch_at_slot(get_current_slot(store)) + + if get_current_slot(store) - compute_start_slot_at_epoch(current_epoch) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + return True + + justified_block = store.blocks[justified_checkpoint.root] + if justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): + return False + if not get_ancestor(store, justified_checkpoint.root, store.blocks[justified_checkpoint.root].slot): + return False + + return True +``` + ### Handlers #### `on_tick` ```python def on_tick(store: Store, time: uint64) -> None: + previous_slot = get_current_slot(store) + + # update store time store.time = time + + current_slot = get_current_slot(store) + # not a new epoch, return + if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): + return + # if new epoch and there are queued_justified_checkpoints, update if any is better than the best in store + if any(store.queued_justified_checkpoints): + best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) + if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.justified_checkpoint = best_justified_checkpoint ``` #### `on_block` @@ -164,7 +209,10 @@ def on_block(store: Store, block: BeaconBlock) -> None: # Update justified checkpoint if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch: - store.justified_checkpoint = state.current_justified_checkpoint + if should_update_justified_checkpoint(store, state.current_justified_checkpoint): + store.justified_checkpoint = state.current_justified_checkpoint + else: + store.queued_justified_checkpoints.append(state.current_justified_checkpoint) # Update finalized checkpoint if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch: diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py new file mode 100644 index 000000000..cd0d20abb --- /dev/null +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -0,0 +1,72 @@ +from eth2spec.test.context import with_all_phases, spec_state_test + + +def run_on_tick(spec, store, time, new_justified_checkpoint=None): + previous_justified_checkpoint = store.justified_checkpoint + + spec.on_tick(store, time) + + assert store.time == time + + if new_justified_checkpoint: + assert store.justified_checkpoint == new_justified_checkpoint + assert store.justified_checkpoint.epoch > previous_justified_checkpoint.epoch + else: + assert store.justified_checkpoint == previous_justified_checkpoint + + +@with_all_phases +@spec_state_test +def test_basic(spec, state): + store = spec.get_genesis_store(state) + run_on_tick(spec, store, store.time + 1) + + +@with_all_phases +@spec_state_test +def test_update_justified_single(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + + +@with_all_phases +@spec_state_test +def test_update_justified_multiple(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = None # remember checkpoint with latest epoch + for i in range(3): + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + i + 1, + root=i.to_bytes(1, byteorder='big') * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + + +@with_all_phases +@spec_state_test +def test_no_update_(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + From 40a21dc3ad55e5420f0656ffc85d9b660e84c90e Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 09:00:04 -0700 Subject: [PATCH 08/33] PR feedback --- README.md | 1 - specs/core/0_beacon-chain.md | 10 ++-------- 2 files changed, 2 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index a6f23db9d..fa103394d 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,6 @@ The following are the broad design goals for Ethereum 2.0: ## For spec contributors - Documentation on the different components used during spec writing can be found here: * [YAML Test Generators](test_generators/README.md) * [Executable Python Spec, with Py-tests](test_libs/pyspec/README.md) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 037e1672f..fc51dd48d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -54,7 +54,6 @@ - [`hash_tree_root`](#hash_tree_root) - [`signing_root`](#signing_root) - [`bls_verify`](#bls_verify) - - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) - [Predicates](#predicates) - [`is_active_validator`](#is_active_validator) @@ -542,10 +541,6 @@ def bytes_to_int(data: bytes) -> uint64: `bls_verify` is a function for verifying a BLS signature, as defined in the [BLS Signature spec](../bls_signature.md#bls_verify). -#### `bls_verify_multiple` - -`bls_verify_multiple` is a function for verifying a BLS signature constructed from multiple messages, as defined in the [BLS Signature spec](../bls_signature.md#bls_verify_multiple). - #### `bls_aggregate_pubkeys` `bls_aggregate_pubkeys` is a function for aggregating multiple BLS public keys into a single aggregate key, as defined in the [BLS Signature spec](../bls_signature.md#bls_aggregate_pubkeys). @@ -1432,9 +1427,8 @@ def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSla assert is_valid_indexed_attestation(state, attestation_2) slashed_any = False - attesting_indices_1 = attestation_1.attesting_indices - attesting_indices_2 = attestation_2.attesting_indices - for index in sorted(set(attesting_indices_1).intersection(attesting_indices_2)): + indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices) + for index in sorted(indices): if is_slashable_validator(state.validators[index], get_current_epoch(state)): slash_validator(state, index) slashed_any = True From 97d7cf519063f306039fa0e9e4958e4033932aa0 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 10:58:45 -0700 Subject: [PATCH 09/33] further test bounce attack --- configs/mainnet.yaml | 5 ++ configs/minimal.yaml | 6 ++ specs/core/0_fork-choice.md | 2 +- .../test/fork_choice/test_on_block.py | 71 +++++++++++++++++++ .../eth2spec/test/fork_choice/test_on_tick.py | 62 ++++++++++++++-- 5 files changed, 141 insertions(+), 5 deletions(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index af446d575..e1b0faa3c 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -23,6 +23,11 @@ MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 65536 MIN_GENESIS_TIME: 1578009600 +# Fork Choice +# --------------------------------------------------------------- +# 2**3 (= 8) +SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 8 + # Deposit contract # --------------------------------------------------------------- diff --git a/configs/minimal.yaml b/configs/minimal.yaml index 53599e83a..fbc961ab1 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -21,6 +21,12 @@ MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 64 # Jan 3, 2020 MIN_GENESIS_TIME: 1578009600 +# +# +# Fork Choice +# --------------------------------------------------------------- +# 2**1 (= 1) +SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 2 # Deposit contract diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 722695f23..f2bb64f2a 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -150,7 +150,7 @@ def get_head(store: Store) -> Hash: def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: current_epoch = compute_epoch_at_slot(get_current_slot(store)) - if get_current_slot(store) - compute_start_slot_at_epoch(current_epoch) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True justified_block = store.blocks[justified_checkpoint.root] diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py index 918c0f79e..36e61f0b5 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py @@ -132,3 +132,74 @@ def test_on_block_before_finalized(spec, state): block = build_empty_block_for_next_slot(spec, state) state_transition_and_sign_block(spec, state, block) run_on_block(spec, store, block, False) + + +@with_all_phases +@spec_state_test +def test_on_block_update_justified_checkpoint_within_safe_slots(spec, state): + # Initialization + store = spec.get_genesis_store(state) + time = 100 + spec.on_tick(store, time) + + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + state, store, last_block = apply_next_epoch_with_attestations(spec, state, store) + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + last_block_root = signing_root(last_block) + + # Mock the justified checkpoint + just_state = store.block_states[last_block_root] + new_justified = spec.Checkpoint( + epoch=just_state.current_justified_checkpoint.epoch + 1, + root=b'\x77' * 32, + ) + just_state.current_justified_checkpoint = new_justified + + block = build_empty_block_for_next_slot(spec, just_state) + state_transition_and_sign_block(spec, deepcopy(just_state), block) + assert spec.get_current_slot(store) % spec.SLOTS_PER_EPOCH < spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED + run_on_block(spec, store, block) + + assert store.justified_checkpoint == new_justified + + +@with_all_phases +@spec_state_test +def test_on_block_outside_safe_slots_and_old_block(spec, state): + # Initialization + store = spec.get_genesis_store(state) + time = 100 + spec.on_tick(store, time) + + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + state, store, last_block = apply_next_epoch_with_attestations(spec, state, store) + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + last_block_root = signing_root(last_block) + + # Mock justified block in store + just_block = build_empty_block_for_next_slot(spec, state) + # Slot is same as justified checkpoint so does not trigger an override in the store + just_block.slot = spec.compute_start_slot_at_epoch(store.justified_checkpoint.epoch) + store.blocks[just_block.hash_tree_root()] = just_block + + # Mock the justified checkpoint + just_state = store.block_states[last_block_root] + new_justified = spec.Checkpoint( + epoch=just_state.current_justified_checkpoint.epoch + 1, + root=just_block.hash_tree_root(), + ) + just_state.current_justified_checkpoint = new_justified + + block = build_empty_block_for_next_slot(spec, just_state) + state_transition_and_sign_block(spec, deepcopy(just_state), block) + + spec.on_tick(store, store.time + spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED * spec.SECONDS_PER_SLOT) + assert spec.get_current_slot(store) % spec.SLOTS_PER_EPOCH >= spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED + run_on_block(spec, store, block) + + assert store.justified_checkpoint != new_justified + assert store.queued_justified_checkpoints[0] == new_justified diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py index cd0d20abb..2ba89ebca 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -32,7 +32,6 @@ def test_update_justified_single(spec, state): epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) @@ -57,7 +56,7 @@ def test_update_justified_multiple(spec, state): @with_all_phases @spec_state_test -def test_no_update_(spec, state): +def test_no_update_same_slot_at_epoch_boundary(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH @@ -65,8 +64,63 @@ def test_no_update_(spec, state): epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + # set store time to already be at epoch boundary + store.time = seconds_per_epoch + run_on_tick(spec, store, store.time + 1) + + +@with_all_phases +@spec_state_test +def test_no_update_not_epoch_boundary(spec, state): + store = spec.get_genesis_store(state) + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + spec.SECONDS_PER_SLOT) + + +@with_all_phases +@spec_state_test +def test_no_update_new_justified_equal_epoch(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + store.justified_checkpoint = spec.Checkpoint( + epoch=new_justified.epoch, + root=b'\44' * 32, + ) + + run_on_tick(spec, store, store.time + seconds_per_epoch) + + +@with_all_phases +@spec_state_test +def test_no_update_new_justified_later_epoch(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + store.justified_checkpoint = spec.Checkpoint( + epoch=new_justified.epoch + 1, + root=b'\44' * 32, + ) + + run_on_tick(spec, store, store.time + seconds_per_epoch) From e20e11e0b54aee2683c9386e6d7d600707daa6f5 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:02:58 -0700 Subject: [PATCH 10/33] wipe queued justified after epoch transition --- specs/core/0_fork-choice.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index f2bb64f2a..2e8deffdb 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -47,7 +47,7 @@ The head block root associated with a `store` is defined as `get_head(store)`. A | Name | Value | Unit | Duration | | - | - | :-: | :-: | -| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `9` | slots | 108 seconds | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3 (= 8)` | slots | 96 seconds | ### Helpers @@ -174,14 +174,15 @@ def on_tick(store: Store, time: uint64) -> None: store.time = time current_slot = get_current_slot(store) - # not a new epoch, return + # Not a new epoch, return if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): return - # if new epoch and there are queued_justified_checkpoints, update if any is better than the best in store + # If new epoch and there are queued_justified_checkpoints, update if any is better than the best in store if any(store.queued_justified_checkpoints): best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: store.justified_checkpoint = best_justified_checkpoint + store.queued_justified_checkpoints = [] ``` #### `on_block` From ba6637b4d98fbfcec9721d813ae9d3deb8445436 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:42:40 -0700 Subject: [PATCH 11/33] remove extra var --- specs/core/0_fork-choice.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 2e8deffdb..edd8a52fa 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -148,8 +148,6 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: - current_epoch = compute_epoch_at_slot(get_current_slot(store)) - if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True From 26162106379e385b0684c49abe856194de9c479d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 12:51:47 -0700 Subject: [PATCH 12/33] minor fmt --- specs/core/0_fork-choice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index edd8a52fa..e08ea09b6 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -47,7 +47,7 @@ The head block root associated with a `store` is defined as `get_head(store)`. A | Name | Value | Unit | Duration | | - | - | :-: | :-: | -| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3 (= 8)` | slots | 96 seconds | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3` (= 8) | slots | 96 seconds | ### Helpers From 4f42f63e4e6cb8dbc0c7237dbfe0e4fee8b660c1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:35:58 -0700 Subject: [PATCH 13/33] only allow attestatiosn to be considered from current and previous epoch --- specs/core/0_fork-choice.md | 3 ++ .../test/fork_choice/test_on_attestation.py | 52 +++++++++++++++++-- 2 files changed, 50 insertions(+), 5 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index e08ea09b6..09c2fd31c 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -224,6 +224,9 @@ def on_block(store: Store, block: BeaconBlock) -> None: def on_attestation(store: Store, attestation: Attestation) -> None: target = attestation.data.target + # Attestations must be from the current or previous epoch + current_epoch = compute_epoch_at_slot(get_current_slot(store)) + assert target.epoch in [current_epoch, current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH] # Cannot calculate the current shuffling if have not seen the target assert target.root in store.blocks diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py index 70375ef27..3a513cf2f 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py @@ -29,10 +29,9 @@ def run_on_attestation(spec, state, store, attestation, valid=True): @with_all_phases @spec_state_test -def test_on_attestation(spec, state): +def test_on_attestation_current_epoch(spec, state): store = spec.get_genesis_store(state) - time = 100 - spec.on_tick(store, time) + spec.on_tick(store, store.time + spec.SECONDS_PER_SLOT * 2) block = build_empty_block_for_next_slot(spec, state) state_transition_and_sign_block(spec, state, block) @@ -41,9 +40,53 @@ def test_on_attestation(spec, state): spec.on_block(store, block) attestation = get_valid_attestation(spec, state, slot=block.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + run_on_attestation(spec, state, store, attestation) +@with_all_phases +@spec_state_test +def test_on_attestation_previous_epoch(spec, state): + store = spec.get_genesis_store(state) + spec.on_tick(store, store.time + spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH) + + block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, block) + + # store block in store + spec.on_block(store, block) + + attestation = get_valid_attestation(spec, state, slot=block.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + 1 + + run_on_attestation(spec, state, store, attestation) + + +@with_all_phases +@spec_state_test +def test_on_attestation_past_epoch(spec, state): + store = spec.get_genesis_store(state) + + # move time forward 2 epochs + time = store.time + 2 * spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + spec.on_tick(store, time) + + # create and store block from 3 epochs ago + block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, block) + spec.on_block(store, block) + + # create attestation for past block + attestation = get_valid_attestation(spec, state, slot=state.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + 2 + + run_on_attestation(spec, state, store, attestation, False) + + @with_all_phases @spec_state_test def test_on_attestation_target_not_in_store(spec, state): @@ -77,8 +120,7 @@ def test_on_attestation_future_epoch(spec, state): spec.on_block(store, block) # move state forward but not store - attestation_slot = block.slot + spec.SLOTS_PER_EPOCH - state.slot = attestation_slot + state.slot = block.slot + spec.SLOTS_PER_EPOCH attestation = get_valid_attestation(spec, state, slot=state.slot) run_on_attestation(spec, state, store, attestation, False) From fc40bff2a8298b717fd5ceb6fa3d3a02d83a2659 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:10:32 -0700 Subject: [PATCH 14/33] use best_justified_checkpoint instead of queued_justified_checkpoints --- specs/core/0_fork-choice.md | 26 +++++------ .../test/fork_choice/test_on_block.py | 2 +- .../eth2spec/test/fork_choice/test_on_tick.py | 43 +++++-------------- 3 files changed, 25 insertions(+), 46 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index e08ea09b6..9e38fd548 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -69,7 +69,7 @@ class Store(object): genesis_time: uint64 justified_checkpoint: Checkpoint finalized_checkpoint: Checkpoint - queued_justified_checkpoints: List[Checkpoint, 2**40] = field(default_factory=list) + best_justified_checkpoint: Checkpoint blocks: Dict[Hash, BeaconBlock] = field(default_factory=dict) block_states: Dict[Hash, BeaconState] = field(default_factory=dict) checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) @@ -89,6 +89,7 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: genesis_time=genesis_state.genesis_time, justified_checkpoint=justified_checkpoint, finalized_checkpoint=finalized_checkpoint, + best_justified_checkpoint=justified_checkpoint, blocks={root: genesis_block}, block_states={root: genesis_state.copy()}, checkpoint_states={justified_checkpoint: genesis_state.copy()}, @@ -147,14 +148,17 @@ def get_head(store: Store) -> Hash: #### `should_update_justified_checkpoint` ```python -def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: +def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True - justified_block = store.blocks[justified_checkpoint.root] - if justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): + new_justified_block = store.blocks[new_justified_checkpoint.root] + if new_justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): return False - if not get_ancestor(store, justified_checkpoint.root, store.blocks[justified_checkpoint.root].slot): + if not ( + get_ancestor(store, new_justified_checkpoint.root, store.blocks[store.justified_checkpoint.root].slot) == + store.justified_checkpoint.root + ): return False return True @@ -175,12 +179,9 @@ def on_tick(store: Store, time: uint64) -> None: # Not a new epoch, return if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): return - # If new epoch and there are queued_justified_checkpoints, update if any is better than the best in store - if any(store.queued_justified_checkpoints): - best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) - if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: - store.justified_checkpoint = best_justified_checkpoint - store.queued_justified_checkpoints = [] + # Update store.justified_checkpoint if a better checkpoint is know + if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.justified_checkpoint = store.best_justified_checkpoint ``` #### `on_block` @@ -208,10 +209,9 @@ def on_block(store: Store, block: BeaconBlock) -> None: # Update justified checkpoint if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.best_justified_checkpoint = state.current_justified_checkpoint if should_update_justified_checkpoint(store, state.current_justified_checkpoint): store.justified_checkpoint = state.current_justified_checkpoint - else: - store.queued_justified_checkpoints.append(state.current_justified_checkpoint) # Update finalized checkpoint if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch: diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py index 36e61f0b5..e42cb1ac4 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py @@ -202,4 +202,4 @@ def test_on_block_outside_safe_slots_and_old_block(spec, state): run_on_block(spec, store, block) assert store.justified_checkpoint != new_justified - assert store.queued_justified_checkpoints[0] == new_justified + assert store.best_justified_checkpoint == new_justified diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py index 2ba89ebca..77222f65c 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -1,7 +1,7 @@ from eth2spec.test.context import with_all_phases, spec_state_test -def run_on_tick(spec, store, time, new_justified_checkpoint=None): +def run_on_tick(spec, store, time, new_justified_checkpoint=False): previous_justified_checkpoint = store.justified_checkpoint spec.on_tick(store, time) @@ -9,8 +9,9 @@ def run_on_tick(spec, store, time, new_justified_checkpoint=None): assert store.time == time if new_justified_checkpoint: - assert store.justified_checkpoint == new_justified_checkpoint + assert store.justified_checkpoint == store.best_justified_checkpoint assert store.justified_checkpoint.epoch > previous_justified_checkpoint.epoch + assert store.justified_checkpoint.root != previous_justified_checkpoint.root else: assert store.justified_checkpoint == previous_justified_checkpoint @@ -28,30 +29,12 @@ def test_update_justified_single(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) - - -@with_all_phases -@spec_state_test -def test_update_justified_multiple(spec, state): - store = spec.get_genesis_store(state) - seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - - new_justified = None # remember checkpoint with latest epoch - for i in range(3): - new_justified = spec.Checkpoint( - epoch=store.justified_checkpoint.epoch + i + 1, - root=i.to_bytes(1, byteorder='big') * 32, - ) - store.queued_justified_checkpoints.append(new_justified) - - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + run_on_tick(spec, store, store.time + seconds_per_epoch, True) @with_all_phases @@ -60,11 +43,10 @@ def test_no_update_same_slot_at_epoch_boundary(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) # set store time to already be at epoch boundary store.time = seconds_per_epoch @@ -77,11 +59,10 @@ def test_no_update_same_slot_at_epoch_boundary(spec, state): def test_no_update_not_epoch_boundary(spec, state): store = spec.get_genesis_store(state) - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) run_on_tick(spec, store, store.time + spec.SECONDS_PER_SLOT) @@ -92,14 +73,13 @@ def test_no_update_new_justified_equal_epoch(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) store.justified_checkpoint = spec.Checkpoint( - epoch=new_justified.epoch, + epoch=store.best_justified_checkpoint.epoch, root=b'\44' * 32, ) @@ -112,14 +92,13 @@ def test_no_update_new_justified_later_epoch(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) store.justified_checkpoint = spec.Checkpoint( - epoch=new_justified.epoch + 1, + epoch=store.best_justified_checkpoint.epoch + 1, root=b'\44' * 32, ) From 09fd49ce89f07728a267bebc9bc3a63cc9428e94 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:20:21 -0700 Subject: [PATCH 15/33] use helper for slots since epoch start --- specs/core/0_fork-choice.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 9e38fd548..ac1de0b3d 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -96,11 +96,20 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: ) ``` +#### `get_current_slot` + ```python def get_current_slot(store: Store) -> Slot: return Slot((store.time - store.genesis_time) // SECONDS_PER_SLOT) ``` +#### `compute_slots_since_epoch_start` + +```python +def compute_slots_since_epoch_start(slot: Slot) -> int: + return slot - compute_start_slot_at_epoch(compute_epoch_at_slot(slot)) +``` + #### `get_ancestor` ```python @@ -149,7 +158,7 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: - if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True new_justified_block = store.blocks[new_justified_checkpoint.root] From a28c02794319d3b7cf6a1c96f541f53b647b151b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:26:06 -0700 Subject: [PATCH 16/33] be explicit about use of genesis epoch for previous epoch in fork choice on_block --- specs/core/0_fork-choice.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 34948fbce..68e681fc7 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -233,9 +233,11 @@ def on_block(store: Store, block: BeaconBlock) -> None: def on_attestation(store: Store, attestation: Attestation) -> None: target = attestation.data.target - # Attestations must be from the current or previous epoch + # Attestations must be from the current or previous epoch current_epoch = compute_epoch_at_slot(get_current_slot(store)) - assert target.epoch in [current_epoch, current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH] + # Use GENESIS_EPOCH for previous when genesis to avoid underflow + previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH + assert target.epoch in [current_epoch, previous_epoch] # Cannot calculate the current shuffling if have not seen the target assert target.root in store.blocks From 9ddae15e2e475602bd5e45d5cef96996d3740e01 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 20:32:25 -0700 Subject: [PATCH 17/33] reduce genesis validator count for mainnet config --- configs/mainnet.yaml | 4 ++-- specs/core/0_beacon-chain.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index af446d575..4096b43b8 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -17,8 +17,8 @@ MIN_PER_EPOCH_CHURN_LIMIT: 4 CHURN_LIMIT_QUOTIENT: 65536 # See issue 563 SHUFFLE_ROUND_COUNT: 90 -# `2**16` (= 65,536) -MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 65536 +# `2**14` (= 16,384) +MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 16384 # Jan 3, 2020 MIN_GENESIS_TIME: 1578009600 diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 540d8192d..5707c6157 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -171,7 +171,7 @@ The following values are (non-configurable) constants used throughout the specif | `MIN_PER_EPOCH_CHURN_LIMIT` | `2**2` (= 4) | | `CHURN_LIMIT_QUOTIENT` | `2**16` (= 65,536) | | `SHUFFLE_ROUND_COUNT` | `90` | -| `MIN_GENESIS_ACTIVE_VALIDATOR_COUNT` | `2**16` (= 65,536) | +| `MIN_GENESIS_ACTIVE_VALIDATOR_COUNT` | `2**14` (= 16,384) | | `MIN_GENESIS_TIME` | `1578009600` (Jan 3, 2020) | - For the safety of committees, `TARGET_COMMITTEE_SIZE` exceeds [the recommended minimum committee size of 111](https://vitalik.ca/files/Ithaca201807_Sharding.pdf); with sufficient active validators (at least `SLOTS_PER_EPOCH * TARGET_COMMITTEE_SIZE`), the shuffling algorithm ensures committee sizes of at least `TARGET_COMMITTEE_SIZE`. (Unbiasable randomness with a Verifiable Delay Function (VDF) will improve committee robustness and lower the safe minimum committee size.) From bf78a711524e964dc616c10cbd3ca561d050b830 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 11:51:53 -0700 Subject: [PATCH 18/33] pr feedback --- specs/core/0_fork-choice.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index ac1de0b3d..468d1e3d6 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -158,6 +158,13 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: + """ + To address the bouncing attack, only update conflicting justified + checkpoints in the fork choice if in the early slots of the epoch. + Otherwise, delay incorporation of new justified checkpoint until next epoch boundary. + + See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion. + """ if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True @@ -186,9 +193,9 @@ def on_tick(store: Store, time: uint64) -> None: current_slot = get_current_slot(store) # Not a new epoch, return - if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): + if not (current_slot > previous_slot and compute_slots_since_epoch_start(current_slot) == 0): return - # Update store.justified_checkpoint if a better checkpoint is know + # Update store.justified_checkpoint if a better checkpoint is known if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: store.justified_checkpoint = store.best_justified_checkpoint ``` From 9b21c0db933679eb381aa5ab4105f4445a9607ff Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:06:23 -0700 Subject: [PATCH 19/33] add note aboutgenesis attestations --- specs/validator/0_beacon-chain-validator.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 166534031..fa7fce8ea 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -291,6 +291,8 @@ A validator is expected to create, sign, and broadcast an attestation during eac A validator should create and broadcast the `attestation` to the associated attestation subnet one-third of the way through the `slot` during which the validator is assigned―that is, `SECONDS_PER_SLOT / 3` seconds after the start of `slot`. +*Note*: Although attestations during `GENESIS_EPOCH` do not count toward FFG finality, these initial attestations do give weight to the fork choice, are rewarded fork, and should be made. + #### Attestation data First, the validator should construct `attestation_data`, an [`AttestationData`](../core/0_beacon-chain.md#attestationdata) object based upon the state at the assigned slot. From 5159c69632a86f63c50abd0516ca5d8548dc7069 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:15:56 -0700 Subject: [PATCH 20/33] cleanup get_eth1_vote --- specs/validator/0_beacon-chain-validator.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f6c3b55da..a985047b6 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -236,11 +236,13 @@ def get_eth1_vote(state: BeaconState, previous_eth1_distance: uint64) -> Eth1Dat new_eth1_data = [get_eth1_data(distance) for distance in range(ETH1_FOLLOW_DISTANCE, 2 * ETH1_FOLLOW_DISTANCE)] all_eth1_data = [get_eth1_data(distance) for distance in range(ETH1_FOLLOW_DISTANCE, previous_eth1_distance)] - valid_votes = [] - for slot, vote in enumerate(state.eth1_data_votes): - period_tail = slot % SLOTS_PER_ETH1_VOTING_PERIOD >= integer_squareroot(SLOTS_PER_ETH1_VOTING_PERIOD) - if vote in new_eth1_data or (period_tail and vote in all_eth1_data): - valid_votes.append(vote) + period_tail = state.slot % SLOTS_PER_ETH1_VOTING_PERIOD >= integer_squareroot(SLOTS_PER_ETH1_VOTING_PERIOD) + if period_tail: + votes_to_consider = all_eth1_data + else: + votes_to_consider = new_eth1_data + + valid_votes = [vote for vote in state.eth1_data_votes if vote in votes_to_consider] return max( valid_votes, From c84dce7b32985befb27b2871b9a7fba278774aa7 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:32:02 -0700 Subject: [PATCH 21/33] make eth1_follow_distance clearer --- specs/validator/0_beacon-chain-validator.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f6c3b55da..86cd1e083 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -229,7 +229,9 @@ def get_epoch_signature(state: BeaconState, block: BeaconBlock, privkey: int) -> The `block.eth1_data` field is for block proposers to vote on recent Eth1 data. This recent data contains an Eth1 block hash as well as the associated deposit root (as calculated by the `get_deposit_root()` method of the deposit contract) and deposit count after execution of the corresponding Eth1 block. If over half of the block proposers in the current Eth1 voting period vote for the same `eth1_data` then `state.eth1_data` updates at the end of the voting period. Each deposit in `block.body.deposits` must verify against `state.eth1_data.eth1_deposit_root`. -Let `get_eth1_data(distance: uint64) -> Eth1Data` be the (subjective) function that returns the Eth1 data at distance `distance` relative to the Eth1 head at the start of the current Eth1 voting period. Let `previous_eth1_distance` be the distance relative to the Eth1 block corresponding to `state.eth1_data.block_hash` at the start of the current Eth1 voting period. An honest block proposer sets `block.eth1_data = get_eth1_vote(state, previous_eth1_distance)` where: +Let `get_eth1_data(distance: uint64) -> Eth1Data` be the (subjective) function that returns the Eth1 data at distance `distance` relative to the Eth1 head at the start of the current Eth1 voting period. Let `previous_eth1_distance` be the distance relative to the Eth1 block corresponding to `eth1_data.block_hash` found in the state at the _start_ of the current Eth1 voting period. Note that `eth1_data` can be updated in the middle of a voting period and thus the starting `eth1_data.block_hash` must be stored separately. + +An honest block proposer sets `block.eth1_data = get_eth1_vote(state, previous_eth1_distance)` where: ```python def get_eth1_vote(state: BeaconState, previous_eth1_distance: uint64) -> Eth1Data: From a029db34102b81bad246007b5b8eba632ea688eb Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Fri, 8 Nov 2019 19:05:14 +0800 Subject: [PATCH 22/33] Update the expected proposer period Since `SECONDS_PER_SLOT` is now `12` --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 38ae343fd..819f1ac98 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -195,7 +195,7 @@ A validator has two primary responsibilities to the beacon chain: [proposing blo A validator is expected to propose a [`BeaconBlock`](../core/0_beacon-chain.md#beaconblock) at the beginning of any slot during which `is_proposer(state, validator_index)` returns `True`. To propose, the validator selects the `BeaconBlock`, `parent`, that in their view of the fork choice is the head of the chain during `slot - 1`. The validator creates, signs, and broadcasts a `block` that is a child of `parent` that satisfies a valid [beacon chain state transition](../core/0_beacon-chain.md#beacon-chain-state-transition-function). -There is one proposer per slot, so if there are N active validators any individual validator will on average be assigned to propose once per N slots (e.g. at 312,500 validators = 10 million ETH, that's once per ~3 weeks). +There is one proposer per slot, so if there are N active validators any individual validator will on average be assigned to propose once per N slots (e.g. at 312,500 validators = 10 million ETH, that's once per ~6 weeks). #### Block header From b376a1387c6386135bdfcd28c916ee69d9b1f48f Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 8 Nov 2019 11:34:14 -0700 Subject: [PATCH 23/33] minor fix to comment in mainnet config --- configs/mainnet.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index e1b0faa3c..a9459fb64 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -58,7 +58,7 @@ BLS_WITHDRAWAL_PREFIX: 0x00 # --------------------------------------------------------------- # 12 seconds SECONDS_PER_SLOT: 12 -# 2**0 (= 1) slots 6 seconds +# 2**0 (= 1) slots 12 seconds MIN_ATTESTATION_INCLUSION_DELAY: 1 # 2**5 (= 32) slots 6.4 minutes SLOTS_PER_EPOCH: 32 From 931ad45c538384fa28b74cf1f49dcd3d523d8397 Mon Sep 17 00:00:00 2001 From: terence tsao Date: Thu, 31 Oct 2019 21:31:08 -0700 Subject: [PATCH 24/33] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 540d8192d..4a952a2c6 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -202,8 +202,8 @@ The following values are (non-configurable) constants used throughout the specif | `SLOTS_PER_EPOCH` | `2**5` (= 32) | slots | 6.4 minutes | | `MIN_SEED_LOOKAHEAD` | `2**0` (= 1) | epochs | 6.4 minutes | | `MAX_SEED_LOOKAHEAD` | `2**2` (= 4) | epochs | 25.6 minutes | -| `SLOTS_PER_ETH1_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours | -| `SLOTS_PER_HISTORICAL_ROOT` | `2**13` (= 8,192) | slots | ~13 hours | +| `SLOTS_PER_ETH1_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~3.4 hours | +| `SLOTS_PER_HISTORICAL_ROOT` | `2**13` (= 8,192) | slots | ~27 hours | | `MIN_VALIDATOR_WITHDRAWABILITY_DELAY` | `2**8` (= 256) | epochs | ~27 hours | | `PERSISTENT_COMMITTEE_PERIOD` | `2**11` (= 2,048) | epochs | 9 days | | `MIN_EPOCHS_TO_INACTIVITY_PENALTY` | `2**2` (= 4) | epochs | 25.6 minutes | From 32e6623915959ef25e1dc7f4d1c70528c8c0c7dc Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 11 Nov 2019 14:27:46 -0700 Subject: [PATCH 25/33] clarify topic subscriptions for attestations --- specs/validator/0_beacon-chain-validator.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 819f1ac98..aff2ba919 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -183,9 +183,13 @@ def is_proposer(state: BeaconState, The beacon chain shufflings are designed to provide a minimum of 1 epoch lookahead on the validator's upcoming committee assignments for attesting dictated by the shuffling and slot. Note that this lookahead does not apply to proposing, which must be checked during the epoch in question. -`get_committee_assignment` should be called at the start of each epoch to get the assignment for the next epoch (`current_epoch + 1`). A validator should plan for future assignments by noting at which future slot they will have to attest. +`get_committee_assignment` should be called at the start of each epoch to get the assignment for the next epoch (`current_epoch + 1`). A validator should plan for future assignments by noting at which future slot they will have to attest and joining the committee index attestation subnet related to their committee assignment. -Specifically, a validator should call `get_committee_assignment(state, next_epoch, validator_index)` when checking for next epoch assignments. +Specifically a validator should: +* Call `get_committee_assignment(state, next_epoch, validator_index)` when checking for next epoch assignments. +* Join the pubsub topic -- `committee_index{committee_index % ATTESTATION_SUBNET_COUNT}_beacon_attestation`. + * If any current peers are subscribed to the topic, the validator simply sends `subscribe` messages for the new topic. + * If no current peers are subscribed to the topic, the validator must discover new peers on this topic. If "topic discovery" is available, use topic discovery to find peers that advertise subscription to the topic. If not, "guess and check" by connecting with a number of random new peers, persisting connections with peers subscribed to the topic and (potentially) dropping the new peers otherwise. ## Beacon chain responsibilities From 29b073bcca4cfea775ddfe94bbcdafc27332a551 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 11 Nov 2019 14:46:30 -0700 Subject: [PATCH 26/33] add random subnet subscriptions for phase 0 stability --- specs/validator/0_beacon-chain-validator.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index aff2ba919..1c316e77c 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -55,6 +55,7 @@ - [Aggregate signature](#aggregate-signature-1) - [Broadcast aggregate](#broadcast-aggregate) - [`AggregateAndProof`](#aggregateandproof) + - [Phase 0 attestation subnet stability](#phase-0-attestation-subnet-stability) - [How to avoid slashing](#how-to-avoid-slashing) - [Proposer slashing](#proposer-slashing) - [Attester slashing](#attester-slashing) @@ -79,6 +80,8 @@ All terminology, constants, functions, and protocol mechanics defined in the [Ph | - | - | :-: | :-: | | `ETH1_FOLLOW_DISTANCE` | `2**10` (= 1,024) | blocks | ~4 hours | | `TARGET_AGGREGATORS_PER_COMMITTEE` | `2**4` (= 16) | validators | | +| `RANDOM_SUBNETS_PER_VALIDATOR` | `2**0` (= 1) | subnets | | +| `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION` | `2**8` (= 256) | epochs | ~27 hours | ## Becoming a validator @@ -418,6 +421,10 @@ Where * `selection_proof` is the signature of the slot (`slot_signature()`). * `aggregate` is the `aggregate_attestation` constructed in the previous section. +## Phase 0 attestation subnet stability + +Because Phase 0 does not have shards and thus does not have Shard Committees, there is no stable backbone to the attestation subnets (`committee_index{subnet_id}_beacon_attestation`). To provide this stability, each validator must randomly select and remain subscribed to `RANDOM_SUBNETS_PER_VALIDATOR` attestation subnets. The lifetime of each random subscription must be _at least_ `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION`. + ## How to avoid slashing "Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed: [proposer slashing](#proposer-slashing) and [attester slashing](#attester-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages a validator has previously signed. From 398b11c26c83cb37d746eaddb97b31db58227ac2 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 12 Nov 2019 08:42:57 -0700 Subject: [PATCH 27/33] add validator constants to config --- configs/mainnet.yaml | 12 ++++++++++++ configs/minimal.yaml | 12 ++++++++++++ 2 files changed, 24 insertions(+) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index a9459fb64..2fdc10648 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -29,6 +29,18 @@ MIN_GENESIS_TIME: 1578009600 SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 8 +# Validator +# --------------------------------------------------------------- +# 2**10 (= 1,024) +ETH1_FOLLOW_DISTANCE: 1024 +# 2**4 (= 16) +TARGET_AGGREGATORS_PER_COMMITTEE: 16 +# 2**0 (= 1) +RANDOM_SUBNETS_PER_VALIDATOR: 1 +# 2**8 (= 256) +EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION: 256 + + # Deposit contract # --------------------------------------------------------------- # **TBD** diff --git a/configs/minimal.yaml b/configs/minimal.yaml index fbc961ab1..7adc82eae 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -28,6 +28,18 @@ MIN_GENESIS_TIME: 1578009600 # 2**1 (= 1) SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 2 +# +# Validator +# --------------------------------------------------------------- +# [customized] process deposits more quickly, but insecure +ETH1_FOLLOW_DISTANCE: 16 +# 2**4 (= 16) +TARGET_AGGREGATORS_PER_COMMITTEE: 16 +# 2**0 (= 1) +RANDOM_SUBNETS_PER_VALIDATOR: 1 +# 2**8 (= 256) +EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION: 256 + # Deposit contract # --------------------------------------------------------------- From 74253bdbafe8fb14324e801e775570885230e369 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 12 Nov 2019 11:29:46 -0700 Subject: [PATCH 28/33] attestations for future blocks not valid in fork choice --- specs/core/0_fork-choice.md | 7 ++ .../test/fork_choice/test_on_attestation.py | 69 ++++++++++++++++--- .../pyspec/eth2spec/test/helpers/state.py | 10 +++ 3 files changed, 76 insertions(+), 10 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 051c6ef8d..88cc84913 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -248,10 +248,17 @@ def on_attestation(store: Store, attestation: Attestation) -> None: # Cannot calculate the current shuffling if have not seen the target assert target.root in store.blocks + # Attestations target be for a known block. If target block is unknown, delay consideration until the block is found + assert target.root in store.blocks # Attestations cannot be from future epochs. If they are, delay consideration until the epoch arrives base_state = store.block_states[target.root].copy() assert store.time >= base_state.genesis_time + compute_start_slot_at_epoch(target.epoch) * SECONDS_PER_SLOT + # Attestations must be for a known block. If block is unknown, delay consideration until the block is found + assert attestation.data.beacon_block_root in store.blocks + # Attestations must not be for blocks in the future. If not, the attestation should not be considered + assert store.blocks[attestation.data.beacon_block_root].slot <= attestation.data.slot + # Store target checkpoint state if not yet seen if target not in store.checkpoint_states: process_slots(base_state, compute_start_slot_at_epoch(target.epoch)) diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py index 8db55cce8..50e3fc070 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py @@ -1,8 +1,7 @@ - from eth2spec.test.context import with_all_phases, spec_state_test from eth2spec.test.helpers.block import build_empty_block_for_next_slot -from eth2spec.test.helpers.attestations import get_valid_attestation -from eth2spec.test.helpers.state import state_transition_and_sign_block +from eth2spec.test.helpers.attestations import get_valid_attestation, sign_attestation +from eth2spec.test.helpers.state import transition_to, state_transition_and_sign_block def run_on_attestation(spec, state, store, attestation, valid=True): @@ -89,18 +88,48 @@ def test_on_attestation_past_epoch(spec, state): @spec_state_test def test_on_attestation_target_not_in_store(spec, state): store = spec.get_genesis_store(state) - time = 100 + time = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH spec.on_tick(store, time) - # move to next epoch to make block new target - state.slot += spec.SLOTS_PER_EPOCH + # move to immediately before next epoch to make block new target + transition_to(spec, state, state.slot + spec.SLOTS_PER_EPOCH - 1) - block = build_empty_block_for_next_slot(spec, state) - state_transition_and_sign_block(spec, state, block) + target_block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, target_block) - # do not add block to store + # do not add target block to store + + attestation = get_valid_attestation(spec, state, slot=target_block.slot) + assert attestation.data.target.root == target_block.signing_root() + + run_on_attestation(spec, state, store, attestation, False) + + +@with_all_phases +@spec_state_test +def test_on_attestation_beacon_block_not_in_store(spec, state): + store = spec.get_genesis_store(state) + time = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + spec.on_tick(store, time) + + # move to immediately before next epoch to make block new target + transition_to(spec, state, state.slot + spec.SLOTS_PER_EPOCH - 1) + + target_block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, target_block) + + # store target in store + spec.on_block(store, target_block) + + head_block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, head_block) + + # do not add head block to store + + attestation = get_valid_attestation(spec, state, slot=head_block.slot) + assert attestation.data.target.root == target_block.signing_root() + assert attestation.data.beacon_block_root == head_block.signing_root() - attestation = get_valid_attestation(spec, state, slot=block.slot) run_on_attestation(spec, state, store, attestation, False) @@ -124,6 +153,26 @@ def test_on_attestation_future_epoch(spec, state): run_on_attestation(spec, state, store, attestation, False) +@with_all_phases +@spec_state_test +def test_on_attestation_future_block(spec, state): + store = spec.get_genesis_store(state) + time = spec.SECONDS_PER_SLOT * 5 + spec.on_tick(store, time) + + block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, block) + + spec.on_block(store, block) + + # attestation for slot immediately prior to the block being attested to + attestation = get_valid_attestation(spec, state, slot=block.slot - 1, signed=False) + attestation.data.beacon_block_root = block.signing_root() + sign_attestation(spec, state, attestation) + + run_on_attestation(spec, state, store, attestation, False) + + @with_all_phases @spec_state_test def test_on_attestation_same_slot(spec, state): diff --git a/test_libs/pyspec/eth2spec/test/helpers/state.py b/test_libs/pyspec/eth2spec/test/helpers/state.py index 27e946cbb..a26621c84 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/state.py +++ b/test_libs/pyspec/eth2spec/test/helpers/state.py @@ -14,6 +14,16 @@ def next_slot(spec, state): spec.process_slots(state, state.slot + 1) +def transition_to(spec, state, slot): + """ + Transition to ``slot``. + """ + assert state.slot <= slot + for _ in range(slot - state.slot): + next_slot(spec, state) + assert state.slot == slot + + def next_epoch(spec, state): """ Transition to the start slot of the next epoch From e3a67fe52c397477097e18bc3570e3dce4ce7b16 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 12 Nov 2019 12:00:30 -0700 Subject: [PATCH 29/33] ensure not all validators switch at once --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 1c316e77c..1434f0e87 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -423,7 +423,7 @@ Where ## Phase 0 attestation subnet stability -Because Phase 0 does not have shards and thus does not have Shard Committees, there is no stable backbone to the attestation subnets (`committee_index{subnet_id}_beacon_attestation`). To provide this stability, each validator must randomly select and remain subscribed to `RANDOM_SUBNETS_PER_VALIDATOR` attestation subnets. The lifetime of each random subscription must be _at least_ `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION`. +Because Phase 0 does not have shards and thus does not have Shard Committees, there is no stable backbone to the attestation subnets (`committee_index{subnet_id}_beacon_attestation`). To provide this stability, each validator must randomly select and remain subscribed to `RANDOM_SUBNETS_PER_VALIDATOR` attestation subnets. The lifetime of each random subscription should be a random number of epochs between `EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION` and `2 * EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION]`. ## How to avoid slashing From c061abdd75cc36e4eb2ada19bdc42b1495c74490 Mon Sep 17 00:00:00 2001 From: protolambda Date: Tue, 12 Nov 2019 21:29:58 +0100 Subject: [PATCH 30/33] Avoid Hash <> hash() type and func confusion, intro Root and change to non-roots to Bytes32 --- scripts/build_spec.py | 14 ++--- specs/core/0_beacon-chain.md | 62 +++++++++---------- specs/core/0_fork-choice.md | 12 ++-- specs/core/1_beacon-chain-misc.md | 6 +- specs/core/1_custody-game.md | 20 +++--- specs/core/1_shard-data-chains.md | 18 +++--- specs/light_client/merkle_proofs.md | 20 +++--- specs/light_client/sync_protocol.md | 6 +- .../pyspec/eth2spec/test/helpers/block.py | 2 +- .../block_processing/test_process_deposit.py | 2 +- .../eth2spec/test/sanity/test_blocks.py | 4 +- .../eth2spec/utils/ssz/test_ssz_typing.py | 10 +-- 12 files changed, 88 insertions(+), 88 deletions(-) diff --git a/scripts/build_spec.py b/scripts/build_spec.py index 9ed9d3ae4..549357cd7 100644 --- a/scripts/build_spec.py +++ b/scripts/build_spec.py @@ -79,26 +79,26 @@ def ceillog2(x: uint64) -> int: SUNDRY_FUNCTIONS = ''' # Monkey patch hash cache _hash = hash -hash_cache: Dict[bytes, Hash] = {} +hash_cache: Dict[bytes, Bytes32] = {} -def get_eth1_data(distance: uint64) -> Hash: +def get_eth1_data(distance: uint64) -> Bytes32: return hash(distance) -def hash(x: bytes) -> Hash: +def hash(x: bytes) -> Bytes32: if x not in hash_cache: - hash_cache[x] = Hash(_hash(x)) + hash_cache[x] = Bytes32(_hash(x)) return hash_cache[x] # Monkey patch validator compute committee code _compute_committee = compute_committee -committee_cache: Dict[Tuple[Hash, Hash, int, int], Sequence[ValidatorIndex]] = {} +committee_cache: Dict[Tuple[Bytes32, Bytes32, int, int], Sequence[ValidatorIndex]] = {} def compute_committee(indices: Sequence[ValidatorIndex], # type: ignore - seed: Hash, + seed: Bytes32, index: int, count: int) -> Sequence[ValidatorIndex]: param_hash = (hash(b''.join(index.to_bytes(length=4, byteorder='little') for index in indices)), seed, index, count) @@ -210,7 +210,7 @@ def combine_constants(old_constants: Dict[str, str], new_constants: Dict[str, st ignored_dependencies = [ - 'bit', 'boolean', 'Vector', 'List', 'Container', 'Hash', 'BLSPubkey', 'BLSSignature', 'Bytes', 'BytesN' + 'bit', 'boolean', 'Vector', 'List', 'Container', 'Root', 'BLSPubkey', 'BLSSignature', 'Bytes', 'BytesN' 'Bytes1', 'Bytes4', 'Bytes32', 'Bytes48', 'Bytes96', 'Bitlist', 'Bitvector', 'uint8', 'uint16', 'uint32', 'uint64', 'uint128', 'uint256', 'bytes', 'byte', 'BytesN' # to be removed after updating spec doc diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4a952a2c6..f7c24e9bc 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -137,7 +137,7 @@ We define the following Python custom types for type hinting and readability: | `CommitteeIndex` | `uint64` | a committee index at a slot | | `ValidatorIndex` | `uint64` | a validator registry index | | `Gwei` | `uint64` | an amount in Gwei | -| `Hash` | `Bytes32` | a hash | +| `Root` | `Bytes32` | a Merkle root | | `Version` | `Bytes4` | a fork version number | | `DomainType` | `Bytes4` | a domain type | | `Domain` | `Bytes8` | a signature domain | @@ -275,7 +275,7 @@ class Fork(Container): ```python class Checkpoint(Container): epoch: Epoch - root: Hash + root: Root ``` #### `Validator` @@ -283,7 +283,7 @@ class Checkpoint(Container): ```python class Validator(Container): pubkey: BLSPubkey - withdrawal_credentials: Hash # Commitment to pubkey for withdrawals + withdrawal_credentials: Bytes32 # Commitment to pubkey for withdrawals effective_balance: Gwei # Balance at stake slashed: boolean # Status epochs @@ -300,7 +300,7 @@ class AttestationData(Container): slot: Slot index: CommitteeIndex # LMD GHOST vote - beacon_block_root: Hash + beacon_block_root: Root # FFG vote source: Checkpoint target: Checkpoint @@ -329,17 +329,17 @@ class PendingAttestation(Container): ```python class Eth1Data(Container): - deposit_root: Hash + deposit_root: Root deposit_count: uint64 - block_hash: Hash + block_hash: Bytes32 ``` #### `HistoricalBatch` ```python class HistoricalBatch(Container): - block_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT] - state_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT] + block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] ``` #### `DepositData` @@ -347,7 +347,7 @@ class HistoricalBatch(Container): ```python class DepositData(Container): pubkey: BLSPubkey - withdrawal_credentials: Hash + withdrawal_credentials: Bytes32 amount: Gwei signature: BLSSignature ``` @@ -357,9 +357,9 @@ class DepositData(Container): ```python class BeaconBlockHeader(Container): slot: Slot - parent_root: Hash - state_root: Hash - body_root: Hash + parent_root: Root + state_root: Root + body_root: Root signature: BLSSignature ``` @@ -395,7 +395,7 @@ class Attestation(Container): ```python class Deposit(Container): - proof: Vector[Hash, DEPOSIT_CONTRACT_TREE_DEPTH + 1] # Merkle path to deposit data list root + proof: Vector[Bytes32, DEPOSIT_CONTRACT_TREE_DEPTH + 1] # Merkle path to deposit data list root data: DepositData ``` @@ -430,8 +430,8 @@ class BeaconBlockBody(Container): ```python class BeaconBlock(Container): slot: Slot - parent_root: Hash - state_root: Hash + parent_root: Root + state_root: Root body: BeaconBlockBody signature: BLSSignature ``` @@ -448,9 +448,9 @@ class BeaconState(Container): fork: Fork # History latest_block_header: BeaconBlockHeader - block_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT] - state_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT] - historical_roots: List[Hash, HISTORICAL_ROOTS_LIMIT] + block_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + state_roots: Vector[Root, SLOTS_PER_HISTORICAL_ROOT] + historical_roots: List[Root, HISTORICAL_ROOTS_LIMIT] # Eth1 eth1_data: Eth1Data eth1_data_votes: List[Eth1Data, SLOTS_PER_ETH1_VOTING_PERIOD] @@ -459,7 +459,7 @@ class BeaconState(Container): validators: List[Validator, VALIDATOR_REGISTRY_LIMIT] balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT] # Randomness - randao_mixes: Vector[Hash, EPOCHS_PER_HISTORICAL_VECTOR] + randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR] # Slashings slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances # Attestations @@ -527,15 +527,15 @@ def bytes_to_int(data: bytes) -> uint64: #### `hash` -`def hash(data: bytes) -> Hash` is SHA256. +`def hash(data: bytes) -> Bytes32` is SHA256. #### `hash_tree_root` -`def hash_tree_root(object: SSZSerializable) -> Hash` is a function for hashing objects into a single root by utilizing a hash tree structure, as defined in the [SSZ spec](../simple-serialize.md#merkleization). +`def hash_tree_root(object: SSZSerializable) -> Root` is a function for hashing objects into a single root by utilizing a hash tree structure, as defined in the [SSZ spec](../simple-serialize.md#merkleization). #### `signing_root` -`def signing_root(object: Container) -> Hash` is a function for computing signing messages, as defined in the [SSZ spec](../simple-serialize.md#self-signed-containers). +`def signing_root(object: Container) -> Root` is a function for computing signing messages, as defined in the [SSZ spec](../simple-serialize.md#self-signed-containers). #### `bls_verify` @@ -611,7 +611,7 @@ def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: Indexe #### `is_valid_merkle_branch` ```python -def is_valid_merkle_branch(leaf: Hash, branch: Sequence[Hash], depth: uint64, index: uint64, root: Hash) -> bool: +def is_valid_merkle_branch(leaf: Bytes32, branch: Sequence[Bytes32], depth: uint64, index: uint64, root: Root) -> bool: """ Check if ``leaf`` at ``index`` verifies against the Merkle ``root`` and ``branch``. """ @@ -629,7 +629,7 @@ def is_valid_merkle_branch(leaf: Hash, branch: Sequence[Hash], depth: uint64, in #### `compute_shuffled_index` ```python -def compute_shuffled_index(index: ValidatorIndex, index_count: uint64, seed: Hash) -> ValidatorIndex: +def compute_shuffled_index(index: ValidatorIndex, index_count: uint64, seed: Bytes32) -> ValidatorIndex: """ Return the shuffled validator index corresponding to ``seed`` (and ``index_count``). """ @@ -652,7 +652,7 @@ def compute_shuffled_index(index: ValidatorIndex, index_count: uint64, seed: Has #### `compute_proposer_index` ```python -def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Hash) -> ValidatorIndex: +def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Bytes32) -> ValidatorIndex: """ Return from ``indices`` a random index sampled by effective balance. """ @@ -672,7 +672,7 @@ def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex] ```python def compute_committee(indices: Sequence[ValidatorIndex], - seed: Hash, + seed: Bytes32, index: uint64, count: uint64) -> Sequence[ValidatorIndex]: """ @@ -749,7 +749,7 @@ def get_previous_epoch(state: BeaconState) -> Epoch: #### `get_block_root` ```python -def get_block_root(state: BeaconState, epoch: Epoch) -> Hash: +def get_block_root(state: BeaconState, epoch: Epoch) -> Root: """ Return the block root at the start of a recent ``epoch``. """ @@ -759,7 +759,7 @@ def get_block_root(state: BeaconState, epoch: Epoch) -> Hash: #### `get_block_root_at_slot` ```python -def get_block_root_at_slot(state: BeaconState, slot: Slot) -> Hash: +def get_block_root_at_slot(state: BeaconState, slot: Slot) -> Root: """ Return the block root at a recent ``slot``. """ @@ -770,7 +770,7 @@ def get_block_root_at_slot(state: BeaconState, slot: Slot) -> Hash: #### `get_randao_mix` ```python -def get_randao_mix(state: BeaconState, epoch: Epoch) -> Hash: +def get_randao_mix(state: BeaconState, epoch: Epoch) -> Bytes32: """ Return the randao mix at a recent ``epoch``. """ @@ -801,7 +801,7 @@ def get_validator_churn_limit(state: BeaconState) -> uint64: #### `get_seed` ```python -def get_seed(state: BeaconState, epoch: Epoch, domain_type: DomainType) -> Hash: +def get_seed(state: BeaconState, epoch: Epoch, domain_type: DomainType) -> Bytes32: """ Return the seed at ``epoch``. """ @@ -996,7 +996,7 @@ Before the Ethereum 2.0 genesis has been triggered, and for every Ethereum 1.0 b - `deposits` is the sequence of all deposits, ordered chronologically, up to (and including) the block with hash `eth1_block_hash` ```python -def initialize_beacon_state_from_eth1(eth1_block_hash: Hash, +def initialize_beacon_state_from_eth1(eth1_block_hash: Bytes32, eth1_timestamp: uint64, deposits: Sequence[Deposit]) -> BeaconState: state = BeaconState( diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 051c6ef8d..4a28137fd 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -57,7 +57,7 @@ The head block root associated with a `store` is defined as `get_head(store)`. A @dataclass(eq=True, frozen=True) class LatestMessage(object): epoch: Epoch - root: Hash + root: Root ``` #### `Store` @@ -70,8 +70,8 @@ class Store(object): justified_checkpoint: Checkpoint finalized_checkpoint: Checkpoint best_justified_checkpoint: Checkpoint - blocks: Dict[Hash, BeaconBlock] = field(default_factory=dict) - block_states: Dict[Hash, BeaconState] = field(default_factory=dict) + blocks: Dict[Root, BeaconBlock] = field(default_factory=dict) + block_states: Dict[Root, BeaconState] = field(default_factory=dict) checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) latest_messages: Dict[ValidatorIndex, LatestMessage] = field(default_factory=dict) ``` @@ -113,7 +113,7 @@ def compute_slots_since_epoch_start(slot: Slot) -> int: #### `get_ancestor` ```python -def get_ancestor(store: Store, root: Hash, slot: Slot) -> Hash: +def get_ancestor(store: Store, root: Root, slot: Slot) -> Root: block = store.blocks[root] if block.slot > slot: return get_ancestor(store, block.parent_root, slot) @@ -126,7 +126,7 @@ def get_ancestor(store: Store, root: Hash, slot: Slot) -> Hash: #### `get_latest_attesting_balance` ```python -def get_latest_attesting_balance(store: Store, root: Hash) -> Gwei: +def get_latest_attesting_balance(store: Store, root: Root) -> Gwei: state = store.checkpoint_states[store.justified_checkpoint] active_indices = get_active_validator_indices(state, get_current_epoch(state)) return Gwei(sum( @@ -139,7 +139,7 @@ def get_latest_attesting_balance(store: Store, root: Hash) -> Gwei: #### `get_head` ```python -def get_head(store: Store) -> Hash: +def get_head(store: Store) -> Root: # Execute the LMD-GHOST fork choice head = store.justified_checkpoint.root justified_slot = compute_start_slot_at_epoch(store.justified_checkpoint.epoch) diff --git a/specs/core/1_beacon-chain-misc.md b/specs/core/1_beacon-chain-misc.md index 3fdafe3ea..d99aca0ef 100644 --- a/specs/core/1_beacon-chain-misc.md +++ b/specs/core/1_beacon-chain-misc.md @@ -62,7 +62,7 @@ class ShardReceiptDelta(Container): ```python class ShardReceiptProof(Container): shard: Shard - proof: List[Hash, PLACEHOLDER] + proof: List[Bytes32, PLACEHOLDER] receipt: List[ShardReceiptDelta, PLACEHOLDER] ``` @@ -109,7 +109,7 @@ def committee_to_compact_committee(state: BeaconState, committee: Sequence[Valid #### `verify_merkle_proof` ```python -def verify_merkle_proof(leaf: Hash, proof: Sequence[Hash], index: GeneralizedIndex, root: Hash) -> bool: +def verify_merkle_proof(leaf: Bytes32, proof: Sequence[Bytes32], index: GeneralizedIndex, root: Root) -> bool: assert len(proof) == get_generalized_index_length(index) for i, h in enumerate(proof): if get_generalized_index_bit(index, i): @@ -199,7 +199,7 @@ Add the following fields to the end of the specified container objects. ```python class BeaconState(Container): # Period committees - period_committee_roots: Vector[Hash, PERIOD_COMMITTEE_ROOT_LENGTH] + period_committee_roots: Vector[Root, PERIOD_COMMITTEE_ROOT_LENGTH] next_shard_receipt_period: Vector[uint64, SHARD_COUNT] ``` diff --git a/specs/core/1_custody-game.md b/specs/core/1_custody-game.md index 67e12a08c..b487d3c34 100644 --- a/specs/core/1_custody-game.md +++ b/specs/core/1_custody-game.md @@ -170,7 +170,7 @@ class CustodyChunkChallengeRecord(Container): challenger_index: ValidatorIndex responder_index: ValidatorIndex inclusion_epoch: Epoch - data_root: Hash + data_root: Root depth: uint64 chunk_index: uint64 ``` @@ -183,9 +183,9 @@ class CustodyBitChallengeRecord(Container): challenger_index: ValidatorIndex responder_index: ValidatorIndex inclusion_epoch: Epoch - data_root: Hash + data_root: Root chunk_count: uint64 - chunk_bits_merkle_root: Hash + chunk_bits_merkle_root: Root responder_key: BLSSignature ``` @@ -196,8 +196,8 @@ class CustodyResponse(Container): challenge_index: uint64 chunk_index: uint64 chunk: BytesN[BYTES_PER_CUSTODY_CHUNK] - data_branch: List[Hash, CUSTODY_DATA_DEPTH] - chunk_bits_branch: List[Hash, CUSTODY_CHUNK_BIT_DEPTH] + data_branch: List[Bytes32, CUSTODY_DATA_DEPTH] + chunk_bits_branch: List[Bytes32, CUSTODY_CHUNK_BIT_DEPTH] chunk_bits_leaf: Bitvector[256] ``` @@ -228,7 +228,7 @@ class EarlyDerivedSecretReveal(Container): # Index of the validator who revealed (whistleblower) masker_index: ValidatorIndex # Mask used to hide the actual reveal signature (prevent reveal from being stolen) - mask: Hash + mask: Bytes32 ``` ### Phase 0 container updates @@ -283,11 +283,11 @@ def ceillog2(x: uint64) -> int: ### `is_valid_merkle_branch_with_mixin` ```python -def is_valid_merkle_branch_with_mixin(leaf: Hash, - branch: Sequence[Hash], +def is_valid_merkle_branch_with_mixin(leaf: Bytes32, + branch: Sequence[Bytes32], depth: uint64, index: uint64, - root: Hash, + root: Root, mixin: uint64) -> bool: value = leaf for i in range(depth): @@ -672,7 +672,7 @@ def process_chunk_challenge_response(state: BeaconState, # Verify chunk index assert response.chunk_index == challenge.chunk_index # Verify bit challenge data is null - assert response.chunk_bits_branch == [] and response.chunk_bits_leaf == Hash() + assert response.chunk_bits_branch == [] and response.chunk_bits_leaf == Bytes32() # Verify minimum delay assert get_current_epoch(state) >= challenge.inclusion_epoch + MAX_SEED_LOOKAHEAD # Verify the chunk matches the crosslink data root diff --git a/specs/core/1_shard-data-chains.md b/specs/core/1_shard-data-chains.md index d9c88e72b..7b951dd11 100644 --- a/specs/core/1_shard-data-chains.md +++ b/specs/core/1_shard-data-chains.md @@ -118,9 +118,9 @@ class Crosslink(Container): class ShardBlock(Container): shard: Shard slot: ShardSlot - beacon_block_root: Hash - parent_root: Hash - state_root: Hash + beacon_block_root: Root + parent_root: Root + state_root: Root body: List[byte, MAX_SHARD_BLOCK_SIZE - SHARD_HEADER_SIZE] block_size_sum: uint64 aggregation_bits: Bitvector[2 * MAX_PERIOD_COMMITTEE_SIZE] @@ -134,10 +134,10 @@ class ShardBlock(Container): class ShardBlockHeader(Container): shard: Shard slot: ShardSlot - beacon_block_root: Hash - parent_root: Hash - state_root: Hash - body_root: Hash + beacon_block_root: Root + parent_root: Root + state_root: Root + body_root: Root block_size_sum: uint64 aggregation_bits: Bitvector[2 * MAX_PERIOD_COMMITTEE_SIZE] attestations: BLSSignature @@ -150,7 +150,7 @@ class ShardBlockHeader(Container): class ShardState(Container): shard: Shard slot: ShardSlot - history_accumulator: Vector[Hash, HISTORY_ACCUMULATOR_DEPTH] + history_accumulator: Vector[Bytes32, HISTORY_ACCUMULATOR_DEPTH] latest_block_header: ShardBlockHeader block_size_sum: uint64 # Fees and rewards @@ -166,7 +166,7 @@ class ShardState(Container): ```python class ShardAttestationData(Container): slot: ShardSlot - parent_root: Hash + parent_root: Root ``` ## Helper functions diff --git a/specs/light_client/merkle_proofs.md b/specs/light_client/merkle_proofs.md index 906e7d241..c89235cb5 100644 --- a/specs/light_client/merkle_proofs.md +++ b/specs/light_client/merkle_proofs.md @@ -59,9 +59,9 @@ In a binary Merkle tree, we define a "generalized index" of a node as `2**depth Note that the generalized index has the convenient property that the two children of node `k` are `2k` and `2k+1`, and also that it equals the position of a node in the linear representation of the Merkle tree that's computed by this function: ```python -def merkle_tree(leaves: Sequence[Hash]) -> Sequence[Hash]: +def merkle_tree(leaves: Sequence[Bytes32]) -> Sequence[Bytes32]: padded_length = get_next_power_of_two(len(leaves)) - o = [Hash()] * padded_length + list(leaves) + [Hash()] * (padded_length - len(leaves)) + o = [Bytes32()] * padded_length + list(leaves) + [Bytes32()] * (padded_length - len(leaves)) for i in range(padded_length - 1, 0, -1): o[i] = hash(o[i * 2] + o[i * 2 + 1]) return o @@ -289,7 +289,7 @@ def get_helper_indices(indices: Sequence[GeneralizedIndex]) -> Sequence[Generali Now we provide the Merkle proof verification functions. First, for single item proofs: ```python -def calculate_merkle_root(leaf: Hash, proof: Sequence[Hash], index: GeneralizedIndex) -> Hash: +def calculate_merkle_root(leaf: Bytes32, proof: Sequence[Bytes32], index: GeneralizedIndex) -> Root: assert len(proof) == get_generalized_index_length(index) for i, h in enumerate(proof): if get_generalized_index_bit(index, i): @@ -300,16 +300,16 @@ def calculate_merkle_root(leaf: Hash, proof: Sequence[Hash], index: GeneralizedI ``` ```python -def verify_merkle_proof(leaf: Hash, proof: Sequence[Hash], index: GeneralizedIndex, root: Hash) -> bool: +def verify_merkle_proof(leaf: Bytes32, proof: Sequence[Bytes32], index: GeneralizedIndex, root: Root) -> bool: return calculate_merkle_root(leaf, proof, index) == root ``` Now for multi-item proofs: ```python -def calculate_multi_merkle_root(leaves: Sequence[Hash], - proof: Sequence[Hash], - indices: Sequence[GeneralizedIndex]) -> Hash: +def calculate_multi_merkle_root(leaves: Sequence[Bytes32], + proof: Sequence[Bytes32], + indices: Sequence[GeneralizedIndex]) -> Root: assert len(leaves) == len(indices) helper_indices = get_helper_indices(indices) assert len(proof) == len(helper_indices) @@ -332,10 +332,10 @@ def calculate_multi_merkle_root(leaves: Sequence[Hash], ``` ```python -def verify_merkle_multiproof(leaves: Sequence[Hash], - proof: Sequence[Hash], +def verify_merkle_multiproof(leaves: Sequence[Bytes32], + proof: Sequence[Bytes32], indices: Sequence[GeneralizedIndex], - root: Hash) -> bool: + root: Root) -> bool: return calculate_multi_merkle_root(leaves, proof, indices) == root ``` diff --git a/specs/light_client/sync_protocol.md b/specs/light_client/sync_protocol.md index 8d5d44fb6..00edb990d 100644 --- a/specs/light_client/sync_protocol.md +++ b/specs/light_client/sync_protocol.md @@ -49,16 +49,16 @@ We define the following Python custom types for type hinting and readability: ```python class LightClientUpdate(container): # Shard block root (and authenticating signature data) - shard_block_root: Hash + shard_block_root: Root fork_version: Version aggregation_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE] signature: BLSSignature # Updated beacon header (and authenticating branch) header: BeaconBlockHeader - header_branch: Vector[Hash, BEACON_CHAIN_ROOT_IN_SHARD_BLOCK_HEADER_DEPTH] + header_branch: Vector[Bytes32, BEACON_CHAIN_ROOT_IN_SHARD_BLOCK_HEADER_DEPTH] # Updated period committee (and authenticating branch) committee: CompactCommittee - committee_branch: Vector[Hash, PERIOD_COMMITTEE_ROOT_IN_BEACON_STATE_DEPTH + log_2(SHARD_COUNT)] + committee_branch: Vector[Bytes32, PERIOD_COMMITTEE_ROOT_IN_BEACON_STATE_DEPTH + log_2(SHARD_COUNT)] ``` ## Helpers diff --git a/test_libs/pyspec/eth2spec/test/helpers/block.py b/test_libs/pyspec/eth2spec/test/helpers/block.py index 779f2e1cf..930311665 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/block.py +++ b/test_libs/pyspec/eth2spec/test/helpers/block.py @@ -59,7 +59,7 @@ def build_empty_block(spec, state, slot=None, signed=False): empty_block.slot = slot empty_block.body.eth1_data.deposit_count = state.eth1_deposit_index previous_block_header = deepcopy(state.latest_block_header) - if previous_block_header.state_root == spec.Hash(): + if previous_block_header.state_root == spec.Root(): previous_block_header.state_root = state.hash_tree_root() empty_block.parent_root = signing_root(previous_block_header) diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_deposit.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_deposit.py index d1ffbd7c9..1cef99394 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_deposit.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_deposit.py @@ -195,7 +195,7 @@ def test_bad_merkle_proof(spec, state): deposit = prepare_state_and_deposit(spec, state, validator_index, amount) # mess up merkle branch - deposit.proof[5] = spec.Hash() + deposit.proof[5] = spec.Bytes32() sign_deposit_data(spec, deposit.data, privkeys[validator_index], state=state) diff --git a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py index bbc4bba3a..da6acb22c 100644 --- a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py +++ b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py @@ -63,7 +63,7 @@ def test_empty_block_transition(spec, state): assert len(state.eth1_data_votes) == pre_eth1_votes + 1 assert spec.get_block_root_at_slot(state, pre_slot) == block.parent_root - assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Hash() + assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Root() @with_all_phases @@ -98,7 +98,7 @@ def test_skipped_slots(spec, state): yield 'post', state assert state.slot == block.slot - assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Hash() + assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Bytes32() for slot in range(pre_slot, state.slot): assert spec.get_block_root_at_slot(state, slot) == block.parent_root diff --git a/test_libs/pyspec/eth2spec/utils/ssz/test_ssz_typing.py b/test_libs/pyspec/eth2spec/utils/ssz/test_ssz_typing.py index d5a53c5fa..f90724782 100644 --- a/test_libs/pyspec/eth2spec/utils/ssz/test_ssz_typing.py +++ b/test_libs/pyspec/eth2spec/utils/ssz/test_ssz_typing.py @@ -208,13 +208,13 @@ def test_bytesn_subclass(): assert issubclass(BytesN[32](b'\xab' * 32).type(), Bytes32) assert issubclass(BytesN[32], Bytes32) - class Hash(Bytes32): + class Root(Bytes32): pass - assert isinstance(Hash(b'\xab' * 32), Bytes32) - assert not isinstance(Hash(b'\xab' * 32), Bytes48) - assert issubclass(Hash(b'\xab' * 32).type(), Bytes32) - assert issubclass(Hash, Bytes32) + assert isinstance(Root(b'\xab' * 32), Bytes32) + assert not isinstance(Root(b'\xab' * 32), Bytes48) + assert issubclass(Root(b'\xab' * 32).type(), Bytes32) + assert issubclass(Root, Bytes32) assert not issubclass(Bytes48, Bytes32) From 1202502bcb112ab36996d0d0693e9d48949e3f8e Mon Sep 17 00:00:00 2001 From: Diederik Loerakker Date: Wed, 13 Nov 2019 00:20:17 +0100 Subject: [PATCH 31/33] Update test_libs/pyspec/eth2spec/test/sanity/test_blocks.py Co-Authored-By: Danny Ryan --- test_libs/pyspec/eth2spec/test/sanity/test_blocks.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py index da6acb22c..ba577b89e 100644 --- a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py +++ b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py @@ -63,7 +63,7 @@ def test_empty_block_transition(spec, state): assert len(state.eth1_data_votes) == pre_eth1_votes + 1 assert spec.get_block_root_at_slot(state, pre_slot) == block.parent_root - assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Root() + assert spec.get_randao_mix(state, spec.get_current_epoch(state)) != spec.Bytes32() @with_all_phases From 117fb12888add6911c8dcda8c9313e72ee0b3ff9 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 12 Nov 2019 16:24:33 -0700 Subject: [PATCH 32/33] add scheduling note to on_attestation --- specs/core/0_fork-choice.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 88cc84913..13d451c3e 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -238,6 +238,12 @@ def on_block(store: Store, block: BeaconBlock) -> None: ```python def on_attestation(store: Store, attestation: Attestation) -> None: + """ + Run ``on_attestation`` upon receiving a new ``attestation`` from either within a block or directly on the wire. + + An ``attestation`` that is asserted as invalid may be valid at a later time, + consider scheduling it for later processing in such case. + """ target = attestation.data.target # Attestations must be from the current or previous epoch From 8fed220736ba370b9dc422b6dec92149bc5cfdda Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Mon, 18 Nov 2019 21:35:12 +0800 Subject: [PATCH 33/33] Update `AggregateAndProof` and rename `slot_signature` to `get_slot_signature` 1. Reorder the fields to fit the container name 2. Rename `AggregateAndProof.index: ValidatorIndex` to `AggregateAndProof.aggregator_index: ValidatorIndex` - Since we also have similar naming style like `ProposerSlashing.proposer_index` or `VoluntaryExit.validator_index` --- specs/networking/p2p-interface.md | 4 ++-- specs/validator/0_beacon-chain-validator.md | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 30ca43fd3..5bb9fff6d 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -174,9 +174,9 @@ There are two primary global topics used to propagate beacon blocks and aggregat - The aggregate attestation defined by `hash_tree_root(aggregate_and_proof.aggregate)` has _not_ already been seen (via aggregate gossip, within a block, or through the creation of an equivalent aggregate locally). - The block being voted for (`aggregate_and_proof.aggregate.data.beacon_block_root`) passes validation. - `aggregate_and_proof.aggregate.data.slot` is within the last `ATTESTATION_PROPAGATION_SLOT_RANGE` slots (`aggregate_and_proof.aggregate.data.slot + ATTESTATION_PROPAGATION_SLOT_RANGE >= current_slot >= aggregate_and_proof.aggregate.data.slot`). - - The validator index is within the aggregate's committee -- i.e. `aggregate_and_proof.index in get_attesting_indices(state, aggregate_and_proof.aggregate.data, aggregate_and_proof.aggregate.aggregation_bits)`. + - The validator index is within the aggregate's committee -- i.e. `aggregate_and_proof.aggregator_index in get_attesting_indices(state, aggregate_and_proof.aggregate.data, aggregate_and_proof.aggregate.aggregation_bits)`. - `aggregate_and_proof.selection_proof` selects the validator as an aggregator for the slot -- i.e. `is_aggregator(state, aggregate_and_proof.aggregate.data.index, aggregate_and_proof.selection_proof)` returns `True`. - - The `aggregate_and_proof.selection_proof` is a valid signature of the `aggregate_and_proof.aggregate.data.slot` by the validator with index `aggregate_and_proof.index`. + - The `aggregate_and_proof.selection_proof` is a valid signature of the `aggregate_and_proof.aggregate.data.slot` by the validator with index `aggregate_and_proof.aggregator_index`. - The signature of `aggregate_and_proof.aggregate` is valid. Additional global topics are used to propagate lower frequency validator messages. Their `TopicName`s are: diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 1434f0e87..43ea7ab07 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -365,7 +365,7 @@ Some validators are selected to locally aggregate attestations with a similar `a A validator is selected to aggregate based upon the return value of `is_aggregator()`. ```python -def slot_signature(state: BeaconState, slot: Slot, privkey: int) -> BLSSignature: +def get_slot_signature(state: BeaconState, slot: Slot, privkey: int) -> BLSSignature: domain = get_domain(state, DOMAIN_BEACON_ATTESTER, compute_epoch_at_slot(slot)) return bls_sign(privkey, hash_tree_root(slot), domain) ``` @@ -411,15 +411,15 @@ Aggregate attestations are broadcast as `AggregateAndProof` objects to prove to ```python class AggregateAndProof(Container): - index: ValidatorIndex - selection_proof: BLSSignature + aggregator_index: ValidatorIndex aggregate: Attestation + selection_proof: BLSSignature ``` Where -* `index` is the validator's `validator_index`. -* `selection_proof` is the signature of the slot (`slot_signature()`). +* `aggregator_index` is the validator's `ValidatorIndex`. * `aggregate` is the `aggregate_attestation` constructed in the previous section. +* `selection_proof` is the signature of the slot (`get_slot_signature()`). ## Phase 0 attestation subnet stability