From 351dcd4bb8ec963b7c38cbe1769c45eea2bfb47d Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Tue, 29 Oct 2019 08:48:16 +0100 Subject: [PATCH 01/23] p2p-interface: clarify that signing_root is used for block requests --- specs/networking/p2p-interface.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 5bccd25a9..662a9be6b 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -320,7 +320,7 @@ Here, `result` represents the 1-byte response code. The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time: -- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `HashTreeRoots`'s. +- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Hash`'s. - `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet. #### SSZ-encoding strategy (with or without Snappy) @@ -403,7 +403,7 @@ The response MUST consist of a single `response_chunk`. Request Content: ``` ( - head_block_root: HashTreeRoot + head_block_root: Hash start_slot: uint64 count: uint64 step: uint64 @@ -417,7 +417,7 @@ Response Content: ) ``` -Requests count beacon blocks from the peer starting from `start_slot` on the chain defined by `head_block_root`. The response MUST contain no more than count blocks. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`. +Requests count beacon blocks from the peer starting from `start_slot` on the chain defined by `head_block_root` (= `signing_root(BeaconBlock)`). The response MUST contain no more than count blocks. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`. The request MUST be encoded as an SSZ-container. @@ -441,7 +441,7 @@ Request Content: ``` ( - []HashTreeRoot + []Hash ) ``` @@ -453,7 +453,7 @@ Response Content: ) ``` -Requests blocks by their block roots. The response is a list of `BeaconBlock` whose length is less than or equal to the number of requested blocks. It may be less in the case that the responding peer is missing blocks. +Requests blocks by block root (= `signing_root(BeaconBlock)`). The response is a list of `BeaconBlock` whose length is less than or equal to the number of requested blocks. It may be less in the case that the responding peer is missing blocks. `BeaconBlocksByRoot` is primarily used to recover recent blocks (e.g. when receiving a block or attestation whose parent is unknown). From 1dc09e8567c9a1103d619fabb338c53fef384812 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Thu, 31 Oct 2019 17:32:08 +0100 Subject: [PATCH 02/23] hash cleanups * one more hash tree root gone for blocks - block hashes are always signing roots! * use simple serialize data types consistently --- specs/networking/p2p-interface.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 662a9be6b..9444ce267 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -320,7 +320,7 @@ Here, `result` represents the 1-byte response code. The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time: -- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Hash`'s. +- `ssz`: the contents are [SSZ-encoded](../simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Bytes32`'s. - `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet. #### SSZ-encoding strategy (with or without Snappy) @@ -344,20 +344,20 @@ constituents individually as `response_chunk`s. For example, the Request, Response Content: ``` ( - head_fork_version: bytes4 - finalized_root: bytes32 + head_fork_version: Bytes4 + finalized_root: Bytes32 finalized_epoch: uint64 - head_root: bytes32 + head_root: Bytes32 head_slot: uint64 ) ``` -The fields are: +The fields are, as seen by the client at the time of sending the message: - `head_fork_version`: The beacon_state `Fork` version. -- `finalized_root`: The latest finalized root the node knows about. -- `finalized_epoch`: The latest finalized epoch the node knows about. -- `head_root`: The block hash tree root corresponding to the head of the chain as seen by the sending node. -- `head_slot`: The slot corresponding to the `head_root`. +- `finalized_root`: The signing root of the latest finalized block. +- `finalized_epoch`: The epoch of the block corresponding to `finalized_root`. +- `head_root`: The signing root of the current head block. +- `head_slot`: The slot of the block corresponding to the `head_root`. The dialing client MUST send a `Status` request upon connection. @@ -403,7 +403,7 @@ The response MUST consist of a single `response_chunk`. Request Content: ``` ( - head_block_root: Hash + head_block_root: Bytes32 start_slot: uint64 count: uint64 step: uint64 @@ -441,7 +441,7 @@ Request Content: ``` ( - []Hash + []Bytes32 ) ``` From af5238ad9aea3fce7157357d1df61280f1b02f34 Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Fri, 1 Nov 2019 08:42:10 +0100 Subject: [PATCH 03/23] Describe which finalized root/epoch to use --- specs/networking/p2p-interface.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/networking/p2p-interface.md b/specs/networking/p2p-interface.md index 9444ce267..30ca43fd3 100644 --- a/specs/networking/p2p-interface.md +++ b/specs/networking/p2p-interface.md @@ -354,8 +354,8 @@ Request, Response Content: The fields are, as seen by the client at the time of sending the message: - `head_fork_version`: The beacon_state `Fork` version. -- `finalized_root`: The signing root of the latest finalized block. -- `finalized_epoch`: The epoch of the block corresponding to `finalized_root`. +- `finalized_root`: `state.finalized_checkpoint.root` for the state corresponding to the head block. +- `finalized_epoch`: `state.finalized_checkpoint.epoch` for the state corresponding to the head block. - `head_root`: The signing root of the current head block. - `head_slot`: The slot of the block corresponding to the `head_root`. From 65b615a4d4cf75a50b29d25c53f1bc5422770ae5 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 1 Nov 2019 21:02:53 -0600 Subject: [PATCH 04/23] remove custody_bits from attestation --- specs/core/0_beacon-chain.md | 33 +++-------- specs/core/0_fork-choice.md | 2 +- .../test/fork_choice/test_on_attestation.py | 13 ++-- .../eth2spec/test/helpers/attestations.py | 4 +- .../test_process_attestation.py | 58 ------------------ .../test_process_attester_slashing.py | 59 +++++-------------- .../test_process_rewards_and_penalties.py | 2 +- .../eth2spec/test/sanity/test_blocks.py | 3 +- 8 files changed, 35 insertions(+), 139 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9f0f21435..632c4963a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -320,8 +320,7 @@ class AttestationDataAndCustodyBit(Container): ```python class IndexedAttestation(Container): - custody_bit_0_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] # Indices with custody bit equal to 0 - custody_bit_1_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] # Indices with custody bit equal to 1 + attesting_indices: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE] data: AttestationData signature: BLSSignature ``` @@ -399,7 +398,6 @@ class AttesterSlashing(Container): class Attestation(Container): aggregation_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE] data: AttestationData - custody_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE] signature: BLSSignature ``` @@ -605,30 +603,21 @@ def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: Indexe """ Check if ``indexed_attestation`` has valid indices and signature. """ - bit_0_indices = indexed_attestation.custody_bit_0_indices - bit_1_indices = indexed_attestation.custody_bit_1_indices + indices = indexed_attestation.attesting_indices - # Verify no index has custody bit equal to 1 [to be removed in phase 1] - if not len(bit_1_indices) == 0: # [to be removed in phase 1] - return False # [to be removed in phase 1] # Verify max number of indices - if not len(bit_0_indices) + len(bit_1_indices) <= MAX_VALIDATORS_PER_COMMITTEE: - return False - # Verify index sets are disjoint - if not len(set(bit_0_indices).intersection(bit_1_indices)) == 0: + if not len(indices) <= MAX_VALIDATORS_PER_COMMITTEE: return False # Verify indices are sorted - if not (bit_0_indices == sorted(bit_0_indices) and bit_1_indices == sorted(bit_1_indices)): + if not indices == sorted(indices): return False # Verify aggregate signature if not bls_verify_multiple( pubkeys=[ - bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_0_indices]), - bls_aggregate_pubkeys([state.validators[i].pubkey for i in bit_1_indices]), + bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), ], message_hashes=[ hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b0)), - hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b1)), ], signature=indexed_attestation.signature, domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch), @@ -922,13 +911,9 @@ def get_indexed_attestation(state: BeaconState, attestation: Attestation) -> Ind Return the indexed attestation corresponding to ``attestation``. """ attesting_indices = get_attesting_indices(state, attestation.data, attestation.aggregation_bits) - custody_bit_1_indices = get_attesting_indices(state, attestation.data, attestation.custody_bits) - assert custody_bit_1_indices.issubset(attesting_indices) - custody_bit_0_indices = attesting_indices.difference(custody_bit_1_indices) return IndexedAttestation( - custody_bit_0_indices=sorted(custody_bit_0_indices), - custody_bit_1_indices=sorted(custody_bit_1_indices), + attesting_indices=sorted(attesting_indices), data=attestation.data, signature=attestation.signature, ) @@ -1460,8 +1445,8 @@ def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSla assert is_valid_indexed_attestation(state, attestation_2) slashed_any = False - attesting_indices_1 = attestation_1.custody_bit_0_indices + attestation_1.custody_bit_1_indices - attesting_indices_2 = attestation_2.custody_bit_0_indices + attestation_2.custody_bit_1_indices + attesting_indices_1 = attestation_1.attesting_indices + attesting_indices_2 = attestation_2.attesting_indices for index in sorted(set(attesting_indices_1).intersection(attesting_indices_2)): if is_slashable_validator(state.validators[index], get_current_epoch(state)): slash_validator(state, index) @@ -1479,7 +1464,7 @@ def process_attestation(state: BeaconState, attestation: Attestation) -> None: assert data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot <= data.slot + SLOTS_PER_EPOCH committee = get_beacon_committee(state, data.slot, data.index) - assert len(attestation.aggregation_bits) == len(attestation.custody_bits) == len(committee) + assert len(attestation.aggregation_bits) == len(committee) pending_attestation = PendingAttestation( data=data, diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index b909ce732..bf9688db7 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -199,7 +199,7 @@ def on_attestation(store: Store, attestation: Attestation) -> None: assert is_valid_indexed_attestation(target_state, indexed_attestation) # Update latest messages - for i in indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices: + for i in indexed_attestation.attesting_indices: if i not in store.latest_messages or target.epoch > store.latest_messages[i].epoch: store.latest_messages[i] = LatestMessage(epoch=target.epoch, root=attestation.data.beacon_block_root) ``` diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py index 70375ef27..39c179b59 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py @@ -1,7 +1,5 @@ -from eth2spec.test.context import with_all_phases, spec_state_test, with_phases - - +from eth2spec.test.context import with_all_phases, spec_state_test from eth2spec.test.helpers.block import build_empty_block_for_next_slot from eth2spec.test.helpers.attestations import get_valid_attestation from eth2spec.test.helpers.state import state_transition_and_sign_block @@ -19,7 +17,7 @@ def run_on_attestation(spec, state, store, attestation, valid=True): indexed_attestation = spec.get_indexed_attestation(state, attestation) spec.on_attestation(store, attestation) assert ( - store.latest_messages[indexed_attestation.custody_bit_0_indices[0]] == + store.latest_messages[indexed_attestation.attesting_indices[0]] == spec.LatestMessage( epoch=attestation.data.target.epoch, root=attestation.data.beacon_block_root, @@ -100,7 +98,7 @@ def test_on_attestation_same_slot(spec, state): run_on_attestation(spec, state, store, attestation, False) -@with_phases(['phase0']) +@with_all_phases @spec_state_test def test_on_attestation_invalid_attestation(spec, state): store = spec.get_genesis_store(state) @@ -113,6 +111,7 @@ def test_on_attestation_invalid_attestation(spec, state): spec.on_block(store, block) attestation = get_valid_attestation(spec, state, slot=block.slot) - # make attestation invalid by setting a phase1-only custody bit - attestation.custody_bits[0] = 1 + # make invalid by using an invalid committee index + attestation.data.index = spec.MAX_COMMITTEES_PER_SLOT * spec.SLOTS_PER_EPOCH + run_on_attestation(spec, state, store, attestation, False) diff --git a/test_libs/pyspec/eth2spec/test/helpers/attestations.py b/test_libs/pyspec/eth2spec/test/helpers/attestations.py index d36e62ace..d00a4cc96 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/attestations.py +++ b/test_libs/pyspec/eth2spec/test/helpers/attestations.py @@ -54,11 +54,9 @@ def get_valid_attestation(spec, state, slot=None, index=None, signed=False): committee_size = len(beacon_committee) aggregation_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE](*([0] * committee_size)) - custody_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE](*([0] * committee_size)) attestation = spec.Attestation( aggregation_bits=aggregation_bits, data=attestation_data, - custody_bits=custody_bits, ) fill_aggregate_attestation(spec, state, attestation) if signed: @@ -83,7 +81,7 @@ def sign_aggregate_attestation(spec, state, attestation_data, participants: List def sign_indexed_attestation(spec, state, indexed_attestation): - participants = indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices + participants = indexed_attestation.attesting_indices indexed_attestation.signature = sign_aggregate_attestation(spec, state, indexed_attestation.data, participants) diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py index 04fa880e0..554d77a00 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py @@ -274,35 +274,6 @@ def test_bad_source_root(spec, state): yield from run_attestation_processing(spec, state, attestation, False) -@with_all_phases -@spec_state_test -def test_inconsistent_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - custody_bits = attestation.custody_bits[:] - custody_bits.append(False) - - attestation.custody_bits = custody_bits - - sign_attestation(spec, state, attestation) - - yield from run_attestation_processing(spec, state, attestation, False) - - -@with_phases(['phase0']) -@spec_state_test -def test_non_empty_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - attestation.custody_bits = attestation.aggregation_bits[:] - - sign_attestation(spec, state, attestation) - - yield from run_attestation_processing(spec, state, attestation, False) - - @with_all_phases @spec_state_test def test_empty_aggregation_bits(spec, state): @@ -344,32 +315,3 @@ def test_too_few_aggregation_bits(spec, state): attestation.aggregation_bits = attestation.aggregation_bits[:-1] yield from run_attestation_processing(spec, state, attestation, False) - - -@with_all_phases -@spec_state_test -def test_too_many_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state, signed=True) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - # one too many bits - attestation.custody_bits.append(0b0) - - yield from run_attestation_processing(spec, state, attestation, False) - - -@with_all_phases -@spec_state_test -def test_too_few_custody_bits(spec, state): - attestation = get_valid_attestation(spec, state) - state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY - - attestation.custody_bits = Bitlist[spec.MAX_VALIDATORS_PER_COMMITTEE]( - *([0b1] + [0b0] * (len(attestation.custody_bits) - 1))) - - sign_attestation(spec, state, attestation) - - # one too few bits - attestation.custody_bits = attestation.custody_bits[:-1] - - yield from run_attestation_processing(spec, state, attestation, False) diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py index 20a510648..85e807ec0 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attester_slashing.py @@ -25,10 +25,7 @@ def run_attester_slashing_processing(spec, state, attester_slashing, valid=True) yield 'post', None return - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices proposer_index = spec.get_beacon_proposer_index(state) pre_proposer_balance = get_balance(state, proposer_index) @@ -112,10 +109,7 @@ def test_success_surround(spec, state): @always_bls def test_success_already_exited_recent(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices for index in slashed_indices: spec.initiate_validator_exit(state, index) @@ -127,10 +121,7 @@ def test_success_already_exited_recent(spec, state): @always_bls def test_success_already_exited_long_ago(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - slashed_indices = ( - attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices - ) + slashed_indices = attester_slashing.attestation_1.attesting_indices for index in slashed_indices: spec.initiate_validator_exit(state, index) state.validators[index].withdrawable_epoch = spec.get_current_epoch(state) + 2 @@ -190,38 +181,23 @@ def test_participants_already_slashed(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) # set all indices to slashed - attestation_1 = attester_slashing.attestation_1 - validator_indices = attestation_1.custody_bit_0_indices + attestation_1.custody_bit_1_indices + validator_indices = attester_slashing.attestation_1.attesting_indices for index in validator_indices: state.validators[index].slashed = True yield from run_attester_slashing_processing(spec, state, attester_slashing, False) -@with_all_phases -@spec_state_test -def test_custody_bit_0_and_1_intersect(spec, state): - attester_slashing = get_valid_attester_slashing(spec, state, signed_1=False, signed_2=True) - - attester_slashing.attestation_1.custody_bit_1_indices.append( - attester_slashing.attestation_1.custody_bit_0_indices[0] - ) - - sign_indexed_attestation(spec, state, attester_slashing.attestation_1) - - yield from run_attester_slashing_processing(spec, state, attester_slashing, False) - - @with_all_phases @spec_state_test @always_bls def test_att1_bad_extra_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices.append(options[len(options) // 2]) # add random index, not previously in attestation. - attester_slashing.attestation_1.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_1.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad extra index is spotted, and slashing is aborted. @@ -234,10 +210,10 @@ def test_att1_bad_extra_index(spec, state): def test_att1_bad_replaced_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices[3] = options[len(options) // 2] # replace with random index, not previously in attestation. - attester_slashing.attestation_1.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_1.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad replaced index is spotted, and slashing is aborted. @@ -250,10 +226,10 @@ def test_att1_bad_replaced_index(spec, state): def test_att2_bad_extra_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices.append(options[len(options) // 2]) # add random index, not previously in attestation. - attester_slashing.attestation_2.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_2.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad extra index is spotted, and slashing is aborted. @@ -266,10 +242,10 @@ def test_att2_bad_extra_index(spec, state): def test_att2_bad_replaced_index(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices options = list(set(range(len(state.validators))) - set(indices)) indices[3] = options[len(options) // 2] # replace with random index, not previously in attestation. - attester_slashing.attestation_2.custody_bit_0_indices = sorted(indices) + attester_slashing.attestation_2.attesting_indices = sorted(indices) # Do not sign the modified attestation (it's ok to slash if attester signed, not if they did not), # see if the bad replaced index is spotted, and slashing is aborted. @@ -278,10 +254,10 @@ def test_att2_bad_replaced_index(spec, state): @with_all_phases @spec_state_test -def test_unsorted_att_1_bit0(spec, state): +def test_unsorted_att_1(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=False, signed_2=True) - indices = attester_slashing.attestation_1.custody_bit_0_indices + indices = attester_slashing.attestation_1.attesting_indices assert len(indices) >= 3 indices[1], indices[2] = indices[2], indices[1] # unsort second and third index sign_indexed_attestation(spec, state, attester_slashing.attestation_1) @@ -291,15 +267,12 @@ def test_unsorted_att_1_bit0(spec, state): @with_all_phases @spec_state_test -def test_unsorted_att_2_bit0(spec, state): +def test_unsorted_att_2(spec, state): attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=False) - indices = attester_slashing.attestation_2.custody_bit_0_indices + indices = attester_slashing.attestation_2.attesting_indices assert len(indices) >= 3 indices[1], indices[2] = indices[2], indices[1] # unsort second and third index sign_indexed_attestation(spec, state, attester_slashing.attestation_2) yield from run_attester_slashing_processing(spec, state, attester_slashing, False) - - -# note: unsorted indices for custody bit 0 are to be introduced in phase 1 testing. diff --git a/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py b/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py index 99ba9d284..7d844b63b 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/epoch_processing/test_process_rewards_and_penalties.py @@ -141,7 +141,7 @@ def test_duplicate_attestation(spec, state): attestation = get_valid_attestation(spec, state, signed=True) indexed_attestation = spec.get_indexed_attestation(state, attestation) - participants = indexed_attestation.custody_bit_0_indices + indexed_attestation.custody_bit_1_indices + participants = indexed_attestation.attesting_indices assert len(participants) > 0 diff --git a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py index bffb1c1fc..bbc4bba3a 100644 --- a/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py +++ b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py @@ -188,8 +188,7 @@ def test_attester_slashing(spec, state): pre_state = deepcopy(state) attester_slashing = get_valid_attester_slashing(spec, state, signed_1=True, signed_2=True) - validator_index = (attester_slashing.attestation_1.custody_bit_0_indices - + attester_slashing.attestation_1.custody_bit_1_indices)[0] + validator_index = attester_slashing.attestation_1.attesting_indices[0] assert not state.validators[validator_index].slashed From 600265a3110b647418836db59768e8fa453a3c1d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 1 Nov 2019 21:12:32 -0600 Subject: [PATCH 05/23] remove AttestationDataAndCustodyBit --- scripts/build_spec.py | 3 +-- specs/core/0_beacon-chain.md | 19 +++-------------- specs/validator/0_beacon-chain-validator.md | 21 +------------------ .../pyspec/eth2spec/fuzzing/test_decoder.py | 2 +- .../eth2spec/test/helpers/attestations.py | 9 ++------ .../test_process_attestation.py | 2 +- 6 files changed, 9 insertions(+), 47 deletions(-) diff --git a/scripts/build_spec.py b/scripts/build_spec.py index e05907014..9ed9d3ae4 100644 --- a/scripts/build_spec.py +++ b/scripts/build_spec.py @@ -24,14 +24,13 @@ from eth2spec.utils.ssz.ssz_impl import ( signing_root, ) from eth2spec.utils.ssz.ssz_typing import ( - bit, boolean, Container, List, Vector, uint64, + boolean, Container, List, Vector, uint64, Bytes1, Bytes4, Bytes8, Bytes32, Bytes48, Bytes96, Bitlist, Bitvector, ) from eth2spec.utils.bls import ( bls_aggregate_signatures, bls_aggregate_pubkeys, bls_verify, - bls_verify_multiple, bls_sign, ) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 632c4963a..037e1672f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -26,7 +26,6 @@ - [`Checkpoint`](#checkpoint) - [`Validator`](#validator) - [`AttestationData`](#attestationdata) - - [`AttestationDataAndCustodyBit`](#attestationdataandcustodybit) - [`IndexedAttestation`](#indexedattestation) - [`PendingAttestation`](#pendingattestation) - [`Eth1Data`](#eth1data) @@ -308,14 +307,6 @@ class AttestationData(Container): target: Checkpoint ``` -#### `AttestationDataAndCustodyBit` - -```python -class AttestationDataAndCustodyBit(Container): - data: AttestationData - custody_bit: bit # Challengeable bit (SSZ-bool, 1 byte) for the custody of shard data -``` - #### `IndexedAttestation` ```python @@ -612,13 +603,9 @@ def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: Indexe if not indices == sorted(indices): return False # Verify aggregate signature - if not bls_verify_multiple( - pubkeys=[ - bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), - ], - message_hashes=[ - hash_tree_root(AttestationDataAndCustodyBit(data=indexed_attestation.data, custody_bit=0b0)), - ], + if not bls_verify( + pubkey=bls_aggregate_pubkeys([state.validators[i].pubkey for i in indices]), + message_hash=hash_tree_root(indexed_attestation.data), signature=indexed_attestation.signature, domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch), ): diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 166534031..236d12a88 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -45,7 +45,6 @@ - [Construct attestation](#construct-attestation) - [Data](#data) - [Aggregation bits](#aggregation-bits) - - [Custody bits](#custody-bits) - [Aggregate signature](#aggregate-signature) - [Broadcast attestation](#broadcast-attestation) - [Attestation aggregation](#attestation-aggregation) @@ -53,7 +52,6 @@ - [Construct aggregate](#construct-aggregate) - [Data](#data-1) - [Aggregation bits](#aggregation-bits-1) - - [Custody bits](#custody-bits-1) - [Aggregate signature](#aggregate-signature-1) - [Broadcast aggregate](#broadcast-aggregate) - [`AggregateAndProof`](#aggregateandproof) @@ -331,25 +329,14 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes *Note*: Calling `get_attesting_indices(state, attestation.data, attestation.aggregation_bits)` should return a list of length equal to 1, containing `validator_index`. -##### Custody bits - -- Let `attestation.custody_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` filled with zeros of length `len(committee)`. - -*Note*: This is a stub for Phase 0. - ##### Aggregate signature Set `attestation.signature = signed_attestation_data` where `signed_attestation_data` is obtained from: ```python def get_signed_attestation_data(state: BeaconState, attestation: IndexedAttestation, privkey: int) -> BLSSignature: - attestation_data_and_custody_bit = AttestationDataAndCustodyBit( - data=attestation.data, - custody_bit=0b0, - ) - domain = get_domain(state, DOMAIN_BEACON_ATTESTER, attestation.data.target.epoch) - return bls_sign(privkey, hash_tree_root(attestation_data_and_custody_bit), domain) + return bls_sign(privkey, hash_tree_root(attestation.data), domain) ``` #### Broadcast attestation @@ -391,12 +378,6 @@ Set `aggregate_attestation.data = attestation_data` where `attestation_data` is Let `aggregate_attestation.aggregation_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` of length `len(committee)`, where each bit set from each individual attestation is set to `0b1`. -##### Custody bits - -- Let `aggregate_attestation.custody_bits` be a `Bitlist[MAX_VALIDATORS_PER_COMMITTEE]` filled with zeros of length `len(committee)`. - -*Note*: This is a stub for Phase 0. - ##### Aggregate signature Set `aggregate_attestation.signature = aggregate_signature` where `aggregate_signature` is obtained from: diff --git a/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py b/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py index 77b52e7a2..9cabefb13 100644 --- a/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py +++ b/test_libs/pyspec/eth2spec/fuzzing/test_decoder.py @@ -9,7 +9,7 @@ def test_decoder(): rng = Random(123) # check these types only, Block covers a lot of operation types already. - for typ in [spec.AttestationDataAndCustodyBit, spec.BeaconState, spec.BeaconBlock]: + for typ in [spec.Attestation, spec.BeaconState, spec.BeaconBlock]: # create a random pyspec value original = random_value.get_random_ssz_object(rng, typ, 100, 10, mode=random_value.RandomizationMode.mode_random, diff --git a/test_libs/pyspec/eth2spec/test/helpers/attestations.py b/test_libs/pyspec/eth2spec/test/helpers/attestations.py index d00a4cc96..299ca32ff 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/attestations.py +++ b/test_libs/pyspec/eth2spec/test/helpers/attestations.py @@ -95,14 +95,9 @@ def sign_attestation(spec, state, attestation): attestation.signature = sign_aggregate_attestation(spec, state, attestation.data, participants) -def get_attestation_signature(spec, state, attestation_data, privkey, custody_bit=0b0): - message_hash = spec.AttestationDataAndCustodyBit( - data=attestation_data, - custody_bit=custody_bit, - ).hash_tree_root() - +def get_attestation_signature(spec, state, attestation_data, privkey): return bls_sign( - message_hash=message_hash, + message_hash=attestation_data.hash_tree_root(), privkey=privkey, domain=spec.get_domain( state=state, diff --git a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py index 554d77a00..f19bc66d3 100644 --- a/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py +++ b/test_libs/pyspec/eth2spec/test/phase_0/block_processing/test_process_attestation.py @@ -2,7 +2,7 @@ from eth2spec.test.context import ( spec_state_test, expect_assertion_error, always_bls, never_bls, - with_all_phases, with_phases, + with_all_phases, spec_test, low_balances, with_custom_state, From 1c99ab2c96be00cdd50fdf1bec56fd3481d6a5b9 Mon Sep 17 00:00:00 2001 From: Paul Hauner Date: Fri, 1 Nov 2019 10:51:21 +1100 Subject: [PATCH 06/23] Specify inclusive range for genesis deposits --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9f0f21435..f00be9a88 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1026,7 +1026,7 @@ Before the Ethereum 2.0 genesis has been triggered, and for every Ethereum 1.0 b - `eth1_block_hash` is the hash of the Ethereum 1.0 block - `eth1_timestamp` is the Unix timestamp corresponding to `eth1_block_hash` -- `deposits` is the sequence of all deposits, ordered chronologically, up to the block with hash `eth1_block_hash` +- `deposits` is the sequence of all deposits, ordered chronologically, up to (and including) the block with hash `eth1_block_hash` ```python def initialize_beacon_state_from_eth1(eth1_block_hash: Hash, From 405e2185981a689ac98871ed3e430435ddb26a04 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 08:55:34 -0700 Subject: [PATCH 07/23] add initial fork choice bounce prevention and tests --- specs/core/0_fork-choice.md | 50 ++++++++++++- .../eth2spec/test/fork_choice/test_on_tick.py | 72 +++++++++++++++++++ 2 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index b909ce732..722695f23 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -43,6 +43,12 @@ The head block root associated with a `store` is defined as `get_head(store)`. A 4) **Manual forks**: Manual forks may arbitrarily change the fork choice rule but are expected to be enacted at epoch transitions, with the fork details reflected in `state.fork`. 5) **Implementation**: The implementation found in this specification is constructed for ease of understanding rather than for optimization in computation, space, or any other resource. A number of optimized alternatives can be found [here](https://github.com/protolambda/lmd-ghost). +### Configuration + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `9` | slots | 108 seconds | + ### Helpers #### `LatestMessage` @@ -60,8 +66,10 @@ class LatestMessage(object): @dataclass class Store(object): time: uint64 + genesis_time: uint64 justified_checkpoint: Checkpoint finalized_checkpoint: Checkpoint + queued_justified_checkpoints: List[Checkpoint, 2**40] = field(default_factory=list) blocks: Dict[Hash, BeaconBlock] = field(default_factory=dict) block_states: Dict[Hash, BeaconState] = field(default_factory=dict) checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) @@ -78,6 +86,7 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: finalized_checkpoint = Checkpoint(epoch=GENESIS_EPOCH, root=root) return Store( time=genesis_state.genesis_time, + genesis_time=genesis_state.genesis_time, justified_checkpoint=justified_checkpoint, finalized_checkpoint=finalized_checkpoint, blocks={root: genesis_block}, @@ -86,6 +95,11 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: ) ``` +```python +def get_current_slot(store: Store) -> Slot: + return Slot((store.time - store.genesis_time) // SECONDS_PER_SLOT) +``` + #### `get_ancestor` ```python @@ -130,13 +144,44 @@ def get_head(store: Store) -> Hash: head = max(children, key=lambda root: (get_latest_attesting_balance(store, root), root)) ``` +#### `should_update_justified_checkpoint` + +```python +def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: + current_epoch = compute_epoch_at_slot(get_current_slot(store)) + + if get_current_slot(store) - compute_start_slot_at_epoch(current_epoch) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + return True + + justified_block = store.blocks[justified_checkpoint.root] + if justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): + return False + if not get_ancestor(store, justified_checkpoint.root, store.blocks[justified_checkpoint.root].slot): + return False + + return True +``` + ### Handlers #### `on_tick` ```python def on_tick(store: Store, time: uint64) -> None: + previous_slot = get_current_slot(store) + + # update store time store.time = time + + current_slot = get_current_slot(store) + # not a new epoch, return + if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): + return + # if new epoch and there are queued_justified_checkpoints, update if any is better than the best in store + if any(store.queued_justified_checkpoints): + best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) + if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.justified_checkpoint = best_justified_checkpoint ``` #### `on_block` @@ -164,7 +209,10 @@ def on_block(store: Store, block: BeaconBlock) -> None: # Update justified checkpoint if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch: - store.justified_checkpoint = state.current_justified_checkpoint + if should_update_justified_checkpoint(store, state.current_justified_checkpoint): + store.justified_checkpoint = state.current_justified_checkpoint + else: + store.queued_justified_checkpoints.append(state.current_justified_checkpoint) # Update finalized checkpoint if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch: diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py new file mode 100644 index 000000000..cd0d20abb --- /dev/null +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -0,0 +1,72 @@ +from eth2spec.test.context import with_all_phases, spec_state_test + + +def run_on_tick(spec, store, time, new_justified_checkpoint=None): + previous_justified_checkpoint = store.justified_checkpoint + + spec.on_tick(store, time) + + assert store.time == time + + if new_justified_checkpoint: + assert store.justified_checkpoint == new_justified_checkpoint + assert store.justified_checkpoint.epoch > previous_justified_checkpoint.epoch + else: + assert store.justified_checkpoint == previous_justified_checkpoint + + +@with_all_phases +@spec_state_test +def test_basic(spec, state): + store = spec.get_genesis_store(state) + run_on_tick(spec, store, store.time + 1) + + +@with_all_phases +@spec_state_test +def test_update_justified_single(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + + +@with_all_phases +@spec_state_test +def test_update_justified_multiple(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = None # remember checkpoint with latest epoch + for i in range(3): + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + i + 1, + root=i.to_bytes(1, byteorder='big') * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + + +@with_all_phases +@spec_state_test +def test_no_update_(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + From 40a21dc3ad55e5420f0656ffc85d9b660e84c90e Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 09:00:04 -0700 Subject: [PATCH 08/23] PR feedback --- README.md | 1 - specs/core/0_beacon-chain.md | 10 ++-------- 2 files changed, 2 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index a6f23db9d..fa103394d 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,6 @@ The following are the broad design goals for Ethereum 2.0: ## For spec contributors - Documentation on the different components used during spec writing can be found here: * [YAML Test Generators](test_generators/README.md) * [Executable Python Spec, with Py-tests](test_libs/pyspec/README.md) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 037e1672f..fc51dd48d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -54,7 +54,6 @@ - [`hash_tree_root`](#hash_tree_root) - [`signing_root`](#signing_root) - [`bls_verify`](#bls_verify) - - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) - [Predicates](#predicates) - [`is_active_validator`](#is_active_validator) @@ -542,10 +541,6 @@ def bytes_to_int(data: bytes) -> uint64: `bls_verify` is a function for verifying a BLS signature, as defined in the [BLS Signature spec](../bls_signature.md#bls_verify). -#### `bls_verify_multiple` - -`bls_verify_multiple` is a function for verifying a BLS signature constructed from multiple messages, as defined in the [BLS Signature spec](../bls_signature.md#bls_verify_multiple). - #### `bls_aggregate_pubkeys` `bls_aggregate_pubkeys` is a function for aggregating multiple BLS public keys into a single aggregate key, as defined in the [BLS Signature spec](../bls_signature.md#bls_aggregate_pubkeys). @@ -1432,9 +1427,8 @@ def process_attester_slashing(state: BeaconState, attester_slashing: AttesterSla assert is_valid_indexed_attestation(state, attestation_2) slashed_any = False - attesting_indices_1 = attestation_1.attesting_indices - attesting_indices_2 = attestation_2.attesting_indices - for index in sorted(set(attesting_indices_1).intersection(attesting_indices_2)): + indices = set(attestation_1.attesting_indices).intersection(attestation_2.attesting_indices) + for index in sorted(indices): if is_slashable_validator(state.validators[index], get_current_epoch(state)): slash_validator(state, index) slashed_any = True From 97d7cf519063f306039fa0e9e4958e4033932aa0 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 10:58:45 -0700 Subject: [PATCH 09/23] further test bounce attack --- configs/mainnet.yaml | 5 ++ configs/minimal.yaml | 6 ++ specs/core/0_fork-choice.md | 2 +- .../test/fork_choice/test_on_block.py | 71 +++++++++++++++++++ .../eth2spec/test/fork_choice/test_on_tick.py | 62 ++++++++++++++-- 5 files changed, 141 insertions(+), 5 deletions(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index af446d575..e1b0faa3c 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -23,6 +23,11 @@ MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 65536 MIN_GENESIS_TIME: 1578009600 +# Fork Choice +# --------------------------------------------------------------- +# 2**3 (= 8) +SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 8 + # Deposit contract # --------------------------------------------------------------- diff --git a/configs/minimal.yaml b/configs/minimal.yaml index 53599e83a..fbc961ab1 100644 --- a/configs/minimal.yaml +++ b/configs/minimal.yaml @@ -21,6 +21,12 @@ MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 64 # Jan 3, 2020 MIN_GENESIS_TIME: 1578009600 +# +# +# Fork Choice +# --------------------------------------------------------------- +# 2**1 (= 1) +SAFE_SLOTS_TO_UPDATE_JUSTIFIED: 2 # Deposit contract diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 722695f23..f2bb64f2a 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -150,7 +150,7 @@ def get_head(store: Store) -> Hash: def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: current_epoch = compute_epoch_at_slot(get_current_slot(store)) - if get_current_slot(store) - compute_start_slot_at_epoch(current_epoch) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True justified_block = store.blocks[justified_checkpoint.root] diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py index 918c0f79e..36e61f0b5 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py @@ -132,3 +132,74 @@ def test_on_block_before_finalized(spec, state): block = build_empty_block_for_next_slot(spec, state) state_transition_and_sign_block(spec, state, block) run_on_block(spec, store, block, False) + + +@with_all_phases +@spec_state_test +def test_on_block_update_justified_checkpoint_within_safe_slots(spec, state): + # Initialization + store = spec.get_genesis_store(state) + time = 100 + spec.on_tick(store, time) + + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + state, store, last_block = apply_next_epoch_with_attestations(spec, state, store) + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + last_block_root = signing_root(last_block) + + # Mock the justified checkpoint + just_state = store.block_states[last_block_root] + new_justified = spec.Checkpoint( + epoch=just_state.current_justified_checkpoint.epoch + 1, + root=b'\x77' * 32, + ) + just_state.current_justified_checkpoint = new_justified + + block = build_empty_block_for_next_slot(spec, just_state) + state_transition_and_sign_block(spec, deepcopy(just_state), block) + assert spec.get_current_slot(store) % spec.SLOTS_PER_EPOCH < spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED + run_on_block(spec, store, block) + + assert store.justified_checkpoint == new_justified + + +@with_all_phases +@spec_state_test +def test_on_block_outside_safe_slots_and_old_block(spec, state): + # Initialization + store = spec.get_genesis_store(state) + time = 100 + spec.on_tick(store, time) + + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + state, store, last_block = apply_next_epoch_with_attestations(spec, state, store) + next_epoch(spec, state) + spec.on_tick(store, store.time + state.slot * spec.SECONDS_PER_SLOT) + last_block_root = signing_root(last_block) + + # Mock justified block in store + just_block = build_empty_block_for_next_slot(spec, state) + # Slot is same as justified checkpoint so does not trigger an override in the store + just_block.slot = spec.compute_start_slot_at_epoch(store.justified_checkpoint.epoch) + store.blocks[just_block.hash_tree_root()] = just_block + + # Mock the justified checkpoint + just_state = store.block_states[last_block_root] + new_justified = spec.Checkpoint( + epoch=just_state.current_justified_checkpoint.epoch + 1, + root=just_block.hash_tree_root(), + ) + just_state.current_justified_checkpoint = new_justified + + block = build_empty_block_for_next_slot(spec, just_state) + state_transition_and_sign_block(spec, deepcopy(just_state), block) + + spec.on_tick(store, store.time + spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED * spec.SECONDS_PER_SLOT) + assert spec.get_current_slot(store) % spec.SLOTS_PER_EPOCH >= spec.SAFE_SLOTS_TO_UPDATE_JUSTIFIED + run_on_block(spec, store, block) + + assert store.justified_checkpoint != new_justified + assert store.queued_justified_checkpoints[0] == new_justified diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py index cd0d20abb..2ba89ebca 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -32,7 +32,6 @@ def test_update_justified_single(spec, state): epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) @@ -57,7 +56,7 @@ def test_update_justified_multiple(spec, state): @with_all_phases @spec_state_test -def test_no_update_(spec, state): +def test_no_update_same_slot_at_epoch_boundary(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH @@ -65,8 +64,63 @@ def test_no_update_(spec, state): epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + # set store time to already be at epoch boundary + store.time = seconds_per_epoch + run_on_tick(spec, store, store.time + 1) + + +@with_all_phases +@spec_state_test +def test_no_update_not_epoch_boundary(spec, state): + store = spec.get_genesis_store(state) + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + run_on_tick(spec, store, store.time + spec.SECONDS_PER_SLOT) + + +@with_all_phases +@spec_state_test +def test_no_update_new_justified_equal_epoch(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + store.justified_checkpoint = spec.Checkpoint( + epoch=new_justified.epoch, + root=b'\44' * 32, + ) + + run_on_tick(spec, store, store.time + seconds_per_epoch) + + +@with_all_phases +@spec_state_test +def test_no_update_new_justified_later_epoch(spec, state): + store = spec.get_genesis_store(state) + seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + + new_justified = spec.Checkpoint( + epoch=store.justified_checkpoint.epoch + 1, + root=b'\x55' * 32, + ) + store.queued_justified_checkpoints.append(new_justified) + + store.justified_checkpoint = spec.Checkpoint( + epoch=new_justified.epoch + 1, + root=b'\44' * 32, + ) + + run_on_tick(spec, store, store.time + seconds_per_epoch) From e20e11e0b54aee2683c9386e6d7d600707daa6f5 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:02:58 -0700 Subject: [PATCH 10/23] wipe queued justified after epoch transition --- specs/core/0_fork-choice.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index f2bb64f2a..2e8deffdb 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -47,7 +47,7 @@ The head block root associated with a `store` is defined as `get_head(store)`. A | Name | Value | Unit | Duration | | - | - | :-: | :-: | -| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `9` | slots | 108 seconds | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3 (= 8)` | slots | 96 seconds | ### Helpers @@ -174,14 +174,15 @@ def on_tick(store: Store, time: uint64) -> None: store.time = time current_slot = get_current_slot(store) - # not a new epoch, return + # Not a new epoch, return if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): return - # if new epoch and there are queued_justified_checkpoints, update if any is better than the best in store + # If new epoch and there are queued_justified_checkpoints, update if any is better than the best in store if any(store.queued_justified_checkpoints): best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: store.justified_checkpoint = best_justified_checkpoint + store.queued_justified_checkpoints = [] ``` #### `on_block` From ba6637b4d98fbfcec9721d813ae9d3deb8445436 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:42:40 -0700 Subject: [PATCH 11/23] remove extra var --- specs/core/0_fork-choice.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 2e8deffdb..edd8a52fa 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -148,8 +148,6 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: - current_epoch = compute_epoch_at_slot(get_current_slot(store)) - if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True From 26162106379e385b0684c49abe856194de9c479d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 12:51:47 -0700 Subject: [PATCH 12/23] minor fmt --- specs/core/0_fork-choice.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index edd8a52fa..e08ea09b6 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -47,7 +47,7 @@ The head block root associated with a `store` is defined as `get_head(store)`. A | Name | Value | Unit | Duration | | - | - | :-: | :-: | -| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3 (= 8)` | slots | 96 seconds | +| `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` | `2**3` (= 8) | slots | 96 seconds | ### Helpers From 4f42f63e4e6cb8dbc0c7237dbfe0e4fee8b660c1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Tue, 5 Nov 2019 11:35:58 -0700 Subject: [PATCH 13/23] only allow attestatiosn to be considered from current and previous epoch --- specs/core/0_fork-choice.md | 3 ++ .../test/fork_choice/test_on_attestation.py | 52 +++++++++++++++++-- 2 files changed, 50 insertions(+), 5 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index e08ea09b6..09c2fd31c 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -224,6 +224,9 @@ def on_block(store: Store, block: BeaconBlock) -> None: def on_attestation(store: Store, attestation: Attestation) -> None: target = attestation.data.target + # Attestations must be from the current or previous epoch + current_epoch = compute_epoch_at_slot(get_current_slot(store)) + assert target.epoch in [current_epoch, current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH] # Cannot calculate the current shuffling if have not seen the target assert target.root in store.blocks diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py index 70375ef27..3a513cf2f 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_attestation.py @@ -29,10 +29,9 @@ def run_on_attestation(spec, state, store, attestation, valid=True): @with_all_phases @spec_state_test -def test_on_attestation(spec, state): +def test_on_attestation_current_epoch(spec, state): store = spec.get_genesis_store(state) - time = 100 - spec.on_tick(store, time) + spec.on_tick(store, store.time + spec.SECONDS_PER_SLOT * 2) block = build_empty_block_for_next_slot(spec, state) state_transition_and_sign_block(spec, state, block) @@ -41,9 +40,53 @@ def test_on_attestation(spec, state): spec.on_block(store, block) attestation = get_valid_attestation(spec, state, slot=block.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + run_on_attestation(spec, state, store, attestation) +@with_all_phases +@spec_state_test +def test_on_attestation_previous_epoch(spec, state): + store = spec.get_genesis_store(state) + spec.on_tick(store, store.time + spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH) + + block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, block) + + # store block in store + spec.on_block(store, block) + + attestation = get_valid_attestation(spec, state, slot=block.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + 1 + + run_on_attestation(spec, state, store, attestation) + + +@with_all_phases +@spec_state_test +def test_on_attestation_past_epoch(spec, state): + store = spec.get_genesis_store(state) + + # move time forward 2 epochs + time = store.time + 2 * spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH + spec.on_tick(store, time) + + # create and store block from 3 epochs ago + block = build_empty_block_for_next_slot(spec, state) + state_transition_and_sign_block(spec, state, block) + spec.on_block(store, block) + + # create attestation for past block + attestation = get_valid_attestation(spec, state, slot=state.slot) + assert attestation.data.target.epoch == spec.GENESIS_EPOCH + assert spec.compute_epoch_at_slot(spec.get_current_slot(store)) == spec.GENESIS_EPOCH + 2 + + run_on_attestation(spec, state, store, attestation, False) + + @with_all_phases @spec_state_test def test_on_attestation_target_not_in_store(spec, state): @@ -77,8 +120,7 @@ def test_on_attestation_future_epoch(spec, state): spec.on_block(store, block) # move state forward but not store - attestation_slot = block.slot + spec.SLOTS_PER_EPOCH - state.slot = attestation_slot + state.slot = block.slot + spec.SLOTS_PER_EPOCH attestation = get_valid_attestation(spec, state, slot=state.slot) run_on_attestation(spec, state, store, attestation, False) From fc40bff2a8298b717fd5ceb6fa3d3a02d83a2659 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:10:32 -0700 Subject: [PATCH 14/23] use best_justified_checkpoint instead of queued_justified_checkpoints --- specs/core/0_fork-choice.md | 26 +++++------ .../test/fork_choice/test_on_block.py | 2 +- .../eth2spec/test/fork_choice/test_on_tick.py | 43 +++++-------------- 3 files changed, 25 insertions(+), 46 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index e08ea09b6..9e38fd548 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -69,7 +69,7 @@ class Store(object): genesis_time: uint64 justified_checkpoint: Checkpoint finalized_checkpoint: Checkpoint - queued_justified_checkpoints: List[Checkpoint, 2**40] = field(default_factory=list) + best_justified_checkpoint: Checkpoint blocks: Dict[Hash, BeaconBlock] = field(default_factory=dict) block_states: Dict[Hash, BeaconState] = field(default_factory=dict) checkpoint_states: Dict[Checkpoint, BeaconState] = field(default_factory=dict) @@ -89,6 +89,7 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: genesis_time=genesis_state.genesis_time, justified_checkpoint=justified_checkpoint, finalized_checkpoint=finalized_checkpoint, + best_justified_checkpoint=justified_checkpoint, blocks={root: genesis_block}, block_states={root: genesis_state.copy()}, checkpoint_states={justified_checkpoint: genesis_state.copy()}, @@ -147,14 +148,17 @@ def get_head(store: Store) -> Hash: #### `should_update_justified_checkpoint` ```python -def should_update_justified_checkpoint(store: Store, justified_checkpoint: Checkpoint) -> bool: +def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True - justified_block = store.blocks[justified_checkpoint.root] - if justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): + new_justified_block = store.blocks[new_justified_checkpoint.root] + if new_justified_block.slot <= compute_start_slot_at_epoch(store.justified_checkpoint.epoch): return False - if not get_ancestor(store, justified_checkpoint.root, store.blocks[justified_checkpoint.root].slot): + if not ( + get_ancestor(store, new_justified_checkpoint.root, store.blocks[store.justified_checkpoint.root].slot) == + store.justified_checkpoint.root + ): return False return True @@ -175,12 +179,9 @@ def on_tick(store: Store, time: uint64) -> None: # Not a new epoch, return if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): return - # If new epoch and there are queued_justified_checkpoints, update if any is better than the best in store - if any(store.queued_justified_checkpoints): - best_justified_checkpoint = max(store.queued_justified_checkpoints, key=lambda checkpoint: checkpoint.epoch) - if best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: - store.justified_checkpoint = best_justified_checkpoint - store.queued_justified_checkpoints = [] + # Update store.justified_checkpoint if a better checkpoint is know + if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.justified_checkpoint = store.best_justified_checkpoint ``` #### `on_block` @@ -208,10 +209,9 @@ def on_block(store: Store, block: BeaconBlock) -> None: # Update justified checkpoint if state.current_justified_checkpoint.epoch > store.justified_checkpoint.epoch: + store.best_justified_checkpoint = state.current_justified_checkpoint if should_update_justified_checkpoint(store, state.current_justified_checkpoint): store.justified_checkpoint = state.current_justified_checkpoint - else: - store.queued_justified_checkpoints.append(state.current_justified_checkpoint) # Update finalized checkpoint if state.finalized_checkpoint.epoch > store.finalized_checkpoint.epoch: diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py index 36e61f0b5..e42cb1ac4 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_block.py @@ -202,4 +202,4 @@ def test_on_block_outside_safe_slots_and_old_block(spec, state): run_on_block(spec, store, block) assert store.justified_checkpoint != new_justified - assert store.queued_justified_checkpoints[0] == new_justified + assert store.best_justified_checkpoint == new_justified diff --git a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py index 2ba89ebca..77222f65c 100644 --- a/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py +++ b/test_libs/pyspec/eth2spec/test/fork_choice/test_on_tick.py @@ -1,7 +1,7 @@ from eth2spec.test.context import with_all_phases, spec_state_test -def run_on_tick(spec, store, time, new_justified_checkpoint=None): +def run_on_tick(spec, store, time, new_justified_checkpoint=False): previous_justified_checkpoint = store.justified_checkpoint spec.on_tick(store, time) @@ -9,8 +9,9 @@ def run_on_tick(spec, store, time, new_justified_checkpoint=None): assert store.time == time if new_justified_checkpoint: - assert store.justified_checkpoint == new_justified_checkpoint + assert store.justified_checkpoint == store.best_justified_checkpoint assert store.justified_checkpoint.epoch > previous_justified_checkpoint.epoch + assert store.justified_checkpoint.root != previous_justified_checkpoint.root else: assert store.justified_checkpoint == previous_justified_checkpoint @@ -28,30 +29,12 @@ def test_update_justified_single(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) - - -@with_all_phases -@spec_state_test -def test_update_justified_multiple(spec, state): - store = spec.get_genesis_store(state) - seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - - new_justified = None # remember checkpoint with latest epoch - for i in range(3): - new_justified = spec.Checkpoint( - epoch=store.justified_checkpoint.epoch + i + 1, - root=i.to_bytes(1, byteorder='big') * 32, - ) - store.queued_justified_checkpoints.append(new_justified) - - run_on_tick(spec, store, store.time + seconds_per_epoch, new_justified) + run_on_tick(spec, store, store.time + seconds_per_epoch, True) @with_all_phases @@ -60,11 +43,10 @@ def test_no_update_same_slot_at_epoch_boundary(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) # set store time to already be at epoch boundary store.time = seconds_per_epoch @@ -77,11 +59,10 @@ def test_no_update_same_slot_at_epoch_boundary(spec, state): def test_no_update_not_epoch_boundary(spec, state): store = spec.get_genesis_store(state) - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) run_on_tick(spec, store, store.time + spec.SECONDS_PER_SLOT) @@ -92,14 +73,13 @@ def test_no_update_new_justified_equal_epoch(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) store.justified_checkpoint = spec.Checkpoint( - epoch=new_justified.epoch, + epoch=store.best_justified_checkpoint.epoch, root=b'\44' * 32, ) @@ -112,14 +92,13 @@ def test_no_update_new_justified_later_epoch(spec, state): store = spec.get_genesis_store(state) seconds_per_epoch = spec.SECONDS_PER_SLOT * spec.SLOTS_PER_EPOCH - new_justified = spec.Checkpoint( + store.best_justified_checkpoint = spec.Checkpoint( epoch=store.justified_checkpoint.epoch + 1, root=b'\x55' * 32, ) - store.queued_justified_checkpoints.append(new_justified) store.justified_checkpoint = spec.Checkpoint( - epoch=new_justified.epoch + 1, + epoch=store.best_justified_checkpoint.epoch + 1, root=b'\44' * 32, ) From 09fd49ce89f07728a267bebc9bc3a63cc9428e94 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:20:21 -0700 Subject: [PATCH 15/23] use helper for slots since epoch start --- specs/core/0_fork-choice.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 9e38fd548..ac1de0b3d 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -96,11 +96,20 @@ def get_genesis_store(genesis_state: BeaconState) -> Store: ) ``` +#### `get_current_slot` + ```python def get_current_slot(store: Store) -> Slot: return Slot((store.time - store.genesis_time) // SECONDS_PER_SLOT) ``` +#### `compute_slots_since_epoch_start` + +```python +def compute_slots_since_epoch_start(slot: Slot) -> int: + return slot - compute_start_slot_at_epoch(compute_epoch_at_slot(slot)) +``` + #### `get_ancestor` ```python @@ -149,7 +158,7 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: - if get_current_slot(store) % SLOTS_PER_EPOCH < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: + if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True new_justified_block = store.blocks[new_justified_checkpoint.root] From a28c02794319d3b7cf6a1c96f541f53b647b151b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 6 Nov 2019 17:26:06 -0700 Subject: [PATCH 16/23] be explicit about use of genesis epoch for previous epoch in fork choice on_block --- specs/core/0_fork-choice.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index 34948fbce..68e681fc7 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -233,9 +233,11 @@ def on_block(store: Store, block: BeaconBlock) -> None: def on_attestation(store: Store, attestation: Attestation) -> None: target = attestation.data.target - # Attestations must be from the current or previous epoch + # Attestations must be from the current or previous epoch current_epoch = compute_epoch_at_slot(get_current_slot(store)) - assert target.epoch in [current_epoch, current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH] + # Use GENESIS_EPOCH for previous when genesis to avoid underflow + previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else GENESIS_EPOCH + assert target.epoch in [current_epoch, previous_epoch] # Cannot calculate the current shuffling if have not seen the target assert target.root in store.blocks From bf78a711524e964dc616c10cbd3ca561d050b830 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 11:51:53 -0700 Subject: [PATCH 17/23] pr feedback --- specs/core/0_fork-choice.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/specs/core/0_fork-choice.md b/specs/core/0_fork-choice.md index ac1de0b3d..468d1e3d6 100644 --- a/specs/core/0_fork-choice.md +++ b/specs/core/0_fork-choice.md @@ -158,6 +158,13 @@ def get_head(store: Store) -> Hash: ```python def should_update_justified_checkpoint(store: Store, new_justified_checkpoint: Checkpoint) -> bool: + """ + To address the bouncing attack, only update conflicting justified + checkpoints in the fork choice if in the early slots of the epoch. + Otherwise, delay incorporation of new justified checkpoint until next epoch boundary. + + See https://ethresear.ch/t/prevention-of-bouncing-attack-on-ffg/6114 for more detailed analysis and discussion. + """ if compute_slots_since_epoch_start(get_current_slot(store)) < SAFE_SLOTS_TO_UPDATE_JUSTIFIED: return True @@ -186,9 +193,9 @@ def on_tick(store: Store, time: uint64) -> None: current_slot = get_current_slot(store) # Not a new epoch, return - if not (current_slot > previous_slot and current_slot % SLOTS_PER_EPOCH == 0): + if not (current_slot > previous_slot and compute_slots_since_epoch_start(current_slot) == 0): return - # Update store.justified_checkpoint if a better checkpoint is know + # Update store.justified_checkpoint if a better checkpoint is known if store.best_justified_checkpoint.epoch > store.justified_checkpoint.epoch: store.justified_checkpoint = store.best_justified_checkpoint ``` From 9b21c0db933679eb381aa5ab4105f4445a9607ff Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:06:23 -0700 Subject: [PATCH 18/23] add note aboutgenesis attestations --- specs/validator/0_beacon-chain-validator.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 166534031..fa7fce8ea 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -291,6 +291,8 @@ A validator is expected to create, sign, and broadcast an attestation during eac A validator should create and broadcast the `attestation` to the associated attestation subnet one-third of the way through the `slot` during which the validator is assigned―that is, `SECONDS_PER_SLOT / 3` seconds after the start of `slot`. +*Note*: Although attestations during `GENESIS_EPOCH` do not count toward FFG finality, these initial attestations do give weight to the fork choice, are rewarded fork, and should be made. + #### Attestation data First, the validator should construct `attestation_data`, an [`AttestationData`](../core/0_beacon-chain.md#attestationdata) object based upon the state at the assigned slot. From 5159c69632a86f63c50abd0516ca5d8548dc7069 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:15:56 -0700 Subject: [PATCH 19/23] cleanup get_eth1_vote --- specs/validator/0_beacon-chain-validator.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f6c3b55da..a985047b6 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -236,11 +236,13 @@ def get_eth1_vote(state: BeaconState, previous_eth1_distance: uint64) -> Eth1Dat new_eth1_data = [get_eth1_data(distance) for distance in range(ETH1_FOLLOW_DISTANCE, 2 * ETH1_FOLLOW_DISTANCE)] all_eth1_data = [get_eth1_data(distance) for distance in range(ETH1_FOLLOW_DISTANCE, previous_eth1_distance)] - valid_votes = [] - for slot, vote in enumerate(state.eth1_data_votes): - period_tail = slot % SLOTS_PER_ETH1_VOTING_PERIOD >= integer_squareroot(SLOTS_PER_ETH1_VOTING_PERIOD) - if vote in new_eth1_data or (period_tail and vote in all_eth1_data): - valid_votes.append(vote) + period_tail = state.slot % SLOTS_PER_ETH1_VOTING_PERIOD >= integer_squareroot(SLOTS_PER_ETH1_VOTING_PERIOD) + if period_tail: + votes_to_consider = all_eth1_data + else: + votes_to_consider = new_eth1_data + + valid_votes = [vote for vote in state.eth1_data_votes if vote in votes_to_consider] return max( valid_votes, From c84dce7b32985befb27b2871b9a7fba278774aa7 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 7 Nov 2019 12:32:02 -0700 Subject: [PATCH 20/23] make eth1_follow_distance clearer --- specs/validator/0_beacon-chain-validator.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f6c3b55da..86cd1e083 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -229,7 +229,9 @@ def get_epoch_signature(state: BeaconState, block: BeaconBlock, privkey: int) -> The `block.eth1_data` field is for block proposers to vote on recent Eth1 data. This recent data contains an Eth1 block hash as well as the associated deposit root (as calculated by the `get_deposit_root()` method of the deposit contract) and deposit count after execution of the corresponding Eth1 block. If over half of the block proposers in the current Eth1 voting period vote for the same `eth1_data` then `state.eth1_data` updates at the end of the voting period. Each deposit in `block.body.deposits` must verify against `state.eth1_data.eth1_deposit_root`. -Let `get_eth1_data(distance: uint64) -> Eth1Data` be the (subjective) function that returns the Eth1 data at distance `distance` relative to the Eth1 head at the start of the current Eth1 voting period. Let `previous_eth1_distance` be the distance relative to the Eth1 block corresponding to `state.eth1_data.block_hash` at the start of the current Eth1 voting period. An honest block proposer sets `block.eth1_data = get_eth1_vote(state, previous_eth1_distance)` where: +Let `get_eth1_data(distance: uint64) -> Eth1Data` be the (subjective) function that returns the Eth1 data at distance `distance` relative to the Eth1 head at the start of the current Eth1 voting period. Let `previous_eth1_distance` be the distance relative to the Eth1 block corresponding to `eth1_data.block_hash` found in the state at the _start_ of the current Eth1 voting period. Note that `eth1_data` can be updated in the middle of a voting period and thus the starting `eth1_data.block_hash` must be stored separately. + +An honest block proposer sets `block.eth1_data = get_eth1_vote(state, previous_eth1_distance)` where: ```python def get_eth1_vote(state: BeaconState, previous_eth1_distance: uint64) -> Eth1Data: From a029db34102b81bad246007b5b8eba632ea688eb Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Fri, 8 Nov 2019 19:05:14 +0800 Subject: [PATCH 21/23] Update the expected proposer period Since `SECONDS_PER_SLOT` is now `12` --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 38ae343fd..819f1ac98 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -195,7 +195,7 @@ A validator has two primary responsibilities to the beacon chain: [proposing blo A validator is expected to propose a [`BeaconBlock`](../core/0_beacon-chain.md#beaconblock) at the beginning of any slot during which `is_proposer(state, validator_index)` returns `True`. To propose, the validator selects the `BeaconBlock`, `parent`, that in their view of the fork choice is the head of the chain during `slot - 1`. The validator creates, signs, and broadcasts a `block` that is a child of `parent` that satisfies a valid [beacon chain state transition](../core/0_beacon-chain.md#beacon-chain-state-transition-function). -There is one proposer per slot, so if there are N active validators any individual validator will on average be assigned to propose once per N slots (e.g. at 312,500 validators = 10 million ETH, that's once per ~3 weeks). +There is one proposer per slot, so if there are N active validators any individual validator will on average be assigned to propose once per N slots (e.g. at 312,500 validators = 10 million ETH, that's once per ~6 weeks). #### Block header From b376a1387c6386135bdfcd28c916ee69d9b1f48f Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 8 Nov 2019 11:34:14 -0700 Subject: [PATCH 22/23] minor fix to comment in mainnet config --- configs/mainnet.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configs/mainnet.yaml b/configs/mainnet.yaml index e1b0faa3c..a9459fb64 100644 --- a/configs/mainnet.yaml +++ b/configs/mainnet.yaml @@ -58,7 +58,7 @@ BLS_WITHDRAWAL_PREFIX: 0x00 # --------------------------------------------------------------- # 12 seconds SECONDS_PER_SLOT: 12 -# 2**0 (= 1) slots 6 seconds +# 2**0 (= 1) slots 12 seconds MIN_ATTESTATION_INCLUSION_DELAY: 1 # 2**5 (= 32) slots 6.4 minutes SLOTS_PER_EPOCH: 32 From 931ad45c538384fa28b74cf1f49dcd3d523d8397 Mon Sep 17 00:00:00 2001 From: terence tsao Date: Thu, 31 Oct 2019 21:31:08 -0700 Subject: [PATCH 23/23] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 540d8192d..4a952a2c6 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -202,8 +202,8 @@ The following values are (non-configurable) constants used throughout the specif | `SLOTS_PER_EPOCH` | `2**5` (= 32) | slots | 6.4 minutes | | `MIN_SEED_LOOKAHEAD` | `2**0` (= 1) | epochs | 6.4 minutes | | `MAX_SEED_LOOKAHEAD` | `2**2` (= 4) | epochs | 25.6 minutes | -| `SLOTS_PER_ETH1_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours | -| `SLOTS_PER_HISTORICAL_ROOT` | `2**13` (= 8,192) | slots | ~13 hours | +| `SLOTS_PER_ETH1_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~3.4 hours | +| `SLOTS_PER_HISTORICAL_ROOT` | `2**13` (= 8,192) | slots | ~27 hours | | `MIN_VALIDATOR_WITHDRAWABILITY_DELAY` | `2**8` (= 256) | epochs | ~27 hours | | `PERSISTENT_COMMITTEE_PERIOD` | `2**11` (= 2,048) | epochs | 9 days | | `MIN_EPOCHS_TO_INACTIVITY_PENALTY` | `2**2` (= 4) | epochs | 25.6 minutes |