From 79d6b49a904cbdba4de9b8c98eba649f381f93d4 Mon Sep 17 00:00:00 2001 From: Giuseppe Bertone Date: Fri, 10 Apr 2020 17:38:37 +0200 Subject: [PATCH 1/9] Fixed target compile_deposit_contract Path of validator_registration.vy contract was wrong --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index e8f3d21bc..e53aaf8a2 100644 --- a/Makefile +++ b/Makefile @@ -117,7 +117,7 @@ install_deposit_contract_compiler: compile_deposit_contract: cd $(DEPOSIT_CONTRACT_COMPILER_DIR); . venv/bin/activate; \ - python3.7 deposit_contract/compile.py contracts/validator_registration.vy + python3.7 deposit_contract/compile.py ../contracts/validator_registration.vy test_compile_deposit_contract: cd $(DEPOSIT_CONTRACT_COMPILER_DIR); . venv/bin/activate; \ From e58cfedb6830d54ab22b08db56a1773f4b0d3797 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 16 Apr 2020 11:12:24 -0600 Subject: [PATCH 2/9] clarify ssz_snappy for gossip --- specs/phase0/p2p-interface.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index c461c2d3b..7d733e48e 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -345,7 +345,9 @@ Topics are post-fixed with an encoding. Encodings define how the payload of a go #### Mainnet -- `ssz_snappy` - All objects are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). Example: The beacon aggregate attestation topic string is `/eth2/beacon_aggregate_and_proof/ssz_snappy`, and the data field of a gossipsub message is an `AggregateAndProof` that has been SSZ-encoded and then compressed with Snappy. +- `ssz_snappy` - All objects are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy) block compression. Example: The beacon aggregate attestation topic string is `/eth2/beacon_aggregate_and_proof/ssz_snappy`, and the data field of a gossipsub message is an `AggregateAndProof` that has been SSZ-encoded and then compressed with Snappy. + +Snappy has two formats: "block" and "frames" (streaming). Gossip messages remain relatively small (100s of bytes to 100s of kilobytes) so [basic snappy block compression](https://github.com/google/snappy/blob/master/format_description.txt) is used to avoid the additional overhead associated with snappy frames. Implementations MUST use a single encoding. Changing an encoding will require coordination between participating implementations. @@ -448,7 +450,7 @@ Here, `result` represents the 1-byte response code. The token of the negotiated protocol ID specifies the type of encoding to be used for the req/resp interaction. Two values are possible at this time: - `ssz`: the contents are [SSZ-encoded](../../ssz/simple-serialize.md). This encoding type MUST be supported by all clients. For objects containing a single field, only the field is SSZ-encoded not a container with a single field. For example, the `BeaconBlocksByRoot` request is an SSZ-encoded list of `Root`'s. -- `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy). MAY be supported in the interoperability testnet; MUST be supported in mainnet. +- `ssz_snappy`: The contents are SSZ-encoded and then compressed with [Snappy](https://github.com/google/snappy) frames compression. MAY be supported in the interoperability testnet; MUST be supported in mainnet. #### SSZ-encoding strategy (with or without Snappy) From 4915014a19bb8d67efac833ecb065072846e6a5d Mon Sep 17 00:00:00 2001 From: Jacek Sieka Date: Mon, 20 Apr 2020 20:01:57 +0200 Subject: [PATCH 3/9] simplify block range request description There's room for ambiguity as to what `count` means - this clarifies that it always relates to the slot, and not the number of blocks in the response which allows clients to request ranges epoch by epoch (for example) without worrying about overlaps caused by empty slots. --- specs/phase0/p2p-interface.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index 7d733e48e..eff6a7c4f 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -463,7 +463,7 @@ Snappy has two formats: "block" and "frames" (streaming). To support large reque Since snappy frame contents [have a maximum size of `65536` bytes](https://github.com/google/snappy/blob/master/framing_format.txt#L104) and frame headers are just `identifier (1) + checksum (4)` bytes, the expected buffering of a single frame is acceptable. -**Encoding-dependent header:** Req/Resp protocols using the `ssz` or `ssz_snappy` encoding strategies MUST encode the length of the raw SSZ bytes, encoded as an unsigned [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints). +**Encoding-dependent header:** Req/Resp protocols using the `ssz` or `ssz_snappy` encoding strategies MUST encode the length of the raw SSZ bytes, encoded as an unsigned [protobuf varint](https://developers.google.com/protocol-buffers/docs/encoding#varints). *Writing*: By first computing and writing the SSZ byte length, the SSZ encoder can then directly write the chunk contents to the stream. If Snappy is applied, it can be passed through a buffered Snappy writer to compress frame by frame. @@ -577,7 +577,7 @@ Response Content: ) ``` -Requests count beacon blocks from the peer starting from `start_slot`, leading up to the current head block as selected by fork choice. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at slots [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. A step value of 1 returns all blocks on the range `[start_slot, start_slot + count)`. +Requests beacon blocks in the slot range `[start_slot, start_slot + count * step)`, leading up to the current head block as selected by fork choice. `step` defines the slot increment between blocks. For example, requesting blocks starting at `start_slot` 2 with a step value of 2 would return the blocks at slots [2, 4, 6, …]. In cases where a slot is empty for a given slot number, no block is returned. For example, if slot 4 were empty in the previous example, the returned array would contain [2, 6, …]. `BeaconBlocksByRange` is primarily used to sync historical blocks. From 508811d6417640c10d4866b12965bbe9e1ed6c13 Mon Sep 17 00:00:00 2001 From: Justin Date: Tue, 21 Apr 2020 08:50:42 +0100 Subject: [PATCH 4/9] =?UTF-8?q?Fix=20#1735=E2=80=94remove=20redundant=20ch?= =?UTF-8?q?eck?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit As per #1735 the check `if not len(indices) <= MAX_VALIDATORS_PER_COMMITTEE: return False` is redundant. As such this PR should be purely cosmetic. --- specs/phase0/beacon-chain.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index 23fa5ceee..a093c1098 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -684,14 +684,10 @@ def is_slashable_attestation_data(data_1: AttestationData, data_2: AttestationDa ```python def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: IndexedAttestation) -> bool: """ - Check if ``indexed_attestation`` has valid indices and signature. + Check if ``indexed_attestation`` has sorted and unique indices and a valid aggregate signature. """ - indices = indexed_attestation.attesting_indices - - # Verify max number of indices - if not len(indices) <= MAX_VALIDATORS_PER_COMMITTEE: - return False # Verify indices are sorted and unique + indices = indexed_attestation.attesting_indices if not indices == sorted(set(indices)): return False # Verify aggregate signature From e2a320ef32f0a7567a7d62008a35c2c4bcd9c312 Mon Sep 17 00:00:00 2001 From: Justin Date: Tue, 21 Apr 2020 08:59:53 +0100 Subject: [PATCH 5/9] Partial fix for #1701 Clarify that state transitions with `uint64` overflows are invalid. --- specs/phase0/beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index 23fa5ceee..01bf36c99 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -1195,7 +1195,7 @@ Let `genesis_block = BeaconBlock(state_root=hash_tree_root(genesis_state))`. ## Beacon chain state transition function -The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. +The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow are also considered invalid. ```python def state_transition(state: BeaconState, signed_block: SignedBeaconBlock, validate_result: bool=True) -> BeaconState: From 3436021e72019349f64910822ef862cddac086d4 Mon Sep 17 00:00:00 2001 From: Justin Date: Tue, 21 Apr 2020 15:34:55 +0100 Subject: [PATCH 6/9] Update beacon-chain.md --- specs/phase0/beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/phase0/beacon-chain.md b/specs/phase0/beacon-chain.md index 01bf36c99..87b207686 100644 --- a/specs/phase0/beacon-chain.md +++ b/specs/phase0/beacon-chain.md @@ -1195,7 +1195,7 @@ Let `genesis_block = BeaconBlock(state_root=hash_tree_root(genesis_state))`. ## Beacon chain state transition function -The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow are also considered invalid. +The post-state corresponding to a pre-state `state` and a signed block `signed_block` is defined as `state_transition(state, signed_block)`. State transitions that trigger an unhandled exception (e.g. a failed `assert` or an out-of-range list access) are considered invalid. State transitions that cause a `uint64` overflow or underflow are also considered invalid. ```python def state_transition(state: BeaconState, signed_block: SignedBeaconBlock, validate_result: bool=True) -> BeaconState: From 3d4122a2f6010d393f52f63d62068ce0641068b9 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 16 Apr 2020 11:56:15 -0600 Subject: [PATCH 7/9] add note about distributing bootnode ENRs prior to genesis --- .circleci/config.yml | 22 +++++++++++----------- specs/phase0/p2p-interface.md | 2 +- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 5c4b77e78..3a67e5528 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -79,16 +79,16 @@ jobs: # Restore git repo at point close to target branch/revision, to speed up checkout - restore_cache: keys: - - v2-specs-repo-{{ .Branch }}-{{ .Revision }} - - v2-specs-repo-{{ .Branch }}- - - v2-specs-repo- + - v3-specs-repo-{{ .Branch }}-{{ .Revision }} + - v3-specs-repo-{{ .Branch }}- + - v3-specs-repo- - checkout - run: name: Clean up git repo to reduce cache size command: git gc # Save the git checkout as a cache, to make cloning next time faster. - save_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} paths: - ~/specs-repo install_pyspec_test: @@ -97,7 +97,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_pyspec_cached_venv - run: name: Install pyspec requirements @@ -109,7 +109,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_pyspec_cached_venv - run: name: Run py-tests @@ -140,7 +140,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_pyspec_cached_venv - run: name: Run linter @@ -152,7 +152,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_deposit_contract_compiler_cached_venv - run: name: Install deposit contract compiler requirements @@ -164,7 +164,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_deposit_contract_tester_cached_venv - run: name: Install deposit contract tester requirements @@ -176,7 +176,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_deposit_contract_compiler_cached_venv - run: name: Run deposit contract compile test @@ -187,7 +187,7 @@ jobs: working_directory: ~/specs-repo steps: - restore_cache: - key: v2-specs-repo-{{ .Branch }}-{{ .Revision }} + key: v3-specs-repo-{{ .Branch }}-{{ .Revision }} - restore_deposit_contract_tester_cached_venv - run: name: Run deposit contract test diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index c461c2d3b..9573eef11 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -755,7 +755,7 @@ where the fields of `ENRForkID` are defined as * `next_fork_version` is the fork version corresponding to the next planned hard fork at a future epoch. If no future fork is planned, set `next_fork_version = current_fork_version` to signal this fact * `next_fork_epoch` is the epoch at which the next fork is planned and the `current_fork_version` will be updated. If no future fork is planned, set `next_fork_epoch = FAR_FUTURE_EPOCH` to signal this fact -*Note*: `fork_digest` is composed of values that are not not known until the genesis block/state are available. Due to this, clients SHOULD NOT form ENRs and begin peer discovery until genesis values are known. +*Note*: `fork_digest` is composed of values that are not not known until the genesis block/state are available. Due to this, clients SHOULD NOT form ENRs and begin peer discovery until genesis values are known. One notable exception to this rule is the distribution of bootnode ENRs prior to genesis. In this case, bootnode ENRs SHOULD be initially distributed with `eth2` field set as `ENRForkID(fork_digest=compute_fork_digest(GENESIS_FORK_VERSION, b'\x00'*32), next_fork_version=GENESIS_FORK_VERSION, next_fork_epoch=FAR_FUTURE_EPOCH)`. After genesis values are known, the bootnodes SHOULD update ENRs to participate in normal discovery operations. Clients SHOULD connect to peers with `fork_digest`, `next_fork_version`, and `next_fork_epoch` that match local values. From 11d164748c58580a34edb798a91387b720e45701 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 22 Apr 2020 14:45:01 -0600 Subject: [PATCH 8/9] add 'valid' when de-deduplication of attestations on gossip subnets --- specs/phase0/p2p-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/phase0/p2p-interface.md b/specs/phase0/p2p-interface.md index c66ec0b6e..d3b9150d1 100644 --- a/specs/phase0/p2p-interface.md +++ b/specs/phase0/p2p-interface.md @@ -319,7 +319,7 @@ Attestation subnets are used to propagate unaggregated attestations to subsectio - The attestation's committee index (`attestation.data.index`) is for the correct subnet. - `attestation.data.slot` is within the last `ATTESTATION_PROPAGATION_SLOT_RANGE` slots (within a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. `attestation.data.slot + ATTESTATION_PROPAGATION_SLOT_RANGE >= current_slot >= attestation.data.slot` (a client MAY queue future attestations for processing at the appropriate slot). - The attestation is unaggregated -- that is, it has exactly one participating validator (`len([bit for bit in attestation.aggregation_bits if bit == 0b1]) == 1`). - - There has been no other attestation seen on an attestation subnet that has an identical `attestation.data.target.epoch` and participating validator index. + - There has been no other valid attestation seen on an attestation subnet that has an identical `attestation.data.target.epoch` and participating validator index. - The block being voted for (`attestation.data.beacon_block_root`) passes validation. - The signature of `attestation` is valid. From 56535e3dbe6127728e37fb6c871e790eaa6dc96b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 24 Apr 2020 10:21:47 -0600 Subject: [PATCH 9/9] bump version to v0.11.2 --- tests/core/pyspec/eth2spec/VERSION.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/core/pyspec/eth2spec/VERSION.txt b/tests/core/pyspec/eth2spec/VERSION.txt index 027934ea1..a8839f70d 100644 --- a/tests/core/pyspec/eth2spec/VERSION.txt +++ b/tests/core/pyspec/eth2spec/VERSION.txt @@ -1 +1 @@ -0.11.1 \ No newline at end of file +0.11.2 \ No newline at end of file