From 1d44023731e6f0d23c123e02893e313bece1e47a Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 14 Jan 2019 21:25:23 -0600 Subject: [PATCH 001/136] initial pass on phase 0 validator doc --- specs/validator/0_beacon-chain-validator.md | 278 ++++++++++++++++++++ 1 file changed, 278 insertions(+) create mode 100644 specs/validator/0_beacon-chain-validator.md diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md new file mode 100644 index 000000000..c34a76ee3 --- /dev/null +++ b/specs/validator/0_beacon-chain-validator.md @@ -0,0 +1,278 @@ +# Ethereum 2.0 Phase 0 -- Honest Validator + +__NOTICE__: This document is a work-in-progress for researchers and implementers. This is an accompanying document to [Ethereum 2.0 Phase 0 -- The Beacon Chain](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) that describes the expected actions of a "validator" participating in the Ethereum 2.0 protocol. + +## Table of Contents + +## Introduction + +This document represents the expected behavior of an "honest validator" with respect to Phase 0 of the Ethereum 2.0 protocol. This document does not distinguish between a "node" and a "validator client". The separation of concerns between these (potentially) two pieces of software is left as a design decision that is outside of scope. + +A validator is an entity that participates in the consensus of the Ethereum 2.0 protocol. This is an optional role for users in which they can post ETH as collateral to seek financial returns in exchange for building and securing the protocol. This is similar to proof of work networks in which a miner provides collateral in the form of hardware/hash-power to seek returns in exchange for building and securing the protocol. + +## Prerequisites + +All terminology, constants, functions, and protocol mechanics defined in the [Phase 0 -- The Beacon Chain](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) doc are requisite for this document and used throughout. Please see the Phase 0 doc before continuing and use as a reference throughout. + +## Becoming a validator + +### Initialization + +A validator must initial many parameters locally before submitting a deposit and joining the validator registry. + +#### BLS public key + +Validator public keys are [G1 points](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#g1-points) on the [BLS12-381 curve](https://z.cash/blog/new-snark-curve). A private key, `privkey`, must be securely generated along with the resultant `pubkey`. This `privkey` must be "hot", that is, constantly available to sign data throughout the lifetime of the validator. + +#### BLS withdrawal key + +A secondary withdrawal private key, `withdrawal_privkey`, must also be securely generated along with the resultant `withdrawal_pubkey`. This `withdrawal_privkey` does not have to be available for signing during the normal lifetime of a validator and can live in "cold storage". + +The validator constructs their `withdrawal_credentials` through the following: +* Set `withdrawal_credentials[:1] == BLS_WITHDRAWAL_PREFIX_BYTE`. +* Set `withdrawal_credentials[1:] == hash(withdrawal_pubkey)[1:]`. + +#### RANDAO commitment + +A validator's RANDAO commitment is the outermost layer of a 32-byte hash-onion. To create this commitment, perform the following steps: + +* Randomly generate a 32-byte `randao_seed`. +* Store this `randao_seed` in a secure location. +* Calculate `randao_commitment = repeat_hash(randao_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not propose more than `n` beacon chain blocks. + +Assuming `>= 100k validators`, on average a validator will have an opportunity to reveal once every `>= 600k seconds`, so `<= 50 times per year`. At this estimate, `n == 5000` would last a century. To be conservative, we recommend `n >= 100k`. + +_Note_: A validator must be able to reveal the next layer deep from their current commitment at any time. There are many strategies that trade off space and computation to be able to provide this reveal. At one end of this trade-off, a validator might only store their `randao_seed` and repeat the `repeat_hash` calculation on the fly to re-calculate the layer `n-1` for the reveal. On the other end of this trade-off, a validator might store _all_ layers of the hash-onion and not have to perform any calculations to retrieve the layer `n-1`. A more sensible strategy might be to store every `m`th layer as cached references to recalculate the intermittent layers as needed. + +#### Custody commitment + +A validator's custody commitment is the outermost layer of a 32-byte hash-onion. To create this commitment, perform the following steps: + +* Randomly generate a 32-byte `custody_seed`. +* Store this `custody_seed` in a secutre location. +* Calculate `custody_commitment = repeat_hash(custody_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not attest to more than `n` beacon chain blocks. + +Assuming a validator changes their `custody_seed` with frequency `>= 1 week`, the validator changes their seed approximately `<= 50 times per year`. At this estimate, `n == 5000` would last a century. To be conservative, we recommend `n >= 100k`. + +See above note on hash-onion caching strategies in [RANDAO commitment](). + +_Note_: although this commitment is being committed to and stored in phase 0, it will not be used until phase 1. + +### Deposit + +In phase 0, all incoming validator deposits originate from the Ethereum 1.0 PoW chain. Deposits are made to the [deposit contract](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#ethereum-10-deposit-contract) located at `DEPOSIT_CONTRACT_ADDRESS`. + +To submit a deposit: + +* Pack the validator's [initialization parameters]() into `deposit_input`, a [`DepositInput`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#depositinput) object. +* Set `deposit_input.proof_of_possession = EMPTY_SIGNATURE`. +* Let `proof_of_possession` be the result of `bls_sign` of the `hash_tree_root(deposit_input)` with `domain=DOMAIN_DEPOSIT`. +* Set `deposit_input.proof_of_possession = proof_of_possession`. +* Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in ETH. + +### Validator index + +Once a validator has been added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`]() contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. + +### Activation + +A validator is activated some amount of slots after being added to the registry. There is a maximum validator churn per finalized epoch so the delay until activation is variable depending upon finality, total active validator balance, and the number of validators in the queue to be activated. + +The function [`is_active_validator`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_active_validator) can be used to check if a validator is active at a given slot. Usage is as follows: + +```python +validator = state.validator_registry[validator_index] +is_active_validator(validator, slot) +``` + +Once a validator is active, the validator is assigned [responsibilities]() until exited. + +## Beacon chain responsibilities + +A validator has two primary responsibilities to the beacon chain -- [proposing blocks]() and [creating attestations](). Proposals happen infrequently, whereas attestations should be created once per epoch. + +### Block proposal + +A validator is expected to propose a [`BeaconBlock`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beaconblock) at the beginning of any slot during which `get_beacon_proposer_index(state, slot)` returns the validator's `validator_index`. To propose, the validator selects the `BeaconBlock`, `parent`, that in their view of the fork choice is the head of the chain during `slot`. The validator is to create, sign, and broadcast a `block` that is a child of `parent` that creates a valid [beacon chain state transition](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-state-transition-function). + +#### Block header + +##### Slot + +Set `block.slot = slot` where `slot` is the current slot at which the validator has been selected to propose. The `parent` selected must satisfy that `parent.slot < block.slot`. + +_Note:_ there might be "skipped" slots between the `parent` and `block`. These skipped slots are processed in the state transition function without per-block processing. + +##### Parent root + +Set `block.parent_root = hash_tree_root(parent)`. + +##### State root + +Set `block.state_root = hash_tree_root(state)` of the resulting `state` of the `parent -> block` state transition. + +_Note_: To calculate `state_root`, the validator should first run the state transition function on an unsigned `block` containing a stub for the `state_root`. It is useful to be able to run a state transition function that does _not_ validate signatures for this purpose. + +##### Randao reveal + +Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's current `randao_commitment` where `n = validator.randao_layers + 1`. `block.randao_reveal` should satisfy `repeat_hash(block.randao_reveal, validator.randao_layers + 1) == validator.randao_commitment`. + +##### Deposit root + +##### Signature + +Set `block.signature = signed_proposal_data` where `signed_proposal_data` is defined as: + +```python +proposal_data = ProposalSignedData( + slot=slot, + shard=BEACON_CHAIN_SHARD_NUMBER, + block_root=hash_tree_root(block), # where `block.sigature == EMPTY_SIGNATURE +) +proposal_root = hash_tree_root(proposal_data) + +signed_proposal_data = bls_sign( + privkey=validator.privkey, # privkey store locally, not in state + message=proposal_root, + domain=get_domain( + state.fork_data, # `state` is the resulting state of `block` transition + state.slot, + DOMAIN_PROPOSAL, + ) +) +``` + +#### Block body + +##### Proposer slashings + +##### Casper slashings + +##### Attestations + +Up to `MAX_ATTESTATIONS` aggregate attestations can be added to the `block`. The attestations added must satify the verification conditions found in [attestation processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestations-1). To maximize profit, the validator should attempt to add aggregate attestations that include the most available that have not previously been added on chain. + +##### Deposits + +##### Exits + +### Attestations + +A validator is expected to create, sign, and broadcast an attestion during each epoch. The slot during which the validator performs this roal is any slot at which `get_shard_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. + +A validator should create and broadcast the attestation halfway through the `slot` during which the validator is assigned -- that is `SLOT_DURATION * 0.5` seconds after the start of `slot`. + +#### Attestation data + +First the validator should construct `attestation_data`, an [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) object based upon the state at the assigned slot. + +##### Slot + +Set `attestation_data.slot = slot` where `slot` is the current slot of which the validator is a member of a committee. + +##### Shard + +Set `attestation_data.shard = shard` where `shard` is the shard associated with the validator's committee defined by `get_shard_committees_at_slot`. + +##### Beacon block root + +Set `attestation_data.beacon_block_root = hash_tree_root(head)` where `head` is the validator's view of the `head` block of the beacon chain during `slot`. + +##### Epoch boundary root + +Set `attestation_data.epoch_boundary_root = hash_tree_root(epoch_boundary)` where `epoch_boundary` is the block at the most recent epoch boundary in the chain defined by `head` -- i.e. the `BeaconBlock` with `slot == head.slot - head.slot % EPOCH_LENGTH`. + +_Note:_ This can be looked up in the state using `get_block_root(state, head.slot - head.slot % EPOCH_LENGTH)`. + +##### Shard block root + +Set `attestation_data.shard_block_root = ZERO_HASH`. + +_Note:_ This is a stub for phase 0. + +##### Latest crosslink root + +Set `attestation_data.latest_crosslink_root = state.latest_crosslinks[shard].shard_block_root` where `state` is the beacon state at `head` and `shard` is the validator's assigned shard. + +##### Justified slot + +Set `attestation_data.justified_slot = state.justified_slot` where `state` is the beacon state at `head`. + +##### Justified block root + +Set `attestation_data.justified_block_root = hash_tree_root(justified_block)` where `justified_block` is the block at `state.justified_slot` in the chain defined by `head`. + +_Note:_ This can be looked up in the state using `get_block_root(state, justified_slot)`. + +#### Construct attestation + +Next the validator creates `attestation`, an [`Attestation`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestation) object. + +##### Data + +Set `attestation.data = attestation_data` where `attestation_data` is the `AttestationData` object defined in the [previous section](). + +##### Participation bitfield + +* Let `participation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. +* Let `index_into_committee` be the index into the validator's `committee` at which `validator_index` is located. +* Set `participation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. +* Set `attestation.participation_bitfield = participation_bitfield`. + +_Note_: Calling `get_attestation_participants(state, attestation.data, attestation.participation_bitfield)` should return `[validator_index]`. + +##### Custody bitfield + +* Let `custody_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. +* Set `attestation.custody_bitfield = custody_bitfield`. + +_Note:_ This is a stub for phase 0. + +##### Aggregate signature + +Set `attestation.aggregate_signature = signed_attestation_data` where `signed_attestation_data` is defined as: + +```python +attestation_data_and_custody_bit = AttestationDataAndCustodyBit( + attestation.data, + False, +) +attestation_message_to_sign = hash_tree_root(attestation_data_and_custody_bit) + +signed_attestation_data = bls_sign( + privkey=validator.privkey, # privkey store locally, not in state + message=attestation_message_to_sign, + domain=get_domain( + state.fork_data, # `state` is the state at `head` + state.slot, + DOMAIN_ATTESTATION, + ) +) +``` + +## How to avoid slashing + +"Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing]() and [attestation slashing](). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. + +_Note_: signed data must be within the same `ForkData` context to conflict. Messages cannot be slashed across forks. + +### Proposal slashing + +To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`]() (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. + +The following helper can be run to check if two proposal messages conflict: + +```python +def proposal_data_is_slashable(proposal_data_1: ProposalSignedData, + proposal_data_2: ProposalSignedData) -> bool: + if (proposal_data_1.slot != proposal_data_2.slot): + return False + if (proposal_data_1.shard != proposal_data_2.shard): + return False + + return proposal_data_1.block_root != proposal_data_2.block_root +``` + +### Casper slashing + +To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`]() objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`]() or [`is_surround_vote`](). From 1070ba2d114c072ec948c5b04531b5db1adca9cc Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 15 Jan 2019 11:33:46 +0800 Subject: [PATCH 002/136] Add ToC --- specs/validator/0_beacon-chain-validator.md | 51 +++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index c34a76ee3..77ec0de6a 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -4,6 +4,57 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers ## Table of Contents + + +- [Ethereum 2.0 Phase 0 -- Honest Validator](#ethereum-20-phase-0----honest-validator) + - [Table of Contents](#table-of-contents) + - [Introduction](#introduction) + - [Prerequisites](#prerequisites) + - [Becoming a validator](#becoming-a-validator) + - [Initialization](#initialization) + - [BLS public key](#bls-public-key) + - [BLS withdrawal key](#bls-withdrawal-key) + - [RANDAO commitment](#randao-commitment) + - [Custody commitment](#custody-commitment) + - [Deposit](#deposit) + - [Validator index](#validator-index) + - [Activation](#activation) + - [Beacon chain responsibilities](#beacon-chain-responsibilities) + - [Block proposal](#block-proposal) + - [Block header](#block-header) + - [Slot](#slot) + - [Parent root](#parent-root) + - [State root](#state-root) + - [Randao reveal](#randao-reveal) + - [Deposit root](#deposit-root) + - [Signature](#signature) + - [Block body](#block-body) + - [Proposer slashings](#proposer-slashings) + - [Casper slashings](#casper-slashings) + - [Attestations](#attestations) + - [Deposits](#deposits) + - [Exits](#exits) + - [Attestations](#attestations-1) + - [Attestation data](#attestation-data) + - [Slot](#slot-1) + - [Shard](#shard) + - [Beacon block root](#beacon-block-root) + - [Epoch boundary root](#epoch-boundary-root) + - [Shard block root](#shard-block-root) + - [Latest crosslink root](#latest-crosslink-root) + - [Justified slot](#justified-slot) + - [Justified block root](#justified-block-root) + - [Construct attestation](#construct-attestation) + - [Data](#data) + - [Participation bitfield](#participation-bitfield) + - [Custody bitfield](#custody-bitfield) + - [Aggregate signature](#aggregate-signature) + - [How to avoid slashing](#how-to-avoid-slashing) + - [Proposal slashing](#proposal-slashing) + - [Casper slashing](#casper-slashing) + + + ## Introduction This document represents the expected behavior of an "honest validator" with respect to Phase 0 of the Ethereum 2.0 protocol. This document does not distinguish between a "node" and a "validator client". The separation of concerns between these (potentially) two pieces of software is left as a design decision that is outside of scope. From d29ce725db96a534c365f0e50d51ae1ea7e10b79 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 14 Jan 2019 21:50:34 -0600 Subject: [PATCH 003/136] add deposit root logic in block proposals --- specs/validator/0_beacon-chain-validator.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 77ec0de6a..f8d50e051 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -170,6 +170,12 @@ Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's cu ##### Deposit root +`block.deposit_root` is a mechanism used by block proposers to vote on a recent deposit root found in the Ethereum 1.0 PoW deposit contract. When consensus is formed `state.latest_deposit_root` is updated, and validator deposits up to this root can be processed. + +* Let `D` be the set of `DepositRootVote` objects `obj` in `state.deposit_root_votes` where `obj.deposit_root` is the deposit root of an eth1.0 block that is (i) part of the canonical chain, (ii) >= 1000 blocks behind the head, and (iii) newer than `state.latest_deposit_root`. +* If `D` is empty, set `block.deposit root` to the deposit root of the 1000th ancestor of the head of the canonical eth1.0 chain. +* If `D` is nonempty, set `block.deposit_root = best_vote.deposit_root` where `best_vote` is the member of `D` that has the highest `vote_count`, breaking ties by favoring newer deposit roots. + ##### Signature Set `block.signature = signed_proposal_data` where `signed_proposal_data` is defined as: From 6b72ae3a3b90b663f28f5637891892aa957d3cd4 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 14 Jan 2019 22:36:33 -0600 Subject: [PATCH 004/136] fill in missing links in phase 0 validator doc --- specs/validator/0_beacon-chain-validator.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f8d50e051..e98795b22 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -105,7 +105,7 @@ A validator's custody commitment is the outermost layer of a 32-byte hash-onion. Assuming a validator changes their `custody_seed` with frequency `>= 1 week`, the validator changes their seed approximately `<= 50 times per year`. At this estimate, `n == 5000` would last a century. To be conservative, we recommend `n >= 100k`. -See above note on hash-onion caching strategies in [RANDAO commitment](). +See above note on hash-onion caching strategies in [RANDAO commitment](#randao-commitment). _Note_: although this commitment is being committed to and stored in phase 0, it will not be used until phase 1. @@ -115,15 +115,18 @@ In phase 0, all incoming validator deposits originate from the Ethereum 1.0 PoW To submit a deposit: -* Pack the validator's [initialization parameters]() into `deposit_input`, a [`DepositInput`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#depositinput) object. +* Pack the validator's [initialization parameters](#initialization) into `deposit_input`, a [`DepositInput`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#depositinput) object. * Set `deposit_input.proof_of_possession = EMPTY_SIGNATURE`. * Let `proof_of_possession` be the result of `bls_sign` of the `hash_tree_root(deposit_input)` with `domain=DOMAIN_DEPOSIT`. * Set `deposit_input.proof_of_possession = proof_of_possession`. +* Let `amount` be the amount of eth to be deposited by the validator where `MIN_DEPOSIT <= amount <= MAX_DEPOSIT`. * Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in ETH. +_Note_: Multiple deposits can be made by the same validator (same initialization params). A singular `Validator` will be added to `state.validator_registry` with each deposit amount being added to the validator's balance. A validator can only be activated when total deposits meet or exceed + ### Validator index -Once a validator has been added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`]() contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. +Once a validator has been added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. ### Activation @@ -136,11 +139,11 @@ validator = state.validator_registry[validator_index] is_active_validator(validator, slot) ``` -Once a validator is active, the validator is assigned [responsibilities]() until exited. +Once a validator is active, the validator is assigned [responsibilities](#beacon-chain-responsibilities) until exited. ## Beacon chain responsibilities -A validator has two primary responsibilities to the beacon chain -- [proposing blocks]() and [creating attestations](). Proposals happen infrequently, whereas attestations should be created once per epoch. +A validator has two primary responsibilities to the beacon chain -- [proposing blocks](block-proposal) and [creating attestations](attestations-1). Proposals happen infrequently, whereas attestations should be created once per epoch. ### Block proposal @@ -267,7 +270,7 @@ Next the validator creates `attestation`, an [`Attestation`](https://github.com/ ##### Data -Set `attestation.data = attestation_data` where `attestation_data` is the `AttestationData` object defined in the [previous section](). +Set `attestation.data = attestation_data` where `attestation_data` is the `AttestationData` object defined in the previous section, [attestation data](#attestation-data). ##### Participation bitfield @@ -309,13 +312,13 @@ signed_attestation_data = bls_sign( ## How to avoid slashing -"Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing]() and [attestation slashing](). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. +"Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing](#proposal-slashing) and [attestation slashing](#casper-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. _Note_: signed data must be within the same `ForkData` context to conflict. Messages cannot be slashed across forks. ### Proposal slashing -To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`]() (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. +To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#proposalsigneddata) (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. The following helper can be run to check if two proposal messages conflict: @@ -332,4 +335,4 @@ def proposal_data_is_slashable(proposal_data_1: ProposalSignedData, ### Casper slashing -To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`]() objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`]() or [`is_surround_vote`](). +To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_surround_vote). From 2881f56c081183a87ac601b2102f972bab4afc07 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 16 Jan 2019 14:44:58 -0600 Subject: [PATCH 005/136] add pr feedback --- specs/validator/0_beacon-chain-validator.md | 24 +++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index e98795b22..c710886f9 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -69,7 +69,7 @@ All terminology, constants, functions, and protocol mechanics defined in the [Ph ### Initialization -A validator must initial many parameters locally before submitting a deposit and joining the validator registry. +A validator must initialize many parameters locally before submitting a deposit and joining the validator registry. #### BLS public key @@ -91,7 +91,7 @@ A validator's RANDAO commitment is the outermost layer of a 32-byte hash-onion. * Store this `randao_seed` in a secure location. * Calculate `randao_commitment = repeat_hash(randao_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not propose more than `n` beacon chain blocks. -Assuming `>= 100k validators`, on average a validator will have an opportunity to reveal once every `>= 600k seconds`, so `<= 50 times per year`. At this estimate, `n == 5000` would last a century. To be conservative, we recommend `n >= 100k`. +Assuming `>= 100k validators`, on average a validator will have an opportunity to reveal once every `>= 600k seconds`, so `<= 50 times per year`. At this estimate, `n == 5000` would last a century. If this value is poorly configured and a validator runs out of layers of to reveal, the validator can no longer propose beacon blocks and should exit. _Note_: A validator must be able to reveal the next layer deep from their current commitment at any time. There are many strategies that trade off space and computation to be able to provide this reveal. At one end of this trade-off, a validator might only store their `randao_seed` and repeat the `repeat_hash` calculation on the fly to re-calculate the layer `n-1` for the reveal. On the other end of this trade-off, a validator might store _all_ layers of the hash-onion and not have to perform any calculations to retrieve the layer `n-1`. A more sensible strategy might be to store every `m`th layer as cached references to recalculate the intermittent layers as needed. @@ -100,10 +100,10 @@ _Note_: A validator must be able to reveal the next layer deep from their curren A validator's custody commitment is the outermost layer of a 32-byte hash-onion. To create this commitment, perform the following steps: * Randomly generate a 32-byte `custody_seed`. -* Store this `custody_seed` in a secutre location. +* Store this `custody_seed` in a secure location. * Calculate `custody_commitment = repeat_hash(custody_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not attest to more than `n` beacon chain blocks. -Assuming a validator changes their `custody_seed` with frequency `>= 1 week`, the validator changes their seed approximately `<= 50 times per year`. At this estimate, `n == 5000` would last a century. To be conservative, we recommend `n >= 100k`. +Assuming a validator changes their `custody_seed` with frequency `>= 1 week`, the validator changes their seed approximately `<= 50 times per year`. At this estimate, `n == 5000` would last a century. If this value is poorly configured and a validator runs out of layers of to reveal, the validator can no longer update their `custody_commitment` and should exit. See above note on hash-onion caching strategies in [RANDAO commitment](#randao-commitment). @@ -218,7 +218,7 @@ Up to `MAX_ATTESTATIONS` aggregate attestations can be added to the `block`. The ### Attestations -A validator is expected to create, sign, and broadcast an attestion during each epoch. The slot during which the validator performs this roal is any slot at which `get_shard_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. +A validator is expected to create, sign, and broadcast an attestation during each epoch. The slot during which the validator performs this role is any slot at which `get_shard_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. A validator should create and broadcast the attestation halfway through the `slot` during which the validator is assigned -- that is `SLOT_DURATION * 0.5` seconds after the start of `slot`. @@ -314,7 +314,7 @@ signed_attestation_data = bls_sign( "Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing](#proposal-slashing) and [attestation slashing](#casper-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. -_Note_: signed data must be within the same `ForkData` context to conflict. Messages cannot be slashed across forks. +_Note_: Signed data must be within the same `Fork` context to conflict. Messages cannot be slashed across forks. ### Proposal slashing @@ -333,6 +333,18 @@ def proposal_data_is_slashable(proposal_data_1: ProposalSignedData, return proposal_data_1.block_root != proposal_data_2.block_root ``` +Specifically, when signing an `BeaconBlock`, a validator should perform the following steps in the following order: +1. Save a record to hard disk that an beacon block has been signed for the `slot` and `shard`. +2. Generate and broadcast the block. + +If the software crashes at some point within this routine, then when the validator comes back online the hard disk has the record of the _potentially_ signed/broadcast block and can effectively avoid slashing. + ### Casper slashing To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_surround_vote). + +Specifically, when signing an `Attestation`, a validator should perform the following steps in the following order: +1. Save a record to hard disk that an attestation has been signed for source -- `attestation_data.justified_slot // EPOCH_LENGTH` -- and target -- `attestation_data.slot // EPOCH_LENGTH`. +2. Generate and broadcast attestation. + +If the software crashes at some point within this routine, then when the validator comes back online the hard disk has the record of the _potentially_ signed/broadcast attestation and can effectively avoid slashing. From 40d4ec33cb4455187de3171703ceeab03a43aaba Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 16 Jan 2019 15:24:59 -0600 Subject: [PATCH 006/136] add basics for all block operations --- specs/validator/0_beacon-chain-validator.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index c710886f9..40cb0eadd 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -206,16 +206,24 @@ signed_proposal_data = bls_sign( ##### Proposer slashings +Up to `MAX_PROPOSER_SLASHINGS` [`ProposerSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposerslashing) objects can be included in the `block`. The proposer slashings must satisfy the verification conditions found in [proposer slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposer-slashings-1). The validator receives a small "whistleblower" reward for each proposer slashing found and included. + ##### Casper slashings +Up to `MAX_CASPER_SLASHINGS` [`CasperSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casperslashing) objects can be included in the `block`. The casper slashings must satisfy the verification conditions found in [casper slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casper-slashings-1). The validator receives a small "whistleblower" reward for each casper slashing found and included. + ##### Attestations -Up to `MAX_ATTESTATIONS` aggregate attestations can be added to the `block`. The attestations added must satify the verification conditions found in [attestation processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestations-1). To maximize profit, the validator should attempt to add aggregate attestations that include the most available that have not previously been added on chain. +Up to `MAX_ATTESTATIONS` aggregate attestations can be included in the `block`. The attestations added must satisfy the verification conditions found in [attestation processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestations-1). To maximize profit, the validator should attempt to add aggregate attestations that include the most available that have not previously been added on chain. ##### Deposits +Up to `MAX_DEPOSITS` [`Deposit`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#deposit) objects can be included in the `block`. These deposits are constructed from the `Deposit` logs from the [Eth1.0 deposit contract](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#ethereum-10-deposit-contract) and must be processed in sequential order. The deposits included in the `block` must satisfy the verification conditions found in [deposits processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#deposits-1). + ##### Exits +Up to `MAX_EXITS` [`Exit`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#exit) objects can be included in the `block`. The exits must satisfy the verification conditions found in [exits processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#exits-1). + ### Attestations A validator is expected to create, sign, and broadcast an attestation during each epoch. The slot during which the validator performs this role is any slot at which `get_shard_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. From 7fc5238b8f0a552251739366d696560a7215095b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 17 Jan 2019 11:38:56 -0600 Subject: [PATCH 007/136] update deposit root section to utilize eth1data --- specs/validator/0_beacon-chain-validator.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 40cb0eadd..687c126c7 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -26,7 +26,7 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Parent root](#parent-root) - [State root](#state-root) - [Randao reveal](#randao-reveal) - - [Deposit root](#deposit-root) + - [Eth1 Data](#eth1-data) - [Signature](#signature) - [Block body](#block-body) - [Proposer slashings](#proposer-slashings) @@ -171,13 +171,21 @@ _Note_: To calculate `state_root`, the validator should first run the state tran Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's current `randao_commitment` where `n = validator.randao_layers + 1`. `block.randao_reveal` should satisfy `repeat_hash(block.randao_reveal, validator.randao_layers + 1) == validator.randao_commitment`. -##### Deposit root +##### Eth1 Data -`block.deposit_root` is a mechanism used by block proposers to vote on a recent deposit root found in the Ethereum 1.0 PoW deposit contract. When consensus is formed `state.latest_deposit_root` is updated, and validator deposits up to this root can be processed. +`block.eth1_data` is a mechanism used by block proposers vote on a recent Ethereum 1.0 block hash and an associated deposit root found in the Ethereum 1.0 deposit contract. When consensus is formed, `state.latest_eth1_data` is updated, and validator deposits up to this root can be processed. -* Let `D` be the set of `DepositRootVote` objects `obj` in `state.deposit_root_votes` where `obj.deposit_root` is the deposit root of an eth1.0 block that is (i) part of the canonical chain, (ii) >= 1000 blocks behind the head, and (iii) newer than `state.latest_deposit_root`. -* If `D` is empty, set `block.deposit root` to the deposit root of the 1000th ancestor of the head of the canonical eth1.0 chain. -* If `D` is nonempty, set `block.deposit_root = best_vote.deposit_root` where `best_vote` is the member of `D` that has the highest `vote_count`, breaking ties by favoring newer deposit roots. +* Let `D` be the set of `Eth1DataVote` objects `vote` in `state.eth1_data_votes` where: + * `vote.eth1_data.block_hash` is the hash of an eth1.0 block that is (i) part of the canonical chain, (ii) >= 1000 blocks behind the head, and (iii) newer than `state.latest_eth1_data.block_data`. + * `vote.eth1_data.deposit_root` is the deposit root of the eth1.0 deposit contract at the block defined by `vote.eth1_data.block_hash`. +* If `D` is empty: + * Let `block_hash` be the block hash of the 1000th ancestory of the head of the canonical eth1.0 chain.to the deposit root of the 1000th ancestor of the head of the canonical eth1.0 chain. + * Let `deposit_root` be the the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. +* If `D` is nonempty: + * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. + * Let `block_hash = best_vote.eth1_data.block_hash`. + * Let `deposit_root = best_vote.eth1_data.deposit_root`. +* Set `block.eth1_data = Eth1Data(deposit_root=deposit_root, block_hash=block_hash)`. ##### Signature From b8f48d20a53bee27d4b58c3511178dd5aa6185b3 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 17 Jan 2019 21:28:14 -0600 Subject: [PATCH 008/136] add follow distance constant and extra details around time to being added to the validator registry --- specs/validator/0_beacon-chain-validator.md | 27 ++++++++++++++++----- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 687c126c7..3af9cfebb 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -16,7 +16,8 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [BLS withdrawal key](#bls-withdrawal-key) - [RANDAO commitment](#randao-commitment) - [Custody commitment](#custody-commitment) - - [Deposit](#deposit) + - [Submit deposit](#submit-deposit) + - [Process deposit](#process-deposit) - [Validator index](#validator-index) - [Activation](#activation) - [Beacon chain responsibilities](#beacon-chain-responsibilities) @@ -65,6 +66,14 @@ A validator is an entity that participates in the consensus of the Ethereum 2.0 All terminology, constants, functions, and protocol mechanics defined in the [Phase 0 -- The Beacon Chain](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md) doc are requisite for this document and used throughout. Please see the Phase 0 doc before continuing and use as a reference throughout. +## Constants + +### Misc + +| Name | Value | Unit | Duration | +| - | - | :-: | +| `ETH1_FOLLOW_DISTANCE` | `2**10` (= 1,024) | blocks | ~4 hours | + ## Becoming a validator ### Initialization @@ -109,7 +118,7 @@ See above note on hash-onion caching strategies in [RANDAO commitment](#randao-c _Note_: although this commitment is being committed to and stored in phase 0, it will not be used until phase 1. -### Deposit +### Submit deposit In phase 0, all incoming validator deposits originate from the Ethereum 1.0 PoW chain. Deposits are made to the [deposit contract](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#ethereum-10-deposit-contract) located at `DEPOSIT_CONTRACT_ADDRESS`. @@ -124,13 +133,17 @@ To submit a deposit: _Note_: Multiple deposits can be made by the same validator (same initialization params). A singular `Validator` will be added to `state.validator_registry` with each deposit amount being added to the validator's balance. A validator can only be activated when total deposits meet or exceed +### Process deposit + +Deposits cannot be processed into the beacon chain until the eth1.0 block in which they were deposited or any of its ancestors is added to the beacon chain `state.eth1_data`. This takes _a minimum_ of `ETH1_FOLLOW_DISTANCE` eth1.0 blocks (~4 hours) plus `ETH1_DATA_VOTING_PERIOD` slots (~1.7 hours). Once the necessary eth1.0 data is added, the deposit will normally be added to a beacon chain block and processed into the `state.validator_registry` within an epoch or two. The validator is then in a queue to be activated. + ### Validator index -Once a validator has been added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. +Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. ### Activation -A validator is activated some amount of slots after being added to the registry. There is a maximum validator churn per finalized epoch so the delay until activation is variable depending upon finality, total active validator balance, and the number of validators in the queue to be activated. +In normal operation, the validator is quickly activated at which point the validator is added to the shuffling and begins validation after an additional `ENTRY_EXIT_DELAY` slots. The function [`is_active_validator`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_active_validator) can be used to check if a validator is active at a given slot. Usage is as follows: @@ -141,6 +154,8 @@ is_active_validator(validator, slot) Once a validator is active, the validator is assigned [responsibilities](#beacon-chain-responsibilities) until exited. +_Note_: There is a maximum validator churn per finalized epoch so the delay until activation is variable depending upon finality, total active validator balance, and the number of validators in the queue to be activated. + ## Beacon chain responsibilities A validator has two primary responsibilities to the beacon chain -- [proposing blocks](block-proposal) and [creating attestations](attestations-1). Proposals happen infrequently, whereas attestations should be created once per epoch. @@ -176,10 +191,10 @@ Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's cu `block.eth1_data` is a mechanism used by block proposers vote on a recent Ethereum 1.0 block hash and an associated deposit root found in the Ethereum 1.0 deposit contract. When consensus is formed, `state.latest_eth1_data` is updated, and validator deposits up to this root can be processed. * Let `D` be the set of `Eth1DataVote` objects `vote` in `state.eth1_data_votes` where: - * `vote.eth1_data.block_hash` is the hash of an eth1.0 block that is (i) part of the canonical chain, (ii) >= 1000 blocks behind the head, and (iii) newer than `state.latest_eth1_data.block_data`. + * `vote.eth1_data.block_hash` is the hash of an eth1.0 block that is (i) part of the canonical chain, (ii) >= `ETH1_FOLLOW_DISTANCE` blocks behind the head, and (iii) newer than `state.latest_eth1_data.block_data`. * `vote.eth1_data.deposit_root` is the deposit root of the eth1.0 deposit contract at the block defined by `vote.eth1_data.block_hash`. * If `D` is empty: - * Let `block_hash` be the block hash of the 1000th ancestory of the head of the canonical eth1.0 chain.to the deposit root of the 1000th ancestor of the head of the canonical eth1.0 chain. + * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`th ancestory of the head of the canonical eth1.0 chain. * Let `deposit_root` be the the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. * If `D` is nonempty: * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. From 421ef9e08d67af14766f32ea5e0cda9d9cc3d5e1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 17 Jan 2019 21:29:38 -0600 Subject: [PATCH 009/136] fix bad link in v guide --- specs/validator/0_beacon-chain-validator.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 3af9cfebb..4b8cd0927 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -349,7 +349,7 @@ _Note_: Signed data must be within the same `Fork` context to conflict. Messages ### Proposal slashing -To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#proposalsigneddata) (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. +To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposalsigneddata) (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. The following helper can be run to check if two proposal messages conflict: @@ -372,7 +372,7 @@ If the software crashes at some point within this routine, then when the validat ### Casper slashing -To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_surround_vote). +To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_surround_vote). Specifically, when signing an `Attestation`, a validator should perform the following steps in the following order: 1. Save a record to hard disk that an attestation has been signed for source -- `attestation_data.justified_slot // EPOCH_LENGTH` -- and target -- `attestation_data.slot // EPOCH_LENGTH`. From 9e75a76fc1f1bbc820269e61441e83ccda471423 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Fri, 18 Jan 2019 21:06:21 -0600 Subject: [PATCH 010/136] Implement #459 Contents: * Peg entries and exits to epoch boundaries * Add a store of historical active index roots * Mix it into the randomness * Remove the delta hash chain Note that the actual light client implementation is beyond the scope of the spec. [Note to reviewers: verify that the invariant added in the PR is correct] Question: * Do we want to also only store epoch-boundary randao values? I don't think we use the epoch-intermediate ones anywhere..... --- specs/core/0_beacon-chain.md | 91 ++++++++++++++---------------------- 1 file changed, 35 insertions(+), 56 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 5640340b6..05d138649 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -18,7 +18,6 @@ - [Reward and penalty quotients](#reward-and-penalty-quotients) - [Status flags](#status-flags) - [Max operations per block](#max-operations-per-block) - - [Validator registry delta flags](#validator-registry-delta-flags) - [Signature domains](#signature-domains) - [Data structures](#data-structures) - [Beacon chain operations](#beacon-chain-operations) @@ -47,7 +46,6 @@ - [`Crosslink`](#crosslink) - [`PendingAttestation`](#pendingattestation) - [`Fork`](#fork) - - [`ValidatorRegistryDeltaBlock`](#validatorregistrydeltablock) - [`Eth1Data`](#eth1data) - [`Eth1DataVote`](#eth1datavote) - [Ethereum 1.0 deposit contract](#ethereum-10-deposit-contract) @@ -73,6 +71,7 @@ - [`get_crosslink_committees_at_slot`](#get_crosslink_committees_at_slot) - [`get_block_root`](#get_block_root) - [`get_randao_mix`](#get_randao_mix) + - [`get_active_index_root`](#get_active_index_root) - [`get_beacon_proposer_index`](#get_beacon_proposer_index) - [`merkle_root`](#merkle_root) - [`get_attestation_participants`](#get_attestation_participants) @@ -168,6 +167,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted | `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes | | `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots | | `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | randao mixes | +| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | index roots | | `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | | `MAX_WITHDRAWALS_PER_EPOCH` | `2**2` (= 4) | withdrawals | @@ -235,13 +235,6 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted | `MAX_DEPOSITS` | `2**4` (= 16) | | `MAX_EXITS` | `2**4` (= 16) | -### Validator registry delta flags - -| Name | Value | -| - | - | -| `ACTIVATION` | `0` | -| `EXIT` | `1` | - ### Signature domains | Name | Value | @@ -478,7 +471,6 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted 'validator_balances': ['uint64'], 'validator_registry_update_slot': 'uint64', 'validator_registry_exit_count': 'uint64', - 'validator_registry_delta_chain_tip': 'hash32', # For light clients to track deltas # Randomness and committees 'latest_randao_mixes': ['hash32'], @@ -487,8 +479,8 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted 'current_epoch_start_shard': 'uint64', 'previous_epoch_calculation_slot': 'uint64', 'current_epoch_calculation_slot': 'uint64', - 'previous_epoch_randao_mix': 'hash32', - 'current_epoch_randao_mix': 'hash32', + 'previous_epoch_seed': 'hash32', + 'current_epoch_seed': 'hash32', # Custody challenges 'custody_challenges': [CustodyChallenge], @@ -502,6 +494,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted # Recent state 'latest_crosslinks': [Crosslink], 'latest_block_roots': ['hash32'], # Needed to process attestations, older to newer + 'latest_index_roots': ['hash32'], 'latest_penalized_balances': ['uint64'], # Balances penalized at every withdrawal period 'latest_attestations': [PendingAttestation], 'batched_block_roots': ['hash32'], @@ -584,18 +577,6 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted } ``` -#### `ValidatorRegistryDeltaBlock` - -```python -{ - 'latest_registry_delta_root': 'hash32', - 'validator_index': 'uint24', - 'pubkey': 'uint384', - 'slot': 'uint64', - 'flag': 'uint64', -} -``` - #### `Eth1Data` ```python @@ -954,7 +935,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, if slot < state_epoch_slot: committees_per_slot = get_previous_epoch_committee_count_per_slot(state) shuffling = get_shuffling( - state.previous_epoch_randao_mix, + state.previous_epoch_seed, state.validator_registry, state.previous_epoch_calculation_slot, ) @@ -962,7 +943,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, else: committees_per_slot = get_current_epoch_committee_count_per_slot(state) shuffling = get_shuffling( - state.current_epoch_randao_mix, + state.current_epoch_seed, state.validator_registry, state.current_epoch_calculation_slot, ) @@ -1007,6 +988,19 @@ def get_randao_mix(state: BeaconState, return state.latest_randao_mixes[slot % LATEST_RANDAO_MIXES_LENGTH] ``` +#### `get_active_index_root` + +```python +def get_active_index_root(state: BeaconState, + slot: int) -> Hash32: + """ + Returns the randao mix at a recent ``slot``. + """ + assert state.slot // EPOCH_LENGTH < slot // EPOCH_LENGTH + LATEST_INDEX_ROOTS_LENGTH + assert slot // EPOCH_LENGTH <= state.slot // EPOCH_LENGTH + return state.latest_index_roots[(slot // EPOCH_LENGTH) % LATEST_INDEX_ROOTS_LENGTH] +``` + #### `get_beacon_proposer_index` ```python @@ -1235,7 +1229,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], validator_balances=[], validator_registry_update_slot=GENESIS_SLOT, validator_registry_exit_count=0, - validator_registry_delta_chain_tip=ZERO_HASH, # Randomness and committees latest_randao_mixes=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH)], @@ -1244,8 +1237,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], current_epoch_start_shard=GENESIS_START_SHARD, previous_epoch_calculation_slot=GENESIS_SLOT, current_epoch_calculation_slot=GENESIS_SLOT, - previous_epoch_randao_mix=ZERO_HASH, - current_epoch_randao_mix=ZERO_HASH, + previous_epoch_seed=ZERO_HASH, + current_epoch_seed=ZERO_HASH, # Custody challenges custody_challenges=[], @@ -1259,6 +1252,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Recent state latest_crosslinks=[Crosslink(slot=GENESIS_SLOT, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)], + latest_index_roots=[ZERO_HASH for _ in range(LATEST_INDEX_ROOTS_LENGTH)], latest_penalized_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)], latest_attestations=[], batched_block_roots=[], @@ -1383,16 +1377,7 @@ Note: All functions in this section mutate `state`. def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_slot = GENESIS_SLOT if genesis else (state.slot + ENTRY_EXIT_DELAY) - state.validator_registry_delta_chain_tip = hash_tree_root( - ValidatorRegistryDeltaBlock( - latest_registry_delta_root=state.validator_registry_delta_chain_tip, - validator_index=index, - pubkey=validator.pubkey, - slot=validator.activation_slot, - flag=ACTIVATION, - ) - ) + validator.activation_slot = GENESIS_SLOT if genesis else (state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY) ``` ```python @@ -1406,22 +1391,13 @@ def exit_validator(state: BeaconState, index: int) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited - if validator.exit_slot <= state.slot + ENTRY_EXIT_DELAY: + if validator.exit_slot <= state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY: return - validator.exit_slot = state.slot + ENTRY_EXIT_DELAY + validator.exit_slot = state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY state.validator_registry_exit_count += 1 validator.exit_count = state.validator_registry_exit_count - state.validator_registry_delta_chain_tip = hash_tree_root( - ValidatorRegistryDeltaBlock( - latest_registry_delta_root=state.validator_registry_delta_chain_tip, - validator_index=index, - pubkey=validator.pubkey, - slot=validator.exit_slot, - flag=EXIT, - ) - ) ``` ```python @@ -1583,7 +1559,7 @@ Verify that `len(block.body.exits) <= MAX_EXITS`. For each `exit` in `block.body.exits`: * Let `validator = state.validator_registry[exit.validator_index]`. -* Verify that `validator.exit_slot > state.slot + ENTRY_EXIT_DELAY`. +* Verify that `validator.exit_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY`. * Verify that `state.slot >= exit.slot`. * Let `exit_message = hash_tree_root(Exit(slot=exit.slot, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. @@ -1763,7 +1739,7 @@ def update_validator_registry(state: BeaconState) -> None: # Activate validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.activation_slot > state.slot + ENTRY_EXIT_DELAY and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: + if validator.activation_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1775,7 +1751,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.exit_slot > state.slot + ENTRY_EXIT_DELAY and validator.status_flags & INITIATED_EXIT: + if validator.exit_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY and validator.status_flags & INITIATED_EXIT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1791,17 +1767,19 @@ and perform the following updates: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` -* Set `state.previous_epoch_randao_mix = state.current_epoch_randao_mix` +* Set `state.previous_epoch_seed = state.current_epoch_seed` * Set `state.current_epoch_calculation_slot = state.slot` * Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH) % SHARD_COUNT` -* Set `state.current_epoch_randao_mix = get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD)` +* Set `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))` If a validator registry update does _not_ happen do the following: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` * Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. -* If `epochs_since_last_registry_change` is an exact power of 2, set `state.current_epoch_calculation_slot = state.slot` and `state.current_epoch_randao_mix = state.latest_randao_mixes[(state.current_epoch_calculation_slot - SEED_LOOKAHEAD) % LATEST_RANDAO_MIXES_LENGTH]`. Note that `state.current_epoch_start_shard` is left unchanged. +* If `epochs_since_last_registry_change` is an exact power of 2, set `state.current_epoch_calculation_slot = state.slot` and `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))`. Note that `state.current_epoch_start_shard` is left unchanged. + +**Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. Regardless of whether or not a validator set change happens, run the following: @@ -1844,6 +1822,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: * Let `e = state.slot // EPOCH_LENGTH`. Set `state.latest_penalized_balances[(e+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[e % LATEST_PENALIZED_EXIT_LENGTH]` * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`. +* Let `epoch = state.slot // EPOCH_LENGTH`. Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` ## State root processing From ba8c44dd9abfe10f671fb556056e76e88a5946fb Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 19 Jan 2019 15:46:09 +0800 Subject: [PATCH 011/136] Fix the new `Eth1Data` fields to `bytes32` --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 0394761ad..3ef3c94e2 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -601,9 +601,9 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted ```python { # Root of the deposit tree - 'deposit_root': 'hash32', + 'deposit_root': 'bytes32', # Block hash - 'block_hash': 'hash32', + 'block_hash': 'bytes32', } ``` From 41813462c3e3b6af6239fea622cdebccd3761c5c Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 19 Jan 2019 15:58:24 +0800 Subject: [PATCH 012/136] Add custom types --- specs/core/0_beacon-chain.md | 59 +++++++++++++++++++++++------------- 1 file changed, 38 insertions(+), 21 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 3ef3c94e2..569296b5f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -50,6 +50,7 @@ - [`ValidatorRegistryDeltaBlock`](#validatorregistrydeltablock) - [`Eth1Data`](#eth1data) - [`Eth1DataVote`](#eth1datavote) + - [Custom Types](#custom-types) - [Ethereum 1.0 deposit contract](#ethereum-10-deposit-contract) - [Deposit arguments](#deposit-arguments) - [Withdrawal credentials](#withdrawal-credentials) @@ -253,6 +254,8 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted ## Data structures +The following data structures are [SimpleSerialize (SSZ)](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md) objects. + ### Beacon chain operations #### Proposer slashings @@ -618,6 +621,20 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted } ``` +## Custom Types + +We define the following Python custom types for type hinting and readability: + +| Name | Type | Description | +| - | - | - | +| `SlotNumber` | unsigned 64-bit integer | the number of a slot | +| `ShardNumber` | unsigned 64-bit integer | the number of a shard | +| `ValidatorIndex` | unsigned 24-bit integer | the index number of validator in the registry | +| `Gwei` | unsigned 64-bit integer | the amount in Gwei | +| `Bytes32` | 32-byte data | the binary data in 32-byte length | +| `BLSPubkey` | 48-byte data | the public key in BLS signature scheme | +| `BLSSignature` | 96-byte data | the signature in BLS signature scheme | + ## Ethereum 1.0 deposit contract The initial deployment phases of Ethereum 2.0 are implemented without consensus changes to Ethereum 1.0. A deposit contract at address `DEPOSIT_CONTRACT_ADDRESS` is added to Ethereum 1.0 for deposits of ETH to the beacon chain. Validator balances will be withdrawable to the shards in phase 2, i.e. when the EVM2.0 is deployed and the shards have state. @@ -788,7 +805,7 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere #### `is_active_validator` ```python -def is_active_validator(validator: Validator, slot: int) -> bool: +def is_active_validator(validator: Validator, slot: SlotNumber) -> bool: """ Checks if ``validator`` is active. """ @@ -798,7 +815,7 @@ def is_active_validator(validator: Validator, slot: int) -> bool: #### `get_active_validator_indices` ```python -def get_active_validator_indices(validators: [Validator], slot: int) -> List[int]: +def get_active_validator_indices(validators: [Validator], slot: SlotNumber) -> List[ValidatorIndex]: """ Gets indices of active validators from ``validators``. """ @@ -890,7 +907,7 @@ def get_committee_count_per_slot(active_validator_count: int) -> int: ```python def get_shuffling(seed: Bytes32, validators: List[Validator], - slot: int) -> List[List[int]] + slot: SlotNumber) -> List[List[ValidatorIndex]] """ Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``slot``. Returns a list of ``EPOCH_LENGTH * committees_per_slot`` committees where each @@ -942,7 +959,7 @@ def get_current_epoch_committee_count_per_slot(state: BeaconState) -> int: ```python def get_crosslink_committees_at_slot(state: BeaconState, - slot: int) -> List[Tuple[List[int], int]]: + slot: SlotNumber) -> List[Tuple[List[ValidatorIndex], ShardNumber]]: """ Returns the list of ``(committee, shard)`` tuples for the ``slot``. """ @@ -983,7 +1000,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, ```python def get_block_root(state: BeaconState, - slot: int) -> Bytes32: + slot: SlotNumber) -> Bytes32: """ Returns the block root at a recent ``slot``. """ @@ -998,7 +1015,7 @@ def get_block_root(state: BeaconState, ```python def get_randao_mix(state: BeaconState, - slot: int) -> Bytes32: + slot: SlotNumber) -> Bytes32: """ Returns the randao mix at a recent ``slot``. """ @@ -1011,7 +1028,7 @@ def get_randao_mix(state: BeaconState, ```python def get_beacon_proposer_index(state: BeaconState, - slot: int) -> int: + slot: SlotNumber) -> ValidatorIndex: """ Returns the beacon proposer index for the ``slot``. """ @@ -1037,7 +1054,7 @@ def merkle_root(values: List[Bytes32]) -> Bytes32: ```python def get_attestation_participants(state: BeaconState, attestation_data: AttestationData, - aggregation_bitfield: bytes) -> List[int]: + aggregation_bitfield: bytes) -> List[ValidatorIndex]: """ Returns the participant indices at for the ``attestation_data`` and ``aggregation_bitfield``. """ @@ -1065,7 +1082,7 @@ def get_attestation_participants(state: BeaconState, #### `get_effective_balance` ```python -def get_effective_balance(state: State, index: int) -> int: +def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: """ Returns the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``. """ @@ -1076,7 +1093,7 @@ def get_effective_balance(state: State, index: int) -> int: ```python def get_fork_version(fork: Fork, - slot: int) -> int: + slot: SlotNumber) -> int: if slot < fork.slot: return fork.previous_version else: @@ -1087,7 +1104,7 @@ def get_fork_version(fork: Fork, ```python def get_domain(fork: Fork, - slot: int, + slot: SlotNumber, domain_type: int) -> int: return get_fork_version( fork, @@ -1294,8 +1311,8 @@ First, a helper function: ```python def validate_proof_of_possession(state: BeaconState, - pubkey: Bytes48, - proof_of_possession: Bytes96, + pubkey: BLSPubkey, + proof_of_possession: BLSSignature, withdrawal_credentials: Bytes32, randao_commitment: Bytes32, custody_commitment: Bytes32) -> bool: @@ -1323,9 +1340,9 @@ Now, to add a [validator](#dfn-validator) or top up an existing [validator](#dfn ```python def process_deposit(state: BeaconState, - pubkey: Bytes48, - amount: int, - proof_of_possession: Bytes96, + pubkey: BLSPubkey, + amount: Gwei, + proof_of_possession: BLSSignature, withdrawal_credentials: Bytes32, randao_commitment: Bytes32, custody_commitment: Bytes32) -> None: @@ -1379,7 +1396,7 @@ def process_deposit(state: BeaconState, Note: All functions in this section mutate `state`. ```python -def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: +def activate_validator(state: BeaconState, index: ValidatorIndex, genesis: bool) -> None: validator = state.validator_registry[index] validator.activation_slot = GENESIS_SLOT if genesis else (state.slot + ENTRY_EXIT_DELAY) @@ -1395,13 +1412,13 @@ def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: ``` ```python -def initiate_validator_exit(state: BeaconState, index: int) -> None: +def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: validator = state.validator_registry[index] validator.status_flags |= INITIATED_EXIT ``` ```python -def exit_validator(state: BeaconState, index: int) -> None: +def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited @@ -1424,7 +1441,7 @@ def exit_validator(state: BeaconState, index: int) -> None: ``` ```python -def penalize_validator(state: BeaconState, index: int) -> None: +def penalize_validator(state: BeaconState, index: ValidatorIndex) -> None: exit_validator(state, index) validator = state.validator_registry[index] state.latest_penalized_balances[(state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) @@ -1437,7 +1454,7 @@ def penalize_validator(state: BeaconState, index: int) -> None: ``` ```python -def prepare_validator_for_withdrawal(state: BeaconState, index: int) -> None: +def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) -> None: validator = state.validator_registry[index] validator.status_flags |= WITHDRAWABLE ``` From 1ae3768ac868b23449ac79bd57edec9bad1f804c Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 19 Jan 2019 16:06:00 +0800 Subject: [PATCH 013/136] Minor fixes --- specs/core/0_beacon-chain.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 569296b5f..60b39c70b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -254,7 +254,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted ## Data structures -The following data structures are [SimpleSerialize (SSZ)](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md) objects. +The following data structures are defined as [SimpleSerialize (SSZ)](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md) objects. ### Beacon chain operations @@ -801,7 +801,7 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere #### `hash_tree_root` -`hash_tree_root` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). +`def hash_tree_root(object: SSZSerializable) -> Bytes32` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). #### `is_active_validator` ```python @@ -815,7 +815,7 @@ def is_active_validator(validator: Validator, slot: SlotNumber) -> bool: #### `get_active_validator_indices` ```python -def get_active_validator_indices(validators: [Validator], slot: SlotNumber) -> List[ValidatorIndex]: +def get_active_validator_indices(validators: List[Validator], slot: SlotNumber) -> List[ValidatorIndex]: """ Gets indices of active validators from ``validators``. """ @@ -1568,7 +1568,7 @@ For each `deposit` in `block.body.deposits`: * Verify that `verify_merkle_branch(hash(serialized_deposit_data), deposit.branch, DEPOSIT_CONTRACT_TREE_DEPTH, deposit.index, state.latest_eth1_data.deposit_root)` is `True`. ```python -def verify_merkle_branch(leaf: Bytes32, branch: [Bytes32], depth: int, index: int, root: Bytes32) -> bool: +def verify_merkle_branch(leaf: Bytes32, branch: List[Bytes32], depth: int, index: int, root: Bytes32) -> bool: value = leaf for i in range(depth): if index // (2**i) % 2: From 958c338c8fe1e9ea8e7494add7d33eecdced5513 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 19 Jan 2019 18:11:07 -0600 Subject: [PATCH 014/136] Update specs/core/0_beacon-chain.md Co-Authored-By: vbuterin --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 3f687618a..afa87a140 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -994,7 +994,7 @@ def get_randao_mix(state: BeaconState, def get_active_index_root(state: BeaconState, slot: int) -> Hash32: """ - Returns the randao mix at a recent ``slot``. + Returns the index root at a recent ``slot``. """ assert state.slot // EPOCH_LENGTH < slot // EPOCH_LENGTH + LATEST_INDEX_ROOTS_LENGTH assert slot // EPOCH_LENGTH <= state.slot // EPOCH_LENGTH From 02725b870e0cc8b6c01427906cafe4a36461a528 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 19 Jan 2019 18:11:14 -0600 Subject: [PATCH 015/136] Update specs/core/0_beacon-chain.md Co-Authored-By: vbuterin --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index afa87a140..25eb7bbe6 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -992,7 +992,7 @@ def get_randao_mix(state: BeaconState, ```python def get_active_index_root(state: BeaconState, - slot: int) -> Hash32: + slot: int) -> Bytes32: """ Returns the index root at a recent ``slot``. """ From 12b217df70ea36fa7322efd00fdd8491cba5ed7f Mon Sep 17 00:00:00 2001 From: vbuterin Date: Sat, 19 Jan 2019 18:13:17 -0600 Subject: [PATCH 016/136] Updated as per hww's suggestions --- specs/core/0_beacon-chain.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 25eb7bbe6..88b3c0384 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -996,9 +996,11 @@ def get_active_index_root(state: BeaconState, """ Returns the index root at a recent ``slot``. """ - assert state.slot // EPOCH_LENGTH < slot // EPOCH_LENGTH + LATEST_INDEX_ROOTS_LENGTH - assert slot // EPOCH_LENGTH <= state.slot // EPOCH_LENGTH - return state.latest_index_roots[(slot // EPOCH_LENGTH) % LATEST_INDEX_ROOTS_LENGTH] + state_epoch = state.slot // EPOCH_LENGTH + given_epoch = slot // EPOCH_LENGTH + assert state_epoch < given_epoch + LATEST_INDEX_ROOTS_LENGTH + assert given_epoch <= state_epoch + return state.latest_index_roots[given_epoch % LATEST_INDEX_ROOTS_LENGTH] ``` #### `get_beacon_proposer_index` @@ -1819,9 +1821,10 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates -* Let `e = state.slot // EPOCH_LENGTH`. Set `state.latest_penalized_balances[(e+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[e % LATEST_PENALIZED_EXIT_LENGTH]` +* Let `epoch = state.slot // EPOCH_LENGTH`. +* Set `state.latest_penalized_balances[(epoch+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[epoch % LATEST_PENALIZED_EXIT_LENGTH]` * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`. -* Let `epoch = state.slot // EPOCH_LENGTH`. Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` +* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` ## State root processing From 0efeed9a56c4544c522afc946c96502a0c3acab2 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 21 Jan 2019 08:39:24 -0600 Subject: [PATCH 017/136] fix constants in validator section --- specs/validator/0_beacon-chain-validator.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 4b8cd0927..1aaeb86ab 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -10,6 +10,8 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Table of Contents](#table-of-contents) - [Introduction](#introduction) - [Prerequisites](#prerequisites) + - [Constants](#constants) + - [Misc](#misc) - [Becoming a validator](#becoming-a-validator) - [Initialization](#initialization) - [BLS public key](#bls-public-key) @@ -71,7 +73,7 @@ All terminology, constants, functions, and protocol mechanics defined in the [Ph ### Misc | Name | Value | Unit | Duration | -| - | - | :-: | +| - | - | :-: | :-: | | `ETH1_FOLLOW_DISTANCE` | `2**10` (= 1,024) | blocks | ~4 hours | ## Becoming a validator From 52696f88067385a2e582b144fdccaecfbdba8321 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 21 Jan 2019 11:07:56 -0600 Subject: [PATCH 018/136] ensure validator links to master --- specs/validator/0_beacon-chain-validator.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 1aaeb86ab..8296ee97c 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -141,7 +141,7 @@ Deposits cannot be processed into the beacon chain until the eth1.0 block in whi ### Validator index -Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. +Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. ### Activation @@ -374,7 +374,7 @@ If the software crashes at some point within this routine, then when the validat ### Casper slashing -To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/bd506e12227850888df167926b52f8fd2db31915/specs/core/0_beacon-chain.md#is_surround_vote). +To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_surround_vote). Specifically, when signing an `Attestation`, a validator should perform the following steps in the following order: 1. Save a record to hard disk that an attestation has been signed for source -- `attestation_data.justified_slot // EPOCH_LENGTH` -- and target -- `attestation_data.slot // EPOCH_LENGTH`. From 460188f9f55ed084f3008c5d1e878d708394a443 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 21 Jan 2019 11:47:23 -0600 Subject: [PATCH 019/136] clarify get_shuffling invariant --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 49db5fdb0..daf0cae7e 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -912,7 +912,7 @@ def get_shuffling(seed: Bytes32, return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) ``` -**Invariant**: if `get_shuffling(seed, validators, slot)` returns some value `x`, it should return the same value `x` for the same `seed` and `slot` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. +**Invariant**: if `get_shuffling(seed, validators, slot)` returns some value `x` and `slot <= state.slot + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `slot` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. From 634740a2f26517973d1948b53f72137b54c49e7f Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 07:56:44 -0600 Subject: [PATCH 020/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 8296ee97c..3f8e8d29d 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -130,7 +130,7 @@ To submit a deposit: * Set `deposit_input.proof_of_possession = EMPTY_SIGNATURE`. * Let `proof_of_possession` be the result of `bls_sign` of the `hash_tree_root(deposit_input)` with `domain=DOMAIN_DEPOSIT`. * Set `deposit_input.proof_of_possession = proof_of_possession`. -* Let `amount` be the amount of eth to be deposited by the validator where `MIN_DEPOSIT <= amount <= MAX_DEPOSIT`. +* Let `amount` be the amount in Gwei to be deposited by the validator where `MIN_DEPOSIT_AMOUNT <= amount <= MAX_DEPOSIT_AMOUNT`. * Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in ETH. _Note_: Multiple deposits can be made by the same validator (same initialization params). A singular `Validator` will be added to `state.validator_registry` with each deposit amount being added to the validator's balance. A validator can only be activated when total deposits meet or exceed From b7de018f4d7121e6f9d764b6db9f5395905d0959 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 07:57:51 -0600 Subject: [PATCH 021/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 3f8e8d29d..b55a25d0a 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -196,7 +196,7 @@ Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's cu * `vote.eth1_data.block_hash` is the hash of an eth1.0 block that is (i) part of the canonical chain, (ii) >= `ETH1_FOLLOW_DISTANCE` blocks behind the head, and (iii) newer than `state.latest_eth1_data.block_data`. * `vote.eth1_data.deposit_root` is the deposit root of the eth1.0 deposit contract at the block defined by `vote.eth1_data.block_hash`. * If `D` is empty: - * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`th ancestory of the head of the canonical eth1.0 chain. + * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`th ancestor of the head of the canonical eth1.0 chain. * Let `deposit_root` be the the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. * If `D` is nonempty: * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. From a934138d8bf52482481977656b380523de3bd2d9 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 07:58:10 -0600 Subject: [PATCH 022/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index b55a25d0a..0b10aeb9f 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -131,7 +131,7 @@ To submit a deposit: * Let `proof_of_possession` be the result of `bls_sign` of the `hash_tree_root(deposit_input)` with `domain=DOMAIN_DEPOSIT`. * Set `deposit_input.proof_of_possession = proof_of_possession`. * Let `amount` be the amount in Gwei to be deposited by the validator where `MIN_DEPOSIT_AMOUNT <= amount <= MAX_DEPOSIT_AMOUNT`. -* Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in ETH. +* Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in Gwei. _Note_: Multiple deposits can be made by the same validator (same initialization params). A singular `Validator` will be added to `state.validator_registry` with each deposit amount being added to the validator's balance. A validator can only be activated when total deposits meet or exceed From daa1b6ebf1ef32876ef94c8591b65f9da1069e9f Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 07:58:29 -0600 Subject: [PATCH 023/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 0b10aeb9f..c79d22347 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -307,7 +307,7 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes ##### Participation bitfield -* Let `participation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. +* Let `aggregation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. * Let `index_into_committee` be the index into the validator's `committee` at which `validator_index` is located. * Set `participation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. * Set `attestation.participation_bitfield = participation_bitfield`. From 05e8d25a80b8668b00b0d64e8da988a07e32f091 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 07:58:55 -0600 Subject: [PATCH 024/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index c79d22347..2521e1cb3 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -197,7 +197,7 @@ Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's cu * `vote.eth1_data.deposit_root` is the deposit root of the eth1.0 deposit contract at the block defined by `vote.eth1_data.block_hash`. * If `D` is empty: * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`th ancestor of the head of the canonical eth1.0 chain. - * Let `deposit_root` be the the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. + * Let `deposit_root` be the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. * If `D` is nonempty: * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. * Let `block_hash = best_vote.eth1_data.block_hash`. From c32a79f940edd06d45626b4b5987592ed7f18059 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 22 Jan 2019 08:03:00 -0600 Subject: [PATCH 025/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 2521e1cb3..dde1610ba 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -235,7 +235,7 @@ Up to `MAX_PROPOSER_SLASHINGS` [`ProposerSlashing`](https://github.com/ethereum/ ##### Casper slashings -Up to `MAX_CASPER_SLASHINGS` [`CasperSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casperslashing) objects can be included in the `block`. The casper slashings must satisfy the verification conditions found in [casper slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casper-slashings-1). The validator receives a small "whistleblower" reward for each casper slashing found and included. +Up to `MAX_CASPER_SLASHINGS` [`CasperSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casperslashing) objects can be included in the `block`. The Casper slashings must satisfy the verification conditions found in [Casper slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casper-slashings-1). The validator receives a small "whistleblower" reward for each Casper slashing found and included. ##### Attestations From a5519a7d3b5c0b675026a2daef9477f06ae795a6 Mon Sep 17 00:00:00 2001 From: Jannik Luhn Date: Tue, 22 Jan 2019 17:14:25 +0100 Subject: [PATCH 026/136] SSZ: List serialization with variable element size Fix #232 --- specs/simple-serialize.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 8630c47c6..cca2969e6 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -143,9 +143,9 @@ Lists are a collection of elements of the same homogeneous type. | Length of serialized list fits into 4 bytes | ``len(serialized) < 2**32`` | -1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``. - * Encode that as a `4-byte` **little endian** `uint32`. -2. Append the elements in a packed manner. +1. Serialize all list elements individually and concatenate them. + +2. Prefix the concatenation with its length encoded as a `4-byte` **little-endian** unsigned integer. * *Note on efficiency*: consider using a container that does not need to iterate over all elements to get its length. For example Python lists, C++ vectors or Rust Vec. From f8670e80692c02a1de4be41bdf9b9ca94948419f Mon Sep 17 00:00:00 2001 From: Jannik Luhn Date: Tue, 22 Jan 2019 17:16:37 +0100 Subject: [PATCH 027/136] SSZ: Remove note on efficiency --- specs/simple-serialize.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index cca2969e6..eff65f49e 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -147,8 +147,6 @@ Lists are a collection of elements of the same homogeneous type. 2. Prefix the concatenation with its length encoded as a `4-byte` **little-endian** unsigned integer. -* *Note on efficiency*: consider using a container that does not need to iterate over all elements to get its length. For example Python lists, C++ vectors or Rust Vec. - **Example in Python** ```python From 1bc6c19dca555271537ace0534521b0f62936e9e Mon Sep 17 00:00:00 2001 From: terence tsao Date: Tue, 22 Jan 2019 10:56:01 -0800 Subject: [PATCH 028/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 56430cd56..24486ddd3 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1715,6 +1715,7 @@ First, update `previous_epoch_calculation_slot` and `previous_epoch_start_shard` * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` +* Set `state.previous_epoch_randao_mix = state.current_epoch_randao_mix` If the following are satisfied: @@ -1769,7 +1770,6 @@ def update_validator_registry(state: BeaconState) -> None: and perform the following updates: -* Set `state.previous_epoch_randao_mix = state.current_epoch_randao_mix` * Set `state.current_epoch_calculation_slot = state.slot` * Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH) % SHARD_COUNT` * Set `state.current_epoch_randao_mix = get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD)` From 6c824ffee462686787b27cb7ad11420513eccb32 Mon Sep 17 00:00:00 2001 From: Justin Date: Tue, 22 Jan 2019 20:45:30 +0000 Subject: [PATCH 029/136] Remove placeholders MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove all custody and VDF placeholders (56 lines less!). A few notes on the placeholder fields and logic: * All placeholder fields were dummy fields that can easily be restored in the phase 1 hard fork. * One special case to the above is `custody_bit_0_validator_indices` in `SlashableVoteData` which was renamed to `validator_indices`. Renaming it back is *not* a spec change because SSZ field names are [no longer part of the spec](https://github.com/ethereum/eth2.0-specs/commit/a9328157a87451ee4f372df272ece158b386ec41#diff-8d8fe480a35579c7be2f976d9b321216). * The placeholder logic was written using generalised functions (e.g. `bls_verify_multiple` vs `bls_verify`, and `indices(slashable_vote_data)` vs `slashable_vote_data.validator_indices`). This generality was unnecessary because it was not triggered when all custody bits were 0. This means we can simplify the logic without being inconsistent with phase 1. Rationale: * Keep phase 0 (likely far harder to deliver than phase 1) as clean as possible * Focus on upgrade paths and incremental releases * Custody is still under research—keep the design space open --- specs/core/0_beacon-chain.md | 76 +++++------------------------------- 1 file changed, 10 insertions(+), 66 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 56430cd56..a47cad399 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -30,7 +30,6 @@ - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) - - [`AttestationDataAndCustodyBit`](#attestationdataandcustodybit) - [Deposits](#deposits) - [`Deposit`](#deposit) - [`DepositData`](#depositdata) @@ -104,7 +103,6 @@ - [Attestations](#attestations-1) - [Deposits](#deposits-1) - [Exits](#exits-1) - - [Custody](#custody) - [Per-epoch processing](#per-epoch-processing) - [Helpers](#helpers) - [Eth1 data](#eth1-data-1) @@ -292,10 +290,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { - # Validator indices with custody bit equal to 0 - 'custody_bit_0_indices': ['uint24'], - # Validator indices with custody bit equal to 1 - 'custody_bit_1_indices': ['uint24'], + # Validator indices + 'validator_indices': ['uint24'], # Attestation data 'data': AttestationData, # Aggregate signature @@ -313,8 +309,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', - # Custody bitfield - 'custody_bitfield': 'bytes', # BLS aggregate signature 'aggregate_signature': 'bytes96', } @@ -343,17 +337,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be } ``` -##### `AttestationDataAndCustodyBit` - -```python -{ - # Attestation data - data: AttestationData, - # Custody bit - custody_bit: bool, -} -``` - #### Deposits ##### `Deposit` @@ -436,16 +419,11 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'proposer_slashings': [ProposerSlashing], 'casper_slashings': [CasperSlashing], 'attestations': [Attestation], - 'custody_reseeds': [CustodyReseed], - 'custody_challenges': [CustodyChallenge], - 'custody_responses': [CustodyResponse], 'deposits': [Deposit], 'exits': [Exit], } ``` -`CustodyReseed`, `CustodyChallenge`, and `CustodyResponse` will be defined in phase 1; for now, put dummy classes as these lists will remain empty throughout phase 0. - #### `ProposalSignedData` ```python @@ -479,7 +457,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be # Randomness and committees 'latest_randao_mixes': ['bytes32'], - 'latest_vdf_outputs': ['bytes32'], 'previous_epoch_start_shard': 'uint64', 'current_epoch_start_shard': 'uint64', 'previous_epoch_calculation_slot': 'uint64', @@ -487,9 +464,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'previous_epoch_randao_mix': 'bytes32', 'current_epoch_randao_mix': 'bytes32', - # Custody challenges - 'custody_challenges': [CustodyChallenge], - # Finality 'previous_justified_slot': 'uint64', 'justified_slot': 'uint64', @@ -531,10 +505,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'exit_count': 'uint64', # Status flags 'status_flags': 'uint64', - # Slot of latest custody reseed - 'latest_custody_reseed_slot': 'uint64', - # Slot of second-latest custody reseed - 'penultimate_custody_reseed_slot': 'uint64', } ``` @@ -557,8 +527,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', - # Custody bitfield - 'custody_bitfield': 'bytes', # Slot the attestation was included 'slot_included': 'uint64', } @@ -1092,24 +1060,14 @@ def get_domain(fork: Fork, ```python def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) -> bool: - if len(vote_data.custody_bit_0_indices) + len(vote_data.custody_bit_1_indices) > MAX_CASPER_VOTES: + if len(vote_data.validator_indices) > MAX_CASPER_VOTES: return False - return bls_verify_multiple( - pubkeys=[ - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_0_indices]), - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_1_indices]), - ], - messages=[ - hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, False)), - hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, True)), - ], + return bls_verify( + pubkey=bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.validator_indices]), + message=hash_tree_root(AttestationData(vote_data.data)), signature=vote_data.aggregate_signature, - domain=get_domain( - state.fork, - vote_data.data.slot, - DOMAIN_ATTESTATION, - ), + domain=get_domain(state.fork, vote_data.data.slot, DOMAIN_ATTESTATION), ) ``` @@ -1198,9 +1156,6 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val proposer_slashings=[], casper_slashings=[], attestations=[], - custody_reseeds=[], - custody_challenges=[], - custody_responses=[], deposits=[], exits=[], ), @@ -1232,7 +1187,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Randomness and committees latest_randao_mixes=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH)], - latest_vdf_outputs=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH // EPOCH_LENGTH)], previous_epoch_start_shard=GENESIS_START_SHARD, current_epoch_start_shard=GENESIS_START_SHARD, previous_epoch_calculation_slot=GENESIS_SLOT, @@ -1240,9 +1194,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], previous_epoch_randao_mix=ZERO_HASH, current_epoch_randao_mix=ZERO_HASH, - # Custody challenges - custody_challenges=[], - # Finality previous_justified_slot=GENESIS_SLOT, justified_slot=GENESIS_SLOT, @@ -1340,8 +1291,6 @@ def process_deposit(state: BeaconState, penalized_slot=FAR_FUTURE_SLOT, exit_count=0, status_flags=0, - latest_custody_reseed_slot=GENESIS_SLOT, - penultimate_custody_reseed_slot=GENESIS_SLOT, ) # Note: In phase 2 registry indices that have been withdrawn for a long time will be recycled. @@ -1489,8 +1438,7 @@ For each `casper_slashing` in `block.body.casper_slashings`: * Let `slashable_vote_data_1 = casper_slashing.slashable_vote_data_1`. * Let `slashable_vote_data_2 = casper_slashing.slashable_vote_data_2`. -* Let `indices(slashable_vote_data) = slashable_vote_data.custody_bit_0_indices + slashable_vote_data.custody_bit_1_indices`. -* Let `intersection = [x for x in indices(slashable_vote_data_1) if x in indices(slashable_vote_data_2)]`. +* Let `intersection = [x for x in slashable_vote_data_1.validator_indices if x in slashable_vote_data_2.validator_indices]`. * Verify that `len(intersection) >= 1`. * Verify that `slashable_vote_data_1.data != slashable_vote_data_2.data`. * Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. @@ -1512,9 +1460,9 @@ For each `attestation` in `block.body.attestations`: * `aggregate_signature` verification: * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. * Let `group_public_key = bls_aggregate_pubkeys([state.validator_registry[v].pubkey for v in participants])`. - * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataAndCustodyBit(attestation.data, False)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. + * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationData(attestation.data)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. -* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`. +* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, slot_included=state.slot)` to `state.latest_attestations`. #### Deposits @@ -1564,10 +1512,6 @@ For each `exit` in `block.body.exits`: * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. * Run `initiate_validator_exit(state, exit.validator_index)`. -#### Custody - -[TO BE REMOVED IN PHASE 1] Verify that `len(block.body.custody_reseeds) == len(block.body.custody_challenges) == len(block.body.custody_responses) == 0`. - ## Per-epoch processing The steps below happen when `state.slot % EPOCH_LENGTH == 0`. From 697545a9e056c08e54e02027f16fdfc0c72c2cde Mon Sep 17 00:00:00 2001 From: vbuterin Date: Tue, 22 Jan 2019 22:47:07 -0600 Subject: [PATCH 030/136] Added entry_exit_effect_slot helper and moved index roots update --- specs/core/0_beacon-chain.md | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index b0e7303c5..56789d822 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -83,6 +83,7 @@ - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) + - [`entry_exit_effect_slot`](#entry_exit_effect_slot) - [`bls_verify`](#bls_verify) - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) @@ -1159,7 +1160,7 @@ def is_surround_vote(attestation_data_1: AttestationData, ```python def integer_squareroot(n: int) -> int: """ - The largest integer ``x`` such that ``x**2`` is less than ``n``. + The largest integer ``x`` such that ``x**2`` is less than or equal to ``n``. """ assert n >= 0 x = n @@ -1170,6 +1171,17 @@ def integer_squareroot(n: int) -> int: return x ``` +#### `entry_exit_effect_slot` + +```python +def entry_exit_effect_slot(n: int) -> int: + """ + An entry or exit triggered in the slot given by the input takes effect at + the slot given by the output. + """ + return (n - n % EPOCH_LENGTH) + EPOCH_LENGTH + ENTRY_EXIT_DELAY +``` + #### `bls_verify` `bls_verify` is a function for verifying a BLS signature, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify). @@ -1372,13 +1384,13 @@ def process_deposit(state: BeaconState, ### Routines for updating validator status -Note: All functions in this section mutate `state`. +Note: All functions in this section mutate `state`. ```python def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_slot = GENESIS_SLOT if genesis else (state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY) + validator.activation_slot = GENESIS_SLOT if genesis else entry_exit_effect_slot(state.slot) ``` ```python @@ -1392,10 +1404,10 @@ def exit_validator(state: BeaconState, index: int) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited - if validator.exit_slot <= state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY: + if validator.exit_slot <= entry_exit_effect_slot(state.slot): return - validator.exit_slot = state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY + validator.exit_slot = entry_exit_effect_slot(state.slot) state.validator_registry_exit_count += 1 validator.exit_count = state.validator_registry_exit_count @@ -1560,7 +1572,7 @@ Verify that `len(block.body.exits) <= MAX_EXITS`. For each `exit` in `block.body.exits`: * Let `validator = state.validator_registry[exit.validator_index]`. -* Verify that `validator.exit_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY`. +* Verify that `validator.exit_slot > entry_exit_effect_slot(state.slot)`. * Verify that `state.slot >= exit.slot`. * Let `exit_message = hash_tree_root(Exit(slot=exit.slot, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. @@ -1711,12 +1723,13 @@ def process_ejections(state: BeaconState) -> None: exit_validator(state, index) ``` -### Validator registry +### Validator registry and shuffling seed data -First, update `previous_epoch_calculation_slot` and `previous_epoch_start_shard`: +First, update `previous_epoch_calculation_slot`, `previous_epoch_start_shard` and `latest_index_roots`: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` +* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` If the following are satisfied: @@ -1745,7 +1758,7 @@ def update_validator_registry(state: BeaconState) -> None: # Activate validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.activation_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: + if validator.activation_slot > entry_exit_effect_slot(state.slot) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1757,7 +1770,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.exit_slot > state.slot - state.slot % EPOCH_LENGTH + ENTRY_EXIT_DELAY and validator.status_flags & INITIATED_EXIT: + if validator.exit_slot > entry_exit_effect_slot(state.slot) and validator.status_flags & INITIATED_EXIT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1825,7 +1838,6 @@ def process_penalties_and_exits(state: BeaconState) -> None: * Let `epoch = state.slot // EPOCH_LENGTH`. * Set `state.latest_penalized_balances[(epoch+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[epoch % LATEST_PENALIZED_EXIT_LENGTH]` * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`. -* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` ## State root processing From f9097bfe8b3832db07b8a263320aff93368b3a68 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 23 Jan 2019 00:22:47 -0600 Subject: [PATCH 031/136] Update specs/core/0_beacon-chain.md Co-Authored-By: vbuterin --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 56789d822..9399bc255 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1384,7 +1384,7 @@ def process_deposit(state: BeaconState, ### Routines for updating validator status -Note: All functions in this section mutate `state`. +Note: All functions in this section mutate `state`. ```python def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: From 109ac8969774188011a94ad2ed36e21687c59f09 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 09:57:48 +0000 Subject: [PATCH 032/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index a47cad399..4d900cfe8 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -30,6 +30,7 @@ - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) + - [`AttestationDataWrapped`](#attestationdatawrapped) - [Deposits](#deposits) - [`Deposit`](#deposit) - [`DepositData`](#depositdata) @@ -290,12 +291,12 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { - # Validator indices - 'validator_indices': ['uint24'], # Attestation data 'data': AttestationData, # Aggregate signature 'aggregate_signature': 'bytes96', + # Validator indices + 'validator_indices': ['uint24'], } ``` @@ -337,6 +338,15 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be } ``` +#### `AttestationDataWrapped` + +```python +{ + # Attestation data + 'data': AttestationData, +} +``` + #### Deposits ##### `Deposit` @@ -1065,7 +1075,7 @@ def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) return bls_verify( pubkey=bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.validator_indices]), - message=hash_tree_root(AttestationData(vote_data.data)), + message=hash_tree_root(AttestationDataWrapped(data=vote_data.data)), signature=vote_data.aggregate_signature, domain=get_domain(state.fork, vote_data.data.slot, DOMAIN_ATTESTATION), ) @@ -1460,7 +1470,7 @@ For each `attestation` in `block.body.attestations`: * `aggregate_signature` verification: * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. * Let `group_public_key = bls_aggregate_pubkeys([state.validator_registry[v].pubkey for v in participants])`. - * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationData(attestation.data)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. + * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataWrapped(data=attestation.data)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. * Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, slot_included=state.slot)` to `state.latest_attestations`. From 6ac5608d0be01a8eb40440bcb2030e3439af0557 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Tue, 22 Jan 2019 01:21:22 +0800 Subject: [PATCH 033/136] Explicit check bytes end --- specs/simple-serialize.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 8630c47c6..9774a9ecd 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -219,14 +219,22 @@ The decoding requires knowledge of the type of the item to be decoded. When performing decoding on an entire serialized string, it also requires knowledge of the order in which the objects have been serialized. -Note: Each return will provide ``deserialized_object, new_index`` keeping track -of the new index. +Note: Each return will provide: +- `deserialized_object` +- `new_index` At each step, the following checks should be made: | Check to perform | Check | |:-------------------------|:-----------------------------------------------------------| -| Ensure sufficient length | ``length(rawbytes) >= current_index + deserialize_length`` | +| Ensure sufficient length | ``len(rawbytes) >= current_index + deserialize_length`` | + +At the final step, the following checks should be made: + +| Check to perform | Check | +|:-------------------------|:-------------------------------------| +| Ensure no extra length | `new_index == len(rawbytes)` | + #### uint @@ -326,6 +334,7 @@ Instantiate a container with the full set of deserialized data, matching each me | rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | + To deserialize: 1. Get the list of the container's fields. From 696a6c140a23dc900cc31878c584daecd022488d Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 12:40:59 +0000 Subject: [PATCH 034/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 121 +++++++++++++++++++++++++++++------ 1 file changed, 101 insertions(+), 20 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4d900cfe8..865c4c175 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -30,7 +30,7 @@ - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) - - [`AttestationDataWrapped`](#attestationdatawrapped) + - [`AttestationDataAndCustodyBit`](#attestationdataandcustodybit) - [Deposits](#deposits) - [`Deposit`](#deposit) - [`DepositData`](#depositdata) @@ -80,10 +80,13 @@ - [`get_effective_balance`](#get_effective_balance) - [`get_fork_version`](#get_fork_version) - [`get_domain`](#get_domain) + - [`get_bitfield_bit`](#get_bitfield_bit) + - [`verify_bitfield`](#verify_bitfield) - [`verify_slashable_vote_data`](#verify_slashable_vote_data) - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) + - [`indices`](#indices) - [`bls_verify`](#bls_verify) - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) @@ -251,6 +254,13 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `DOMAIN_EXIT` | `3` | | `DOMAIN_RANDAO` | `4` | +### Custody bits + +| Name | Value | +| - | - | +| `CUSTODY_BIT_ZERO` | `False` | +| `CUSTODY_BIT_ONE` | `True` | + ## Data structures ### Beacon chain operations @@ -291,12 +301,14 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { + # Validator indices with custody bit equal to 0 + 'custody_bit_0_indices': ['uint24'], + # Validator indices with custody bit equal to 1 + 'custody_bit_1_indices': ['uint24'], # Attestation data 'data': AttestationData, # Aggregate signature 'aggregate_signature': 'bytes96', - # Validator indices - 'validator_indices': ['uint24'], } ``` @@ -310,6 +322,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', + # Custody bitfield + 'custody_bitfield': 'bytes', # BLS aggregate signature 'aggregate_signature': 'bytes96', } @@ -338,12 +352,14 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be } ``` -#### `AttestationDataWrapped` +##### `AttestationDataAndCustodyBit` ```python { # Attestation data 'data': AttestationData, + # Custody bit + 'custody_bit': bool, } ``` @@ -537,6 +553,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', + # Custody bitfield + 'custody_bitfield': 'bytes', # Slot the attestation was included 'slot_included': 'uint64', } @@ -1008,22 +1026,20 @@ def merkle_root(values: List[Bytes32]) -> Bytes32: ```python def get_attestation_participants(state: BeaconState, attestation_data: AttestationData, - aggregation_bitfield: bytes) -> List[int]: + bitfield: bytes) -> List[int]: """ - Returns the participant indices at for the ``attestation_data`` and ``aggregation_bitfield``. + Returns the participant indices at for the ``attestation_data`` and ``bitfield``. """ - # Find the committee in the list with the desired shard crosslink_committees = get_crosslink_committees_at_slot(state, attestation_data.slot) assert attestation_data.shard in [shard for _, shard in crosslink_committees] crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0] - assert len(aggregation_bitfield) == (len(committee) + 7) // 8 # Find the participating attesters in the committee participants = [] for i, validator_index in enumerate(crosslink_committee): - aggregation_bit = (aggregation_bitfield[i // 8] >> (7 - (i % 8))) % 2 + aggregation_bit = get_bitfield_bit(bitfield, i) if aggregation_bit == 1: participants.append(validator_index) return participants @@ -1066,18 +1082,51 @@ def get_domain(fork: Fork, ) * 2**32 + domain_type ``` +#### `get_bitfield_bit` + +```python +def get_bitfield_bit(bitfield: bytes, i: int) -> int: + return (bitfield[i // 8] >> (7 - (i % 8))) % 2 +``` + +#### `verify_bitfield` + +```python +def verify_bitfield(bitfield: bytes, size: int) -> bool: + ``` + Verify ``bitfield`` against the ``size``. + if len(bitfield) != (size + 7) // 8: + return False + + for i in range(size + 1, size + size % 8 + 8): + if get_bitfield_bit(bitfield, i) != 0: + return False + + return True +``` + #### `verify_slashable_vote_data` ```python -def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) -> bool: - if len(vote_data.validator_indices) > MAX_CASPER_VOTES: +def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: SlashableVoteData) -> bool: + # [TO BE REMOVED IN PHASE 1] + if len(slashable_vote_data.custody_bit_1_indices) > 0: + return False + + if len(indices(slashable_vote_data)) > MAX_CASPER_VOTES: return False return bls_verify( - pubkey=bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.validator_indices]), - message=hash_tree_root(AttestationDataWrapped(data=vote_data.data)), - signature=vote_data.aggregate_signature, - domain=get_domain(state.fork, vote_data.data.slot, DOMAIN_ATTESTATION), + pubkeys=[ + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in slashable_vote_data.custody_bit_0_indices]), + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in slashable_vote_data.custody_bit_1_indices]), + ], + messages=[ + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, CUSTODY_BIT_ZERO)), + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, CUSTODY_BIT_ONE)), + ], + signature=slashable_vote_data.aggregate_signature, + domain=get_domain(state.fork, slashable_vote_data.data.slot, DOMAIN_ATTESTATION), ) ``` @@ -1135,6 +1184,16 @@ def integer_squareroot(n: int) -> int: return x ``` +#### `indices` + +```python +def indices(slashable_vote_data: SlashableVoteData) -> List[int]: + """ + Return all indicies in ``slashable_vote_data``. + """ + return slashable_vote_data.custody_bit_0_indices + slashable_vote_data.custody_bit_1_indices +``` + #### `bls_verify` `bls_verify` is a function for verifying a BLS signature, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify). @@ -1448,7 +1507,7 @@ For each `casper_slashing` in `block.body.casper_slashings`: * Let `slashable_vote_data_1 = casper_slashing.slashable_vote_data_1`. * Let `slashable_vote_data_2 = casper_slashing.slashable_vote_data_2`. -* Let `intersection = [x for x in slashable_vote_data_1.validator_indices if x in slashable_vote_data_2.validator_indices]`. +* Let `intersection = [x for x in indices(slashable_vote_data_1) if x in indices(slashable_vote_data_2)]`. * Verify that `len(intersection) >= 1`. * Verify that `slashable_vote_data_1.data != slashable_vote_data_2.data`. * Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. @@ -1467,10 +1526,32 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.justified_slot` is equal to `state.justified_slot if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_slot`. * Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_slot)`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. -* `aggregate_signature` verification: - * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. - * Let `group_public_key = bls_aggregate_pubkeys([state.validator_registry[v].pubkey for v in participants])`. - * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataWrapped(data=attestation.data)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. +* `attestation.aggregate_signature` verification: + +```python + assert verify_bitfield(attestation.aggregation_bitfield) + assert verify_bitfield(attestation.custody_bitfield) + assert attestation.custody_bitfield & attestation.aggregation_bitfield == attestation.custody_bitfield + + participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield) + custody_bit_0_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) + custody_bit_1_participants = [i in participants for i not in custody_bit_0_participants] + + assert bls_verify_multiple( + pubkeys=[ + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_0_participants]), + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_participants]), + ], + messages=[ + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=CUSTODY_BIT_ZERO)), + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=CUSTODY_BIT_ONE)), + ], + signature=attestation.aggregate_signature, + domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION), + ) + +``` + * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. * Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, slot_included=state.slot)` to `state.latest_attestations`. From ee081e160ca93eddd3bab9c59ed2d6a6a5979fb2 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 12:47:11 +0000 Subject: [PATCH 035/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 865c4c175..ecbcdebb1 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1526,13 +1526,17 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.justified_slot` is equal to `state.justified_slot if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_slot`. * Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_slot)`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. -* `attestation.aggregate_signature` verification: +* Verify bitfields: ```python assert verify_bitfield(attestation.aggregation_bitfield) assert verify_bitfield(attestation.custody_bitfield) assert attestation.custody_bitfield & attestation.aggregation_bitfield == attestation.custody_bitfield +``` +* Verify aggregate signature: + +```python participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield) custody_bit_0_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) custody_bit_1_participants = [i in participants for i not in custody_bit_0_participants] From ac85e06e81f0664a5177478a9f3d37efeb255004 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 12:48:54 +0000 Subject: [PATCH 036/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ecbcdebb1..3a79ee71d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1109,8 +1109,7 @@ def verify_bitfield(bitfield: bytes, size: int) -> bool: ```python def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: SlashableVoteData) -> bool: - # [TO BE REMOVED IN PHASE 1] - if len(slashable_vote_data.custody_bit_1_indices) > 0: + if len(slashable_vote_data.custody_bit_1_indices) > 0: # [TO BE REMOVED IN PHASE 1] return False if len(indices(slashable_vote_data)) > MAX_CASPER_VOTES: @@ -1532,6 +1531,7 @@ For each `attestation` in `block.body.attestations`: assert verify_bitfield(attestation.aggregation_bitfield) assert verify_bitfield(attestation.custody_bitfield) assert attestation.custody_bitfield & attestation.aggregation_bitfield == attestation.custody_bitfield + assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] ``` * Verify aggregate signature: From 1e48535edd4e80779008a17a1eef1eb66d13ca32 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 12:57:06 +0000 Subject: [PATCH 037/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 3a79ee71d..151e5f696 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1557,7 +1557,7 @@ For each `attestation` in `block.body.attestations`: ``` * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. -* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, slot_included=state.slot)` to `state.latest_attestations`. +* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`. #### Deposits From e3a5720c1b7587e69cdebb453a6ae07e9d198e80 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 13:03:33 +0000 Subject: [PATCH 038/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 151e5f696..585ff5223 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -20,6 +20,7 @@ - [Max operations per block](#max-operations-per-block) - [Validator registry delta flags](#validator-registry-delta-flags) - [Signature domains](#signature-domains) + - [Custody bits](#custody-bits) - [Data structures](#data-structures) - [Beacon chain operations](#beacon-chain-operations) - [Proposer slashings](#proposer-slashings) From d1efd65d8a53a1e69b087cf30f7830eb36dd1acf Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 13:24:35 +0000 Subject: [PATCH 039/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 46 +++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 19 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 585ff5223..ac3a00a37 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -302,10 +302,10 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { - # Validator indices with custody bit equal to 0 - 'custody_bit_0_indices': ['uint24'], - # Validator indices with custody bit equal to 1 - 'custody_bit_1_indices': ['uint24'], + # Validator indices + 'validator_indices': '[uint24]', + # Custody bitfield + 'custody_bitfield': 'bytes', # Attestation data 'data': AttestationData, # Aggregate signature @@ -1110,16 +1110,34 @@ def verify_bitfield(bitfield: bytes, size: int) -> bool: ```python def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: SlashableVoteData) -> bool: - if len(slashable_vote_data.custody_bit_1_indices) > 0: # [TO BE REMOVED IN PHASE 1] + if slashable_vote_data.custody_bitfield != 0: # [TO BE REMOVED IN PHASE 1] return False - if len(indices(slashable_vote_data)) > MAX_CASPER_VOTES: + if len(slashable_vote_data.validator_indices) == 0: return False + for i in range(len(slashable_vote_data.validator_indices) - 1): + if slashable_vote_data.validator_indices[i] >= slashable_vote_data.validator_indices[i + 1]: + return False + + if not verify_bitfield(slashable_vote_data.custody_bitfield, len(slashable_vote_data.validator_indices)): + return False + + if len(slashable_vote_data.validator_indices) > MAX_CASPER_VOTES: + return False + + custody_bit_0_indices = [] + custody_bit_1_indices = [] + for i, validator_index in enumerate(slashable_vote_data.validator_indices): + if get_bitfield_bit(slashable_vote_data.custody_bitfield, i) == 0: + custody_bit_0_indices.append(validator_index) + else: + custody_bit_1_indices.append(validator_index) + return bls_verify( pubkeys=[ - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in slashable_vote_data.custody_bit_0_indices]), - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in slashable_vote_data.custody_bit_1_indices]), + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_0_indices]), + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, CUSTODY_BIT_ZERO)), @@ -1184,16 +1202,6 @@ def integer_squareroot(n: int) -> int: return x ``` -#### `indices` - -```python -def indices(slashable_vote_data: SlashableVoteData) -> List[int]: - """ - Return all indicies in ``slashable_vote_data``. - """ - return slashable_vote_data.custody_bit_0_indices + slashable_vote_data.custody_bit_1_indices -``` - #### `bls_verify` `bls_verify` is a function for verifying a BLS signature, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify). @@ -1507,7 +1515,7 @@ For each `casper_slashing` in `block.body.casper_slashings`: * Let `slashable_vote_data_1 = casper_slashing.slashable_vote_data_1`. * Let `slashable_vote_data_2 = casper_slashing.slashable_vote_data_2`. -* Let `intersection = [x for x in indices(slashable_vote_data_1) if x in indices(slashable_vote_data_2)]`. +* Let `intersection = [x for x in slashable_vote_data_1.validator_indices if x in slashable_vote_data_2.validator_indices]`. * Verify that `len(intersection) >= 1`. * Verify that `slashable_vote_data_1.data != slashable_vote_data_2.data`. * Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. From 022dd0cc3d8ad239bb484cbf16d1f45bd360c951 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 13:26:23 +0000 Subject: [PATCH 040/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ac3a00a37..8c6197927 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1561,8 +1561,7 @@ For each `attestation` in `block.body.attestations`: ], signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION), - ) - + ) ``` * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. From 6f3cb7e7e5991366f5dbfbc729d739edcf698a94 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 13:28:43 +0000 Subject: [PATCH 041/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8c6197927..a708a30a8 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -255,13 +255,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `DOMAIN_EXIT` | `3` | | `DOMAIN_RANDAO` | `4` | -### Custody bits - -| Name | Value | -| - | - | -| `CUSTODY_BIT_ZERO` | `False` | -| `CUSTODY_BIT_ONE` | `True` | - ## Data structures ### Beacon chain operations @@ -1140,8 +1133,8 @@ def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: Slashabl bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, CUSTODY_BIT_ZERO)), - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, CUSTODY_BIT_ONE)), + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, 0)), + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, 1)), ], signature=slashable_vote_data.aggregate_signature, domain=get_domain(state.fork, slashable_vote_data.data.slot, DOMAIN_ATTESTATION), @@ -1556,8 +1549,8 @@ For each `attestation` in `block.body.attestations`: bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_participants]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=CUSTODY_BIT_ZERO)), - hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=CUSTODY_BIT_ONE)), + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=0)), + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=1)), ], signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION), From dfaf24c75e3df6aaaabab88bfd6115c23928dc77 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 23 Jan 2019 13:52:52 +0000 Subject: [PATCH 042/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index a708a30a8..5c0663242 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -20,7 +20,6 @@ - [Max operations per block](#max-operations-per-block) - [Validator registry delta flags](#validator-registry-delta-flags) - [Signature domains](#signature-domains) - - [Custody bits](#custody-bits) - [Data structures](#data-structures) - [Beacon chain operations](#beacon-chain-operations) - [Proposer slashings](#proposer-slashings) @@ -1080,6 +1079,9 @@ def get_domain(fork: Fork, ```python def get_bitfield_bit(bitfield: bytes, i: int) -> int: + """ + Extract the bit in ``bitfield`` at position ``i``. + """ return (bitfield[i // 8] >> (7 - (i % 8))) % 2 ``` @@ -1087,12 +1089,13 @@ def get_bitfield_bit(bitfield: bytes, i: int) -> int: ```python def verify_bitfield(bitfield: bytes, size: int) -> bool: - ``` + """ Verify ``bitfield`` against the ``size``. + """ if len(bitfield) != (size + 7) // 8: return False - for i in range(size + 1, size + size % 8 + 8): + for i in range(size + 1, size - size % 8 + 8): if get_bitfield_bit(bitfield, i) != 0: return False @@ -1103,6 +1106,9 @@ def verify_bitfield(bitfield: bytes, size: int) -> bool: ```python def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: SlashableVoteData) -> bool: + """ + Verify validity of ``slashable_vote_data`` fields. + """ if slashable_vote_data.custody_bitfield != 0: # [TO BE REMOVED IN PHASE 1] return False From dd075edd06e5069b38a0cd929f220db95765472d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 23 Jan 2019 15:30:05 -0600 Subject: [PATCH 043/136] epoch trans at end of epoch and fix finality conditions --- specs/core/0_beacon-chain.md | 39 +++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index d58d49df8..cedd48172 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1570,10 +1570,13 @@ For each `exit` in `block.body.exits`: ## Per-epoch processing -The steps below happen when `state.slot % EPOCH_LENGTH == 0`. +The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. ### Helpers +* Let `current_epoch_start_slot = state.slot - (EPOCH_LENGTH + 1)`. +* Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > EPOCH_LENGTH` else `current_epoch_start_slot`. + All [validators](#dfn-validator): * Let `active_validator_indices = get_active_validator_indices(state.validator_registry, state.slot)`. @@ -1581,23 +1584,23 @@ All [validators](#dfn-validator): [Validators](#dfn-Validator) attesting during the current epoch: -* Let `current_epoch_attestations = [a for a in state.latest_attestations if state.slot - EPOCH_LENGTH <= a.data.slot < state.slot]`. (Note: this is the set of attestations of slots in the epoch `state.slot-EPOCH_LENGTH...state.slot-1`, _not_ attestations that got included in the chain during the epoch `state.slot-EPOCH_LENGTH...state.slot-1`.) +* Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch_start_slot <= a.data.slot <= state.slot]`. (Note: this is the set of attestations of slots in the epoch `current_epoch_start_slot...state.slot`, _not_ attestations that got included in the chain during the epoch `current_epoch_start_slot...state.slot`.) * Validators justifying the epoch boundary block at the start of the current epoch: - * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, state.slot-EPOCH_LENGTH) and a.data.justified_slot == state.justified_slot]`. + * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, current_epoch_start_slot) and a.data.justified_slot == state.justified_slot]`. * Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_boundary_attestations]`. * Let `current_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in current_epoch_boundary_attester_indices])`. [Validators](#dfn-Validator) attesting during the previous epoch: * Validators that made an attestation during the previous epoch: - * Let `previous_epoch_attestations = [a for a in state.latest_attestations if state.slot - 2 * EPOCH_LENGTH <= a.data.slot < state.slot - EPOCH_LENGTH]`. + * Let `previous_epoch_attestations = [a for a in state.latest_attestations if previous_epoch_start_slot <= a.data.slot < current_epoch_start_slot]`. * Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_attestations]`. * Validators targeting the previous justified slot: * Let `previous_epoch_justified_attestations = [a for a in current_epoch_attestations + previous_epoch_attestations if a.data.justified_slot == state.previous_justified_slot]`. * Let `previous_epoch_justified_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_justified_attestations]`. * Let `previous_epoch_justified_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_justified_attester_indices])`. * Validators justifying the epoch boundary block at the start of the previous epoch: - * Let `previous_epoch_boundary_attestations = [a for a in previous_epoch_justified_attestations if a.data.epoch_boundary_root == get_block_root(state, state.slot - 2 * EPOCH_LENGTH)]`. + * Let `previous_epoch_boundary_attestations = [a for a in previous_epoch_justified_attestations if a.data.epoch_boundary_root == get_block_root(state, previous_epoch_start_slot)]`. * Let `previous_epoch_boundary_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_boundary_attestations]`. * Let `previous_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_boundary_attester_indices])`. * Validators attesting to the expected beacon chain head during the previous epoch: @@ -1623,23 +1626,31 @@ Define the following helpers to process attestation inclusion rewards and inclus ### Eth1 data -If `state.slot % ETH1_DATA_VOTING_PERIOD == 0`: +If `state.slot % ETH1_DATA_VOTING_PERIOD == ETH1_DATA_VOTING_PERIOD - 1`: * Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. ### Justification -* Set `state.previous_justified_slot = state.justified_slot`. +First, update the justification bitfield: + +* Let `new_justified_slot = state.justified_slot`. * Set `state.justification_bitfield = (state.justification_bitfield * 2) % 2**64`. -* Set `state.justification_bitfield |= 2` and `state.justified_slot = state.slot - 2 * EPOCH_LENGTH` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. -* Set `state.justification_bitfield |= 1` and `state.justified_slot = state.slot - 1 * EPOCH_LENGTH` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield |= 2` and `new_justified_slot = previous_epoch_start_slot` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield |= 1` and `new_justified_slot = current_epoch_start_slot` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. -Set `state.finalized_slot = state.previous_justified_slot` if any of the following are true: +Next, update last finalized slot if possible: -* `state.previous_justified_slot == state.slot - 2 * EPOCH_LENGTH and state.justification_bitfield % 4 == 3` -* `state.previous_justified_slot == state.slot - 3 * EPOCH_LENGTH and state.justification_bitfield % 8 == 7` -* `state.previous_justified_slot == state.slot - 4 * EPOCH_LENGTH and state.justification_bitfield % 16 in (15, 14)` +* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 4 == 0b11 and state.justified_slot == previous_epoch_start_slot`. +* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. +* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 8 == 0b111 and state.justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. +* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_slot == previous_epoch_start_slot - 2 * EPOCH_LENGTH`. + +Finally, update the following: + +* Set `state.previous_justified_slot = state.justified_slot`. +* Set `state.justified_slot = justified_slot`. ### Crosslinks @@ -1819,7 +1830,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates * Let `e = state.slot // EPOCH_LENGTH`. Set `state.latest_penalized_balances[(e+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[e % LATEST_PENALIZED_EXIT_LENGTH]` -* Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`. +* Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < current_epoch_start_slot`. ## State root processing From b18de2c86cdd812d072728ee094b3f7915474460 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 23 Jan 2019 15:41:25 -0600 Subject: [PATCH 044/136] clena up get_crosslink_committees_at_slots --- specs/core/0_beacon-chain.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index cedd48172..013676f55 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -942,24 +942,25 @@ def get_crosslink_committees_at_slot(state: BeaconState, state_epoch_slot = state.slot - (state.slot % EPOCH_LENGTH) assert state_epoch_slot <= slot + EPOCH_LENGTH assert slot < state_epoch_slot + EPOCH_LENGTH - offset = slot % EPOCH_LENGTH if slot < state_epoch_slot: committees_per_slot = get_previous_epoch_committee_count_per_slot(state) - shuffling = get_shuffling( - state.previous_epoch_randao_mix, - state.validator_registry, - state.previous_epoch_calculation_slot, - ) - slot_start_shard = (state.previous_epoch_start_shard + committees_per_slot * offset) % SHARD_COUNT + seed = state.previous_epoch_randao_mix + shuffling_slot = state.previous_epoch_calculation_slot + shuffling_start_shard = state.previous_epoch_start_shard else: committees_per_slot = get_current_epoch_committee_count_per_slot(state) - shuffling = get_shuffling( - state.current_epoch_randao_mix, - state.validator_registry, - state.current_epoch_calculation_slot, - ) - slot_start_shard = (state.current_epoch_start_shard + committees_per_slot * offset) % SHARD_COUNT + seed = state.current_epoch_randao_mix + shuffling_slot = state.current_epoch_calculation_slot + shuffling_start_shard = state.current_epoch_start_shard + + shuffling = get_shuffling( + seed, + state.validator_registry, + shuffling_slot, + ) + offset = slot % EPOCH_LENGTH + slot_start_shard = (shuffling_start_shard + committees_per_slot * offset) % SHARD_COUNT return [ ( From 1aceaeb793f620de01087fee1221029f486dcd6e Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 23 Jan 2019 17:14:38 -0600 Subject: [PATCH 045/136] reverse ordering of finality conditions --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 013676f55..dfb41ab47 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1643,10 +1643,10 @@ First, update the justification bitfield: Next, update last finalized slot if possible: -* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 4 == 0b11 and state.justified_slot == previous_epoch_start_slot`. +* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_slot == previous_epoch_start_slot - 2 * EPOCH_LENGTH`. * Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. * Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 8 == 0b111 and state.justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. -* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_slot == previous_epoch_start_slot - 2 * EPOCH_LENGTH`. +* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 4 == 0b11 and state.justified_slot == previous_epoch_start_slot`. Finally, update the following: From 028eba903ea542394167ea838e1d7b3669464b10 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 23 Jan 2019 17:15:53 -0600 Subject: [PATCH 046/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index dde1610ba..86d0bd14c 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -309,7 +309,7 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes * Let `aggregation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. * Let `index_into_committee` be the index into the validator's `committee` at which `validator_index` is located. -* Set `participation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. +* Set `aggregation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. * Set `attestation.participation_bitfield = participation_bitfield`. _Note_: Calling `get_attestation_participants(state, attestation.data, attestation.participation_bitfield)` should return `[validator_index]`. From 4a566469a5cb0c27b26c606931c4e1017e8c3fa2 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 23 Jan 2019 17:16:04 -0600 Subject: [PATCH 047/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 86d0bd14c..61539d974 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -310,7 +310,7 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes * Let `aggregation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. * Let `index_into_committee` be the index into the validator's `committee` at which `validator_index` is located. * Set `aggregation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. -* Set `attestation.participation_bitfield = participation_bitfield`. +* Set `attestation.aggregation_bitfield = aggregation_bitfield`. _Note_: Calling `get_attestation_participants(state, attestation.data, attestation.participation_bitfield)` should return `[validator_index]`. From b7c2f33dcbc8ec35e29abeeabc254283c02e99d7 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 23 Jan 2019 17:26:11 -0600 Subject: [PATCH 048/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 61539d974..8ac6fdf45 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -312,7 +312,7 @@ Set `attestation.data = attestation_data` where `attestation_data` is the `Attes * Set `aggregation_bitfield[index_into_committee // 8] |= 2 ** (index_into_committee % 8)`. * Set `attestation.aggregation_bitfield = aggregation_bitfield`. -_Note_: Calling `get_attestation_participants(state, attestation.data, attestation.participation_bitfield)` should return `[validator_index]`. +_Note_: Calling `get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)` should return a list of length equal to 1, containing `validator_index`. ##### Custody bitfield From 59b301f7af7b91925083bcf68d460a44bbc8254e Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 23 Jan 2019 17:31:27 -0600 Subject: [PATCH 049/136] Update specs/validator/0_beacon-chain-validator.md Co-Authored-By: djrtwo --- specs/validator/0_beacon-chain-validator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 8ac6fdf45..6d3a1976b 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -151,7 +151,7 @@ The function [`is_active_validator`](https://github.com/ethereum/eth2.0-specs/bl ```python validator = state.validator_registry[validator_index] -is_active_validator(validator, slot) +is_active = is_active_validator(validator, slot) ``` Once a validator is active, the validator is assigned [responsibilities](#beacon-chain-responsibilities) until exited. From df85857c2edb700f9882eae888561a6eb99519b6 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Wed, 23 Jan 2019 20:35:26 -0600 Subject: [PATCH 050/136] A few more state.slot -> current/previous/next epoch start slot changes --- specs/core/0_beacon-chain.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index dfb41ab47..2fa4c151c 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1575,12 +1575,13 @@ The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. ### Helpers +* Let `next_epoch_start_slot = state.slot + 1`. * Let `current_epoch_start_slot = state.slot - (EPOCH_LENGTH + 1)`. * Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > EPOCH_LENGTH` else `current_epoch_start_slot`. All [validators](#dfn-validator): -* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, state.slot)`. +* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch_start_slot)`. * Let `total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices])`. [Validators](#dfn-Validator) attesting during the current epoch: @@ -1611,7 +1612,7 @@ All [validators](#dfn-validator): **Note**: `previous_epoch_boundary_attesting_balance` balance might be marginally different than `current_epoch_boundary_attesting_balance` during the previous epoch transition. Due to the tight bound on validator churn each epoch and small per-epoch rewards/penalties, the potential balance difference is very low and only marginally affects consensus safety. -For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * Let `shard_block_root` be `state.latest_crosslinks[shard].shard_block_root` * Let `attesting_validator_indices(crosslink_committee, shard_block_root)` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_attestations + previous_epoch_attestations if a.data.shard == shard and a.data.shard_block_root == shard_block_root]`. @@ -1655,9 +1656,9 @@ Finally, update the following: ### Crosslinks -For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: -* Set `state.latest_crosslinks[shard] = Crosslink(slot=state.slot, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. +* Set `state.latest_crosslinks[shard] = Crosslink(slot=current_epoch_start_slot, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. ### Rewards and penalties @@ -1671,7 +1672,7 @@ First, we define some additional helpers: Note: When applying penalties in the following balance recalculations implementers should make sure the `uint64` does not underflow. -* Let `epochs_since_finality = (state.slot - state.finalized_slot) // EPOCH_LENGTH`. +* Let `epochs_since_finality = (next_epoch_start_slot - state.finalized_slot) // EPOCH_LENGTH`. Case 1: `epochs_since_finality <= 4`: @@ -1692,7 +1693,7 @@ Case 2: `epochs_since_finality > 4`: * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_justified_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_head_attester_indices`, loses `base_reward(state, index)`. -* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_slot <= state.slot`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. +* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_slot <= current_epoch_start_slot`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` loses `base_reward(state, index) - base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` #### Attestation inclusion @@ -1701,7 +1702,7 @@ For each `index` in `previous_epoch_attester_indices`, we determine the proposer #### Crosslinks -For every `slot in range(state.slot - 2 * EPOCH_LENGTH, state.slot - EPOCH_LENGTH)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(previous_epoch_start_slot, current_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * If `index in attesting_validators(crosslink_committee)`, `state.validator_balances[index] += base_reward(state, index) * total_attesting_balance(crosslink_committee) // total_balance(crosslink_committee))`. * If `index not in attesting_validators(crosslink_committee)`, `state.validator_balances[index] -= base_reward(state, index)`. @@ -1716,7 +1717,7 @@ def process_ejections(state: BeaconState) -> None: Iterate through the validator registry and eject active validators with balance below ``EJECTION_BALANCE``. """ - for index in get_active_validator_indices(state.validator_registry, state.slot): + for index in get_active_validator_indices(state.validator_registry, current_epoch_start_slot): if state.validator_balances[index] < EJECTION_BALANCE: exit_validator(state, index) ``` @@ -1743,7 +1744,7 @@ def update_validator_registry(state: BeaconState) -> None: Note that this function mutates ``state``. """ # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, state.slot) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch_start_slot) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) From 5bb02a9d09cd2fe8a5358d1854bb1c9c888bf798 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Thu, 24 Jan 2019 15:53:52 +0800 Subject: [PATCH 051/136] Update specs/simple-serialize.md Co-Authored-By: ChihChengLiang --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 9774a9ecd..60b436e6d 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -331,7 +331,7 @@ Instantiate a container with the full set of deserialized data, matching each me | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | From c2112f0bfcf122856105d0690cccef9156dfc484 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 15:57:50 +0800 Subject: [PATCH 052/136] PR feedback: remove unnecessary newline --- specs/simple-serialize.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 60b436e6d..524a010d3 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -331,10 +331,9 @@ Instantiate a container with the full set of deserialized data, matching each me | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | - To deserialize: 1. Get the list of the container's fields. From 88ffae633527a02c6de68df9c40b0e98615ffc83 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:08:56 +0800 Subject: [PATCH 053/136] define deserialized_object and new_index --- specs/simple-serialize.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 524a010d3..e0528b562 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -54,6 +54,9 @@ overhead. | `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. | | `value` | The value to serialize. | | `rawbytes` | Raw serialized bytes. | +| `deserialized_object` | The deserialized data in the data structure of your programming language. | +| `new_index` | An index to keep track the latest position where the `rawbytes` have been deserialized. | + ## Constants From 14432e91a3fd0f66806faccc533c384c0fb5eb63 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:11:04 +0800 Subject: [PATCH 054/136] add code block to variable in the table --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index e0528b562..70245aebe 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -306,7 +306,7 @@ entire length of the list. | Check to perform | code | |:------------------------------------------|:----------------------------------------------------------------| -| rawbytes has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | +| ``rawbytes`` has enough left for length | ``len(rawbytes) > current_index + LENGTH_BYTES`` | | list is not greater than serialized bytes | ``len(rawbytes) > current_index + LENGTH_BYTES + total_length`` | ```python From d41215aeececf065befefd2b99830a434ba58d01 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:11:45 +0800 Subject: [PATCH 055/136] rename Terminology to Variables and Functions --- specs/simple-serialize.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 70245aebe..f69544ffa 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -43,7 +43,7 @@ protocol for use in the Ethereum 2.0 Beacon Chain. The core feature of `ssz` is the simplicity of the serialization with low overhead. -## Terminology +## Variables and Functions | Term | Definition | |:-------------|:-----------------------------------------------------------------------------------------------| From 5dfa4e005b2d9e07e684cecc027c70407463bda0 Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:12:43 +0800 Subject: [PATCH 056/136] rename byte_order to byteorder --- specs/simple-serialize.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index f69544ffa..5edaa6e10 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -48,10 +48,10 @@ overhead. | Term | Definition | |:-------------|:-----------------------------------------------------------------------------------------------| | `little` | Little endian. | -| `byte_order` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. | +| `byteorder` | Specifies [endianness](https://en.wikipedia.org/wiki/Endianness): big endian or little endian. | | `len` | Length/number of bytes. | -| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byte_order``. | -| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byte_order``. | +| `to_bytes` | Convert to bytes. Should take parameters ``size`` and ``byteorder``. | +| `from_bytes` | Convert from bytes to object. Should take ``bytes`` and ``byteorder``. | | `value` | The value to serialize. | | `rawbytes` | Raw serialized bytes. | | `deserialized_object` | The deserialized data in the data structure of your programming language. | From 45c064a2d6cbba2fb284b0e8fae7d83742dab31c Mon Sep 17 00:00:00 2001 From: Chih Cheng Liang Date: Thu, 24 Jan 2019 16:24:05 +0800 Subject: [PATCH 057/136] remove all unnecessary newline --- specs/simple-serialize.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index 5edaa6e10..150c71b62 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -57,7 +57,6 @@ overhead. | `deserialized_object` | The deserialized data in the data structure of your programming language. | | `new_index` | An index to keep track the latest position where the `rawbytes` have been deserialized. | - ## Constants | Constant | Value | Definition | @@ -75,7 +74,6 @@ overhead. |:---------:|:-----------------------------------------------------------| | `uintN` | Type of `N` bits unsigned integer, where ``N % 8 == 0``. | - Convert directly to bytes the size of the int. (e.g. ``uint16 = 2 bytes``) All integers are serialized as **little endian**. @@ -145,7 +143,6 @@ Lists are a collection of elements of the same homogeneous type. |:--------------------------------------------|:----------------------------| | Length of serialized list fits into 4 bytes | ``len(serialized) < 2**32`` | - 1. Get the number of raw bytes to serialize: it is ``len(list) * sizeof(element)``. * Encode that as a `4-byte` **little endian** `uint32`. 2. Append the elements in a packed manner. @@ -174,7 +171,6 @@ A container represents a heterogenous, associative collection of key-value pairs To serialize a container, obtain the list of its field's names in the specified order. For each field name in this list, obtain the corresponding value and serialize it. Tightly pack the complete set of serialized values in the same order as the field names into a buffer. Calculate the size of this buffer of serialized bytes and encode as a `4-byte` **little endian** `uint32`. Prepend the encoded length to the buffer. The result of this concatenation is the final serialized value of the container. - | Check to perform | Code | |:--------------------------------------------|:----------------------------| | Length of serialized fields fits into 4 bytes | ``len(serialized) < 2**32`` | @@ -238,7 +234,6 @@ At the final step, the following checks should be made: |:-------------------------|:-------------------------------------| | Ensure no extra length | `new_index == len(rawbytes)` | - #### uint Convert directly from bytes into integer utilising the number of bytes the same @@ -453,6 +448,5 @@ return hash(b''.join([hash_tree_root(getattr(x, field)) for field in value.field | Go | [ https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz ](https://github.com/prysmaticlabs/prysm/tree/master/shared/ssz) | Go implementation of SSZ mantained by Prysmatic Labs | | Swift | [ https://github.com/yeeth/SimpleSerialize.swift ](https://github.com/yeeth/SimpleSerialize.swift) | Swift implementation maintained SSZ | - ## Copyright Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From a182fdaa6f3ca43ff7c5f57fcf3bc34d5b2d8869 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 24 Jan 2019 22:07:41 -0700 Subject: [PATCH 058/136] pr feedback --- specs/validator/0_beacon-chain-validator.md | 56 ++++++++------------- 1 file changed, 20 insertions(+), 36 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 6d3a1976b..f1e650884 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -16,8 +16,6 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Initialization](#initialization) - [BLS public key](#bls-public-key) - [BLS withdrawal key](#bls-withdrawal-key) - - [RANDAO commitment](#randao-commitment) - - [Custody commitment](#custody-commitment) - [Submit deposit](#submit-deposit) - [Process deposit](#process-deposit) - [Validator index](#validator-index) @@ -60,9 +58,9 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers ## Introduction -This document represents the expected behavior of an "honest validator" with respect to Phase 0 of the Ethereum 2.0 protocol. This document does not distinguish between a "node" and a "validator client". The separation of concerns between these (potentially) two pieces of software is left as a design decision that is outside of scope. +This document represents the expected behavior of an "honest validator" with respect to Phase 0 of the Ethereum 2.0 protocol. This document does not distinguish between a "node" (ie. the functionality of following and reading the beacon chain) and a "validator client" (ie. the functionality of actively participating in consensus). The separation of concerns between these (potentially) two pieces of software is left as a design decision that is out of scope. -A validator is an entity that participates in the consensus of the Ethereum 2.0 protocol. This is an optional role for users in which they can post ETH as collateral to seek financial returns in exchange for building and securing the protocol. This is similar to proof of work networks in which a miner provides collateral in the form of hardware/hash-power to seek returns in exchange for building and securing the protocol. +A validator is an entity that participates in the consensus of the Ethereum 2.0 protocol. This is an optional role for users in which they can post ETH as collateral and verify and attest to the validity of blocks to seek financial returns in exchange for building and securing the protocol. This is similar to proof of work networks in which a miner provides collateral in the form of hardware/hash-power to seek returns in exchange for building and securing the protocol. ## Prerequisites @@ -94,32 +92,6 @@ The validator constructs their `withdrawal_credentials` through the following: * Set `withdrawal_credentials[:1] == BLS_WITHDRAWAL_PREFIX_BYTE`. * Set `withdrawal_credentials[1:] == hash(withdrawal_pubkey)[1:]`. -#### RANDAO commitment - -A validator's RANDAO commitment is the outermost layer of a 32-byte hash-onion. To create this commitment, perform the following steps: - -* Randomly generate a 32-byte `randao_seed`. -* Store this `randao_seed` in a secure location. -* Calculate `randao_commitment = repeat_hash(randao_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not propose more than `n` beacon chain blocks. - -Assuming `>= 100k validators`, on average a validator will have an opportunity to reveal once every `>= 600k seconds`, so `<= 50 times per year`. At this estimate, `n == 5000` would last a century. If this value is poorly configured and a validator runs out of layers of to reveal, the validator can no longer propose beacon blocks and should exit. - -_Note_: A validator must be able to reveal the next layer deep from their current commitment at any time. There are many strategies that trade off space and computation to be able to provide this reveal. At one end of this trade-off, a validator might only store their `randao_seed` and repeat the `repeat_hash` calculation on the fly to re-calculate the layer `n-1` for the reveal. On the other end of this trade-off, a validator might store _all_ layers of the hash-onion and not have to perform any calculations to retrieve the layer `n-1`. A more sensible strategy might be to store every `m`th layer as cached references to recalculate the intermittent layers as needed. - -#### Custody commitment - -A validator's custody commitment is the outermost layer of a 32-byte hash-onion. To create this commitment, perform the following steps: - -* Randomly generate a 32-byte `custody_seed`. -* Store this `custody_seed` in a secure location. -* Calculate `custody_commitment = repeat_hash(custody_seed, n)` where `n` is large enough such that within the lifetime of the validator, the validator will not attest to more than `n` beacon chain blocks. - -Assuming a validator changes their `custody_seed` with frequency `>= 1 week`, the validator changes their seed approximately `<= 50 times per year`. At this estimate, `n == 5000` would last a century. If this value is poorly configured and a validator runs out of layers of to reveal, the validator can no longer update their `custody_commitment` and should exit. - -See above note on hash-onion caching strategies in [RANDAO commitment](#randao-commitment). - -_Note_: although this commitment is being committed to and stored in phase 0, it will not be used until phase 1. - ### Submit deposit In phase 0, all incoming validator deposits originate from the Ethereum 1.0 PoW chain. Deposits are made to the [deposit contract](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#ethereum-10-deposit-contract) located at `DEPOSIT_CONTRACT_ADDRESS`. @@ -133,7 +105,7 @@ To submit a deposit: * Let `amount` be the amount in Gwei to be deposited by the validator where `MIN_DEPOSIT_AMOUNT <= amount <= MAX_DEPOSIT_AMOUNT`. * Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in Gwei. -_Note_: Multiple deposits can be made by the same validator (same initialization params). A singular `Validator` will be added to `state.validator_registry` with each deposit amount being added to the validator's balance. A validator can only be activated when total deposits meet or exceed +_Note_: Deposits made for the same `pubkey` are treated as for the same validator. A singular `Validator` will be added to `state.validator_registry` with each additional deposit amount added to the validator's balance. A validator can only be activated when total deposits for the validator pubkey meet or exceed `MAX_DEPOSIT_AMOUNT`. ### Process deposit @@ -141,7 +113,7 @@ Deposits cannot be processed into the beacon chain until the eth1.0 block in whi ### Validator index -Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. +Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. A validator's `validator_index` is guaranteed to not change from the time of initial deposit until the validator exists and fully withdraws. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. ### Activation @@ -186,7 +158,19 @@ _Note_: To calculate `state_root`, the validator should first run the state tran ##### Randao reveal -Set `block.randao_reveal` to the `n`th layer deep reveal from the validator's current `randao_commitment` where `n = validator.randao_layers + 1`. `block.randao_reveal` should satisfy `repeat_hash(block.randao_reveal, validator.randao_layers + 1) == validator.randao_commitment`. +Set `block.randao_reveal = reveal_signature` where `reveal_signature` is defined as: + +```python +reveal_signature = bls_sign( + privkey=validator.privkey, # privkey store locally, not in state + message=int_to_bytes32(validator.proposer_slots + 1), + domain=get_domain( + fork_data, # `fork_data` is the fork_data at the slot `block.slot` + block.slot, + DOMAIN_RANDAO, + ) +) +``` ##### Eth1 Data @@ -220,8 +204,8 @@ signed_proposal_data = bls_sign( privkey=validator.privkey, # privkey store locally, not in state message=proposal_root, domain=get_domain( - state.fork_data, # `state` is the resulting state of `block` transition - state.slot, + fork_data, # `fork_data` is the fork_data at the slot `block.slot` + block.slot, DOMAIN_PROPOSAL, ) ) @@ -239,7 +223,7 @@ Up to `MAX_CASPER_SLASHINGS` [`CasperSlashing`](https://github.com/ethereum/eth2 ##### Attestations -Up to `MAX_ATTESTATIONS` aggregate attestations can be included in the `block`. The attestations added must satisfy the verification conditions found in [attestation processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestations-1). To maximize profit, the validator should attempt to add aggregate attestations that include the most available that have not previously been added on chain. +Up to `MAX_ATTESTATIONS` aggregate attestations can be included in the `block`. The attestations added must satisfy the verification conditions found in [attestation processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestations-1). To maximize profit, the validator should attempt to create aggregate attestations that include singular attestations from the largest number of validators whose signatures from the same epoch have not previously been added on chain. ##### Deposits From 1614f2a9d798b042a248ab7f8b19b781054230b8 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 24 Jan 2019 23:11:40 -0700 Subject: [PATCH 059/136] simplify slashing instructions in vlaidator guide --- specs/validator/0_beacon-chain-validator.md | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index f1e650884..29ec0c7e0 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -331,27 +331,16 @@ signed_attestation_data = bls_sign( "Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing](#proposal-slashing) and [attestation slashing](#casper-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. -_Note_: Signed data must be within the same `Fork` context to conflict. Messages cannot be slashed across forks. +_Note_: Signed data must be within a sequential `Fork` context to conflict. Messages cannot be slashed across diverging forks. If the previous fork version is 1 and the chain splits into fork 2 and 102, messages from 1 can slashable against messages in forks 1, 2, and 102. Messages in 2 cannot be slashable against messages in 102 and vice versa. ### Proposal slashing -To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposalsigneddata) (suggest renaming `ProposalData`) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. +To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposalsigneddata) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. In phase 0, proposals are only made for the beacon chain (`shard == BEACON_CHAIN_SHARD_NUMBER`). -The following helper can be run to check if two proposal messages conflict: - -```python -def proposal_data_is_slashable(proposal_data_1: ProposalSignedData, - proposal_data_2: ProposalSignedData) -> bool: - if (proposal_data_1.slot != proposal_data_2.slot): - return False - if (proposal_data_1.shard != proposal_data_2.shard): - return False - - return proposal_data_1.block_root != proposal_data_2.block_root -``` +In phase 0, as long as the validator does not sign two different beacon chain proposals for the same slot, the validator is safe against proposal slashings. Specifically, when signing an `BeaconBlock`, a validator should perform the following steps in the following order: -1. Save a record to hard disk that an beacon block has been signed for the `slot` and `shard`. +1. Save a record to hard disk that an beacon block has been signed for the `slot=slot` and `shard=BEACON_CHAIN_SHARD_NUMBER`. 2. Generate and broadcast the block. If the software crashes at some point within this routine, then when the validator comes back online the hard disk has the record of the _potentially_ signed/broadcast block and can effectively avoid slashing. From 0254bc8d177dc916ee902c1cfc1ed5ad7538f6dd Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Thu, 24 Jan 2019 23:17:56 -0700 Subject: [PATCH 060/136] pr feedback --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index daf0cae7e..0cc239866 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -912,7 +912,7 @@ def get_shuffling(seed: Bytes32, return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) ``` -**Invariant**: if `get_shuffling(seed, validators, slot)` returns some value `x` and `slot <= state.slot + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `slot` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. +**Invariant**: if `get_shuffling(seed, validators, slot)` returns some value `x` for some `slot <= state.slot + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `slot` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. From f27905583f0cdfc2a78c98ead5d03902bcddd597 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Fri, 25 Jan 2019 16:01:35 +0800 Subject: [PATCH 061/136] PR feedback --- specs/core/0_beacon-chain.md | 40 ++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 15 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 93e773976..382caeca7 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -746,26 +746,36 @@ Beacon block production is significantly different because of the proof of stake The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a [validator](#dfn-validator) `v` subjectively calculates the beacon chain head as follows. -* Let `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. +* Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. * Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) * Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `EPOCH_LENGTH` slots. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. -* Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. -* Let `get_latest_attestation(store, validator)` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. -* Let `get_latest_attestation_target(store, validator)` be the target block in the attestation `get_latest_attestation(store, validator)`. -* The head is `lmd_ghost(store, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. +* Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. +* Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. +* Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. +* Let `get_children(block: BeaconBlock) -> List[BeaconBlock]` returns the children blocks of the given `block`. +* Let `justified_head_state` be the `BeaconState` object after processing `justified_head`. +* The `head` is `lmd_ghost(store, justified_head_state, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. ```python -def lmd_ghost(store, start): - validators = start.state.validator_registry - active_validators = [validators[i] for i in - get_active_validator_indices(validators, start.state.slot)] - attestation_targets = [get_latest_attestation_target(store, validator) - for validator in active_validators] - def get_vote_count(block): - return len([target for target in attestation_targets if - get_ancestor(store, target, block.slot) == block]) +def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) -> BeaconBlock: + validators = start_state.validator_registry + active_validators = [ + validators[i] + for i in get_active_validator_indices(validators, start_state.slot) + ] + attestation_targets = [ + get_latest_attestation_target(store, validator) + for validator in active_validators + ] - head = start + def get_vote_count(block: BeaconBlock) -> int: + return len([ + target + for target in attestation_targets + if get_ancestor(store, target, block.slot) == block + ]) + + head = start_block while 1: children = get_children(head) if len(children) == 0: From 21cecba6bb24bdbb0e57cefab76c2b550527c89a Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Fri, 25 Jan 2019 16:06:05 +0800 Subject: [PATCH 062/136] Update `get_children` def --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 382caeca7..a4440e924 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -752,7 +752,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi * Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. * Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. * Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. -* Let `get_children(block: BeaconBlock) -> List[BeaconBlock]` returns the children blocks of the given `block`. +* Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the children blocks of the given `block`. * Let `justified_head_state` be the `BeaconState` object after processing `justified_head`. * The `head` is `lmd_ghost(store, justified_head_state, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. @@ -777,7 +777,7 @@ def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) head = start_block while 1: - children = get_children(head) + children = get_children(store, head) if len(children) == 0: return head head = max(children, key=get_vote_count) From f61d3643521b9ad06642c8a2026be22fe9f5735d Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 14:56:10 -0700 Subject: [PATCH 063/136] clean up top language in validator registry section --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4cecb4ddd..ddfb833f8 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1701,7 +1701,7 @@ def process_ejections(state: BeaconState) -> None: ### Validator registry and shuffling seed data -First, update `previous_epoch_calculation_slot`, `previous_epoch_start_shard` and `latest_index_roots`: +First, update the following: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` From f96cd871b1e9b48f54b23cab106937d5f9d66cc3 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 15:25:19 -0700 Subject: [PATCH 064/136] update ordering of assignments --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ddfb833f8..dc9fadd54 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1705,8 +1705,8 @@ First, update the following: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` -* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` * Set `state.previous_epoch_seed = state.current_epoch_seed` +* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` If the following are satisfied: From 86faacdcd2fdebda137b0ce661553d7e4286e4b4 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 15:27:27 -0700 Subject: [PATCH 065/136] clarify assignments in val reg not change --- specs/core/0_beacon-chain.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index dc9fadd54..b6dc26a7a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1768,7 +1768,10 @@ and perform the following updates: If a validator registry update does _not_ happen do the following: * Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. -* If `epochs_since_last_registry_change` is an exact power of 2, set `state.current_epoch_calculation_slot = state.slot` and `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))`. Note that `state.current_epoch_start_shard` is left unchanged. +* If `epochs_since_last_registry_change` is an exact power of 2: + * Set `state.current_epoch_calculation_slot = state.slot` + * Set `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))`. + * _Note_ that `state.current_epoch_start_shard` is left unchanged. **Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. From 56037726210fa6e79be3afe974c6284ad4eb18fe Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 15:28:08 -0700 Subject: [PATCH 066/136] add missing period --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index b6dc26a7a..d949af716 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1769,7 +1769,7 @@ If a validator registry update does _not_ happen do the following: * Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. * If `epochs_since_last_registry_change` is an exact power of 2: - * Set `state.current_epoch_calculation_slot = state.slot` + * Set `state.current_epoch_calculation_slot = state.slot`. * Set `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))`. * _Note_ that `state.current_epoch_start_shard` is left unchanged. From 85d39af1cad0e4869944be8bdcbfb75b7c3c64f6 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 15:28:49 -0700 Subject: [PATCH 067/136] add missing period --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index d949af716..8eaa2b5e4 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1815,7 +1815,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates * Let `epoch = state.slot // EPOCH_LENGTH`. -* Set `state.latest_penalized_balances[(epoch+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[epoch % LATEST_PENALIZED_EXIT_LENGTH]` +* Set `state.latest_penalized_balances[(epoch+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[epoch % LATEST_PENALIZED_EXIT_LENGTH]`. * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < state.slot - EPOCH_LENGTH`. ## State root processing From ddf420ba3981e16225d708600960eb2d7843f08b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 16:07:54 -0700 Subject: [PATCH 068/136] work through issues with epochs, seeds, etc) --- specs/core/0_beacon-chain.md | 46 ++++++++++++++++++++++++++++-------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8addc31a4..f924d2dc7 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -72,6 +72,7 @@ - [`get_block_root`](#get_block_root) - [`get_randao_mix`](#get_randao_mix) - [`get_active_index_root`](#get_active_index_root) + - [`generate_seed`](#generate_seed) - [`get_beacon_proposer_index`](#get_beacon_proposer_index) - [`merkle_root`](#merkle_root) - [`get_attestation_participants`](#get_attestation_participants) @@ -998,6 +999,27 @@ def get_active_index_root(state: BeaconState, return state.latest_index_roots[given_epoch % LATEST_INDEX_ROOTS_LENGTH] ``` +#### `generate_seed` + +```python +def generate_seed(state: BeaconState, + slot: int) -> Bytes32: + """ + Generate a seed for the given ``slot``. + """ + if slot < SEED_LOOKAHEAD: + randao_mix_slot = GENESIS_SLOT + else: + randao_mix_slot = slot - SEED_LOOKAHEAD + + return hash( + get_randao_mix(state, randao_mix_slot) + + get_active_index_root(state, slot) + ) + +``` + + #### `get_beacon_proposer_index` ```python @@ -1285,6 +1307,9 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) + state.latest_index_roots[GENESIS_SLOT // EPOCH_LENGTH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, GENESIS_SLOT)) + state.current_epoch_seed = generate_seed(state, GENESIS_SLOT) + return state ``` @@ -1568,6 +1593,8 @@ The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. * Let `next_epoch_start_slot = state.slot + 1`. * Let `current_epoch_start_slot = state.slot - (EPOCH_LENGTH + 1)`. * Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > EPOCH_LENGTH` else `current_epoch_start_slot`. +* Let `next_epoch = next_epoch_start_slot // EPOCH_LENGTH`. +* Let `current_epoch = current_epoch_start_slot // EPOCH_LENGTH`. All [validators](#dfn-validator): @@ -1576,7 +1603,7 @@ All [validators](#dfn-validator): [Validators](#dfn-Validator) attesting during the current epoch: -* Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch_start_slot <= a.data.slot <= state.slot]`. (Note: this is the set of attestations of slots in the epoch `current_epoch_start_slot...state.slot`, _not_ attestations that got included in the chain during the epoch `current_epoch_start_slot...state.slot`.) +* Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch_start_slot <= a.data.slot < next_epoch_start_slot]`. (Note: this is the set of attestations of slots in the epoch `current_epoch_start_slot...next_epoch_start_slot`, _not_ attestations that got included in the chain during the epoch `current_epoch_start_slot...next_epoch_start_slot`.) * Validators justifying the epoch boundary block at the start of the current epoch: * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, current_epoch_start_slot) and a.data.justified_slot == state.justified_slot]`. * Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_boundary_attestations]`. @@ -1719,7 +1746,7 @@ First, update the following: * Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` * Set `state.previous_epoch_seed = state.current_epoch_seed` -* Set `state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, state.slot))` +* Set `state.latest_index_roots[next_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch_start_slot))` If the following are satisfied: @@ -1769,21 +1796,21 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validator exit_validator(state, index) - state.validator_registry_update_slot = state.slot + state.validator_registry_update_slot = state.slot - (state.slot % EPOCH_LENGTH) ``` and perform the following updates: -* Set `state.current_epoch_calculation_slot = state.slot` +* Set `state.current_epoch_calculation_slot = next_epoch_start_slot` * Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH) % SHARD_COUNT` -* Set `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))` +* Set `state.current_epoch_seed = generate_seed(state, state.current_epoch_calculation_slot)` If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = (state.slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. +* Let `epochs_since_last_registry_change = (current_epoch_start_slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. * If `epochs_since_last_registry_change` is an exact power of 2: - * Set `state.current_epoch_calculation_slot = state.slot`. - * Set `state.current_epoch_seed = hash(get_randao_mix(state, state.current_epoch_calculation_slot - SEED_LOOKAHEAD) + get_active_index_root(state, state.current_epoch_calculation_slot))`. + * Set `state.current_epoch_calculation_slot = next_epoch_start_slot`. + * Set `state.current_epoch_seed = generate_seed(state, state.current_epoch_calculation_slot)` * _Note_ that `state.current_epoch_start_shard` is left unchanged. **Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. @@ -1827,8 +1854,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates -* Let `epoch = state.slot // EPOCH_LENGTH`. -* Set `state.latest_penalized_balances[(epoch+1) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[epoch % LATEST_PENALIZED_EXIT_LENGTH]`. +* Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch % LATEST_PENALIZED_EXIT_LENGTH]`. * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < current_epoch_start_slot`. ## State root processing From 6f830179c6b397823b0aa4776a64620b65e35f67 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 16:50:15 -0700 Subject: [PATCH 069/136] remove extra blank line --- specs/core/0_beacon-chain.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index f924d2dc7..ed0ecf444 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1019,7 +1019,6 @@ def generate_seed(state: BeaconState, ``` - #### `get_beacon_proposer_index` ```python From e34cfc8de8f9663a9431dcda3ca52c87bf190321 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 17:26:27 -0700 Subject: [PATCH 070/136] fix preivous_epoch_start_slot to be based upon GENESIS_SLOT --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ed0ecf444..20879dd41 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1591,7 +1591,7 @@ The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. * Let `next_epoch_start_slot = state.slot + 1`. * Let `current_epoch_start_slot = state.slot - (EPOCH_LENGTH + 1)`. -* Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > EPOCH_LENGTH` else `current_epoch_start_slot`. +* Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > GENESIS_SLOT + EPOCH_LENGTH` else `current_epoch_start_slot`. * Let `next_epoch = next_epoch_start_slot // EPOCH_LENGTH`. * Let `current_epoch = current_epoch_start_slot // EPOCH_LENGTH`. From a212038cd3c37a05fc99fb6da48a2c10678993b1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 17:33:15 -0700 Subject: [PATCH 071/136] begin translation to epochs --- specs/core/0_beacon-chain.md | 200 ++++++++++++++++++----------------- 1 file changed, 103 insertions(+), 97 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index ed0ecf444..0f283103b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -60,6 +60,7 @@ - [Helper functions](#helper-functions) - [`hash`](#hash) - [`hash_tree_root`](#hash_tree_root) + - [`slot_to_epoch`](#slot_to_epoch) - [`is_active_validator`](#is_active_validator) - [`get_active_validator_indices`](#get_active_validator_indices) - [`shuffle`](#shuffle) @@ -84,7 +85,7 @@ - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) - - [`entry_exit_effect_slot`](#entry_exit_effect_slot) + - [`entry_exit_effect_epoch`](#entry_exit_effect_epoch) - [`bls_verify`](#bls_verify) - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) @@ -191,7 +192,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `GENESIS_FORK_VERSION` | `0` | | `GENESIS_SLOT` | `0` | | `GENESIS_START_SHARD` | `0` | -| `FAR_FUTURE_SLOT` | `2**64 - 1` | +| `FAR_FUTURE_EPOCH` | `2**64 - 1` | | `ZERO_HASH` | `int_to_bytes32(0)` | | `EMPTY_SIGNATURE` | `int_to_bytes96(0)` | | `BLS_WITHDRAWAL_PREFIX_BYTE` | `int_to_bytes1(0)` | @@ -203,10 +204,10 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `SLOT_DURATION` | `6` | seconds | 6 seconds | | `MIN_ATTESTATION_INCLUSION_DELAY` | `2**2` (= 4) | slots | 24 seconds | | `EPOCH_LENGTH` | `2**6` (= 64) | slots | 6.4 minutes | -| `SEED_LOOKAHEAD` | `2**6` (= 64) | slots | 6.4 minutes | -| `ENTRY_EXIT_DELAY` | `2**8` (= 256) | slots | 25.6 minutes | +| `SEED_LOOKAHEAD` | `2**0` (= 1) | epochs | 6.4 minutes | +| `ENTRY_EXIT_DELAY` | `2**2` (= 4) | epochs | 25.6 minutes | | `ETH1_DATA_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours | -| `MIN_VALIDATOR_WITHDRAWAL_TIME` | `2**14` (= 16,384) | slots | ~27 hours | +| `MIN_VALIDATOR_WITHDRAWAL_EPOCHS` | `2**8` (= 256) | epochs | ~27 hours | ### Reward and penalty quotients @@ -468,7 +469,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be # Validator registry 'validator_registry': [Validator], 'validator_balances': ['uint64'], - 'validator_registry_update_slot': 'uint64', + 'validator_registry_update_epoch': 'uint64', 'validator_registry_exit_count': 'uint64', # Randomness and committees @@ -476,8 +477,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'latest_vdf_outputs': ['bytes32'], 'previous_epoch_start_shard': 'uint64', 'current_epoch_start_shard': 'uint64', - 'previous_epoch_calculation_slot': 'uint64', - 'current_epoch_calculation_slot': 'uint64', + 'previous_calculation_epoch': 'uint64', + 'current_calculation_epoch': 'uint64', 'previous_epoch_seed': 'bytes32', 'current_epoch_seed': 'bytes32', @@ -514,14 +515,14 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'withdrawal_credentials': 'bytes32', # Number of proposer slots since genesis 'proposer_slots': 'uint64', - # Slot when validator activated - 'activation_slot': 'uint64', - # Slot when validator exited - 'exit_slot': 'uint64', - # Slot when validator withdrew - 'withdrawal_slot': 'uint64', - # Slot when validator was penalized - 'penalized_slot': 'uint64', + # Epoch when validator activated + 'activation_epoch': 'uint64', + # Epoch when validator exited + 'exit_epoch': 'uint64', + # Epoch when validator withdrew + 'withdrawal_epoch': 'uint64', + # Epoch when validator was penalized + 'penalized_epoch': 'uint64', # Exit counter when validator exited 'exit_count': 'uint64', # Status flags @@ -762,13 +763,20 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere `hash_tree_root` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). +#### `slot_to_epoch` + +```python +def slot_to_epoch(slot: int) -> int: + return slot // EPOCH_LENGTH +``` + #### `is_active_validator` ```python def is_active_validator(validator: Validator, slot: int) -> bool: """ Checks if ``validator`` is active. """ - return validator.activation_slot <= slot < validator.exit_slot + return validator.activation_epoch <= slot_to_epoch(slot) < validator.exit_epoch ``` #### `get_active_validator_indices` @@ -866,29 +874,26 @@ def get_committee_count_per_slot(active_validator_count: int) -> int: ```python def get_shuffling(seed: Bytes32, validators: List[Validator], - slot: int) -> List[List[int]] + epoch: int) -> List[List[int]] """ - Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``slot``. + Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. Returns a list of ``EPOCH_LENGTH * committees_per_slot`` committees where each committee is itself a list of validator indices. """ - # Normalizes slot to start of epoch boundary - slot -= slot % EPOCH_LENGTH - active_validator_indices = get_active_validator_indices(validators, slot) committees_per_slot = get_committee_count_per_slot(len(active_validator_indices)) # Shuffle - seed = xor(seed, int_to_bytes32(slot)) + seed = xor(seed, int_to_bytes32(epoch)) shuffled_active_validator_indices = shuffle(active_validator_indices, seed) # Split the shuffled list into epoch_length * committees_per_slot pieces return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) ``` -**Invariant**: if `get_shuffling(seed, validators, slot)` returns some value `x` for some `slot <= state.slot + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `slot` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. +**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= slot_to_epoch(state.slot) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. @@ -898,7 +903,7 @@ def get_shuffling(seed: Bytes32, def get_previous_epoch_committee_count_per_slot(state: BeaconState) -> int: previous_active_validators = get_active_validator_indices( state.validator_registry, - state.previous_epoch_calculation_slot, + state.previous_calculation_epoch, ) return get_committee_count_per_slot(len(previous_active_validators)) ``` @@ -909,7 +914,7 @@ def get_previous_epoch_committee_count_per_slot(state: BeaconState) -> int: def get_current_epoch_committee_count_per_slot(state: BeaconState) -> int: current_active_validators = get_active_validator_indices( state.validator_registry, - state.current_epoch_calculation_slot, + state.current_calculation_epoch, ) return get_committee_count_per_slot(len(current_active_validators)) ``` @@ -929,18 +934,18 @@ def get_crosslink_committees_at_slot(state: BeaconState, if slot < state_epoch_slot: committees_per_slot = get_previous_epoch_committee_count_per_slot(state) seed = state.previous_epoch_seed - shuffling_slot = state.previous_epoch_calculation_slot + shuffling_epoch = state.previous_calculation_epoch shuffling_start_shard = state.previous_epoch_start_shard else: committees_per_slot = get_current_epoch_committee_count_per_slot(state) seed = state.current_epoch_seed - shuffling_slot = state.current_epoch_calculation_slot + shuffling_epoch = state.current_calculation_epoch shuffling_start_shard = state.current_epoch_start_shard shuffling = get_shuffling( seed, state.validator_registry, - shuffling_slot, + shuffling_epoch, ) offset = slot % EPOCH_LENGTH slot_start_shard = (shuffling_start_shard + committees_per_slot * offset) % SHARD_COUNT @@ -988,33 +993,32 @@ def get_randao_mix(state: BeaconState, ```python def get_active_index_root(state: BeaconState, - slot: int) -> Bytes32: + epoch: int) -> Bytes32: """ Returns the index root at a recent ``slot``. """ - state_epoch = state.slot // EPOCH_LENGTH - given_epoch = slot // EPOCH_LENGTH - assert state_epoch < given_epoch + LATEST_INDEX_ROOTS_LENGTH - assert given_epoch <= state_epoch - return state.latest_index_roots[given_epoch % LATEST_INDEX_ROOTS_LENGTH] + state_epoch = slot_to_epoch(state.slot) + assert state_epoch < epoch + LATEST_INDEX_ROOTS_LENGTH + assert epoch <= state_epoch + return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] ``` #### `generate_seed` ```python def generate_seed(state: BeaconState, - slot: int) -> Bytes32: + epoch: int) -> Bytes32: """ - Generate a seed for the given ``slot``. + Generate a seed for the given ``epoch``. """ - if slot < SEED_LOOKAHEAD: - randao_mix_slot = GENESIS_SLOT + if epoch < SEED_LOOKAHEAD: + randao_mix_epoch = slot_to_epoch(GENESIS_SLOT) else: - randao_mix_slot = slot - SEED_LOOKAHEAD + randao_mix_epoch = epoch - SEED_LOOKAHEAD return hash( - get_randao_mix(state, randao_mix_slot) + - get_active_index_root(state, slot) + get_randao_mix(state, randao_mix_epoch * EPOCH_LENGTH) + + get_active_index_root(state, epoch) ) ``` @@ -1142,8 +1146,8 @@ def is_double_vote(attestation_data_1: AttestationData, Returns True if the provided ``AttestationData`` are slashable due to a 'double vote'. """ - target_epoch_1 = attestation_data_1.slot // EPOCH_LENGTH - target_epoch_2 = attestation_data_2.slot // EPOCH_LENGTH + target_epoch_1 = slot_to_epoch(attestation_data_1.slot) + target_epoch_2 = slot_to_epoch(attestation_data_2.slot) return target_epoch_1 == target_epoch_2 ``` @@ -1159,10 +1163,10 @@ def is_surround_vote(attestation_data_1: AttestationData, Note: parameter order matters as this function only checks that ``attestation_data_1`` surrounds ``attestation_data_2``. """ - source_epoch_1 = attestation_data_1.justified_slot // EPOCH_LENGTH - source_epoch_2 = attestation_data_2.justified_slot // EPOCH_LENGTH - target_epoch_1 = attestation_data_1.slot // EPOCH_LENGTH - target_epoch_2 = attestation_data_2.slot // EPOCH_LENGTH + source_epoch_1 = slot_to_epoch(attestation_data_1.justified_slot) + source_epoch_2 = slot_to_epoch(attestation_data_2.justified_slot) + target_epoch_1 = slot_to_epoch(attestation_data_1.slot) + target_epoch_2 = slot_to_epoch(attestation_data_2.slot) return ( (source_epoch_1 < source_epoch_2) and (source_epoch_2 + 1 == target_epoch_2) and @@ -1186,15 +1190,15 @@ def integer_squareroot(n: int) -> int: return x ``` -#### `entry_exit_effect_slot` +#### `entry_exit_effect_epoch` ```python -def entry_exit_effect_slot(n: int) -> int: +def entry_exit_effect_epoch(slot: int) -> int: """ An entry or exit triggered in the slot given by the input takes effect at - the slot given by the output. + the epoch given by the output. """ - return (n - n % EPOCH_LENGTH) + EPOCH_LENGTH + ENTRY_EXIT_DELAY + return slot_to_epoch(slot) + 1 + ENTRY_EXIT_DELAY ``` #### `bls_verify` @@ -1256,7 +1260,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Validator registry validator_registry=[], validator_balances=[], - validator_registry_update_slot=GENESIS_SLOT, + validator_registry_update_epoch=slot_to_epoch(GENESIS_SLOT), validator_registry_exit_count=0, # Randomness and committees @@ -1264,8 +1268,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], latest_vdf_outputs=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH // EPOCH_LENGTH)], previous_epoch_start_shard=GENESIS_START_SHARD, current_epoch_start_shard=GENESIS_START_SHARD, - previous_epoch_calculation_slot=GENESIS_SLOT, - current_epoch_calculation_slot=GENESIS_SLOT, + previous_calculation_epoch=slot_to_epoch(GENESIS_SLOT), + current_calculation_epoch=slot_to_epoch(GENESIS_SLOT), previous_epoch_seed=ZERO_HASH, current_epoch_seed=ZERO_HASH, @@ -1306,8 +1310,9 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) - state.latest_index_roots[GENESIS_SLOT // EPOCH_LENGTH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, GENESIS_SLOT)) - state.current_epoch_seed = generate_seed(state, GENESIS_SLOT) + genesis_epoch = slot_to_epoch(GENESIS_SLOT) + state.latest_index_roots[genesis_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, genesis_epoch)) + state.current_epoch_seed = generate_seed(state, genesis_epoch) return state ``` @@ -1367,10 +1372,10 @@ def process_deposit(state: BeaconState, pubkey=pubkey, withdrawal_credentials=withdrawal_credentials, proposer_slots=0, - activation_slot=FAR_FUTURE_SLOT, - exit_slot=FAR_FUTURE_SLOT, - withdrawal_slot=FAR_FUTURE_SLOT, - penalized_slot=FAR_FUTURE_SLOT, + activation_epoch=FAR_FUTURE_EPOCH, + exit_epch=FAR_FUTURE_EPOCH, + withdrawal_epoch=FAR_FUTURE_EPOCH, + penalized_epoch=FAR_FUTURE_EPOCH, exit_count=0, status_flags=0, latest_custody_reseed_slot=GENESIS_SLOT, @@ -1396,7 +1401,7 @@ Note: All functions in this section mutate `state`. def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_slot = GENESIS_SLOT if genesis else entry_exit_effect_slot(state.slot) + validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else entry_exit_effect_epoch(state.slot) ``` ```python @@ -1410,10 +1415,10 @@ def exit_validator(state: BeaconState, index: int) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited - if validator.exit_slot <= entry_exit_effect_slot(state.slot): + if validator.exit_epoch <= entry_exit_effect_epoch(state.slot): return - validator.exit_slot = entry_exit_effect_slot(state.slot) + validator.exit_epoch = entry_exit_effect_epoch(state.slot) state.validator_registry_exit_count += 1 validator.exit_count = state.validator_registry_exit_count @@ -1423,13 +1428,13 @@ def exit_validator(state: BeaconState, index: int) -> None: def penalize_validator(state: BeaconState, index: int) -> None: exit_validator(state, index) validator = state.validator_registry[index] - state.latest_penalized_balances[(state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) + state.latest_penalized_balances[slot_to_epoch(state.slot) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) whistleblower_index = get_beacon_proposer_index(state, state.slot) whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT state.validator_balances[whistleblower_index] += whistleblower_reward state.validator_balances[index] -= whistleblower_reward - validator.penalized_slot = state.slot + validator.penalized_epoch = slot_to_epoch(state.slot) ``` ```python @@ -1573,7 +1578,7 @@ Verify that `len(block.body.exits) <= MAX_EXITS`. For each `exit` in `block.body.exits`: * Let `validator = state.validator_registry[exit.validator_index]`. -* Verify that `validator.exit_slot > entry_exit_effect_slot(state.slot)`. +* Verify that `validator.exit_epoch > entry_exit_effect_epoch(state.slot)`. * Verify that `state.slot >= exit.slot`. * Let `exit_message = hash_tree_root(Exit(slot=exit.slot, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. @@ -1589,15 +1594,16 @@ The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. ### Helpers -* Let `next_epoch_start_slot = state.slot + 1`. -* Let `current_epoch_start_slot = state.slot - (EPOCH_LENGTH + 1)`. -* Let `previous_epoch_start_slot = state.slot - 2 * EPOCH_LENGTH + 1` if `state.slot > EPOCH_LENGTH` else `current_epoch_start_slot`. -* Let `next_epoch = next_epoch_start_slot // EPOCH_LENGTH`. -* Let `current_epoch = current_epoch_start_slot // EPOCH_LENGTH`. +* Let `current_epoch = slot_to_epoch(state.slot)`. +* Let `current_epoch_start_slot = current_epoch * EPOCH_LENGTH`. +* Let `next_epoch = slot_to_epoch(state.slot + 1)`. +* Let `next_epoch_start_slot = next_epoch * EPOCH_LENGTH`. +* Let `previous_epoch = current_epoch - 1 if current_epoch > slot_to_epoch(GENESIS_SLOT) else current_epoch`. +* Let `previous_epoch_start_slot = previous_epoch * EPOCH_LENGTH`. All [validators](#dfn-validator): -* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch_start_slot)`. +* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch)`. * Let `total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices])`. [Validators](#dfn-Validator) attesting during the current epoch: @@ -1688,7 +1694,7 @@ First, we define some additional helpers: Note: When applying penalties in the following balance recalculations implementers should make sure the `uint64` does not underflow. -* Let `epochs_since_finality = (next_epoch_start_slot - state.finalized_slot) // EPOCH_LENGTH`. +* Let `epochs_since_finality = next_epoch - slot_to_epoch(state.finalized_slot)`. Case 1: `epochs_since_finality <= 4`: @@ -1709,7 +1715,7 @@ Case 2: `epochs_since_finality > 4`: * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_justified_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_head_attester_indices`, loses `base_reward(state, index)`. -* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_slot <= current_epoch_start_slot`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. +* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` loses `base_reward(state, index) - base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` #### Attestation inclusion @@ -1733,7 +1739,7 @@ def process_ejections(state: BeaconState) -> None: Iterate through the validator registry and eject active validators with balance below ``EJECTION_BALANCE``. """ - for index in get_active_validator_indices(state.validator_registry, current_epoch_start_slot): + for index in get_active_validator_indices(state.validator_registry, current_epoch): if state.validator_balances[index] < EJECTION_BALANCE: exit_validator(state, index) ``` @@ -1742,15 +1748,15 @@ def process_ejections(state: BeaconState) -> None: First, update the following: -* Set `state.previous_epoch_calculation_slot = state.current_epoch_calculation_slot` -* Set `state.previous_epoch_start_shard = state.current_epoch_start_shard` -* Set `state.previous_epoch_seed = state.current_epoch_seed` -* Set `state.latest_index_roots[next_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch_start_slot))` +* Set `state.previous_calculation_epoch = state.current_calculation_epoch`. +* Set `state.previous_epoch_start_shard = state.current_epoch_start_shard`. +* Set `state.previous_epoch_seed = state.current_epoch_seed`. +* Set `state.latest_index_roots[next_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch_start_slot))`. If the following are satisfied: -* `state.finalized_slot > state.validator_registry_update_slot` -* `state.latest_crosslinks[shard].slot > state.validator_registry_update_slot` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) +* `slot_to_epoch(state.finalized_slot) > state.validator_registry_update_epoch` +* `slot_to_epoch(state.latest_crosslinks[shard].slot) > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) update the validator registry and associated fields by running @@ -1761,7 +1767,7 @@ def update_validator_registry(state: BeaconState) -> None: Note that this function mutates ``state``. """ # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch_start_slot) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) @@ -1774,7 +1780,7 @@ def update_validator_registry(state: BeaconState) -> None: # Activate validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.activation_slot > entry_exit_effect_slot(state.slot) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: + if validator.activation_epoch > entry_exit_effect_epoch(state.slot) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1786,7 +1792,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.exit_slot > entry_exit_effect_slot(state.slot) and validator.status_flags & INITIATED_EXIT: + if validator.exit_epoch > entry_exit_effect_epoch(state.slot) and validator.status_flags & INITIATED_EXIT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1795,21 +1801,21 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validator exit_validator(state, index) - state.validator_registry_update_slot = state.slot - (state.slot % EPOCH_LENGTH) + state.validator_registry_update_epoch = slot_to_epoch(state.slot) ``` and perform the following updates: -* Set `state.current_epoch_calculation_slot = next_epoch_start_slot` +* Set `state.current_calculation_epoch = next_epoch` * Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH) % SHARD_COUNT` -* Set `state.current_epoch_seed = generate_seed(state, state.current_epoch_calculation_slot)` +* Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = (current_epoch_start_slot - state.validator_registry_update_slot) // EPOCH_LENGTH`. +* Let `epochs_since_last_registry_change = (current_epoch - state.validator_registry_update_epoch)`. * If `epochs_since_last_registry_change` is an exact power of 2: - * Set `state.current_epoch_calculation_slot = next_epoch_start_slot`. - * Set `state.current_epoch_seed = generate_seed(state, state.current_epoch_calculation_slot)` + * Set `state.current_calculation_epoch = next_epoch`. + * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` * _Note_ that `state.current_epoch_start_shard` is left unchanged. **Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. @@ -1824,8 +1830,8 @@ def process_penalties_and_exits(state: BeaconState) -> None: total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) for index, validator in enumerate(state.validator_registry): - if (state.slot // EPOCH_LENGTH) == (validator.penalized_slot // EPOCH_LENGTH) + LATEST_PENALIZED_EXIT_LENGTH // 2: - e = (state.slot // EPOCH_LENGTH) % LATEST_PENALIZED_EXIT_LENGTH + if slot_to_epoch(state.slot) == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: + e = slot_to_epoch(state.slot) % LATEST_PENALIZED_EXIT_LENGTH total_at_start = state.latest_penalized_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH] total_at_end = state.latest_penalized_balances[e] total_penalties = total_at_end - total_at_start @@ -1834,11 +1840,11 @@ def process_penalties_and_exits(state: BeaconState) -> None: def eligible(index): validator = state.validator_registry[index] - if validator.penalized_slot <= state.slot: - PENALIZED_WITHDRAWAL_TIME = LATEST_PENALIZED_EXIT_LENGTH * EPOCH_LENGTH // 2 - return state.slot >= validator.penalized_slot + PENALIZED_WITHDRAWAL_TIME + if validator.penalized_epoch <= slot_to_epoch(state.slot): + PENALIZED_WITHDRAWAL_EPOCHS = LATEST_PENALIZED_EXIT_LENGTH // 2 + return slot_to_epoch(state.slot) >= validator.penalized_epoch + PENALIZED_WITHDRAWAL_EPOCHS else: - return state.slot >= validator.exit_slot + MIN_VALIDATOR_WITHDRAWAL_TIME + return slot_to_epoch(state.slot) >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS all_indices = list(range(len(state.validator_registry))) eligible_indices = filter(eligible, all_indices) From f943361a23883b387cd52c00bea24d7605fccc5b Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Fri, 25 Jan 2019 17:40:44 -0700 Subject: [PATCH 072/136] convert voting period to epochs --- specs/core/0_beacon-chain.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 0f283103b..372191e8f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -206,7 +206,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `EPOCH_LENGTH` | `2**6` (= 64) | slots | 6.4 minutes | | `SEED_LOOKAHEAD` | `2**0` (= 1) | epochs | 6.4 minutes | | `ENTRY_EXIT_DELAY` | `2**2` (= 4) | epochs | 25.6 minutes | -| `ETH1_DATA_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours | +| `ETH1_DATA_VOTING_PERIOD` | `2**4` (= 16) | epochs | ~1.7 hours | | `MIN_VALIDATOR_WITHDRAWAL_EPOCHS` | `2**8` (= 256) | epochs | ~27 hours | ### Reward and penalty quotients @@ -1590,7 +1590,7 @@ For each `exit` in `block.body.exits`: ## Per-epoch processing -The steps below happen when `state.slot % EPOCH_LENGTH == EPOCH_LENGTH - 1`. +The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. ### Helpers @@ -1650,9 +1650,9 @@ Define the following helpers to process attestation inclusion rewards and inclus ### Eth1 data -If `state.slot % ETH1_DATA_VOTING_PERIOD == ETH1_DATA_VOTING_PERIOD - 1`: +If `slot_to_epoch(state.slot) % ETH1_DATA_VOTING_PERIOD == 0`: -* Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD` for some `eth1_data_vote` in `state.eth1_data_votes`. +* Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. ### Justification @@ -1812,7 +1812,7 @@ and perform the following updates: If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = (current_epoch - state.validator_registry_update_epoch)`. +* Let `epochs_since_last_registry_change = current_epoch - state.validator_registry_update_epoch`. * If `epochs_since_last_registry_change` is an exact power of 2: * Set `state.current_calculation_epoch = next_epoch`. * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` From 235425aaf0088e8d3ffed687a0ec8215bc493a8d Mon Sep 17 00:00:00 2001 From: Paul Hauner Date: Sat, 26 Jan 2019 14:58:48 +1100 Subject: [PATCH 073/136] Fix field name error on eth1_data `eth1_data_vote.data` > `eth1_data_vote.eth1_data` --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 0c0869b6f..d3ee1b9d5 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1615,7 +1615,7 @@ Define the following helpers to process attestation inclusion rewards and inclus If `state.slot % ETH1_DATA_VOTING_PERIOD == 0`: -* Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD` for some `eth1_data_vote` in `state.eth1_data_votes`. +* Set `state.latest_eth1_data = eth1_data_vote.eth1_data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. ### Justification From feaf689c9401434b97cd2f68b22baaae74368861 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 16:02:49 +0800 Subject: [PATCH 074/136] Apply suggestions from code review Co-Authored-By: hwwhww --- specs/core/0_beacon-chain.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index a4440e924..ea2d6eeb0 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -622,11 +622,11 @@ We define the following Python custom types for type hinting and readability: | - | - | - | | `SlotNumber` | unsigned 64-bit integer | the number of a slot | | `ShardNumber` | unsigned 64-bit integer | the number of a shard | -| `ValidatorIndex` | unsigned 24-bit integer | the index number of validator in the registry | -| `Gwei` | unsigned 64-bit integer | the amount in Gwei | -| `Bytes32` | 32-byte data | the binary data in 32-byte length | -| `BLSPubkey` | 48-byte data | the public key in BLS signature scheme | -| `BLSSignature` | 96-byte data | the signature in BLS signature scheme | +| `ValidatorIndex` | unsigned 24-bit integer | the index number of a validator in the registry | +| `Gwei` | unsigned 64-bit integer | an amount in Gwei | +| `Bytes32` | 32-byte data | binary data with 32-byte length | +| `BLSPubkey` | 48-byte data | a public key in BLS signature scheme | +| `BLSSignature` | 96-byte data | a signature in BLS signature scheme | ## Ethereum 1.0 deposit contract @@ -746,14 +746,14 @@ Beacon block production is significantly different because of the proof of stake The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a [validator](#dfn-validator) `v` subjectively calculates the beacon chain head as follows. -* Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. +* Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not yet included in any chain are still included in `store`. * Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) * Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `EPOCH_LENGTH` slots. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. * Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. * Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. * Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. -* Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the children blocks of the given `block`. -* Let `justified_head_state` be the `BeaconState` object after processing `justified_head`. +* Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the child blocks of the given `block`. +* Let `justified_head_state` be the resulting `BeaconState` object from processing the chain up to the `justified_head`. * The `head` is `lmd_ghost(store, justified_head_state, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. ```python From 8c91be9e74c3ed5cc19695deb891cd72c08c0f8f Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Sat, 26 Jan 2019 16:07:15 +0800 Subject: [PATCH 075/136] Add custom type hinting for `get_active_index_root` --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 5013b9c95..af121905f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -116,7 +116,7 @@ - [Attestation inclusion](#attestation-inclusion) - [Crosslinks](#crosslinks-1) - [Ejections](#ejections) - - [Validator registry](#validator-registry) + - [Validator registry and shuffling seed data](#validator-registry-and-shuffling-seed-data) - [Final updates](#final-updates) - [State root processing](#state-root-processing) - [References](#references) @@ -1013,7 +1013,7 @@ def get_randao_mix(state: BeaconState, ```python def get_active_index_root(state: BeaconState, - slot: int) -> Bytes32: + slot: SlotNumber) -> Bytes32: """ Returns the index root at a recent ``slot``. """ From 0b827a04476f0fef05276aebb153e6ec9a51765a Mon Sep 17 00:00:00 2001 From: Paul Hauner Date: Sat, 26 Jan 2019 21:38:27 +1100 Subject: [PATCH 076/136] Change `PENALIZED_WITHDRAWAL_TIME` variable It's not in the "Constants" list and it is assigned to. --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index af121905f..c7bef5c25 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1823,8 +1823,8 @@ def process_penalties_and_exits(state: BeaconState) -> None: def eligible(index): validator = state.validator_registry[index] if validator.penalized_slot <= state.slot: - PENALIZED_WITHDRAWAL_TIME = LATEST_PENALIZED_EXIT_LENGTH * EPOCH_LENGTH // 2 - return state.slot >= validator.penalized_slot + PENALIZED_WITHDRAWAL_TIME + penalized_withdrawal_time = LATEST_PENALIZED_EXIT_LENGTH * EPOCH_LENGTH // 2 + return state.slot >= validator.penalized_slot + penalized_withdrawal_time else: return state.slot >= validator.exit_slot + MIN_VALIDATOR_WITHDRAWAL_TIME From 0e90dd8ad021735669c219281b94fb8e39ac0e06 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 07:31:09 -0700 Subject: [PATCH 077/136] add current_epoch helper --- specs/core/0_beacon-chain.md | 53 ++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 372191e8f..49b4d0077 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -61,6 +61,7 @@ - [`hash`](#hash) - [`hash_tree_root`](#hash_tree_root) - [`slot_to_epoch`](#slot_to_epoch) + - [`current_epoch`](#current_epoch) - [`is_active_validator`](#is_active_validator) - [`get_active_validator_indices`](#get_active_validator_indices) - [`shuffle`](#shuffle) @@ -770,6 +771,14 @@ def slot_to_epoch(slot: int) -> int: return slot // EPOCH_LENGTH ``` +#### `current_epoch` + +```python +def current_epoch(state: BeaconState) -> int: + return slot_to_epoch(state.slot) +``` + + #### `is_active_validator` ```python def is_active_validator(validator: Validator, slot: int) -> bool: @@ -893,7 +902,7 @@ def get_shuffling(seed: Bytes32, return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) ``` -**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= slot_to_epoch(state.slot) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. +**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. @@ -997,9 +1006,8 @@ def get_active_index_root(state: BeaconState, """ Returns the index root at a recent ``slot``. """ - state_epoch = slot_to_epoch(state.slot) - assert state_epoch < epoch + LATEST_INDEX_ROOTS_LENGTH - assert epoch <= state_epoch + assert current_epoch(state) < epoch + LATEST_INDEX_ROOTS_LENGTH + assert epoch <= current_epoch(state) return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] ``` @@ -1428,13 +1436,13 @@ def exit_validator(state: BeaconState, index: int) -> None: def penalize_validator(state: BeaconState, index: int) -> None: exit_validator(state, index) validator = state.validator_registry[index] - state.latest_penalized_balances[slot_to_epoch(state.slot) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) + state.latest_penalized_balances[current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) whistleblower_index = get_beacon_proposer_index(state, state.slot) whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT state.validator_balances[whistleblower_index] += whistleblower_reward state.validator_balances[index] -= whistleblower_reward - validator.penalized_epoch = slot_to_epoch(state.slot) + validator.penalized_epoch = current_epoch(state) ``` ```python @@ -1594,16 +1602,15 @@ The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. ### Helpers -* Let `current_epoch = slot_to_epoch(state.slot)`. -* Let `current_epoch_start_slot = current_epoch * EPOCH_LENGTH`. -* Let `next_epoch = slot_to_epoch(state.slot + 1)`. +* Let `current_epoch_start_slot = current_epoch(state) * EPOCH_LENGTH`. +* Let `next_epoch = current_epoch(state) + 1`. * Let `next_epoch_start_slot = next_epoch * EPOCH_LENGTH`. -* Let `previous_epoch = current_epoch - 1 if current_epoch > slot_to_epoch(GENESIS_SLOT) else current_epoch`. +* Let `previous_epoch = current_epoch(state) - 1 if current_epoch(state) > slot_to_epoch(GENESIS_SLOT) else current_epoch(state)`. * Let `previous_epoch_start_slot = previous_epoch * EPOCH_LENGTH`. All [validators](#dfn-validator): -* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch)`. +* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state))`. * Let `total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices])`. [Validators](#dfn-Validator) attesting during the current epoch: @@ -1650,7 +1657,7 @@ Define the following helpers to process attestation inclusion rewards and inclus ### Eth1 data -If `slot_to_epoch(state.slot) % ETH1_DATA_VOTING_PERIOD == 0`: +If `current_epoch(state) % ETH1_DATA_VOTING_PERIOD == 0`: * Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. @@ -1715,7 +1722,7 @@ Case 2: `epochs_since_finality > 4`: * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_justified_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_head_attester_indices`, loses `base_reward(state, index)`. -* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. +* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch(state)`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` loses `base_reward(state, index) - base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` #### Attestation inclusion @@ -1739,7 +1746,7 @@ def process_ejections(state: BeaconState) -> None: Iterate through the validator registry and eject active validators with balance below ``EJECTION_BALANCE``. """ - for index in get_active_validator_indices(state.validator_registry, current_epoch): + for index in get_active_validator_indices(state.validator_registry, current_epoch(state)): if state.validator_balances[index] < EJECTION_BALANCE: exit_validator(state, index) ``` @@ -1767,7 +1774,7 @@ def update_validator_registry(state: BeaconState) -> None: Note that this function mutates ``state``. """ # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state)) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) @@ -1801,7 +1808,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validator exit_validator(state, index) - state.validator_registry_update_epoch = slot_to_epoch(state.slot) + state.validator_registry_update_epoch = current_epoch(state) ``` and perform the following updates: @@ -1812,7 +1819,7 @@ and perform the following updates: If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = current_epoch - state.validator_registry_update_epoch`. +* Let `epochs_since_last_registry_change = current_epoch(state) - state.validator_registry_update_epoch`. * If `epochs_since_last_registry_change` is an exact power of 2: * Set `state.current_calculation_epoch = next_epoch`. * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` @@ -1830,8 +1837,8 @@ def process_penalties_and_exits(state: BeaconState) -> None: total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) for index, validator in enumerate(state.validator_registry): - if slot_to_epoch(state.slot) == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: - e = slot_to_epoch(state.slot) % LATEST_PENALIZED_EXIT_LENGTH + if current_epoch(state) == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: + e = current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH total_at_start = state.latest_penalized_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH] total_at_end = state.latest_penalized_balances[e] total_penalties = total_at_end - total_at_start @@ -1840,11 +1847,11 @@ def process_penalties_and_exits(state: BeaconState) -> None: def eligible(index): validator = state.validator_registry[index] - if validator.penalized_epoch <= slot_to_epoch(state.slot): + if validator.penalized_epoch <= current_epoch(state): PENALIZED_WITHDRAWAL_EPOCHS = LATEST_PENALIZED_EXIT_LENGTH // 2 - return slot_to_epoch(state.slot) >= validator.penalized_epoch + PENALIZED_WITHDRAWAL_EPOCHS + return current_epoch(state) >= validator.penalized_epoch + PENALIZED_WITHDRAWAL_EPOCHS else: - return slot_to_epoch(state.slot) >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS + return current_epoch(state) >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS all_indices = list(range(len(state.validator_registry))) eligible_indices = filter(eligible, all_indices) @@ -1859,7 +1866,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates -* Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch % LATEST_PENALIZED_EXIT_LENGTH]`. +* Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH]`. * Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < current_epoch_start_slot`. ## State root processing From c9494dbf8813f65b0d138f9b173aea2959f4a051 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 07:55:32 -0700 Subject: [PATCH 078/136] change crosslink.slot to crosslink.epoch --- specs/core/0_beacon-chain.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 49b4d0077..76f69f274 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -539,8 +539,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { - # Slot number - 'slot': 'uint64', + # Epoch number + 'epoch': 'uint64', # Shard block root 'shard_block_root': 'bytes32', } @@ -1291,7 +1291,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], finalized_slot=GENESIS_SLOT, # Recent state - latest_crosslinks=[Crosslink(slot=GENESIS_SLOT, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], + latest_crosslinks=[Crosslink(epoch=slot_to_epoch(GENESIS_SLOT), shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)], latest_index_roots=[ZERO_HASH for _ in range(LATEST_INDEX_ROOTS_LENGTH)], latest_penalized_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)], @@ -1318,7 +1318,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) - genesis_epoch = slot_to_epoch(GENESIS_SLOT) + genesis_epoch = current_epoch(state) state.latest_index_roots[genesis_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, genesis_epoch)) state.current_epoch_seed = generate_seed(state, genesis_epoch) @@ -1687,7 +1687,7 @@ Finally, update the following: For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: -* Set `state.latest_crosslinks[shard] = Crosslink(slot=current_epoch_start_slot, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. +* Set `state.latest_crosslinks[shard] = Crosslink(epoch=current_epoch(state), shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. ### Rewards and penalties @@ -1763,7 +1763,7 @@ First, update the following: If the following are satisfied: * `slot_to_epoch(state.finalized_slot) > state.validator_registry_update_epoch` -* `slot_to_epoch(state.latest_crosslinks[shard].slot) > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) +* `state.latest_crosslinks[shard].epoch > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) update the validator registry and associated fields by running From ae5dfab217e48e20c6f50543f017e0aa0ede11f1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 08:16:32 -0700 Subject: [PATCH 079/136] convert finality vars to epochs and do some more epohh cleaning --- specs/core/0_beacon-chain.md | 108 +++++++++++++++++------------------ 1 file changed, 54 insertions(+), 54 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 76f69f274..57158fdd4 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -333,8 +333,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'shard_block_root': 'bytes32', # Last crosslink's hash of root 'latest_crosslink_root': 'bytes32', - # Slot of the last justified beacon block - 'justified_slot': 'uint64', + # Last justified epoch in the beacon state + 'justified_epoch': 'uint64', # Hash of the last justified beacon block 'justified_block_root': 'bytes32', } @@ -487,10 +487,10 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'custody_challenges': [CustodyChallenge], # Finality - 'previous_justified_slot': 'uint64', - 'justified_slot': 'uint64', + 'previous_justified_epoch': 'uint64', + 'justified_epoch': 'uint64', 'justification_bitfield': 'uint64', - 'finalized_slot': 'uint64', + 'finalized_epoch': 'uint64', # Recent state 'latest_crosslinks': [Crosslink], @@ -726,7 +726,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi def lmd_ghost(store, start): validators = start.state.validator_registry active_validators = [validators[i] for i in - get_active_validator_indices(validators, start.state.slot)] + get_active_validator_indices(validators, current_epoch(start.state))] attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] def get_vote_count(block): @@ -781,21 +781,21 @@ def current_epoch(state: BeaconState) -> int: #### `is_active_validator` ```python -def is_active_validator(validator: Validator, slot: int) -> bool: +def is_active_validator(validator: Validator, epoch: int) -> bool: """ Checks if ``validator`` is active. """ - return validator.activation_epoch <= slot_to_epoch(slot) < validator.exit_epoch + return validator.activation_epoch <= epoch < validator.exit_epoch ``` #### `get_active_validator_indices` ```python -def get_active_validator_indices(validators: [Validator], slot: int) -> List[int]: +def get_active_validator_indices(validators: [Validator], epoch: int) -> List[int]: """ Gets indices of active validators from ``validators``. """ - return [i for i, v in enumerate(validators) if is_active_validator(v, slot)] + return [i for i, v in enumerate(validators) if is_active_validator(v, epoch)] ``` #### `shuffle` @@ -890,7 +890,7 @@ def get_shuffling(seed: Bytes32, committee is itself a list of validator indices. """ - active_validator_indices = get_active_validator_indices(validators, slot) + active_validator_indices = get_active_validator_indices(validators, epoch) committees_per_slot = get_committee_count_per_slot(len(active_validator_indices)) @@ -936,11 +936,11 @@ def get_crosslink_committees_at_slot(state: BeaconState, """ Returns the list of ``(committee, shard)`` tuples for the ``slot``. """ - state_epoch_slot = state.slot - (state.slot % EPOCH_LENGTH) - assert state_epoch_slot <= slot + EPOCH_LENGTH - assert slot < state_epoch_slot + EPOCH_LENGTH + current_epoch_slot = current_epoch(state) * EPOCH_LENGTH + assert current_epoch_slot <= slot + EPOCH_LENGTH + assert slot < current_epoch_slot + EPOCH_LENGTH - if slot < state_epoch_slot: + if slot < current_epoch_slot: committees_per_slot = get_previous_epoch_committee_count_per_slot(state) seed = state.previous_epoch_seed shuffling_epoch = state.previous_calculation_epoch @@ -1171,8 +1171,8 @@ def is_surround_vote(attestation_data_1: AttestationData, Note: parameter order matters as this function only checks that ``attestation_data_1`` surrounds ``attestation_data_2``. """ - source_epoch_1 = slot_to_epoch(attestation_data_1.justified_slot) - source_epoch_2 = slot_to_epoch(attestation_data_2.justified_slot) + source_epoch_1 = attestation_data_1.justified_epoch + source_epoch_2 = attestation_data_2.justified_epoch target_epoch_1 = slot_to_epoch(attestation_data_1.slot) target_epoch_2 = slot_to_epoch(attestation_data_2.slot) return ( @@ -1201,12 +1201,12 @@ def integer_squareroot(n: int) -> int: #### `entry_exit_effect_epoch` ```python -def entry_exit_effect_epoch(slot: int) -> int: +def entry_exit_effect_epoch(epoch: int) -> int: """ - An entry or exit triggered in the slot given by the input takes effect at + An entry or exit triggered in the ``epoch`` given by the input takes effect at the epoch given by the output. """ - return slot_to_epoch(slot) + 1 + ENTRY_EXIT_DELAY + return epoch + 1 + ENTRY_EXIT_DELAY ``` #### `bls_verify` @@ -1255,6 +1255,7 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val def get_initial_beacon_state(initial_validator_deposits: List[Deposit], genesis_time: int, latest_eth1_data: Eth1Data) -> BeaconState: + genesis_epoch = slot_to_epoch(GENESIS_SLOT) state = BeaconState( # Misc slot=GENESIS_SLOT, @@ -1268,7 +1269,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Validator registry validator_registry=[], validator_balances=[], - validator_registry_update_epoch=slot_to_epoch(GENESIS_SLOT), + validator_registry_update_epoch=genesis_epoch, validator_registry_exit_count=0, # Randomness and committees @@ -1276,8 +1277,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], latest_vdf_outputs=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH // EPOCH_LENGTH)], previous_epoch_start_shard=GENESIS_START_SHARD, current_epoch_start_shard=GENESIS_START_SHARD, - previous_calculation_epoch=slot_to_epoch(GENESIS_SLOT), - current_calculation_epoch=slot_to_epoch(GENESIS_SLOT), + previous_calculation_epoch=genesis_epoch, + current_calculation_epoch=genesis_epoch, previous_epoch_seed=ZERO_HASH, current_epoch_seed=ZERO_HASH, @@ -1285,13 +1286,13 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], custody_challenges=[], # Finality - previous_justified_slot=GENESIS_SLOT, - justified_slot=GENESIS_SLOT, + previous_justified_epoch=genesis_epoch, + justified_epoch=genesis_epoch, justification_bitfield=0, - finalized_slot=GENESIS_SLOT, + finalized_epoch=genesis_epoch, # Recent state - latest_crosslinks=[Crosslink(epoch=slot_to_epoch(GENESIS_SLOT), shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], + latest_crosslinks=[Crosslink(epoch=genesis_epoch, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)], latest_index_roots=[ZERO_HASH for _ in range(LATEST_INDEX_ROOTS_LENGTH)], latest_penalized_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)], @@ -1318,7 +1319,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) - genesis_epoch = current_epoch(state) state.latest_index_roots[genesis_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, genesis_epoch)) state.current_epoch_seed = generate_seed(state, genesis_epoch) @@ -1409,7 +1409,7 @@ Note: All functions in this section mutate `state`. def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else entry_exit_effect_epoch(state.slot) + validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else entry_exit_effect_epoch(current_epoch(state)) ``` ```python @@ -1423,10 +1423,10 @@ def exit_validator(state: BeaconState, index: int) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited - if validator.exit_epoch <= entry_exit_effect_epoch(state.slot): + if validator.exit_epoch <= entry_exit_effect_epoch(current_epoch(state)): return - validator.exit_epoch = entry_exit_effect_epoch(state.slot) + validator.exit_epoch = entry_exit_effect_epoch(current_epoch(state)) state.validator_registry_exit_count += 1 validator.exit_count = state.validator_registry_exit_count @@ -1504,7 +1504,7 @@ For each `proposer_slashing` in `block.body.proposer_slashings`: * Verify that `proposer_slashing.proposal_data_1.slot == proposer_slashing.proposal_data_2.slot`. * Verify that `proposer_slashing.proposal_data_1.shard == proposer_slashing.proposal_data_2.shard`. * Verify that `proposer_slashing.proposal_data_1.block_root != proposer_slashing.proposal_data_2.block_root`. -* Verify that `proposer.penalized_slot > state.slot`. +* Verify that `proposer.penalized_epoch > current_epoch(state)`. * Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork, proposer_slashing.proposal_data_1.slot, DOMAIN_PROPOSAL))`. * Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, proposer_slashing.proposal_data_2.slot, DOMAIN_PROPOSAL))`. * Run `penalize_validator(state, proposer_slashing.proposer_index)`. @@ -1524,7 +1524,7 @@ For each `casper_slashing` in `block.body.casper_slashings`: * Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. * Verify that `verify_slashable_vote_data(state, slashable_vote_data_1)`. * Verify that `verify_slashable_vote_data(state, slashable_vote_data_2)`. -* For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_slot > state.slot`. +* For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_epoch > current_epoch(state)`. #### Attestations @@ -1534,8 +1534,8 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot`. * Verify that `attestation.data.slot + EPOCH_LENGTH >= state.slot`. -* Verify that `attestation.data.justified_slot` is equal to `state.justified_slot if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_slot`. -* Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_slot)`. +* Verify that `attestation.data.justified_epoch` is equal to `state.justified_epoch if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_epoch`. +* Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_epoch * EPOCH_LENGTH)`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. * `aggregate_signature` verification: * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. @@ -1586,7 +1586,7 @@ Verify that `len(block.body.exits) <= MAX_EXITS`. For each `exit` in `block.body.exits`: * Let `validator = state.validator_registry[exit.validator_index]`. -* Verify that `validator.exit_epoch > entry_exit_effect_epoch(state.slot)`. +* Verify that `validator.exit_epoch > entry_exit_effect_epoch(current_epoch(state))`. * Verify that `state.slot >= exit.slot`. * Let `exit_message = hash_tree_root(Exit(slot=exit.slot, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. @@ -1617,7 +1617,7 @@ All [validators](#dfn-validator): * Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch_start_slot <= a.data.slot < next_epoch_start_slot]`. (Note: this is the set of attestations of slots in the epoch `current_epoch_start_slot...next_epoch_start_slot`, _not_ attestations that got included in the chain during the epoch `current_epoch_start_slot...next_epoch_start_slot`.) * Validators justifying the epoch boundary block at the start of the current epoch: - * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, current_epoch_start_slot) and a.data.justified_slot == state.justified_slot]`. + * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, current_epoch_start_slot) and a.data.justified_epoch == state.justified_epoch]`. * Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_boundary_attestations]`. * Let `current_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in current_epoch_boundary_attester_indices])`. @@ -1627,7 +1627,7 @@ All [validators](#dfn-validator): * Let `previous_epoch_attestations = [a for a in state.latest_attestations if previous_epoch_start_slot <= a.data.slot < current_epoch_start_slot]`. * Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_attestations]`. * Validators targeting the previous justified slot: - * Let `previous_epoch_justified_attestations = [a for a in current_epoch_attestations + previous_epoch_attestations if a.data.justified_slot == state.previous_justified_slot]`. + * Let `previous_epoch_justified_attestations = [a for a in current_epoch_attestations + previous_epoch_attestations if a.data.justified_epoch == state.previous_justified_epoch]`. * Let `previous_epoch_justified_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_justified_attestations]`. * Let `previous_epoch_justified_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_justified_attester_indices])`. * Validators justifying the epoch boundary block at the start of the previous epoch: @@ -1666,22 +1666,22 @@ If `current_epoch(state) % ETH1_DATA_VOTING_PERIOD == 0`: First, update the justification bitfield: -* Let `new_justified_slot = state.justified_slot`. +* Let `new_justified_epoch = state.justified_epoch`. * Set `state.justification_bitfield = (state.justification_bitfield * 2) % 2**64`. -* Set `state.justification_bitfield |= 2` and `new_justified_slot = previous_epoch_start_slot` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. -* Set `state.justification_bitfield |= 1` and `new_justified_slot = current_epoch_start_slot` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield |= 2` and `new_justified_epoch = previous_epoch` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch(state)` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. Next, update last finalized slot if possible: -* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_slot == previous_epoch_start_slot - 2 * EPOCH_LENGTH`. -* Set `state.finalized_slot = state.previous_justified_slot` if `(state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. -* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 8 == 0b111 and state.justified_slot == previous_epoch_start_slot - EPOCH_LENGTH`. -* Set `state.finalized_slot = state.justified_slot` if `(state.justification_bitfield >> 0) % 4 == 0b11 and state.justified_slot == previous_epoch_start_slot`. +* Set `state.finalized_epoch = state.previous_justified_epoch` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_epoch == previous_epoch - 2`. +* Set `state.finalized_epoch = state.previous_justified_epoch` if `(state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_epoch == previous_epoch - 1`. +* Set `state.finalized_epoch = state.justified_epoch` if `(state.justification_bitfield >> 0) % 8 == 0b111 and state.justified_epoch == previous_epoch - 1`. +* Set `state.finalized_epoch = state.justified_epoch` if `(state.justification_bitfield >> 0) % 4 == 0b11 and state.justified_epoch == previous_epoch`. Finally, update the following: -* Set `state.previous_justified_slot = state.justified_slot`. -* Set `state.justified_slot = justified_slot`. +* Set `state.previous_justified_epoch = state.justified_epoch`. +* Set `state.justified_epoch = new_justified_epoch`. ### Crosslinks @@ -1701,7 +1701,7 @@ First, we define some additional helpers: Note: When applying penalties in the following balance recalculations implementers should make sure the `uint64` does not underflow. -* Let `epochs_since_finality = next_epoch - slot_to_epoch(state.finalized_slot)`. +* Let `epochs_since_finality = next_epoch - state.finalized_epoch`. Case 1: `epochs_since_finality <= 4`: @@ -1758,11 +1758,11 @@ First, update the following: * Set `state.previous_calculation_epoch = state.current_calculation_epoch`. * Set `state.previous_epoch_start_shard = state.current_epoch_start_shard`. * Set `state.previous_epoch_seed = state.current_epoch_seed`. -* Set `state.latest_index_roots[next_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch_start_slot))`. +* Set `state.latest_index_roots[next_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch))`. If the following are satisfied: -* `slot_to_epoch(state.finalized_slot) > state.validator_registry_update_epoch` +* `state.finalized_epoch > state.validator_registry_update_epoch` * `state.latest_crosslinks[shard].epoch > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) update the validator registry and associated fields by running @@ -1787,7 +1787,7 @@ def update_validator_registry(state: BeaconState) -> None: # Activate validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.activation_epoch > entry_exit_effect_epoch(state.slot) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: + if validator.activation_epoch > entry_exit_effect_epoch(current_epoch(state)) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1799,7 +1799,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.exit_epoch > entry_exit_effect_epoch(state.slot) and validator.status_flags & INITIATED_EXIT: + if validator.exit_epoch > entry_exit_effect_epoch(current_epoch(state)) and validator.status_flags & INITIATED_EXIT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1832,7 +1832,7 @@ Regardless of whether or not a validator set change happens, run the following: ```python def process_penalties_and_exits(state: BeaconState) -> None: # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, state.slot) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state)) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) @@ -1867,7 +1867,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates * Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH]`. -* Remove any `attestation` in `state.latest_attestations` such that `attestation.data.slot < current_epoch_start_slot`. +* Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch(state)`. ## State root processing From 6e16cdc00e6e5492e5046b562db82666293d31b3 Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 18:59:07 +0000 Subject: [PATCH 080/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 5c0663242..4da97cc54 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -167,7 +167,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `EJECTION_BALANCE` | `2**4 * 1e9` (= 16,000,000,000) | Gwei | | `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - | | `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - | -| `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes | +| `MAX_CASPER_VOTES` | `2**12` (= 4,096) | votes | | `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots | | `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | randao mixes | | `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | @@ -232,7 +232,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | Name | Value | | - | - | | `MAX_PROPOSER_SLASHINGS` | `2**4` (= 16) | -| `MAX_CASPER_SLASHINGS` | `2**4` (= 16) | +| `MAX_CASPER_SLASHINGS` | `2**0` (= 1) | | `MAX_ATTESTATIONS` | `2**7` (= 128) | | `MAX_DEPOSITS` | `2**4` (= 16) | | `MAX_EXITS` | `2**4` (= 16) | @@ -1029,6 +1029,8 @@ def get_attestation_participants(state: BeaconState, assert attestation_data.shard in [shard for _, shard in crosslink_committees] crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0] + assert verify_bitfield(bitfield, len(crosslink_committee)) + # Find the participating attesters in the committee participants = [] for i, validator_index in enumerate(crosslink_committee): @@ -1088,14 +1090,14 @@ def get_bitfield_bit(bitfield: bytes, i: int) -> int: #### `verify_bitfield` ```python -def verify_bitfield(bitfield: bytes, size: int) -> bool: +def verify_bitfield(bitfield: bytes, committee_size: int) -> bool: """ - Verify ``bitfield`` against the ``size``. + Verify ``bitfield`` against the ``committee_size``. """ - if len(bitfield) != (size + 7) // 8: + if len(bitfield) != (committee_size + 7) // 8: return False - for i in range(size + 1, size - size % 8 + 8): + for i in range(committee_size + 1, committee_size - committee_size % 8 + 8): if get_bitfield_bit(bitfield, i) != 0: return False @@ -1109,7 +1111,7 @@ def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: Slashabl """ Verify validity of ``slashable_vote_data`` fields. """ - if slashable_vote_data.custody_bitfield != 0: # [TO BE REMOVED IN PHASE 1] + if slashable_vote_data.custody_bitfield != b'\x00' * len(slashable_vote_data.custody_bitfield): # [TO BE REMOVED IN PHASE 1] return False if len(slashable_vote_data.validator_indices) == 0: @@ -1536,9 +1538,10 @@ For each `attestation` in `block.body.attestations`: * Verify bitfields: ```python - assert verify_bitfield(attestation.aggregation_bitfield) - assert verify_bitfield(attestation.custody_bitfield) - assert attestation.custody_bitfield & attestation.aggregation_bitfield == attestation.custody_bitfield + for i in range(len(crosslink_committee)): + if get_bitfield_bit(attestation.aggregation_bitfield) == 0: + assert get_bitfield_bit(attestation.custody_bitfield) == 0 + assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] ``` From c3aff31da862e0a585405597f8da85ade2a04ca2 Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 19:06:46 +0000 Subject: [PATCH 081/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 898f9ae3d..906d3e1b5 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1551,19 +1551,15 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.justified_slot` is equal to `state.justified_slot if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_slot`. * Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_slot)`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. -* Verify bitfields: +* Verify bitfields and aggregate signature: ```python + assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] + for i in range(len(crosslink_committee)): if get_bitfield_bit(attestation.aggregation_bitfield) == 0: assert get_bitfield_bit(attestation.custody_bitfield) == 0 - assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] -``` - -* Verify aggregate signature: - -```python participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield) custody_bit_0_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) custody_bit_1_participants = [i in participants for i not in custody_bit_0_participants] From a81092627a4bfedc0427cd5b9ffa1e491d16f321 Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 19:10:21 +0000 Subject: [PATCH 082/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 906d3e1b5..0648a351c 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -167,7 +167,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `EJECTION_BALANCE` | `2**4 * 1e9` (= 16,000,000,000) | Gwei | | `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - | | `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - | -| `MAX_CASPER_VOTES` | `2**12` (= 4,096) | votes | +| `MAX_INDICES_PER_SLASHABLE_VOTE` | `2**12` (= 4,096) | votes | | `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots | | `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | randao mixes | | `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | index roots | @@ -1147,7 +1147,7 @@ def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: Slashabl if not verify_bitfield(slashable_vote_data.custody_bitfield, len(slashable_vote_data.validator_indices)): return False - if len(slashable_vote_data.validator_indices) > MAX_CASPER_VOTES: + if len(slashable_vote_data.validator_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: return False custody_bit_0_indices = [] From 1a4107876823f4c6060f25ba79cff89c988a2a1c Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 12:13:12 -0700 Subject: [PATCH 083/136] randao reveal is signed epoch number (#498) --- specs/core/0_beacon-chain.md | 8 ++------ specs/validator/0_beacon-chain-validator.md | 6 +++--- 2 files changed, 5 insertions(+), 9 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index c7bef5c25..5a185f63f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -514,8 +514,6 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git 'pubkey': 'bytes48', # Withdrawal credentials 'withdrawal_credentials': 'bytes32', - # Number of proposer slots since genesis - 'proposer_slots': 'uint64', # Slot when validator activated 'activation_slot': 'uint64', # Slot when validator exited @@ -1368,7 +1366,6 @@ def process_deposit(state: BeaconState, validator = Validator( pubkey=pubkey, withdrawal_credentials=withdrawal_credentials, - proposer_slots=0, activation_slot=FAR_FUTURE_SLOT, exit_slot=FAR_FUTURE_SLOT, withdrawal_slot=FAR_FUTURE_SLOT, @@ -1447,7 +1444,6 @@ Below are the processing steps that happen at every slot. ### Misc counters * Set `state.slot += 1`. -* Set `state.validator_registry[get_beacon_proposer_index(state, state.slot)].proposer_slots += 1`. * Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = state.latest_randao_mixes[(state.slot - 1) % LATEST_RANDAO_MIXES_LENGTH]` ### Block roots @@ -1473,8 +1469,8 @@ Below are the processing steps that happen at every `block`. ### RANDAO * Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(proposer.proposer_slots), signature=block.randao_reveal, domain=get_domain(state.fork, state.slot, DOMAIN_RANDAO))`. -* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = hash(state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] + block.randao_reveal)`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(state.slot // EPOCH_LENGTH), signature=block.randao_reveal, domain=get_domain(state.fork, state.slot, DOMAIN_RANDAO))`. +* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = xor(state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH], hash(block.randao_reveal))`. ### Eth1 data diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 29ec0c7e0..83ebb5751 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -158,12 +158,12 @@ _Note_: To calculate `state_root`, the validator should first run the state tran ##### Randao reveal -Set `block.randao_reveal = reveal_signature` where `reveal_signature` is defined as: +Set `block.randao_reveal = epoch_signature` where `epoch_signature` is defined as: ```python -reveal_signature = bls_sign( +epoch_signature = bls_sign( privkey=validator.privkey, # privkey store locally, not in state - message=int_to_bytes32(validator.proposer_slots + 1), + message=int_to_bytes32(block.slot // EPOCH_LENGTH), domain=get_domain( fork_data, # `fork_data` is the fork_data at the slot `block.slot` block.slot, From cfb1cb57b8bd7db23f8a7feaaf33b708036f4c11 Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 19:25:18 +0000 Subject: [PATCH 084/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 54 ++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 0648a351c..9d1fd14b7 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -25,7 +25,7 @@ - [`ProposerSlashing`](#proposerslashing) - [Casper slashings](#casper-slashings) - [`CasperSlashing`](#casperslashing) - - [`SlashableVoteData`](#slashablevotedata) + - [`SlashableVote`](#SlashableVote) - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) @@ -82,7 +82,7 @@ - [`get_domain`](#get_domain) - [`get_bitfield_bit`](#get_bitfield_bit) - [`verify_bitfield`](#verify_bitfield) - - [`verify_slashable_vote_data`](#verify_slashable_vote_data) + - [`verify_slashable_vote`](#verify_slashable_vote) - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) @@ -280,13 +280,13 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { # First batch of votes - 'slashable_vote_data_1': SlashableVoteData, + 'slashable_vote_1': SlashableVote, # Second batch of votes - 'slashable_vote_data_2': SlashableVoteData, + 'slashable_vote_2': SlashableVote, } ``` -##### `SlashableVoteData` +##### `SlashableVote` ```python { @@ -1127,33 +1127,33 @@ def verify_bitfield(bitfield: bytes, committee_size: int) -> bool: return True ``` -#### `verify_slashable_vote_data` +#### `verify_slashable_vote` ```python -def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: SlashableVoteData) -> bool: +def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> bool: """ - Verify validity of ``slashable_vote_data`` fields. + Verify validity of ``slashable_vote`` fields. """ - if slashable_vote_data.custody_bitfield != b'\x00' * len(slashable_vote_data.custody_bitfield): # [TO BE REMOVED IN PHASE 1] + if slashable_vote.custody_bitfield != b'\x00' * len(slashable_vote.custody_bitfield): # [TO BE REMOVED IN PHASE 1] return False - if len(slashable_vote_data.validator_indices) == 0: + if len(slashable_vote.validator_indices) == 0: return False - for i in range(len(slashable_vote_data.validator_indices) - 1): - if slashable_vote_data.validator_indices[i] >= slashable_vote_data.validator_indices[i + 1]: + for i in range(len(slashable_vote.validator_indices) - 1): + if slashable_vote.validator_indices[i] >= slashable_vote.validator_indices[i + 1]: return False - if not verify_bitfield(slashable_vote_data.custody_bitfield, len(slashable_vote_data.validator_indices)): + if not verify_bitfield(slashable_vote.custody_bitfield, len(slashable_vote.validator_indices)): return False - if len(slashable_vote_data.validator_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: + if len(slashable_vote.validator_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: return False custody_bit_0_indices = [] custody_bit_1_indices = [] - for i, validator_index in enumerate(slashable_vote_data.validator_indices): - if get_bitfield_bit(slashable_vote_data.custody_bitfield, i) == 0: + for i, validator_index in enumerate(slashable_vote.validator_indices): + if get_bitfield_bit(slashable_vote.custody_bitfield, i) == 0: custody_bit_0_indices.append(validator_index) else: custody_bit_1_indices.append(validator_index) @@ -1164,11 +1164,11 @@ def verify_slashable_vote_data(state: BeaconState, slashable_vote_data: Slashabl bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, 0)), - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote_data.data, 1)), + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote.data, 0)), + hash_tree_root(AttestationDataAndCustodyBit(slashable_vote.data, 1)), ], - signature=slashable_vote_data.aggregate_signature, - domain=get_domain(state.fork, slashable_vote_data.data.slot, DOMAIN_ATTESTATION), + signature=slashable_vote.aggregate_signature, + domain=get_domain(state.fork, slashable_vote.data.slot, DOMAIN_ATTESTATION), ) ``` @@ -1530,14 +1530,14 @@ Verify that `len(block.body.casper_slashings) <= MAX_CASPER_SLASHINGS`. For each `casper_slashing` in `block.body.casper_slashings`: -* Let `slashable_vote_data_1 = casper_slashing.slashable_vote_data_1`. -* Let `slashable_vote_data_2 = casper_slashing.slashable_vote_data_2`. -* Let `intersection = [x for x in slashable_vote_data_1.validator_indices if x in slashable_vote_data_2.validator_indices]`. +* Let `slashable_vote_1 = casper_slashing.slashable_vote_1`. +* Let `slashable_vote_2 = casper_slashing.slashable_vote_2`. +* Let `intersection = [x for x in slashable_vote_1.validator_indices if x in slashable_vote_2.validator_indices]`. * Verify that `len(intersection) >= 1`. -* Verify that `slashable_vote_data_1.data != slashable_vote_data_2.data`. -* Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. -* Verify that `verify_slashable_vote_data(state, slashable_vote_data_1)`. -* Verify that `verify_slashable_vote_data(state, slashable_vote_data_2)`. +* Verify that `slashable_vote_1.data != slashable_vote_2.data`. +* Verify that `is_double_vote(slashable_vote_1.data, slashable_vote_2.data)` or `is_surround_vote(slashable_vote_1.data, slashable_vote_2.data)`. +* Verify that `verify_slashable_vote(state, slashable_vote_1)`. +* Verify that `verify_slashable_vote(state, slashable_vote_2)`. * For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_slot > state.slot`. #### Attestations From aa909bddf3f32115ecfbd2bb5ddbeb0efab6f976 Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 19:25:58 +0000 Subject: [PATCH 085/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9d1fd14b7..9a3590d9a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -25,7 +25,7 @@ - [`ProposerSlashing`](#proposerslashing) - [Casper slashings](#casper-slashings) - [`CasperSlashing`](#casperslashing) - - [`SlashableVote`](#SlashableVote) + - [`SlashableVote`](#slashablevote) - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) From d282a36c63d51b1db2fe65dc98e75be173858c1f Mon Sep 17 00:00:00 2001 From: Justin Date: Sat, 26 Jan 2019 19:27:22 +0000 Subject: [PATCH 086/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9a3590d9a..33226aeee 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1554,7 +1554,7 @@ For each `attestation` in `block.body.attestations`: * Verify bitfields and aggregate signature: ```python - assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] + assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] for i in range(len(crosslink_committee)): if get_bitfield_bit(attestation.aggregation_bitfield) == 0: From b1ea3b26674d352861544099dc56d0feabf76790 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 15:27:50 -0700 Subject: [PATCH 087/136] address pr feedback --- specs/core/0_beacon-chain.md | 155 ++++++++++++++++++----------------- 1 file changed, 79 insertions(+), 76 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 57158fdd4..d919ff824 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -61,7 +61,7 @@ - [`hash`](#hash) - [`hash_tree_root`](#hash_tree_root) - [`slot_to_epoch`](#slot_to_epoch) - - [`current_epoch`](#current_epoch) + - [`get_current_epoch`](#get_current_epoch) - [`is_active_validator`](#is_active_validator) - [`get_active_validator_indices`](#get_active_validator_indices) - [`shuffle`](#shuffle) @@ -86,7 +86,7 @@ - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) - - [`entry_exit_effect_epoch`](#entry_exit_effect_epoch) + - [`get_entry_exit_effect_epoch`](#get_entry_exit_effect_epoch) - [`bls_verify`](#bls_verify) - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) @@ -192,6 +192,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | - | - | | `GENESIS_FORK_VERSION` | `0` | | `GENESIS_SLOT` | `0` | +| `GENESIS_EPOCH` | `slot_to_epoch(GENESIS_SLOT)` | | `GENESIS_START_SHARD` | `0` | | `FAR_FUTURE_EPOCH` | `2**64 - 1` | | `ZERO_HASH` | `int_to_bytes32(0)` | @@ -398,8 +399,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ```python { - # Minimum slot for processing exit - 'slot': 'uint64', + # Minimum epoch for processing exit + 'epoch': 'uint64', # Index of the exiting validator 'validator_index': 'uint24', # Validator signature @@ -569,8 +570,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be 'previous_version': 'uint64', # Current fork version 'current_version': 'uint64', - # Fork slot number - 'slot': 'uint64', + # Fork epoch number + 'epoch': 'uint64', } ``` @@ -715,8 +716,8 @@ Beacon block production is significantly different because of the proof of stake The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a [validator](#dfn-validator) `v` subjectively calculates the beacon chain head as follows. * Let `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not part of any chain are still included in `store`. -* Let `finalized_head` be the finalized block with the highest slot number. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) -* Let `justified_head` be the descendant of `finalized_head` with the highest slot number that has been justified for at least `EPOCH_LENGTH` slots. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. +* Let `finalized_head` be the finalized block with the highest epoch. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) +* Let `justified_head` be the descendant of `finalized_head` with the highest epoch that has been justified for at least 1 epoch. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. * Let `get_ancestor(store, block, slot)` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store, block, slot): return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. * Let `get_latest_attestation(store, validator)` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. * Let `get_latest_attestation_target(store, validator)` be the target block in the attestation `get_latest_attestation(store, validator)`. @@ -726,7 +727,7 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi def lmd_ghost(store, start): validators = start.state.validator_registry active_validators = [validators[i] for i in - get_active_validator_indices(validators, current_epoch(start.state))] + get_active_validator_indices(validators, get_current_epoch(start.state))] attestation_targets = [get_latest_attestation_target(store, validator) for validator in active_validators] def get_vote_count(block): @@ -771,10 +772,10 @@ def slot_to_epoch(slot: int) -> int: return slot // EPOCH_LENGTH ``` -#### `current_epoch` +#### `get_current_epoch` ```python -def current_epoch(state: BeaconState) -> int: +def get_current_epoch(state: BeaconState) -> int: return slot_to_epoch(state.slot) ``` @@ -902,7 +903,7 @@ def get_shuffling(seed: Bytes32, return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) ``` -**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. +**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= get_current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. @@ -936,7 +937,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, """ Returns the list of ``(committee, shard)`` tuples for the ``slot``. """ - current_epoch_slot = current_epoch(state) * EPOCH_LENGTH + current_epoch_slot = get_current_epoch(state) * EPOCH_LENGTH assert current_epoch_slot <= slot + EPOCH_LENGTH assert slot < current_epoch_slot + EPOCH_LENGTH @@ -1004,10 +1005,10 @@ def get_randao_mix(state: BeaconState, def get_active_index_root(state: BeaconState, epoch: int) -> Bytes32: """ - Returns the index root at a recent ``slot``. + Returns the index root at a recent ``epoch``. """ - assert current_epoch(state) < epoch + LATEST_INDEX_ROOTS_LENGTH - assert epoch <= current_epoch(state) + assert get_current_epoch(state) < epoch + LATEST_INDEX_ROOTS_LENGTH + assert epoch <= get_current_epoch(state) return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] ``` @@ -1100,8 +1101,8 @@ def get_effective_balance(state: State, index: int) -> int: ```python def get_fork_version(fork: Fork, - slot: int) -> int: - if slot < fork.slot: + epoch: int) -> int: + if epoch < fork.epoch: return fork.previous_version else: return fork.current_version @@ -1111,11 +1112,11 @@ def get_fork_version(fork: Fork, ```python def get_domain(fork: Fork, - slot: int, + epoch: int, domain_type: int) -> int: return get_fork_version( fork, - slot + epoch, ) * 2**32 + domain_type ``` @@ -1138,7 +1139,7 @@ def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) signature=vote_data.aggregate_signature, domain=get_domain( state.fork, - vote_data.data.slot, + slot_to_epoch(vote_data.data.slot), DOMAIN_ATTESTATION, ), ) @@ -1198,10 +1199,10 @@ def integer_squareroot(n: int) -> int: return x ``` -#### `entry_exit_effect_epoch` +#### `get_entry_exit_effect_epoch` ```python -def entry_exit_effect_epoch(epoch: int) -> int: +def get_entry_exit_effect_epoch(epoch: int) -> int: """ An entry or exit triggered in the ``epoch`` given by the input takes effect at the epoch given by the output. @@ -1255,7 +1256,6 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val def get_initial_beacon_state(initial_validator_deposits: List[Deposit], genesis_time: int, latest_eth1_data: Eth1Data) -> BeaconState: - genesis_epoch = slot_to_epoch(GENESIS_SLOT) state = BeaconState( # Misc slot=GENESIS_SLOT, @@ -1263,13 +1263,13 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], fork=Fork( previous_version=GENESIS_FORK_VERSION, current_version=GENESIS_FORK_VERSION, - slot=GENESIS_SLOT, + epoch=GENESIS_EPOCH, ), # Validator registry validator_registry=[], validator_balances=[], - validator_registry_update_epoch=genesis_epoch, + validator_registry_update_epoch=GENESIS_EPOCH, validator_registry_exit_count=0, # Randomness and committees @@ -1277,8 +1277,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], latest_vdf_outputs=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH // EPOCH_LENGTH)], previous_epoch_start_shard=GENESIS_START_SHARD, current_epoch_start_shard=GENESIS_START_SHARD, - previous_calculation_epoch=genesis_epoch, - current_calculation_epoch=genesis_epoch, + previous_calculation_epoch=GENESIS_EPOCH, + current_calculation_epoch=GENESIS_EPOCH, previous_epoch_seed=ZERO_HASH, current_epoch_seed=ZERO_HASH, @@ -1286,13 +1286,13 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], custody_challenges=[], # Finality - previous_justified_epoch=genesis_epoch, - justified_epoch=genesis_epoch, + previous_justified_epoch=GENESIS_EPOCH, + justified_epoch=GENESIS_EPOCH, justification_bitfield=0, - finalized_epoch=genesis_epoch, + finalized_epoch=GENESIS_EPOCH, # Recent state - latest_crosslinks=[Crosslink(epoch=genesis_epoch, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], + latest_crosslinks=[Crosslink(epoch=GENESIS_EPOCH, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)], latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)], latest_index_roots=[ZERO_HASH for _ in range(LATEST_INDEX_ROOTS_LENGTH)], latest_penalized_balances=[0 for _ in range(LATEST_PENALIZED_EXIT_LENGTH)], @@ -1319,8 +1319,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) - state.latest_index_roots[genesis_epoch % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, genesis_epoch)) - state.current_epoch_seed = generate_seed(state, genesis_epoch) + state.latest_index_roots[GENESIS_EPOCH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, GENESIS_EPOCH)) + state.current_epoch_seed = generate_seed(state, GENESIS_EPOCH) return state ``` @@ -1346,7 +1346,7 @@ def validate_proof_of_possession(state: BeaconState, signature=proof_of_possession, domain=get_domain( state.fork, - state.slot, + get_current_epoch(state), DOMAIN_DEPOSIT, ) ) @@ -1381,7 +1381,7 @@ def process_deposit(state: BeaconState, withdrawal_credentials=withdrawal_credentials, proposer_slots=0, activation_epoch=FAR_FUTURE_EPOCH, - exit_epch=FAR_FUTURE_EPOCH, + exit_epoch=FAR_FUTURE_EPOCH, withdrawal_epoch=FAR_FUTURE_EPOCH, penalized_epoch=FAR_FUTURE_EPOCH, exit_count=0, @@ -1409,7 +1409,7 @@ Note: All functions in this section mutate `state`. def activate_validator(state: BeaconState, index: int, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else entry_exit_effect_epoch(current_epoch(state)) + validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) ``` ```python @@ -1423,10 +1423,10 @@ def exit_validator(state: BeaconState, index: int) -> None: validator = state.validator_registry[index] # The following updates only occur if not previous exited - if validator.exit_epoch <= entry_exit_effect_epoch(current_epoch(state)): + if validator.exit_epoch <= get_entry_exit_effect_epoch(get_current_epoch(state)): return - validator.exit_epoch = entry_exit_effect_epoch(current_epoch(state)) + validator.exit_epoch = get_entry_exit_effect_epoch(get_current_epoch(state)) state.validator_registry_exit_count += 1 validator.exit_count = state.validator_registry_exit_count @@ -1436,13 +1436,13 @@ def exit_validator(state: BeaconState, index: int) -> None: def penalize_validator(state: BeaconState, index: int) -> None: exit_validator(state, index) validator = state.validator_registry[index] - state.latest_penalized_balances[current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) + state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) whistleblower_index = get_beacon_proposer_index(state, state.slot) whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT state.validator_balances[whistleblower_index] += whistleblower_reward state.validator_balances[index] -= whistleblower_reward - validator.penalized_epoch = current_epoch(state) + validator.penalized_epoch = get_current_epoch(state) ``` ```python @@ -1459,7 +1459,7 @@ Below are the processing steps that happen at every slot. * Set `state.slot += 1`. * Set `state.validator_registry[get_beacon_proposer_index(state, state.slot)].proposer_slots += 1`. -* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = state.latest_randao_mixes[(state.slot - 1) % LATEST_RANDAO_MIXES_LENGTH]` +* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, state.slot - 1)`. ### Block roots @@ -1479,13 +1479,13 @@ Below are the processing steps that happen at every `block`. * Let `block_without_signature_root` be the `hash_tree_root` of `block` where `block.signature` is set to `EMPTY_SIGNATURE`. * Let `proposal_root = hash_tree_root(ProposalSignedData(state.slot, BEACON_CHAIN_SHARD_NUMBER, block_without_signature_root))`. -* Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message=proposal_root, signature=block.signature, domain=get_domain(state.fork, state.slot, DOMAIN_PROPOSAL))`. +* Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message=proposal_root, signature=block.signature, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_PROPOSAL))`. ### RANDAO * Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(proposer.proposer_slots), signature=block.randao_reveal, domain=get_domain(state.fork, state.slot, DOMAIN_RANDAO))`. -* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = hash(state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] + block.randao_reveal)`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(proposer.proposer_slots), signature=block.randao_reveal, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_RANDAO))`. +* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = hash(get_randao_mix(state, state.slot) + block.randao_reveal)`. ### Eth1 data @@ -1504,9 +1504,9 @@ For each `proposer_slashing` in `block.body.proposer_slashings`: * Verify that `proposer_slashing.proposal_data_1.slot == proposer_slashing.proposal_data_2.slot`. * Verify that `proposer_slashing.proposal_data_1.shard == proposer_slashing.proposal_data_2.shard`. * Verify that `proposer_slashing.proposal_data_1.block_root != proposer_slashing.proposal_data_2.block_root`. -* Verify that `proposer.penalized_epoch > current_epoch(state)`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork, proposer_slashing.proposal_data_1.slot, DOMAIN_PROPOSAL))`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, proposer_slashing.proposal_data_2.slot, DOMAIN_PROPOSAL))`. +* Verify that `proposer.penalized_epoch > get_current_epoch(state)`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_1.slot), DOMAIN_PROPOSAL))`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_2.slot), DOMAIN_PROPOSAL))`. * Run `penalize_validator(state, proposer_slashing.proposer_index)`. #### Casper slashings @@ -1524,7 +1524,7 @@ For each `casper_slashing` in `block.body.casper_slashings`: * Verify that `is_double_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)` or `is_surround_vote(slashable_vote_data_1.data, slashable_vote_data_2.data)`. * Verify that `verify_slashable_vote_data(state, slashable_vote_data_1)`. * Verify that `verify_slashable_vote_data(state, slashable_vote_data_2)`. -* For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_epoch > current_epoch(state)`. +* For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_epoch > get_current_epoch(state)`. #### Attestations @@ -1540,7 +1540,7 @@ For each `attestation` in `block.body.attestations`: * `aggregate_signature` verification: * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. * Let `group_public_key = bls_aggregate_pubkeys([state.validator_registry[v].pubkey for v in participants])`. - * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataAndCustodyBit(attestation.data, False)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION))`. + * Verify that `bls_verify(pubkey=group_public_key, message=hash_tree_root(AttestationDataAndCustodyBit(attestation.data, False)), signature=attestation.aggregate_signature, domain=get_domain(state.fork, slot_to_epoch(attestation.data.slot), DOMAIN_ATTESTATION))`. * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. * Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`. @@ -1586,10 +1586,10 @@ Verify that `len(block.body.exits) <= MAX_EXITS`. For each `exit` in `block.body.exits`: * Let `validator = state.validator_registry[exit.validator_index]`. -* Verify that `validator.exit_epoch > entry_exit_effect_epoch(current_epoch(state))`. -* Verify that `state.slot >= exit.slot`. -* Let `exit_message = hash_tree_root(Exit(slot=exit.slot, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. -* Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.slot, DOMAIN_EXIT))`. +* Verify that `validator.exit_epoch > get_entry_exit_effect_epoch(get_current_epoch(state))`. +* Verify that `get_current_epoch(state) >= exit.epoch`. +* Let `exit_message = hash_tree_root(Exit(epoch=exit.epoch, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. +* Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.epoch, DOMAIN_EXIT))`. * Run `initiate_validator_exit(state, exit.validator_index)`. #### Custody @@ -1602,15 +1602,16 @@ The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. ### Helpers -* Let `current_epoch_start_slot = current_epoch(state) * EPOCH_LENGTH`. -* Let `next_epoch = current_epoch(state) + 1`. -* Let `next_epoch_start_slot = next_epoch * EPOCH_LENGTH`. -* Let `previous_epoch = current_epoch(state) - 1 if current_epoch(state) > slot_to_epoch(GENESIS_SLOT) else current_epoch(state)`. +* Let `current_epoch = get_current_epoch(state)`. +* Let `current_epoch_start_slot = current_epoch * EPOCH_LENGTH`. +* Let `previous_epoch = current_epoch - 1 if current_epoch > slot_to_epoch(GENESIS_SLOT) else current_epoch`. * Let `previous_epoch_start_slot = previous_epoch * EPOCH_LENGTH`. +* Let `next_epoch = current_epoch + 1`. +* Let `next_epoch_start_slot = next_epoch * EPOCH_LENGTH`. All [validators](#dfn-validator): -* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state))`. +* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch)`. * Let `total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices])`. [Validators](#dfn-Validator) attesting during the current epoch: @@ -1657,7 +1658,7 @@ Define the following helpers to process attestation inclusion rewards and inclus ### Eth1 data -If `current_epoch(state) % ETH1_DATA_VOTING_PERIOD == 0`: +If `current_epoch % ETH1_DATA_VOTING_PERIOD == 0`: * Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. @@ -1669,7 +1670,7 @@ First, update the justification bitfield: * Let `new_justified_epoch = state.justified_epoch`. * Set `state.justification_bitfield = (state.justification_bitfield * 2) % 2**64`. * Set `state.justification_bitfield |= 2` and `new_justified_epoch = previous_epoch` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. -* Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch(state)` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. Next, update last finalized slot if possible: @@ -1687,7 +1688,7 @@ Finally, update the following: For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: -* Set `state.latest_crosslinks[shard] = Crosslink(epoch=current_epoch(state), shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. +* Set `state.latest_crosslinks[shard] = Crosslink(epoch=current_epoch, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. ### Rewards and penalties @@ -1722,7 +1723,7 @@ Case 2: `epochs_since_finality > 4`: * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_justified_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices`, loses `inactivity_penalty(state, index, epochs_since_finality)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_head_attester_indices`, loses `base_reward(state, index)`. -* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch(state)`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. +* Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` loses `base_reward(state, index) - base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` #### Attestation inclusion @@ -1773,8 +1774,9 @@ def update_validator_registry(state: BeaconState) -> None: Update validator registry. Note that this function mutates ``state``. """ + current_epoch = get_current_epoch(state) # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state)) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) @@ -1787,7 +1789,7 @@ def update_validator_registry(state: BeaconState) -> None: # Activate validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.activation_epoch > entry_exit_effect_epoch(current_epoch(state)) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: + if validator.activation_epoch > get_entry_exit_effect_epoch(current_epoch) and state.validator_balances[index] >= MAX_DEPOSIT_AMOUNT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1799,7 +1801,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validators within the allowable balance churn balance_churn = 0 for index, validator in enumerate(state.validator_registry): - if validator.exit_epoch > entry_exit_effect_epoch(current_epoch(state)) and validator.status_flags & INITIATED_EXIT: + if validator.exit_epoch > get_entry_exit_effect_epoch(current_epoch) and validator.status_flags & INITIATED_EXIT: # Check the balance churn would be within the allowance balance_churn += get_effective_balance(state, index) if balance_churn > max_balance_churn: @@ -1808,7 +1810,7 @@ def update_validator_registry(state: BeaconState) -> None: # Exit validator exit_validator(state, index) - state.validator_registry_update_epoch = current_epoch(state) + state.validator_registry_update_epoch = current_epoch ``` and perform the following updates: @@ -1819,7 +1821,7 @@ and perform the following updates: If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = current_epoch(state) - state.validator_registry_update_epoch`. +* Let `epochs_since_last_registry_change = current_epoch - state.validator_registry_update_epoch`. * If `epochs_since_last_registry_change` is an exact power of 2: * Set `state.current_calculation_epoch = next_epoch`. * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` @@ -1831,14 +1833,15 @@ Regardless of whether or not a validator set change happens, run the following: ```python def process_penalties_and_exits(state: BeaconState) -> None: + current_epoch = get_current_epoch(state) # The active validators - active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch(state)) + active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch) # The total effective balance of active validators total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) for index, validator in enumerate(state.validator_registry): - if current_epoch(state) == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: - e = current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH + if current_epoch == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: + e = current_epoch % LATEST_PENALIZED_EXIT_LENGTH total_at_start = state.latest_penalized_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH] total_at_end = state.latest_penalized_balances[e] total_penalties = total_at_end - total_at_start @@ -1847,11 +1850,11 @@ def process_penalties_and_exits(state: BeaconState) -> None: def eligible(index): validator = state.validator_registry[index] - if validator.penalized_epoch <= current_epoch(state): + if validator.penalized_epoch <= current_epoch: PENALIZED_WITHDRAWAL_EPOCHS = LATEST_PENALIZED_EXIT_LENGTH // 2 - return current_epoch(state) >= validator.penalized_epoch + PENALIZED_WITHDRAWAL_EPOCHS + return current_epoch >= validator.penalized_epoch + PENALIZED_WITHDRAWAL_EPOCHS else: - return current_epoch(state) >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS + return current_epoch >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS all_indices = list(range(len(state.validator_registry))) eligible_indices = filter(eligible, all_indices) @@ -1866,8 +1869,8 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates -* Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH]`. -* Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch(state)`. +* Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch % LATEST_PENALIZED_EXIT_LENGTH]`. +* Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch`. ## State root processing From 8da467df7ca72aef010f3130d3bf2f03e2b5ae01 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sat, 26 Jan 2019 15:39:57 -0700 Subject: [PATCH 088/136] pr feedback --- specs/core/0_beacon-chain.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index d919ff824..3344d295d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -937,11 +937,14 @@ def get_crosslink_committees_at_slot(state: BeaconState, """ Returns the list of ``(committee, shard)`` tuples for the ``slot``. """ - current_epoch_slot = get_current_epoch(state) * EPOCH_LENGTH - assert current_epoch_slot <= slot + EPOCH_LENGTH - assert slot < current_epoch_slot + EPOCH_LENGTH + epoch = slot_to_epoch(slot) + current_epoch = get_current_epoch(state) + previous_epoch = current_epoch - 1 if epoch > GENESIS_EPOCH else current_epoch + next_epoch = current_epoch + 1 - if slot < current_epoch_slot: + assert previous_epoch <= epoch < next_epoch + + if epoch < current_epoch: committees_per_slot = get_previous_epoch_committee_count_per_slot(state) seed = state.previous_epoch_seed shuffling_epoch = state.previous_calculation_epoch From 82ecc8c86856fc504db96504aa93488f6218510d Mon Sep 17 00:00:00 2001 From: Justin Date: Sun, 27 Jan 2019 09:01:11 +0000 Subject: [PATCH 089/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 48 ++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 33226aeee..8ac9fb47c 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -261,7 +261,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { # Proposer index - 'proposer_index': 'uint24', + 'proposer_index': 'uint64', # First proposal data 'proposal_data_1': ProposalSignedData, # First proposal signature @@ -291,7 +291,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { # Validator indices - 'validator_indices': '[uint24]', + 'validator_indices': '[uint64]', # Custody bitfield 'custody_bitfield': 'bytes', # Attestation data @@ -402,7 +402,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git # Minimum slot for processing exit 'slot': 'uint64', # Index of the exiting validator - 'validator_index': 'uint24', + 'validator_index': 'uint64', # Validator signature 'signature': 'bytes96', } @@ -588,15 +588,15 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git We define the following Python custom types for type hinting and readability: -| Name | Type | Description | +| Name | SSZ equivalent | Description | | - | - | - | -| `SlotNumber` | unsigned 64-bit integer | the number of a slot | -| `ShardNumber` | unsigned 64-bit integer | the number of a shard | -| `ValidatorIndex` | unsigned 24-bit integer | the index number of a validator in the registry | -| `Gwei` | unsigned 64-bit integer | an amount in Gwei | -| `Bytes32` | 32-byte data | binary data with 32-byte length | -| `BLSPubkey` | 48-byte data | a public key in BLS signature scheme | -| `BLSSignature` | 96-byte data | a signature in BLS signature scheme | +| `SlotNumber` | `uint64` | a slot number | +| `ShardNumber` | `uint64` | a shard number | +| `ValidatorIndex` | `uint64` | an index in the validator registry | +| `Gwei` | `uint64` | an amount in Gwei | +| `Bytes32` | `bytes32` | 32 bytes of binary data | +| `BLSPubkey` | `bytes48` | a BLS public key | +| `BLSSignature` | `bytes96` | a BLS signature | ## Ethereum 1.0 deposit contract @@ -1058,7 +1058,7 @@ def get_attestation_participants(state: BeaconState, participants = [] for i, validator_index in enumerate(crosslink_committee): aggregation_bit = get_bitfield_bit(bitfield, i) - if aggregation_bit == 1: + if aggregation_bit == 0b1: participants.append(validator_index) return participants ``` @@ -1121,7 +1121,7 @@ def verify_bitfield(bitfield: bytes, committee_size: int) -> bool: return False for i in range(committee_size + 1, committee_size - committee_size % 8 + 8): - if get_bitfield_bit(bitfield, i) != 0: + if get_bitfield_bit(bitfield, i) == 0b1: return False return True @@ -1142,7 +1142,7 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> for i in range(len(slashable_vote.validator_indices) - 1): if slashable_vote.validator_indices[i] >= slashable_vote.validator_indices[i + 1]: - return False + return False if not verify_bitfield(slashable_vote.custody_bitfield, len(slashable_vote.validator_indices)): return False @@ -1153,7 +1153,7 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> custody_bit_0_indices = [] custody_bit_1_indices = [] for i, validator_index in enumerate(slashable_vote.validator_indices): - if get_bitfield_bit(slashable_vote.custody_bitfield, i) == 0: + if get_bitfield_bit(slashable_vote.custody_bitfield, i) == 0b0: custody_bit_0_indices.append(validator_index) else: custody_bit_1_indices.append(validator_index) @@ -1164,8 +1164,8 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote.data, 0)), - hash_tree_root(AttestationDataAndCustodyBit(slashable_vote.data, 1)), + hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b0)), + hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b1)), ], signature=slashable_vote.aggregate_signature, domain=get_domain(state.fork, slashable_vote.data.slot, DOMAIN_ATTESTATION), @@ -1554,15 +1554,15 @@ For each `attestation` in `block.body.attestations`: * Verify bitfields and aggregate signature: ```python - assert attestation.custody_bitfield == 0 # [TO BE REMOVED IN PHASE 1] + assert attestation.custody_bitfield == b'\x00' * len(attestation.custody_bitfield) # [TO BE REMOVED IN PHASE 1] for i in range(len(crosslink_committee)): - if get_bitfield_bit(attestation.aggregation_bitfield) == 0: - assert get_bitfield_bit(attestation.custody_bitfield) == 0 + if get_bitfield_bit(attestation.aggregation_bitfield) == 0b0: + assert get_bitfield_bit(attestation.custody_bitfield) == 0b0 participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield) - custody_bit_0_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) - custody_bit_1_participants = [i in participants for i not in custody_bit_0_participants] + custody_bit_1_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) + custody_bit_0_participants = [i in participants for i not in custody_bit_1_participants] assert bls_verify_multiple( pubkeys=[ @@ -1570,8 +1570,8 @@ For each `attestation` in `block.body.attestations`: bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_participants]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=0)), - hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=1)), + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=0b0)), + hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=0b1)), ], signature=attestation.aggregate_signature, domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION), From e1e1e2359a46e46c19ab7f68c92f70e94704ff49 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 07:54:46 -0700 Subject: [PATCH 090/136] pr feedback --- specs/core/0_beacon-chain.md | 40 ++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 225c7addc..97e7a7dd1 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -63,6 +63,7 @@ - [`hash_tree_root`](#hash_tree_root) - [`slot_to_epoch`](#slot_to_epoch) - [`get_current_epoch`](#get_current_epoch) + - [`get_epoch_start_slot`](#get_epoch_start_slot) - [`is_active_validator`](#is_active_validator) - [`get_active_validator_indices`](#get_active_validator_indices) - [`shuffle`](#shuffle) @@ -794,17 +795,23 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere #### `slot_to_epoch` ```python -def slot_to_epoch(slot: int) -> int: +def slot_to_epoch(slot: SlotNumber) -> EpochNumber: return slot // EPOCH_LENGTH ``` #### `get_current_epoch` ```python -def get_current_epoch(state: BeaconState) -> int: +def get_current_epoch(state: BeaconState) -> EpochNumber: return slot_to_epoch(state.slot) ``` +#### `get_epoch_start_slot` + +```python +def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: + return slot_to_epoch(state.slot) +``` #### `is_active_validator` ```python @@ -1050,12 +1057,12 @@ def generate_seed(state: BeaconState, Generate a seed for the given ``epoch``. """ if epoch < SEED_LOOKAHEAD: - randao_mix_epoch = slot_to_epoch(GENESIS_SLOT) + randao_mix_epoch = GENESIS_EPOCH else: randao_mix_epoch = epoch - SEED_LOOKAHEAD return hash( - get_randao_mix(state, randao_mix_epoch * EPOCH_LENGTH) + + get_randao_mix(state, get_epoch_start_slot(randao_mix_epoch)) + get_active_index_root(state, epoch) ) @@ -1437,7 +1444,7 @@ Note: All functions in this section mutate `state`. def activate_validator(state: BeaconState, index: ValidatorIndex, genesis: bool) -> None: validator = state.validator_registry[index] - validator.activation_epoch = slot_to_epoch(GENESIS_SLOT) if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) + validator.activation_epoch = GENESIS_EPOCH if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) ``` ```python @@ -1562,7 +1569,7 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot`. * Verify that `attestation.data.slot + EPOCH_LENGTH >= state.slot`. * Verify that `attestation.data.justified_epoch` is equal to `state.justified_epoch if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_epoch`. -* Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, attestation.data.justified_epoch * EPOCH_LENGTH)`. +* Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, get_epoch_start_slot(attestation.data.justified_epoch))`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. * `aggregate_signature` verification: * Let `participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield)`. @@ -1630,11 +1637,8 @@ The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. ### Helpers * Let `current_epoch = get_current_epoch(state)`. -* Let `current_epoch_start_slot = current_epoch * EPOCH_LENGTH`. -* Let `previous_epoch = current_epoch - 1 if current_epoch > slot_to_epoch(GENESIS_SLOT) else current_epoch`. -* Let `previous_epoch_start_slot = previous_epoch * EPOCH_LENGTH`. +* Let `previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch`. * Let `next_epoch = current_epoch + 1`. -* Let `next_epoch_start_slot = next_epoch * EPOCH_LENGTH`. All [validators](#dfn-validator): @@ -1643,23 +1647,23 @@ All [validators](#dfn-validator): [Validators](#dfn-Validator) attesting during the current epoch: -* Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch_start_slot <= a.data.slot < next_epoch_start_slot]`. (Note: this is the set of attestations of slots in the epoch `current_epoch_start_slot...next_epoch_start_slot`, _not_ attestations that got included in the chain during the epoch `current_epoch_start_slot...next_epoch_start_slot`.) +* Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch == slot_to_epoch(a.data.slot)]`. (Note: this is the set of attestations of slots in the epoch `current_epoch`, _not_ attestations that got included in the chain during the epoch `current_epoch`.) * Validators justifying the epoch boundary block at the start of the current epoch: - * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, current_epoch_start_slot) and a.data.justified_epoch == state.justified_epoch]`. + * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, get_epoch_start_slot(current_epoch)) and a.data.justified_epoch == state.justified_epoch]`. * Let `current_epoch_boundary_attester_indices` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_boundary_attestations]`. * Let `current_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in current_epoch_boundary_attester_indices])`. [Validators](#dfn-Validator) attesting during the previous epoch: * Validators that made an attestation during the previous epoch: - * Let `previous_epoch_attestations = [a for a in state.latest_attestations if previous_epoch_start_slot <= a.data.slot < current_epoch_start_slot]`. + * Let `previous_epoch_attestations = [a for a in state.latest_attestations if previous_epoch == slot_to_epoch(a.data.slot)]`. * Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_attestations]`. * Validators targeting the previous justified slot: * Let `previous_epoch_justified_attestations = [a for a in current_epoch_attestations + previous_epoch_attestations if a.data.justified_epoch == state.previous_justified_epoch]`. * Let `previous_epoch_justified_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_justified_attestations]`. * Let `previous_epoch_justified_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_justified_attester_indices])`. * Validators justifying the epoch boundary block at the start of the previous epoch: - * Let `previous_epoch_boundary_attestations = [a for a in previous_epoch_justified_attestations if a.data.epoch_boundary_root == get_block_root(state, previous_epoch_start_slot)]`. + * Let `previous_epoch_boundary_attestations = [a for a in previous_epoch_justified_attestations if a.data.epoch_boundary_root == get_block_root(state, get_epoch_start_slot(previous_epoch))]`. * Let `previous_epoch_boundary_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_boundary_attestations]`. * Let `previous_epoch_boundary_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_boundary_attester_indices])`. * Validators attesting to the expected beacon chain head during the previous epoch: @@ -1669,7 +1673,7 @@ All [validators](#dfn-validator): **Note**: `previous_epoch_boundary_attesting_balance` balance might be marginally different than `current_epoch_boundary_attesting_balance` during the previous epoch transition. Due to the tight bound on validator churn each epoch and small per-epoch rewards/penalties, the potential balance difference is very low and only marginally affects consensus safety. -For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(next_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * Let `shard_block_root` be `state.latest_crosslinks[shard].shard_block_root` * Let `attesting_validator_indices(crosslink_committee, shard_block_root)` be the union of the [validator](#dfn-validator) index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in current_epoch_attestations + previous_epoch_attestations if a.data.shard == shard and a.data.shard_block_root == shard_block_root]`. @@ -1699,7 +1703,7 @@ First, update the justification bitfield: * Set `state.justification_bitfield |= 2` and `new_justified_epoch = previous_epoch` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. * Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. -Next, update last finalized slot if possible: +Next, update last finalized epoch if possible: * Set `state.finalized_epoch = state.previous_justified_epoch` if `(state.justification_bitfield >> 1) % 8 == 0b111 and state.previous_justified_epoch == previous_epoch - 2`. * Set `state.finalized_epoch = state.previous_justified_epoch` if `(state.justification_bitfield >> 1) % 4 == 0b11 and state.previous_justified_epoch == previous_epoch - 1`. @@ -1713,7 +1717,7 @@ Finally, update the following: ### Crosslinks -For every `slot in range(previous_epoch_start_slot, next_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(next_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * Set `state.latest_crosslinks[shard] = Crosslink(epoch=current_epoch, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. @@ -1759,7 +1763,7 @@ For each `index` in `previous_epoch_attester_indices`, we determine the proposer #### Crosslinks -For every `slot in range(previous_epoch_start_slot, current_epoch_start_slot)`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(current_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * If `index in attesting_validators(crosslink_committee)`, `state.validator_balances[index] += base_reward(state, index) * total_attesting_balance(crosslink_committee) // total_balance(crosslink_committee))`. * If `index not in attesting_validators(crosslink_committee)`, `state.validator_balances[index] -= base_reward(state, index)`. From 9d18760521ae05c1edfd0ca6a1fb104c02506af9 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 09:56:26 -0700 Subject: [PATCH 091/136] convert some functions to be per epoch --- specs/core/0_beacon-chain.md | 39 ++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 97e7a7dd1..2ead71784 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -68,10 +68,10 @@ - [`get_active_validator_indices`](#get_active_validator_indices) - [`shuffle`](#shuffle) - [`split`](#split) - - [`get_committee_count_per_slot`](#get_committee_count_per_slot) + - [`get_epoch_committee_count`](#get_epoch_committee_count) - [`get_shuffling`](#get_shuffling) - - [`get_previous_epoch_committee_count_per_slot`](#get_previous_epoch_committee_count_per_slot) - - [`get_current_epoch_committee_count_per_slot`](#get_current_epoch_committee_count_per_slot) + - [`get_previous_epoch_committee_count`](#get_previous_epoch_committee_count) + - [`get_current_epoch_committee_count`](#get_current_epoch_committee_count) - [`get_crosslink_committees_at_slot`](#get_crosslink_committees_at_slot) - [`get_block_root`](#get_block_root) - [`get_randao_mix`](#get_randao_mix) @@ -899,17 +899,17 @@ def split(values: List[Any], split_count: int) -> List[List[Any]]: ] ``` -#### `get_committee_count_per_slot` +#### `get_epoch_committee_count` ```python -def get_committee_count_per_slot(active_validator_count: int) -> int: +def get_epoch_committee_count(active_validator_count: int) -> int: return max( 1, min( SHARD_COUNT // EPOCH_LENGTH, active_validator_count // EPOCH_LENGTH // TARGET_COMMITTEE_SIZE, ) - ) + ) * EPOCH_LENGTH ``` #### `get_shuffling` @@ -926,40 +926,40 @@ def get_shuffling(seed: Bytes32, active_validator_indices = get_active_validator_indices(validators, epoch) - committees_per_slot = get_committee_count_per_slot(len(active_validator_indices)) + committees_per_epoch = get_epoch_committee_count(len(active_validator_indices)) # Shuffle seed = xor(seed, int_to_bytes32(epoch)) shuffled_active_validator_indices = shuffle(active_validator_indices, seed) - # Split the shuffled list into epoch_length * committees_per_slot pieces - return split(shuffled_active_validator_indices, committees_per_slot * EPOCH_LENGTH) + # Split the shuffled list into committees_per_epoch pieces + return split(shuffled_active_validator_indices, committees_per_epoch) ``` **Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= get_current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. -#### `get_previous_epoch_committee_count_per_slot` +#### `get_previous_epoch_committee_count` ```python -def get_previous_epoch_committee_count_per_slot(state: BeaconState) -> int: +def get_previous_epoch_committee_count(state: BeaconState) -> int: previous_active_validators = get_active_validator_indices( state.validator_registry, state.previous_calculation_epoch, ) - return get_committee_count_per_slot(len(previous_active_validators)) + return get_epoch_committee_count(len(previous_active_validators)) ``` -#### `get_current_epoch_committee_count_per_slot` +#### `get_current_epoch_committee_count` ```python -def get_current_epoch_committee_count_per_slot(state: BeaconState) -> int: +def get_current_epoch_committee_count(state: BeaconState) -> int: current_active_validators = get_active_validator_indices( state.validator_registry, state.current_calculation_epoch, ) - return get_committee_count_per_slot(len(current_active_validators)) + return get_epoch_committee_count(len(current_active_validators)) ``` #### `get_crosslink_committees_at_slot` @@ -978,12 +978,12 @@ def get_crosslink_committees_at_slot(state: BeaconState, assert previous_epoch <= epoch < next_epoch if epoch < current_epoch: - committees_per_slot = get_previous_epoch_committee_count_per_slot(state) + committees_per_epoch = get_previous_epoch_committee_count(state) seed = state.previous_epoch_seed shuffling_epoch = state.previous_calculation_epoch shuffling_start_shard = state.previous_epoch_start_shard else: - committees_per_slot = get_current_epoch_committee_count_per_slot(state) + committees_per_epoch = get_current_epoch_committee_count(state) seed = state.current_epoch_seed shuffling_epoch = state.current_calculation_epoch shuffling_start_shard = state.current_epoch_start_shard @@ -994,6 +994,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, shuffling_epoch, ) offset = slot % EPOCH_LENGTH + committees_per_slot = committees_per_epoch // EPOCH_LENGTH slot_start_shard = (shuffling_start_shard + committees_per_slot * offset) % SHARD_COUNT return [ @@ -1795,7 +1796,7 @@ First, update the following: If the following are satisfied: * `state.finalized_epoch > state.validator_registry_update_epoch` -* `state.latest_crosslinks[shard].epoch > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH)]` (that is, for every shard in the current committees) +* `state.latest_crosslinks[shard].epoch > state.validator_registry_update_epoch` for every shard number `shard` in `[(state.current_epoch_start_shard + i) % SHARD_COUNT for i in range(get_current_epoch_committee_count(state))]` (that is, for every shard in the current committees) update the validator registry and associated fields by running @@ -1847,7 +1848,7 @@ def update_validator_registry(state: BeaconState) -> None: and perform the following updates: * Set `state.current_calculation_epoch = next_epoch` -* Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count_per_slot(state) * EPOCH_LENGTH) % SHARD_COUNT` +* Set `state.current_epoch_start_shard = (state.current_epoch_start_shard + get_current_epoch_committee_count(state)) % SHARD_COUNT` * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` If a validator registry update does _not_ happen do the following: From 12619995bbe05fde92e1be5f239a9fae583dfe23 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 10:00:03 -0700 Subject: [PATCH 092/136] pr feedback --- specs/core/0_beacon-chain.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 2ead71784..44ca758b4 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -810,7 +810,7 @@ def get_current_epoch(state: BeaconState) -> EpochNumber: ```python def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: - return slot_to_epoch(state.slot) + return epoch * EPOCH_LENGTH ``` #### `is_active_validator` @@ -920,7 +920,7 @@ def get_shuffling(seed: Bytes32, epoch: EpochNumber) -> List[List[ValidatorIndex]] """ Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. - Returns a list of ``EPOCH_LENGTH * committees_per_slot`` committees where each + Returns a list of ``committees_per_epoch`` committees where each committee is itself a list of validator indices. """ @@ -1053,7 +1053,7 @@ def get_active_index_root(state: BeaconState, ```python def generate_seed(state: BeaconState, - epoch: int) -> Bytes32: + epoch: EpochNumber) -> Bytes32: """ Generate a seed for the given ``epoch``. """ From 8fbaa255963b03236b665d4b5d65379862ec2c6e Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 10:01:04 -0700 Subject: [PATCH 093/136] pr feedback --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 44ca758b4..95289793d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1239,7 +1239,7 @@ def integer_squareroot(n: int) -> int: #### `get_entry_exit_effect_epoch` ```python -def get_entry_exit_effect_epoch(epoch: int) -> int: +def get_entry_exit_effect_epoch(epoch: EpochNumber) -> EpochNumber: """ An entry or exit triggered in the ``epoch`` given by the input takes effect at the epoch given by the output. From e41eeeedb61e45a154c0559949dc7308e7f93d93 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 10:22:27 -0700 Subject: [PATCH 094/136] make randao_mixes per epoch --- specs/core/0_beacon-chain.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 95289793d..068584fa5 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -171,9 +171,9 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - | | `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - | | `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes | -| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots | -| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | randao mixes | -| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | index roots | +| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | slots | +| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | +| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | | `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | | `MAX_WITHDRAWALS_PER_EPOCH` | `2**2` (= 4) | withdrawals | @@ -1027,13 +1027,13 @@ def get_block_root(state: BeaconState, ```python def get_randao_mix(state: BeaconState, - slot: SlotNumber) -> Bytes32: + epoch: EpochNumber) -> Bytes32: """ Returns the randao mix at a recent ``slot``. """ - assert state.slot < slot + LATEST_RANDAO_MIXES_LENGTH - assert slot <= state.slot - return state.latest_randao_mixes[slot % LATEST_RANDAO_MIXES_LENGTH] + assert get_current_epoch(state) < epoch + LATEST_RANDAO_MIXES_LENGTH + assert epoch <= get_current_epoch(state) + return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] ``` #### `get_active_index_root` @@ -1494,7 +1494,6 @@ Below are the processing steps that happen at every slot. ### Misc counters * Set `state.slot += 1`. -* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, state.slot - 1)`. ### Block roots @@ -1520,7 +1519,7 @@ Below are the processing steps that happen at every `block`. * Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. * Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(get_current_epoch(state)), signature=block.randao_reveal, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_RANDAO))`. -* Set `state.latest_randao_mixes[state.slot % LATEST_RANDAO_MIXES_LENGTH] = xor(get_randao_mix(state, state.slot), hash(block.randao_reveal))`. +* Set `state.latest_randao_mixes[get_current_epoch(state) % LATEST_RANDAO_MIXES_LENGTH] = xor(get_randao_mix(state, get_current_epoch(state)), hash(block.randao_reveal))`. ### Eth1 data @@ -1902,6 +1901,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: ### Final updates * Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch % LATEST_PENALIZED_EXIT_LENGTH]`. +* Set `state.latest_randao_mixes[next_epoch % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, current_epoch)`. * Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch`. ## State root processing From b0e71f4c80e0d3a82d6c5a62220c6ccbaa002b90 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 10:23:28 -0700 Subject: [PATCH 095/136] pr feedback --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 068584fa5..6659ee3f4 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1689,7 +1689,7 @@ Define the following helpers to process attestation inclusion rewards and inclus ### Eth1 data -If `current_epoch % ETH1_DATA_VOTING_PERIOD == 0`: +If `next_epoch % ETH1_DATA_VOTING_PERIOD == 0`: * Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes`. * Set `state.eth1_data_votes = []`. From ad46b9cb1248f2d1023899b4a2fe4fd033e53389 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 10:31:01 -0700 Subject: [PATCH 096/136] pr feedback --- specs/core/0_beacon-chain.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 6659ee3f4..35435ed7f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -774,7 +774,7 @@ def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) We now define the state transition function. At a high level the state transition is made up of two parts: 1. The per-slot transitions, which happens every slot, and only affects a parts of the `state`. -2. The per-epoch transitions, which happens at every epoch boundary (i.e. `state.slot % EPOCH_LENGTH == 0`), and affects the entire `state`. +2. The per-epoch transitions, which happens in the last slot of every epoch (i.e. `(state.slot + 1) % EPOCH_LENGTH == 0`), and affects the entire `state`. The per-slot transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-slot activity in the `BeaconState`. The per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. @@ -1057,13 +1057,10 @@ def generate_seed(state: BeaconState, """ Generate a seed for the given ``epoch``. """ - if epoch < SEED_LOOKAHEAD: - randao_mix_epoch = GENESIS_EPOCH - else: - randao_mix_epoch = epoch - SEED_LOOKAHEAD + randao_mix_epoch = epoch + LATEST_RANDAO_MIXES_LENGTH - SEED_LOOKAHEAD return hash( - get_randao_mix(state, get_epoch_start_slot(randao_mix_epoch)) + + get_randao_mix(state, randao_mix_epoch) + get_active_index_root(state, epoch) ) @@ -1568,7 +1565,7 @@ For each `attestation` in `block.body.attestations`: * Verify that `attestation.data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot`. * Verify that `attestation.data.slot + EPOCH_LENGTH >= state.slot`. -* Verify that `attestation.data.justified_epoch` is equal to `state.justified_epoch if attestation.data.slot >= state.slot - (state.slot % EPOCH_LENGTH) else state.previous_justified_epoch`. +* Verify that `attestation.data.justified_epoch` is equal to `state.justified_epoch if attestation.data.slot >= get_epoch_start_slot(get_current_epoch(state)) else state.previous_justified_epoch`. * Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, get_epoch_start_slot(attestation.data.justified_epoch))`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. * `aggregate_signature` verification: From 41827d26ecf50d4b02a0e1cb944414a3e399eea6 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Mon, 28 Jan 2019 01:49:15 +0800 Subject: [PATCH 097/136] Add the brief of `per_block` processing, and nudge some wording --- specs/core/0_beacon-chain.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 5a185f63f..fd5cd7d35 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -765,12 +765,13 @@ def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) ## Beacon chain state transition function -We now define the state transition function. At a high level the state transition is made up of two parts: +We now define the state transition function. At a high level the state transition is made up of three parts: -1. The per-slot transitions, which happens every slot, and only affects a parts of the `state`. -2. The per-epoch transitions, which happens at every epoch boundary (i.e. `state.slot % EPOCH_LENGTH == 0`), and affects the entire `state`. +1. The per-slot transitions, which happens every slot. +2. The per-block transitions, which happens at every block. +3. The per-epoch transitions, which happens at every epoch boundary (i.e. `state.slot % EPOCH_LENGTH == 0`). -The per-slot transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-slot activity in the `BeaconState`. The per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. +The per-slot transitions focus on the slot counter and block roots records updates; the per-block transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-block activity in the `BeaconState`; the per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. ### Helper functions From f9b53e858369f6c889f3a99f4850a02b34f51e0f Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 17:20:15 -0700 Subject: [PATCH 098/136] pr feedback --- specs/core/0_beacon-chain.md | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 35435ed7f..21744ecbc 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -96,10 +96,10 @@ - [Routine for processing deposits](#routine-for-processing-deposits) - [Routines for updating validator status](#routines-for-updating-validator-status) - [Per-slot processing](#per-slot-processing) - - [Misc counters](#misc-counters) + - [Slot](#slot) - [Block roots](#block-roots) - [Per-block processing](#per-block-processing) - - [Slot](#slot) + - [Slot](#slot-1) - [Proposer signature](#proposer-signature) - [RANDAO](#randao) - [Eth1 data](#eth1-data) @@ -1488,7 +1488,7 @@ def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) Below are the processing steps that happen at every slot. -### Misc counters +### Slot * Set `state.slot += 1`. @@ -1637,13 +1637,9 @@ The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. * Let `previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch`. * Let `next_epoch = current_epoch + 1`. -All [validators](#dfn-validator): - -* Let `active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch)`. -* Let `total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices])`. - [Validators](#dfn-Validator) attesting during the current epoch: +* Let `current_total_balance = sum([get_effective_balance(state, i) for i in get_active_validator_indices(state.validator_registry, current_epoch)])`. * Let `current_epoch_attestations = [a for a in state.latest_attestations if current_epoch == slot_to_epoch(a.data.slot)]`. (Note: this is the set of attestations of slots in the epoch `current_epoch`, _not_ attestations that got included in the chain during the epoch `current_epoch`.) * Validators justifying the epoch boundary block at the start of the current epoch: * Let `current_epoch_boundary_attestations = [a for a in current_epoch_attestations if a.data.epoch_boundary_root == get_block_root(state, get_epoch_start_slot(current_epoch)) and a.data.justified_epoch == state.justified_epoch]`. @@ -1652,6 +1648,7 @@ All [validators](#dfn-validator): [Validators](#dfn-Validator) attesting during the previous epoch: +* Let `previous_total_balance = sum([get_effective_balance(state, i) for i in get_active_validator_indices(state.validator_registry, previous_epoch)])`. * Validators that made an attestation during the previous epoch: * Let `previous_epoch_attestations = [a for a in state.latest_attestations if previous_epoch == slot_to_epoch(a.data.slot)]`. * Let `previous_epoch_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_attestations]`. @@ -1668,7 +1665,7 @@ All [validators](#dfn-validator): * Let `previous_epoch_head_attester_indices` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.aggregation_bitfield) for a in previous_epoch_head_attestations]`. * Let `previous_epoch_head_attesting_balance = sum([get_effective_balance(state, i) for i in previous_epoch_head_attester_indices])`. -**Note**: `previous_epoch_boundary_attesting_balance` balance might be marginally different than `current_epoch_boundary_attesting_balance` during the previous epoch transition. Due to the tight bound on validator churn each epoch and small per-epoch rewards/penalties, the potential balance difference is very low and only marginally affects consensus safety. +**Note**: `previous_total_balance` and `previous_epoch_boundary_attesting_balance` balance might be marginally different than the actual balances during previous epoch transition. Due to the tight bound on validator churn each epoch and small per-epoch rewards/penalties, the potential balance difference is very low and only marginally affects consensus safety. For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(next_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: @@ -1696,9 +1693,9 @@ If `next_epoch % ETH1_DATA_VOTING_PERIOD == 0`: First, update the justification bitfield: * Let `new_justified_epoch = state.justified_epoch`. -* Set `state.justification_bitfield = (state.justification_bitfield * 2) % 2**64`. -* Set `state.justification_bitfield |= 2` and `new_justified_epoch = previous_epoch` if `3 * previous_epoch_boundary_attesting_balance >= 2 * total_balance`. -* Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch` if `3 * current_epoch_boundary_attesting_balance >= 2 * total_balance`. +* Set `state.justification_bitfield = state.justification_bitfield << 1`. +* Set `state.justification_bitfield |= 2` and `new_justified_epoch = previous_epoch` if `3 * previous_epoch_boundary_attesting_balance >= 2 * previous_total_balance`. +* Set `state.justification_bitfield |= 1` and `new_justified_epoch = current_epoch` if `3 * current_epoch_boundary_attesting_balance >= 2 * current_total_balance`. Next, update last finalized epoch if possible: @@ -1722,7 +1719,7 @@ For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_s First, we define some additional helpers: -* Let `base_reward_quotient = integer_squareroot(total_balance) // BASE_REWARD_QUOTIENT`. +* Let `base_reward_quotient = integer_squareroot(previous_total_balance) // BASE_REWARD_QUOTIENT`. * Let `base_reward(state, index) = get_effective_balance(state, index) // base_reward_quotient // 5` for any validator with the given `index`. * Let `inactivity_penalty(state, index, epochs_since_finality) = base_reward(state, index) + get_effective_balance(state, index) * epochs_since_finality // INACTIVITY_PENALTY_QUOTIENT // 2` for any validator with the given `index`. @@ -1735,13 +1732,13 @@ Note: When applying penalties in the following balance recalculations implemente Case 1: `epochs_since_finality <= 4`: * Expected FFG source: - * Any [validator](#dfn-validator) `index` in `previous_epoch_justified_attester_indices` gains `base_reward(state, index) * previous_epoch_justified_attesting_balance // total_balance`. + * Any [validator](#dfn-validator) `index` in `previous_epoch_justified_attester_indices` gains `base_reward(state, index) * previous_epoch_justified_attesting_balance // previous_total_balance`. * Any [active validator](#dfn-active-validator) `v` not in `previous_epoch_justified_attester_indices` loses `base_reward(state, index)`. * Expected FFG target: - * Any [validator](#dfn-validator) `index` in `previous_epoch_boundary_attester_indices` gains `base_reward(state, index) * previous_epoch_boundary_attesting_balance // total_balance`. + * Any [validator](#dfn-validator) `index` in `previous_epoch_boundary_attester_indices` gains `base_reward(state, index) * previous_epoch_boundary_attesting_balance // previous_total_balance`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices` loses `base_reward(state, index)`. * Expected beacon chain head: - * Any [validator](#dfn-validator) `index` in `previous_epoch_head_attester_indices` gains `base_reward(state, index) * previous_epoch_head_attesting_balance // total_balance)`. + * Any [validator](#dfn-validator) `index` in `previous_epoch_head_attester_indices` gains `base_reward(state, index) * previous_epoch_head_attesting_balance // previous_total_balance)`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_head_attester_indices` loses `base_reward(state, index)`. * Inclusion distance: * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` gains `base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` From 1947fc0ff32b522f1c0d8bff9159a9f90848bc71 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 17:25:29 -0700 Subject: [PATCH 099/136] pr feedback --- specs/core/0_beacon-chain.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 21744ecbc..10a41506f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -15,6 +15,7 @@ - [Deposit contract](#deposit-contract) - [Initial values](#initial-values) - [Time parameters](#time-parameters) + - [State list lengths](#state-list-lengths) - [Reward and penalty quotients](#reward-and-penalty-quotients) - [Status flags](#status-flags) - [Max operations per block](#max-operations-per-block) @@ -171,10 +172,6 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - | | `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - | | `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes | -| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | slots | -| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | -| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | -| `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | | `MAX_WITHDRAWALS_PER_EPOCH` | `2**2` (= 4) | withdrawals | * For the safety of crosslinks `TARGET_COMMITTEE_SIZE` exceeds [the recommended minimum committee size of 111](https://vitalik.ca/files/Ithaca201807_Sharding.pdf); with sufficient active validators (at least `EPOCH_LENGTH * TARGET_COMMITTEE_SIZE`), the shuffling algorithm ensures committee sizes at least `TARGET_COMMITTEE_SIZE`. (Unbiasable randomness with a Verifiable Delay Function (VDF) will improve committee robustness and lower the safe minimum committee size.) @@ -213,6 +210,14 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `ETH1_DATA_VOTING_PERIOD` | `2**4` (= 16) | epochs | ~1.7 hours | | `MIN_VALIDATOR_WITHDRAWAL_EPOCHS` | `2**8` (= 256) | epochs | ~27 hours | +### State list lengths + +| Name | Value | Unit | +| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | slots | +| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | +| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | +| `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | + ### Reward and penalty quotients | Name | Value | @@ -1757,10 +1762,12 @@ For each `index` in `previous_epoch_attester_indices`, we determine the proposer #### Crosslinks -For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(current_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: +For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(current_epoch))`: -* If `index in attesting_validators(crosslink_committee)`, `state.validator_balances[index] += base_reward(state, index) * total_attesting_balance(crosslink_committee) // total_balance(crosslink_committee))`. -* If `index not in attesting_validators(crosslink_committee)`, `state.validator_balances[index] -= base_reward(state, index)`. +* Let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. +* For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`: + * If `index in attesting_validators(crosslink_committee)`, `state.validator_balances[index] += base_reward(state, index) * total_attesting_balance(crosslink_committee) // total_balance(crosslink_committee))`. + * If `index not in attesting_validators(crosslink_committee)`, `state.validator_balances[index] -= base_reward(state, index)`. ### Ejections From 7b9c033578eeca3d2e55275c681d25ab7dd44d51 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 20:40:30 -0700 Subject: [PATCH 100/136] remove extra line --- specs/core/0_beacon-chain.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 10a41506f..3ce31da06 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1068,7 +1068,6 @@ def generate_seed(state: BeaconState, get_randao_mix(state, randao_mix_epoch) + get_active_index_root(state, epoch) ) - ``` #### `get_beacon_proposer_index` From c9a2f9258e792c24bdc02d39038984b14367a1c8 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Sun, 27 Jan 2019 21:29:47 -0700 Subject: [PATCH 101/136] pr feedback --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 3ce31da06..4d021bc3d 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1689,7 +1689,7 @@ Define the following helpers to process attestation inclusion rewards and inclus If `next_epoch % ETH1_DATA_VOTING_PERIOD == 0`: -* Set `state.latest_eth1_data = eth1_data_vote.data` if `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes`. +* If `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes` (ie. more than half the votes in this voting period were for that value), set `state.latest_eth1_data = eth1_data_vote.data`. * Set `state.eth1_data_votes = []`. ### Justification From 656eae6f6ad85de5f4b9493ca0a4f8ca16d2e261 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Sun, 27 Jan 2019 23:12:37 -0600 Subject: [PATCH 102/136] Minimum slot number, simplify excessive anti-underflow logic Adds a minimum slot number large enough that integer underflows involving epochs and slots will not happen; simplifies some logic that was more complex to handle them. --- specs/core/0_beacon-chain.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4d021bc3d..6e07c3857 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -190,7 +190,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | Name | Value | | - | - | | `GENESIS_FORK_VERSION` | `0` | -| `GENESIS_SLOT` | `0` | +| `GENESIS_SLOT` | `2**19` | | `GENESIS_EPOCH` | `slot_to_epoch(GENESIS_SLOT)` | | `GENESIS_START_SHARD` | `0` | | `FAR_FUTURE_EPOCH` | `2**64 - 1` | @@ -198,6 +198,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `EMPTY_SIGNATURE` | `int_to_bytes96(0)` | | `BLS_WITHDRAWAL_PREFIX_BYTE` | `int_to_bytes1(0)` | +* `GENESIS_SLOT` should be at least as large in terms of time as the largest of the time parameters or state list lengths below (ie. it should be at least as large as any value measured in slots, and at least `EPOCH_LENGTH` times as large as any value measured in epochs). + ### Time parameters | Name | Value | Unit | Duration | @@ -212,11 +214,12 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ### State list lengths -| Name | Value | Unit | -| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | slots | -| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | -| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | -| `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | slots | ~13 hours | +| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | +| `LATEST_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | +| `LATEST_PENALIZED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days | ### Reward and penalty quotients @@ -1036,8 +1039,7 @@ def get_randao_mix(state: BeaconState, """ Returns the randao mix at a recent ``slot``. """ - assert get_current_epoch(state) < epoch + LATEST_RANDAO_MIXES_LENGTH - assert epoch <= get_current_epoch(state) + assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state) return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] ``` @@ -1049,8 +1051,7 @@ def get_active_index_root(state: BeaconState, """ Returns the index root at a recent ``epoch``. """ - assert get_current_epoch(state) < epoch + LATEST_INDEX_ROOTS_LENGTH - assert epoch <= get_current_epoch(state) + assert get_current_epoch(state) - LATEST_INDEX_ROOTS_LENGTH < epoch <= get_current_epoch(state) return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] ``` @@ -1062,10 +1063,9 @@ def generate_seed(state: BeaconState, """ Generate a seed for the given ``epoch``. """ - randao_mix_epoch = epoch + LATEST_RANDAO_MIXES_LENGTH - SEED_LOOKAHEAD return hash( - get_randao_mix(state, randao_mix_epoch) + + get_randao_mix(state, epoch - SEED_LOOKAHEAD) + get_active_index_root(state, epoch) ) ``` From ca3d0b8b534b19646fa9936dc8a25c6686c9972e Mon Sep 17 00:00:00 2001 From: vbuterin Date: Sun, 27 Jan 2019 23:36:38 -0600 Subject: [PATCH 103/136] slot -> epoch --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 6e07c3857..2606389e2 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1037,7 +1037,7 @@ def get_block_root(state: BeaconState, def get_randao_mix(state: BeaconState, epoch: EpochNumber) -> Bytes32: """ - Returns the randao mix at a recent ``slot``. + Returns the randao mix at a recent ``epoch``. """ assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state) return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] From ce73a8327f40fcec81f7582bfed43815c1b417fd Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Mon, 28 Jan 2019 08:26:48 -0700 Subject: [PATCH 104/136] Update specs/core/0_beacon-chain.md Co-Authored-By: djrtwo --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 2606389e2..a2634c11a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -980,7 +980,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, """ epoch = slot_to_epoch(slot) current_epoch = get_current_epoch(state) - previous_epoch = current_epoch - 1 if epoch > GENESIS_EPOCH else current_epoch + previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch next_epoch = current_epoch + 1 assert previous_epoch <= epoch < next_epoch From 90e2a57614f0c3fff4102bc80ffa89949b2b2960 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Mon, 28 Jan 2019 10:28:05 -0600 Subject: [PATCH 105/136] Modular squareroot clarification --- specs/bls_signature.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/bls_signature.md b/specs/bls_signature.md index 4dd479a6a..034de479b 100644 --- a/specs/bls_signature.md +++ b/specs/bls_signature.md @@ -86,13 +86,15 @@ def hash_to_G2(message: bytes32, domain: uint64) -> [uint384]: ### `modular_squareroot` -`modular_squareroot(x)` returns a solution `y` to `y**2 % q == x`, and `None` if none exists. If there are two solutions the one with higher imaginary component is favored; if both solutions have equal imaginary component the one with higher real component is favored. +`modular_squareroot(x)` returns a solution `y` to `y**2 % q == x`, and `None` if none exists. If there are two solutions the one with higher imaginary component is favored; if both solutions have equal imaginary component the one with higher real component is favored (note that this is equivalent to saying that the single solution with either imaginary component > p/2 or imaginary component zero and real component > p/2 is favored). + +The following is a sample implementation; implementers are free to implement modular square roots as they wish. Note that `x2 = -x1` is an _additive modular inverse_ so real and imaginary coefficients remain in `[0 .. q-1]` ```python Fq2_order = q ** 2 - 1 eighth_roots_of_unity = [Fq2([1,1]) ** ((Fq2_order * k) // 8) for k in range(8)] -def modular_squareroot(value: int) -> int: +def modular_squareroot(value: Fq2) -> Fq2: candidate_squareroot = value ** ((Fq2_order + 8) // 16) check = candidate_squareroot ** 2 / value if check in eighth_roots_of_unity[::2]: From 7b7cf7cf758b5c09640052fccb37f07b1aecb0f8 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Mon, 28 Jan 2019 12:01:51 -0600 Subject: [PATCH 106/136] Add link to honest validator implementation doc to main readme --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index f3aba3f64..c5c88daf9 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ Accompanying documents can be found in [specs](specs) and include * [SimpleSerialize (SSZ) spec](specs/simple-serialize.md) * [BLS signature verification](specs/bls_signature.md) * [General test format](specs/test-format.md) +* [Honest validator implementation doc](specs/validator/0_beacon-chain-validator.md) ## Design goals The following are the broad design goals for Ethereum 2.0: From a14600e3a47fddf60910b7693d1b5cc635602dd2 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Mon, 28 Jan 2019 12:24:09 -0600 Subject: [PATCH 107/136] Edits * Removed comment beside `latest_block_roots` "Needed to process attestations, older to newer" because it is no longer accurate * Clarified per-slot processing * Casper slashing -> attester slashing --- specs/core/0_beacon-chain.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 50351106d..c0e247108 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -24,8 +24,8 @@ - [Beacon chain operations](#beacon-chain-operations) - [Proposer slashings](#proposer-slashings) - [`ProposerSlashing`](#proposerslashing) - - [Casper slashings](#casper-slashings) - - [`CasperSlashing`](#casperslashing) + - [Attester slashings](#attester-slashings) + - [`AttesterSlashing`](#attesterslashing) - [`SlashableVoteData`](#slashablevotedata) - [Attestations](#attestations) - [`Attestation`](#attestation) @@ -106,7 +106,7 @@ - [Eth1 data](#eth1-data) - [Operations](#operations) - [Proposer slashings](#proposer-slashings-1) - - [Casper slashings](#casper-slashings-1) + - [Attester slashings](#attester-slashings-1) - [Attestations](#attestations-1) - [Deposits](#deposits-1) - [Exits](#exits-1) @@ -245,7 +245,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | Name | Value | | - | - | | `MAX_PROPOSER_SLASHINGS` | `2**4` (= 16) | -| `MAX_CASPER_SLASHINGS` | `2**4` (= 16) | +| `MAX_ATTESTER_SLASHINGS` | `2**4` (= 16) | | `MAX_ATTESTATIONS` | `2**7` (= 128) | | `MAX_DEPOSITS` | `2**4` (= 16) | | `MAX_EXITS` | `2**4` (= 16) | @@ -285,9 +285,9 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git } ``` -#### Casper slashings +#### Attester slashings -##### `CasperSlashing` +##### `AttesterSlashing` ```python { @@ -444,7 +444,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { 'proposer_slashings': [ProposerSlashing], - 'casper_slashings': [CasperSlashing], + 'attester_slashings': [AttesterSlashing], 'attestations': [Attestation], 'custody_reseeds': [CustodyReseed], 'custody_challenges': [CustodyChallenge], @@ -507,7 +507,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git # Recent state 'latest_crosslinks': [Crosslink], - 'latest_block_roots': ['bytes32'], # Needed to process attestations, older to newer + 'latest_block_roots': ['bytes32'], 'latest_index_roots': ['bytes32'], 'latest_penalized_balances': ['uint64'], # Balances penalized at every withdrawal period 'latest_attestations': [PendingAttestation], @@ -1278,7 +1278,7 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val signature=EMPTY_SIGNATURE, body=BeaconBlockBody( proposer_slashings=[], - casper_slashings=[], + attester_slashings=[], attestations=[], custody_reseeds=[], custody_challenges=[], @@ -1491,7 +1491,7 @@ def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) ## Per-slot processing -Below are the processing steps that happen at every slot. +Below are the processing steps that happen at every slot. If there are skipped slots between a block and its parent block, run the steps in this section once for each skipped slot and then once for the slot containing the new block. ### Slot @@ -1545,14 +1545,14 @@ For each `proposer_slashing` in `block.body.proposer_slashings`: * Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_2.slot), DOMAIN_PROPOSAL))`. * Run `penalize_validator(state, proposer_slashing.proposer_index)`. -#### Casper slashings +#### Attester slashings -Verify that `len(block.body.casper_slashings) <= MAX_CASPER_SLASHINGS`. +Verify that `len(block.body.attester_slashings) <= MAX_ATTESTER_SLASHINGS`. -For each `casper_slashing` in `block.body.casper_slashings`: +For each `attester_slashing` in `block.body.attester_slashings`: -* Let `slashable_vote_data_1 = casper_slashing.slashable_vote_data_1`. -* Let `slashable_vote_data_2 = casper_slashing.slashable_vote_data_2`. +* Let `slashable_vote_data_1 = attester_slashing.slashable_vote_data_1`. +* Let `slashable_vote_data_2 = attester_slashing.slashable_vote_data_2`. * Let `indices(slashable_vote_data) = slashable_vote_data.custody_bit_0_indices + slashable_vote_data.custody_bit_1_indices`. * Let `intersection = [x for x in indices(slashable_vote_data_1) if x in indices(slashable_vote_data_2)]`. * Verify that `len(intersection) >= 1`. From 9b7d94f07babca4bbd1359e4b9276759af0704d0 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 28 Jan 2019 18:31:13 +0000 Subject: [PATCH 108/136] Apply suggestions from code review Co-Authored-By: JustinDrake --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 6a04eb2fb..bb5a0b850 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -361,7 +361,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git # Attestation data 'data': AttestationData, # Custody bit - 'custody_bit': bool, + 'custody_bit': 'bool', } ``` @@ -1624,7 +1624,7 @@ For each `attestation` in `block.body.attestations`: hash_tree_root(AttestationDataAndCustodyBit(data=attestation.data, custody_bit=0b1)), ], signature=attestation.aggregate_signature, - domain=get_domain(state.fork, attestation.data.slot, DOMAIN_ATTESTATION), + domain=get_domain(state.fork, slot_to_epoch(attestation.data.slot), DOMAIN_ATTESTATION), ) ``` From 053b8ec2a54e65471d8a8801c8ee25fd64341af6 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 28 Jan 2019 12:15:43 -0700 Subject: [PATCH 109/136] cleanup state transition organization and notes --- specs/core/0_beacon-chain.md | 1528 +++++++++++++++++----------------- 1 file changed, 773 insertions(+), 755 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index c0e247108..f57ad9a6f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -50,80 +50,86 @@ - [`Eth1Data`](#eth1data) - [`Eth1DataVote`](#eth1datavote) - [Custom Types](#custom-types) + - [Helper functions](#helper-functions) + - [`hash`](#hash) + - [`hash_tree_root`](#hash_tree_root) + - [`slot_to_epoch`](#slot_to_epoch) + - [`get_current_epoch`](#get_current_epoch) + - [`get_epoch_start_slot`](#get_epoch_start_slot) + - [`is_active_validator`](#is_active_validator) + - [`get_active_validator_indices`](#get_active_validator_indices) + - [`shuffle`](#shuffle) + - [`split`](#split) + - [`get_epoch_committee_count`](#get_epoch_committee_count) + - [`get_shuffling`](#get_shuffling) + - [`get_previous_epoch_committee_count`](#get_previous_epoch_committee_count) + - [`get_current_epoch_committee_count`](#get_current_epoch_committee_count) + - [`get_crosslink_committees_at_slot`](#get_crosslink_committees_at_slot) + - [`get_block_root`](#get_block_root) + - [`get_randao_mix`](#get_randao_mix) + - [`get_active_index_root`](#get_active_index_root) + - [`generate_seed`](#generate_seed) + - [`get_beacon_proposer_index`](#get_beacon_proposer_index) + - [`merkle_root`](#merkle_root) + - [`get_attestation_participants`](#get_attestation_participants) + - [`int_to_bytes1`, `int_to_bytes2`, ...](#int_to_bytes1-int_to_bytes2-) + - [`get_effective_balance`](#get_effective_balance) + - [`get_fork_version`](#get_fork_version) + - [`get_domain`](#get_domain) + - [`verify_slashable_vote_data`](#verify_slashable_vote_data) + - [`is_double_vote`](#is_double_vote) + - [`is_surround_vote`](#is_surround_vote) + - [`integer_squareroot`](#integer_squareroot) + - [`get_entry_exit_effect_epoch`](#get_entry_exit_effect_epoch) + - [`bls_verify`](#bls_verify) + - [`bls_verify_multiple`](#bls_verify_multiple) + - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) + - [`validate_proof_of_possession`](#validate_proof_of_possession) + - [`process_deposit`](#process_deposit) + - [`Routines for updating validator status`](#routines-for-updating-validator-status) + - [`activate_validator`](#activate_validator) + - [`initiate_validator_exit`](#initiate_validator_exit) + - [`exit_validator`](#exit_validator) + - [`penalize_validator`](#penalize_validator) + - [`prepare_validator_for_withdrawal`](#prepare_validator_for_withdrawal) - [Ethereum 1.0 deposit contract](#ethereum-10-deposit-contract) - [Deposit arguments](#deposit-arguments) - [Withdrawal credentials](#withdrawal-credentials) - [`Deposit` logs](#deposit-logs) - [`ChainStart` log](#chainstart-log) - [Vyper code](#vyper-code) + - [On startup](#on-startup) - [Beacon chain processing](#beacon-chain-processing) - [Beacon chain fork choice rule](#beacon-chain-fork-choice-rule) - [Beacon chain state transition function](#beacon-chain-state-transition-function) - - [Helper functions](#helper-functions) - - [`hash`](#hash) - - [`hash_tree_root`](#hash_tree_root) - - [`slot_to_epoch`](#slot_to_epoch) - - [`get_current_epoch`](#get_current_epoch) - - [`get_epoch_start_slot`](#get_epoch_start_slot) - - [`is_active_validator`](#is_active_validator) - - [`get_active_validator_indices`](#get_active_validator_indices) - - [`shuffle`](#shuffle) - - [`split`](#split) - - [`get_epoch_committee_count`](#get_epoch_committee_count) - - [`get_shuffling`](#get_shuffling) - - [`get_previous_epoch_committee_count`](#get_previous_epoch_committee_count) - - [`get_current_epoch_committee_count`](#get_current_epoch_committee_count) - - [`get_crosslink_committees_at_slot`](#get_crosslink_committees_at_slot) - - [`get_block_root`](#get_block_root) - - [`get_randao_mix`](#get_randao_mix) - - [`get_active_index_root`](#get_active_index_root) - - [`generate_seed`](#generate_seed) - - [`get_beacon_proposer_index`](#get_beacon_proposer_index) - - [`merkle_root`](#merkle_root) - - [`get_attestation_participants`](#get_attestation_participants) - - [`int_to_bytes1`, `int_to_bytes2`, ...](#int_to_bytes1-int_to_bytes2-) - - [`get_effective_balance`](#get_effective_balance) - - [`get_fork_version`](#get_fork_version) - - [`get_domain`](#get_domain) - - [`verify_slashable_vote_data`](#verify_slashable_vote_data) - - [`is_double_vote`](#is_double_vote) - - [`is_surround_vote`](#is_surround_vote) - - [`integer_squareroot`](#integer_squareroot) - - [`get_entry_exit_effect_epoch`](#get_entry_exit_effect_epoch) - - [`bls_verify`](#bls_verify) - - [`bls_verify_multiple`](#bls_verify_multiple) - - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) - - [On startup](#on-startup) - - [Routine for processing deposits](#routine-for-processing-deposits) - - [Routines for updating validator status](#routines-for-updating-validator-status) - - [Per-slot processing](#per-slot-processing) - - [Slot](#slot) - - [Block roots](#block-roots) - - [Per-block processing](#per-block-processing) - - [Slot](#slot-1) - - [Proposer signature](#proposer-signature) - - [RANDAO](#randao) - - [Eth1 data](#eth1-data) - - [Operations](#operations) - - [Proposer slashings](#proposer-slashings-1) - - [Attester slashings](#attester-slashings-1) - - [Attestations](#attestations-1) - - [Deposits](#deposits-1) - - [Exits](#exits-1) - - [Custody](#custody) - - [Per-epoch processing](#per-epoch-processing) - - [Helpers](#helpers) - - [Eth1 data](#eth1-data-1) - - [Justification](#justification) - - [Crosslinks](#crosslinks) - - [Rewards and penalties](#rewards-and-penalties) - - [Justification and finalization](#justification-and-finalization) - - [Attestation inclusion](#attestation-inclusion) - - [Crosslinks](#crosslinks-1) - - [Ejections](#ejections) - - [Validator registry and shuffling seed data](#validator-registry-and-shuffling-seed-data) - - [Final updates](#final-updates) - - [State root processing](#state-root-processing) + - [Per-slot processing](#per-slot-processing) + - [Slot](#slot) + - [Block roots](#block-roots) + - [Per-block processing](#per-block-processing) + - [Slot](#slot-1) + - [Proposer signature](#proposer-signature) + - [RANDAO](#randao) + - [Eth1 data](#eth1-data) + - [Operations](#operations) + - [Proposer slashings](#proposer-slashings-1) + - [Attester slashings](#attester-slashings-1) + - [Attestations](#attestations-1) + - [Deposits](#deposits-1) + - [Exits](#exits-1) + - [Custody](#custody) + - [Per-epoch processing](#per-epoch-processing) + - [Helpers](#helpers) + - [Eth1 data](#eth1-data-1) + - [Justification](#justification) + - [Crosslinks](#crosslinks) + - [Rewards and penalties](#rewards-and-penalties) + - [Justification and finalization](#justification-and-finalization) + - [Attestation inclusion](#attestation-inclusion) + - [Crosslinks](#crosslinks-1) + - [Ejections](#ejections) + - [Validator registry and shuffling seed data](#validator-registry-and-shuffling-seed-data) + - [Final updates](#final-updates) + - [State root verification](#state-root-verification) - [References](#references) - [Normative](#normative) - [Informative](#informative) @@ -622,6 +628,616 @@ We define the following Python custom types for type hinting and readability: | `BLSPubkey` | 48-byte data | a public key in BLS signature scheme | | `BLSSignature` | 96-byte data | a signature in BLS signature scheme | +## Helper functions + +Note: The definitions below are for specification purposes and are not necessarily optimal implementations. + +### `hash` + +The hash function is denoted by `hash`. In Phase 0 the beacon chain is deployed with the same hash function as Ethereum 1.0, i.e. Keccak-256 (also incorrectly known as SHA3). + +Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethereum 2.0 deployment phase. + +### `hash_tree_root` + +`def hash_tree_root(object: SSZSerializable) -> Bytes32` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). + +### `slot_to_epoch` + +```python +def slot_to_epoch(slot: SlotNumber) -> EpochNumber: + return slot // EPOCH_LENGTH +``` + +### `get_current_epoch` + +```python +def get_current_epoch(state: BeaconState) -> EpochNumber: + return slot_to_epoch(state.slot) +``` + +### `get_epoch_start_slot` + +```python +def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: + return epoch * EPOCH_LENGTH +``` + +### `is_active_validator` +```python +def is_active_validator(validator: Validator, epoch: EpochNumber) -> bool: + """ + Checks if ``validator`` is active. + """ + return validator.activation_epoch <= epoch < validator.exit_epoch +``` + +### `get_active_validator_indices` + +```python +def get_active_validator_indices(validators: List[Validator], epoch: EpochNumber) -> List[ValidatorIndex]: + """ + Gets indices of active validators from ``validators``. + """ + return [i for i, v in enumerate(validators) if is_active_validator(v, epoch)] +``` + +### `shuffle` + +```python +def shuffle(values: List[Any], seed: Bytes32) -> List[Any]: + """ + Returns the shuffled ``values`` with ``seed`` as entropy. + """ + values_count = len(values) + + # Entropy is consumed from the seed in 3-byte (24 bit) chunks. + rand_bytes = 3 + # The highest possible result of the RNG. + rand_max = 2 ** (rand_bytes * 8) - 1 + + # The range of the RNG places an upper-bound on the size of the list that + # may be shuffled. It is a logic error to supply an oversized list. + assert values_count < rand_max + + output = [x for x in values] + source = seed + index = 0 + while index < values_count - 1: + # Re-hash the `source` to obtain a new pattern of bytes. + source = hash(source) + # Iterate through the `source` bytes in 3-byte chunks. + for position in range(0, 32 - (32 % rand_bytes), rand_bytes): + # Determine the number of indices remaining in `values` and exit + # once the last index is reached. + remaining = values_count - index + if remaining == 1: + break + + # Read 3-bytes of `source` as a 24-bit big-endian integer. + sample_from_source = int.from_bytes(source[position:position + rand_bytes], 'big') + + # Sample values greater than or equal to `sample_max` will cause + # modulo bias when mapped into the `remaining` range. + sample_max = rand_max - rand_max % remaining + + # Perform a swap if the consumed entropy will not cause modulo bias. + if sample_from_source < sample_max: + # Select a replacement index for the current index. + replacement_position = (sample_from_source % remaining) + index + # Swap the current index with the replacement index. + output[index], output[replacement_position] = output[replacement_position], output[index] + index += 1 + else: + # The sample causes modulo bias. A new sample should be read. + pass + + return output +``` + +### `split` + +```python +def split(values: List[Any], split_count: int) -> List[List[Any]]: + """ + Splits ``values`` into ``split_count`` pieces. + """ + list_length = len(values) + return [ + values[(list_length * i // split_count): (list_length * (i + 1) // split_count)] + for i in range(split_count) + ] +``` + +### `get_epoch_committee_count` + +```python +def get_epoch_committee_count(active_validator_count: int) -> int: + return max( + 1, + min( + SHARD_COUNT // EPOCH_LENGTH, + active_validator_count // EPOCH_LENGTH // TARGET_COMMITTEE_SIZE, + ) + ) * EPOCH_LENGTH +``` + +### `get_shuffling` + +```python +def get_shuffling(seed: Bytes32, + validators: List[Validator], + epoch: EpochNumber) -> List[List[ValidatorIndex]] + """ + Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. + Returns a list of ``committees_per_epoch`` committees where each + committee is itself a list of validator indices. + """ + + active_validator_indices = get_active_validator_indices(validators, epoch) + + committees_per_epoch = get_epoch_committee_count(len(active_validator_indices)) + + # Shuffle + seed = xor(seed, int_to_bytes32(epoch)) + shuffled_active_validator_indices = shuffle(active_validator_indices, seed) + + # Split the shuffled list into committees_per_epoch pieces + return split(shuffled_active_validator_indices, committees_per_epoch) +``` + +**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= get_current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. + +**Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. + +#### `get_previous_epoch_committee_count` + +```python +def get_previous_epoch_committee_count(state: BeaconState) -> int: + previous_active_validators = get_active_validator_indices( + state.validator_registry, + state.previous_calculation_epoch, + ) + return get_epoch_committee_count(len(previous_active_validators)) +``` + +### `get_current_epoch_committee_count` + +```python +def get_current_epoch_committee_count(state: BeaconState) -> int: + current_active_validators = get_active_validator_indices( + state.validator_registry, + state.current_calculation_epoch, + ) + return get_epoch_committee_count(len(current_active_validators)) +``` + +### `get_crosslink_committees_at_slot` + +```python +def get_crosslink_committees_at_slot(state: BeaconState, + slot: SlotNumber) -> List[Tuple[List[ValidatorIndex], ShardNumber]]: + """ + Returns the list of ``(committee, shard)`` tuples for the ``slot``. + """ + epoch = slot_to_epoch(slot) + current_epoch = get_current_epoch(state) + previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch + next_epoch = current_epoch + 1 + + assert previous_epoch <= epoch < next_epoch + + if epoch < current_epoch: + committees_per_epoch = get_previous_epoch_committee_count(state) + seed = state.previous_epoch_seed + shuffling_epoch = state.previous_calculation_epoch + shuffling_start_shard = state.previous_epoch_start_shard + else: + committees_per_epoch = get_current_epoch_committee_count(state) + seed = state.current_epoch_seed + shuffling_epoch = state.current_calculation_epoch + shuffling_start_shard = state.current_epoch_start_shard + + shuffling = get_shuffling( + seed, + state.validator_registry, + shuffling_epoch, + ) + offset = slot % EPOCH_LENGTH + committees_per_slot = committees_per_epoch // EPOCH_LENGTH + slot_start_shard = (shuffling_start_shard + committees_per_slot * offset) % SHARD_COUNT + + return [ + ( + shuffling[committees_per_slot * offset + i], + (slot_start_shard + i) % SHARD_COUNT, + ) + for i in range(committees_per_slot) + ] +``` + +**Note**: we plan to replace the shuffling algorithm with a pointwise-evaluable shuffle (see https://github.com/ethereum/eth2.0-specs/issues/323), which will allow calculation of the committees for each slot individually. + +### `get_block_root` + +```python +def get_block_root(state: BeaconState, + slot: SlotNumber) -> Bytes32: + """ + Returns the block root at a recent ``slot``. + """ + assert state.slot <= slot + LATEST_BLOCK_ROOTS_LENGTH + assert slot < state.slot + return state.latest_block_roots[slot % LATEST_BLOCK_ROOTS_LENGTH] +``` + +`get_block_root(_, s)` should always return `hash_tree_root` of the block in the beacon chain at slot `s`, and `get_crosslink_committees_at_slot(_, s)` should not change unless the [validator](#dfn-validator) registry changes. + +### `get_randao_mix` + +```python +def get_randao_mix(state: BeaconState, + epoch: EpochNumber) -> Bytes32: + """ + Returns the randao mix at a recent ``epoch``. + """ + assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state) + return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] +``` + +### `get_active_index_root` + +```python +def get_active_index_root(state: BeaconState, + epoch: EpochNumber) -> Bytes32: + """ + Returns the index root at a recent ``epoch``. + """ + assert get_current_epoch(state) - LATEST_INDEX_ROOTS_LENGTH < epoch <= get_current_epoch(state) + return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] +``` + +### `generate_seed` + +```python +def generate_seed(state: BeaconState, + epoch: EpochNumber) -> Bytes32: + """ + Generate a seed for the given ``epoch``. + """ + + return hash( + get_randao_mix(state, epoch - SEED_LOOKAHEAD) + + get_active_index_root(state, epoch) + ) +``` + +### `get_beacon_proposer_index` + +```python +def get_beacon_proposer_index(state: BeaconState, + slot: SlotNumber) -> ValidatorIndex: + """ + Returns the beacon proposer index for the ``slot``. + """ + first_committee, _ = get_crosslink_committees_at_slot(state, slot)[0] + return first_committee[slot % len(first_committee)] +``` + +### `merkle_root` + +```python +def merkle_root(values: List[Bytes32]) -> Bytes32: + """ + Merkleize ``values`` (where ``len(values)`` is a power of two) and return the Merkle root. + """ + o = [0] * len(values) + values + for i in range(len(values) - 1, 0, -1): + o[i] = hash(o[i * 2] + o[i * 2 + 1]) + return o[1] +``` + +### `get_attestation_participants` + +```python +def get_attestation_participants(state: BeaconState, + attestation_data: AttestationData, + aggregation_bitfield: bytes) -> List[ValidatorIndex]: + """ + Returns the participant indices at for the ``attestation_data`` and ``aggregation_bitfield``. + """ + + # Find the committee in the list with the desired shard + crosslink_committees = get_crosslink_committees_at_slot(state, attestation_data.slot) + + assert attestation_data.shard in [shard for _, shard in crosslink_committees] + crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0] + assert len(aggregation_bitfield) == (len(crosslink_committee) + 7) // 8 + + # Find the participating attesters in the committee + participants = [] + for i, validator_index in enumerate(crosslink_committee): + aggregation_bit = (aggregation_bitfield[i // 8] >> (7 - (i % 8))) % 2 + if aggregation_bit == 1: + participants.append(validator_index) + return participants +``` + +### `int_to_bytes1`, `int_to_bytes2`, ... + +`int_to_bytes1(x): return x.to_bytes(1, 'big')`, `int_to_bytes2(x): return x.to_bytes(2, 'big')`, and so on for all integers, particularly 1, 2, 3, 4, 8, 32, 48, 96. + +### `get_effective_balance` + +```python +def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: + """ + Returns the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``. + """ + return min(state.validator_balances[index], MAX_DEPOSIT_AMOUNT) +``` + +### `get_fork_version` + +```python +def get_fork_version(fork: Fork, + epoch: EpochNumber) -> int: + if epoch < fork.epoch: + return fork.previous_version + else: + return fork.current_version +``` + +### `get_domain` + +```python +def get_domain(fork: Fork, + epoch: EpochNumber, + domain_type: int) -> int: + return get_fork_version( + fork, + epoch, + ) * 2**32 + domain_type +``` + +### `verify_slashable_vote_data` + +```python +def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) -> bool: + if len(vote_data.custody_bit_0_indices) + len(vote_data.custody_bit_1_indices) > MAX_CASPER_VOTES: + return False + + return bls_verify_multiple( + pubkeys=[ + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_0_indices]), + bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_1_indices]), + ], + messages=[ + hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, False)), + hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, True)), + ], + signature=vote_data.aggregate_signature, + domain=get_domain( + state.fork, + slot_to_epoch(vote_data.data.slot), + DOMAIN_ATTESTATION, + ), + ) +``` + +### `is_double_vote` + +```python +def is_double_vote(attestation_data_1: AttestationData, + attestation_data_2: AttestationData) -> bool + """ + Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. + Returns True if the provided ``AttestationData`` are slashable + due to a 'double vote'. + """ + target_epoch_1 = slot_to_epoch(attestation_data_1.slot) + target_epoch_2 = slot_to_epoch(attestation_data_2.slot) + return target_epoch_1 == target_epoch_2 +``` + +### `is_surround_vote` + +```python +def is_surround_vote(attestation_data_1: AttestationData, + attestation_data_2: AttestationData) -> bool: + """ + Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. + Returns True if the provided ``AttestationData`` are slashable + due to a 'surround vote'. + Note: parameter order matters as this function only checks + that ``attestation_data_1`` surrounds ``attestation_data_2``. + """ + source_epoch_1 = attestation_data_1.justified_epoch + source_epoch_2 = attestation_data_2.justified_epoch + target_epoch_1 = slot_to_epoch(attestation_data_1.slot) + target_epoch_2 = slot_to_epoch(attestation_data_2.slot) + return ( + (source_epoch_1 < source_epoch_2) and + (source_epoch_2 + 1 == target_epoch_2) and + (target_epoch_2 < target_epoch_1) + ) +``` + +### `integer_squareroot` + +```python +def integer_squareroot(n: int) -> int: + """ + The largest integer ``x`` such that ``x**2`` is less than or equal to ``n``. + """ + assert n >= 0 + x = n + y = (x + 1) // 2 + while y < x: + x = y + y = (x + n // x) // 2 + return x +``` + +### `get_entry_exit_effect_epoch` + +```python +def get_entry_exit_effect_epoch(epoch: EpochNumber) -> EpochNumber: + """ + An entry or exit triggered in the ``epoch`` given by the input takes effect at + the epoch given by the output. + """ + return epoch + 1 + ENTRY_EXIT_DELAY +``` + +### `bls_verify` + +`bls_verify` is a function for verifying a BLS signature, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify). + +### `bls_verify_multiple` + +`bls_verify_multiple` is a function for verifying a BLS signature constructed from multiple messages, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify_multiple). + +### `bls_aggregate_pubkeys` + +`bls_aggregate_pubkeys` is a function for aggregating multiple BLS public keys into a single aggregate key, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_aggregate_pubkeys). + + +### `validate_proof_of_possession` + +```python +def validate_proof_of_possession(state: BeaconState, + pubkey: BLSPubkey, + proof_of_possession: BLSSignature, + withdrawal_credentials: Bytes32) -> bool: + proof_of_possession_data = DepositInput( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + proof_of_possession=EMPTY_SIGNATURE, + ) + + return bls_verify( + pubkey=pubkey, + message=hash_tree_root(proof_of_possession_data), + signature=proof_of_possession, + domain=get_domain( + state.fork, + get_current_epoch(state), + DOMAIN_DEPOSIT, + ) + ) +``` + +### `process_deposit` + +Used to add a [validator](#dfn-validator) or top up an existing [validator](#dfn-validator)'s balance by some `deposit` amount: + +```python +def process_deposit(state: BeaconState, + pubkey: BLSPubkey, + amount: Gwei, + proof_of_possession: BLSSignature, + withdrawal_credentials: Bytes32) -> None: + """ + Process a deposit from Ethereum 1.0. + Note that this function mutates ``state``. + """ + # Validate the given `proof_of_possession` + assert validate_proof_of_possession( + state, + pubkey, + proof_of_possession, + withdrawal_credentials, + ) + + validator_pubkeys = [v.pubkey for v in state.validator_registry] + + if pubkey not in validator_pubkeys: + # Add new validator + validator = Validator( + pubkey=pubkey, + withdrawal_credentials=withdrawal_credentials, + activation_epoch=FAR_FUTURE_EPOCH, + exit_epoch=FAR_FUTURE_EPOCH, + withdrawal_epoch=FAR_FUTURE_EPOCH, + penalized_epoch=FAR_FUTURE_EPOCH, + exit_count=0, + status_flags=0, + latest_custody_reseed_slot=GENESIS_SLOT, + penultimate_custody_reseed_slot=GENESIS_SLOT, + ) + + # Note: In phase 2 registry indices that have been withdrawn for a long time will be recycled. + state.validator_registry.append(validator) + state.validator_balances.append(amount) + else: + # Increase balance by deposit amount + index = validator_pubkeys.index(pubkey) + assert state.validator_registry[index].withdrawal_credentials == withdrawal_credentials + + state.validator_balances[index] += amount +``` + +### Routines for updating validator status + +Note: All functions in this section mutate `state`. + +#### `activate_validator` + +```python +def activate_validator(state: BeaconState, index: ValidatorIndex, genesis: bool) -> None: + validator = state.validator_registry[index] + + validator.activation_epoch = GENESIS_EPOCH if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) +``` + +#### `initiate_validator_exit` + +```python +def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: + validator = state.validator_registry[index] + validator.status_flags |= INITIATED_EXIT +``` + +#### `exit_validator` + +```python +def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: + validator = state.validator_registry[index] + + # The following updates only occur if not previous exited + if validator.exit_epoch <= get_entry_exit_effect_epoch(get_current_epoch(state)): + return + + validator.exit_epoch = get_entry_exit_effect_epoch(get_current_epoch(state)) + + state.validator_registry_exit_count += 1 + validator.exit_count = state.validator_registry_exit_count +``` + +#### `penalize_validator` + +```python +def penalize_validator(state: BeaconState, index: ValidatorIndex) -> None: + exit_validator(state, index) + validator = state.validator_registry[index] + state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) + + whistleblower_index = get_beacon_proposer_index(state, state.slot) + whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT + state.validator_balances[whistleblower_index] += whistleblower_reward + state.validator_balances[index] -= whistleblower_reward + validator.penalized_epoch = get_current_epoch(state) +``` + +#### `prepare_validator_for_withdrawal` + +```python +def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) -> None: + validator = state.validator_registry[index] + validator.status_flags |= WITHDRAWABLE +``` + ## Ethereum 1.0 deposit contract The initial deployment phases of Ethereum 2.0 are implemented without consensus changes to Ethereum 1.0. A deposit contract at address `DEPOSIT_CONTRACT_ADDRESS` is added to Ethereum 1.0 for deposits of ETH to the beacon chain. Validator balances will be withdrawable to the shards in phase 2, i.e. when the EVM2.0 is deployed and the shards have state. @@ -715,553 +1331,7 @@ def get_branch(leaf: uint256) -> bytes32[32]: # size is DEPOSIT_CONTRACT_TREE_DE return branch ``` -## Beacon chain processing - -The beacon chain is the system chain for Ethereum 2.0. The main responsibilities of the beacon chain are: - -* Store and maintain the registry of [validators](#dfn-validator) -* Process crosslinks (see above) -* Process its per-slot consensus, as well as the finality gadget - -Processing the beacon chain is similar to processing the Ethereum 1.0 chain. Clients download and process blocks, and maintain a view of what is the current "canonical chain", terminating at the current "head". However, because of the beacon chain's relationship with Ethereum 1.0, and because it is a proof-of-stake chain, there are differences. - -For a beacon chain block, `block`, to be processed by a node, the following conditions must be met: - -* The parent block with root `block.parent_root` has been processed and accepted. -* The node has processed its `state` up to slot, `block.slot - 1`. -* An Ethereum 1.0 block pointed to by the `state.latest_eth1_data.block_hash` has been processed and accepted. -* The node's local clock time is greater than or equal to `state.genesis_time + block.slot * SLOT_DURATION`. - -If these conditions are not met, the client should delay processing the beacon block until the conditions are all satisfied. - -Beacon block production is significantly different because of the proof of stake mechanism. A client simply checks what it thinks is the canonical chain when it should create a block, and looks up what its slot number is; when the slot arrives, it either proposes or attests to a block as required. Note that this requires each node to have a clock that is roughly (i.e. within `SLOT_DURATION` seconds) synchronized with the other nodes. - -### Beacon chain fork choice rule - -The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a [validator](#dfn-validator) `v` subjectively calculates the beacon chain head as follows. - -* Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not yet included in any chain are still included in `store`. -* Let `finalized_head` be the finalized block with the highest epoch. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) -* Let `justified_head` be the descendant of `finalized_head` with the highest epoch that has been justified for at least 1 epoch. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. -* Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. -* Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. -* Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. -* Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the child blocks of the given `block`. -* Let `justified_head_state` be the resulting `BeaconState` object from processing the chain up to the `justified_head`. -* The `head` is `lmd_ghost(store, justified_head_state, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. - -```python -def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) -> BeaconBlock: - validators = start_state.validator_registry - active_validators = [ - validators[i] - for i in get_active_validator_indices(validators, start_state.slot) - ] - attestation_targets = [ - get_latest_attestation_target(store, validator) - for validator in active_validators - ] - - def get_vote_count(block: BeaconBlock) -> int: - return len([ - target - for target in attestation_targets - if get_ancestor(store, target, block.slot) == block - ]) - - head = start_block - while 1: - children = get_children(store, head) - if len(children) == 0: - return head - head = max(children, key=get_vote_count) -``` - -## Beacon chain state transition function - -We now define the state transition function. At a high level the state transition is made up of three parts: - -1. The per-slot transitions, which happens every slot. -2. The per-block transitions, which happens at every block. -3. The per-epoch transitions, which happens at every epoch boundary (i.e. `(state.slot + 1) % EPOCH_LENGTH == 0`). - -The per-slot transitions focus on the slot counter and block roots records updates; the per-block transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-block activity in the `BeaconState`; the per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. - -### Helper functions - -Note: The definitions below are for specification purposes and are not necessarily optimal implementations. - -#### `hash` - -The hash function is denoted by `hash`. In Phase 0 the beacon chain is deployed with the same hash function as Ethereum 1.0, i.e. Keccak-256 (also incorrectly known as SHA3). - -Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethereum 2.0 deployment phase. - -#### `hash_tree_root` - -`def hash_tree_root(object: SSZSerializable) -> Bytes32` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). - -#### `slot_to_epoch` - -```python -def slot_to_epoch(slot: SlotNumber) -> EpochNumber: - return slot // EPOCH_LENGTH -``` - -#### `get_current_epoch` - -```python -def get_current_epoch(state: BeaconState) -> EpochNumber: - return slot_to_epoch(state.slot) -``` - -#### `get_epoch_start_slot` - -```python -def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: - return epoch * EPOCH_LENGTH -``` - -#### `is_active_validator` -```python -def is_active_validator(validator: Validator, epoch: EpochNumber) -> bool: - """ - Checks if ``validator`` is active. - """ - return validator.activation_epoch <= epoch < validator.exit_epoch -``` - -#### `get_active_validator_indices` - -```python -def get_active_validator_indices(validators: List[Validator], epoch: EpochNumber) -> List[ValidatorIndex]: - """ - Gets indices of active validators from ``validators``. - """ - return [i for i, v in enumerate(validators) if is_active_validator(v, epoch)] -``` - -#### `shuffle` - -```python -def shuffle(values: List[Any], seed: Bytes32) -> List[Any]: - """ - Returns the shuffled ``values`` with ``seed`` as entropy. - """ - values_count = len(values) - - # Entropy is consumed from the seed in 3-byte (24 bit) chunks. - rand_bytes = 3 - # The highest possible result of the RNG. - rand_max = 2 ** (rand_bytes * 8) - 1 - - # The range of the RNG places an upper-bound on the size of the list that - # may be shuffled. It is a logic error to supply an oversized list. - assert values_count < rand_max - - output = [x for x in values] - source = seed - index = 0 - while index < values_count - 1: - # Re-hash the `source` to obtain a new pattern of bytes. - source = hash(source) - # Iterate through the `source` bytes in 3-byte chunks. - for position in range(0, 32 - (32 % rand_bytes), rand_bytes): - # Determine the number of indices remaining in `values` and exit - # once the last index is reached. - remaining = values_count - index - if remaining == 1: - break - - # Read 3-bytes of `source` as a 24-bit big-endian integer. - sample_from_source = int.from_bytes(source[position:position + rand_bytes], 'big') - - # Sample values greater than or equal to `sample_max` will cause - # modulo bias when mapped into the `remaining` range. - sample_max = rand_max - rand_max % remaining - - # Perform a swap if the consumed entropy will not cause modulo bias. - if sample_from_source < sample_max: - # Select a replacement index for the current index. - replacement_position = (sample_from_source % remaining) + index - # Swap the current index with the replacement index. - output[index], output[replacement_position] = output[replacement_position], output[index] - index += 1 - else: - # The sample causes modulo bias. A new sample should be read. - pass - - return output -``` - -#### `split` - -```python -def split(values: List[Any], split_count: int) -> List[List[Any]]: - """ - Splits ``values`` into ``split_count`` pieces. - """ - list_length = len(values) - return [ - values[(list_length * i // split_count): (list_length * (i + 1) // split_count)] - for i in range(split_count) - ] -``` - -#### `get_epoch_committee_count` - -```python -def get_epoch_committee_count(active_validator_count: int) -> int: - return max( - 1, - min( - SHARD_COUNT // EPOCH_LENGTH, - active_validator_count // EPOCH_LENGTH // TARGET_COMMITTEE_SIZE, - ) - ) * EPOCH_LENGTH -``` - -#### `get_shuffling` - -```python -def get_shuffling(seed: Bytes32, - validators: List[Validator], - epoch: EpochNumber) -> List[List[ValidatorIndex]] - """ - Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. - Returns a list of ``committees_per_epoch`` committees where each - committee is itself a list of validator indices. - """ - - active_validator_indices = get_active_validator_indices(validators, epoch) - - committees_per_epoch = get_epoch_committee_count(len(active_validator_indices)) - - # Shuffle - seed = xor(seed, int_to_bytes32(epoch)) - shuffled_active_validator_indices = shuffle(active_validator_indices, seed) - - # Split the shuffled list into committees_per_epoch pieces - return split(shuffled_active_validator_indices, committees_per_epoch) -``` - -**Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= get_current_epoch(state) + ENTRY_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. - -**Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. - -#### `get_previous_epoch_committee_count` - -```python -def get_previous_epoch_committee_count(state: BeaconState) -> int: - previous_active_validators = get_active_validator_indices( - state.validator_registry, - state.previous_calculation_epoch, - ) - return get_epoch_committee_count(len(previous_active_validators)) -``` - -#### `get_current_epoch_committee_count` - -```python -def get_current_epoch_committee_count(state: BeaconState) -> int: - current_active_validators = get_active_validator_indices( - state.validator_registry, - state.current_calculation_epoch, - ) - return get_epoch_committee_count(len(current_active_validators)) -``` - -#### `get_crosslink_committees_at_slot` - -```python -def get_crosslink_committees_at_slot(state: BeaconState, - slot: SlotNumber) -> List[Tuple[List[ValidatorIndex], ShardNumber]]: - """ - Returns the list of ``(committee, shard)`` tuples for the ``slot``. - """ - epoch = slot_to_epoch(slot) - current_epoch = get_current_epoch(state) - previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch - next_epoch = current_epoch + 1 - - assert previous_epoch <= epoch < next_epoch - - if epoch < current_epoch: - committees_per_epoch = get_previous_epoch_committee_count(state) - seed = state.previous_epoch_seed - shuffling_epoch = state.previous_calculation_epoch - shuffling_start_shard = state.previous_epoch_start_shard - else: - committees_per_epoch = get_current_epoch_committee_count(state) - seed = state.current_epoch_seed - shuffling_epoch = state.current_calculation_epoch - shuffling_start_shard = state.current_epoch_start_shard - - shuffling = get_shuffling( - seed, - state.validator_registry, - shuffling_epoch, - ) - offset = slot % EPOCH_LENGTH - committees_per_slot = committees_per_epoch // EPOCH_LENGTH - slot_start_shard = (shuffling_start_shard + committees_per_slot * offset) % SHARD_COUNT - - return [ - ( - shuffling[committees_per_slot * offset + i], - (slot_start_shard + i) % SHARD_COUNT, - ) - for i in range(committees_per_slot) - ] -``` - -**Note**: we plan to replace the shuffling algorithm with a pointwise-evaluable shuffle (see https://github.com/ethereum/eth2.0-specs/issues/323), which will allow calculation of the committees for each slot individually. - -#### `get_block_root` - -```python -def get_block_root(state: BeaconState, - slot: SlotNumber) -> Bytes32: - """ - Returns the block root at a recent ``slot``. - """ - assert state.slot <= slot + LATEST_BLOCK_ROOTS_LENGTH - assert slot < state.slot - return state.latest_block_roots[slot % LATEST_BLOCK_ROOTS_LENGTH] -``` - -`get_block_root(_, s)` should always return `hash_tree_root` of the block in the beacon chain at slot `s`, and `get_crosslink_committees_at_slot(_, s)` should not change unless the [validator](#dfn-validator) registry changes. - -#### `get_randao_mix` - -```python -def get_randao_mix(state: BeaconState, - epoch: EpochNumber) -> Bytes32: - """ - Returns the randao mix at a recent ``epoch``. - """ - assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state) - return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] -``` - -#### `get_active_index_root` - -```python -def get_active_index_root(state: BeaconState, - epoch: EpochNumber) -> Bytes32: - """ - Returns the index root at a recent ``epoch``. - """ - assert get_current_epoch(state) - LATEST_INDEX_ROOTS_LENGTH < epoch <= get_current_epoch(state) - return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] -``` - -#### `generate_seed` - -```python -def generate_seed(state: BeaconState, - epoch: EpochNumber) -> Bytes32: - """ - Generate a seed for the given ``epoch``. - """ - - return hash( - get_randao_mix(state, epoch - SEED_LOOKAHEAD) + - get_active_index_root(state, epoch) - ) -``` - -#### `get_beacon_proposer_index` - -```python -def get_beacon_proposer_index(state: BeaconState, - slot: SlotNumber) -> ValidatorIndex: - """ - Returns the beacon proposer index for the ``slot``. - """ - first_committee, _ = get_crosslink_committees_at_slot(state, slot)[0] - return first_committee[slot % len(first_committee)] -``` - -#### `merkle_root` - -```python -def merkle_root(values: List[Bytes32]) -> Bytes32: - """ - Merkleize ``values`` (where ``len(values)`` is a power of two) and return the Merkle root. - """ - o = [0] * len(values) + values - for i in range(len(values) - 1, 0, -1): - o[i] = hash(o[i * 2] + o[i * 2 + 1]) - return o[1] -``` - -#### `get_attestation_participants` - -```python -def get_attestation_participants(state: BeaconState, - attestation_data: AttestationData, - aggregation_bitfield: bytes) -> List[ValidatorIndex]: - """ - Returns the participant indices at for the ``attestation_data`` and ``aggregation_bitfield``. - """ - - # Find the committee in the list with the desired shard - crosslink_committees = get_crosslink_committees_at_slot(state, attestation_data.slot) - - assert attestation_data.shard in [shard for _, shard in crosslink_committees] - crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0] - assert len(aggregation_bitfield) == (len(crosslink_committee) + 7) // 8 - - # Find the participating attesters in the committee - participants = [] - for i, validator_index in enumerate(crosslink_committee): - aggregation_bit = (aggregation_bitfield[i // 8] >> (7 - (i % 8))) % 2 - if aggregation_bit == 1: - participants.append(validator_index) - return participants -``` - -#### `int_to_bytes1`, `int_to_bytes2`, ... - -`int_to_bytes1(x): return x.to_bytes(1, 'big')`, `int_to_bytes2(x): return x.to_bytes(2, 'big')`, and so on for all integers, particularly 1, 2, 3, 4, 8, 32, 48, 96. - -#### `get_effective_balance` - -```python -def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: - """ - Returns the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``. - """ - return min(state.validator_balances[index], MAX_DEPOSIT_AMOUNT) -``` - -#### `get_fork_version` - -```python -def get_fork_version(fork: Fork, - epoch: EpochNumber) -> int: - if epoch < fork.epoch: - return fork.previous_version - else: - return fork.current_version -``` - -#### `get_domain` - -```python -def get_domain(fork: Fork, - epoch: EpochNumber, - domain_type: int) -> int: - return get_fork_version( - fork, - epoch, - ) * 2**32 + domain_type -``` - -#### `verify_slashable_vote_data` - -```python -def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) -> bool: - if len(vote_data.custody_bit_0_indices) + len(vote_data.custody_bit_1_indices) > MAX_CASPER_VOTES: - return False - - return bls_verify_multiple( - pubkeys=[ - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_0_indices]), - bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in vote_data.custody_bit_1_indices]), - ], - messages=[ - hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, False)), - hash_tree_root(AttestationDataAndCustodyBit(vote_data.data, True)), - ], - signature=vote_data.aggregate_signature, - domain=get_domain( - state.fork, - slot_to_epoch(vote_data.data.slot), - DOMAIN_ATTESTATION, - ), - ) -``` - -#### `is_double_vote` - -```python -def is_double_vote(attestation_data_1: AttestationData, - attestation_data_2: AttestationData) -> bool - """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable - due to a 'double vote'. - """ - target_epoch_1 = slot_to_epoch(attestation_data_1.slot) - target_epoch_2 = slot_to_epoch(attestation_data_2.slot) - return target_epoch_1 == target_epoch_2 -``` - -#### `is_surround_vote` - -```python -def is_surround_vote(attestation_data_1: AttestationData, - attestation_data_2: AttestationData) -> bool: - """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable - due to a 'surround vote'. - Note: parameter order matters as this function only checks - that ``attestation_data_1`` surrounds ``attestation_data_2``. - """ - source_epoch_1 = attestation_data_1.justified_epoch - source_epoch_2 = attestation_data_2.justified_epoch - target_epoch_1 = slot_to_epoch(attestation_data_1.slot) - target_epoch_2 = slot_to_epoch(attestation_data_2.slot) - return ( - (source_epoch_1 < source_epoch_2) and - (source_epoch_2 + 1 == target_epoch_2) and - (target_epoch_2 < target_epoch_1) - ) -``` - -#### `integer_squareroot` - -```python -def integer_squareroot(n: int) -> int: - """ - The largest integer ``x`` such that ``x**2`` is less than or equal to ``n``. - """ - assert n >= 0 - x = n - y = (x + 1) // 2 - while y < x: - x = y - y = (x + n // x) // 2 - return x -``` - -#### `get_entry_exit_effect_epoch` - -```python -def get_entry_exit_effect_epoch(epoch: EpochNumber) -> EpochNumber: - """ - An entry or exit triggered in the ``epoch`` given by the input takes effect at - the epoch given by the output. - """ - return epoch + 1 + ENTRY_EXIT_DELAY -``` - -#### `bls_verify` - -`bls_verify` is a function for verifying a BLS signature, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify). - -#### `bls_verify_multiple` - -`bls_verify_multiple` is a function for verifying a BLS signature constructed from multiple messages, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_verify_multiple). - -#### `bls_aggregate_pubkeys` - -`bls_aggregate_pubkeys` is a function for aggregating multiple BLS public keys into a single aggregate key, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_aggregate_pubkeys). - -### On startup +## On startup A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following values. Other validity rules (e.g. requiring a signature) do not apply. @@ -1364,173 +1434,121 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], return state ``` -### Routine for processing deposits +## Beacon chain processing -First, a helper function: +The beacon chain is the system chain for Ethereum 2.0. The main responsibilities of the beacon chain are: + +* Store and maintain the registry of [validators](#dfn-validator) +* Process crosslinks (see above) +* Process its per-block consensus, as well as the finality gadget + +Processing the beacon chain is similar to processing the Ethereum 1.0 chain. Clients download and process blocks, and maintain a view of what is the current "canonical chain", terminating at the current "head". However, because of the beacon chain's relationship with Ethereum 1.0, and because it is a proof-of-stake chain, there are differences. + +For a beacon chain block, `block`, to be processed by a node, the following conditions must be met: + +* The parent block with root `block.parent_root` has been processed and accepted. +* An Ethereum 1.0 block pointed to by the `state.latest_eth1_data.block_hash` has been processed and accepted. +* The node's local clock time is greater than or equal to `state.genesis_time + block.slot * SLOT_DURATION`. + +If these conditions are not met, the client should delay processing the beacon block until the conditions are all satisfied. + +Beacon block production is significantly different because of the proof of stake mechanism. A client simply checks what it thinks is the canonical chain when it should create a block, and looks up what its slot number is; when the slot arrives, it either proposes or attests to a block as required. Note that this requires each node to have a clock that is roughly (i.e. within `SLOT_DURATION` seconds) synchronized with the other nodes. + +### Beacon chain fork choice rule + +The beacon chain fork choice rule is a hybrid that combines justification and finality with Latest Message Driven (LMD) Greediest Heaviest Observed SubTree (GHOST). At any point in time a [validator](#dfn-validator) `v` subjectively calculates the beacon chain head as follows. + +* Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not yet included in any chain are still included in `store`. +* Let `finalized_head` be the finalized block with the highest epoch. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) +* Let `justified_head` be the descendant of `finalized_head` with the highest epoch that has been justified for at least 1 epoch. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. +* Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. +* Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. +* Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. +* Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the child blocks of the given `block`. +* Let `justified_head_state` be the resulting `BeaconState` object from processing the chain up to the `justified_head`. +* The `head` is `lmd_ghost(store, justified_head_state, justified_head)` where the function `lmd_ghost` is defined below. Note that the implementation below is suboptimal; there are implementations that compute the head in time logarithmic in slot count. ```python -def validate_proof_of_possession(state: BeaconState, - pubkey: BLSPubkey, - proof_of_possession: BLSSignature, - withdrawal_credentials: Bytes32) -> bool: - proof_of_possession_data = DepositInput( - pubkey=pubkey, - withdrawal_credentials=withdrawal_credentials, - proof_of_possession=EMPTY_SIGNATURE, - ) +def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) -> BeaconBlock: + validators = start_state.validator_registry + active_validators = [ + validators[i] + for i in get_active_validator_indices(validators, start_state.slot) + ] + attestation_targets = [ + get_latest_attestation_target(store, validator) + for validator in active_validators + ] - return bls_verify( - pubkey=pubkey, - message=hash_tree_root(proof_of_possession_data), - signature=proof_of_possession, - domain=get_domain( - state.fork, - get_current_epoch(state), - DOMAIN_DEPOSIT, - ) - ) + def get_vote_count(block: BeaconBlock) -> int: + return len([ + target + for target in attestation_targets + if get_ancestor(store, target, block.slot) == block + ]) + + head = start_block + while 1: + children = get_children(store, head) + if len(children) == 0: + return head + head = max(children, key=get_vote_count) ``` -Now, to add a [validator](#dfn-validator) or top up an existing [validator](#dfn-validator)'s balance by some `deposit` amount: +## Beacon chain state transition function -```python -def process_deposit(state: BeaconState, - pubkey: BLSPubkey, - amount: Gwei, - proof_of_possession: BLSSignature, - withdrawal_credentials: Bytes32) -> None: - """ - Process a deposit from Ethereum 1.0. - Note that this function mutates ``state``. - """ - # Validate the given `proof_of_possession` - assert validate_proof_of_possession( - state, - pubkey, - proof_of_possession, - withdrawal_credentials, - ) +We now define the state transition function. This is a parent to child block transition. At a high level the state transition is made up of three parts: - validator_pubkeys = [v.pubkey for v in state.validator_registry] +1. The per-slot transitions, which happens every slot. +2. The per-block transitions, which happens at every block. +3. The per-epoch transitions, which happens at every epoch boundary (i.e. `(state.slot + 1) % EPOCH_LENGTH == 0`). - if pubkey not in validator_pubkeys: - # Add new validator - validator = Validator( - pubkey=pubkey, - withdrawal_credentials=withdrawal_credentials, - activation_epoch=FAR_FUTURE_EPOCH, - exit_epoch=FAR_FUTURE_EPOCH, - withdrawal_epoch=FAR_FUTURE_EPOCH, - penalized_epoch=FAR_FUTURE_EPOCH, - exit_count=0, - status_flags=0, - latest_custody_reseed_slot=GENESIS_SLOT, - penultimate_custody_reseed_slot=GENESIS_SLOT, - ) +The per-slot transitions focus on the slot counter and block roots records updates; the per-block transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-block activity in the `BeaconState`; the per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. - # Note: In phase 2 registry indices that have been withdrawn for a long time will be recycled. - state.validator_registry.append(validator) - state.validator_balances.append(amount) - else: - # Increase balance by deposit amount - index = validator_pubkeys.index(pubkey) - assert state.validator_registry[index].withdrawal_credentials == withdrawal_credentials +_Note_: If there are skipped slots between a block and its parent block, run the steps in the [per-slot](#per-slot-processing) and [per-epoch](#per-epoch-processing) sections once for each skipped slot and then once for the slot containing the new block. - state.validator_balances[index] += amount -``` +### Per-slot processing -### Routines for updating validator status +Below are the processing steps that happen at every slot. -Note: All functions in this section mutate `state`. - -```python -def activate_validator(state: BeaconState, index: ValidatorIndex, genesis: bool) -> None: - validator = state.validator_registry[index] - - validator.activation_epoch = GENESIS_EPOCH if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) -``` - -```python -def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: - validator = state.validator_registry[index] - validator.status_flags |= INITIATED_EXIT -``` - -```python -def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: - validator = state.validator_registry[index] - - # The following updates only occur if not previous exited - if validator.exit_epoch <= get_entry_exit_effect_epoch(get_current_epoch(state)): - return - - validator.exit_epoch = get_entry_exit_effect_epoch(get_current_epoch(state)) - - state.validator_registry_exit_count += 1 - validator.exit_count = state.validator_registry_exit_count -``` - -```python -def penalize_validator(state: BeaconState, index: ValidatorIndex) -> None: - exit_validator(state, index) - validator = state.validator_registry[index] - state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) - - whistleblower_index = get_beacon_proposer_index(state, state.slot) - whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT - state.validator_balances[whistleblower_index] += whistleblower_reward - state.validator_balances[index] -= whistleblower_reward - validator.penalized_epoch = get_current_epoch(state) -``` - -```python -def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) -> None: - validator = state.validator_registry[index] - validator.status_flags |= WITHDRAWABLE -``` - -## Per-slot processing - -Below are the processing steps that happen at every slot. If there are skipped slots between a block and its parent block, run the steps in this section once for each skipped slot and then once for the slot containing the new block. - -### Slot +#### Slot * Set `state.slot += 1`. -### Block roots +#### Block roots * Let `previous_block_root` be the `tree_hash_root` of the previous beacon block processed in the chain. * Set `state.latest_block_roots[(state.slot - 1) % LATEST_BLOCK_ROOTS_LENGTH] = previous_block_root`. * If `state.slot % LATEST_BLOCK_ROOTS_LENGTH == 0` append `merkle_root(state.latest_block_roots)` to `state.batched_block_roots`. -## Per-block processing +### Per-block processing Below are the processing steps that happen at every `block`. -### Slot +#### Slot * Verify that `block.slot == state.slot`. -### Proposer signature +#### Proposer signature * Let `block_without_signature_root` be the `hash_tree_root` of `block` where `block.signature` is set to `EMPTY_SIGNATURE`. * Let `proposal_root = hash_tree_root(ProposalSignedData(state.slot, BEACON_CHAIN_SHARD_NUMBER, block_without_signature_root))`. * Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message=proposal_root, signature=block.signature, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_PROPOSAL))`. -### RANDAO +#### RANDAO * Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. * Verify that `bls_verify(pubkey=proposer.pubkey, message=int_to_bytes32(get_current_epoch(state)), signature=block.randao_reveal, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_RANDAO))`. * Set `state.latest_randao_mixes[get_current_epoch(state) % LATEST_RANDAO_MIXES_LENGTH] = xor(get_randao_mix(state, get_current_epoch(state)), hash(block.randao_reveal))`. -### Eth1 data +#### Eth1 data * If `block.eth1_data` equals `eth1_data_vote.eth1_data` for some `eth1_data_vote` in `state.eth1_data_votes`, set `eth1_data_vote.vote_count += 1`. * Otherwise, append to `state.eth1_data_votes` a new `Eth1DataVote(eth1_data=block.eth1_data, vote_count=1)`. -### Operations +#### Operations -#### Proposer slashings +##### Proposer slashings Verify that `len(block.body.proposer_slashings) <= MAX_PROPOSER_SLASHINGS`. @@ -1545,7 +1563,7 @@ For each `proposer_slashing` in `block.body.proposer_slashings`: * Verify that `bls_verify(pubkey=proposer.pubkey, message=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_2.slot), DOMAIN_PROPOSAL))`. * Run `penalize_validator(state, proposer_slashing.proposer_index)`. -#### Attester slashings +##### Attester slashings Verify that `len(block.body.attester_slashings) <= MAX_ATTESTER_SLASHINGS`. @@ -1562,7 +1580,7 @@ For each `attester_slashing` in `block.body.attester_slashings`: * Verify that `verify_slashable_vote_data(state, slashable_vote_data_2)`. * For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_epoch > get_current_epoch(state)`. -#### Attestations +##### Attestations Verify that `len(block.body.attestations) <= MAX_ATTESTATIONS`. @@ -1580,7 +1598,7 @@ For each `attestation` in `block.body.attestations`: * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. * Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`. -#### Deposits +##### Deposits Verify that `len(block.body.deposits) <= MAX_DEPOSITS`. @@ -1615,7 +1633,7 @@ process_deposit( ) ``` -#### Exits +##### Exits Verify that `len(block.body.exits) <= MAX_EXITS`. @@ -1628,15 +1646,15 @@ For each `exit` in `block.body.exits`: * Verify that `bls_verify(pubkey=validator.pubkey, message=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.epoch, DOMAIN_EXIT))`. * Run `initiate_validator_exit(state, exit.validator_index)`. -#### Custody +##### Custody [TO BE REMOVED IN PHASE 1] Verify that `len(block.body.custody_reseeds) == len(block.body.custody_challenges) == len(block.body.custody_responses) == 0`. -## Per-epoch processing +### Per-epoch processing The steps below happen when `(state.slot + 1) % EPOCH_LENGTH == 0`. -### Helpers +#### Helpers * Let `current_epoch = get_current_epoch(state)`. * Let `previous_epoch = current_epoch - 1 if current_epoch > GENESIS_EPOCH else current_epoch`. @@ -1686,14 +1704,14 @@ Define the following helpers to process attestation inclusion rewards and inclus * Let `inclusion_slot(state, index) = a.slot_included` for the attestation `a` where `index` is in `get_attestation_participants(state, a.data, a.aggregation_bitfield)`. If multiple attestations are applicable, the attestation with lowest `slot_included` is considered. * Let `inclusion_distance(state, index) = a.slot_included - a.data.slot` where `a` is the above attestation. -### Eth1 data +#### Eth1 data If `next_epoch % ETH1_DATA_VOTING_PERIOD == 0`: * If `eth1_data_vote.vote_count * 2 > ETH1_DATA_VOTING_PERIOD * EPOCH_LENGTH` for some `eth1_data_vote` in `state.eth1_data_votes` (ie. more than half the votes in this voting period were for that value), set `state.latest_eth1_data = eth1_data_vote.data`. * Set `state.eth1_data_votes = []`. -### Justification +#### Justification First, update the justification bitfield: @@ -1714,13 +1732,13 @@ Finally, update the following: * Set `state.previous_justified_epoch = state.justified_epoch`. * Set `state.justified_epoch = new_justified_epoch`. -### Crosslinks +#### Crosslinks For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(next_epoch))`, let `crosslink_committees_at_slot = get_crosslink_committees_at_slot(state, slot)`. For every `(crosslink_committee, shard)` in `crosslink_committees_at_slot`, compute: * Set `state.latest_crosslinks[shard] = Crosslink(epoch=current_epoch, shard_block_root=winning_root(crosslink_committee))` if `3 * total_attesting_balance(crosslink_committee) >= 2 * total_balance(crosslink_committee)`. -### Rewards and penalties +#### Rewards and penalties First, we define some additional helpers: @@ -1728,7 +1746,7 @@ First, we define some additional helpers: * Let `base_reward(state, index) = get_effective_balance(state, index) // base_reward_quotient // 5` for any validator with the given `index`. * Let `inactivity_penalty(state, index, epochs_since_finality) = base_reward(state, index) + get_effective_balance(state, index) * epochs_since_finality // INACTIVITY_PENALTY_QUOTIENT // 2` for any validator with the given `index`. -#### Justification and finalization +##### Justification and finalization Note: When applying penalties in the following balance recalculations implementers should make sure the `uint64` does not underflow. @@ -1756,11 +1774,11 @@ Case 2: `epochs_since_finality > 4`: * Any [active_validator](#dfn-active-validator) `index` with `validator.penalized_epoch <= current_epoch`, loses `2 * inactivity_penalty(state, index, epochs_since_finality) + base_reward(state, index)`. * Any [validator](#dfn-validator) `index` in `previous_epoch_attester_indices` loses `base_reward(state, index) - base_reward(state, index) * MIN_ATTESTATION_INCLUSION_DELAY // inclusion_distance(state, index)` -#### Attestation inclusion +##### Attestation inclusion For each `index` in `previous_epoch_attester_indices`, we determine the proposer `proposer_index = get_beacon_proposer_index(state, inclusion_slot(state, index))` and set `state.validator_balances[proposer_index] += base_reward(state, index) // INCLUDER_REWARD_QUOTIENT`. -#### Crosslinks +##### Crosslinks For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_slot(current_epoch))`: @@ -1769,7 +1787,7 @@ For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_s * If `index in attesting_validators(crosslink_committee)`, `state.validator_balances[index] += base_reward(state, index) * total_attesting_balance(crosslink_committee) // total_balance(crosslink_committee))`. * If `index not in attesting_validators(crosslink_committee)`, `state.validator_balances[index] -= base_reward(state, index)`. -### Ejections +#### Ejections * Run `process_ejections(state)`. @@ -1784,7 +1802,7 @@ def process_ejections(state: BeaconState) -> None: exit_validator(state, index) ``` -### Validator registry and shuffling seed data +#### Validator registry and shuffling seed data First, update the following: @@ -1899,13 +1917,13 @@ def process_penalties_and_exits(state: BeaconState) -> None: break ``` -### Final updates +#### Final updates * Set `state.latest_penalized_balances[(next_epoch) % LATEST_PENALIZED_EXIT_LENGTH] = state.latest_penalized_balances[current_epoch % LATEST_PENALIZED_EXIT_LENGTH]`. * Set `state.latest_randao_mixes[next_epoch % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, current_epoch)`. * Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch`. -## State root processing +### State root verification Verify `block.state_root == hash_tree_root(state)` if there exists a `block` for the slot being processed. From f51ce8ff14d2a4bbb00b88d8cd7183f8334ba593 Mon Sep 17 00:00:00 2001 From: Justin Date: Mon, 28 Jan 2019 19:16:11 +0000 Subject: [PATCH 110/136] Further deemphasize "Casper" --- specs/core/0_beacon-chain.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index c0e247108..1f2b53090 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -145,8 +145,8 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be ## Terminology -* **Validator** - a participant in the Casper/sharding consensus system. You can become one by depositing 32 ETH into the Casper mechanism. -* **Active validator** - a [validator](#dfn-validator) currently participating in the protocol which the Casper mechanism looks to produce and attest to blocks, crosslinks and other consensus objects. +* **Validator** - a registered participant in the beacon chain. You can become one by sending Ether into the Ethereum 1.0 deposit contract. +* **Active validator** - an active participant in the Ethereum 2.0 consensus invited to, among other things, propose and attest to blocks and vote for crosslinks. * **Committee** - a (pseudo-) randomly sampled subset of [active validators](#dfn-active-validator). When a committee is referred to collectively, as in "this committee attests to X", this is assumed to mean "some subset of that committee that contains enough [validators](#dfn-validator) that the protocol recognizes it as representing the committee". * **Proposer** - the [validator](#dfn-validator) that creates a beacon chain block * **Attester** - a [validator](#dfn-validator) that is part of a committee that needs to sign off on a beacon chain block while simultaneously creating a link (crosslink) to a recent shard block on a particular shard chain. @@ -171,7 +171,7 @@ Code snippets appearing in `this style` are to be interpreted as Python code. Be | `EJECTION_BALANCE` | `2**4 * 1e9` (= 16,000,000,000) | Gwei | | `MAX_BALANCE_CHURN_QUOTIENT` | `2**5` (= 32) | - | | `BEACON_CHAIN_SHARD_NUMBER` | `2**64 - 1` | - | -| `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes | +| `MAX_INDICES_PER_SLASHABLE_VOTE` | `2**10` (= 1,024) | votes | | `MAX_WITHDRAWALS_PER_EPOCH` | `2**2` (= 4) | withdrawals | * For the safety of crosslinks `TARGET_COMMITTEE_SIZE` exceeds [the recommended minimum committee size of 111](https://vitalik.ca/files/Ithaca201807_Sharding.pdf); with sufficient active validators (at least `EPOCH_LENGTH * TARGET_COMMITTEE_SIZE`), the shuffling algorithm ensures committee sizes at least `TARGET_COMMITTEE_SIZE`. (Unbiasable randomness with a Verifiable Delay Function (VDF) will improve committee robustness and lower the safe minimum committee size.) @@ -1163,7 +1163,7 @@ def get_domain(fork: Fork, ```python def verify_slashable_vote_data(state: BeaconState, vote_data: SlashableVoteData) -> bool: - if len(vote_data.custody_bit_0_indices) + len(vote_data.custody_bit_1_indices) > MAX_CASPER_VOTES: + if len(vote_data.custody_bit_0_indices) + len(vote_data.custody_bit_1_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: return False return bls_verify_multiple( From b2855f9be44fea6610ce0c708b1778b9dfb39c14 Mon Sep 17 00:00:00 2001 From: Justin Date: Mon, 28 Jan 2019 21:13:25 +0000 Subject: [PATCH 111/136] Stricter aggregation_bitfield validation Make sure that there's at least one vote in attestations (`bls_verify_multiple` can return `True` with zero votes) --- specs/core/0_beacon-chain.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 910ebd378..56b5813f4 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1604,8 +1604,9 @@ For each `attestation` in `block.body.attestations`: * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. * Verify bitfields and aggregate signature: -```python +```python assert attestation.custody_bitfield == b'\x00' * len(attestation.custody_bitfield) # [TO BE REMOVED IN PHASE 1] + assert attestation.aggregation_bitfield != b'\x00' * len(attestation.aggregation_bitfield) for i in range(len(crosslink_committee)): if get_bitfield_bit(attestation.aggregation_bitfield) == 0b0: From 3068e6483d0ffcc718c74d1dadb591c8564ae1a9 Mon Sep 17 00:00:00 2001 From: Justin Date: Mon, 28 Jan 2019 21:13:58 +0000 Subject: [PATCH 112/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 56b5813f4..9990c5d73 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1604,7 +1604,7 @@ For each `attestation` in `block.body.attestations`: * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. * Verify bitfields and aggregate signature: -```python +```python assert attestation.custody_bitfield == b'\x00' * len(attestation.custody_bitfield) # [TO BE REMOVED IN PHASE 1] assert attestation.aggregation_bitfield != b'\x00' * len(attestation.aggregation_bitfield) From bb0619986ccc8d0635dda445beed90b3315c4a29 Mon Sep 17 00:00:00 2001 From: vbuterin Date: Mon, 28 Jan 2019 16:20:04 -0600 Subject: [PATCH 113/136] Change -> update As per #358 --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8c9d8a846..a6e9f49ee 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1919,8 +1919,8 @@ and perform the following updates: If a validator registry update does _not_ happen do the following: -* Let `epochs_since_last_registry_change = current_epoch - state.validator_registry_update_epoch`. -* If `epochs_since_last_registry_change` is an exact power of 2: +* Let `epochs_since_last_registry_update = current_epoch - state.validator_registry_update_epoch`. +* If `epochs_since_last_registry_update` is an exact power of 2: * Set `state.current_calculation_epoch = next_epoch`. * Set `state.current_epoch_seed = generate_seed(state, state.current_calculation_epoch)` * _Note_ that `state.current_epoch_start_shard` is left unchanged. From 30ceb8f439151d5b5d63f163f66dc84c3e36b221 Mon Sep 17 00:00:00 2001 From: hukkinj1 Date: Mon, 28 Jan 2019 14:42:31 -0800 Subject: [PATCH 114/136] Removed a trailing space (#514) --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 9990c5d73..808a4082f 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1627,7 +1627,7 @@ For each `attestation` in `block.body.attestations`: ], signature=attestation.aggregate_signature, domain=get_domain(state.fork, slot_to_epoch(attestation.data.slot), DOMAIN_ATTESTATION), - ) + ) ``` * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. From 7da716b0e9db2ecf54dc5fbb2536a8c9eb2d9630 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 28 Jan 2019 21:18:22 -0700 Subject: [PATCH 115/136] pr feedback --- specs/core/0_beacon-chain.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index a6e9f49ee..4adcacdcc 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1527,11 +1527,11 @@ def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) ## Beacon chain state transition function -We now define the state transition function. This is a parent to child block transition. At a high level the state transition is made up of three parts: +We now define the state transition function. At a high level the state transition is made up of three parts: -1. The per-slot transitions, which happens every slot. +1. The per-slot transitions, which happens at the start of every slot. 2. The per-block transitions, which happens at every block. -3. The per-epoch transitions, which happens at every epoch boundary (i.e. `(state.slot + 1) % EPOCH_LENGTH == 0`). +3. The per-epoch transitions, which happens at the end of the last slot of every epoch (i.e. `(state.slot + 1) % EPOCH_LENGTH == 0`). The per-slot transitions focus on the slot counter and block roots records updates; the per-block transitions generally focus on verifying aggregate signatures and saving temporary records relating to the per-block activity in the `BeaconState`; the per-epoch transitions focus on the [validator](#dfn-validator) registry, including adjusting balances and activating and exiting [validators](#dfn-validator), as well as processing crosslinks and managing block justification/finalization. From cbb1571a90dce4244946f5eff5d99c7a08091532 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 28 Jan 2019 21:25:59 -0700 Subject: [PATCH 116/136] remove last ref to casper --- specs/core/0_beacon-chain.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4adcacdcc..a82985b37 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1599,8 +1599,8 @@ Verify that `len(block.body.attester_slashings) <= MAX_ATTESTER_SLASHINGS`. For each `attester_slashing` in `block.body.attester_slashings`: -* Let `slashable_vote_1 = casper_slashing.slashable_vote_1`. -* Let `slashable_vote_2 = casper_slashing.slashable_vote_2`. +* Let `slashable_vote_1 = attester_slashing.slashable_vote_1`. +* Let `slashable_vote_2 = attester_slashing.slashable_vote_2`. * Let `intersection = [x for x in slashable_vote_1.validator_indices if x in slashable_vote_2.validator_indices]`. * Verify that `len(intersection) >= 1`. * Verify that `slashable_vote_1.data != slashable_vote_2.data`. From 6a3b229cb23cec8003e2b88177d9ec2c6cf19f0f Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Mon, 28 Jan 2019 21:32:36 -0700 Subject: [PATCH 117/136] minor formatting issue --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index a82985b37..f166c0d7b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -88,7 +88,7 @@ - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) - [`validate_proof_of_possession`](#validate_proof_of_possession) - [`process_deposit`](#process_deposit) - - [`Routines for updating validator status`](#routines-for-updating-validator-status) + - [Routines for updating validator status](#routines-for-updating-validator-status) - [`activate_validator`](#activate_validator) - [`initiate_validator_exit`](#initiate_validator_exit) - [`exit_validator`](#exit_validator) From c87a9bfd2ce798ba3ee301400b4bbbd9082d935a Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 29 Jan 2019 21:17:05 +0800 Subject: [PATCH 118/136] Fix ToC and #189 --- specs/core/0_beacon-chain.md | 69 +++++++++++++++++++++--------------- 1 file changed, 40 insertions(+), 29 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 7fa25abcb..65a930547 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -104,33 +104,33 @@ - [Beacon chain processing](#beacon-chain-processing) - [Beacon chain fork choice rule](#beacon-chain-fork-choice-rule) - [Beacon chain state transition function](#beacon-chain-state-transition-function) - - [Per-slot processing](#per-slot-processing) - - [Slot](#slot) - - [Block roots](#block-roots) - - [Per-block processing](#per-block-processing) - - [Slot](#slot-1) - - [Proposer signature](#proposer-signature) - - [RANDAO](#randao) - - [Eth1 data](#eth1-data) - - [Operations](#operations) - - [Proposer slashings](#proposer-slashings-1) - - [Attester slashings](#attester-slashings-1) - - [Attestations](#attestations-1) - - [Deposits](#deposits-1) - - [Exits](#exits-1) - - [Per-epoch processing](#per-epoch-processing) - - [Helpers](#helpers) - - [Eth1 data](#eth1-data-1) - - [Justification](#justification) - - [Crosslinks](#crosslinks) - - [Rewards and penalties](#rewards-and-penalties) - - [Justification and finalization](#justification-and-finalization) - - [Attestation inclusion](#attestation-inclusion) - - [Crosslinks](#crosslinks-1) - - [Ejections](#ejections) - - [Validator registry and shuffling seed data](#validator-registry-and-shuffling-seed-data) - - [Final updates](#final-updates) - - [State root verification](#state-root-verification) + - [Per-slot processing](#per-slot-processing) + - [Slot](#slot) + - [Block roots](#block-roots) + - [Per-block processing](#per-block-processing) + - [Slot](#slot-1) + - [Proposer signature](#proposer-signature) + - [RANDAO](#randao) + - [Eth1 data](#eth1-data) + - [Operations](#operations) + - [Proposer slashings](#proposer-slashings-1) + - [Attester slashings](#attester-slashings-1) + - [Attestations](#attestations-1) + - [Deposits](#deposits-1) + - [Exits](#exits-1) + - [Per-epoch processing](#per-epoch-processing) + - [Helpers](#helpers) + - [Eth1 data](#eth1-data-1) + - [Justification](#justification) + - [Crosslinks](#crosslinks) + - [Rewards and penalties](#rewards-and-penalties) + - [Justification and finalization](#justification-and-finalization) + - [Attestation inclusion](#attestation-inclusion) + - [Crosslinks](#crosslinks-1) + - [Ejections](#ejections) + - [Validator registry and shuffling seed data](#validator-registry-and-shuffling-seed-data) + - [Final updates](#final-updates) + - [State root verification](#state-root-verification) - [References](#references) - [Normative](#normative) - [Informative](#informative) @@ -778,7 +778,7 @@ def get_shuffling(seed: Bytes32, **Note**: this definition and the next few definitions make heavy use of repetitive computing. Production implementations are expected to appropriately use caching/memoization to avoid redoing work. -#### `get_previous_epoch_committee_count` +### `get_previous_epoch_committee_count` ```python def get_previous_epoch_committee_count(state: BeaconState) -> int: @@ -1491,7 +1491,18 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi * Abstractly define `Store` as the type of storage object for the chain data and `store` be the set of attestations and blocks that the [validator](#dfn-validator) `v` has observed and verified (in particular, block ancestors must be recursively verified). Attestations not yet included in any chain are still included in `store`. * Let `finalized_head` be the finalized block with the highest epoch. (A block `B` is finalized if there is a descendant of `B` in `store` the processing of which sets `B` as finalized.) * Let `justified_head` be the descendant of `finalized_head` with the highest epoch that has been justified for at least 1 epoch. (A block `B` is justified if there is a descendant of `B` in `store` the processing of which sets `B` as justified.) If no such descendant exists set `justified_head` to `finalized_head`. -* Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as `def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: return block if block.slot == slot else get_ancestor(store, store.get_parent(block), slot)`. +* Let `get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock` be the ancestor of `block` with slot number `slot`. The `get_ancestor` function can be defined recursively as: + +```python +def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: + if block.slot == slot: + return block + elif block.slot < slot: + return None + else: + return get_ancestor(store, store.get_parent(block), slot) +``` + * Let `get_latest_attestation(store: Store, validator: Validator) -> Attestation` be the attestation with the highest slot number in `store` from `validator`. If several such attestations exist, use the one the [validator](#dfn-validator) `v` observed first. * Let `get_latest_attestation_target(store: Store, validator: Validator) -> BeaconBlock` be the target block in the attestation `get_latest_attestation(store, validator)`. * Let `get_children(store: Store, block: BeaconBlock) -> List[BeaconBlock]` returns the child blocks of the given `block`. From c6ca68db00a40acd205dabcb08038eb7a2dd5c0d Mon Sep 17 00:00:00 2001 From: vbuterin Date: Tue, 29 Jan 2019 07:52:42 -0600 Subject: [PATCH 119/136] Clarify further "choose the greater" mechanism --- specs/bls_signature.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/bls_signature.md b/specs/bls_signature.md index 034de479b..44007d82e 100644 --- a/specs/bls_signature.md +++ b/specs/bls_signature.md @@ -88,7 +88,7 @@ def hash_to_G2(message: bytes32, domain: uint64) -> [uint384]: `modular_squareroot(x)` returns a solution `y` to `y**2 % q == x`, and `None` if none exists. If there are two solutions the one with higher imaginary component is favored; if both solutions have equal imaginary component the one with higher real component is favored (note that this is equivalent to saying that the single solution with either imaginary component > p/2 or imaginary component zero and real component > p/2 is favored). -The following is a sample implementation; implementers are free to implement modular square roots as they wish. Note that `x2 = -x1` is an _additive modular inverse_ so real and imaginary coefficients remain in `[0 .. q-1]` +The following is a sample implementation; implementers are free to implement modular square roots as they wish. Note that `x2 = -x1` is an _additive modular inverse_ so real and imaginary coefficients remain in `[0 .. q-1]`. `coerce_to_int` is a function that takes Fq elements (ie. integers mod q) and converts them to regular integers. ```python Fq2_order = q ** 2 - 1 @@ -100,7 +100,9 @@ def modular_squareroot(value: Fq2) -> Fq2: if check in eighth_roots_of_unity[::2]: x1 = candidate_squareroot / eighth_roots_of_unity[eighth_roots_of_unity.index(check) // 2] x2 = -x1 - return x1 if (x1.coeffs[1].n, x1.coeffs[0].n) > (x2.coeffs[1].n, x2.coeffs[0].n) else x2 + x1_re, x1_im = coerce_to_int(x1.coeffs[0]), coerce_to_int(x1.coeffs[1]) + x2_re, x2_im = coerce_to_int(x2.coeffs[0]), coerce_to_int(x2.coeffs[1]) + return x1 if (x1_im > x2_im or (x1_im == x2_im and x1_re > x2_re)) else x2 return None ``` From 51f8ed8992b7e6a3661f5c8696210f6fc9d6e429 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Tue, 29 Jan 2019 15:57:05 -0600 Subject: [PATCH 120/136] Update specs/bls_signature.md Co-Authored-By: vbuterin --- specs/bls_signature.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/bls_signature.md b/specs/bls_signature.md index 44007d82e..fd9bae58e 100644 --- a/specs/bls_signature.md +++ b/specs/bls_signature.md @@ -88,7 +88,7 @@ def hash_to_G2(message: bytes32, domain: uint64) -> [uint384]: `modular_squareroot(x)` returns a solution `y` to `y**2 % q == x`, and `None` if none exists. If there are two solutions the one with higher imaginary component is favored; if both solutions have equal imaginary component the one with higher real component is favored (note that this is equivalent to saying that the single solution with either imaginary component > p/2 or imaginary component zero and real component > p/2 is favored). -The following is a sample implementation; implementers are free to implement modular square roots as they wish. Note that `x2 = -x1` is an _additive modular inverse_ so real and imaginary coefficients remain in `[0 .. q-1]`. `coerce_to_int` is a function that takes Fq elements (ie. integers mod q) and converts them to regular integers. +The following is a sample implementation; implementers are free to implement modular square roots as they wish. Note that `x2 = -x1` is an _additive modular inverse_ so real and imaginary coefficients remain in `[0 .. q-1]`. `coerce_to_int(element: Fq) -> int` is a function that takes Fq element `element` (ie. integers `mod q`) and converts it to a regular integer. ```python Fq2_order = q ** 2 - 1 From 31c557670651d23341ef2c0d3ba1bec0b1be8c25 Mon Sep 17 00:00:00 2001 From: Dean Eigenmann Date: Wed, 30 Jan 2019 00:35:45 +0100 Subject: [PATCH 121/136] Update 0_beacon-chain.md --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index d804a90bc..8b14b4594 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1458,7 +1458,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: activate_validator(state, validator_index, True) - state.latest_index_roots[GENESIS_EPOCH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, GENESIS_EPOCH)) + state.latest_index_roots[GENESIS_EPOCH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state.validator_registry, GENESIS_EPOCH)) state.current_epoch_seed = generate_seed(state, GENESIS_EPOCH) return state From 5e0691f3dfe3c0b6bbd9a557c38acfa5d8f7dbf7 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 30 Jan 2019 17:25:39 +0800 Subject: [PATCH 122/136] Refactor and add docstring. --- specs/core/0_beacon-chain.md | 93 +++++++++++++++++++++++++++++------- 1 file changed, 77 insertions(+), 16 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8b14b4594..3df21ed0e 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -310,7 +310,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { # Validator indices - 'validator_indices': '[uint64]', + 'validator_indices': ['uint64'], # Custody bitfield 'custody_bitfield': 'bytes', # Attestation data @@ -634,6 +634,9 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere ```python def slot_to_epoch(slot: SlotNumber) -> EpochNumber: + """ + Returns the epoch number of the given ``slot``. + """ return slot // EPOCH_LENGTH ``` @@ -641,6 +644,9 @@ def slot_to_epoch(slot: SlotNumber) -> EpochNumber: ```python def get_current_epoch(state: BeaconState) -> EpochNumber: + """ + Returns the current epoch of the given ``state``. + """ return slot_to_epoch(state.slot) ``` @@ -648,6 +654,9 @@ def get_current_epoch(state: BeaconState) -> EpochNumber: ```python def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: + """ + Returns the starting slot of the given ``epoch``. + """ return epoch * EPOCH_LENGTH ``` @@ -741,6 +750,9 @@ def split(values: List[Any], split_count: int) -> List[List[Any]]: ```python def get_epoch_committee_count(active_validator_count: int) -> int: + """ + Returns the number of committees in one epoch. + """ return max( 1, min( @@ -782,6 +794,9 @@ def get_shuffling(seed: Bytes32, ```python def get_previous_epoch_committee_count(state: BeaconState) -> int: + """ + Returns the number of committees in the previous epoch of the given ``state``. + """ previous_active_validators = get_active_validator_indices( state.validator_registry, state.previous_calculation_epoch, @@ -793,6 +808,9 @@ def get_previous_epoch_committee_count(state: BeaconState) -> int: ```python def get_current_epoch_committee_count(state: BeaconState) -> int: + """ + Returns the number of committees in the current epoch of the given ``state``. + """ current_active_validators = get_active_validator_indices( state.validator_registry, state.current_calculation_epoch, @@ -970,6 +988,9 @@ def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: ```python def get_fork_version(fork: Fork, epoch: EpochNumber) -> int: + """ + Returns the fork version of the given ``epoch``. + """ if epoch < fork.epoch: return fork.previous_version else: @@ -982,10 +1003,11 @@ def get_fork_version(fork: Fork, def get_domain(fork: Fork, epoch: EpochNumber, domain_type: int) -> int: - return get_fork_version( - fork, - epoch, - ) * 2**32 + domain_type + """ + Get the domain number that represents the fork meta and signature domain. + """ + fork_version = get_fork_version(fork, epoch) + return fork_version * 2**32 + domain_type ``` ### `get_bitfield_bit` @@ -1052,8 +1074,8 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b0)), - hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b1)), + hash_tree_root(AttestationDataAndCustodyBit(data=slashable_vote.data, custody_bit=0b0)), + hash_tree_root(AttestationDataAndCustodyBit(data=slashable_vote.data, custody_bit=0b1)), ], signature=slashable_vote.aggregate_signature, domain=get_domain( @@ -1068,7 +1090,7 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> ```python def is_double_vote(attestation_data_1: AttestationData, - attestation_data_2: AttestationData) -> bool + attestation_data_2: AttestationData) -> bool: """ Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. Returns True if the provided ``AttestationData`` are slashable @@ -1148,6 +1170,9 @@ def validate_proof_of_possession(state: BeaconState, pubkey: BLSPubkey, proof_of_possession: BLSSignature, withdrawal_credentials: Bytes32) -> bool: + """ + Verify the given ``proof_of_possession``. + """ proof_of_possession_data = DepositInput( pubkey=pubkey, withdrawal_credentials=withdrawal_credentials, @@ -1221,16 +1246,24 @@ Note: All functions in this section mutate `state`. #### `activate_validator` ```python -def activate_validator(state: BeaconState, index: ValidatorIndex, genesis: bool) -> None: +def activate_validator(state: BeaconState, index: ValidatorIndex, is_genesis: bool) -> None: + """ + Activate the validator of the given ``index``. + Note that this function mutates ``state``. + """ validator = state.validator_registry[index] - validator.activation_epoch = GENESIS_EPOCH if genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) + validator.activation_epoch = GENESIS_EPOCH if is_genesis else get_entry_exit_effect_epoch(get_current_epoch(state)) ``` #### `initiate_validator_exit` ```python def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: + """ + Initiate the validator of the given ``index``. + Note that this function mutates ``state``. + """ validator = state.validator_registry[index] validator.status_flags |= INITIATED_EXIT ``` @@ -1239,6 +1272,10 @@ def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None: ```python def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: + """ + Exit the validator of the given ``index``. + Note that this function mutates ``state``. + """ validator = state.validator_registry[index] # The following updates only occur if not previous exited @@ -1255,6 +1292,10 @@ def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: ```python def penalize_validator(state: BeaconState, index: ValidatorIndex) -> None: + """ + Penalize the validator of the given ``index``. + Note that this function mutates ``state``. + """ exit_validator(state, index) validator = state.validator_registry[index] state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) @@ -1270,6 +1311,10 @@ def penalize_validator(state: BeaconState, index: ValidatorIndex) -> None: ```python def prepare_validator_for_withdrawal(state: BeaconState, index: ValidatorIndex) -> None: + """ + Set the validator with the given ``index`` with ``WITHDRAWABLE`` flag. + Note that this function mutates ``state``. + """ validator = state.validator_registry[index] validator.status_flags |= WITHDRAWABLE ``` @@ -1398,6 +1443,9 @@ A valid block with slot `GENESIS_SLOT` (a "genesis block") has the following val def get_initial_beacon_state(initial_validator_deposits: List[Deposit], genesis_time: int, latest_eth1_data: Eth1Data) -> BeaconState: + """ + Get the initial ``BeaconState``. + """ state = BeaconState( # Misc slot=GENESIS_SLOT, @@ -1456,7 +1504,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Process initial activations for validator_index, _ in enumerate(state.validator_registry): if get_effective_balance(state, validator_index) >= MAX_DEPOSIT_AMOUNT: - activate_validator(state, validator_index, True) + activate_validator(state, validator_index, is_genesis=True) state.latest_index_roots[GENESIS_EPOCH % LATEST_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state.validator_registry, GENESIS_EPOCH)) state.current_epoch_seed = generate_seed(state, GENESIS_EPOCH) @@ -1495,6 +1543,9 @@ The beacon chain fork choice rule is a hybrid that combines justification and fi ```python def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBlock: + """ + Get the ancestor of ``block`` with slot number ``slot``; return ``None`` if not found. + """ if block.slot == slot: return block elif block.slot < slot: @@ -1511,6 +1562,9 @@ def get_ancestor(store: Store, block: BeaconBlock, slot: SlotNumber) -> BeaconBl ```python def lmd_ghost(store: Store, start_state: BeaconState, start_block: BeaconBlock) -> BeaconBlock: + """ + Execute the LMD-GHOST algorithm to find the head ``BeaconBlock``. + """ validators = start_state.validator_registry active_validators = [ validators[i] @@ -1676,6 +1730,9 @@ For each `deposit` in `block.body.deposits`: ```python def verify_merkle_branch(leaf: Bytes32, branch: List[Bytes32], depth: int, index: int, root: Bytes32) -> bool: + """ + Verify that the given ``leaf`` is on the merkle branch ``branch``. + """ value = leaf for i in range(depth): if index // (2**i) % 2: @@ -1906,7 +1963,7 @@ def update_validator_registry(state: BeaconState) -> None: break # Activate validator - activate_validator(state, index, False) + activate_validator(state, index, is_genesis=False) # Exit validators within the allowable balance churn balance_churn = 0 @@ -1943,17 +2000,21 @@ Regardless of whether or not a validator set change happens, run the following: ```python def process_penalties_and_exits(state: BeaconState) -> None: + """ + Process the penalties and prepare the validators who are eligible to withdrawal. + Note that this function mutates ``state``. + """ current_epoch = get_current_epoch(state) # The active validators active_validator_indices = get_active_validator_indices(state.validator_registry, current_epoch) # The total effective balance of active validators - total_balance = sum([get_effective_balance(state, i) for i in active_validator_indices]) + total_balance = sum(get_effective_balance(state, i) for i in active_validator_indices) for index, validator in enumerate(state.validator_registry): if current_epoch == validator.penalized_epoch + LATEST_PENALIZED_EXIT_LENGTH // 2: - e = current_epoch % LATEST_PENALIZED_EXIT_LENGTH - total_at_start = state.latest_penalized_balances[(e + 1) % LATEST_PENALIZED_EXIT_LENGTH] - total_at_end = state.latest_penalized_balances[e] + epoch_index = current_epoch % LATEST_PENALIZED_EXIT_LENGTH + total_at_start = state.latest_penalized_balances[(epoch_index + 1) % LATEST_PENALIZED_EXIT_LENGTH] + total_at_end = state.latest_penalized_balances[epoch_index] total_penalties = total_at_end - total_at_start penalty = get_effective_balance(state, index) * min(total_penalties * 3, total_balance) // total_balance state.validator_balances[index] -= penalty From f4798370aa8bd968f62720ecf665f9631df1b850 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 30 Jan 2019 05:38:45 -0800 Subject: [PATCH 123/136] Attester slashing fixes * Fix bug whereby `intersection` (replaced by `slashable_indices `) could be non-empty, but no validator gets penalized because they were all already penalized. * Rename `SlashableVote` to `SlashableAttestation` (for consistency with `Attestation`, and to avoid having "Vote" be a synonym of "Attestation"). * Rename `slot_included` to `inclusion_slot` for consistency with the `_slot` (and `_epoch`) postfix convention * Homogenise ordering of fields in `Attestation` and `SlashableAttestation` objects (one has an `aggregation_bitfield` instead of the other's `validator_indices`). * Homogenise ordering of fields in `Attestation` and `PendingAttestation` objects (one has an `aggregate_signature` instead of the other's `inclusion_slot`). * Light presentation cleanup --- specs/core/0_beacon-chain.md | 92 +++++++++++++++++------------------- 1 file changed, 43 insertions(+), 49 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8b14b4594..28d2a8d58 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -26,7 +26,7 @@ - [`ProposerSlashing`](#proposerslashing) - [Attester slashings](#attester-slashings) - [`AttesterSlashing`](#attesterslashing) - - [`SlashableVote`](#slashablevote) + - [`SlashableAttestation`](#slashableattestation) - [Attestations](#attestations) - [`Attestation`](#attestation) - [`AttestationData`](#attestationdata) @@ -78,7 +78,7 @@ - [`get_domain`](#get_domain) - [`get_bitfield_bit`](#get_bitfield_bit) - [`verify_bitfield`](#verify_bitfield) - - [`verify_slashable_vote`](#verify_slashable_vote) + - [`verify_slashable_attestation`](#verify_slashable_attestation) - [`is_double_vote`](#is_double_vote) - [`is_surround_vote`](#is_surround_vote) - [`integer_squareroot`](#integer_squareroot) @@ -298,23 +298,23 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { - # First batch of votes - 'slashable_vote_1': SlashableVote, - # Second batch of votes - 'slashable_vote_2': SlashableVote, + # First slashable attestation + 'slashable_attestation_1': SlashableAttestation, + # Second slashable attestation + 'slashable_attestation_2': SlashableAttestation, } ``` -##### `SlashableVote` +##### `SlashableAttestation` ```python { # Validator indices 'validator_indices': '[uint64]', - # Custody bitfield - 'custody_bitfield': 'bytes', # Attestation data 'data': AttestationData, + # Custody bitfield + 'custody_bitfield': 'bytes', # Aggregate signature 'aggregate_signature': 'bytes96', } @@ -326,10 +326,10 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { - # Attestation data - 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', + # Attestation data + 'data': AttestationData, # Custody bitfield 'custody_bitfield': 'bytes', # BLS aggregate signature @@ -555,14 +555,14 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { - # Signed data - 'data': AttestationData, # Attester aggregation bitfield 'aggregation_bitfield': 'bytes', + # Attestation data + 'data': AttestationData, # Custody bitfield 'custody_bitfield': 'bytes', - # Slot the attestation was included - 'slot_included': 'uint64', + # Inclusion slot + 'inclusion_slot': 'uint64', } ``` @@ -1015,33 +1015,33 @@ def verify_bitfield(bitfield: bytes, committee_size: int) -> bool: return True ``` -### `verify_slashable_vote` +### `verify_slashable_attestation` ```python -def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> bool: +def verify_slashable_attestation(state: BeaconState, slashable_attestation: SlashableAttestation) -> bool: """ - Verify validity of ``slashable_vote`` fields. + Verify validity of ``slashable_attestation`` fields. """ - if slashable_vote.custody_bitfield != b'\x00' * len(slashable_vote.custody_bitfield): # [TO BE REMOVED IN PHASE 1] + if slashable_attestation.custody_bitfield != b'\x00' * len(slashable_attestation.custody_bitfield): # [TO BE REMOVED IN PHASE 1] return False - if len(slashable_vote.validator_indices) == 0: + if len(slashable_attestation.validator_indices) == 0: return False - for i in range(len(slashable_vote.validator_indices) - 1): - if slashable_vote.validator_indices[i] >= slashable_vote.validator_indices[i + 1]: + for i in range(len(slashable_attestation.validator_indices) - 1): + if slashable_attestation.validator_indices[i] >= slashable_attestation.validator_indices[i + 1]: return False - if not verify_bitfield(slashable_vote.custody_bitfield, len(slashable_vote.validator_indices)): + if not verify_bitfield(slashable_attestation.custody_bitfield, len(slashable_attestation.validator_indices)): return False - if len(slashable_vote.validator_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: + if len(slashable_attestation.validator_indices) > MAX_INDICES_PER_SLASHABLE_VOTE: return False custody_bit_0_indices = [] custody_bit_1_indices = [] - for i, validator_index in enumerate(slashable_vote.validator_indices): - if get_bitfield_bit(slashable_vote.custody_bitfield, i) == 0b0: + for i, validator_index in enumerate(slashable_attestation.validator_indices): + if get_bitfield_bit(slashable_attestation.custody_bitfield, i) == 0b0: custody_bit_0_indices.append(validator_index) else: custody_bit_1_indices.append(validator_index) @@ -1052,10 +1052,10 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> bls_aggregate_pubkeys([state.validator_registry[i].pubkey for i in custody_bit_1_indices]), ], messages=[ - hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b0)), - hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_vote.data, custody_bit=0b1)), + hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_attestation.data, custody_bit=0b0)), + hash_tree_root(AttestationDataAndCustodyBit(attestation_data=slashable_attestation.data, custody_bit=0b1)), ], - signature=slashable_vote.aggregate_signature, + signature=slashable_attestation.aggregate_signature, domain=get_domain( state.fork, slot_to_epoch(vote_data.data.slot), @@ -1070,9 +1070,7 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> def is_double_vote(attestation_data_1: AttestationData, attestation_data_2: AttestationData) -> bool """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable - due to a 'double vote'. + Checks if the two ``AttestationData`` have the same target. """ target_epoch_1 = slot_to_epoch(attestation_data_1.slot) target_epoch_2 = slot_to_epoch(attestation_data_2.slot) @@ -1085,11 +1083,7 @@ def is_double_vote(attestation_data_1: AttestationData, def is_surround_vote(attestation_data_1: AttestationData, attestation_data_2: AttestationData) -> bool: """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable - due to a 'surround vote'. - Note: parameter order matters as this function only checks - that ``attestation_data_1`` surrounds ``attestation_data_2``. + Checks if ``attestation_data_1`` surrounds ``attestation_data_2``. """ source_epoch_1 = attestation_data_1.justified_epoch source_epoch_2 = attestation_data_2.justified_epoch @@ -1610,15 +1604,15 @@ Verify that `len(block.body.attester_slashings) <= MAX_ATTESTER_SLASHINGS`. For each `attester_slashing` in `block.body.attester_slashings`: -* Let `slashable_vote_1 = attester_slashing.slashable_vote_1`. -* Let `slashable_vote_2 = attester_slashing.slashable_vote_2`. -* Let `intersection = [x for x in slashable_vote_1.validator_indices if x in slashable_vote_2.validator_indices]`. -* Verify that `len(intersection) >= 1`. -* Verify that `slashable_vote_1.data != slashable_vote_2.data`. -* Verify that `is_double_vote(slashable_vote_1.data, slashable_vote_2.data)` or `is_surround_vote(slashable_vote_1.data, slashable_vote_2.data)`. -* Verify that `verify_slashable_vote(state, slashable_vote_1)`. -* Verify that `verify_slashable_vote(state, slashable_vote_2)`. -* For each [validator](#dfn-validator) index `i` in `intersection` run `penalize_validator(state, i)` if `state.validator_registry[i].penalized_epoch > get_current_epoch(state)`. +* Let `slashable_attestation_1 = attester_slashing.slashable_attestation_1`. +* Let `slashable_attestation_2 = attester_slashing.slashable_attestation_2`. +* Verify that `slashable_attestation_1.data != slashable_attestation_2.data`. +* Verify that `is_double_vote(slashable_attestation_1.data, slashable_attestation_2.data)` or `is_surround_vote(slashable_attestation_1.data, slashable_attestation_2.data)`. +* Verify that `verify_slashable_attestation(state, slashable_attestation_1)`. +* Verify that `verify_slashable_attestation(state, slashable_attestation_2)`. +* Let `slashable_indices = [index for index in slashable_attestation_1.validator_indices if index in slashable_attestation_2.validator_indices and state.validator_registry[index].penalized_epoch > get_current_epoch(state)]`. +* Verify that `len(slashable_indices) >= 1`. +* Run `penalize_validator(state, index)` for each `index` in `slashable_indices`. ##### Attestations @@ -1660,7 +1654,7 @@ For each `attestation` in `block.body.attestations`: ``` * [TO BE REMOVED IN PHASE 1] Verify that `attestation.data.shard_block_root == ZERO_HASH`. -* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, slot_included=state.slot)` to `state.latest_attestations`. +* Append `PendingAttestation(data=attestation.data, aggregation_bitfield=attestation.aggregation_bitfield, custody_bitfield=attestation.custody_bitfield, inclusion_slot=state.slot)` to `state.latest_attestations`. ##### Deposits @@ -1761,8 +1755,8 @@ For every `slot in range(get_epoch_start_slot(previous_epoch), get_epoch_start_s Define the following helpers to process attestation inclusion rewards and inclusion distance reward/penalty. For every attestation `a` in `previous_epoch_attestations`: -* Let `inclusion_slot(state, index) = a.slot_included` for the attestation `a` where `index` is in `get_attestation_participants(state, a.data, a.aggregation_bitfield)`. If multiple attestations are applicable, the attestation with lowest `slot_included` is considered. -* Let `inclusion_distance(state, index) = a.slot_included - a.data.slot` where `a` is the above attestation. +* Let `inclusion_slot(state, index) = a.inclusion_slot` for the attestation `a` where `index` is in `get_attestation_participants(state, a.data, a.aggregation_bitfield)`. If multiple attestations are applicable, the attestation with lowest `inclusion_slot` is considered. +* Let `inclusion_distance(state, index) = a.inclusion_slot - a.data.slot` where `a` is the above attestation. #### Eth1 data From 8663773fa2da7589db446b9207a31498a43b71a9 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 30 Jan 2019 06:32:44 -0800 Subject: [PATCH 124/136] Cleanup attestation inclusion logic * (Logic change) Introduce invariant that attestations can be included for `EPOCH_LENGTH`. * Simplify presentation by related checks into a single line. --- specs/core/0_beacon-chain.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8b14b4594..6c74ef0b9 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1626,8 +1626,7 @@ Verify that `len(block.body.attestations) <= MAX_ATTESTATIONS`. For each `attestation` in `block.body.attestations`: -* Verify that `attestation.data.slot + MIN_ATTESTATION_INCLUSION_DELAY <= state.slot`. -* Verify that `attestation.data.slot + EPOCH_LENGTH >= state.slot`. +* Verify that `attestation.data.slot <= state.slot - MIN_ATTESTATION_INCLUSION_DELAY < attestation.data.slot + EPOCH_LENGTH`. * Verify that `attestation.data.justified_epoch` is equal to `state.justified_epoch if attestation.data.slot >= get_epoch_start_slot(get_current_epoch(state)) else state.previous_justified_epoch`. * Verify that `attestation.data.justified_block_root` is equal to `get_block_root(state, get_epoch_start_slot(attestation.data.justified_epoch))`. * Verify that either `attestation.data.latest_crosslink_root` or `attestation.data.shard_block_root` equals `state.latest_crosslinks[shard].shard_block_root`. From 5889e9c6dbe07f654987885d1add6d32792752b1 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 30 Jan 2019 06:43:02 -0800 Subject: [PATCH 125/136] Remove exit counts * Simplify by removing `exit_epoch` in `BeaconState` and `Validator`. (Also less storage for full nodes.) * Replace with `exit_epoch` when ordering validator with withdrawal. Slight loss of granularity, but that's actually a good thing because it reduces the opportunity for manipulation by proposers. --- specs/core/0_beacon-chain.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 8b14b4594..49868123a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -486,7 +486,6 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git 'validator_registry': [Validator], 'validator_balances': ['uint64'], 'validator_registry_update_epoch': 'uint64', - 'validator_registry_exit_count': 'uint64', # Randomness and committees 'latest_randao_mixes': ['bytes32'], @@ -533,8 +532,6 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git 'withdrawal_epoch': 'uint64', # Epoch when validator was penalized 'penalized_epoch': 'uint64', - # Exit counter when validator exited - 'exit_count': 'uint64', # Status flags 'status_flags': 'uint64', } @@ -1199,7 +1196,6 @@ def process_deposit(state: BeaconState, exit_epoch=FAR_FUTURE_EPOCH, withdrawal_epoch=FAR_FUTURE_EPOCH, penalized_epoch=FAR_FUTURE_EPOCH, - exit_count=0, status_flags=0, ) @@ -1246,9 +1242,6 @@ def exit_validator(state: BeaconState, index: ValidatorIndex) -> None: return validator.exit_epoch = get_entry_exit_effect_epoch(get_current_epoch(state)) - - state.validator_registry_exit_count += 1 - validator.exit_count = state.validator_registry_exit_count ``` #### `penalize_validator` @@ -1412,7 +1405,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], validator_registry=[], validator_balances=[], validator_registry_update_epoch=GENESIS_EPOCH, - validator_registry_exit_count=0, # Randomness and committees latest_randao_mixes=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH)], @@ -1968,7 +1960,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: all_indices = list(range(len(state.validator_registry))) eligible_indices = filter(eligible, all_indices) - sorted_indices = sorted(eligible_indices, key=lambda index: state.validator_registry[index].exit_count) + sorted_indices = sorted(eligible_indices, key=lambda index: state.validator_registry[index].exit_epoch) withdrawn_so_far = 0 for index in sorted_indices: prepare_validator_for_withdrawal(state, index) From 8040044f6925bb915e3eb50765ed8010c679be8a Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 30 Jan 2019 06:46:43 -0800 Subject: [PATCH 126/136] fix is_surround as per #410 (#523) --- specs/core/0_beacon-chain.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 6c74ef0b9..e4b2d498a 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1095,11 +1095,8 @@ def is_surround_vote(attestation_data_1: AttestationData, source_epoch_2 = attestation_data_2.justified_epoch target_epoch_1 = slot_to_epoch(attestation_data_1.slot) target_epoch_2 = slot_to_epoch(attestation_data_2.slot) - return ( - (source_epoch_1 < source_epoch_2) and - (source_epoch_2 + 1 == target_epoch_2) and - (target_epoch_2 < target_epoch_1) - ) + + return source_epoch_1 < source_epoch_2 and target_epoch_2 < target_epoch_1 ``` ### `integer_squareroot` From 014016cf3ca0283bc2e61932edf02087eaf3cb60 Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 30 Jan 2019 06:50:24 -0800 Subject: [PATCH 127/136] Remove extraneous latest_vdf_outputs --- specs/core/0_beacon-chain.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index e4b2d498a..dcd9eba4b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1413,7 +1413,6 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit], # Randomness and committees latest_randao_mixes=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH)], - latest_vdf_outputs=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH // EPOCH_LENGTH)], previous_epoch_start_shard=GENESIS_START_SHARD, current_epoch_start_shard=GENESIS_START_SHARD, previous_calculation_epoch=GENESIS_EPOCH, From 8c2ed0b683ecc84d79f95624683e7c2c79980cc4 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 30 Jan 2019 06:53:25 -0800 Subject: [PATCH 128/136] validator guide to recent spec changes (#518) --- specs/core/0_beacon-chain.md | 4 +- specs/validator/0_beacon-chain-validator.md | 82 +++++++++++---------- 2 files changed, 44 insertions(+), 42 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index e4b2d498a..125ed2c1b 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1634,8 +1634,8 @@ For each `attestation` in `block.body.attestations`: assert attestation.aggregation_bitfield != b'\x00' * len(attestation.aggregation_bitfield) for i in range(len(crosslink_committee)): - if get_bitfield_bit(attestation.aggregation_bitfield) == 0b0: - assert get_bitfield_bit(attestation.custody_bitfield) == 0b0 + if get_bitfield_bit(attestation.aggregation_bitfield, i) == 0b0: + assert get_bitfield_bit(attestation.custody_bitfield, i) == 0b0 participants = get_attestation_participants(state, attestation.data, attestation.aggregation_bitfield) custody_bit_1_participants = get_attestation_participants(state, attestation.data, attestation.custody_bitfield) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 83ebb5751..80b0fe048 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -31,7 +31,7 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Signature](#signature) - [Block body](#block-body) - [Proposer slashings](#proposer-slashings) - - [Casper slashings](#casper-slashings) + - [Attester slashings](#attester-slashings) - [Attestations](#attestations) - [Deposits](#deposits) - [Exits](#exits) @@ -51,8 +51,8 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Custody bitfield](#custody-bitfield) - [Aggregate signature](#aggregate-signature) - [How to avoid slashing](#how-to-avoid-slashing) - - [Proposal slashing](#proposal-slashing) - - [Casper slashing](#casper-slashing) + - [Proposer slashing](#proposer-slashing) + - [Attester slashing](#attester-slashing) @@ -88,7 +88,7 @@ Validator public keys are [G1 points](https://github.com/ethereum/eth2.0-specs/b A secondary withdrawal private key, `withdrawal_privkey`, must also be securely generated along with the resultant `withdrawal_pubkey`. This `withdrawal_privkey` does not have to be available for signing during the normal lifetime of a validator and can live in "cold storage". -The validator constructs their `withdrawal_credentials` through the following: +The validator constructs their `withdrawal_credentials` via the following: * Set `withdrawal_credentials[:1] == BLS_WITHDRAWAL_PREFIX_BYTE`. * Set `withdrawal_credentials[1:] == hash(withdrawal_pubkey)[1:]`. @@ -98,35 +98,35 @@ In phase 0, all incoming validator deposits originate from the Ethereum 1.0 PoW To submit a deposit: -* Pack the validator's [initialization parameters](#initialization) into `deposit_input`, a [`DepositInput`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#depositinput) object. +* Pack the validator's [initialization parameters](#initialization) into `deposit_input`, a [`DepositInput`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#depositinput) SSZ object. * Set `deposit_input.proof_of_possession = EMPTY_SIGNATURE`. * Let `proof_of_possession` be the result of `bls_sign` of the `hash_tree_root(deposit_input)` with `domain=DOMAIN_DEPOSIT`. * Set `deposit_input.proof_of_possession = proof_of_possession`. * Let `amount` be the amount in Gwei to be deposited by the validator where `MIN_DEPOSIT_AMOUNT <= amount <= MAX_DEPOSIT_AMOUNT`. -* Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `deposit_input` as the singular `bytes` input along with a deposit `amount` in Gwei. +* Send a transaction on the Ethereum 1.0 chain to `DEPOSIT_CONTRACT_ADDRESS` executing `deposit` along with `serialize(deposit_input)` as the singular `bytes` input along with a deposit `amount` in Gwei. _Note_: Deposits made for the same `pubkey` are treated as for the same validator. A singular `Validator` will be added to `state.validator_registry` with each additional deposit amount added to the validator's balance. A validator can only be activated when total deposits for the validator pubkey meet or exceed `MAX_DEPOSIT_AMOUNT`. ### Process deposit -Deposits cannot be processed into the beacon chain until the eth1.0 block in which they were deposited or any of its ancestors is added to the beacon chain `state.eth1_data`. This takes _a minimum_ of `ETH1_FOLLOW_DISTANCE` eth1.0 blocks (~4 hours) plus `ETH1_DATA_VOTING_PERIOD` slots (~1.7 hours). Once the necessary eth1.0 data is added, the deposit will normally be added to a beacon chain block and processed into the `state.validator_registry` within an epoch or two. The validator is then in a queue to be activated. +Deposits cannot be processed into the beacon chain until the eth1.0 block in which they were deposited or any of its descendants is added to the beacon chain `state.eth1_data`. This takes _a minimum_ of `ETH1_FOLLOW_DISTANCE` eth1.0 blocks (~4 hours) plus `ETH1_DATA_VOTING_PERIOD` epochs (~1.7 hours). Once the requisite eth1.0 data is added, the deposit will normally be added to a beacon chain block and processed into the `state.validator_registry` within an epoch or two. The validator is then in a queue to be activated. ### Validator index -Once a validator has been processed and added to the state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#validatorrecord) contains the `pubkey` specified in the validator's deposit. A validator's `validator_index` is guaranteed to not change from the time of initial deposit until the validator exists and fully withdraws. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. +Once a validator has been processed and added to the beacon state's `validator_registry`, the validator's `validator_index` is defined by the index into the registry at which the [`ValidatorRecord`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#validator) contains the `pubkey` specified in the validator's deposit. A validator's `validator_index` is guaranteed to not change from the time of initial deposit until the validator exits and fully withdraws. This `validator_index` is used throughout the specification to dictate validator roles and responsibilities at any point and should be stored locally. ### Activation -In normal operation, the validator is quickly activated at which point the validator is added to the shuffling and begins validation after an additional `ENTRY_EXIT_DELAY` slots. +In normal operation, the validator is quickly activated at which point the validator is added to the shuffling and begins validation after an additional `ENTRY_EXIT_DELAY` epochs (25.6 minutes). -The function [`is_active_validator`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_active_validator) can be used to check if a validator is active at a given slot. Usage is as follows: +The function [`is_active_validator`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_active_validator) can be used to check if a validator is active during a given epoch. Usage is as follows: ```python validator = state.validator_registry[validator_index] -is_active = is_active_validator(validator, slot) +is_active = is_active_validator(validator, epoch) ``` -Once a validator is active, the validator is assigned [responsibilities](#beacon-chain-responsibilities) until exited. +Once a validator is activated, the validator is assigned [responsibilities](#beacon-chain-responsibilities) until exited. _Note_: There is a maximum validator churn per finalized epoch so the delay until activation is variable depending upon finality, total active validator balance, and the number of validators in the queue to be activated. @@ -136,7 +136,9 @@ A validator has two primary responsibilities to the beacon chain -- [proposing b ### Block proposal -A validator is expected to propose a [`BeaconBlock`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beaconblock) at the beginning of any slot during which `get_beacon_proposer_index(state, slot)` returns the validator's `validator_index`. To propose, the validator selects the `BeaconBlock`, `parent`, that in their view of the fork choice is the head of the chain during `slot`. The validator is to create, sign, and broadcast a `block` that is a child of `parent` that creates a valid [beacon chain state transition](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-state-transition-function). +A validator is expected to propose a [`BeaconBlock`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beaconblock) at the beginning of any slot during which `get_beacon_proposer_index(state, slot)` returns the validator's `validator_index`. To propose, the validator selects the `BeaconBlock`, `parent`, that in their view of the fork choice is the head of the chain during `slot`. The validator is to create, sign, and broadcast a `block` that is a child of `parent` and that executes a valid [beacon chain state transition](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#beacon-chain-state-transition-function). + +There is one proposer per slot, so if there are N active validators any individual validator will on average be assigned to propose once per N slots (eg. at 312500 validators = 10 million ETH, that's once per ~3 weeks). #### Block header @@ -163,25 +165,25 @@ Set `block.randao_reveal = epoch_signature` where `epoch_signature` is defined a ```python epoch_signature = bls_sign( privkey=validator.privkey, # privkey store locally, not in state - message=int_to_bytes32(block.slot // EPOCH_LENGTH), + message=int_to_bytes32(slot_to_epoch(block.slot)), domain=get_domain( - fork_data, # `fork_data` is the fork_data at the slot `block.slot` - block.slot, - DOMAIN_RANDAO, + fork=fork, # `fork` is the fork object at the slot `block.slot` + epoch=slot_to_epoch(block.slot), + domain_type=DOMAIN_RANDAO, ) ) ``` ##### Eth1 Data -`block.eth1_data` is a mechanism used by block proposers vote on a recent Ethereum 1.0 block hash and an associated deposit root found in the Ethereum 1.0 deposit contract. When consensus is formed, `state.latest_eth1_data` is updated, and validator deposits up to this root can be processed. +`block.eth1_data` is a mechanism used by block proposers vote on a recent Ethereum 1.0 block hash and an associated deposit root found in the Ethereum 1.0 deposit contract. When consensus is formed, `state.latest_eth1_data` is updated, and validator deposits up to this root can be processed. The deposit root can be calculated by calling the `get_deposit_root()` function of the deposit contract using the post-state of the block hash. * Let `D` be the set of `Eth1DataVote` objects `vote` in `state.eth1_data_votes` where: * `vote.eth1_data.block_hash` is the hash of an eth1.0 block that is (i) part of the canonical chain, (ii) >= `ETH1_FOLLOW_DISTANCE` blocks behind the head, and (iii) newer than `state.latest_eth1_data.block_data`. * `vote.eth1_data.deposit_root` is the deposit root of the eth1.0 deposit contract at the block defined by `vote.eth1_data.block_hash`. * If `D` is empty: - * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`th ancestor of the head of the canonical eth1.0 chain. - * Let `deposit_root` be the deposit root of the eth1.0 deposit contract at the block defined by `block_hash`. + * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`'th ancestor of the head of the canonical eth1.0 chain. + * Let `deposit_root` be the deposit root of the eth1.0 deposit contract in the post-state of the block referenced by `block_hash` * If `D` is nonempty: * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. * Let `block_hash = best_vote.eth1_data.block_hash`. @@ -204,9 +206,9 @@ signed_proposal_data = bls_sign( privkey=validator.privkey, # privkey store locally, not in state message=proposal_root, domain=get_domain( - fork_data, # `fork_data` is the fork_data at the slot `block.slot` - block.slot, - DOMAIN_PROPOSAL, + fork=fork, # `fork` is the fork object at the slot `block.slot` + epoch=slot_to_epoch(block.slot), + domain_type=DOMAIN_PROPOSAL, ) ) ``` @@ -217,9 +219,9 @@ signed_proposal_data = bls_sign( Up to `MAX_PROPOSER_SLASHINGS` [`ProposerSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposerslashing) objects can be included in the `block`. The proposer slashings must satisfy the verification conditions found in [proposer slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposer-slashings-1). The validator receives a small "whistleblower" reward for each proposer slashing found and included. -##### Casper slashings +##### Attester slashings -Up to `MAX_CASPER_SLASHINGS` [`CasperSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casperslashing) objects can be included in the `block`. The Casper slashings must satisfy the verification conditions found in [Casper slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#casper-slashings-1). The validator receives a small "whistleblower" reward for each Casper slashing found and included. +Up to `MAX_ATTESTER_SLASHINGS` [`AttesterSlashing`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attesterslashing) objects can be included in the `block`. The attester slashings must satisfy the verification conditions found in [Attester slashings processing](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attester-slashings-1). The validator receives a small "whistleblower" reward for each attester slashing found and included. ##### Attestations @@ -235,7 +237,7 @@ Up to `MAX_EXITS` [`Exit`](https://github.com/ethereum/eth2.0-specs/blob/master/ ### Attestations -A validator is expected to create, sign, and broadcast an attestation during each epoch. The slot during which the validator performs this role is any slot at which `get_shard_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. +A validator is expected to create, sign, and broadcast an attestation during each epoch. The slot during which the validator performs this role is any slot at which `get_crosslink_committees_at_slot(state, slot)` contains a committee that contains `validator_index`. A validator should create and broadcast the attestation halfway through the `slot` during which the validator is assigned -- that is `SLOT_DURATION * 0.5` seconds after the start of `slot`. @@ -257,9 +259,9 @@ Set `attestation_data.beacon_block_root = hash_tree_root(head)` where `head` is ##### Epoch boundary root -Set `attestation_data.epoch_boundary_root = hash_tree_root(epoch_boundary)` where `epoch_boundary` is the block at the most recent epoch boundary in the chain defined by `head` -- i.e. the `BeaconBlock` with `slot == head.slot - head.slot % EPOCH_LENGTH`. +Set `attestation_data.epoch_boundary_root = hash_tree_root(epoch_boundary)` where `epoch_boundary` is the block at the most recent epoch boundary in the chain defined by `head` -- i.e. the `BeaconBlock` where `block.slot == get_epoch_start_slot(head.slot)`. -_Note:_ This can be looked up in the state using `get_block_root(state, head.slot - head.slot % EPOCH_LENGTH)`. +_Note:_ This can be looked up in the state using `get_block_root(state, get_epoch_start_slot(head.slot))`. ##### Shard block root @@ -311,8 +313,8 @@ Set `attestation.aggregate_signature = signed_attestation_data` where `signed_at ```python attestation_data_and_custody_bit = AttestationDataAndCustodyBit( - attestation.data, - False, + data=attestation.data, + custody_bit=0b0, ) attestation_message_to_sign = hash_tree_root(attestation_data_and_custody_bit) @@ -320,24 +322,24 @@ signed_attestation_data = bls_sign( privkey=validator.privkey, # privkey store locally, not in state message=attestation_message_to_sign, domain=get_domain( - state.fork_data, # `state` is the state at `head` - state.slot, - DOMAIN_ATTESTATION, + fork=fork, # `fork` is the fork object at the slot, `attestation_data.slot` + epoch=slot_to_epoch(attestation_data.slot), + domain_type=DOMAIN_ATTESTATION, ) ) ``` ## How to avoid slashing -"Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposal slashing](#proposal-slashing) and [attestation slashing](#casper-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages you have previously signed. +"Slashing" is the burning of some amount of validator funds and immediate ejection from the active validator set. In Phase 0, there are two ways in which funds can be slashed -- [proposer slashing](#proposer-slashing) and [attester slashing](#attester-slashing). Although being slashed has serious repercussions, it is simple enough to avoid being slashed all together by remaining _consistent_ with respect to the messages a validator has previously signed. _Note_: Signed data must be within a sequential `Fork` context to conflict. Messages cannot be slashed across diverging forks. If the previous fork version is 1 and the chain splits into fork 2 and 102, messages from 1 can slashable against messages in forks 1, 2, and 102. Messages in 2 cannot be slashable against messages in 102 and vice versa. -### Proposal slashing +### Proposer slashing -To avoid "proposal slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposalsigneddata) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. In phase 0, proposals are only made for the beacon chain (`shard == BEACON_CHAIN_SHARD_NUMBER`). +To avoid "proposer slashings", a validator must not sign two conflicting [`ProposalSignedData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#proposalsigneddata) where conflicting is defined as having the same `slot` and `shard` but a different `block_root`. In phase 0, proposals are only made for the beacon chain (`shard == BEACON_CHAIN_SHARD_NUMBER`). -In phase 0, as long as the validator does not sign two different beacon chain proposals for the same slot, the validator is safe against proposal slashings. +_In phase 0, as long as the validator does not sign two different beacon chain proposals for the same slot, the validator is safe against proposer slashings._ Specifically, when signing an `BeaconBlock`, a validator should perform the following steps in the following order: 1. Save a record to hard disk that an beacon block has been signed for the `slot=slot` and `shard=BEACON_CHAIN_SHARD_NUMBER`. @@ -345,12 +347,12 @@ Specifically, when signing an `BeaconBlock`, a validator should perform the foll If the software crashes at some point within this routine, then when the validator comes back online the hard disk has the record of the _potentially_ signed/broadcast block and can effectively avoid slashing. -### Casper slashing +### Attester slashing -To avoid "Casper slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_surround_vote). +To avoid "attester slashings", a validator must not sign two conflicting [`AttestationData`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#attestationdata) objects where conflicting is defined as a set of two attestations that satisfy either [`is_double_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_double_vote) or [`is_surround_vote`](https://github.com/ethereum/eth2.0-specs/blob/master/specs/core/0_beacon-chain.md#is_surround_vote). Specifically, when signing an `Attestation`, a validator should perform the following steps in the following order: -1. Save a record to hard disk that an attestation has been signed for source -- `attestation_data.justified_slot // EPOCH_LENGTH` -- and target -- `attestation_data.slot // EPOCH_LENGTH`. +1. Save a record to hard disk that an attestation has been signed for source -- `attestation_data.justified_epoch` -- and target -- `slot_to_epoch(attestation_data.slot)`. 2. Generate and broadcast attestation. If the software crashes at some point within this routine, then when the validator comes back online the hard disk has the record of the _potentially_ signed/broadcast attestation and can effectively avoid slashing. From 0f70b200caab6678bf643b1317dad2f18cf38030 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 30 Jan 2019 23:01:38 +0800 Subject: [PATCH 129/136] imperative --- specs/core/0_beacon-chain.md | 48 ++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 3df21ed0e..4aaace782 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -635,7 +635,7 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere ```python def slot_to_epoch(slot: SlotNumber) -> EpochNumber: """ - Returns the epoch number of the given ``slot``. + Return the epoch number of the given ``slot``. """ return slot // EPOCH_LENGTH ``` @@ -645,7 +645,7 @@ def slot_to_epoch(slot: SlotNumber) -> EpochNumber: ```python def get_current_epoch(state: BeaconState) -> EpochNumber: """ - Returns the current epoch of the given ``state``. + Return the current epoch of the given ``state``. """ return slot_to_epoch(state.slot) ``` @@ -655,7 +655,7 @@ def get_current_epoch(state: BeaconState) -> EpochNumber: ```python def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: """ - Returns the starting slot of the given ``epoch``. + Return the starting slot of the given ``epoch``. """ return epoch * EPOCH_LENGTH ``` @@ -664,7 +664,7 @@ def get_epoch_start_slot(epoch: EpochNumber) -> SlotNumber: ```python def is_active_validator(validator: Validator, epoch: EpochNumber) -> bool: """ - Checks if ``validator`` is active. + Check if ``validator`` is active. """ return validator.activation_epoch <= epoch < validator.exit_epoch ``` @@ -674,7 +674,7 @@ def is_active_validator(validator: Validator, epoch: EpochNumber) -> bool: ```python def get_active_validator_indices(validators: List[Validator], epoch: EpochNumber) -> List[ValidatorIndex]: """ - Gets indices of active validators from ``validators``. + Get indices of active validators from ``validators``. """ return [i for i, v in enumerate(validators) if is_active_validator(v, epoch)] ``` @@ -684,7 +684,7 @@ def get_active_validator_indices(validators: List[Validator], epoch: EpochNumber ```python def shuffle(values: List[Any], seed: Bytes32) -> List[Any]: """ - Returns the shuffled ``values`` with ``seed`` as entropy. + Return the shuffled ``values`` with ``seed`` as entropy. """ values_count = len(values) @@ -751,7 +751,7 @@ def split(values: List[Any], split_count: int) -> List[List[Any]]: ```python def get_epoch_committee_count(active_validator_count: int) -> int: """ - Returns the number of committees in one epoch. + Return the number of committees in one epoch. """ return max( 1, @@ -769,8 +769,8 @@ def get_shuffling(seed: Bytes32, validators: List[Validator], epoch: EpochNumber) -> List[List[ValidatorIndex]] """ - Shuffles ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. - Returns a list of ``committees_per_epoch`` committees where each + Shuffle ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. + Return a list of ``committees_per_epoch`` committees where each committee is itself a list of validator indices. """ @@ -795,7 +795,7 @@ def get_shuffling(seed: Bytes32, ```python def get_previous_epoch_committee_count(state: BeaconState) -> int: """ - Returns the number of committees in the previous epoch of the given ``state``. + Return the number of committees in the previous epoch of the given ``state``. """ previous_active_validators = get_active_validator_indices( state.validator_registry, @@ -809,7 +809,7 @@ def get_previous_epoch_committee_count(state: BeaconState) -> int: ```python def get_current_epoch_committee_count(state: BeaconState) -> int: """ - Returns the number of committees in the current epoch of the given ``state``. + Return the number of committees in the current epoch of the given ``state``. """ current_active_validators = get_active_validator_indices( state.validator_registry, @@ -824,7 +824,7 @@ def get_current_epoch_committee_count(state: BeaconState) -> int: def get_crosslink_committees_at_slot(state: BeaconState, slot: SlotNumber) -> List[Tuple[List[ValidatorIndex], ShardNumber]]: """ - Returns the list of ``(committee, shard)`` tuples for the ``slot``. + Return the list of ``(committee, shard)`` tuples for the ``slot``. """ epoch = slot_to_epoch(slot) current_epoch = get_current_epoch(state) @@ -870,7 +870,7 @@ def get_crosslink_committees_at_slot(state: BeaconState, def get_block_root(state: BeaconState, slot: SlotNumber) -> Bytes32: """ - Returns the block root at a recent ``slot``. + Return the block root at a recent ``slot``. """ assert state.slot <= slot + LATEST_BLOCK_ROOTS_LENGTH assert slot < state.slot @@ -885,7 +885,7 @@ def get_block_root(state: BeaconState, def get_randao_mix(state: BeaconState, epoch: EpochNumber) -> Bytes32: """ - Returns the randao mix at a recent ``epoch``. + Return the randao mix at a recent ``epoch``. """ assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state) return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH] @@ -897,7 +897,7 @@ def get_randao_mix(state: BeaconState, def get_active_index_root(state: BeaconState, epoch: EpochNumber) -> Bytes32: """ - Returns the index root at a recent ``epoch``. + Return the index root at a recent ``epoch``. """ assert get_current_epoch(state) - LATEST_INDEX_ROOTS_LENGTH < epoch <= get_current_epoch(state) return state.latest_index_roots[epoch % LATEST_INDEX_ROOTS_LENGTH] @@ -924,7 +924,7 @@ def generate_seed(state: BeaconState, def get_beacon_proposer_index(state: BeaconState, slot: SlotNumber) -> ValidatorIndex: """ - Returns the beacon proposer index for the ``slot``. + Return the beacon proposer index for the ``slot``. """ first_committee, _ = get_crosslink_committees_at_slot(state, slot)[0] return first_committee[slot % len(first_committee)] @@ -950,7 +950,7 @@ def get_attestation_participants(state: BeaconState, attestation_data: AttestationData, bitfield: bytes) -> List[ValidatorIndex]: """ - Returns the participant indices at for the ``attestation_data`` and ``bitfield``. + Return the participant indices at for the ``attestation_data`` and ``bitfield``. """ # Find the committee in the list with the desired shard crosslink_committees = get_crosslink_committees_at_slot(state, attestation_data.slot) @@ -978,7 +978,7 @@ def get_attestation_participants(state: BeaconState, ```python def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: """ - Returns the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``. + Return the effective balance (also known as "balance at stake") for a ``validator`` with the given ``index``. """ return min(state.validator_balances[index], MAX_DEPOSIT_AMOUNT) ``` @@ -989,7 +989,7 @@ def get_effective_balance(state: State, index: ValidatorIndex) -> Gwei: def get_fork_version(fork: Fork, epoch: EpochNumber) -> int: """ - Returns the fork version of the given ``epoch``. + Return the fork version of the given ``epoch``. """ if epoch < fork.epoch: return fork.previous_version @@ -1092,8 +1092,8 @@ def verify_slashable_vote(state: BeaconState, slashable_vote: SlashableVote) -> def is_double_vote(attestation_data_1: AttestationData, attestation_data_2: AttestationData) -> bool: """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable + Assume ``attestation_data_1`` is distinct from ``attestation_data_2``. + Return True if the provided ``AttestationData`` are slashable due to a 'double vote'. """ target_epoch_1 = slot_to_epoch(attestation_data_1.slot) @@ -1107,8 +1107,8 @@ def is_double_vote(attestation_data_1: AttestationData, def is_surround_vote(attestation_data_1: AttestationData, attestation_data_2: AttestationData) -> bool: """ - Assumes ``attestation_data_1`` is distinct from ``attestation_data_2``. - Returns True if the provided ``AttestationData`` are slashable + Assume ``attestation_data_1`` is distinct from ``attestation_data_2``. + Return True if the provided ``AttestationData`` are slashable due to a 'surround vote'. Note: parameter order matters as this function only checks that ``attestation_data_1`` surrounds ``attestation_data_2``. @@ -1995,7 +1995,7 @@ If a validator registry update does _not_ happen do the following: * _Note_ that `state.current_epoch_start_shard` is left unchanged. **Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. - + Regardless of whether or not a validator set change happens, run the following: ```python From 18e430393e011cfe2d9a0a133c7ce145706e76fb Mon Sep 17 00:00:00 2001 From: vbuterin Date: Wed, 30 Jan 2019 15:03:42 +0000 Subject: [PATCH 130/136] Update specs/core/0_beacon-chain.md Co-Authored-By: JustinDrake --- specs/core/0_beacon-chain.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 49868123a..8d4359733 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1960,6 +1960,7 @@ def process_penalties_and_exits(state: BeaconState) -> None: all_indices = list(range(len(state.validator_registry))) eligible_indices = filter(eligible, all_indices) + # Sort in order of exit epoch, and validators that exit within the same epoch exit in order of validator index sorted_indices = sorted(eligible_indices, key=lambda index: state.validator_registry[index].exit_epoch) withdrawn_so_far = 0 for index in sorted_indices: From 3f31285232be44df9933edec11ce825d6527de6e Mon Sep 17 00:00:00 2001 From: Justin Date: Wed, 30 Jan 2019 15:46:27 +0000 Subject: [PATCH 131/136] Fix typo (#528) --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 82038242a..5ba1b242c 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1811,7 +1811,7 @@ Case 1: `epochs_since_finality <= 4`: * Expected FFG source: * Any [validator](#dfn-validator) `index` in `previous_epoch_justified_attester_indices` gains `base_reward(state, index) * previous_epoch_justified_attesting_balance // previous_total_balance`. - * Any [active validator](#dfn-active-validator) `v` not in `previous_epoch_justified_attester_indices` loses `base_reward(state, index)`. + * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_justified_attester_indices` loses `base_reward(state, index)`. * Expected FFG target: * Any [validator](#dfn-validator) `index` in `previous_epoch_boundary_attester_indices` gains `base_reward(state, index) * previous_epoch_boundary_attesting_balance // previous_total_balance`. * Any [active validator](#dfn-active-validator) `index` not in `previous_epoch_boundary_attester_indices` loses `base_reward(state, index)`. From cc259f98d967af0b8e8cde0ab76a34dfde3e79fe Mon Sep 17 00:00:00 2001 From: terence tsao Date: Wed, 30 Jan 2019 16:47:57 +0100 Subject: [PATCH 132/136] Update 0_beacon-chain-validator.md (#531) --- specs/validator/0_beacon-chain-validator.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 80b0fe048..1a244f263 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -47,7 +47,7 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Justified block root](#justified-block-root) - [Construct attestation](#construct-attestation) - [Data](#data) - - [Participation bitfield](#participation-bitfield) + - [Aggregation bitfield](#aggregation-bitfield) - [Custody bitfield](#custody-bitfield) - [Aggregate signature](#aggregate-signature) - [How to avoid slashing](#how-to-avoid-slashing) @@ -291,7 +291,7 @@ Next the validator creates `attestation`, an [`Attestation`](https://github.com/ Set `attestation.data = attestation_data` where `attestation_data` is the `AttestationData` object defined in the previous section, [attestation data](#attestation-data). -##### Participation bitfield +##### Aggregation bitfield * Let `aggregation_bitfield` be a byte array filled with zeros of length `(len(committee) + 7) // 8`. * Let `index_into_committee` be the index into the validator's `committee` at which `validator_index` is located. From 5aba345eec51f6109354ec52c5a6f17badf85360 Mon Sep 17 00:00:00 2001 From: Hsiao-Wei Wang Date: Wed, 30 Jan 2019 23:50:17 +0800 Subject: [PATCH 133/136] Remove extraneous space --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index 4aaace782..27c2f5c41 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1995,7 +1995,7 @@ If a validator registry update does _not_ happen do the following: * _Note_ that `state.current_epoch_start_shard` is left unchanged. **Invariant**: the active index root that is hashed into the shuffling seed actually is the `hash_tree_root` of the validator set that is used for that epoch. - + Regardless of whether or not a validator set change happens, run the following: ```python From 989aae5dc665c59c7cd5e3b19b64078402ace460 Mon Sep 17 00:00:00 2001 From: Ben Edgington Date: Wed, 30 Jan 2019 15:50:37 +0000 Subject: [PATCH 134/136] Fix missed updates from slot to epoch (#530) --- specs/validator/0_beacon-chain-validator.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 1a244f263..cacfebe7d 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -43,7 +43,7 @@ __NOTICE__: This document is a work-in-progress for researchers and implementers - [Epoch boundary root](#epoch-boundary-root) - [Shard block root](#shard-block-root) - [Latest crosslink root](#latest-crosslink-root) - - [Justified slot](#justified-slot) + - [Justified epoch](#justified-epoch) - [Justified block root](#justified-block-root) - [Construct attestation](#construct-attestation) - [Data](#data) @@ -251,7 +251,7 @@ Set `attestation_data.slot = slot` where `slot` is the current slot of which the ##### Shard -Set `attestation_data.shard = shard` where `shard` is the shard associated with the validator's committee defined by `get_shard_committees_at_slot`. +Set `attestation_data.shard = shard` where `shard` is the shard associated with the validator's committee defined by `get_crosslink_committees_at_slot`. ##### Beacon block root @@ -273,15 +273,15 @@ _Note:_ This is a stub for phase 0. Set `attestation_data.latest_crosslink_root = state.latest_crosslinks[shard].shard_block_root` where `state` is the beacon state at `head` and `shard` is the validator's assigned shard. -##### Justified slot +##### Justified epoch -Set `attestation_data.justified_slot = state.justified_slot` where `state` is the beacon state at `head`. +Set `attestation_data.justified_epoch = state.justified_epoch` where `state` is the beacon state at `head`. ##### Justified block root -Set `attestation_data.justified_block_root = hash_tree_root(justified_block)` where `justified_block` is the block at `state.justified_slot` in the chain defined by `head`. +Set `attestation_data.justified_block_root = hash_tree_root(justified_block)` where `justified_block` is the block at `state.justified_epoch` in the chain defined by `head`. -_Note:_ This can be looked up in the state using `get_block_root(state, justified_slot)`. +_Note:_ This can be looked up in the state using `get_block_root(state, justified_epoch)`. #### Construct attestation From ad5583c79f0c0be64f53e5247de1e2535442e058 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 30 Jan 2019 11:03:52 -0800 Subject: [PATCH 135/136] remove extra line --- specs/core/0_beacon-chain.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index e4a1b1b23..d8eaa2547 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -311,7 +311,6 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git { # Validator indices 'validator_indices': ['uint64'], - # Custody bitfield # Attestation data 'data': AttestationData, # Custody bitfield From ed80daa489e3e53c40dba79ba59447bd16ca4da1 Mon Sep 17 00:00:00 2001 From: Danny Ryan Date: Wed, 30 Jan 2019 11:04:35 -0800 Subject: [PATCH 136/136] typo in comment --- specs/core/0_beacon-chain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index d8eaa2547..e5a8687f2 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -1089,7 +1089,7 @@ def verify_slashable_attestation(state: BeaconState, slashable_attestation: Slas def is_double_vote(attestation_data_1: AttestationData, attestation_data_2: AttestationData) -> bool: """ - Check if the ``attestation_data_1`` and ``attestation_data_2`` have the same target. + Check if ``attestation_data_1`` and ``attestation_data_2`` have the same target. """ target_epoch_1 = slot_to_epoch(attestation_data_1.slot) target_epoch_2 = slot_to_epoch(attestation_data_2.slot)