Big refactor + simplification

This commit is contained in:
vbuterin 2018-11-26 17:12:39 -05:00 committed by GitHub
parent 386dab1140
commit 170962c80f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 105 additions and 93 deletions

View File

@ -51,7 +51,7 @@ The primary source of load on the beacon chain are "attestations". Attestations
| `POW_RECEIPT_ROOT_VOTING_PERIOD` | 2**10 (= 1,024) | slots | ~1.7 hours |
| `SLASHING_WHISTLEBLOWER_REWARD_DENOMINATOR` | 2**9 (= 512) |
| `BASE_REWARD_QUOTIENT` | 2**11 (= 2,048) | — |
| `INCLUDER_REWARD_QUOTIENT` | 2**14 (= 16,384) | — |
| `INCLUDER_REWARD_SHARE_QUOTIENT` | 2**3 (= 8) | — |
| `MAX_VALIDATOR_CHURN_QUOTIENT` | 2**5 (= 32) | — |
| `POW_CONTRACT_MERKLE_TREE_DEPTH` | 2**5 (= 32) | - |
| `MAX_ATTESTATION_COUNT` | 2**7 (= 128) | - |
@ -138,29 +138,39 @@ An `AttestationRecord` has the following fields:
```python
{
# Slot number
'slot': 'uint64',
# Shard number
'shard': 'uint64',
# Beacon block hashes not part of the current chain, oldest to newest
'parent_hashes': ['hash32'],
# Shard block hash being attested to
'shard_block_hash': 'hash32',
# Last crosslink hash
'last_crosslink_hash': 'hash32',
# Root of data between last hash and this one
'shard_block_combined_data_root': 'hash32',
data: AttestationSignedData
# Attester participation bitfield (2 bits per attester)
'attester_bitfield': 'bytes',
# Slot of last justified beacon block
'justified_slot': 'uint64',
# Hash of last justified beacon block
'justified_block_hash': 'hash32',
# Proof of custody bitfield
'poc_bitfield': 'bytes',
# BLS aggregate signature
'aggregate_sig': ['uint384']
}
```
`AttestationSignedData`:
```python
{
# Slot number
'slot': 'uint64',
# Shard number
'shard': 'uint64',
# Hash of the block we're signing
'block_hash': 'hash32',
# Hash of the ancestor at the cycle boundary
'cycle_boundary_hash': 'hash32',
# Shard block hash being attested to
'shard_block_hash': 'hash32',
# Last crosslink hash
'last_crosslink_hash': 'hash32',
# Slot of last justified beacon block
'justified_slot': 'uint64',
# Hash of last justified beacon block
'justified_block_hash': 'hash32',
}
```
A `ProposalSignedData` has the following fields:
```python
@ -174,27 +184,6 @@ A `ProposalSignedData` has the following fields:
}
```
An `AttestationSignedData` has the following fields:
```python
{
# Slot number
'slot': 'uint64',
# Shard number
'shard': 'uint64',
# CYCLE_LENGTH parent hashes
'parent_hashes': ['hash32'],
# Shard block hash
'shard_block_hash': 'hash32',
# Last crosslink hash
'last_crosslink_hash': 'hash32',
# Root of data between last hash and this one
'shard_block_combined_data_root': 'hash32',
# Slot of last justified beacon block referenced in the attestation
'justified_slot': 'uint64'
}
```
A `SpecialRecord` has the following fields:
```python
@ -251,7 +240,7 @@ The `BeaconState` has the following fields:
'post_fork_version': 'uint64',
'fork_slot_number': 'uint64',
# Attestations not yet processed
'pending_attestations': [AttestationRecord],
'pending_attestations': [ProcessedAttestations],
# recent beacon block hashes needed to process attestations, older to newer
'recent_block_hashes': ['hash32'],
# RANDAO state
@ -328,6 +317,21 @@ A `CandidatePoWReceiptRootRecord` object contains the following fields:
}
```
A `ProcessedAttestation` object has the following fields:
```python
{
# Signed data
'data': AttestationSignedData,
# Attester participation bitfield (2 bits per attester)
'attester_bitfield': 'bytes',
# Proof of custody bitfield
'poc_bitfield': 'bytes',
# Slot in which it was included
'slot_included': 'uint64'
}
```
## Beacon chain processing
The beacon chain is the "main chain" of the PoS system. The beacon chain's main responsibilities are:
@ -557,19 +561,17 @@ The following is a function that determines the validators that participated in
```python
def get_attestation_participants(state: State,
attestation: AttestationRecord) -> Tuple[List[int], List[int]]:
sncs_for_slot = get_shards_and_committees_for_slot(state, attestation.slot)
snc = [x for x in sncs_for_slot if x.shard == attestation.shard][0]
assert len(attestation.attester_bitfield) == ceil_div8(len(snc.committee) * 2)
bit0_participants, bit1_participants = [], []
attestation_data: AttestationSignedData,
attester_bitfield: bytes) -> List[int]:
sncs_for_slot = get_shards_and_committees_for_slot(state, attestation_data.slot)
snc = [x for x in sncs_for_slot if x.shard == attestation_data.shard][0]
assert len(attester_bitfield) == ceil_div8(len(snc.committee))
participants = []
for i, vindex in snc.committee:
bits = (attestation.attester_bitfield[i//4] >> (3 - (i % 4)) * 2) % 4
assert bits in (0, 2, 3)
if bits == 2:
bit0_participants.append(vindex)
elif bits == 3:
bit1_participants.append(vindex)
return bit0_participants, bit1_participants
bit = (attester_bitfield[i//8] >> (7 - (i % 8))) % 2
if bit == 1:
participants.append(vindex)
return participants
```
We define another set of helpers to be used throughout: `bytes1(x): return x.to_bytes(1, 'big')`, `bytes2(x): return x.to_bytes(2, 'big')`, and so on for all integers, particularly 1, 2, 3, 4, 8, 32.
@ -860,21 +862,18 @@ def update_ancestor_hashes(parent_ancestor_hashes: List[Hash32],
Verify that there are at most `MAX_ATTESTATION_COUNT` `AttestationRecord` objects.
For each `AttestationRecord` object:
For each `AttestationRecord` object `obj`:
* Verify that `slot <= block.slot - MIN_ATTESTATION_INCLUSION_DELAY` and `slot >= max(parent.slot - CYCLE_LENGTH + 1, 0)`.
* Verify that `justified_slot` is equal to `justification_source if slot >= block.slot - (block.slot % CYCLE_LENGTH) else prev_cycle_justification_source`
* Verify that `justified_block_hash` is the hash of the block in the current chain at the slot -- `justified_slot`.
* Verify that either `last_crosslink_hash` or `shard_block_hash` equals `state.crosslinks[shard].shard_block_hash`.
* Compute `full_parent_hashes` = `[get_block_hash(state, block, slot - CYCLE_LENGTH + i) for i in range(1, CYCLE_LENGTH - len(parent_hashes) + 1)] + parent_hashes` (eg, if `CYCLE_LENGTH = 4`, `slot = 5`, the actual block hashes starting from slot 0 are `Z A B C D E F G H I J`, and `parent_hashes = [D', E']` then `full_parent_hashes = [B, C, D' E']`). Note that when *creating* an attestation for a block, the hash of that block itself won't yet be in the `state`, so you would need to add it explicitly.
* Verify that `obj.data.slot <= block.slot - MIN_ATTESTATION_INCLUSION_DELAY` and `obj.data.slot >= max(parent.slot - CYCLE_LENGTH + 1, 0)`.
* Verify that `obj.data.justified_slot` is equal to `justification_source if state.last_state_recalculation_slot else prev_cycle_justification_source`
* Verify that `obj.data.justified_block_hash` is the hash of the block in the current chain at the slot -- `obj.data.justified_slot`.
* Verify that either `obj.data.last_crosslink_hash` or `obj.data.shard_block_hash` equals `state.crosslinks[shard].shard_block_hash`.
* `aggregate_sig` verification:
* Let `bit0_attestation_indices, bit1_attestation_indices = get_attestation_participants(state, obj)` (and verify that the method returns successfully)
* Let `bit0_group_public_key = BLSAddPubkeys(bit0_attestation_indices)` and `bit1_group_public_key = BLSAddPubkeys(bit1_attestation_indices)`.
* Let `data = AttestationSignedData(slot, shard, parent_hashes, shard_block_hash, last_crosslinked_hash, shard_block_combined_data_root, justified_slot)`.
* Check `BLSVerify(pubkey=group_public_key, msg=data, sig=aggregate_sig, domain=get_domain(state, slot, DOMAIN_ATTESTATION))`.
* Let `participants = get_attestation_participants(state, obj.data, obj.attester_bitfield)`
* Let `group_public_key = BLSAddPubkeys([state.validators[v].pubkey for v in participants])`
* Check `BLSVerify(pubkey=group_public_key, msg=obj.data, sig=aggregate_sig, domain=get_domain(state, slot, DOMAIN_ATTESTATION))`.
* [TO BE REMOVED IN PHASE 1] Verify that `shard_block_hash == bytes([0] * 32)`.
Extend the list of `AttestationRecord` objects in the `state` with those included in the block, ordering the new additions in the same order as they came in the block, and replacing `obj.parent_hashes` with the calculated value of `full_parent_hashes`.
* Append `ProcessedAttestation(data=obj.data, attester_bitfield=obj.attester_bitfield, poc_bitfield=obj.poc_bitfield, slot_included=block.slot)` to `state.pending_attestations`.
### Verify proposer signature
@ -1001,39 +1000,52 @@ _Note: `last_state_recalculation_slot` will always be a multiple of `CYCLE_LENGT
#### Precomputation
All validators:
* Let `active_validators = [state.validators[i] for i in get_active_validator_indices(state.validators)]`.
* Let `total_balance = sum([balance_at_stake(v) for v in active_validators])`.
* Let `this_cycle_attestations = [a for a in state.pending_attestations if s <= a.slot < s + CYCLE_LENGTH]`. (note: this is the set of attestations _of slots in the cycle `s...s+CYCLE_LENGTH-1`_, not attestations _that got included in the chain during the cycle `s...s+CYCLE_LENGTH-1`_)
Validators justifying the cycle boundary block at the start of the current cycle:
* Let `this_cycle_attestations = [a for a in state.pending_attestations if s <= a.data.slot < s + CYCLE_LENGTH]`. (note: this is the set of attestations _of slots in the cycle `s...s+CYCLE_LENGTH-1`_, not attestations _that got included in the chain during the cycle `s...s+CYCLE_LENGTH-1`_)
* Let `this_cycle_boundary_attestations = [a for a in this_cycle_attestations if a.data.cycle_boundary_hash == get_block_hash(state, block, s) a.justified_slot == state.justification_source]`.
* Let `this_cycle_boundary_attesters` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.attester_bitfield) for a in this_cycle_boundary_attestations]`.
* Let `this_cycle_boundary_attesting_balance = sum([balance_at_stake(v) for v in this_cycle_boundary_attesters])`.
Validators justifying the cycle boundary block at the start of the previous cycle:
* Let `prev_cycle_attestations = [a for a in state.pending_attestations if s - CYCLE_LENGTH <= a.slot < s]`.
* Let `this_cycle_s_attestations = [a for a in this_cycle_attestations if get_block_hash(state, block, s) in a.parent_hashes and a.justified_slot == state.justification_source]`.
* Let `prev_s_attestations = [a for a in this_cycle_attestations + prev_cycle_attestations if get_block_hash(state, block, s - CYCLE_LENGTH) in a.parent_hashes and a.justified_slot == state.prev_cycle_justification_source]`.
* Let `s_attesters` be the union of the validator index sets given by `[get_attestation_participants(state, a) for a in this_cycle_s_attestations]`.
* Let `prev_s_attesters` be the union of the validator index sets given by `[get_attestation_participants(state, a) for a in prev_s_attestations]`.
* Let `total_balance_attesting_at_s = sum([balance_at_stake(v) for v in s_attesters])`.
* Let `total_balance_attesting_at_prev_s = sum([balance_at_stake(v) for v in prev_s_attesters])`.
* For every `ShardAndCommittee` object `obj` in `shard_and_committee_for_slots`, let:
* `attesting_validators(obj, shard_block_hash)` be the union of the validator index sets given by `[get_attestation_participants(state, a) for a in this_cycle_attestations + prev_cycle_attestations if a.shard == obj.shard and a.shard_block_hash == shard_block_hash]`
* `attesting_validators(obj)` be equal to `attesting_validators(obj, shard_block_hash)` for the value of `shard_block_hash` such that `sum([balance_at_stake(v) for v in attesting_validators(obj, shard_block_hash)])` is maximized (ties broken by favoring lower `shard_block_hash` values)
* `total_attesting_balance(obj)` be the maximal sum balance
* `winning_hash(obj)` be the winning `shard_block_hash` value
* `total_balance(obj) = sum([balance_at_stake(v) for v in obj.committee])`
* Let `prev_cycle_boundary_attestations = [a for a in this_cycle_attestations + prev_cycle_attestations if a.cycle_boundary_hash == get_block_hash(state, block, s - CYCLE_LENGTH) and a.justified_slot == state.prev_cycle_justification_source]`.
* Let `prev_cycle_boundary_attesters` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.attester_bitfield) for a in prev_cycle_boundary_attestations]`.
* Let `prev_cycle_boundary_attesting_balance = sum([balance_at_stake(v) for v in prev_cycle_boundary_attesters])`.
For every `ShardAndCommittee` object `obj` in `shard_and_committee_for_slots`, let:
* `attesting_validators(obj, shard_block_hash)` be the union of the validator index sets given by `[get_attestation_participants(state, a.data, a.attester_bitfield) for a in this_cycle_attestations + prev_cycle_attestations if a.shard == obj.shard and a.shard_block_hash == shard_block_hash]`
* `attesting_validators(obj)` be equal to `attesting_validators(obj, shard_block_hash)` for the value of `shard_block_hash` such that `sum([balance_at_stake(v) for v in attesting_validators(obj, shard_block_hash)])` is maximized (ties broken by favoring lower `shard_block_hash` values)
* `total_attesting_balance(obj)` be the sum of the balances-at-stake of `attesting_validators(obj)`
* `winning_hash(obj)` be the winning `shard_block_hash` value
* `total_balance(obj) = sum([balance_at_stake(v) for v in obj.committee])`
Let `inclusion_slot(a)` equal the slot in which attestation `a` was included, and `inclusion_distance(a) = inclusion_slot(a) - a.slot`. Let `inclusion_slot(v)` equal `inclusion_distance(a)` for the attestation `a` where `v` is in `get_attestation_participants(state, a)`, and define `inclusion_distance(v)` similarly. We define a function `adjust_for_inclusion_distance(magnitude, dist)` which adjusts the reward of an attestation based on how long it took to get included (the longer, the lower the reward). Returns a value between 0 and `magnitude`
Let `inclusion_slot(v)` equal `a.slot_included` for the attestation `a` where `v` is in `get_attestation_participants(state, a.data, a.attester_bitfield)`, and `inclusion_distance(v) = a.slot_included - a.data.slot` for the same attestation. We define a function `adjust_for_inclusion_distance(magnitude, dist)` which adjusts the reward of an attestation based on how long it took to get included (the longer, the lower the reward). Returns a value between 0 and `magnitude`
```python
def adjust_for_inclusion_distance(magnitude: int, dist: int) -> int:
return magnitude // 2 + (magnitude // 2) * MIN_ATTESTATION_INCLUSION_DELAY // dist
```
For any validator `v`, `base_reward(v) = balance_at_stake(v) // BASE_REWARD_QUOTIENT`
#### Adjust justified slots and crosslink status
* Set `state.justified_slot_bitfield = (state.justified_slot_bitfield * 2) % 2**64`.
* If `3 * total_balance_attesting_at_prev_s >= 2 * total_balance` then set `state.justified_slot_bitfield &= 2` (ie. flip the second lowest bit to 1) and `new_justification_source = prev_s`.
* If `3 * total_balance_attesting_at_s >= 2 * total_balance` then set `state.justified_slot_bitfield &= 1` (ie. flip the lowest bit to 1) and `new_justification_source = s`.
* If `justification_source == prev_s and state.justified_slot_bitfield % 4 == 3`, set `last_finalized_slot = justification_source`.
* If `justification_source == prev_s - 2 * CYCLE_LENGTH and state.justified_slot_bitfield % 16 in (15, 14)`, set `last_finalized_slot = justification_source`.
* If `justification_source == prev_s - CYCLE_LENGTH and state.justified_slot_bitfield % 8 == 7`, set `last_finalized_slot = justification_source`.
* Set `prev_cycle_justification_source = justification_source` and `justification_source = new_justification_source`.
* If `3 * prev_cycle_boundary_attesting_balance >= 2 * total_balance` then set `state.justified_slot_bitfield &= 2` (ie. flip the second lowest bit to 1) and `new_justification_source = s - CYCLE_LENGTH`.
* If `3 * this_cycle_boundary_attesting_balance >= 2 * total_balance` then set `state.justified_slot_bitfield &= 1` (ie. flip the lowest bit to 1) and `new_justification_source = s`.
* If `state.justification_source == s - CYCLE_LENGTH and state.justified_slot_bitfield % 4 == 3`, set `last_finalized_slot = justification_source`.
* If `state.justification_source == s - CYCLE_LENGTH - 2 * CYCLE_LENGTH and state.justified_slot_bitfield % 16 in (15, 14)`, set `last_finalized_slot = justification_source`.
* If `state.justification_source == s - CYCLE_LENGTH - CYCLE_LENGTH and state.justified_slot_bitfield % 8 == 7`, set `state.last_finalized_slot = state.justification_source`.
* Set `state.prev_cycle_justification_source = state.justification_source` and if `new_justification_source` has been set, set `state.justification_source = new_justification_source`.
For every `ShardAndCommittee` object `obj`:
@ -1046,26 +1058,26 @@ Note: When applying penalties in the following balance recalculations implemente
* Let `total_balance_in_eth = total_balance // GWEI_PER_ETH`.
* Let `reward_quotient = BASE_REWARD_QUOTIENT * int_sqrt(total_balance_in_eth)`. (The per-slot maximum interest rate is `2/reward_quotient`.)
* Let `quadratic_penalty_quotient = SQRT_E_DROP_TIME**2`. (The portion lost by offline validators after `D` cycles is about `D*D/2/quadratic_penalty_quotient`.)
* Let `time_since_finality = slot - last_finalized_slot`.
* Let `time_since_finality = slot - state.last_finalized_slot`.
Case 1: `time_since_finality <= 4 * CYCLE_LENGTH`:
* Any validator `v` in `prev_s_attesters` gains `adjust_for_inclusion_distance(balance_at_stake(v) // reward_quotient * (prev_s_attesters - total_balance) // total_balance, inclusion_distance(v))`.
* Any active validator `v` not in `prev_s_attesters` loses `balance_at_stake(v) // reward_quotient`.
* Any validator `v` in `prev_cycle_boundary_attesters` gains `adjust_for_inclusion_distance(base_reward(v) * prev_cycle_boundary_attesting_balance // total_balance, inclusion_distance(v))`.
* Any active validator `v` not in `prev_cycle_boundary_attesters` loses `base_reward(v)`.
Case 2: `time_since_finality > 4 * CYCLE_LENGTH`:
* Any validator in `prev_s_attesters` sees their balance unchanged.
* Any active validator `v` not in `prev_s_attesters`, and any validator with `status == PENALIZED`, loses `balance_at_stake(v) // reward_quotient + balance_at_stake(v) * time_since_finality // quadratic_penalty_quotient`.
* Any validator in `prev_cycle_boundary_attesters` sees their balance unchanged.
* Any active validator `v` not in `prev_cycle_boundary_attesters`, and any validator with `status == PENALIZED`, loses `base_reward(v) + balance_at_stake(v) * time_since_finality // quadratic_penalty_quotient`.
For each `v` in `prev_s_attesters`, the validator `proposer = get_beacon_proposer(state, inclusion_slot(v))` gains `balance_at_stake(proposer) // INCLUDER_REWARD_QUOTIENT`.
For each `v` in `prev_cycle_boundary_attesters`, the validator `proposer = get_beacon_proposer(state, inclusion_slot(v))` gains `base_reward(v) // INCLUDER_REWARD_SHARE_QUOTIENT`.
#### Balance recalculations related to crosslink rewards
For every `ShardAndCommittee` object `obj` in `shard_and_committee_for_slots[:CYCLE_LENGTH]` (ie. the objects corresponding to the slot before the current one), for each `v in obj.committee`, where `v = state.validators[b]` adjust balances as follows:
For every `ShardAndCommittee` object `obj` in `shard_and_committee_for_slots[:CYCLE_LENGTH]` (ie. the objects corresponding to the cycle before the current one), for each `v` in `[validators[index] for index in obj.committee]`, adjust balances as follows:
* If `v in attesting_validators(obj)`, `v.balance += adjust_for_inclusion_distance(balance_at_stake(v) // reward_quotient * total_attesting_balance(obj) // total_balance(obj)), inclusion_distance(v))`.
* If `v not in attesting_validators(obj)`, `v.balance -= balance_at_stake(v) // reward_quotient`.
* If `v in attesting_validators(obj)`, `v.balance += adjust_for_inclusion_distance(base_reward(v) * total_attesting_balance(obj) // total_balance(obj)), inclusion_distance(v))`.
* If `v not in attesting_validators(obj)`, `v.balance -= base_reward(v)`.
#### PoW chain related rules
@ -1174,7 +1186,7 @@ def change_validators(validators: List[ValidatorRecord], current_slot: int) -> N
And perform the following updates to the `state`:
* Set `state.validator_set_change_slot = s`
* Set `state.validator_set_change_slot = s + CYCLE_LENGTH`
* Set `state.shard_and_committee_for_slots[:CYCLE_LENGTH] = state.shard_and_committee_for_slots[CYCLE_LENGTH:]`
* Let `state.next_start_shard = (shard_and_committee_for_slots[-1][-1].shard + 1) % SHARD_COUNT`
* Set `state.shard_and_committee_for_slots[CYCLE_LENGTH:] = get_new_shuffling(state.next_shuffling_seed, validators, next_start_shard)`