2019-11-15 20:11:42 +00:00
# Ethereum 2.0 Phase 1 -- The Beacon Chain for Shards
2019-10-12 03:05:08 +00:00
**Notice**: This document is a work-in-progress for researchers and implementers.
## Table of contents
<!-- TOC -->
2019-11-15 20:11:42 +00:00
TODO
2019-10-12 03:05:08 +00:00
<!-- /TOC -->
## Introduction
2019-11-15 20:11:42 +00:00
This document describes the extensions made to the Phase 0 design of The Beacon Chain
to facilitate the new shards as part of Phase 1 of Eth2.
2019-10-12 03:05:08 +00:00
2019-11-18 23:40:02 +00:00
## Custom types
We define the following Python custom types for type hinting and readability:
| Name | SSZ equivalent | Description |
| - | - | - |
| `Shard` | `uint64` | a shard number |
2019-10-12 03:05:08 +00:00
## Configuration
2019-11-15 20:11:42 +00:00
Configuration is not namespaced. Instead it is strictly an extension;
no constants of phase 0 change, but new constants are adopted for changing behaviors.
2019-10-12 03:05:08 +00:00
### Misc
2019-10-13 06:52:51 +00:00
| Name | Value | Unit | Duration |
| - | - | - | - |
2019-10-12 03:05:08 +00:00
| `MAX_SHARDS` | `2**10` (= 1024) |
| `ACTIVE_SHARDS` | `2**6` (= 64) |
2019-10-13 06:52:51 +00:00
| `ONLINE_PERIOD` | `2**3` (= 8) | epochs | ~51 min |
| `LIGHT_CLIENT_COMMITTEE_SIZE` | `2**7` (= 128) |
2019-11-05 18:31:59 +00:00
| `LIGHT_CLIENT_COMMITTEE_PERIOD` | `2**8` (= 256) | epochs | ~27 hours |
2019-11-12 14:13:47 +00:00
| `SHARD_COMMITTEE_PERIOD` | `2**8` (= 256) | epochs | ~27 hours |
2019-10-29 17:43:13 +00:00
| `SHARD_BLOCK_CHUNK_SIZE` | `2**18` (= 262,144) | |
2019-11-20 03:43:32 +00:00
| `MAX_SHARD_BLOCK_CHUNKS` | `2**2` (= 4) | |
2019-11-20 03:15:15 +00:00
| `TARGET_SHARD_BLOCK_SIZE` | `3 * 2**16` (= 196,608) | |
2019-10-29 17:43:13 +00:00
| `SHARD_BLOCK_OFFSETS` | `[1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233]` | |
| `MAX_SHARD_BLOCKS_PER_ATTESTATION` | `len(SHARD_BLOCK_OFFSETS)` | |
2019-11-20 03:43:32 +00:00
| `EMPTY_CHUNK_ROOT` | `hash_tree_root(BytesN[SHARD_BLOCK_CHUNK_SIZE]())` | |
2019-10-29 18:12:55 +00:00
| `MAX_GASPRICE` | `2**14` (= 16,384) | Gwei | |
2019-11-07 13:21:36 +00:00
| `MIN_GASPRICE` | `2**5` (= 32) | Gwei | |
2019-10-29 18:12:55 +00:00
| `GASPRICE_ADJUSTMENT_COEFFICIENT` | `2**3` (= 8) | |
2019-11-18 23:40:02 +00:00
| `DOMAIN_LIGHT_CLIENT` | `192` | |
2019-11-12 14:13:47 +00:00
| `DOMAIN_SHARD_COMMITTEE` | `192` | |
2019-11-03 20:06:19 +00:00
| `DOMAIN_SHARD_PROPOSAL` | `193` | |
2019-10-12 03:05:08 +00:00
## Containers
2019-11-03 20:06:19 +00:00
### `ShardBlockWrapper`
2019-11-06 22:19:00 +00:00
_Wrapper for being broadcasted over the network._
2019-11-03 20:06:19 +00:00
```python
class ShardBlockWrapper(Container):
shard_parent_root: Hash
beacon_parent_root: Hash
slot: Slot
2019-11-20 03:43:32 +00:00
body: BytesN[MAX_SHARD_BLOCK_CHUNKS * SHARD_BLOCK_CHUNK_SIZE]
2019-11-03 20:06:19 +00:00
signature: BLSSignature
```
2019-11-06 22:19:00 +00:00
### `ShardSignableHeader`
2019-11-03 20:06:19 +00:00
```python
2019-11-06 22:19:00 +00:00
class ShardSignableHeader(Container):
2019-11-03 20:06:19 +00:00
shard_parent_root: Hash
beacon_parent_root: Hash
slot: Slot
body_root: Hash
```
2019-10-29 17:43:13 +00:00
### `ShardState`
2019-10-27 01:01:10 +00:00
2019-10-29 17:43:13 +00:00
```python
class ShardState(Container):
slot: Slot
gasprice: Gwei
2019-11-06 22:19:00 +00:00
data: Hash
2019-11-12 13:27:34 +00:00
latest_block_root: Hash
2019-10-29 17:43:13 +00:00
```
2019-10-27 01:01:10 +00:00
2019-11-15 20:11:42 +00:00
### New `AttestationData`
2019-10-12 03:05:08 +00:00
```python
2019-11-18 22:07:50 +00:00
class AttestationData(phase0.AttestationData):
2019-10-12 03:05:08 +00:00
slot: Slot
2019-10-27 01:01:10 +00:00
index: CommitteeIndex
2019-10-12 03:05:08 +00:00
# LMD GHOST vote
beacon_block_root: Hash
# FFG vote
source: Checkpoint
target: Checkpoint
2019-11-12 13:27:34 +00:00
# Current-slot shard block root
head_shard_root: Hash
2019-11-01 14:58:13 +00:00
# Shard transition root
shard_transition_root: Hash
2019-10-27 01:01:10 +00:00
```
2019-10-29 17:43:13 +00:00
### `ShardTransition`
2019-10-27 01:01:10 +00:00
```python
2019-10-29 18:33:29 +00:00
class ShardTransition(Container):
2019-10-29 17:43:13 +00:00
# Starting from slot
start_slot: Slot
2019-10-27 01:01:10 +00:00
# Shard block lengths
2019-11-01 14:58:13 +00:00
shard_block_lengths: List[uint64, MAX_SHARD_BLOCKS_PER_ATTESTATION]
2019-10-12 03:05:08 +00:00
# Shard data roots
2019-11-01 14:58:13 +00:00
shard_data_roots: List[List[Hash, MAX_SHARD_BLOCK_CHUNKS], MAX_SHARD_BLOCKS_PER_ATTESTATION]
2019-11-06 22:19:00 +00:00
# Intermediate shard states
2019-11-06 20:53:01 +00:00
shard_states: List[ShardState, MAX_SHARD_BLOCKS_PER_ATTESTATION]
2019-11-03 20:06:19 +00:00
# Proposer signature aggregate
proposer_signature_aggregate: BLSSignature
2019-10-12 03:05:08 +00:00
```
2019-11-15 20:11:42 +00:00
### New `Attestation`
2019-10-12 03:05:08 +00:00
```python
class Attestation(Container):
aggregation_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE]
data: AttestationData
2019-10-29 17:43:13 +00:00
custody_bits: List[Bitlist[MAX_VALIDATORS_PER_COMMITTEE], MAX_SHARD_BLOCKS_PER_ATTESTATION]
2019-10-27 01:01:10 +00:00
signature: BLSSignature
```
2019-11-12 13:27:34 +00:00
### `AttestationAndCommittee`
2019-10-27 01:01:10 +00:00
```python
2019-11-12 13:27:34 +00:00
class AttestationAndCommittee(Container):
2019-11-04 16:50:09 +00:00
committee: List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE]
attestation: Attestation
2019-10-27 01:01:10 +00:00
```
2019-10-13 06:52:51 +00:00
### `CompactCommittee`
```python
class CompactCommittee(Container):
pubkeys: List[BLSPubkey, MAX_VALIDATORS_PER_COMMITTEE]
compact_validators: List[uint64, MAX_VALIDATORS_PER_COMMITTEE]
```
2019-10-27 01:01:10 +00:00
### `AttestationCustodyBitWrapper`
2019-10-28 09:18:27 +00:00
```python
2019-10-27 01:01:10 +00:00
class AttestationCustodyBitWrapper(Container):
attestation_root: Hash
2019-11-01 14:58:13 +00:00
block_index: uint64
2019-11-18 21:04:29 +00:00
bit: boolean
2019-10-27 01:01:10 +00:00
```
2019-11-16 10:13:47 +00:00
### New extended `PendingAttestation`
2019-11-01 14:58:13 +00:00
```python
2019-11-19 19:16:40 +00:00
class PendingAttestation(Container):
2019-11-01 14:58:13 +00:00
aggregation_bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE]
data: AttestationData
inclusion_delay: Slot
proposer_index: ValidatorIndex
2019-11-18 21:04:29 +00:00
crosslink_success: boolean
2019-11-01 14:58:13 +00:00
```
2019-11-15 20:11:42 +00:00
### New extended `Validator`
```python
2019-11-19 19:16:40 +00:00
class Validator(Container):
2019-11-15 20:11:42 +00:00
pubkey: BLSPubkey
withdrawal_credentials: Hash # Commitment to pubkey for withdrawals
effective_balance: Gwei # Balance at stake
slashed: boolean
# Status epochs
activation_eligibility_epoch: Epoch # When criteria for activation were met
activation_epoch: Epoch
exit_epoch: Epoch
withdrawable_epoch: Epoch # When validator can withdraw funds
2019-11-16 10:13:47 +00:00
# Custody game
2019-11-15 20:11:42 +00:00
# next_custody_secret_to_reveal is initialised to the custody period
# (of the particular validator) in which the validator is activated
# = get_custody_period_for_validator(...)
next_custody_secret_to_reveal: uint64
max_reveal_lateness: Epoch
```
2019-10-12 14:59:51 +00:00
2019-11-15 22:42:28 +00:00
### New extended `BeaconBlockBody`
```python
2019-11-19 19:16:40 +00:00
class BeaconBlockBody(Container):
2019-11-15 22:42:28 +00:00
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Slashings
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
# Attesting
attestations: List[Attestation, MAX_ATTESTATIONS]
2019-11-16 10:17:39 +00:00
# Entry & exit
2019-11-15 22:42:28 +00:00
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[VoluntaryExit, MAX_VOLUNTARY_EXITS]
# Custody game
2019-11-20 03:15:15 +00:00
custody_slashings: List[CustodySlashing, MAX_CUSTODY_SLASHINGS]
2019-11-16 10:13:47 +00:00
custody_key_reveals: List[CustodyKeyReveal, MAX_CUSTODY_KEY_REVEALS]
early_derived_secret_reveals: List[EarlyDerivedSecretReveal, MAX_EARLY_DERIVED_SECRET_REVEALS]
2019-11-15 22:42:28 +00:00
# Shards
shard_transitions: Vector[ShardTransition, MAX_SHARDS]
# Light clients
light_client_signature_bitfield: Bitlist[LIGHT_CLIENT_COMMITTEE_SIZE]
light_client_signature: BLSSignature
```
2019-11-15 20:11:42 +00:00
### New extended `BeaconBlock`
2019-10-12 14:59:51 +00:00
2019-11-16 10:13:47 +00:00
Note that the `body` has a new `BeaconBlockBody` definition.
2019-10-12 14:59:51 +00:00
```python
2019-11-19 19:16:40 +00:00
class BeaconBlock(Container):
2019-11-15 20:11:42 +00:00
slot: Slot
parent_root: Hash
state_root: Hash
body: BeaconBlockBody
signature: BLSSignature
```
### New extended `BeaconState`
2019-11-16 10:13:47 +00:00
Note that aside from the new additions, `Validator` and `PendingAttestation` have new definitions.
2019-11-15 20:11:42 +00:00
```python
2019-11-19 19:16:40 +00:00
class BeaconState(Container):
2019-11-15 20:11:42 +00:00
# Versioning
genesis_time: uint64
slot: Slot
fork: Fork
# History
latest_block_header: BeaconBlockHeader
block_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT]
state_roots: Vector[Hash, SLOTS_PER_HISTORICAL_ROOT]
historical_roots: List[Hash, HISTORICAL_ROOTS_LIMIT]
# Eth1
eth1_data: Eth1Data
eth1_data_votes: List[Eth1Data, SLOTS_PER_ETH1_VOTING_PERIOD]
eth1_deposit_index: uint64
# Registry
validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
# Randomness
randao_mixes: Vector[Hash, EPOCHS_PER_HISTORICAL_VECTOR]
# Slashings
slashings: Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR] # Per-epoch sums of slashed effective balances
# Attestations
previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
# Finality
justification_bits: Bitvector[JUSTIFICATION_BITS_LENGTH] # Bit set for every recent justified epoch
previous_justified_checkpoint: Checkpoint # Previous epoch snapshot
current_justified_checkpoint: Checkpoint
finalized_checkpoint: Checkpoint
# Phase 1
2019-11-16 11:23:45 +00:00
shard_states: List[ShardState, MAX_SHARDS]
2019-11-15 20:11:42 +00:00
online_countdown: Bytes[VALIDATOR_REGISTRY_LIMIT]
current_light_committee: CompactCommittee
next_light_committee: CompactCommittee
2019-11-16 11:23:45 +00:00
# Custody game
2019-11-15 20:11:42 +00:00
# Future derived secrets already exposed; contains the indices of the exposed validator
# at RANDAO reveal period % EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS
2019-11-19 19:16:40 +00:00
exposed_derived_secrets: Vector[List[ValidatorIndex, MAX_EARLY_DERIVED_SECRET_REVEALS * SLOTS_PER_EPOCH],
2019-11-15 20:11:42 +00:00
EARLY_DERIVED_SECRET_PENALTY_MAX_FUTURE_EPOCHS]
2019-10-12 14:59:51 +00:00
```
2019-11-15 20:11:42 +00:00
## Helper functions
### Crypto
#### `bls_verify_multiple`
`bls_verify_multiple` is a function for verifying a BLS signature constructed from multiple messages, as defined in the [BLS Signature spec ](../bls_signature.md#bls_verify_multiple ).
### Misc
#### `pack_compact_validator`
2019-10-13 06:52:51 +00:00
```python
def pack_compact_validator(index: int, slashed: bool, balance_in_increments: int) -> int:
"""
Creates a compact validator object representing index, slashed status, and compressed balance.
Takes as input balance-in-increments (// EFFECTIVE_BALANCE_INCREMENT) to preserve symmetry with
the unpacking function.
"""
return (index < < 16 ) + ( slashed < < 15 ) + balance_in_increments
```
2019-11-15 20:11:42 +00:00
#### `committee_to_compact_committee`
2019-10-13 06:52:51 +00:00
```python
def committee_to_compact_committee(state: BeaconState, committee: Sequence[ValidatorIndex]) -> CompactCommittee:
"""
Given a state and a list of validator indices, outputs the CompactCommittee representing them.
"""
validators = [state.validators[i] for i in committee]
compact_validators = [
pack_compact_validator(i, v.slashed, v.effective_balance // EFFECTIVE_BALANCE_INCREMENT)
for i, v in zip(committee, validators)
]
pubkeys = [v.pubkey for v in validators]
return CompactCommittee(pubkeys=pubkeys, compact_validators=compact_validators)
```
2019-11-15 20:11:42 +00:00
#### `chunks_to_body_root`
```python
def chunks_to_body_root(chunks):
2019-11-20 03:15:15 +00:00
return hash_tree_root(Vector[Hash, MAX_SHARD_BLOCK_CHUNKS](
2019-11-20 03:43:32 +00:00
chunks + [EMPTY_CHUNK_ROOT] * (MAX_SHARD_BLOCK_CHUNKS - len(chunks))
2019-11-20 03:15:15 +00:00
))
2019-11-15 20:11:42 +00:00
```
### Beacon state accessors
2019-11-16 10:13:47 +00:00
#### `get_online_validator_indices`
2019-11-15 20:11:42 +00:00
```python
2019-11-16 10:13:47 +00:00
def get_online_validator_indices(state: BeaconState) -> Set[ValidatorIndex]:
2019-11-15 20:11:42 +00:00
active_validators = get_active_validator_indices(state, get_current_epoch(state))
return set([i for i in active_validators if state.online_countdown[i] != 0])
```
#### `get_shard_committee`
2019-11-12 14:13:47 +00:00
```python
def get_shard_committee(beacon_state: BeaconState, epoch: Epoch, shard: Shard) -> Sequence[ValidatorIndex]:
source_epoch = epoch - epoch % SHARD_COMMITTEE_PERIOD
if source_epoch > 0:
source_epoch -= SHARD_COMMITTEE_PERIOD
active_validator_indices = get_active_validator_indices(beacon_state, source_epoch)
seed = get_seed(beacon_state, source_epoch, DOMAIN_SHARD_COMMITTEE)
return compute_committee(active_validator_indices, seed, 0, ACTIVE_SHARDS)
```
2019-11-15 20:11:42 +00:00
#### `get_shard_proposer_index`
2019-11-12 14:13:47 +00:00
```python
def get_shard_proposer_index(beacon_state: BeaconState, slot: Slot, shard: Shard) -> ValidatorIndex:
2019-11-18 23:40:02 +00:00
committee = get_shard_committee(beacon_state, compute_epoch_at_slot(slot), shard)
r = bytes_to_int(get_seed(beacon_state, get_current_epoch(beacon_state), DOMAIN_SHARD_COMMITTEE)[:8])
2019-11-12 14:13:47 +00:00
return committee[r % len(committee)]
```
2019-11-15 20:11:42 +00:00
#### `get_light_client_committee`
2019-10-27 01:01:10 +00:00
```python
def get_light_client_committee(beacon_state: BeaconState, epoch: Epoch) -> Sequence[ValidatorIndex]:
2019-11-01 14:58:13 +00:00
source_epoch = epoch - epoch % LIGHT_CLIENT_COMMITTEE_PERIOD
if source_epoch > 0:
source_epoch -= LIGHT_CLIENT_COMMITTEE_PERIOD
active_validator_indices = get_active_validator_indices(beacon_state, source_epoch)
2019-11-18 23:40:02 +00:00
seed = get_seed(beacon_state, source_epoch, DOMAIN_LIGHT_CLIENT)
2019-10-27 01:01:10 +00:00
return compute_committee(active_validator_indices, seed, 0, ACTIVE_SHARDS)[:TARGET_COMMITTEE_SIZE]
```
2019-11-15 20:11:42 +00:00
#### `get_indexed_attestation`
2019-10-27 01:01:10 +00:00
```python
2019-11-12 13:27:34 +00:00
def get_indexed_attestation(beacon_state: BeaconState, attestation: Attestation) -> AttestationAndCommittee:
2019-11-04 16:50:09 +00:00
committee = get_beacon_committee(beacon_state, attestation.data.slot, attestation.data.index)
2019-11-12 13:27:34 +00:00
return AttestationAndCommittee(committee, attestation)
2019-10-27 01:01:10 +00:00
```
2019-11-15 20:11:42 +00:00
#### `get_updated_gasprice`
2019-10-29 17:43:13 +00:00
```python
2019-11-06 22:19:00 +00:00
def get_updated_gasprice(prev_gasprice: Gwei, length: uint8) -> Gwei:
2019-11-20 03:15:15 +00:00
if length > TARGET_SHARD_BLOCK_SIZE:
delta = (prev_gasprice * (length - TARGET_SHARD_BLOCK_SIZE)
2019-11-20 03:43:32 +00:00
// TARGET_SHARD_BLOCK_SIZE // GASPRICE_ADJUSTMENT_COEFFICIENT)
2019-10-29 18:12:55 +00:00
return min(prev_gasprice + delta, MAX_GASPRICE)
2019-10-29 17:43:13 +00:00
else:
2019-11-20 03:15:15 +00:00
delta = (prev_gasprice * (TARGET_SHARD_BLOCK_SIZE - length)
2019-11-20 03:43:32 +00:00
// TARGET_SHARD_BLOCK_SIZE // GASPRICE_ADJUSTMENT_COEFFICIENT)
2019-11-07 13:21:36 +00:00
return max(prev_gasprice, MIN_GASPRICE + delta) - delta
2019-10-29 17:43:13 +00:00
```
2019-11-18 23:40:02 +00:00
#### `get_start_shard`
```python
def get_start_shard(state: BeaconState, slot: Slot) -> Shard:
# TODO: implement start shard logic
return Shard(0)
```
2019-11-15 20:11:42 +00:00
#### `get_shard`
```python
def get_shard(state: BeaconState, attestation: Attestation) -> Shard:
return Shard((attestation.data.index + get_start_shard(state, attestation.data.slot)) % ACTIVE_SHARDS)
```
#### `get_offset_slots`
```python
def get_offset_slots(state: BeaconState, start_slot: Slot) -> Sequence[Slot]:
return [start_slot + x for x in SHARD_BLOCK_OFFSETS if start_slot + x < state.slot ]
```
### Predicates
2019-11-16 10:13:47 +00:00
#### Updated `is_valid_indexed_attestation`
Note that this replaces the Phase 0 `is_valid_indexed_attestation` .
2019-10-27 01:01:10 +00:00
2019-10-28 09:18:27 +00:00
```python
2019-11-12 13:27:34 +00:00
def is_valid_indexed_attestation(state: BeaconState, indexed_attestation: AttestationAndCommittee) -> bool:
2019-10-27 01:01:10 +00:00
"""
Check if ``indexed_attestation`` has valid indices and signature.
"""
# Verify aggregate signature
all_pubkeys = []
all_message_hashes = []
2019-11-04 16:50:09 +00:00
aggregation_bits = indexed_attestation.attestation.aggregation_bits
assert len(aggregation_bits) == len(indexed_attestation.committee)
for i, custody_bits in enumerate(indexed_attestation.attestation.custody_bits):
assert len(custody_bits) == len(indexed_attestation.committee)
for participant, abit, cbit in zip(indexed_attestation.committee, aggregation_bits, custody_bits):
if abit:
all_pubkeys.append(state.validators[participant].pubkey)
# Note: only 2N distinct message hashes
2019-11-05 18:32:49 +00:00
all_message_hashes.append(hash_tree_root(
AttestationCustodyBitWrapper(hash_tree_root(indexed_attestation.data), i, cbit)
))
2019-11-04 16:50:09 +00:00
else:
2019-11-18 22:07:50 +00:00
assert not cbit
2019-10-27 01:01:10 +00:00
return bls_verify_multiple(
pubkeys=all_pubkeys,
message_hashes=all_message_hashes,
signature=indexed_attestation.signature,
domain=get_domain(state, DOMAIN_BEACON_ATTESTER, indexed_attestation.data.target.epoch),
)
```
2019-11-03 16:17:46 +00:00
2019-11-15 20:11:42 +00:00
### Block processing
2019-11-06 22:19:00 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
process_randao(state, block.body)
process_eth1_data(state, block.body)
verify_shard_transition_false_positives(state, block)
process_light_client_signatures(state, block)
process_operations(state, block.body)
2019-11-12 13:27:34 +00:00
```
2019-10-12 03:05:08 +00:00
2019-11-15 20:11:42 +00:00
#### Operations
2019-10-12 03:05:08 +00:00
2019-10-13 08:42:55 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
# Verify that outstanding deposits are processed up to the maximum number of deposits
assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)
2019-11-20 03:43:32 +00:00
def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
2019-11-15 20:11:42 +00:00
for operation in operations:
fn(state, operation)
2019-11-16 10:13:47 +00:00
for_ops(body.proposer_slashings, process_proposer_slashing)
for_ops(body.attester_slashings, process_attester_slashing)
2019-10-13 06:52:51 +00:00
2019-11-15 20:11:42 +00:00
# New attestation processing
2019-11-15 22:42:28 +00:00
process_attestations(state, body, body.attestations)
2019-10-13 06:52:51 +00:00
2019-11-16 10:13:47 +00:00
for_ops(body.deposits, process_deposit)
for_ops(body.voluntary_exits, process_voluntary_exit)
2019-11-15 20:11:42 +00:00
2019-11-15 22:42:28 +00:00
# See custody game spec.
process_custody_game_operations(state, body)
2019-11-15 20:11:42 +00:00
# TODO process_operations(body.shard_receipt_proofs, process_shard_receipt_proofs)
2019-10-12 03:05:08 +00:00
```
2019-11-15 20:11:42 +00:00
##### New Attestation processing
2019-11-03 16:17:46 +00:00
2019-11-15 20:11:42 +00:00
###### `validate_attestation`
2019-10-12 03:05:08 +00:00
```python
2019-11-05 18:33:26 +00:00
def validate_attestation(state: BeaconState, attestation: Attestation) -> None:
2019-10-12 03:05:08 +00:00
data = attestation.data
2019-10-12 14:59:51 +00:00
assert data.index < ACTIVE_SHARDS
2019-11-03 16:17:46 +00:00
shard = get_shard(state, attestation)
2019-10-12 03:05:08 +00:00
# Signature check
assert is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))
# Type 1: on-time attestations
2019-11-12 13:27:34 +00:00
if attestation.custody_bits != []:
2019-10-29 17:43:13 +00:00
# Correct slot
assert data.slot == state.slot
2019-10-12 03:05:08 +00:00
# Correct data root count
2019-11-06 22:19:00 +00:00
assert len(attestation.custody_bits) == len(get_offset_slots(state, state.shard_next_slots[shard]))
2019-10-12 03:05:08 +00:00
# Correct parent block root
assert data.beacon_block_root == get_block_root_at_slot(state, state.slot - 1)
# Type 2: delayed attestations
else:
2019-11-18 23:40:02 +00:00
assert state.slot - compute_start_slot_at_epoch(compute_epoch_at_slot(data.slot)) < SLOTS_PER_EPOCH
2019-11-01 14:58:13 +00:00
assert data.shard_transition_root == Hash()
2019-11-03 16:17:46 +00:00
```
2019-10-12 03:05:08 +00:00
2019-11-15 20:11:42 +00:00
###### `apply_shard_transition`
2019-11-03 16:17:46 +00:00
```python
def apply_shard_transition(state: BeaconState, shard: Shard, transition: ShardTransition) -> None:
# Slot the attestation starts counting from
start_slot = state.shard_next_slots[shard]
2019-11-03 20:06:19 +00:00
2019-11-03 16:17:46 +00:00
# Correct data root count
2019-11-06 22:19:00 +00:00
offset_slots = get_offset_slots(state, start_slot)
2019-11-18 22:07:50 +00:00
assert (
len(transition.shard_data_roots)
== len(transition.shard_states)
== len(transition.shard_block_lengths)
== len(offset_slots)
)
2019-11-03 16:17:46 +00:00
assert transition.start_slot == start_slot
2019-11-03 20:06:19 +00:00
# Reonstruct shard headers
headers = []
proposers = []
2019-11-12 13:27:34 +00:00
shard_parent_root = state.shard_states[shard].latest_block_root
2019-11-03 20:06:19 +00:00
for i in range(len(offset_slots)):
if any(transition.shard_data_roots):
2019-11-06 22:19:00 +00:00
headers.append(ShardSignableHeader(
2019-11-05 18:33:41 +00:00
shard_parent_root=shard_parent_root,
2019-11-18 22:07:50 +00:00
parent_hash=get_block_root_at_slot(state, state.slot - 1),
2019-11-03 20:06:19 +00:00
slot=offset_slots[i],
body_root=chunks_to_body_root(transition.shard_data_roots[i])
))
2019-11-12 14:13:47 +00:00
proposers.append(get_shard_proposer_index(state, shard, offset_slots[i]))
2019-11-03 20:06:19 +00:00
shard_parent_root = hash_tree_root(headers[-1])
2019-11-03 16:17:46 +00:00
# Verify correct calculation of gas prices and slots and chunk roots
prev_gasprice = state.shard_states[shard].gasprice
for i in range(len(offset_slots)):
2019-11-18 22:07:50 +00:00
shard_state = transition.shard_states[i]
block_length = transition.shard_block_lengths[i]
chunks = transition.shard_data_roots[i]
2019-11-18 23:40:02 +00:00
assert shard_state.gasprice == get_updated_gasprice(prev_gasprice, block_length)
2019-11-03 16:17:46 +00:00
assert shard_state.slot == offset_slots[i]
assert len(chunks) == block_length // SHARD_BLOCK_CHUNK_SIZE
prev_gasprice = shard_state.gasprice
2019-11-04 16:50:09 +00:00
# Verify combined proposer signature
2019-11-03 20:06:19 +00:00
assert bls_verify_multiple(
pubkeys=[state.validators[proposer].pubkey for proposer in proposers],
message_hashes=[hash_tree_root(header) for header in headers],
2019-11-06 20:53:17 +00:00
signature=transition.proposer_signature_aggregate,
2019-11-03 20:06:19 +00:00
domain=DOMAIN_SHARD_PROPOSAL
)
2019-11-03 16:17:46 +00:00
# Save updated state
state.shard_states[shard] = transition.shard_states[-1]
state.shard_states[shard].slot = state.slot - 1
```
2019-10-12 14:59:51 +00:00
2019-11-15 20:11:42 +00:00
###### `process_attestations`
2019-10-12 03:05:08 +00:00
2019-11-03 16:17:46 +00:00
```python
2019-11-15 22:42:28 +00:00
def process_attestations(state: BeaconState, block_body: BeaconBlockBody, attestations: Sequence[Attestation]) -> None:
2019-11-03 16:17:46 +00:00
# Basic validation
for attestation in attestations:
2019-11-18 22:07:50 +00:00
validate_attestation(state, attestation)
2019-11-03 16:17:46 +00:00
# Process crosslinks
2019-11-16 10:13:47 +00:00
online_indices = get_online_validator_indices(state)
2019-11-03 16:17:46 +00:00
winners = set()
for shard in range(ACTIVE_SHARDS):
2019-11-03 23:49:50 +00:00
success = False
2019-11-03 16:17:46 +00:00
# All attestations in the block for this shard
2019-11-18 22:07:50 +00:00
this_shard_attestations = [
attestation for attestation in attestations
if get_shard(state, attestation) == shard and attestation.data.slot == state.slot
]
2019-11-03 16:17:46 +00:00
# The committee for this shard
this_shard_committee = get_beacon_committee(state, get_current_epoch(state), shard)
# Loop over all shard transition roots
2019-11-18 22:07:50 +00:00
shard_transition_roots = set([a.data.shard_transition_root for a in this_shard_attestations])
for shard_transition_root in sorted(shard_transition_roots):
2019-11-03 16:17:46 +00:00
all_participants = set()
participating_attestations = []
for attestation in this_shard_attestations:
participating_attestations.append(attestation)
if attestation.data.shard_transition_root == shard_transition_root:
2019-11-18 22:07:50 +00:00
participants = get_attesting_indices(state, attestation.data, attestation.aggregation_bits)
all_participants = all_participants.union(participants)
2019-11-15 22:42:28 +00:00
if (
get_total_balance(state, online_indices.intersection(all_participants)) * 3 >=
get_total_balance(state, online_indices.intersection(this_shard_committee)) * 2
and success is False
):
# Attestation < - > shard transition consistency
assert shard_transition_root == hash_tree_root(block_body.shard_transition)
2019-11-18 22:07:50 +00:00
assert (
attestation.data.head_shard_root
== chunks_to_body_root(block_body.shard_transition.shard_data_roots[-1])
)
2019-11-15 22:42:28 +00:00
# Apply transition
apply_shard_transition(state, shard, block_body.shard_transition)
# Apply proposer reward and cost
2019-11-18 23:40:02 +00:00
beacon_proposer_index = get_beacon_proposer_index(state)
2019-11-15 22:42:28 +00:00
estimated_attester_reward = sum([get_base_reward(state, attester) for attester in all_participants])
2019-11-19 19:16:40 +00:00
proposer_reward = estimated_attester_reward // PROPOSER_REWARD_QUOTIENT
2019-11-18 23:40:02 +00:00
increase_balance(state, beacon_proposer_index, proposer_reward)
2019-11-18 22:07:50 +00:00
states_slots_lengths = zip(
block_body.shard_transition.shard_states,
2019-11-18 23:40:02 +00:00
get_offset_slots(state, state.shard_next_slots[get_shard(attestation)]),
2019-11-18 22:07:50 +00:00
block_body.shard_transition.shard_block_lengths
)
for shard_state, slot, length in states_slots_lengths:
proposer_index = get_shard_proposer_index(state, shard, slot)
decrease_balance(state, proposer_index, shard_state.gasprice * length)
2019-11-15 22:42:28 +00:00
winners.add((shard, shard_transition_root))
success = True
2019-11-03 23:49:50 +00:00
if not success:
2019-11-15 22:42:28 +00:00
assert block_body.shard_transitions[shard] == ShardTransition()
2019-11-03 16:17:46 +00:00
for attestation in attestations:
2019-11-18 22:07:50 +00:00
is_winning_transition = (get_shard(state, attestation), attestation.shard_transition_root) in winners
2019-11-03 16:17:46 +00:00
pending_attestation = PendingAttestation(
aggregation_bits=attestation.aggregation_bits,
data=attestation.data,
2019-11-18 23:40:02 +00:00
inclusion_delay=state.slot - attestation.data.slot,
2019-11-18 22:07:50 +00:00
crosslink_success=is_winning_transition and attestation.data.slot == state.slot,
2019-11-03 16:17:46 +00:00
proposer_index=proposer_index
)
if attestation.data.target.epoch == get_current_epoch(state):
assert attestation.data.source == state.current_justified_checkpoint
state.current_epoch_attestations.append(pending_attestation)
else:
assert attestation.data.source == state.previous_justified_checkpoint
state.previous_epoch_attestations.append(pending_attestation)
2019-10-12 03:05:08 +00:00
```
2019-11-15 20:11:42 +00:00
#### Shard transition false positives
2019-10-29 17:43:13 +00:00
```python
2019-11-15 22:42:28 +00:00
def verify_shard_transition_false_positives(state: BeaconState, block_body: BeaconBlockBody) -> None:
2019-10-29 17:43:13 +00:00
# Verify that a `shard_transition` in a block is empty if an attestation was not processed for it
2019-11-16 11:23:45 +00:00
for shard in range(ACTIVE_SHARDS):
2019-10-29 17:43:13 +00:00
if state.shard_states[shard].slot != state.slot - 1:
2019-11-15 22:42:28 +00:00
assert block_body.shard_transition[shard] == ShardTransition()
2019-10-29 17:43:13 +00:00
```
2019-10-13 08:13:52 +00:00
2019-11-15 20:11:42 +00:00
#### Light client processing
2019-10-13 06:52:51 +00:00
```python
2019-11-15 22:42:28 +00:00
def process_light_client_signatures(state: BeaconState, block_body: BeaconBlockBody) -> None:
2019-11-01 14:58:13 +00:00
committee = get_light_client_committee(state, get_current_epoch(state))
2019-11-15 22:42:28 +00:00
assert len(block_body.light_client_signature_bitfield) == len(committee)
2019-11-05 20:16:56 +00:00
total_reward = Gwei(0)
2019-10-27 01:01:10 +00:00
signer_keys = []
2019-11-19 19:16:40 +00:00
for i, participant_bit in enumerate(block_body.light_client_signature_bitfield):
if participant_bit:
2019-10-27 01:01:10 +00:00
signer_keys.append(state.validators[committee[i]].pubkey)
2019-10-29 18:33:29 +00:00
increase_balance(state, committee[i], get_base_reward(state, committee[i]))
2019-11-05 20:17:07 +00:00
total_reward += get_base_reward(state, committee[i])
2019-11-01 14:58:13 +00:00
2019-11-19 19:16:40 +00:00
increase_balance(state, get_beacon_proposer_index(state), total_reward // PROPOSER_REWARD_QUOTIENT)
2019-10-27 01:01:10 +00:00
assert bls_verify(
pubkey=bls_aggregate_pubkeys(signer_keys),
message_hash=get_block_root_at_slot(state, state.slot - 1),
2019-11-15 22:42:28 +00:00
signature=block_body.light_client_signature,
2019-11-19 19:16:40 +00:00
domain=DOMAIN_LIGHT_CLIENT
2019-10-27 01:01:10 +00:00
)
2019-10-13 06:52:51 +00:00
```
2019-11-15 20:11:42 +00:00
2019-10-12 14:59:51 +00:00
### Epoch transition
2019-11-15 20:11:42 +00:00
This epoch transition overrides the phase0 epoch transition:
2019-10-12 14:59:51 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_epoch(state: BeaconState) -> None:
process_justification_and_finalization(state)
process_rewards_and_penalties(state)
process_registry_updates(state)
2019-11-15 22:42:28 +00:00
process_reveal_deadlines(state)
2019-11-15 20:11:42 +00:00
process_slashings(state)
process_final_updates(state)
2019-11-15 22:42:28 +00:00
process_custody_final_updates(state)
process_online_tracking(state)
process_light_client_committee_updates(state)
```
2019-11-15 20:11:42 +00:00
2019-11-15 22:52:57 +00:00
#### Custody game updates
`process_reveal_deadlines` and `process_custody_final_updates` are defined in [the Custody Game spec ](./1_custody-game.md ),
2019-11-15 22:42:28 +00:00
#### Online-tracking
```python
def process_online_tracking(state: BeaconState) -> None:
2019-10-27 01:01:10 +00:00
# Slowly remove validators from the "online" set if they do not show up
for index in range(len(state.validators)):
if state.online_countdown[index] != 0:
state.online_countdown[index] = state.online_countdown[index] - 1
2019-11-06 22:19:00 +00:00
# Process pending attestations
for pending_attestation in state.current_epoch_attestations + state.previous_epoch_attestations:
for index in get_attesting_indices(state, pending_attestation.data, pending_attestation.aggregation_bits):
state.online_countdown[index] = ONLINE_PERIOD
2019-10-12 14:59:51 +00:00
```
2019-11-15 22:42:28 +00:00
#### Light client committee updates
```python
def process_light_client_committee_updates(state: BeaconState) -> None:
# Update light client committees
if get_current_epoch(state) % LIGHT_CLIENT_COMMITTEE_PERIOD == 0:
state.current_light_committee = state.next_light_committee
new_committee = get_light_client_committee(state, get_current_epoch(state) + LIGHT_CLIENT_COMMITTEE_PERIOD)
state.next_light_committee = committee_to_compact_committee(state, new_committee)
```