eth2.0-specs/specs/core/1_shard-data-chains.md

426 lines
15 KiB
Markdown
Raw Normal View History

# Ethereum 2.0 Phase 1 -- Shard Data Chains
2019-05-06 15:30:32 +00:00
**Notice**: This document is a work-in-progress for researchers and implementers.
2019-05-06 15:30:32 +00:00
## Table of contents
<!-- TOC -->
2019-05-06 15:30:32 +00:00
- [Ethereum 2.0 Phase 1 -- Shard Data Chains](#ethereum-20-phase-1----shard-data-chains)
- [Table of contents](#table-of-contents)
- [Introduction](#introduction)
2019-06-17 21:19:44 +00:00
- [Configuration](#configuration)
- [Misc](#misc)
2019-06-17 21:19:44 +00:00
- [Initial values](#initial-values)
- [Time parameters](#time-parameters)
- [Signature domains](#signature-domains)
2019-06-30 20:57:42 +00:00
- [TODO PLACEHOLDER](#todo-placeholder)
- [Data structures](#data-structures)
- [`ShardBlockBody`](#shardblockbody)
2019-05-07 12:23:28 +00:00
- [`ShardAttestation`](#shardattestation)
- [`ShardBlock`](#shardblock)
- [`ShardBlockHeader`](#shardblockheader)
- [Helper functions](#helper-functions)
- [`get_period_committee`](#get_period_committee)
- [`get_switchover_epoch`](#get_switchover_epoch)
- [`get_persistent_committee`](#get_persistent_committee)
- [`get_shard_proposer_index`](#get_shard_proposer_index)
- [`get_shard_header`](#get_shard_header)
- [`verify_shard_attestation_signature`](#verify_shard_attestation_signature)
- [`compute_crosslink_data_root`](#compute_crosslink_data_root)
- [Object validity](#object-validity)
- [Shard blocks](#shard-blocks)
- [Shard attestations](#shard-attestations)
- [Beacon attestations](#beacon-attestations)
- [Shard fork choice rule](#shard-fork-choice-rule)
<!-- /TOC -->
## Introduction
This document describes the shard data layer and the shard fork choice rule in Phase 1 of Ethereum 2.0.
2019-06-17 21:19:44 +00:00
## Configuration
### Misc
| Name | Value |
| - | - |
| `BYTES_PER_SHARD_BLOCK_BODY` | `2**14` (= 16,384) |
| `MAX_SHARD_ATTESTIONS` | `2**4` (= 16) |
2019-06-17 21:19:44 +00:00
### Initial values
| Name | Value |
2019-06-06 09:39:22 +00:00
| `PHASE_1_FORK_EPOCH` | **TBD** |
| `PHASE_1_FORK_SLOT` | **TBD** |
| `GENESIS_SHARD_SLOT` | 0 |
### Time parameters
| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `CROSSLINK_LOOKBACK` | `2**0` (= 1) | epochs | 6.2 minutes |
| `PERSISTENT_COMMITTEE_PERIOD` | `2**11` (= 2,048) | epochs | ~9 days |
2019-06-30 20:12:02 +00:00
### Signature domain types
The following types are defined, mapping into `DomainType` (little endian):
| Name | Value |
| - | - |
| `DOMAIN_SHARD_PROPOSER` | `128` |
| `DOMAIN_SHARD_ATTESTER` | `129` |
2019-06-19 00:14:13 +00:00
### TODO PLACEHOLDER
| Name | Value |
| - | - |
| `PLACEHOLDER` | `2**32` |
## Data structures
### `ShardBlockBody`
```python
2019-06-05 13:29:26 +00:00
class ShardBlockBody(Container):
2019-06-19 00:14:13 +00:00
data: Vector[Bytes[PLACEHOLDER], BYTES_PER_SHARD_BLOCK_BODY]
2019-05-07 12:23:28 +00:00
```
### `ShardAttestation`
```python
2019-06-05 13:29:26 +00:00
class ShardAttestation(Container):
class data(Container):
slot: Slot
shard: Shard
2019-06-05 13:29:26 +00:00
shard_block_root: Bytes32
2019-06-28 11:23:22 +00:00
aggregation_bits: Bitlist[PLACEHOLDER]
2019-06-15 22:25:37 +00:00
aggregate_signature: BLSSignature
```
### `ShardBlock`
```python
2019-06-05 13:29:26 +00:00
class ShardBlock(Container):
slot: Slot
shard: Shard
2019-06-05 13:29:26 +00:00
beacon_chain_root: Bytes32
parent_root: Bytes32
data: ShardBlockBody
state_root: Bytes32
2019-06-19 00:14:13 +00:00
attestations: List[ShardAttestation, PLACEHOLDER]
2019-06-15 22:25:37 +00:00
signature: BLSSignature
```
### `ShardBlockHeader`
```python
2019-06-05 13:29:26 +00:00
class ShardBlockHeader(Container):
slot: Slot
shard: Shard
2019-06-05 13:29:26 +00:00
beacon_chain_root: Bytes32
parent_root: Bytes32
body_root: Bytes32
state_root: Bytes32
2019-06-19 00:14:13 +00:00
attestations: List[ShardAttestation, PLACEHOLDER]
2019-06-15 22:25:37 +00:00
signature: BLSSignature
```
2019-02-08 09:54:02 +00:00
## Helper functions
### `get_period_committee`
```python
def get_period_committee(state: BeaconState,
epoch: Epoch,
shard: Shard,
index: uint64,
count: uint64) -> Sequence[ValidatorIndex]:
"""
Return committee for a period. Used to construct persistent committees.
"""
2019-05-01 14:21:38 +00:00
return compute_committee(
indices=get_active_validator_indices(state, epoch),
2019-06-30 09:02:18 +00:00
seed=get_seed(state, epoch),
2019-05-01 14:21:38 +00:00
index=shard * count + index,
count=SHARD_COUNT * count,
)
```
### `get_switchover_epoch`
```python
def get_switchover_epoch(state: BeaconState, epoch: Epoch, index: ValidatorIndex) -> int:
earlier_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD * 2)
2019-06-30 09:02:18 +00:00
return (bytes_to_int(hash(get_seed(state, earlier_start_epoch) + int_to_bytes(index, length=3)[0:8]))
% PERSISTENT_COMMITTEE_PERIOD)
```
### `get_persistent_committee`
2019-02-08 09:54:02 +00:00
```python
def get_persistent_committee(state: BeaconState,
shard: Shard,
2019-06-22 16:12:42 +00:00
slot: Slot) -> Sequence[ValidatorIndex]:
2019-02-09 04:10:54 +00:00
"""
Return the persistent committee for the given ``shard`` at the given ``slot``.
2019-02-09 04:10:54 +00:00
"""
epoch = compute_epoch_of_slot(slot)
earlier_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD * 2)
later_start_epoch = Epoch(epoch - (epoch % PERSISTENT_COMMITTEE_PERIOD) - PERSISTENT_COMMITTEE_PERIOD)
committee_count = max(
len(get_active_validator_indices(state, earlier_start_epoch)) //
(SHARD_COUNT * TARGET_COMMITTEE_SIZE),
len(get_active_validator_indices(state, later_start_epoch)) //
(SHARD_COUNT * TARGET_COMMITTEE_SIZE),
) + 1
2019-05-01 08:09:24 +00:00
index = slot % committee_count
earlier_committee = get_period_committee(state, earlier_start_epoch, shard, index, committee_count)
later_committee = get_period_committee(state, later_start_epoch, shard, index, committee_count)
2019-02-10 06:09:34 +00:00
# Take not-yet-cycled-out validators from earlier committee and already-cycled-in validators from
# later committee; return a sorted list of the union of the two, deduplicated
2019-06-22 22:17:54 +00:00
return sorted(list(set(
2019-06-30 20:59:12 +00:00
[i for i in earlier_committee if epoch % PERSISTENT_COMMITTEE_PERIOD < get_switchover_epoch(state, epoch, i)]
+ [i for i in later_committee if epoch % PERSISTENT_COMMITTEE_PERIOD >= get_switchover_epoch(state, epoch, i)]
2019-06-22 22:17:54 +00:00
)))
2019-02-08 09:54:02 +00:00
```
### `get_shard_proposer_index`
2019-02-08 09:54:02 +00:00
```python
def get_shard_proposer_index(state: BeaconState,
shard: Shard,
slot: Slot) -> Optional[ValidatorIndex]:
# Randomly shift persistent committee
2019-06-22 16:12:42 +00:00
persistent_committee = list(get_persistent_committee(state, shard, slot))
2019-05-07 09:57:41 +00:00
seed = hash(state.current_shuffling_seed + int_to_bytes(shard, length=8) + int_to_bytes(slot, length=8))
random_index = bytes_to_int(seed[0:8]) % len(persistent_committee)
persistent_committee = persistent_committee[random_index:] + persistent_committee[:random_index]
# Search for an active proposer
for index in persistent_committee:
2019-06-09 19:41:21 +00:00
if is_active_validator(state.validators[index], get_current_epoch(state)):
return index
# No block can be proposed if no validator is active
return None
```
### `get_shard_header`
```python
def get_shard_header(block: ShardBlock) -> ShardBlockHeader:
return ShardBlockHeader(
slot=block.slot,
shard=block.shard,
beacon_chain_root=block.beacon_chain_root,
parent_root=block.parent_root,
body_root=hash_tree_root(block.body),
state_root=block.state_root,
attestations=block.attestations,
signature=block.signature,
)
```
### `verify_shard_attestation_signature`
```python
def verify_shard_attestation_signature(state: BeaconState,
attestation: ShardAttestation) -> None:
data = attestation.data
2019-05-24 17:59:22 +00:00
persistent_committee = get_persistent_committee(state, data.shard, data.slot)
pubkeys = []
for i, index in enumerate(persistent_committee):
2019-06-28 11:23:22 +00:00
if attestation.aggregation_bits[i]:
2019-06-09 19:41:21 +00:00
validator = state.validators[index]
assert is_active_validator(validator, get_current_epoch(state))
pubkeys.append(validator.pubkey)
assert bls_verify(
pubkey=bls_aggregate_pubkeys(pubkeys),
2019-05-24 17:59:22 +00:00
message_hash=data.shard_block_root,
signature=attestation.aggregate_signature,
domain=get_domain(state, DOMAIN_SHARD_ATTESTER, compute_epoch_of_slot(data.slot))
)
```
### `compute_crosslink_data_root`
```python
2019-06-22 16:12:42 +00:00
def compute_crosslink_data_root(blocks: Sequence[ShardBlock]) -> Bytes32:
def is_power_of_two(value: uint64) -> bool:
return (value > 0) and (value & (value - 1) == 0)
def pad_to_power_of_2(values: MutableSequence[bytes]) -> Sequence[bytes]:
while not is_power_of_two(len(values)):
values.append(b'\x00' * BYTES_PER_SHARD_BLOCK_BODY)
return values
def hash_tree_root_of_bytes(data: bytes) -> bytes:
return hash_tree_root([data[i:i + 32] for i in range(0, len(data), 32)])
def zpad(data: bytes, length: uint64) -> bytes:
2019-05-28 07:58:51 +00:00
return data + b'\x00' * (length - len(data))
return hash(
# TODO untested code.
# Need to either pass a typed list to hash-tree-root, or merkleize_chunks(values, pad_to=2**x)
hash_tree_root(pad_to_power_of_2([
2019-05-28 07:58:51 +00:00
hash_tree_root_of_bytes(
zpad(serialize(get_shard_header(block)), BYTES_PER_SHARD_BLOCK_BODY)
) for block in blocks
]))
+ hash_tree_root(pad_to_power_of_2([
2019-06-05 13:29:26 +00:00
hash_tree_root_of_bytes(block.body) for block in blocks
]))
)
```
## Object validity
### Shard blocks
Let:
2019-06-30 19:25:58 +00:00
- `beacon_blocks` be the `BeaconBlock` list such that `beacon_blocks[slot]` is the canonical `BeaconBlock` at slot `slot`
- `beacon_state` be the canonical `BeaconState` after processing `beacon_blocks[-1]`
- `valid_shard_blocks` be the list of valid `ShardBlock`, recursively defined
- `candidate` be a candidate `ShardBlock` for which validity is to be determined by running `is_valid_shard_block`
```python
2019-06-22 16:12:42 +00:00
def is_valid_shard_block(beacon_blocks: Sequence[BeaconBlock],
beacon_state: BeaconState,
2019-06-22 16:12:42 +00:00
valid_shard_blocks: Sequence[ShardBlock],
candidate: ShardBlock) -> bool:
# Check if block is already determined valid
for _, block in enumerate(valid_shard_blocks):
if candidate == block:
return True
# Check slot number
2019-06-06 09:39:22 +00:00
assert candidate.slot >= PHASE_1_FORK_SLOT
# Check shard number
2019-05-24 17:59:22 +00:00
assert candidate.shard <= SHARD_COUNT
# Check beacon block
2019-05-24 17:59:22 +00:00
beacon_block = beacon_blocks[candidate.slot]
assert candidate.beacon_block_root == signing_root(beacon_block)
2019-05-26 12:14:48 +00:00
assert beacon_block.slot <= candidate.slot
# Check state root
2019-06-30 15:10:22 +00:00
assert candidate.state_root == Hash() # [to be removed in phase 2]
# Check parent block
2019-06-06 09:39:22 +00:00
if candidate.slot == PHASE_1_FORK_SLOT:
2019-06-30 15:10:22 +00:00
assert candidate.parent_root == Hash()
else:
parent_block = next(
(block for block in valid_shard_blocks if signing_root(block) == candidate.parent_root),
None
)
assert parent_block is not None
2019-05-24 17:59:22 +00:00
assert parent_block.shard == candidate.shard
assert parent_block.slot < candidate.slot
2019-04-08 01:51:13 +00:00
assert signing_root(beacon_blocks[parent_block.slot]) == parent_block.beacon_chain_root
# Check attestations
2019-05-24 17:59:22 +00:00
assert len(candidate.attestations) <= MAX_SHARD_ATTESTIONS
for _, attestation in enumerate(candidate.attestations):
assert max(GENESIS_SHARD_SLOT, candidate.slot - SLOTS_PER_EPOCH) <= attestation.data.slot
assert attestation.data.slot <= candidate.slot - MIN_ATTESTATION_INCLUSION_DELAY
assert attestation.data.crosslink.shard == candidate.shard
verify_shard_attestation_signature(beacon_state, attestation)
# Check signature
2019-05-24 17:59:22 +00:00
proposer_index = get_shard_proposer_index(beacon_state, candidate.shard, candidate.slot)
assert proposer_index is not None
assert bls_verify(
2019-06-09 19:41:21 +00:00
pubkey=beacon_state.validators[proposer_index].pubkey,
2019-06-22 16:12:42 +00:00
message_hash=signing_root(candidate),
2019-05-24 17:59:22 +00:00
signature=candidate.signature,
domain=get_domain(beacon_state, DOMAIN_SHARD_PROPOSER, compute_epoch_of_slot(candidate.slot)),
)
return True
```
### Shard attestations
Let:
2019-06-30 19:25:58 +00:00
- `valid_shard_blocks` be the list of valid `ShardBlock`
- `beacon_state` be the canonical `BeaconState`
- `candidate` be a candidate `ShardAttestation` for which validity is to be determined by running `is_valid_shard_attestation`
```python
2019-06-22 16:12:42 +00:00
def is_valid_shard_attestation(valid_shard_blocks: Sequence[ShardBlock],
beacon_state: BeaconState,
2019-05-24 17:59:22 +00:00
candidate: ShardAttestation) -> bool:
# Check shard block
shard_block = next(
2019-05-26 12:14:48 +00:00
(block for block in valid_shard_blocks if signing_root(block) == candidate.data.shard_block_root),
None,
)
assert shard_block is not None
2019-05-24 17:59:22 +00:00
assert shard_block.slot == candidate.data.slot
assert shard_block.shard == candidate.data.shard
# Check signature
2019-05-24 17:59:22 +00:00
verify_shard_attestation_signature(beacon_state, candidate)
return True
```
### Beacon attestations
Let:
2019-06-30 19:25:58 +00:00
- `shard` be a valid `Shard`
- `shard_blocks` be the `ShardBlock` list such that `shard_blocks[slot]` is the canonical `ShardBlock` for shard `shard` at slot `slot`
- `beacon_state` be the canonical `BeaconState`
- `valid_attestations` be the set of valid `Attestation` objects, recursively defined
- `candidate` be a candidate `Attestation` which is valid under Phase 0 rules, and for which validity is to be determined under Phase 1 rules by running `is_valid_beacon_attestation`
```python
def is_valid_beacon_attestation(shard: Shard,
2019-06-22 16:12:42 +00:00
shard_blocks: Sequence[ShardBlock],
beacon_state: BeaconState,
valid_attestations: Set[Attestation],
candidate: Attestation) -> bool:
# Check if attestation is already determined valid
for attestation in valid_attestations:
if candidate == attestation:
return True
# Check previous attestation
2019-06-06 09:39:22 +00:00
if candidate.data.previous_crosslink.epoch <= PHASE_1_FORK_EPOCH:
2019-06-30 15:10:22 +00:00
assert candidate.data.previous_crosslink.data_root == Hash()
else:
previous_attestation = next(
2019-06-30 20:59:12 +00:00
(attestation for attestation in valid_attestations
2019-06-30 21:14:16 +00:00
if attestation.data.crosslink.data_root == candidate.data.previous_crosslink.data_root),
None,
)
assert previous_attestation is not None
assert candidate.data.previous_attestation.epoch < compute_epoch_of_slot(candidate.data.slot)
# Check crosslink data root
2019-06-09 19:41:21 +00:00
start_epoch = beacon_state.crosslinks[shard].epoch
end_epoch = min(compute_epoch_of_slot(candidate.data.slot) - CROSSLINK_LOOKBACK,
2019-06-30 18:58:02 +00:00
start_epoch + MAX_EPOCHS_PER_CROSSLINK)
blocks = []
for slot in range(start_epoch * SLOTS_PER_EPOCH, end_epoch * SLOTS_PER_EPOCH):
blocks.append(shard_blocks[slot])
2019-05-06 17:26:14 +00:00
assert candidate.data.crosslink.data_root == compute_crosslink_data_root(blocks)
return True
```
## Shard fork choice rule
2019-04-16 17:03:22 +00:00
The fork choice rule for any shard is LMD GHOST using the shard attestations of the persistent committee and the beacon chain attestations of the crosslink committee currently assigned to that shard, but instead of being rooted in the genesis it is rooted in the block referenced in the most recent accepted crosslink (i.e. `state.crosslinks[shard].shard_block_root`). Only blocks whose `beacon_chain_root` is the block in the main beacon chain at the specified `slot` should be considered. (If the beacon chain skips a slot, then the block at that slot is considered to be the block in the beacon chain at the highest slot lower than that slot.)