2021-03-17 23:20:53 +00:00
# Ethereum 2.0 Sharding -- Beacon Chain changes
2019-10-12 03:05:08 +00:00
**Notice**: This document is a work-in-progress for researchers and implementers.
## Table of contents
2021-03-17 23:07:15 +00:00
<!-- TOC -->
2020-01-24 23:43:43 +00:00
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE - RUN doctoc TO UPDATE -->
2020-09-23 20:55:22 +00:00
2020-01-24 23:43:43 +00:00
- [Introduction ](#introduction )
- [Custom types ](#custom-types )
2021-03-26 23:47:10 +00:00
- [Constants ](#constants )
2020-12-11 08:10:50 +00:00
- [Misc ](#misc )
2021-05-06 16:28:46 +00:00
- [Domain types ](#domain-types )
- [Preset ](#preset )
- [Misc ](#misc-1 )
- [Shard block samples ](#shard-block-samples )
2020-12-11 08:10:50 +00:00
- [Precomputed size verification points ](#precomputed-size-verification-points )
- [Gwei values ](#gwei-values )
2021-05-06 16:28:46 +00:00
- [Configuration ](#configuration )
2020-01-24 23:43:43 +00:00
- [Updated containers ](#updated-containers )
2020-12-11 08:10:50 +00:00
- [`AttestationData` ](#attestationdata )
2021-03-18 00:26:03 +00:00
- [`BeaconBlockBody` ](#beaconblockbody )
2020-12-11 08:10:50 +00:00
- [`BeaconState` ](#beaconstate )
2020-01-24 23:43:43 +00:00
- [New containers ](#new-containers )
2020-12-11 08:10:50 +00:00
- [`DataCommitment` ](#datacommitment )
2021-04-04 00:45:57 +00:00
- [`ShardBlobBodySummary` ](#shardblobbodysummary )
- [`ShardBlobHeader` ](#shardblobheader )
- [`SignedShardBlobHeader` ](#signedshardblobheader )
2020-12-11 08:10:50 +00:00
- [`PendingShardHeader` ](#pendingshardheader )
2021-04-04 00:45:57 +00:00
- [`ShardBlobReference` ](#shardblobreference )
- [`SignedShardBlobReference` ](#signedshardblobreference )
- [`ShardProposerSlashing` ](#shardproposerslashing )
2020-01-24 23:43:43 +00:00
- [Helper functions ](#helper-functions )
2021-05-06 16:28:46 +00:00
- [Misc ](#misc-2 )
2021-01-04 18:17:30 +00:00
- [`next_power_of_two` ](#next_power_of_two )
2020-12-11 08:10:50 +00:00
- [`compute_previous_slot` ](#compute_previous_slot )
- [`compute_updated_gasprice` ](#compute_updated_gasprice )
- [`compute_committee_source_epoch` ](#compute_committee_source_epoch )
- [Beacon state accessors ](#beacon-state-accessors )
- [Updated `get_committee_count_per_slot` ](#updated-get_committee_count_per_slot )
- [`get_active_shard_count` ](#get_active_shard_count )
- [`get_shard_committee` ](#get_shard_committee )
2021-01-04 18:17:30 +00:00
- [`compute_proposer_index` ](#compute_proposer_index )
2020-12-11 08:10:50 +00:00
- [`get_shard_proposer_index` ](#get_shard_proposer_index )
- [`get_start_shard` ](#get_start_shard )
2021-01-04 18:17:30 +00:00
- [`compute_shard_from_committee_index` ](#compute_shard_from_committee_index )
- [`compute_committee_index_from_shard` ](#compute_committee_index_from_shard )
2020-12-11 08:10:50 +00:00
- [Block processing ](#block-processing )
- [Operations ](#operations )
- [New Attestation processing ](#new-attestation-processing )
- [Updated `process_attestation` ](#updated-process_attestation )
- [`update_pending_votes` ](#update_pending_votes )
- [`process_shard_header` ](#process_shard_header )
2021-04-04 00:45:57 +00:00
- [Shard Proposer slashings ](#shard-proposer-slashings )
2020-12-11 08:10:50 +00:00
- [Epoch transition ](#epoch-transition )
- [Pending headers ](#pending-headers )
2021-03-18 00:06:37 +00:00
- [Shard epoch increment ](#shard-epoch-increment )
2019-10-12 03:05:08 +00:00
2020-01-24 23:43:43 +00:00
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
2021-03-17 23:07:15 +00:00
<!-- /TOC -->
2019-10-12 03:05:08 +00:00
## Introduction
2020-12-09 07:29:21 +00:00
This document describes the extensions made to the Phase 0 design of The Beacon Chain to support data sharding,
based on the ideas [here ](https://hackmd.io/G-Iy5jqyT7CXWEz8Ssos8g ) and more broadly [here ](https://arxiv.org/abs/1809.09044 ),
using KZG10 commitments to commit to data to remove any need for fraud proofs (and hence, safety-critical synchrony assumptions) in the design.
2019-10-12 03:05:08 +00:00
2019-11-18 23:40:02 +00:00
## Custom types
We define the following Python custom types for type hinting and readability:
| Name | SSZ equivalent | Description |
| - | - | - |
2020-12-10 06:29:09 +00:00
| `Shard` | `uint64` | A shard number |
2021-04-05 14:39:04 +00:00
| `BLSCommitment` | `Bytes48` | A G1 curve point |
2021-03-18 00:09:15 +00:00
| `BLSPoint` | `uint256` | A number `x` in the range `0 <= x < MODULUS` |
2019-11-18 23:40:02 +00:00
2021-03-26 23:47:10 +00:00
## Constants
The following values are (non-configurable) constants used throughout the specification.
2021-05-06 16:28:46 +00:00
### Misc
2021-03-26 23:47:10 +00:00
| Name | Value | Notes |
| - | - | - |
| `PRIMITIVE_ROOT_OF_UNITY` | `5` | Primitive root of unity of the BLS12_381 (inner) modulus |
2021-03-29 23:33:17 +00:00
| `DATA_AVAILABILITY_INVERSE_CODING_RATE` | `2**1` (= 2) | Factor by which samples are extended for data availability encoding |
2021-03-26 23:47:10 +00:00
| `POINTS_PER_SAMPLE` | `uint64(2**3)` (= 8) | 31 * 8 = 248 bytes |
| `MODULUS` | `0x73eda753299d7d483339d80809a1d80553bda402fffe5bfeffffffff00000001` (curve order of BLS12_381) |
2021-05-06 16:28:46 +00:00
### Domain types
| Name | Value |
| - | - |
| `DOMAIN_SHARD_PROPOSER` | `DomainType('0x80000000')` |
| `DOMAIN_SHARD_COMMITTEE` | `DomainType('0x81000000')` |
## Preset
2019-10-12 03:05:08 +00:00
### Misc
2020-12-10 02:21:21 +00:00
| Name | Value | Notes |
| - | - | - |
2021-03-29 23:33:17 +00:00
| `MAX_SHARDS` | `uint64(2**10)` (= 1,024) | Theoretical max shard count (used to determine data structure sizes) |
2020-12-14 22:57:53 +00:00
| `GASPRICE_ADJUSTMENT_COEFFICIENT` | `uint64(2**3)` (= 8) | Gasprice may decrease/increase by at most exp(1 / this value) *per epoch* |
2021-04-06 01:17:07 +00:00
| `MAX_SHARD_PROPOSER_SLASHINGS` | `2**4` (= 16) | Maximum amount of shard proposer slashing operations per block |
2021-05-06 16:28:46 +00:00
| `MAX_SHARD_HEADERS_PER_SHARD` | `4` | |
2020-12-16 14:14:21 +00:00
2021-05-06 16:28:46 +00:00
### Shard block samples
2020-06-09 22:38:40 +00:00
2020-12-09 07:29:21 +00:00
| Name | Value | Notes |
2020-06-15 07:27:37 +00:00
| - | - | - |
2020-12-09 07:29:21 +00:00
| `MAX_SAMPLES_PER_BLOCK` | `uint64(2**11)` (= 2,048) | 248 * 2,048 = 507,904 bytes |
| `TARGET_SAMPLES_PER_BLOCK` | `uint64(2**10)` (= 1,024) | 248 * 1,024 = 253,952 bytes |
2020-06-09 20:34:46 +00:00
2020-12-09 07:29:21 +00:00
### Precomputed size verification points
2020-06-09 20:34:46 +00:00
| Name | Value |
| - | - |
2020-12-28 17:38:09 +00:00
| `G1_SETUP` | Type `List[G1]` . The G1-side trusted setup `[G, G*s, G*s**2....]` ; note that the first point is the generator. |
| `G2_SETUP` | Type `List[G2]` . The G2-side trusted setup `[G, G*s, G*s**2....]` |
2021-04-05 20:13:54 +00:00
| `ROOT_OF_UNITY` | `pow(PRIMITIVE_ROOT_OF_UNITY, (MODULUS - 1) // int(MAX_SAMPLES_PER_BLOCK * POINTS_PER_SAMPLE), MODULUS)` |
2020-06-09 20:34:46 +00:00
2020-12-09 07:29:21 +00:00
### Gwei values
| Name | Value | Unit | Description |
| - | - | - | - |
2020-12-18 13:57:05 +00:00
| `MAX_GASPRICE` | `Gwei(2**33)` (= 8,589,934,592) | Gwei | Max gasprice charged for a TARGET-sized shard block |
2020-12-14 22:57:53 +00:00
| `MIN_GASPRICE` | `Gwei(2**3)` (= 8) | Gwei | Min gasprice charged for a TARGET-sized shard block |
2020-06-09 20:34:46 +00:00
2021-05-06 16:28:46 +00:00
## Configuration
2020-04-02 22:48:02 +00:00
2021-05-06 16:28:46 +00:00
| Name | Value | Notes |
| - | - | - |
| `INITIAL_ACTIVE_SHARDS` | `uint64(2**6)` (= 64) | Initial shard count |
2019-10-12 03:05:08 +00:00
2019-11-22 22:45:55 +00:00
## Updated containers
2019-10-12 03:05:08 +00:00
2021-03-18 00:06:37 +00:00
The following containers have updated definitions to support Sharding.
2019-11-03 20:06:19 +00:00
2020-12-09 07:29:21 +00:00
### `AttestationData`
2019-10-27 01:01:10 +00:00
2019-10-29 17:43:13 +00:00
```python
2019-11-22 22:45:55 +00:00
class AttestationData(Container):
2019-10-12 03:05:08 +00:00
slot: Slot
2019-10-27 01:01:10 +00:00
index: CommitteeIndex
2019-10-12 03:05:08 +00:00
# LMD GHOST vote
2019-12-05 19:36:48 +00:00
beacon_block_root: Root
2019-10-12 03:05:08 +00:00
# FFG vote
source: Checkpoint
target: Checkpoint
2020-12-09 07:29:21 +00:00
# Shard header root
2021-03-29 23:33:17 +00:00
shard_header_root: Root # [New in Sharding]
2019-11-15 20:11:42 +00:00
```
2021-03-18 00:12:55 +00:00
### `BeaconBlockBody`
2020-12-10 03:47:02 +00:00
```python
2021-03-27 00:28:52 +00:00
class BeaconBlockBody(merge.BeaconBlockBody): # [extends The Merge block body]
2021-04-04 00:45:57 +00:00
shard_proposer_slashings: List[ShardProposerSlashing, MAX_SHARD_PROPOSER_SLASHINGS]
shard_headers: List[SignedShardBlobHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD]
2020-12-10 03:47:02 +00:00
```
2020-12-09 07:29:21 +00:00
### `BeaconState`
2019-11-16 10:13:47 +00:00
2019-11-15 20:11:42 +00:00
```python
2021-04-29 13:39:57 +00:00
class BeaconState(merge.BeaconState): # [extends The Merge state]
2021-03-27 00:28:52 +00:00
# [Updated fields]
2020-12-11 08:17:16 +00:00
previous_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
current_epoch_attestations: List[PendingAttestation, MAX_ATTESTATIONS * SLOTS_PER_EPOCH]
2021-03-27 00:28:52 +00:00
# [New fields]
2021-03-26 23:47:10 +00:00
previous_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD * SLOTS_PER_EPOCH]
current_epoch_pending_shard_headers: List[PendingShardHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD * SLOTS_PER_EPOCH]
2020-12-28 16:53:50 +00:00
grandparent_epoch_confirmed_commitments: Vector[Vector[DataCommitment, SLOTS_PER_EPOCH], MAX_SHARDS]
2020-12-09 07:29:21 +00:00
shard_gasprice: uint64
2020-12-10 03:47:02 +00:00
current_epoch_start_shard: Shard
2019-10-12 14:59:51 +00:00
```
2019-11-22 22:45:55 +00:00
## New containers
2021-03-18 00:06:37 +00:00
The shard data itself is network-layer only, and can be found in the [P2P specification ](./p2p-interface.md ).
The beacon chain registers just the commitments of the shard data.
2019-11-22 22:45:55 +00:00
2020-12-10 06:42:43 +00:00
### `DataCommitment`
```python
class DataCommitment(Container):
2020-12-28 16:24:31 +00:00
# KZG10 commitment to the data
2020-12-10 06:42:43 +00:00
point: BLSCommitment
# Length of the data in samples
2020-12-28 16:01:47 +00:00
length: uint64
2020-12-10 06:42:43 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobBodySummary`
2019-11-22 22:45:55 +00:00
```python
2021-04-04 00:45:57 +00:00
class ShardBlobBodySummary(Container):
2020-12-10 06:42:43 +00:00
# The actual data commitment
commitment: DataCommitment
2020-12-28 16:36:28 +00:00
# Proof that the degree < commitment.length
2021-01-02 21:31:25 +00:00
degree_proof: BLSCommitment
2021-04-04 00:45:57 +00:00
# Hash-tree-root as summary of the data field
data_root: Root
2021-04-06 17:31:20 +00:00
# Latest block root of the Beacon Chain, before shard_blob.slot
beacon_block_root: Root
2020-03-31 04:37:36 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobHeader`
```python
class ShardBlobHeader(Container):
# Slot and shard that this header is intended for
slot: Slot
shard: Shard
2021-05-27 01:57:59 +00:00
# SSZ-summary of ShardBlobBody
2021-04-04 00:45:57 +00:00
body_summary: ShardBlobBodySummary
# Proposer of the shard-blob
proposer_index: ValidatorIndex
```
2021-01-01 20:52:29 +00:00
2021-04-04 00:45:57 +00:00
### `SignedShardBlobHeader`
2021-01-04 18:17:30 +00:00
```python
2021-04-04 00:45:57 +00:00
class SignedShardBlobHeader(Container):
message: ShardBlobHeader
2021-01-04 18:17:30 +00:00
signature: BLSSignature
```
2020-12-09 07:29:21 +00:00
### `PendingShardHeader`
2020-03-31 04:37:36 +00:00
```python
2020-12-09 07:29:21 +00:00
class PendingShardHeader(Container):
# Slot and shard that this header is intended for
2020-12-28 16:14:05 +00:00
slot: Slot
2020-05-28 13:32:27 +00:00
shard: Shard
2020-12-28 16:24:31 +00:00
# KZG10 commitment to the data
2020-12-28 17:21:46 +00:00
commitment: DataCommitment
2021-01-04 18:17:30 +00:00
# hash_tree_root of the ShardHeader (stored so that attestations can be checked against it)
2020-12-28 16:55:06 +00:00
root: Root
2020-12-09 07:29:21 +00:00
# Who voted for the header
2020-12-28 16:56:43 +00:00
votes: Bitlist[MAX_VALIDATORS_PER_COMMITTEE]
2020-12-09 07:29:21 +00:00
# Has this header been confirmed?
2021-04-05 15:22:53 +00:00
confirmed: boolean
2019-11-22 22:45:55 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobReference`
```python
class ShardBlobReference(Container):
2021-04-06 01:17:07 +00:00
# Slot and shard that this reference is intended for
2021-04-04 00:45:57 +00:00
slot: Slot
shard: Shard
2021-05-27 01:57:59 +00:00
# Hash-tree-root of ShardBlobBody
2021-04-04 00:45:57 +00:00
body_root: Root
# Proposer of the shard-blob
proposer_index: ValidatorIndex
```
### `SignedShardBlobReference`
```python
2021-04-06 01:17:07 +00:00
class SignedShardBlobReference(Container):
2021-04-04 00:45:57 +00:00
message: ShardBlobReference
signature: BLSSignature
```
### `ShardProposerSlashing`
```python
class ShardProposerSlashing(Container):
2021-04-06 01:17:07 +00:00
signed_reference_1: SignedShardBlobReference
signed_reference_2: SignedShardBlobReference
2021-04-04 00:45:57 +00:00
```
2019-11-15 20:11:42 +00:00
## Helper functions
### Misc
2020-12-18 20:33:34 +00:00
#### `next_power_of_two`
```python
2021-04-23 15:47:25 +00:00
def next_power_of_two(x: int) -> int:
2020-12-18 20:33:34 +00:00
return 2 ** ((x - 1).bit_length())
```
2020-04-28 03:28:21 +00:00
#### `compute_previous_slot`
2020-01-05 19:11:55 +00:00
```python
2020-04-28 03:28:21 +00:00
def compute_previous_slot(slot: Slot) -> Slot:
2020-01-05 19:11:55 +00:00
if slot > 0:
return Slot(slot - 1)
else:
return Slot(0)
```
2020-04-16 11:43:48 +00:00
#### `compute_updated_gasprice`
```python
2020-12-10 02:21:21 +00:00
def compute_updated_gasprice(prev_gasprice: Gwei, shard_block_length: uint64, adjustment_quotient: uint64) -> Gwei:
2020-12-09 07:29:21 +00:00
if shard_block_length > TARGET_SAMPLES_PER_BLOCK:
2020-12-16 03:55:31 +00:00
delta = max(1, prev_gasprice * (shard_block_length - TARGET_SAMPLES_PER_BLOCK)
// TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
2020-04-16 11:43:48 +00:00
return min(prev_gasprice + delta, MAX_GASPRICE)
else:
2020-12-16 03:55:31 +00:00
delta = max(1, prev_gasprice * (TARGET_SAMPLES_PER_BLOCK - shard_block_length)
// TARGET_SAMPLES_PER_BLOCK // adjustment_quotient)
2020-04-16 11:43:48 +00:00
return max(prev_gasprice, MIN_GASPRICE + delta) - delta
```
2020-05-29 19:09:42 +00:00
#### `compute_committee_source_epoch`
```python
def compute_committee_source_epoch(epoch: Epoch, period: uint64) -> Epoch:
"""
Return the source epoch for computing the committee.
"""
2020-09-03 11:24:17 +00:00
source_epoch = Epoch(epoch - epoch % period)
2020-05-29 19:09:42 +00:00
if source_epoch >= period:
source_epoch -= period # `period` epochs lookahead
return source_epoch
```
2019-11-15 20:11:42 +00:00
### Beacon state accessors
2020-08-10 19:02:22 +00:00
#### Updated `get_committee_count_per_slot`
2020-08-05 18:44:31 +00:00
```python
def get_committee_count_per_slot(state: BeaconState, epoch: Epoch) -> uint64:
"""
Return the number of committees in each slot for the given ``epoch``.
"""
return max(uint64(1), min(
2020-12-10 03:47:02 +00:00
get_active_shard_count(state, epoch),
2020-08-05 18:44:31 +00:00
uint64(len(get_active_validator_indices(state, epoch))) // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
))
```
2020-01-05 19:20:20 +00:00
#### `get_active_shard_count`
```python
2020-12-10 03:47:02 +00:00
def get_active_shard_count(state: BeaconState, epoch: Epoch) -> uint64:
2020-08-10 19:02:22 +00:00
"""
Return the number of active shards.
Note that this puts an upper bound on the number of committees per slot.
"""
2020-08-05 18:44:31 +00:00
return INITIAL_ACTIVE_SHARDS
2020-01-05 19:20:20 +00:00
```
2019-11-15 20:11:42 +00:00
#### `get_shard_committee`
2019-11-12 14:13:47 +00:00
```python
def get_shard_committee(beacon_state: BeaconState, epoch: Epoch, shard: Shard) -> Sequence[ValidatorIndex]:
2020-05-28 14:39:52 +00:00
"""
Return the shard committee of the given ``epoch`` of the given ``shard``.
"""
2020-05-29 19:09:42 +00:00
source_epoch = compute_committee_source_epoch(epoch, SHARD_COMMITTEE_PERIOD)
2019-11-12 14:13:47 +00:00
active_validator_indices = get_active_validator_indices(beacon_state, source_epoch)
seed = get_seed(beacon_state, source_epoch, DOMAIN_SHARD_COMMITTEE)
2020-04-02 03:20:22 +00:00
return compute_committee(
indices=active_validator_indices,
seed=seed,
index=shard,
2020-12-10 03:47:02 +00:00
count=get_active_shard_count(beacon_state, epoch),
2020-04-02 03:20:22 +00:00
)
2019-11-12 14:13:47 +00:00
```
2020-12-18 13:57:05 +00:00
#### `compute_proposer_index`
2021-01-04 18:17:30 +00:00
Updated version to get a proposer index that will only allow proposers with a certain minimum balance,
ensuring that the balance is always sufficient to cover gas costs.
2020-12-18 13:57:05 +00:00
```python
2021-01-04 18:17:30 +00:00
def compute_proposer_index(beacon_state: BeaconState,
indices: Sequence[ValidatorIndex],
seed: Bytes32,
2021-04-23 15:47:25 +00:00
min_effective_balance: Gwei = Gwei(0)) -> ValidatorIndex:
2020-12-18 13:57:05 +00:00
"""
Return from ``indices`` a random index sampled by effective balance.
"""
assert len(indices) > 0
MAX_RANDOM_BYTE = 2**8 - 1
i = uint64(0)
total = uint64(len(indices))
while True:
candidate_index = indices[compute_shuffled_index(i % total, total, seed)]
random_byte = hash(seed + uint_to_bytes(uint64(i // 32)))[i % 32]
2021-01-04 18:17:30 +00:00
effective_balance = beacon_state.validators[candidate_index].effective_balance
if effective_balance < = min_effective_balance:
continue
if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
2020-12-18 13:57:05 +00:00
return candidate_index
i += 1
```
2019-11-15 20:11:42 +00:00
#### `get_shard_proposer_index`
2019-11-12 14:13:47 +00:00
```python
def get_shard_proposer_index(beacon_state: BeaconState, slot: Slot, shard: Shard) -> ValidatorIndex:
2020-06-04 03:19:04 +00:00
"""
Return the proposer's index of shard block at ``slot``.
"""
2020-06-03 21:06:04 +00:00
epoch = compute_epoch_at_slot(slot)
2020-06-04 03:19:04 +00:00
committee = get_shard_committee(beacon_state, epoch, shard)
2021-05-05 14:40:23 +00:00
seed = hash(get_seed(beacon_state, epoch, DOMAIN_SHARD_PROPOSER) + uint_to_bytes(slot))
2020-12-18 13:57:05 +00:00
2021-01-04 18:17:30 +00:00
# Proposer must have sufficient balance to pay for worst case fee burn
EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION = (
2021-05-05 17:17:09 +00:00
EFFECTIVE_BALANCE_INCREMENT - EFFECTIVE_BALANCE_INCREMENT
2021-01-04 18:17:30 +00:00
* HYSTERESIS_DOWNWARD_MULTIPLIER // HYSTERESIS_QUOTIENT
)
min_effective_balance = (
beacon_state.shard_gasprice * MAX_SAMPLES_PER_BLOCK // TARGET_SAMPLES_PER_BLOCK
+ EFFECTIVE_BALANCE_MAX_DOWNWARD_DEVIATION
)
return compute_proposer_index(beacon_state, committee, seed, min_effective_balance)
2020-05-29 15:50:18 +00:00
```
2019-11-18 23:40:02 +00:00
#### `get_start_shard`
```python
def get_start_shard(state: BeaconState, slot: Slot) -> Shard:
2020-05-29 15:50:18 +00:00
"""
Return the start shard at ``slot``.
"""
current_epoch_start_slot = compute_start_slot_at_epoch(get_current_epoch(state))
2020-12-10 03:47:02 +00:00
shard = state.current_epoch_start_shard
if slot > current_epoch_start_slot:
2020-05-29 15:50:18 +00:00
# Current epoch or the next epoch lookahead
2020-12-10 03:47:02 +00:00
for _slot in range(current_epoch_start_slot, slot):
committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
shard = (shard + committee_count) % active_shard_count
elif slot < current_epoch_start_slot:
2020-05-29 15:50:18 +00:00
# Previous epoch
2020-12-10 03:47:02 +00:00
for _slot in list(range(slot, current_epoch_start_slot))[::-1]:
committee_count = get_committee_count_per_slot(state, compute_epoch_at_slot(Slot(_slot)))
active_shard_count = get_active_shard_count(state, compute_epoch_at_slot(Slot(_slot)))
2020-05-29 15:50:18 +00:00
# Ensure positive
2020-12-10 03:47:02 +00:00
shard = (shard + active_shard_count - committee_count) % active_shard_count
return Shard(shard)
2019-11-18 23:40:02 +00:00
```
2020-12-31 01:56:29 +00:00
#### `compute_shard_from_committee_index`
```python
def compute_shard_from_committee_index(state: BeaconState, slot: Slot, index: CommitteeIndex) -> Shard:
active_shards = get_active_shard_count(state, compute_epoch_at_slot(slot))
return Shard((index + get_start_shard(state, slot)) % active_shards)
```
#### `compute_committee_index_from_shard`
```python
def compute_committee_index_from_shard(state: BeaconState, slot: Slot, shard: Shard) -> CommitteeIndex:
2021-05-27 12:13:13 +00:00
"""
Returns either committee index for ``shard`` at ``slot`` or ``None`` if no committee
"""
2020-12-31 01:56:29 +00:00
active_shards = get_active_shard_count(state, compute_epoch_at_slot(slot))
2021-05-27 12:13:13 +00:00
index = (active_shards + shard - get_start_shard(state, slot)) % active_shards
if index >= get_committee_count_per_slot(state, compute_epoch_at_slot(slot)):
return None
else:
return CommitteeIndex(index)
2020-12-31 01:56:29 +00:00
```
2019-11-15 20:11:42 +00:00
### Block processing
2019-11-06 22:19:00 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
process_randao(state, block.body)
process_eth1_data(state, block.body)
2021-03-29 23:33:17 +00:00
process_operations(state, block.body) # [Modified in Sharding]
2021-05-18 11:14:43 +00:00
# Pre-merge, skip execution payload processing
if is_execution_enabled(state, block):
process_execution_payload(state, block.body.execution_payload, EXECUTION_ENGINE) # [New in Merge]
2019-11-12 13:27:34 +00:00
```
2019-11-15 20:11:42 +00:00
#### Operations
2019-10-12 03:05:08 +00:00
2019-10-13 08:42:55 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
# Verify that outstanding deposits are processed up to the maximum number of deposits
assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)
2020-02-06 18:58:21 +00:00
2019-11-20 03:43:32 +00:00
def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
2019-11-15 20:11:42 +00:00
for operation in operations:
fn(state, operation)
2020-02-06 18:58:21 +00:00
2019-11-16 10:13:47 +00:00
for_ops(body.proposer_slashings, process_proposer_slashing)
for_ops(body.attester_slashings, process_attester_slashing)
2021-04-04 00:45:57 +00:00
# New shard proposer slashing processing
for_ops(body.shard_proposer_slashings, process_shard_proposer_slashing)
2021-01-04 18:17:30 +00:00
# Limit is dynamic based on active shard count
assert len(body.shard_headers) < = MAX_SHARD_HEADERS_PER_SHARD * get_active_shard_count(state, get_current_epoch(state))
for_ops(body.shard_headers, process_shard_header)
2019-11-15 20:11:42 +00:00
# New attestation processing
2020-02-06 18:58:21 +00:00
for_ops(body.attestations, process_attestation)
2019-11-16 10:13:47 +00:00
for_ops(body.deposits, process_deposit)
for_ops(body.voluntary_exits, process_voluntary_exit)
2019-10-12 03:05:08 +00:00
```
2020-12-09 07:29:21 +00:00
### New Attestation processing
2019-11-03 16:17:46 +00:00
2020-12-09 07:29:21 +00:00
#### Updated `process_attestation`
2020-05-28 19:20:36 +00:00
```python
def process_attestation(state: BeaconState, attestation: Attestation) -> None:
2020-12-09 07:29:21 +00:00
phase0.process_attestation(state, attestation)
2020-12-31 01:35:27 +00:00
update_pending_votes(state, attestation)
2020-05-28 19:20:36 +00:00
```
2020-12-09 07:29:21 +00:00
#### `update_pending_votes`
2019-11-03 16:17:46 +00:00
```python
2020-12-09 07:29:21 +00:00
def update_pending_votes(state: BeaconState, attestation: Attestation) -> None:
2021-01-04 18:17:30 +00:00
# Find and update the PendingShardHeader object, invalid block if pending header not in state
if compute_epoch_at_slot(attestation.data.slot) == get_current_epoch(state):
2020-12-13 00:36:45 +00:00
pending_headers = state.current_epoch_pending_shard_headers
2020-12-09 07:29:21 +00:00
else:
2020-12-13 00:36:45 +00:00
pending_headers = state.previous_epoch_pending_shard_headers
2021-05-05 10:31:19 +00:00
attestation_shard = compute_shard_from_committee_index(
2020-12-09 07:29:21 +00:00
state,
2020-12-31 01:56:29 +00:00
attestation.data.slot,
2020-12-09 07:29:21 +00:00
attestation.data.index,
)
2021-05-05 10:31:19 +00:00
pending_header = None
for header in pending_headers:
if (
header.root == attestation.data.shard_header_root
and header.slot == attestation.data.slot
and header.shard == attestation_shard
):
pending_header = header
assert pending_header is not None
2021-01-04 18:17:30 +00:00
for i in range(len(pending_header.votes)):
pending_header.votes[i] = pending_header.votes[i] or attestation.aggregation_bits[i]
2019-11-03 20:06:19 +00:00
2020-12-09 07:29:21 +00:00
# Check if the PendingShardHeader is eligible for expedited confirmation
# Requirement 1: nothing else confirmed
all_candidates = [
c for c in pending_headers if
(c.slot, c.shard) == (pending_header.slot, pending_header.shard)
]
2021-01-04 18:17:30 +00:00
if True in [c.confirmed for c in all_candidates]:
return
# Requirement 2: >= 2/3 of balance attesting
2021-03-17 21:41:35 +00:00
participants = get_attesting_indices(state, attestation.data, pending_header.votes)
2021-01-04 18:17:30 +00:00
participants_balance = get_total_balance(state, participants)
full_committee = get_beacon_committee(state, attestation.data.slot, attestation.data.index)
2021-04-24 15:21:59 +00:00
full_committee_balance = get_total_balance(state, set(full_committee))
2021-01-04 18:17:30 +00:00
if participants_balance * 3 >= full_committee_balance * 2:
pending_header.confirmed = True
2020-12-09 07:29:21 +00:00
```
2020-12-10 03:47:02 +00:00
#### `process_shard_header`
2020-12-09 07:29:21 +00:00
```python
2020-12-10 03:47:02 +00:00
def process_shard_header(state: BeaconState,
2021-04-04 00:45:57 +00:00
signed_header: SignedShardBlobHeader) -> None:
2020-12-09 07:29:21 +00:00
header = signed_header.message
2021-05-27 12:13:13 +00:00
committee_index = compute_committee_index_from_shard(state, header.slot, header.shard)
2021-04-06 15:07:29 +00:00
# Verify the header is not 0, and not from the future.
assert Slot(0) < header.slot < = state . slot
2021-04-04 00:45:57 +00:00
header_epoch = compute_epoch_at_slot(header.slot)
# Verify that the header is within the processing time window
assert header_epoch in [get_previous_epoch(state), get_current_epoch(state)]
# Verify that the shard is active
assert header.shard < get_active_shard_count ( state , header_epoch )
2021-05-27 12:13:13 +00:00
# Verify that shard has a committee at slot
assert committee_index is not None
2021-04-06 17:31:20 +00:00
# Verify that the block root matches,
# to ensure the header will only be included in this specific Beacon Chain sub-tree.
2021-04-23 21:12:28 +00:00
assert header.body_summary.beacon_block_root == get_block_root_at_slot(state, header.slot - 1)
2021-04-04 00:45:57 +00:00
# Verify proposer
assert header.proposer_index == get_shard_proposer_index(state, header.slot, header.shard)
2020-12-09 07:29:21 +00:00
# Verify signature
2021-04-30 11:14:09 +00:00
signing_root = compute_signing_root(header, get_domain(state, DOMAIN_SHARD_PROPOSER))
2021-04-06 01:17:07 +00:00
assert bls.Verify(state.validators[header.proposer_index].pubkey, signing_root, signed_header.signature)
2021-01-04 18:17:30 +00:00
2020-12-28 17:38:09 +00:00
# Verify the length by verifying the degree.
2021-04-04 00:45:57 +00:00
body_summary = header.body_summary
if body_summary.commitment.length == 0:
assert body_summary.degree_proof == G1_SETUP[0]
2019-11-18 22:07:50 +00:00
assert (
2021-04-04 00:45:57 +00:00
bls.Pairing(body_summary.degree_proof, G2_SETUP[0])
2021-04-06 17:49:48 +00:00
== bls.Pairing(body_summary.commitment.point, G2_SETUP[-body_summary.commitment.length])
2019-11-18 22:07:50 +00:00
)
2021-01-04 18:17:30 +00:00
2020-12-09 07:29:21 +00:00
# Get the correct pending header list
2021-04-04 00:45:57 +00:00
if header_epoch == get_current_epoch(state):
2020-12-13 00:36:45 +00:00
pending_headers = state.current_epoch_pending_shard_headers
2020-12-09 07:29:21 +00:00
else:
2020-12-13 00:36:45 +00:00
pending_headers = state.previous_epoch_pending_shard_headers
2021-01-04 18:17:30 +00:00
2021-04-04 00:45:57 +00:00
header_root = hash_tree_root(header)
2020-12-09 07:29:21 +00:00
# Check that this header is not yet in the pending list
2021-01-04 18:17:30 +00:00
assert header_root not in [pending_header.root for pending_header in pending_headers]
2020-12-09 07:29:21 +00:00
# Include it in the pending list
2021-05-27 12:13:13 +00:00
committee_length = len(get_beacon_committee(state, header.slot, committee_index))
2020-12-09 07:29:21 +00:00
pending_headers.append(PendingShardHeader(
slot=header.slot,
shard=header.shard,
2021-04-04 00:45:57 +00:00
commitment=body_summary.commitment,
2020-12-09 07:29:21 +00:00
root=header_root,
2020-12-28 16:03:20 +00:00
votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length),
2021-01-04 18:17:30 +00:00
confirmed=False,
2020-12-09 07:29:21 +00:00
))
2020-02-06 18:58:21 +00:00
```
2020-12-09 07:29:21 +00:00
The degree proof works as follows. For a block `B` with length `l` (so `l` values in `[0...l - 1]` , seen as a polynomial `B(X)` which takes these values),
the length proof is the commitment to the polynomial `B(X) * X**(MAX_DEGREE + 1 - l)` ,
where `MAX_DEGREE` is the maximum power of `s` available in the setup, which is `MAX_DEGREE = len(G2_SETUP) - 1` .
The goal is to ensure that a proof can only be constructed if `deg(B) < l` (there are not hidden higher-order terms in the polynomial, which would thwart reconstruction).
2020-02-06 18:58:21 +00:00
2021-04-04 00:45:57 +00:00
##### Shard Proposer slashings
```python
def process_shard_proposer_slashing(state: BeaconState, proposer_slashing: ShardProposerSlashing) -> None:
2021-04-06 01:17:07 +00:00
reference_1 = proposer_slashing.signed_reference_1.message
reference_2 = proposer_slashing.signed_reference_2.message
2021-04-04 00:45:57 +00:00
# Verify header slots match
2021-04-06 01:17:07 +00:00
assert reference_1.slot == reference_2.slot
2021-04-04 00:45:57 +00:00
# Verify header shards match
2021-04-06 01:17:07 +00:00
assert reference_1.shard == reference_2.shard
2021-04-04 00:45:57 +00:00
# Verify header proposer indices match
2021-04-06 01:17:07 +00:00
assert reference_1.proposer_index == reference_2.proposer_index
2021-04-04 00:45:57 +00:00
# Verify the headers are different (i.e. different body)
2021-04-06 01:17:07 +00:00
assert reference_1 != reference_2
2021-04-04 00:45:57 +00:00
# Verify the proposer is slashable
2021-04-06 01:17:07 +00:00
proposer = state.validators[reference_1.proposer_index]
2021-04-04 00:45:57 +00:00
assert is_slashable_validator(proposer, get_current_epoch(state))
# Verify signatures
2021-04-06 01:17:07 +00:00
for signed_header in (proposer_slashing.signed_reference_1, proposer_slashing.signed_reference_2):
2021-04-04 00:45:57 +00:00
domain = get_domain(state, DOMAIN_SHARD_PROPOSER, compute_epoch_at_slot(signed_header.message.slot))
signing_root = compute_signing_root(signed_header.message, domain)
assert bls.Verify(proposer.pubkey, signing_root, signed_header.signature)
2021-04-06 01:17:07 +00:00
slash_validator(state, reference_1.proposer_index)
2021-04-04 00:45:57 +00:00
```
2019-10-12 14:59:51 +00:00
### Epoch transition
2021-03-27 00:28:52 +00:00
This epoch transition overrides the Merge epoch transition:
2019-11-15 20:11:42 +00:00
2019-10-12 14:59:51 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_epoch(state: BeaconState) -> None:
process_justification_and_finalization(state)
process_rewards_and_penalties(state)
process_registry_updates(state)
2020-12-09 07:29:21 +00:00
2019-11-15 20:11:42 +00:00
process_slashings(state)
2020-12-09 07:29:21 +00:00
# Sharding
process_pending_headers(state)
2021-04-24 14:46:55 +00:00
charge_confirmed_header_fees(state)
2020-12-09 07:29:21 +00:00
reset_pending_headers(state)
# Final updates
# Phase 0
2021-01-27 06:42:50 +00:00
process_eth1_data_reset(state)
process_effective_balance_updates(state)
process_slashings_reset(state)
process_randao_mixes_reset(state)
process_historical_roots_update(state)
2021-01-19 13:41:34 +00:00
process_participation_record_updates(state)
2021-03-18 00:06:37 +00:00
process_shard_epoch_increment(state)
2019-11-15 22:42:28 +00:00
```
2019-11-15 20:11:42 +00:00
2020-12-09 07:29:21 +00:00
#### Pending headers
```python
2021-01-04 18:17:30 +00:00
def process_pending_headers(state: BeaconState) -> None:
# Pending header processing applies to the previous epoch.
# Skip if `GENESIS_EPOCH` because no prior epoch to process.
if get_current_epoch(state) == GENESIS_EPOCH:
return
2021-04-22 17:38:21 +00:00
previous_epoch = get_previous_epoch(state)
previous_epoch_start_slot = compute_start_slot_at_epoch(previous_epoch)
2021-01-04 18:17:30 +00:00
for slot in range(previous_epoch_start_slot, previous_epoch_start_slot + SLOTS_PER_EPOCH):
2021-04-30 11:35:34 +00:00
for shard_index in range(get_active_shard_count(state, previous_epoch)):
2021-04-27 10:44:22 +00:00
shard = Shard(shard_index)
2020-12-09 07:29:21 +00:00
# Pending headers for this (slot, shard) combo
candidates = [
2021-01-04 18:17:30 +00:00
c for c in state.previous_epoch_pending_shard_headers
if (c.slot, c.shard) == (slot, shard)
2020-12-09 07:29:21 +00:00
]
2021-01-04 18:17:30 +00:00
# If any candidates already confirmed, skip
if True in [c.confirmed for c in candidates]:
continue
2021-04-22 19:08:44 +00:00
# The entire committee (and its balance)
2021-04-27 14:07:56 +00:00
index = compute_committee_index_from_shard(state, slot, shard)
2021-05-27 12:13:13 +00:00
if index is None:
# the shard had no committee on this slot
continue
2021-04-27 14:07:56 +00:00
full_committee = get_beacon_committee(state, slot, index)
2021-01-04 18:17:30 +00:00
# The set of voters who voted for each header (and their total balances)
voting_sets = [
2021-04-24 15:21:59 +00:00
set(v for i, v in enumerate(full_committee) if c.votes[i])
2021-01-04 18:17:30 +00:00
for c in candidates
]
voting_balances = [
get_total_balance(state, voters)
for voters in voting_sets
]
# Get the index with the most total balance voting for them.
# NOTE: if two choices get exactly the same voting balance,
# the candidate earlier in the list wins
if max(voting_balances) > 0:
winning_index = voting_balances.index(max(voting_balances))
else:
# If no votes, zero wins
winning_index = [c.root for c in candidates].index(Root())
candidates[winning_index].confirmed = True
for slot_index in range(SLOTS_PER_EPOCH):
2021-04-30 11:35:34 +00:00
for shard in range(MAX_SHARDS):
2021-01-04 18:17:30 +00:00
state.grandparent_epoch_confirmed_commitments[shard][slot_index] = DataCommitment()
2021-04-05 13:53:03 +00:00
confirmed_headers = [candidate for candidate in state.previous_epoch_pending_shard_headers if candidate.confirmed]
2021-01-04 18:17:30 +00:00
for header in confirmed_headers:
2021-04-24 14:38:09 +00:00
state.grandparent_epoch_confirmed_commitments[header.shard][header.slot % SLOTS_PER_EPOCH] = header.commitment
2021-01-04 18:17:30 +00:00
```
2020-12-09 07:29:21 +00:00
```python
def charge_confirmed_header_fees(state: BeaconState) -> None:
new_gasprice = state.shard_gasprice
2021-05-10 00:08:54 +00:00
previous_epoch = get_previous_epoch(state)
2021-01-04 18:17:30 +00:00
adjustment_quotient = (
2021-05-10 00:08:54 +00:00
get_active_shard_count(state, previous_epoch)
2021-01-04 18:17:30 +00:00
* SLOTS_PER_EPOCH * GASPRICE_ADJUSTMENT_COEFFICIENT
)
2021-04-30 11:39:37 +00:00
previous_epoch_start_slot = compute_start_slot_at_epoch(previous_epoch)
2021-01-04 18:17:30 +00:00
for slot in range(previous_epoch_start_slot, previous_epoch_start_slot + SLOTS_PER_EPOCH):
2021-04-30 11:39:37 +00:00
for shard_index in range(get_active_shard_count(state, previous_epoch)):
2021-04-27 12:44:25 +00:00
shard = Shard(shard_index)
2020-12-09 07:29:21 +00:00
confirmed_candidates = [
2021-01-04 18:17:30 +00:00
c for c in state.previous_epoch_pending_shard_headers
if (c.slot, c.shard, c.confirmed) == (slot, shard, True)
2020-12-09 07:29:21 +00:00
]
2021-01-04 18:17:30 +00:00
if not any(confirmed_candidates):
continue
candidate = confirmed_candidates[0]
# Charge EIP 1559 fee
2021-04-24 14:32:05 +00:00
proposer = get_shard_proposer_index(state, slot, shard)
2021-01-04 18:17:30 +00:00
fee = (
(state.shard_gasprice * candidate.commitment.length)
// TARGET_SAMPLES_PER_BLOCK
)
decrease_balance(state, proposer, fee)
# Track updated gas price
new_gasprice = compute_updated_gasprice(
new_gasprice,
candidate.commitment.length,
adjustment_quotient,
)
2020-12-09 07:29:21 +00:00
state.shard_gasprice = new_gasprice
```
```python
2020-12-09 07:29:21 +00:00
def reset_pending_headers(state: BeaconState) -> None:
2020-12-13 00:36:45 +00:00
state.previous_epoch_pending_shard_headers = state.current_epoch_pending_shard_headers
state.current_epoch_pending_shard_headers = []
2021-01-04 18:17:30 +00:00
# Add dummy "empty" PendingShardHeader (default vote for if no shard header available)
next_epoch = get_current_epoch(state) + 1
next_epoch_start_slot = compute_start_slot_at_epoch(next_epoch)
2021-05-14 19:02:46 +00:00
committees_per_slot = get_committee_count_per_slot(state, next_epoch)
2021-04-24 14:33:53 +00:00
for slot in range(next_epoch_start_slot, next_epoch_start_slot + SLOTS_PER_EPOCH):
2021-05-14 19:02:46 +00:00
for index in range(committees_per_slot):
2021-04-27 14:09:16 +00:00
committee_index = CommitteeIndex(index)
shard = compute_shard_from_committee_index(state, slot, committee_index)
committee_length = len(get_beacon_committee(state, slot, committee_index))
2020-12-13 00:36:45 +00:00
state.current_epoch_pending_shard_headers.append(PendingShardHeader(
2020-12-09 07:29:21 +00:00
slot=slot,
shard=shard,
2020-12-10 06:42:43 +00:00
commitment=DataCommitment(),
2020-12-09 07:29:21 +00:00
root=Root(),
2020-12-28 16:03:20 +00:00
votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length),
2021-01-04 18:17:30 +00:00
confirmed=False,
2020-12-09 07:29:21 +00:00
))
2019-10-12 14:59:51 +00:00
```
2019-11-15 22:42:28 +00:00
2021-03-18 00:06:37 +00:00
#### Shard epoch increment
2019-11-15 22:42:28 +00:00
2021-03-18 00:06:37 +00:00
```python
def process_shard_epoch_increment(state: BeaconState) -> None:
# Update current_epoch_start_shard
state.current_epoch_start_shard = get_start_shard(state, Slot(state.slot + 1))
```