2021-08-18 23:11:38 +00:00
# Sharding -- The Beacon Chain
2019-10-12 03:05:08 +00:00
**Notice**: This document is a work-in-progress for researchers and implementers.
## Table of contents
2021-03-17 23:07:15 +00:00
<!-- TOC -->
2020-01-24 23:43:43 +00:00
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE - RUN doctoc TO UPDATE -->
2020-09-23 20:55:22 +00:00
2020-01-24 23:43:43 +00:00
- [Introduction ](#introduction )
2021-06-18 00:21:21 +00:00
- [Glossary ](#glossary )
2020-01-24 23:43:43 +00:00
- [Custom types ](#custom-types )
2021-03-26 23:47:10 +00:00
- [Constants ](#constants )
2020-12-11 08:10:50 +00:00
- [Misc ](#misc )
2021-05-06 16:28:46 +00:00
- [Domain types ](#domain-types )
2021-06-03 15:32:35 +00:00
- [Shard Work Status ](#shard-work-status )
2021-05-06 16:28:46 +00:00
- [Misc ](#misc-1 )
2021-07-26 13:26:55 +00:00
- [Participation flag indices ](#participation-flag-indices )
- [Incentivization weights ](#incentivization-weights )
- [Preset ](#preset )
- [Misc ](#misc-2 )
2021-06-21 22:10:47 +00:00
- [Shard blob samples ](#shard-blob-samples )
2020-12-11 08:10:50 +00:00
- [Precomputed size verification points ](#precomputed-size-verification-points )
- [Gwei values ](#gwei-values )
2021-05-06 16:28:46 +00:00
- [Configuration ](#configuration )
2020-01-24 23:43:43 +00:00
- [Updated containers ](#updated-containers )
2020-12-11 08:10:50 +00:00
- [`AttestationData` ](#attestationdata )
2021-03-18 00:26:03 +00:00
- [`BeaconBlockBody` ](#beaconblockbody )
2020-12-11 08:10:50 +00:00
- [`BeaconState` ](#beaconstate )
2020-01-24 23:43:43 +00:00
- [New containers ](#new-containers )
2021-06-23 21:33:46 +00:00
- [`Builder` ](#builder )
2020-12-11 08:10:50 +00:00
- [`DataCommitment` ](#datacommitment )
2021-07-26 13:26:55 +00:00
- [`AttestedDataCommitment` ](#attesteddatacommitment )
2021-08-23 11:46:06 +00:00
- [`ShardBlobBody` ](#shardblobbody )
2021-04-04 00:45:57 +00:00
- [`ShardBlobBodySummary` ](#shardblobbodysummary )
2021-06-18 00:21:21 +00:00
- [`ShardBlob` ](#shardblob )
2021-04-04 00:45:57 +00:00
- [`ShardBlobHeader` ](#shardblobheader )
2021-06-18 00:21:21 +00:00
- [`SignedShardBlob` ](#signedshardblob )
2021-04-04 00:45:57 +00:00
- [`SignedShardBlobHeader` ](#signedshardblobheader )
2020-12-11 08:10:50 +00:00
- [`PendingShardHeader` ](#pendingshardheader )
2021-04-04 00:45:57 +00:00
- [`ShardBlobReference` ](#shardblobreference )
- [`ShardProposerSlashing` ](#shardproposerslashing )
2021-05-29 19:28:00 +00:00
- [`ShardWork` ](#shardwork )
2020-01-24 23:43:43 +00:00
- [Helper functions ](#helper-functions )
2021-07-26 13:26:55 +00:00
- [Misc ](#misc-3 )
2021-01-04 18:17:30 +00:00
- [`next_power_of_two` ](#next_power_of_two )
2020-12-11 08:10:50 +00:00
- [`compute_previous_slot` ](#compute_previous_slot )
2021-07-26 13:26:55 +00:00
- [`compute_updated_sample_price` ](#compute_updated_sample_price )
2020-12-11 08:10:50 +00:00
- [`compute_committee_source_epoch` ](#compute_committee_source_epoch )
2021-07-26 13:26:55 +00:00
- [`batch_apply_participation_flag` ](#batch_apply_participation_flag )
2020-12-11 08:10:50 +00:00
- [Beacon state accessors ](#beacon-state-accessors )
- [Updated `get_committee_count_per_slot` ](#updated-get_committee_count_per_slot )
- [`get_active_shard_count` ](#get_active_shard_count )
- [`get_shard_proposer_index` ](#get_shard_proposer_index )
- [`get_start_shard` ](#get_start_shard )
2021-01-04 18:17:30 +00:00
- [`compute_shard_from_committee_index` ](#compute_shard_from_committee_index )
- [`compute_committee_index_from_shard` ](#compute_committee_index_from_shard )
2020-12-11 08:10:50 +00:00
- [Block processing ](#block-processing )
- [Operations ](#operations )
2021-05-29 21:55:16 +00:00
- [Extended Attestation processing ](#extended-attestation-processing )
- [`process_shard_header` ](#process_shard_header )
- [`process_shard_proposer_slashing` ](#process_shard_proposer_slashing )
2020-12-11 08:10:50 +00:00
- [Epoch transition ](#epoch-transition )
2021-05-29 21:55:16 +00:00
- [`process_pending_shard_confirmations` ](#process_pending_shard_confirmations )
- [`reset_pending_shard_work` ](#reset_pending_shard_work )
2019-10-12 03:05:08 +00:00
2020-01-24 23:43:43 +00:00
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
2021-03-17 23:07:15 +00:00
<!-- /TOC -->
2019-10-12 03:05:08 +00:00
## Introduction
2020-12-09 07:29:21 +00:00
This document describes the extensions made to the Phase 0 design of The Beacon Chain to support data sharding,
based on the ideas [here ](https://hackmd.io/G-Iy5jqyT7CXWEz8Ssos8g ) and more broadly [here ](https://arxiv.org/abs/1809.09044 ),
using KZG10 commitments to commit to data to remove any need for fraud proofs (and hence, safety-critical synchrony assumptions) in the design.
2021-06-18 00:21:21 +00:00
### Glossary
- **Data**: A list of KZG points, to translate a byte string into
- **Blob**: Data with commitments and meta-data, like a flattened bundle of L2 transactions.
2021-06-23 21:33:46 +00:00
- **Builder**: Independent actor that builds blobs and bids for proposal slots via fee-paying blob-headers, responsible for availability.
- **Shard proposer**: Validator taking bids from blob builders for shard data opportunity, co-signs with builder to propose the blob.
2019-10-12 03:05:08 +00:00
2019-11-18 23:40:02 +00:00
## Custom types
We define the following Python custom types for type hinting and readability:
| Name | SSZ equivalent | Description |
| - | - | - |
2020-12-10 06:29:09 +00:00
| `Shard` | `uint64` | A shard number |
2021-04-05 14:39:04 +00:00
| `BLSCommitment` | `Bytes48` | A G1 curve point |
2021-03-18 00:09:15 +00:00
| `BLSPoint` | `uint256` | A number `x` in the range `0 <= x < MODULUS` |
2021-06-18 00:21:21 +00:00
| `BuilderIndex` | `uint64` | Builder registry index |
2019-11-18 23:40:02 +00:00
2021-03-26 23:47:10 +00:00
## Constants
The following values are (non-configurable) constants used throughout the specification.
2021-05-06 16:28:46 +00:00
### Misc
2021-03-26 23:47:10 +00:00
| Name | Value | Notes |
| - | - | - |
| `PRIMITIVE_ROOT_OF_UNITY` | `5` | Primitive root of unity of the BLS12_381 (inner) modulus |
2021-03-29 23:33:17 +00:00
| `DATA_AVAILABILITY_INVERSE_CODING_RATE` | `2**1` (= 2) | Factor by which samples are extended for data availability encoding |
2021-03-26 23:47:10 +00:00
| `POINTS_PER_SAMPLE` | `uint64(2**3)` (= 8) | 31 * 8 = 248 bytes |
| `MODULUS` | `0x73eda753299d7d483339d80809a1d80553bda402fffe5bfeffffffff00000001` (curve order of BLS12_381) |
2021-05-06 16:28:46 +00:00
### Domain types
| Name | Value |
| - | - |
2021-06-20 22:55:45 +00:00
| `DOMAIN_SHARD_BLOB` | `DomainType('0x80000000')` |
2021-05-06 16:28:46 +00:00
2021-06-03 15:32:35 +00:00
### Shard Work Status
2021-05-28 18:18:29 +00:00
| Name | Value | Notes |
| - | - | - |
2021-06-03 15:32:35 +00:00
| `SHARD_WORK_UNCONFIRMED` | `0` | Unconfirmed, nullified after confirmation time elapses |
| `SHARD_WORK_CONFIRMED` | `1` | Confirmed, reduced to just the commitment |
| `SHARD_WORK_PENDING` | `2` | Pending, a list of competing headers |
2021-05-28 18:18:29 +00:00
2021-07-26 13:26:55 +00:00
### Misc
TODO: `PARTICIPATION_FLAG_WEIGHTS` backwards-compatibility is difficult, depends on usage.
| Name | Value |
| - | - |
| `PARTICIPATION_FLAG_WEIGHTS` | `[TIMELY_SOURCE_WEIGHT, TIMELY_TARGET_WEIGHT, TIMELY_HEAD_WEIGHT, TIMELY_SHARD_WEIGHT]` |
### Participation flag indices
| Name | Value |
| - | - |
| `TIMELY_SHARD_FLAG_INDEX` | `3` |
### Incentivization weights
TODO: determine weight for shard attestations
| Name | Value |
| - | - |
| `TIMELY_SHARD_WEIGHT` | `uint64(8)` |
TODO: `WEIGHT_DENOMINATOR` needs to be adjusted, but this breaks a lot of Altair code.
2021-05-06 16:28:46 +00:00
## Preset
2019-10-12 03:05:08 +00:00
### Misc
2020-12-10 02:21:21 +00:00
| Name | Value | Notes |
| - | - | - |
2021-03-29 23:33:17 +00:00
| `MAX_SHARDS` | `uint64(2**10)` (= 1,024) | Theoretical max shard count (used to determine data structure sizes) |
2021-07-30 20:22:43 +00:00
| `INITIAL_ACTIVE_SHARDS` | `uint64(2**6)` (= 64) | Initial shard count |
2021-07-26 13:26:55 +00:00
| `SAMPLE_PRICE_ADJUSTMENT_COEFFICIENT` | `uint64(2**3)` (= 8) | Sample price may decrease/increase by at most exp(1 / this value) *per epoch* |
2021-04-06 01:17:07 +00:00
| `MAX_SHARD_PROPOSER_SLASHINGS` | `2**4` (= 16) | Maximum amount of shard proposer slashing operations per block |
2021-05-06 16:28:46 +00:00
| `MAX_SHARD_HEADERS_PER_SHARD` | `4` | |
2021-05-29 20:39:25 +00:00
| `SHARD_STATE_MEMORY_SLOTS` | `uint64(2**8)` (= 256) | Number of slots for which shard commitments and confirmation status is directly available in the state |
2021-06-23 21:33:46 +00:00
| `BLOB_BUILDER_REGISTRY_LIMIT` | `uint64(2**40)` (= 1,099,511,627,776) | shard blob builders |
2020-12-16 14:14:21 +00:00
2021-06-21 21:33:10 +00:00
### Shard blob samples
2020-06-09 22:38:40 +00:00
2020-12-09 07:29:21 +00:00
| Name | Value | Notes |
2020-06-15 07:27:37 +00:00
| - | - | - |
2021-06-21 21:33:10 +00:00
| `MAX_SAMPLES_PER_BLOB` | `uint64(2**11)` (= 2,048) | 248 * 2,048 = 507,904 bytes |
| `TARGET_SAMPLES_PER_BLOB` | `uint64(2**10)` (= 1,024) | 248 * 1,024 = 253,952 bytes |
2020-06-09 20:34:46 +00:00
2020-12-09 07:29:21 +00:00
### Precomputed size verification points
2020-06-09 20:34:46 +00:00
| Name | Value |
| - | - |
2020-12-28 17:38:09 +00:00
| `G1_SETUP` | Type `List[G1]` . The G1-side trusted setup `[G, G*s, G*s**2....]` ; note that the first point is the generator. |
| `G2_SETUP` | Type `List[G2]` . The G2-side trusted setup `[G, G*s, G*s**2....]` |
2021-06-21 21:33:10 +00:00
| `ROOT_OF_UNITY` | `pow(PRIMITIVE_ROOT_OF_UNITY, (MODULUS - 1) // int(MAX_SAMPLES_PER_BLOB * POINTS_PER_SAMPLE), MODULUS)` |
2020-06-09 20:34:46 +00:00
2020-12-09 07:29:21 +00:00
### Gwei values
| Name | Value | Unit | Description |
| - | - | - | - |
2021-07-26 13:26:55 +00:00
| `MAX_SAMPLE_PRICE` | `Gwei(2**33)` (= 8,589,934,592) | Gwei | Max sample charged for a TARGET-sized shard blob |
| `MIN_SAMPLE_PRICE` | `Gwei(2**3)` (= 8) | Gwei | Min sample price charged for a TARGET-sized shard blob |
2020-06-09 20:34:46 +00:00
2021-05-06 16:28:46 +00:00
## Configuration
2020-04-02 22:48:02 +00:00
2021-08-04 10:44:42 +00:00
Note: Some preset variables may become run-time configurable for testnets, but default to a preset while the spec is unstable.
2021-07-30 20:22:43 +00:00
E.g. `INITIAL_ACTIVE_SHARDS` , `MAX_SAMPLES_PER_BLOB` and `TARGET_SAMPLES_PER_BLOB` .
2019-10-12 03:05:08 +00:00
2019-11-22 22:45:55 +00:00
## Updated containers
2019-10-12 03:05:08 +00:00
2021-03-18 00:06:37 +00:00
The following containers have updated definitions to support Sharding.
2019-11-03 20:06:19 +00:00
2020-12-09 07:29:21 +00:00
### `AttestationData`
2019-10-27 01:01:10 +00:00
2019-10-29 17:43:13 +00:00
```python
2019-11-22 22:45:55 +00:00
class AttestationData(Container):
2019-10-12 03:05:08 +00:00
slot: Slot
2019-10-27 01:01:10 +00:00
index: CommitteeIndex
2019-10-12 03:05:08 +00:00
# LMD GHOST vote
2019-12-05 19:36:48 +00:00
beacon_block_root: Root
2019-10-12 03:05:08 +00:00
# FFG vote
source: Checkpoint
target: Checkpoint
2021-06-23 21:33:46 +00:00
# Hash-tree-root of ShardBlob
2021-06-21 21:33:10 +00:00
shard_blob_root: Root # [New in Sharding]
2019-11-15 20:11:42 +00:00
```
2021-03-18 00:12:55 +00:00
### `BeaconBlockBody`
2020-12-10 03:47:02 +00:00
```python
2021-03-27 00:28:52 +00:00
class BeaconBlockBody(merge.BeaconBlockBody): # [extends The Merge block body]
2021-04-04 00:45:57 +00:00
shard_proposer_slashings: List[ShardProposerSlashing, MAX_SHARD_PROPOSER_SLASHINGS]
2021-06-23 21:33:46 +00:00
shard_headers: List[SignedShardBlobHeader, MAX_SHARDS * MAX_SHARD_HEADERS_PER_SHARD]
2020-12-10 03:47:02 +00:00
```
2020-12-09 07:29:21 +00:00
### `BeaconState`
2019-11-16 10:13:47 +00:00
2019-11-15 20:11:42 +00:00
```python
2021-07-22 14:36:41 +00:00
class BeaconState(merge.BeaconState):
2021-06-23 21:33:46 +00:00
# Blob builder registry.
blob_builders: List[Builder, BLOB_BUILDER_REGISTRY_LIMIT]
blob_builder_balances: List[Gwei, BLOB_BUILDER_REGISTRY_LIMIT]
2021-05-28 19:26:06 +00:00
# A ring buffer of the latest slots, with information per active shard.
2021-05-29 19:28:00 +00:00
shard_buffer: Vector[List[ShardWork, MAX_SHARDS], SHARD_STATE_MEMORY_SLOTS]
2021-07-26 13:26:55 +00:00
shard_sample_price: uint64
2019-10-12 14:59:51 +00:00
```
2019-11-22 22:45:55 +00:00
## New containers
2021-06-23 21:33:46 +00:00
### `Builder`
```python
class Builder(Container):
pubkey: BLSPubkey
# TODO: fields for either an expiry mechanism (refunding execution account with remaining balance)
# and/or a builder-transaction mechanism.
```
2020-12-10 06:42:43 +00:00
### `DataCommitment`
```python
class DataCommitment(Container):
2020-12-28 16:24:31 +00:00
# KZG10 commitment to the data
2020-12-10 06:42:43 +00:00
point: BLSCommitment
# Length of the data in samples
2021-08-04 12:05:21 +00:00
samples_count: uint64
2020-12-10 06:42:43 +00:00
```
2021-07-26 13:26:55 +00:00
### `AttestedDataCommitment`
```python
class AttestedDataCommitment(Container):
# KZG10 commitment to the data, and length
commitment: DataCommitment
# hash_tree_root of the ShardBlobHeader (stored so that attestations can be checked against it)
root: Root
# The proposer who included the shard-header
includer_index: ValidatorIndex
```
2021-08-23 11:21:38 +00:00
### `ShardBlobBody`
2021-06-18 00:21:21 +00:00
Unsigned shard data, bundled by a shard-builder.
Unique, signing different bodies as shard proposer for the same `(slot, shard)` is slashable.
```python
class ShardBlobBody(Container):
# The actual data commitment
commitment: DataCommitment
2021-08-04 12:05:21 +00:00
# Proof that the degree < commitment.samples_count * POINTS_PER_SAMPLE
2021-06-18 00:21:21 +00:00
degree_proof: BLSCommitment
# The actual data. Should match the commitment and degree proof.
2021-06-21 21:33:10 +00:00
data: List[BLSPoint, POINTS_PER_SAMPLE * MAX_SAMPLES_PER_BLOB]
2021-07-26 13:26:55 +00:00
# fee payment fields (EIP 1559 like)
# TODO: express in MWei instead?
max_priority_fee_per_sample: Gwei
max_fee_per_sample: Gwei
2021-06-18 00:21:21 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobBodySummary`
2019-11-22 22:45:55 +00:00
2021-06-18 00:21:21 +00:00
Summary version of the `ShardBlobBody` , omitting the data payload, while preserving the data-commitments.
The commitments are not further collapsed to a single hash,
to avoid an extra network roundtrip between proposer and builder, to include the header on-chain more quickly.
2019-11-22 22:45:55 +00:00
```python
2021-04-04 00:45:57 +00:00
class ShardBlobBodySummary(Container):
2020-12-10 06:42:43 +00:00
# The actual data commitment
commitment: DataCommitment
2021-08-04 12:05:21 +00:00
# Proof that the degree < commitment.samples_count * POINTS_PER_SAMPLE
2021-01-02 21:31:25 +00:00
degree_proof: BLSCommitment
2021-04-04 00:45:57 +00:00
# Hash-tree-root as summary of the data field
data_root: Root
2021-07-26 13:26:55 +00:00
# fee payment fields (EIP 1559 like)
# TODO: express in MWei instead?
max_priority_fee_per_sample: Gwei
max_fee_per_sample: Gwei
2021-06-18 00:21:21 +00:00
```
### `ShardBlob`
`ShardBlobBody` wrapped with the header data that is unique to the shard blob proposal.
```python
class ShardBlob(Container):
slot: Slot
shard: Shard
2021-06-20 22:55:45 +00:00
# Builder of the data, pays data-fee to proposer
builder_index: BuilderIndex
2021-06-23 21:33:46 +00:00
# Proposer of the shard-blob
proposer_index: ValidatorIndex
2021-06-18 00:21:21 +00:00
# Blob contents
body: ShardBlobBody
2020-03-31 04:37:36 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobHeader`
2021-06-20 22:55:45 +00:00
Header version of `ShardBlob` .
2021-06-18 00:21:21 +00:00
2021-04-04 00:45:57 +00:00
```python
class ShardBlobHeader(Container):
slot: Slot
shard: Shard
2021-06-20 22:55:45 +00:00
# Builder of the data, pays data-fee to proposer
builder_index: BuilderIndex
2021-06-23 21:33:46 +00:00
# Proposer of the shard-blob
proposer_index: ValidatorIndex
2021-06-18 00:21:21 +00:00
# Blob contents, without the full data
2021-04-04 00:45:57 +00:00
body_summary: ShardBlobBodySummary
2021-06-18 00:21:21 +00:00
```
### `SignedShardBlob`
2021-06-21 21:33:10 +00:00
Full blob data, signed by the shard builder (ensuring fee payment) and shard proposer (ensuring a single proposal).
2021-06-18 00:21:21 +00:00
```python
class SignedShardBlob(Container):
message: ShardBlob
signature: BLSSignature
2021-04-04 00:45:57 +00:00
```
2021-01-01 20:52:29 +00:00
2021-04-04 00:45:57 +00:00
### `SignedShardBlobHeader`
2021-01-04 18:17:30 +00:00
2021-06-18 00:21:21 +00:00
Header of the blob, the signature is equally applicable to `SignedShardBlob` .
2021-06-21 21:33:10 +00:00
Shard proposers can accept `SignedShardBlobHeader` as a data-transaction by co-signing the header.
2021-06-18 00:21:21 +00:00
2021-01-04 18:17:30 +00:00
```python
2021-04-04 00:45:57 +00:00
class SignedShardBlobHeader(Container):
message: ShardBlobHeader
2021-06-20 22:55:45 +00:00
# Signature by builder.
# Once accepted by proposer, the signatures is the aggregate of both.
2021-06-18 00:21:21 +00:00
signature: BLSSignature
```
2020-12-09 07:29:21 +00:00
### `PendingShardHeader`
2020-03-31 04:37:36 +00:00
```python
2020-12-09 07:29:21 +00:00
class PendingShardHeader(Container):
2021-07-26 13:26:55 +00:00
# The commitment that is attested
attested: AttestedDataCommitment
2020-12-09 07:29:21 +00:00
# Who voted for the header
2020-12-28 16:56:43 +00:00
votes: Bitlist[MAX_VALIDATORS_PER_COMMITTEE]
2021-05-28 19:56:13 +00:00
# Sum of effective balances of votes
weight: Gwei
2021-05-31 16:55:08 +00:00
# When the header was last updated, as reference for weight accuracy
update_slot: Slot
2019-11-22 22:45:55 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardBlobReference`
2021-06-18 00:21:21 +00:00
Reference version of `ShardBlobHeader` , substituting the body for just a hash-tree-root.
2021-04-04 00:45:57 +00:00
```python
class ShardBlobReference(Container):
slot: Slot
shard: Shard
2021-06-20 22:55:45 +00:00
# Builder of the data
builder_index: BuilderIndex
2021-06-23 21:33:46 +00:00
# Proposer of the shard-blob
proposer_index: ValidatorIndex
2021-06-20 22:55:45 +00:00
# Blob hash-tree-root for slashing reference
body_root: Root
2021-06-18 00:21:21 +00:00
```
2021-04-04 00:45:57 +00:00
### `ShardProposerSlashing`
```python
class ShardProposerSlashing(Container):
2021-06-20 22:55:45 +00:00
slot: Slot
shard: Shard
proposer_index: ValidatorIndex
2021-06-21 21:33:10 +00:00
builder_index_1: BuilderIndex
builder_index_2: BuilderIndex
2021-06-20 22:55:45 +00:00
body_root_1: Root
body_root_2: Root
signature_1: BLSSignature
signature_2: BLSSignature
2021-04-04 00:45:57 +00:00
```
2021-05-29 19:28:00 +00:00
### `ShardWork`
2021-05-28 19:26:06 +00:00
```python
class ShardWork(Container):
2021-07-14 11:19:00 +00:00
# Upon confirmation the data is reduced to just the commitment.
2021-06-03 15:32:35 +00:00
status: Union[ # See Shard Work Status enum
None, # SHARD_WORK_UNCONFIRMED
2021-07-27 12:48:21 +00:00
AttestedDataCommitment, # SHARD_WORK_CONFIRMED
2021-06-03 15:32:35 +00:00
List[PendingShardHeader, MAX_SHARD_HEADERS_PER_SHARD] # SHARD_WORK_PENDING
2021-05-28 19:26:06 +00:00
]
```
2019-11-15 20:11:42 +00:00
## Helper functions
### Misc
2020-12-18 20:33:34 +00:00
#### `next_power_of_two`
```python
2021-04-23 15:47:25 +00:00
def next_power_of_two(x: int) -> int:
2020-12-18 20:33:34 +00:00
return 2 ** ((x - 1).bit_length())
```
2020-04-28 03:28:21 +00:00
#### `compute_previous_slot`
2020-01-05 19:11:55 +00:00
```python
2020-04-28 03:28:21 +00:00
def compute_previous_slot(slot: Slot) -> Slot:
2020-01-05 19:11:55 +00:00
if slot > 0:
return Slot(slot - 1)
else:
return Slot(0)
```
2021-07-26 13:26:55 +00:00
#### `compute_updated_sample_price`
2020-04-16 11:43:48 +00:00
```python
2021-08-04 21:25:25 +00:00
def compute_updated_sample_price(prev_price: Gwei, samples_length: uint64, active_shards: uint64) -> Gwei:
2021-07-26 14:27:19 +00:00
adjustment_quotient = active_shards * SLOTS_PER_EPOCH * SAMPLE_PRICE_ADJUSTMENT_COEFFICIENT
2021-08-04 21:25:25 +00:00
if samples_length > TARGET_SAMPLES_PER_BLOB:
delta = max(1, prev_price * (samples_length - TARGET_SAMPLES_PER_BLOB) // TARGET_SAMPLES_PER_BLOB // adjustment_quotient)
2021-07-26 13:26:55 +00:00
return min(prev_price + delta, MAX_SAMPLE_PRICE)
2020-04-16 11:43:48 +00:00
else:
2021-08-04 21:25:25 +00:00
delta = max(1, prev_price * (TARGET_SAMPLES_PER_BLOB - samples_length) // TARGET_SAMPLES_PER_BLOB // adjustment_quotient)
2021-07-26 13:26:55 +00:00
return max(prev_price, MIN_SAMPLE_PRICE + delta) - delta
2020-04-16 11:43:48 +00:00
```
2020-05-29 19:09:42 +00:00
#### `compute_committee_source_epoch`
```python
def compute_committee_source_epoch(epoch: Epoch, period: uint64) -> Epoch:
"""
Return the source epoch for computing the committee.
"""
2020-09-03 11:24:17 +00:00
source_epoch = Epoch(epoch - epoch % period)
2020-05-29 19:09:42 +00:00
if source_epoch >= period:
source_epoch -= period # `period` epochs lookahead
return source_epoch
```
2021-07-26 13:26:55 +00:00
#### `batch_apply_participation_flag`
```python
def batch_apply_participation_flag(state: BeaconState, bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE],
epoch: Epoch, full_committee: Sequence[ValidatorIndex], flag_index: int):
if epoch == get_current_epoch(state):
epoch_participation = state.current_epoch_participation
else:
epoch_participation = state.previous_epoch_participation
for bit, index in zip(bits, full_committee):
if bit:
epoch_participation[index] = add_flag(epoch_participation[index], flag_index)
```
2019-11-15 20:11:42 +00:00
### Beacon state accessors
2020-08-10 19:02:22 +00:00
#### Updated `get_committee_count_per_slot`
2020-08-05 18:44:31 +00:00
```python
def get_committee_count_per_slot(state: BeaconState, epoch: Epoch) -> uint64:
"""
Return the number of committees in each slot for the given ``epoch``.
"""
return max(uint64(1), min(
2020-12-10 03:47:02 +00:00
get_active_shard_count(state, epoch),
2020-08-05 18:44:31 +00:00
uint64(len(get_active_validator_indices(state, epoch))) // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
))
```
2020-01-05 19:20:20 +00:00
#### `get_active_shard_count`
```python
2020-12-10 03:47:02 +00:00
def get_active_shard_count(state: BeaconState, epoch: Epoch) -> uint64:
2020-08-10 19:02:22 +00:00
"""
Return the number of active shards.
Note that this puts an upper bound on the number of committees per slot.
"""
2020-08-05 18:44:31 +00:00
return INITIAL_ACTIVE_SHARDS
2020-01-05 19:20:20 +00:00
```
2019-11-15 20:11:42 +00:00
#### `get_shard_proposer_index`
2019-11-12 14:13:47 +00:00
```python
2021-08-23 17:09:01 +00:00
def get_shard_proposer_index(state: BeaconState, slot: Slot, shard: Shard) -> ValidatorIndex:
2020-06-04 03:19:04 +00:00
"""
Return the proposer's index of shard block at ``slot``.
"""
2020-06-03 21:06:04 +00:00
epoch = compute_epoch_at_slot(slot)
2021-08-23 17:09:01 +00:00
seed = hash(get_seed(state, epoch, DOMAIN_SHARD_BLOB) + uint_to_bytes(slot) + uint_to_bytes(shard))
2021-06-17 21:17:47 +00:00
indices = get_active_validator_indices(state, epoch)
2021-08-23 17:09:01 +00:00
return compute_proposer_index(state, indices, seed)
2020-05-29 15:50:18 +00:00
```
2019-11-18 23:40:02 +00:00
#### `get_start_shard`
```python
def get_start_shard(state: BeaconState, slot: Slot) -> Shard:
2020-05-29 15:50:18 +00:00
"""
Return the start shard at ``slot``.
"""
2021-08-23 17:09:41 +00:00
epoch = compute_epoch_at_slot(Slot(slot))
2021-06-21 17:27:29 +00:00
committee_count = get_committee_count_per_slot(state, epoch)
active_shard_count = get_active_shard_count(state, epoch)
return committee_count * slot % active_shard_count
2019-11-18 23:40:02 +00:00
```
2020-12-31 01:56:29 +00:00
#### `compute_shard_from_committee_index`
```python
def compute_shard_from_committee_index(state: BeaconState, slot: Slot, index: CommitteeIndex) -> Shard:
active_shards = get_active_shard_count(state, compute_epoch_at_slot(slot))
2021-05-28 19:26:06 +00:00
assert index < active_shards
2020-12-31 01:56:29 +00:00
return Shard((index + get_start_shard(state, slot)) % active_shards)
```
#### `compute_committee_index_from_shard`
```python
def compute_committee_index_from_shard(state: BeaconState, slot: Slot, shard: Shard) -> CommitteeIndex:
2021-05-28 19:26:06 +00:00
epoch = compute_epoch_at_slot(slot)
active_shards = get_active_shard_count(state, epoch)
index = CommitteeIndex((active_shards + shard - get_start_shard(state, slot)) % active_shards)
2021-06-01 20:38:36 +00:00
assert index < get_committee_count_per_slot ( state , epoch )
2021-05-28 19:26:06 +00:00
return index
2020-12-31 01:56:29 +00:00
```
2019-11-15 20:11:42 +00:00
### Block processing
2019-11-06 22:19:00 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
2021-09-01 12:42:58 +00:00
# is_execution_enabled is omitted, execution is enabled by default.
process_execution_payload(state, block.body.execution_payload, EXECUTION_ENGINE)
2019-11-15 20:11:42 +00:00
process_randao(state, block.body)
process_eth1_data(state, block.body)
2021-03-29 23:33:17 +00:00
process_operations(state, block.body) # [Modified in Sharding]
2021-07-22 14:36:41 +00:00
process_sync_aggregate(state, block.body.sync_aggregate)
2019-11-12 13:27:34 +00:00
```
2019-11-15 20:11:42 +00:00
#### Operations
2019-10-12 03:05:08 +00:00
2019-10-13 08:42:55 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_operations(state: BeaconState, body: BeaconBlockBody) -> None:
# Verify that outstanding deposits are processed up to the maximum number of deposits
assert len(body.deposits) == min(MAX_DEPOSITS, state.eth1_data.deposit_count - state.eth1_deposit_index)
2020-02-06 18:58:21 +00:00
2019-11-20 03:43:32 +00:00
def for_ops(operations: Sequence[Any], fn: Callable[[BeaconState, Any], None]) -> None:
2019-11-15 20:11:42 +00:00
for operation in operations:
fn(state, operation)
2020-02-06 18:58:21 +00:00
2019-11-16 10:13:47 +00:00
for_ops(body.proposer_slashings, process_proposer_slashing)
for_ops(body.attester_slashings, process_attester_slashing)
2021-04-04 00:45:57 +00:00
# New shard proposer slashing processing
for_ops(body.shard_proposer_slashings, process_shard_proposer_slashing)
2021-07-26 14:27:19 +00:00
# Limit is dynamic: based on active shard count
2021-01-04 18:17:30 +00:00
assert len(body.shard_headers) < = MAX_SHARD_HEADERS_PER_SHARD * get_active_shard_count(state, get_current_epoch(state))
for_ops(body.shard_headers, process_shard_header)
2021-07-26 14:27:19 +00:00
2019-11-15 20:11:42 +00:00
# New attestation processing
2020-02-06 18:58:21 +00:00
for_ops(body.attestations, process_attestation)
2019-11-16 10:13:47 +00:00
for_ops(body.deposits, process_deposit)
for_ops(body.voluntary_exits, process_voluntary_exit)
2021-07-26 14:27:19 +00:00
# TODO: to avoid parallel shards racing, and avoid inclusion-order problems,
# update the fee price per slot, instead of per header.
# state.shard_sample_price = compute_updated_sample_price(state.shard_sample_price, ?, shard_count)
2019-10-12 03:05:08 +00:00
```
2021-05-29 21:55:16 +00:00
##### Extended Attestation processing
2020-05-28 19:20:36 +00:00
```python
def process_attestation(state: BeaconState, attestation: Attestation) -> None:
2021-07-22 14:36:41 +00:00
altair.process_attestation(state, attestation)
2021-07-26 13:26:55 +00:00
process_attested_shard_work(state, attestation)
2020-05-28 19:20:36 +00:00
```
2019-11-03 16:17:46 +00:00
```python
2021-07-26 13:26:55 +00:00
def process_attested_shard_work(state: BeaconState, attestation: Attestation) -> None:
2021-05-05 10:31:19 +00:00
attestation_shard = compute_shard_from_committee_index(
2020-12-09 07:29:21 +00:00
state,
2020-12-31 01:56:29 +00:00
attestation.data.slot,
2020-12-09 07:29:21 +00:00
attestation.data.index,
)
2021-07-26 13:26:55 +00:00
full_committee = get_beacon_committee(state, attestation.data.slot, attestation.data.index)
2021-05-28 18:18:29 +00:00
buffer_index = attestation.data.slot % SHARD_STATE_MEMORY_SLOTS
2021-05-28 19:26:06 +00:00
committee_work = state.shard_buffer[buffer_index][attestation_shard]
2021-05-28 18:18:29 +00:00
2021-05-31 16:55:08 +00:00
# Skip attestation vote accounting if the header is not pending
2021-06-03 15:32:35 +00:00
if committee_work.status.selector != SHARD_WORK_PENDING:
2021-07-26 13:26:55 +00:00
# If the data was already confirmed, check if this matches, to apply the flag to the attesters.
if committee_work.status.selector == SHARD_WORK_CONFIRMED:
2021-07-28 20:00:23 +00:00
attested: AttestedDataCommitment = committee_work.status.value
2021-07-26 13:26:55 +00:00
if attested.root == attestation.data.shard_blob_root:
batch_apply_participation_flag(state, attestation.aggregation_bits,
attestation.data.target.epoch,
full_committee, TIMELY_SHARD_FLAG_INDEX)
2021-01-04 18:17:30 +00:00
return
2021-05-28 19:26:06 +00:00
current_headers: Sequence[PendingShardHeader] = committee_work.status.value
2021-05-28 18:18:29 +00:00
# Find the corresponding header, abort if it cannot be found
2021-07-14 11:19:00 +00:00
header_index = len(current_headers)
for i, header in enumerate(current_headers):
2021-07-26 13:26:55 +00:00
if attestation.data.shard_blob_root == header.attested.root:
2021-07-14 11:19:00 +00:00
header_index = i
break
2021-07-26 13:26:55 +00:00
2021-07-14 11:19:00 +00:00
# Attestations for an unknown header do not count towards shard confirmations, but can otherwise be valid.
if header_index == len(current_headers):
2021-07-26 13:26:55 +00:00
# Note: Attestations may be re-included if headers are included late.
2021-07-14 11:19:00 +00:00
return
2021-05-28 18:18:29 +00:00
2021-05-31 15:03:06 +00:00
pending_header: PendingShardHeader = current_headers[header_index]
2021-05-31 16:55:08 +00:00
# The weight may be outdated if it is not the initial weight, and from a previous epoch
if pending_header.weight != 0 and compute_epoch_at_slot(pending_header.update_slot) < get_current_epoch ( state ) :
pending_header.weight = sum(state.validators[index].effective_balance for index, bit
in zip(full_committee, pending_header.votes) if bit)
pending_header.update_slot = state.slot
2021-06-03 15:32:35 +00:00
full_committee_balance = Gwei(0)
2021-05-31 16:55:08 +00:00
# Update votes bitfield in the state, update weights
2021-05-28 18:18:29 +00:00
for i, bit in enumerate(attestation.aggregation_bits):
2021-05-28 19:56:13 +00:00
weight = state.validators[full_committee[i]].effective_balance
2021-05-31 16:55:08 +00:00
full_committee_balance += weight
2021-05-28 18:18:29 +00:00
if bit:
2021-05-28 19:56:13 +00:00
if not pending_header.votes[i]:
pending_header.weight += weight
2021-05-31 16:55:08 +00:00
pending_header.votes[i] = True
2021-05-28 18:18:29 +00:00
# Check if the PendingShardHeader is eligible for expedited confirmation, requiring 2/3 of balance attesting
2021-05-31 16:55:08 +00:00
if pending_header.weight * 3 >= full_committee_balance * 2:
2021-07-26 13:26:55 +00:00
# participants of the winning header are remembered with participation flags
batch_apply_participation_flag(state, pending_header.votes, attestation.data.target.epoch,
full_committee, TIMELY_SHARD_FLAG_INDEX)
if pending_header.attested.commitment == DataCommitment():
2021-05-28 19:26:06 +00:00
# The committee voted to not confirm anything
2021-06-16 09:03:20 +00:00
state.shard_buffer[buffer_index][attestation_shard].status.change(
2021-06-03 15:32:35 +00:00
selector=SHARD_WORK_UNCONFIRMED,
2021-05-28 19:26:06 +00:00
value=None,
)
else:
2021-06-16 09:03:20 +00:00
state.shard_buffer[buffer_index][attestation_shard].status.change(
2021-06-03 15:32:35 +00:00
selector=SHARD_WORK_CONFIRMED,
2021-07-26 13:26:55 +00:00
value=pending_header.attested,
2021-05-28 19:26:06 +00:00
)
2020-12-09 07:29:21 +00:00
```
2021-05-29 21:55:16 +00:00
##### `process_shard_header`
2020-12-09 07:29:21 +00:00
```python
2021-06-20 22:55:45 +00:00
def process_shard_header(state: BeaconState, signed_header: SignedShardBlobHeader) -> None:
header: ShardBlobHeader = signed_header.message
slot = header.slot
shard = header.shard
2021-06-18 00:21:21 +00:00
2021-04-06 15:07:29 +00:00
# Verify the header is not 0, and not from the future.
2021-06-18 00:21:21 +00:00
assert Slot(0) < slot < = state . slot
header_epoch = compute_epoch_at_slot(slot)
2021-04-04 00:45:57 +00:00
# Verify that the header is within the processing time window
assert header_epoch in [get_previous_epoch(state), get_current_epoch(state)]
2021-07-26 14:27:19 +00:00
# Verify that the shard is valid
shard_count = get_active_shard_count(state, header_epoch)
assert shard < shard_count
# Verify that a committee is able to attest this (slot, shard)
2021-07-26 13:26:55 +00:00
start_shard = get_start_shard(state, slot)
2021-07-26 14:27:19 +00:00
committee_index = (shard_count + shard - start_shard) % shard_count
2021-07-26 13:26:55 +00:00
committees_per_slot = get_committee_count_per_slot(state, header_epoch)
2021-07-26 14:27:19 +00:00
assert committee_index < = committees_per_slot
2021-05-28 18:18:29 +00:00
# Check that this data is still pending
2021-06-18 00:21:21 +00:00
committee_work = state.shard_buffer[slot % SHARD_STATE_MEMORY_SLOTS][shard]
2021-06-03 15:32:35 +00:00
assert committee_work.status.selector == SHARD_WORK_PENDING
2021-05-28 18:18:29 +00:00
# Check that this header is not yet in the pending list
2021-06-16 11:06:17 +00:00
current_headers: List[PendingShardHeader, MAX_SHARD_HEADERS_PER_SHARD] = committee_work.status.value
2021-06-20 22:55:45 +00:00
header_root = hash_tree_root(header)
2021-07-26 13:26:55 +00:00
assert header_root not in [pending_header.attested.root for pending_header in current_headers]
2021-05-28 18:18:29 +00:00
2021-06-20 22:55:45 +00:00
# Verify proposer matches
assert header.proposer_index == get_shard_proposer_index(state, slot, shard)
2021-06-18 00:21:21 +00:00
2021-06-20 22:55:45 +00:00
# Verify builder and proposer aggregate signature
blob_signing_root = compute_signing_root(header, get_domain(state, DOMAIN_SHARD_BLOB))
2021-06-23 21:33:46 +00:00
builder_pubkey = state.blob_builders[header.builder_index].pubkey
2021-06-20 22:55:45 +00:00
proposer_pubkey = state.validators[header.proposer_index].pubkey
assert bls.FastAggregateVerify([builder_pubkey, proposer_pubkey], blob_signing_root, signed_header.signature)
2021-01-04 18:17:30 +00:00
2020-12-28 17:38:09 +00:00
# Verify the length by verifying the degree.
2021-04-04 00:45:57 +00:00
body_summary = header.body_summary
2021-08-04 12:05:21 +00:00
points_count = body_summary.commitment.samples_count * POINTS_PER_SAMPLE
if points_count == 0:
2021-04-04 00:45:57 +00:00
assert body_summary.degree_proof == G1_SETUP[0]
2019-11-18 22:07:50 +00:00
assert (
2021-04-04 00:45:57 +00:00
bls.Pairing(body_summary.degree_proof, G2_SETUP[0])
2021-08-04 12:05:21 +00:00
== bls.Pairing(body_summary.commitment.point, G2_SETUP[-points_count])
2019-11-18 22:07:50 +00:00
)
2021-01-04 18:17:30 +00:00
2021-07-26 13:26:55 +00:00
# Charge EIP 1559 fee, builder pays for opportunity, and is responsible for later availability,
# or fail to publish at their own expense.
2021-08-04 12:05:21 +00:00
samples = body_summary.commitment.samples_count
2021-07-26 13:26:55 +00:00
# TODO: overflows, need bigger int type
2021-07-30 19:54:55 +00:00
max_fee = body_summary.max_fee_per_sample * samples
2021-07-26 13:26:55 +00:00
# Builder must have sufficient balance, even if max_fee is not completely utilized
2021-07-27 12:48:21 +00:00
assert state.blob_builder_balances[header.builder_index] >= max_fee
2021-07-26 13:26:55 +00:00
base_fee = state.shard_sample_price * samples
# Base fee must be paid
assert max_fee >= base_fee
# Remaining fee goes towards proposer for prioritizing, up to a maximum
2021-07-30 19:54:55 +00:00
max_priority_fee = body_summary.max_priority_fee_per_sample * samples
2021-07-26 13:26:55 +00:00
priority_fee = min(max_fee - base_fee, max_priority_fee)
# Burn base fee, take priority fee
2021-07-27 12:48:21 +00:00
# priority_fee < = max_fee - base_fee, thus priority_fee + base_fee < = max_fee, thus sufficient balance.
state.blob_builder_balances[header.builder_index] -= base_fee + priority_fee
2021-07-26 13:26:55 +00:00
# Pay out priority fee
increase_balance(state, header.proposer_index, priority_fee)
2021-05-28 18:18:29 +00:00
# Initialize the pending header
2021-06-18 00:21:21 +00:00
index = compute_committee_index_from_shard(state, slot, shard)
committee_length = len(get_beacon_committee(state, slot, index))
2021-05-28 18:18:29 +00:00
initial_votes = Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length)
pending_header = PendingShardHeader(
2021-07-26 13:26:55 +00:00
attested=AttestedDataCommitment(
2021-07-30 19:54:55 +00:00
commitment=body_summary.commitment,
2021-07-26 13:26:55 +00:00
root=header_root,
includer_index=get_beacon_proposer_index(state),
2021-08-23 11:21:38 +00:00
),
2021-05-28 18:18:29 +00:00
votes=initial_votes,
2021-05-28 19:56:13 +00:00
weight=0,
2021-05-31 16:55:08 +00:00
update_slot=state.slot,
2021-05-28 18:18:29 +00:00
)
# Include it in the pending list
2021-06-16 10:33:56 +00:00
current_headers.append(pending_header)
2020-02-06 18:58:21 +00:00
```
2020-12-09 07:29:21 +00:00
The degree proof works as follows. For a block `B` with length `l` (so `l` values in `[0...l - 1]` , seen as a polynomial `B(X)` which takes these values),
the length proof is the commitment to the polynomial `B(X) * X**(MAX_DEGREE + 1 - l)` ,
where `MAX_DEGREE` is the maximum power of `s` available in the setup, which is `MAX_DEGREE = len(G2_SETUP) - 1` .
The goal is to ensure that a proof can only be constructed if `deg(B) < l` (there are not hidden higher-order terms in the polynomial, which would thwart reconstruction).
2020-02-06 18:58:21 +00:00
2021-05-29 21:55:16 +00:00
##### `process_shard_proposer_slashing`
2021-04-04 00:45:57 +00:00
```python
def process_shard_proposer_slashing(state: BeaconState, proposer_slashing: ShardProposerSlashing) -> None:
2021-06-20 22:55:45 +00:00
slot = proposer_slashing.slot
shard = proposer_slashing.shard
proposer_index = proposer_slashing.proposer_index
reference_1 = ShardBlobReference(slot=slot, shard=shard,
2021-06-21 21:33:10 +00:00
proposer_index=proposer_index,
builder_index=proposer_slashing.builder_index_1,
2021-06-23 21:33:46 +00:00
body_root=proposer_slashing.body_root_1)
2021-06-20 22:55:45 +00:00
reference_2 = ShardBlobReference(slot=slot, shard=shard,
2021-06-21 21:33:10 +00:00
proposer_index=proposer_index,
2021-06-23 21:33:46 +00:00
builder_index=proposer_slashing.builder_index_2,
body_root=proposer_slashing.body_root_2)
# Verify the signed messages are different
assert reference_1 != reference_2
# Verify the proposer is slashable
proposer = state.validators[proposer_index]
assert is_slashable_validator(proposer, get_current_epoch(state))
2021-06-21 21:33:10 +00:00
# The builders are not slashed, the proposer co-signed with them
2021-06-23 21:33:46 +00:00
builder_pubkey_1 = state.blob_builders[proposer_slashing.builder_index_1].pubkey
builder_pubkey_2 = state.blob_builders[proposer_slashing.builder_index_2].pubkey
2021-06-20 22:55:45 +00:00
domain = get_domain(state, DOMAIN_SHARD_PROPOSER, compute_epoch_at_slot(slot))
signing_root_1 = compute_signing_root(reference_1, domain)
signing_root_2 = compute_signing_root(reference_2, domain)
2021-06-21 21:33:10 +00:00
assert bls.FastAggregateVerify([builder_pubkey_1, proposer.pubkey], signing_root_1, proposer_slashing.signature_1)
assert bls.FastAggregateVerify([builder_pubkey_2, proposer.pubkey], signing_root_2, proposer_slashing.signature_2)
2021-06-20 22:55:45 +00:00
slash_validator(state, proposer_index)
2021-04-04 00:45:57 +00:00
```
2019-10-12 14:59:51 +00:00
### Epoch transition
2021-03-27 00:28:52 +00:00
This epoch transition overrides the Merge epoch transition:
2019-11-15 20:11:42 +00:00
2019-10-12 14:59:51 +00:00
```python
2019-11-15 20:11:42 +00:00
def process_epoch(state: BeaconState) -> None:
2021-07-22 14:36:41 +00:00
# Sharding pre-processing
2021-05-29 21:55:16 +00:00
process_pending_shard_confirmations(state)
reset_pending_shard_work(state)
2020-12-09 07:29:21 +00:00
2021-07-22 14:36:41 +00:00
# Base functionality
2019-11-15 20:11:42 +00:00
process_justification_and_finalization(state)
2021-07-22 14:36:41 +00:00
process_inactivity_updates(state)
2021-07-26 13:26:55 +00:00
process_rewards_and_penalties(state) # Note: modified, see new TIMELY_SHARD_FLAG_INDEX
2019-11-15 20:11:42 +00:00
process_registry_updates(state)
process_slashings(state)
2021-01-27 06:42:50 +00:00
process_eth1_data_reset(state)
process_effective_balance_updates(state)
process_slashings_reset(state)
process_randao_mixes_reset(state)
process_historical_roots_update(state)
2021-07-22 14:36:41 +00:00
process_participation_flag_updates(state)
process_sync_committee_updates(state)
2019-11-15 22:42:28 +00:00
```
2019-11-15 20:11:42 +00:00
2021-05-29 21:55:16 +00:00
#### `process_pending_shard_confirmations`
2020-12-09 07:29:21 +00:00
```python
2021-05-29 21:55:16 +00:00
def process_pending_shard_confirmations(state: BeaconState) -> None:
2021-01-04 18:17:30 +00:00
# Pending header processing applies to the previous epoch.
# Skip if `GENESIS_EPOCH` because no prior epoch to process.
if get_current_epoch(state) == GENESIS_EPOCH:
return
2021-04-22 17:38:21 +00:00
previous_epoch = get_previous_epoch(state)
previous_epoch_start_slot = compute_start_slot_at_epoch(previous_epoch)
2021-05-28 18:18:29 +00:00
# Mark stale headers as unconfirmed
2021-01-04 18:17:30 +00:00
for slot in range(previous_epoch_start_slot, previous_epoch_start_slot + SLOTS_PER_EPOCH):
2021-05-28 18:18:29 +00:00
buffer_index = slot % SHARD_STATE_MEMORY_SLOTS
2021-05-28 19:26:06 +00:00
for shard_index in range(len(state.shard_buffer[buffer_index])):
2021-05-28 19:56:13 +00:00
committee_work = state.shard_buffer[buffer_index][shard_index]
2021-06-03 17:11:47 +00:00
if committee_work.status.selector == SHARD_WORK_PENDING:
winning_header = max(committee_work.status.value, key=lambda header: header.weight)
2021-07-26 13:26:55 +00:00
if winning_header.attested.commitment == DataCommitment():
2021-06-03 17:11:47 +00:00
committee_work.status.change(selector=SHARD_WORK_UNCONFIRMED, value=None)
2021-05-28 19:56:13 +00:00
else:
2021-07-26 13:26:55 +00:00
committee_work.status.change(selector=SHARD_WORK_CONFIRMED, value=winning_header.attested)
2020-12-09 07:29:21 +00:00
```
2021-05-29 21:55:16 +00:00
#### `reset_pending_shard_work`
2020-12-09 07:29:21 +00:00
```python
2021-05-29 21:55:16 +00:00
def reset_pending_shard_work(state: BeaconState) -> None:
2021-05-28 19:26:06 +00:00
# Add dummy "empty" PendingShardHeader (default vote if no shard header is available)
2021-01-04 18:17:30 +00:00
next_epoch = get_current_epoch(state) + 1
next_epoch_start_slot = compute_start_slot_at_epoch(next_epoch)
2021-05-14 19:02:46 +00:00
committees_per_slot = get_committee_count_per_slot(state, next_epoch)
2021-05-28 18:18:29 +00:00
active_shards = get_active_shard_count(state, next_epoch)
2021-05-28 19:26:06 +00:00
2021-04-24 14:33:53 +00:00
for slot in range(next_epoch_start_slot, next_epoch_start_slot + SLOTS_PER_EPOCH):
2021-05-28 18:18:29 +00:00
buffer_index = slot % SHARD_STATE_MEMORY_SLOTS
2021-05-28 19:26:06 +00:00
# Reset the shard work tracking
2021-05-29 19:28:00 +00:00
state.shard_buffer[buffer_index] = [ShardWork() for _ in range(active_shards)]
2021-05-28 18:18:29 +00:00
start_shard = get_start_shard(state, slot)
2021-05-31 16:55:08 +00:00
for committee_index in range(committees_per_slot):
shard = (start_shard + committee_index) % active_shards
# a committee is available, initialize a pending shard-header list
2021-06-16 16:07:24 +00:00
committee_length = len(get_beacon_committee(state, slot, CommitteeIndex(committee_index)))
2021-06-16 09:03:20 +00:00
state.shard_buffer[buffer_index][shard].status.change(
2021-06-03 15:32:35 +00:00
selector=SHARD_WORK_PENDING,
2021-05-31 16:55:08 +00:00
value=List[PendingShardHeader, MAX_SHARD_HEADERS_PER_SHARD](
PendingShardHeader(
2021-08-23 11:21:38 +00:00
attested=AttestedDataCommitment(),
2021-05-31 16:55:08 +00:00
votes=Bitlist[MAX_VALIDATORS_PER_COMMITTEE]([0] * committee_length),
weight=0,
update_slot=slot,
2021-05-28 18:18:29 +00:00
)
)
2021-05-31 16:55:08 +00:00
)
2021-06-03 15:32:35 +00:00
# a shard without committee available defaults to SHARD_WORK_UNCONFIRMED.
2019-10-12 14:59:51 +00:00
```