eth2.0-specs/specs/_features/sharding/beacon-chain.md

# Sharding -- The Beacon Chain

**Notice**: This document is a work-in-progress for researchers and implementers.

## Table of contents

<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Introduction](#introduction)
  - [Glossary](#glossary)
- [Constants](#constants)
  - [Misc](#misc)
  - [Domain types](#domain-types)
- [Preset](#preset)
  - [Misc](#misc-1)
  - [Time parameters](#time-parameters)
  - [Shard blob samples](#shard-blob-samples)
- [Configuration](#configuration)
  - [Time parameters](#time-parameters-1)
- [Containers](#containers)
  - [New Containers](#new-containers)
    - [`BuilderBlockBid`](#builderblockbid)
    - [`BuilderBlockBidWithRecipientAddress`](#builderblockbidwithrecipientaddress)
    - [`ShardedCommitmentsContainer`](#shardedcommitmentscontainer)
    - [`ShardSample`](#shardsample)
  - [Extended Containers](#extended-containers)
    - [`BeaconState`](#beaconstate)
    - [`BuilderBlockData`](#builderblockdata)
    - [`BeaconBlockBody`](#beaconblockbody)
- [Helper functions](#helper-functions)
  - [Block processing](#block-processing)
    - [`is_builder_block_slot`](#is_builder_block_slot)
  - [Beacon state accessors](#beacon-state-accessors)
    - [`get_active_shard_count`](#get_active_shard_count)
- [Beacon chain state transition function](#beacon-chain-state-transition-function)
  - [Block processing](#block-processing-1)
    - [`process_block`](#process_block)
    - [Block header](#block-header)
    - [Builder Block Bid](#builder-block-bid)
    - [Sharded data](#sharded-data)
    - [Execution payload](#execution-payload)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->


## Introduction

This document describes the extensions made to the Phase 0 design of The Beacon Chain to support data sharding,
based on the ideas [here](https://notes.ethereum.org/@dankrad/new_sharding) and more broadly [here](https://arxiv.org/abs/1809.09044),
using KZG10 commitments to commit to data to remove any need for fraud proofs (and hence, safety-critical synchrony assumptions) in the design.

### Glossary

- **Data**: A list of KZG points, to translate a byte string into
- **Blob**: Data with commitments and meta-data, like a flattened bundle of L2 transactions.


## Constants

The following values are (non-configurable) constants used throughout the specification.

### Misc

| Name | Value | Notes |
| - | - | - |
| `FIELD_ELEMENTS_PER_SAMPLE` | `uint64(2**4)` (= 16) | 31 * 16 = 496 bytes |

### Domain types

| Name | Value |
| - | - |
| `DOMAIN_SHARD_SAMPLE`     | `DomainType('0x10000000')` |

## Preset

### Misc

| Name | Value | Notes |
| - | - | - |
| `MAX_SHARDS` | `uint64(2**12)` (= 4,096) | Theoretical max shard count (used to determine data structure sizes) |
| `ACTIVE_SHARDS` | `uint64(2**8)` (= 256) | Initial shard count |
| `MAX_PROPOSER_BLOCKS_BETWEEN_BUILDER_BLOCKS` | `uint64(2**4)` (= 16) | TODO: Need to define what happens if there were more blocks without builder blocks | 

### Time parameters

With the introduction of builder blocks the number of slots per epoch is doubled (it counts beacon blocks and builder blocks).

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `SLOTS_PER_EPOCH` | `uint64(2**6)` (= 64) | slots | 8:32 minutes |

### Shard blob samples

| Name | Value | Notes |
| - | - | - |
| `SAMPLES_PER_BLOB` | `uint64(2**9)` (= 512) | 248 * 512 = 126,976 bytes |

## Configuration

Note: Some preset variables may become run-time configurable for testnets, but default to a preset while the spec is unstable.  
E.g. `ACTIVE_SHARDS` and `SAMPLES_PER_BLOB`.

### Time parameters

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `SECONDS_PER_SLOT` | `uint64(8)` | seconds | 8 seconds |

## Containers

### New Containers

#### `BuilderBlockBid`

```python
class BuilderBlockBid(Container):
    slot: Slot
    parent_block_root: Root

    execution_payload_root: Root

    sharded_data_commitment_root: Root # Root of the sharded data (only data, not beacon/builder block commitments)

    sharded_data_commitment_count: uint64 # Count of sharded data commitments

    bid: Gwei # Block builder bid paid to proposer

    validator_index: ValidatorIndex # Validator index for this bid
    
    # Block builders use an Eth1 address -- need signature as
    # block bid and data gas base fees will be charged to this address
    signature_y_parity: bool
    signature_r: uint256
    signature_s: uint256    
```

#### `BuilderBlockBidWithRecipientAddress`

```python
class BuilderBlockBidWithRecipientAddress(Container):
    builder_block_bid: Union[None, BuilderBlockBid]
    recipient_address: ExecutionAddress # Address to receive the block builder bid
```

#### `ShardedCommitmentsContainer`

```python
class ShardedCommitmentsContainer(Container):
    sharded_commitments: List[KZGCommitment, 2 * MAX_SHARDS]

    # Aggregate degree proof for all sharded_commitments
    degree_proof: KZGCommitment

    # The sizes of the blocks encoded in the commitments (last builder and all beacon blocks since)
    included_block_sizes: List[uint64, MAX_PROPOSER_BLOCKS_BETWEEN_BUILDER_BLOCKS + 1]
    
    # Number of commitments that are for sharded data (no blocks)
    included_sharded_data_commitments: uint64

    # Random evaluation of beacon blocks + execution payload (this helps with quick verification)
    block_verification_kzg_proof: KZGCommitment
```

#### `ShardSample`

```python
class ShardSample(Container):
    slot: Slot
    row: uint64
    column: uint64
    data: Vector[BLSFieldElement, FIELD_ELEMENTS_PER_SAMPLE]
    proof: KZGCommitment
    builder: ValidatorIndex
    signature: BLSSignature
```

### Extended Containers

#### `BeaconState`

```python
class BeaconState(bellatrix.BeaconState):
    blocks_since_builder_block: List[BeaconBlock, MAX_PROPOSER_BLOCKS_BETWEEN_BUILDER_BLOCKS]
```

#### `BuilderBlockData`

```python
class BuilderBlockData(Container):
    execution_payload: ExecutionPayload
    sharded_commitments_container: ShardedCommitmentsContainer
```    

#### `BeaconBlockBody`

```python
class BeaconBlockBody(altair.BeaconBlockBody):
    payload_data: Union[BuilderBlockBid, BuilderBlockData]
```

## Helper functions

### Block processing

#### `is_builder_block_slot`

```python
def is_builder_block_slot(slot: Slot) -> bool:
    return slot % 2 == 1
```

### Beacon state accessors

#### `get_active_shard_count`

```python
def get_active_shard_count(state: BeaconState, epoch: Epoch) -> uint64:
    """
    Return the number of active shards.
    Note that this puts an upper bound on the number of committees per slot.
    """
    return ACTIVE_SHARDS
```

## Beacon chain state transition function

### Block processing

#### `process_block`

```python
def process_block(state: BeaconState, block: BeaconBlock) -> None:
    process_block_header(state, block)
    verify_builder_block_bid(state, block)
    process_sharded_data(state, block)
    if is_execution_enabled(state, block.body):
        process_execution_payload(state, block, EXECUTION_ENGINE)

    if not is_builder_block_slot(block.slot):
        process_randao(state, block.body)

    process_eth1_data(state, block.body)
    process_operations(state, block.body)
    process_sync_aggregate(state, block.body.sync_aggregate)

    if is_builder_block_slot(block.slot):
        state.blocks_since_builder_block = []
    state.blocks_since_builder_block.append(block)
```

#### Block header

```python
def process_block_header(state: BeaconState, block: BeaconBlock) -> None:
    # Verify that the slots match
    assert block.slot == state.slot
    # Verify that the block is newer than latest block header
    assert block.slot > state.latest_block_header.slot
    # Verify that proposer index is the correct index
    if not is_builder_block_slot(block.slot):
        assert block.proposer_index == get_beacon_proposer_index(state)
    # Verify that the parent matches
    assert block.parent_root == hash_tree_root(state.latest_block_header)
    # Cache current block as the new latest block
    state.latest_block_header = BeaconBlockHeader(
        slot=block.slot,
        proposer_index=block.proposer_index,
        parent_root=block.parent_root,
        state_root=Bytes32(),  # Overwritten in the next process_slot call
        body_root=hash_tree_root(block.body),
    )

    # Verify proposer is not slashed
    proposer = state.validators[block.proposer_index]
    assert not proposer.slashed
```

#### Builder Block Bid

```python
def verify_builder_block_bid(state: BeaconState, block: BeaconBlock) -> None:
    if is_builder_block_slot(block.slot):
        # Get last builder block bid
        assert state.blocks_since_builder_block[-1].body.payload_data.selector == 0
        builder_block_bid = state.blocks_since_builder_block[-1].body.payload_data.value.builder_block_bid
        assert builder_block_bid.slot + 1 == block.slot

        assert block.body.payload_data.selector == 1 # Verify that builder block does not contain bid

        builder_block_data = block.body.payload_data.value

        assert builder_block_bid.execution_payload_root == hash_tree_root(builder_block_data.execution_payload)

        assert builder_block_bid.sharded_data_commitment_count == builder_block_data.included_sharded_data_commitments

        assert builder_block_bid.sharded_data_commitment_root == hash_tree_root(builder_block_data.sharded_commitments[-builder_block_bid.included_sharded_data_commitments:])

        assert builder_block_bid.validator_index == block.proposer_index

    else:
        assert block.body.payload_data.selector == 0

        builder_block_bid = block.body.payload_data.value.builder_block_bid
        assert builder_block_bid.slot == block.slot
        assert builder_block_bid.parent_block_root == block.parent_root
        # We do not check that the builder address exists or has sufficient balance here.
        # If it does not have sufficient balance, the block proposer loses out, so it is their
        # responsibility to check.

        # Check that the builder is a slashable validator. We can probably reduce this requirement and only
        # ensure that they have 1 ETH in their account as a DOS protection.
        builder = state.validators[builder_block_bid.validator_index]
        assert is_slashable_validator(builder, get_current_epoch(state))
```

#### Sharded data

```python
def process_sharded_data(state: BeaconState, block: BeaconBlock) -> None:
    if is_builder_block_slot(block.slot):
        assert block.body.payload_data.selector == 1
        sharded_commitments_container = block.body.payload_data.value.sharded_commitments_container

        # Verify not too many commitments
        assert len(sharded_commitments_container.sharded_commitments) // 2 <= get_active_shard_count(state, get_current_epoch(state))

        # Verify the degree proof
        r = hash_to_bls_field(sharded_commitments_container.sharded_commitments, 0)
        r_powers = compute_powers(r, len(sharded_commitments_container.sharded_commitments))
        combined_commitment = elliptic_curve_lincomb(sharded_commitments_container.sharded_commitments, r_powers)

        payload_field_elements_per_blob = SAMPLES_PER_BLOB * FIELD_ELEMENTS_PER_SAMPLE // 2

        verify_degree_proof(combined_commitment, payload_field_elements_per_blob, sharded_commitments_container.degree_proof)

        # Verify that the 2*N commitments lie on a degree < N polynomial
        low_degree_check(sharded_commitments_container.sharded_commitments)

        # Verify that blocks since the last builder block have been included
        blocks_chunked = [bytes_to_field_elements(ssz_serialize(block)) for block in state.blocks_since_builder_block]
        block_vectors = []

        for block_chunked in blocks_chunked:
            for i in range(0, len(block_chunked), payload_field_elements_per_blob):
                block_vectors.append(block_chunked[i:i + payload_field_elements_per_blob])

        number_of_blobs = len(block_vectors)
        r = hash_to_bls_field(sharded_commitments_container.sharded_commitments[:number_of_blobs], 0)
        x = hash_to_bls_field(sharded_commitments_container.sharded_commitments[:number_of_blobs], 1)

        r_powers = compute_powers(r, number_of_blobs)
        combined_vector = vector_lincomb(block_vectors, r_powers)
        combined_commitment = elliptic_curve_lincomb(sharded_commitments_container.sharded_commitments[:number_of_blobs], r_powers)
        y = evaluate_polynomial_in_evaluation_form(combined_vector, x)

        verify_kzg_proof(combined_commitment, x, y, sharded_commitments_container.block_verification_kzg_proof)

        # Verify that number of sharded data commitments is correctly indicated
        assert 2 * (number_of_blobs + included_sharded_data_commitments) == len(sharded_commitments_container.sharded_commitments)
```

#### Execution payload

```python
def process_execution_payload(state: BeaconState, block: BeaconBlock, execution_engine: ExecutionEngine) -> None:
    if is_builder_block_slot(block.slot):
        assert block.body.payload_data.selector == 1
        payload = block.body.payload_data.value.execution_payload

        # Verify consistency of the parent hash with respect to the previous execution payload header
        if is_merge_transition_complete(state):
            assert payload.parent_hash == state.latest_execution_payload_header.block_hash
        # Verify random
        assert payload.random == get_randao_mix(state, get_current_epoch(state))
        # Verify timestamp
        assert payload.timestamp == compute_timestamp_at_slot(state, state.slot)

        # Get sharded data commitments
        sharded_commitments_container = block.body.sharded_commitments_container
        sharded_data_commitments = sharded_commitments_container.sharded_commitments[-sharded_commitments_container.included_sharded_data_commitments:]

        # Get all unprocessed builder block bids
        unprocessed_builder_block_bid_with_recipient_addresses = []
        for block in state.blocks_since_builder_block[1:]:
            unprocessed_builder_block_bid_with_recipient_addresses.append(block.body.builder_block_bid_with_recipient_address.value)

        # Verify the execution payload is valid
        # The execution engine gets two extra payloads: One for the sharded data commitments (these are needed to verify type 3 transactions)
        # and one for all so far unprocessed builder block bids:
        # * The execution engine needs to transfer the balance from the bidder to the proposer.
        # * The execution engine needs to deduct data gas fees from the bidder balances
        assert execution_engine.execute_payload(payload,
                                                sharded_data_commitments,
                                                unprocessed_builder_block_bid_with_recipient_addresses)

        # Cache execution payload header
        state.latest_execution_payload_header = ExecutionPayloadHeader(
            parent_hash=payload.parent_hash,
            fee_recipient=payload.fee_recipient,
            state_root=payload.state_root,
            receipt_root=payload.receipt_root,
            logs_bloom=payload.logs_bloom,
            random=payload.random,
            block_number=payload.block_number,
            gas_limit=payload.gas_limit,
            gas_used=payload.gas_used,
            timestamp=payload.timestamp,
            extra_data=payload.extra_data,
            base_fee_per_gas=payload.base_fee_per_gas,
            block_hash=payload.block_hash,
            transactions_root=hash_tree_root(payload.transactions),
        )
```