eth2.0-specs/specs/eip4844/beacon-chain.md

304 lines
11 KiB
Markdown
Raw Normal View History

2022-03-10 05:31:11 +00:00
# EIP-4844 -- The Beacon Chain
**Notice**: This document is a work-in-progress for researchers and implementers.
## Table of contents
<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
- [Introduction](#introduction)
- [Custom types](#custom-types)
- [Constants](#constants)
- [Blob](#blob)
- [Preset](#preset)
- [Execution](#execution)
2022-03-10 05:31:11 +00:00
- [Configuration](#configuration)
- [Containers](#containers)
- [Extended containers](#extended-containers)
- [`BeaconBlockBody`](#beaconblockbody)
- [`ExecutionPayload`](#executionpayload)
- [`ExecutionPayloadHeader`](#executionpayloadheader)
2022-03-10 05:31:11 +00:00
- [Helper functions](#helper-functions)
- [Misc](#misc)
- [`kzg_commitment_to_versioned_hash`](#kzg_commitment_to_versioned_hash)
2022-03-10 05:31:11 +00:00
- [`tx_peek_blob_versioned_hashes`](#tx_peek_blob_versioned_hashes)
- [`verify_kzg_commitments_against_transactions`](#verify_kzg_commitments_against_transactions)
2022-03-10 05:31:11 +00:00
- [Beacon chain state transition function](#beacon-chain-state-transition-function)
- [Block processing](#block-processing)
- [Execution payload](#execution-payload)
- [`process_execution_payload`](#process_execution_payload)
- [Blob KZG commitments](#blob-kzg-commitments)
2022-03-10 05:31:11 +00:00
- [Testing](#testing)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->
## Introduction
This upgrade adds blobs to the beacon chain as part of EIP-4844. This is an extension of the Capella upgrade.
2022-03-10 05:31:11 +00:00
## Custom types
| Name | SSZ equivalent | Description |
| - | - | - |
| `VersionedHash` | `Bytes32` | |
Free the blobs This PR reintroduces and further decouples blocks and blobs in EIP-4844, so as to improve network and processing performance. Block and blob processing, for the purpose of gossip validation, are independent: they can both be propagated and gossip-validated in parallel - the decoupled design allows 4 important optimizations (or, if you are so inclined, removes 4 unnecessary pessimizations): * Blocks and blobs travel on independent meshes allowing for better parallelization and utilization of high-bandwidth peers * Re-broadcasting after validation can start earlier allowing more efficient use of upload bandwidth - blocks for example can be rebroadcast to peers while blobs are still being downloaded * bandwidth-reduction techniques such as per-peer deduplication are more efficient because of the smaller message size * gossip verification happens independently for blocks and blobs, allowing better sharing / use of CPU and I/O resources in clients With growing block sizes and additional blob data to stream, the network streaming time becomes a dominant factor in propagation times - on a 100mbit line, streaming 1mb to 8 peers takes ~1s - this process is repeated for each hop in both incoming and outgoing directions. This design in particular sends each blob on a separate subnet, thus maximising the potential for parallelisation and providing a natural path for growing the number of blobs per block should the network be judged to be able to handle it. Changes compared to the current design include: * `BlobsSidecar` is split into individual `BlobSidecar` containers - each container is signed individually by the proposer * the signature is used during gossip validation but later dropped. * KZG commitment verification is moved out of the gossip pipeline and instead done before fork choice addition, when both block and sidecars have arrived * clients may verify individual blob commitments earlier * more generally and similar to block verification, gossip propagation is performed solely based on trivial consistency checks and proposer signature verification * by-root blob requests are done per-blob, so as to retain the ability to fill in blobs one-by-one assuming clients generally receive blobs from gossip * by-range blob requests are done per-block, so as to simplify historical sync * range and root requests are limited to `128` entries for both blocks and blobs - practically, the current higher limit of `1024` for blocks does not get used and keeping the limits consistent simplifies implementation - with the merge, block sizes have grown significantly and clients generally fetch smaller chunks.
2023-02-07 09:55:51 +00:00
| `BlobIndex` | `uint64` | |
2022-03-10 05:31:11 +00:00
## Constants
### Blob
2022-03-10 05:31:11 +00:00
| Name | Value |
| - | - |
| `BLOB_TX_TYPE` | `uint8(0x05)` |
Free the blobs This PR reintroduces and further decouples blocks and blobs in EIP-4844, so as to improve network and processing performance. Block and blob processing, for the purpose of gossip validation, are independent: they can both be propagated and gossip-validated in parallel - the decoupled design allows 4 important optimizations (or, if you are so inclined, removes 4 unnecessary pessimizations): * Blocks and blobs travel on independent meshes allowing for better parallelization and utilization of high-bandwidth peers * Re-broadcasting after validation can start earlier allowing more efficient use of upload bandwidth - blocks for example can be rebroadcast to peers while blobs are still being downloaded * bandwidth-reduction techniques such as per-peer deduplication are more efficient because of the smaller message size * gossip verification happens independently for blocks and blobs, allowing better sharing / use of CPU and I/O resources in clients With growing block sizes and additional blob data to stream, the network streaming time becomes a dominant factor in propagation times - on a 100mbit line, streaming 1mb to 8 peers takes ~1s - this process is repeated for each hop in both incoming and outgoing directions. This design in particular sends each blob on a separate subnet, thus maximising the potential for parallelisation and providing a natural path for growing the number of blobs per block should the network be judged to be able to handle it. Changes compared to the current design include: * `BlobsSidecar` is split into individual `BlobSidecar` containers - each container is signed individually by the proposer * the signature is used during gossip validation but later dropped. * KZG commitment verification is moved out of the gossip pipeline and instead done before fork choice addition, when both block and sidecars have arrived * clients may verify individual blob commitments earlier * more generally and similar to block verification, gossip propagation is performed solely based on trivial consistency checks and proposer signature verification * by-root blob requests are done per-blob, so as to retain the ability to fill in blobs one-by-one assuming clients generally receive blobs from gossip * by-range blob requests are done per-block, so as to simplify historical sync * range and root requests are limited to `128` entries for both blocks and blobs - practically, the current higher limit of `1024` for blocks does not get used and keeping the limits consistent simplifies implementation - with the merge, block sizes have grown significantly and clients generally fetch smaller chunks.
2023-02-07 09:55:51 +00:00
| `VERSIONED_HASH_VERSION_KZG` | `Bytes1('0x01')` |
2022-03-10 05:31:11 +00:00
## Preset
### Execution
| Name | Value |
| - | - |
| `MAX_BLOBS_PER_BLOCK` | `uint64(2**2)` (= 4) |
2022-03-10 05:31:11 +00:00
## Configuration
## Containers
### Extended containers
#### `BeaconBlockBody`
Note: `BeaconBlock` and `SignedBeaconBlock` types are updated indirectly.
```python
class BeaconBlockBody(Container):
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Operations
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
sync_aggregate: SyncAggregate
# Execution
2022-11-17 20:51:19 +00:00
execution_payload: ExecutionPayload # [Modified in EIP-4844]
bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES]
blob_kzg_commitments: List[KZGCommitment, MAX_BLOBS_PER_BLOCK] # [New in EIP-4844]
2022-03-10 05:31:11 +00:00
```
#### `ExecutionPayload`
```python
class ExecutionPayload(Container):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32 # 'difficulty' in the yellow paper
block_number: uint64 # 'number' in the yellow paper
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
2022-11-11 22:25:06 +00:00
excess_data_gas: uint256 # [New in EIP-4844]
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]
withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]
```
#### `ExecutionPayloadHeader`
```python
class ExecutionPayloadHeader(Container):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32
block_number: uint64
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
2022-11-11 22:25:06 +00:00
excess_data_gas: uint256 # [New in EIP-4844]
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions_root: Root
withdrawals_root: Root
```
2022-03-10 05:31:11 +00:00
## Helper functions
### Misc
#### `kzg_commitment_to_versioned_hash`
```python
def kzg_commitment_to_versioned_hash(kzg_commitment: KZGCommitment) -> VersionedHash:
return VERSIONED_HASH_VERSION_KZG + hash(kzg_commitment)[1:]
```
2022-03-10 05:31:11 +00:00
#### `tx_peek_blob_versioned_hashes`
This function retrieves the hashes from the `SignedBlobTransaction` as defined in EIP-4844, using SSZ offsets.
Offsets are little-endian `uint32` values, as defined in the [SSZ specification](../../ssz/simple-serialize.md).
See [the full details of `blob_versioned_hashes` offset calculation](https://gist.github.com/protolambda/23bd106b66f6d4bb854ce46044aa3ca3).
2022-03-10 05:31:11 +00:00
```python
def tx_peek_blob_versioned_hashes(opaque_tx: Transaction) -> Sequence[VersionedHash]:
assert opaque_tx[0] == BLOB_TX_TYPE
message_offset = 1 + uint32.decode_bytes(opaque_tx[1:5])
# field offset: 32 + 8 + 32 + 32 + 8 + 4 + 32 + 4 + 4 + 32 = 188
blob_versioned_hashes_offset = (
message_offset
+ uint32.decode_bytes(opaque_tx[(message_offset + 188):(message_offset + 192)])
)
return [
VersionedHash(opaque_tx[x:(x + 32)])
for x in range(blob_versioned_hashes_offset, len(opaque_tx), 32)
]
2022-03-10 05:31:11 +00:00
```
#### `verify_kzg_commitments_against_transactions`
2022-03-10 05:31:11 +00:00
```python
def verify_kzg_commitments_against_transactions(transactions: Sequence[Transaction],
kzg_commitments: Sequence[KZGCommitment]) -> bool:
2022-11-28 12:16:18 +00:00
all_versioned_hashes: List[VersionedHash] = []
for tx in transactions:
if tx[0] == BLOB_TX_TYPE:
2022-07-15 15:37:32 +00:00
all_versioned_hashes += tx_peek_blob_versioned_hashes(tx)
return all_versioned_hashes == [kzg_commitment_to_versioned_hash(commitment) for commitment in kzg_commitments]
2022-03-10 05:31:11 +00:00
```
## Beacon chain state transition function
### Block processing
```python
def process_block(state: BeaconState, block: BeaconBlock) -> None:
process_block_header(state, block)
if is_execution_enabled(state, block.body):
process_withdrawals(state, block.body.execution_payload)
process_execution_payload(state, block.body.execution_payload, EXECUTION_ENGINE) # [Modified in EIP-4844]
2022-03-10 05:31:11 +00:00
process_randao(state, block.body)
process_eth1_data(state, block.body)
process_operations(state, block.body)
process_sync_aggregate(state, block.body.sync_aggregate)
process_blob_kzg_commitments(state, block.body) # [New in EIP-4844]
2022-03-10 05:31:11 +00:00
```
#### Execution payload
##### `process_execution_payload`
```python
def process_execution_payload(state: BeaconState, payload: ExecutionPayload, execution_engine: ExecutionEngine) -> None:
# Verify consistency of the parent hash with respect to the previous execution payload header
if is_merge_transition_complete(state):
assert payload.parent_hash == state.latest_execution_payload_header.block_hash
# Verify prev_randao
assert payload.prev_randao == get_randao_mix(state, get_current_epoch(state))
# Verify timestamp
assert payload.timestamp == compute_timestamp_at_slot(state, state.slot)
# Verify the execution payload is valid
assert execution_engine.notify_new_payload(payload)
# Cache execution payload header
state.latest_execution_payload_header = ExecutionPayloadHeader(
parent_hash=payload.parent_hash,
fee_recipient=payload.fee_recipient,
state_root=payload.state_root,
receipts_root=payload.receipts_root,
logs_bloom=payload.logs_bloom,
prev_randao=payload.prev_randao,
block_number=payload.block_number,
gas_limit=payload.gas_limit,
gas_used=payload.gas_used,
timestamp=payload.timestamp,
extra_data=payload.extra_data,
base_fee_per_gas=payload.base_fee_per_gas,
2022-11-11 22:24:05 +00:00
excess_data_gas=payload.excess_data_gas, # [New in EIP-4844]
block_hash=payload.block_hash,
transactions_root=hash_tree_root(payload.transactions),
withdrawals_root=hash_tree_root(payload.withdrawals),
)
```
#### Blob KZG commitments
2022-03-10 05:31:11 +00:00
```python
2022-11-28 12:16:18 +00:00
def process_blob_kzg_commitments(state: BeaconState, body: BeaconBlockBody) -> None:
2022-11-28 10:51:38 +00:00
# pylint: disable=unused-argument
assert verify_kzg_commitments_against_transactions(body.execution_payload.transactions, body.blob_kzg_commitments)
2022-03-10 05:31:11 +00:00
```
## Testing
*Note*: The function `initialize_beacon_state_from_eth1` is modified for pure EIP-4844 testing only.
Free the blobs This PR reintroduces and further decouples blocks and blobs in EIP-4844, so as to improve network and processing performance. Block and blob processing, for the purpose of gossip validation, are independent: they can both be propagated and gossip-validated in parallel - the decoupled design allows 4 important optimizations (or, if you are so inclined, removes 4 unnecessary pessimizations): * Blocks and blobs travel on independent meshes allowing for better parallelization and utilization of high-bandwidth peers * Re-broadcasting after validation can start earlier allowing more efficient use of upload bandwidth - blocks for example can be rebroadcast to peers while blobs are still being downloaded * bandwidth-reduction techniques such as per-peer deduplication are more efficient because of the smaller message size * gossip verification happens independently for blocks and blobs, allowing better sharing / use of CPU and I/O resources in clients With growing block sizes and additional blob data to stream, the network streaming time becomes a dominant factor in propagation times - on a 100mbit line, streaming 1mb to 8 peers takes ~1s - this process is repeated for each hop in both incoming and outgoing directions. This design in particular sends each blob on a separate subnet, thus maximising the potential for parallelisation and providing a natural path for growing the number of blobs per block should the network be judged to be able to handle it. Changes compared to the current design include: * `BlobsSidecar` is split into individual `BlobSidecar` containers - each container is signed individually by the proposer * the signature is used during gossip validation but later dropped. * KZG commitment verification is moved out of the gossip pipeline and instead done before fork choice addition, when both block and sidecars have arrived * clients may verify individual blob commitments earlier * more generally and similar to block verification, gossip propagation is performed solely based on trivial consistency checks and proposer signature verification * by-root blob requests are done per-blob, so as to retain the ability to fill in blobs one-by-one assuming clients generally receive blobs from gossip * by-range blob requests are done per-block, so as to simplify historical sync * range and root requests are limited to `128` entries for both blocks and blobs - practically, the current higher limit of `1024` for blocks does not get used and keeping the limits consistent simplifies implementation - with the merge, block sizes have grown significantly and clients generally fetch smaller chunks.
2023-02-07 09:55:51 +00:00
The `BeaconState` initialization is unchanged, except for the use of the updated `eip4844.BeaconBlockBody` type
when initializing the first body-root.
2022-03-10 05:31:11 +00:00
```python
def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32,
eth1_timestamp: uint64,
deposits: Sequence[Deposit],
execution_payload_header: ExecutionPayloadHeader=ExecutionPayloadHeader()
) -> BeaconState:
fork = Fork(
previous_version=EIP4844_FORK_VERSION, # [Modified in EIP-4844] for testing only
current_version=EIP4844_FORK_VERSION, # [Modified in EIP-4844]
epoch=GENESIS_EPOCH,
)
state = BeaconState(
genesis_time=eth1_timestamp + GENESIS_DELAY,
fork=fork,
eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))),
latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR, # Seed RANDAO with Eth1 entropy
)
# Process deposits
leaves = list(map(lambda deposit: deposit.data, deposits))
for index, deposit in enumerate(deposits):
deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
process_deposit(state, deposit)
# Process activations
for index, validator in enumerate(state.validators):
balance = state.balances[index]
validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
validator.activation_eligibility_epoch = GENESIS_EPOCH
validator.activation_epoch = GENESIS_EPOCH
# Set genesis validators root for domain separation and chain versioning
state.genesis_validators_root = hash_tree_root(state.validators)
# Fill in sync committees
# Note: A duplicate committee is assigned for the current and next committee at genesis
state.current_sync_committee = get_next_sync_committee(state)
state.next_sync_committee = get_next_sync_committee(state)
# Initialize the execution payload header
# If empty, will initialize a chain that has not yet gone through the Merge transition
state.latest_execution_payload_header = execution_payload_header
return state
2022-03-10 05:31:11 +00:00
```