2023-02-07 23:22:28 +00:00
# Deneb -- The Beacon Chain
2022-03-10 05:31:11 +00:00
**Notice**: This document is a work-in-progress for researchers and implementers.
## Table of contents
<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE - RUN doctoc TO UPDATE -->
- [Introduction ](#introduction )
- [Custom types ](#custom-types )
- [Constants ](#constants )
2023-02-15 08:10:31 +00:00
- [Domain types ](#domain-types )
2022-07-13 10:13:30 +00:00
- [Blob ](#blob )
- [Preset ](#preset )
- [Execution ](#execution )
2022-03-10 05:31:11 +00:00
- [Configuration ](#configuration )
- [Containers ](#containers )
- [Extended containers ](#extended-containers )
- [`BeaconBlockBody` ](#beaconblockbody )
2022-09-21 13:10:37 +00:00
- [`ExecutionPayload` ](#executionpayload )
- [`ExecutionPayloadHeader` ](#executionpayloadheader )
2022-03-10 05:31:11 +00:00
- [Helper functions ](#helper-functions )
- [Misc ](#misc )
2022-07-13 10:13:30 +00:00
- [`kzg_commitment_to_versioned_hash` ](#kzg_commitment_to_versioned_hash )
2022-03-10 05:31:11 +00:00
- [Beacon chain state transition function ](#beacon-chain-state-transition-function )
2023-05-15 16:27:00 +00:00
- [Execution engine ](#execution-engine )
- [Request data ](#request-data )
- [Modified `NewPayloadRequest` ](#modified-newpayloadrequest )
- [Engine APIs ](#engine-apis )
2023-05-24 03:55:22 +00:00
- [`is_valid_versioned_hashes` ](#is_valid_versioned_hashes )
- [Modified `verify_and_notify_new_payload` ](#modified-verify_and_notify_new_payload )
2022-03-10 05:31:11 +00:00
- [Block processing ](#block-processing )
2022-09-21 13:10:37 +00:00
- [Execution payload ](#execution-payload )
2023-05-30 13:12:47 +00:00
- [Modified `process_execution_payload` ](#modified-process_execution_payload )
- [Modified `process_voluntary_exit` ](#modified-process_voluntary_exit )
2022-03-10 05:31:11 +00:00
- [Testing ](#testing )
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->
## Introduction
2023-06-07 09:45:39 +00:00
Deneb is a consensus-layer upgrade containing a number of features. Including:
2023-06-12 08:02:28 +00:00
* [EIP-4844 ](https://eips.ethereum.org/EIPS/eip-4844 ): Shard Blob Transactions scale data-availability of Ethereum in a simple, forwards-compatible manner
* [EIP-7044 ](https://github.com/ethereum/EIPs/pull/7044 ): Perpetually Valid Signed Voluntary Exits
2023-05-20 14:05:22 +00:00
2022-03-10 05:31:11 +00:00
## Custom types
| Name | SSZ equivalent | Description |
| - | - | - |
2023-06-08 07:05:46 +00:00
| `VersionedHash` | `Bytes32` | *[New in Deneb:EIP4844]* |
| `BlobIndex` | `uint64` | *[New in Deneb:EIP4844]* |
2022-03-10 05:31:11 +00:00
## Constants
2023-02-14 12:39:59 +00:00
### Domain types
| Name | Value |
| - | - |
| `DOMAIN_BLOB_SIDECAR` | `DomainType('0x0B000000')` |
2022-07-13 10:13:30 +00:00
### Blob
2022-03-10 05:31:11 +00:00
| Name | Value |
| - | - |
2023-04-06 09:20:07 +00:00
| `BLOB_TX_TYPE` | `uint8(0x03)` |
Free the blobs
This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.
Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):
* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients
With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.
This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.
Changes compared to the current design include:
* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
* the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
* clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
2023-02-07 09:55:51 +00:00
| `VERSIONED_HASH_VERSION_KZG` | `Bytes1('0x01')` |
2022-03-10 05:31:11 +00:00
2022-07-13 10:13:30 +00:00
## Preset
### Execution
2023-05-24 13:15:01 +00:00
| Name | Value | Description |
| - | - | - |
2023-06-08 07:05:46 +00:00
| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `uint64(2**12)` (= 4096) | *[New in Deneb:EIP4844]* hardfork independent fixed theoretical limit same as `LIMIT_BLOBS_PER_TX` (see EIP 4844) |
2023-06-09 15:10:00 +00:00
| `MAX_BLOBS_PER_BLOCK` | `uint64(6)` | *[New in Deneb:EIP4844]* maximum number of blobs in a single block limited by `MAX_BLOB_COMMITMENTS_PER_BLOCK` |
2023-06-07 09:45:39 +00:00
*Note*: The blob transactions are packed into the execution payload by the EL/builder with their corresponding blobs being independently transmitted
and are limited by `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` . However the CL limit is independently defined by `MAX_BLOBS_PER_BLOCK` .
2022-07-13 10:13:30 +00:00
2022-03-10 05:31:11 +00:00
## Configuration
## Containers
### Extended containers
#### `BeaconBlockBody`
Note: `BeaconBlock` and `SignedBeaconBlock` types are updated indirectly.
```python
class BeaconBlockBody(Container):
randao_reveal: BLSSignature
eth1_data: Eth1Data # Eth1 data vote
graffiti: Bytes32 # Arbitrary data
# Operations
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
sync_aggregate: SyncAggregate
# Execution
2023-06-07 09:45:39 +00:00
execution_payload: ExecutionPayload # [Modified in Deneb:EIP4844]
2022-10-20 14:30:49 +00:00
bls_to_execution_changes: List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES]
2023-06-07 09:45:39 +00:00
blob_kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK] # [New in Deneb:EIP4844]
2022-03-10 05:31:11 +00:00
```
2022-09-21 13:10:37 +00:00
#### `ExecutionPayload`
```python
class ExecutionPayload(Container):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress # 'beneficiary' in the yellow paper
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32 # 'difficulty' in the yellow paper
block_number: uint64 # 'number' in the yellow paper
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions: List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]
2022-10-20 14:30:49 +00:00
withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]
2023-06-07 09:45:39 +00:00
data_gas_used: uint64 # [New in Deneb:EIP4844]
excess_data_gas: uint64 # [New in Deneb:EIP4844]
2022-09-21 13:10:37 +00:00
```
#### `ExecutionPayloadHeader`
```python
class ExecutionPayloadHeader(Container):
# Execution block header fields
parent_hash: Hash32
fee_recipient: ExecutionAddress
state_root: Bytes32
receipts_root: Bytes32
logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM]
prev_randao: Bytes32
block_number: uint64
gas_limit: uint64
gas_used: uint64
timestamp: uint64
extra_data: ByteList[MAX_EXTRA_DATA_BYTES]
base_fee_per_gas: uint256
# Extra payload fields
block_hash: Hash32 # Hash of execution block
transactions_root: Root
2022-10-20 14:30:49 +00:00
withdrawals_root: Root
2023-06-07 09:45:39 +00:00
data_gas_used: uint64 # [New in Deneb:EIP4844]
excess_data_gas: uint64 # [New in Deneb:EIP4844]
2022-09-21 13:10:37 +00:00
```
2022-03-10 05:31:11 +00:00
## Helper functions
2022-06-22 12:13:41 +00:00
### Misc
2022-03-14 17:54:54 +00:00
2022-07-13 10:12:31 +00:00
#### `kzg_commitment_to_versioned_hash`
2022-03-14 17:54:54 +00:00
```python
2022-07-13 10:12:31 +00:00
def kzg_commitment_to_versioned_hash(kzg_commitment: KZGCommitment) -> VersionedHash:
return VERSIONED_HASH_VERSION_KZG + hash(kzg_commitment)[1:]
2022-03-14 17:54:54 +00:00
```
2023-05-15 16:27:00 +00:00
## Beacon chain state transition function
### Execution engine
#### Request data
2022-03-10 05:31:11 +00:00
2023-05-15 16:27:00 +00:00
##### Modified `NewPayloadRequest`
2022-03-10 05:31:11 +00:00
```python
2023-05-15 16:27:00 +00:00
@dataclass
class NewPayloadRequest(object):
execution_payload: ExecutionPayload
versioned_hashes: Sequence[VersionedHash]
2022-03-10 05:31:11 +00:00
```
2023-05-15 16:27:00 +00:00
#### Engine APIs
2023-05-24 03:55:22 +00:00
##### `is_valid_versioned_hashes`
2022-03-10 05:31:11 +00:00
```python
2023-05-23 14:25:43 +00:00
def is_valid_versioned_hashes(self: ExecutionEngine, new_payload_request: NewPayloadRequest) -> bool:
"""
Return ``True`` if and only if the version hashes computed by the blob transactions of
``new_payload_request.execution_payload`` matches ``new_payload_request.version_hashes``.
"""
...
2022-03-10 05:31:11 +00:00
```
2023-05-24 03:55:22 +00:00
##### Modified `verify_and_notify_new_payload`
2022-03-10 05:31:11 +00:00
```python
2023-05-23 14:25:43 +00:00
def verify_and_notify_new_payload(self: ExecutionEngine,
new_payload_request: NewPayloadRequest) -> bool:
2023-05-15 16:27:00 +00:00
"""
2023-05-23 07:16:12 +00:00
Return ``True`` if and only if ``new_payload_request`` is valid with respect to ``self.execution_state``.
2023-05-15 16:27:00 +00:00
"""
2023-05-24 04:55:49 +00:00
if not self.is_valid_block_hash(new_payload_request.execution_payload):
return False
2023-06-07 09:45:39 +00:00
# [New in Deneb:EIP4844]
2023-05-24 04:55:49 +00:00
if not self.is_valid_versioned_hashes(new_payload_request):
return False
if not self.notify_new_payload(new_payload_request.execution_payload):
return False
2023-05-23 14:25:43 +00:00
return True
2022-03-10 05:31:11 +00:00
```
2023-05-23 15:55:35 +00:00
### Block processing
2022-09-21 13:10:37 +00:00
#### Execution payload
2023-05-30 13:12:47 +00:00
##### Modified `process_execution_payload`
2022-09-21 13:10:37 +00:00
```python
2023-05-15 16:27:00 +00:00
def process_execution_payload(state: BeaconState, body: BeaconBlockBody, execution_engine: ExecutionEngine) -> None:
payload = body.execution_payload
2022-09-21 13:10:37 +00:00
# Verify consistency of the parent hash with respect to the previous execution payload header
2023-05-19 17:14:10 +00:00
assert payload.parent_hash == state.latest_execution_payload_header.block_hash
2022-09-21 13:10:37 +00:00
# Verify prev_randao
assert payload.prev_randao == get_randao_mix(state, get_current_epoch(state))
# Verify timestamp
assert payload.timestamp == compute_timestamp_at_slot(state, state.slot)
2023-05-24 07:38:55 +00:00
2023-06-07 09:45:39 +00:00
# [New in Deneb:EIP4844] Verify commitments are under limit
2023-05-24 07:38:55 +00:00
assert len(body.blob_kzg_commitments) < = MAX_BLOBS_PER_BLOCK
2022-09-21 13:10:37 +00:00
# Verify the execution payload is valid
2023-06-07 09:45:39 +00:00
# [Modified in Deneb:EIP4844] Pass `versioned_hashes` to Execution Engine
2023-05-15 16:27:00 +00:00
versioned_hashes = [kzg_commitment_to_versioned_hash(commitment) for commitment in body.blob_kzg_commitments]
2023-05-23 14:25:43 +00:00
assert execution_engine.verify_and_notify_new_payload(
2023-05-16 15:58:01 +00:00
NewPayloadRequest(execution_payload=payload, versioned_hashes=versioned_hashes)
)
2022-10-20 14:30:49 +00:00
2022-09-21 13:10:37 +00:00
# Cache execution payload header
state.latest_execution_payload_header = ExecutionPayloadHeader(
parent_hash=payload.parent_hash,
fee_recipient=payload.fee_recipient,
state_root=payload.state_root,
receipts_root=payload.receipts_root,
logs_bloom=payload.logs_bloom,
prev_randao=payload.prev_randao,
block_number=payload.block_number,
gas_limit=payload.gas_limit,
gas_used=payload.gas_used,
timestamp=payload.timestamp,
extra_data=payload.extra_data,
base_fee_per_gas=payload.base_fee_per_gas,
block_hash=payload.block_hash,
transactions_root=hash_tree_root(payload.transactions),
2022-10-24 16:44:11 +00:00
withdrawals_root=hash_tree_root(payload.withdrawals),
2023-06-07 09:45:39 +00:00
data_gas_used=payload.data_gas_used, # [New in Deneb:EIP4844]
excess_data_gas=payload.excess_data_gas, # [New in Deneb:EIP4844]
2022-10-20 18:39:18 +00:00
)
2022-09-21 13:10:37 +00:00
```
2023-05-30 13:12:47 +00:00
#### Modified `process_voluntary_exit`
2023-03-11 12:28:49 +00:00
```python
def process_voluntary_exit(state: BeaconState, signed_voluntary_exit: SignedVoluntaryExit) -> None:
voluntary_exit = signed_voluntary_exit.message
validator = state.validators[voluntary_exit.validator_index]
# Verify the validator is active
assert is_active_validator(validator, get_current_epoch(state))
# Verify exit has not been initiated
assert validator.exit_epoch == FAR_FUTURE_EPOCH
# Exits must specify an epoch when they become valid; they are not valid before then
assert get_current_epoch(state) >= voluntary_exit.epoch
# Verify the validator has been active long enough
assert get_current_epoch(state) >= validator.activation_epoch + SHARD_COMMITTEE_PERIOD
# Verify signature
2023-06-07 10:40:02 +00:00
# [Modified in Deneb:EIP7044]
2023-05-19 15:32:49 +00:00
domain = compute_domain(DOMAIN_VOLUNTARY_EXIT, CAPELLA_FORK_VERSION, state.genesis_validators_root)
2023-03-11 12:28:49 +00:00
signing_root = compute_signing_root(voluntary_exit, domain)
assert bls.Verify(validator.pubkey, signing_root, signed_voluntary_exit.signature)
# Initiate exit
initiate_validator_exit(state, voluntary_exit.validator_index)
```
2022-03-10 05:31:11 +00:00
## Testing
2023-02-07 23:22:28 +00:00
*Note*: The function `initialize_beacon_state_from_eth1` is modified for pure Deneb testing only.
2022-03-10 05:31:11 +00:00
2023-02-10 09:44:31 +00:00
The `BeaconState` initialization is unchanged, except for the use of the updated `deneb.BeaconBlockBody` type
2022-07-13 10:13:30 +00:00
when initializing the first body-root.
2022-03-10 05:31:11 +00:00
```python
2022-07-13 10:13:30 +00:00
def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32,
eth1_timestamp: uint64,
deposits: Sequence[Deposit],
execution_payload_header: ExecutionPayloadHeader=ExecutionPayloadHeader()
) -> BeaconState:
fork = Fork(
2023-01-23 14:08:34 +00:00
previous_version=DENEB_FORK_VERSION, # [Modified in Deneb] for testing only
current_version=DENEB_FORK_VERSION, # [Modified in Deneb]
2022-07-13 10:13:30 +00:00
epoch=GENESIS_EPOCH,
)
state = BeaconState(
genesis_time=eth1_timestamp + GENESIS_DELAY,
fork=fork,
eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))),
latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR, # Seed RANDAO with Eth1 entropy
)
# Process deposits
leaves = list(map(lambda deposit: deposit.data, deposits))
for index, deposit in enumerate(deposits):
deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
process_deposit(state, deposit)
# Process activations
for index, validator in enumerate(state.validators):
balance = state.balances[index]
validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
validator.activation_eligibility_epoch = GENESIS_EPOCH
validator.activation_epoch = GENESIS_EPOCH
# Set genesis validators root for domain separation and chain versioning
state.genesis_validators_root = hash_tree_root(state.validators)
# Fill in sync committees
# Note: A duplicate committee is assigned for the current and next committee at genesis
state.current_sync_committee = get_next_sync_committee(state)
state.next_sync_committee = get_next_sync_committee(state)
# Initialize the execution payload header
# If empty, will initialize a chain that has not yet gone through the Merge transition
state.latest_execution_payload_header = execution_payload_header
return state
2022-03-10 05:31:11 +00:00
```