EIP-4844: Further improvements on the spec

- Move constants around
- Implement missing functions to make the spec executable

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

specs/eip4844/beacon-chain.md

@@ -11,19 +11,22 @@
 - [Introduction](#introduction)
 - [Custom types](#custom-types)
 - [Constants](#constants)
+  - [Blob](#blob)
   - [Domain types](#domain-types)
+- [Preset](#preset)
+  - [Execution](#execution)
 - [Configuration](#configuration)
 - [Containers](#containers)
   - [Extended containers](#extended-containers)
     - [`BeaconBlockBody`](#beaconblockbody)
 - [Helper functions](#helper-functions)
   - [Misc](#misc)
-    - [`kzg_to_versioned_hash`](#kzg_to_versioned_hash)
+    - [`kzg_commitment_to_versioned_hash`](#kzg_commitment_to_versioned_hash)
     - [`tx_peek_blob_versioned_hashes`](#tx_peek_blob_versioned_hashes)
-    - [`verify_kzgs_against_transactions`](#verify_kzgs_against_transactions)
+    - [`verify_kzg_commitments_against_transactions`](#verify_kzg_commitments_against_transactions)
 - [Beacon chain state transition function](#beacon-chain-state-transition-function)
   - [Block processing](#block-processing)
-    - [Blob KZGs](#blob-kzgs)
+    - [Blob KZG commitments](#blob-kzg-commitments)
 - [Testing](#testing)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -43,10 +46,13 @@ This upgrade adds blobs to the beacon chain as part of EIP-4844.
 
 ## Constants
 
+### Blob
+
 | Name | Value |
 | - | - |
 | `BLOB_TX_TYPE` | `uint8(0x05)` |
-| `FIELD_ELEMENTS_PER_BLOB` | `4096` |
+| `FIELD_ELEMENTS_PER_BLOB` | `uint64(4096)` |
+| `VERSIONED_HASH_VERSION_KZG` | `Bytes1(0x01)` | 
 
 ### Domain types
 
@@ -54,6 +60,14 @@ This upgrade adds blobs to the beacon chain as part of EIP-4844.
 | - | - |
 | `DOMAIN_BLOBS_SIDECAR` | `DomainType('0x0a000000')` |
 
+## Preset
+
+### Execution
+
+| Name | Value |
+| - | - |
+| `MAX_BLOBS_PER_BLOCK` | `uint64(2**4)` (= 16) |
+
 ## Configuration
 
 
@@ -97,14 +111,21 @@ def kzg_commitment_to_versioned_hash(kzg_commitment: KZGCommitment) -> Versioned
 
 This function retrieves the hashes from the `SignedBlobTransaction` as defined in EIP-4844, using SSZ offsets.
 Offsets are little-endian `uint32` values, as defined in the [SSZ specification](../../ssz/simple-serialize.md).
+See [the full details of `blob_versioned_hashes` offset calculation](https://gist.github.com/protolambda/23bd106b66f6d4bb854ce46044aa3ca3).
 
 ```python
 def tx_peek_blob_versioned_hashes(opaque_tx: Transaction) -> Sequence[VersionedHash]:
     assert opaque_tx[0] == BLOB_TX_TYPE
     message_offset = 1 + uint32.decode_bytes(opaque_tx[1:5])
     # field offset: 32 + 8 + 32 + 32 + 8 + 4 + 32 + 4 + 4 = 156
-    blob_versioned_hashes_offset = uint32.decode_bytes(opaque_tx[message_offset+156:message_offset+160])
+    blob_versioned_hashes_offset = (
-    return [VersionedHash(opaque_tx[x:x+32]) for x in range(blob_versioned_hashes_offset, len(opaque_tx), 32)]
+        message_offset
+        + uint32.decode_bytes(opaque_tx[(message_offset + 156):(message_offset + 160)])
+    )
+    return [
+        VersionedHash(opaque_tx[x:(x + 32)])
+        for x in range(blob_versioned_hashes_offset, len(opaque_tx), 32)
+    ]
 ```
 
 #### `verify_kzg_commitments_against_transactions`
@@ -147,9 +168,53 @@ def process_blob_kzg_commitments(state: BeaconState, body: BeaconBlockBody):
 *Note*: The function `initialize_beacon_state_from_eth1` is modified for pure EIP-4844 testing only.
 
 The `BeaconState` initialization is unchanged, except for the use of the updated `eip4844.BeaconBlockBody` type 
-when initializing the first body-root:
+when initializing the first body-root.
 
 ```python
-state.latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
+def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32,
-```
+                                      eth1_timestamp: uint64,
+                                      deposits: Sequence[Deposit],
+                                      execution_payload_header: ExecutionPayloadHeader=ExecutionPayloadHeader()
+                                      ) -> BeaconState:
+    fork = Fork(
+        previous_version=EIP4844_FORK_VERSION,  # [Modified in EIP-4844] for testing only
+        current_version=EIP4844_FORK_VERSION,  # [Modified in EIP-4844]
+        epoch=GENESIS_EPOCH,
+    )
+    state = BeaconState(
+        genesis_time=eth1_timestamp + GENESIS_DELAY,
+        fork=fork,
+        eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))),
+        latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
+        randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR,  # Seed RANDAO with Eth1 entropy
+    )
 
+    # Process deposits
+    leaves = list(map(lambda deposit: deposit.data, deposits))
+    for index, deposit in enumerate(deposits):
+        deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
+        state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
+        process_deposit(state, deposit)
+
+    # Process activations
+    for index, validator in enumerate(state.validators):
+        balance = state.balances[index]
+        validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
+        if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
+            validator.activation_eligibility_epoch = GENESIS_EPOCH
+            validator.activation_epoch = GENESIS_EPOCH
+
+    # Set genesis validators root for domain separation and chain versioning
+    state.genesis_validators_root = hash_tree_root(state.validators)
+
+    # Fill in sync committees
+    # Note: A duplicate committee is assigned for the current and next committee at genesis
+    state.current_sync_committee = get_next_sync_committee(state)
+    state.next_sync_committee = get_next_sync_committee(state)
+
+    # Initialize the execution payload header
+    # If empty, will initialize a chain that has not yet gone through the Merge transition
+    state.latest_execution_payload_header = execution_payload_header
+
+    return state
+```

specs/eip4844/fork.md

@@ -39,5 +39,54 @@ Note that for the pure EIP-4844 networks, we don't apply `upgrade_to_eip4844` si
 
 ### Upgrading the state
 
-The `eip4844.BeaconState` format is equal to the `bellatrix.BeaconState` format, no upgrade has to be performed.
+Since the `eip4844.BeaconState` format is equal to the `bellatrix.BeaconState` format, we only have to update `BeaconState.fork`.
 
+```python
+def upgrade_to_eip4844(pre: bellatrix.BeaconState) -> BeaconState:
+    # TODO: if Capella gets scheduled, add sync it with Capella.BeaconState
+    epoch = bellatrix.get_current_epoch(pre)
+    post = BeaconState(
+        # Versioning
+        genesis_time=pre.genesis_time,
+        genesis_validators_root=pre.genesis_validators_root,
+        slot=pre.slot,
+        fork=Fork(
+            previous_version=pre.fork.current_version,
+            current_version=EIP4844_FORK_VERSION,  # [Modified in EIP4844]
+            epoch=epoch,
+        ),
+        # History
+        latest_block_header=pre.latest_block_header,
+        block_roots=pre.block_roots,
+        state_roots=pre.state_roots,
+        historical_roots=pre.historical_roots,
+        # Eth1
+        eth1_data=pre.eth1_data,
+        eth1_data_votes=pre.eth1_data_votes,
+        eth1_deposit_index=pre.eth1_deposit_index,
+        # Registry
+        validators=pre.validators,
+        balances=pre.balances,
+        # Randomness
+        randao_mixes=pre.randao_mixes,
+        # Slashings
+        slashings=pre.slashings,
+        # Participation
+        previous_epoch_participation=pre.previous_epoch_participation,
+        current_epoch_participation=pre.current_epoch_participation,
+        # Finality
+        justification_bits=pre.justification_bits,
+        previous_justified_checkpoint=pre.previous_justified_checkpoint,
+        current_justified_checkpoint=pre.current_justified_checkpoint,
+        finalized_checkpoint=pre.finalized_checkpoint,
+        # Inactivity
+        inactivity_scores=pre.inactivity_scores,
+        # Sync
+        current_sync_committee=pre.current_sync_committee,
+        next_sync_committee=pre.next_sync_committee,
+        # Execution-layer
+        latest_execution_payload_header=pre.latest_execution_payload_header,
+    )
+
+    return post
+```

specs/eip4844/p2p-interface.md

@@ -10,7 +10,6 @@ The specification of these changes continues in the same format as the network s
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
-  - [Preset](#preset)
   - [Configuration](#configuration)
   - [Containers](#containers)
     - [`BlobsSidecar`](#blobssidecar)
@@ -32,13 +31,6 @@ The specification of these changes continues in the same format as the network s
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
 
-
-## Preset
-
-| Name | Value |
-| - | - |
-| `MAX_BLOBS_PER_BLOCK` | `uint64(2**4)` (= 16) |
-
 ## Configuration
 
 | Name                                     | Value                         | Description                                                         |
@@ -46,8 +38,6 @@ The specification of these changes continues in the same format as the network s
 | `MAX_REQUEST_BLOBS_SIDECARS`             | `2**7` (= 128)                | Maximum number of blobs sidecars in a single request                |
 | `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` | `2**13` (= 8192, ~1.2 months) | The minimum epoch range over which a node must serve blobs sidecars |
 
-
-
 ## Containers
 
 ### `BlobsSidecar`
@@ -68,7 +58,6 @@ class SignedBlobsSidecar(Container):
     signature: BLSSignature
 ```
 
-
 ## The gossip domain: gossipsub
 
 Some gossip meshes are upgraded in the fork of EIP4844 to support upgraded types.

specs/eip4844/polynomial-commitments.md

@@ -16,9 +16,11 @@
     - [`bls_modular_inverse`](#bls_modular_inverse)
     - [`div`](#div)
     - [`lincomb`](#lincomb)
+    - [`matrix_lincomb`](#matrix_lincomb)
   - [KZG](#kzg)
-    - [`blob_to_kzg`](#blob_to_kzg)
+    - [`blob_to_kzg_commitment`](#blob_to_kzg_commitment)
     - [`verify_kzg_proof`](#verify_kzg_proof)
+    - [`compute_kzg_proof`](#compute_kzg_proof)
   - [Polynomials](#polynomials)
     - [`evaluate_polynomial_in_evaluation_form`](#evaluate_polynomial_in_evaluation_form)
 

specs/eip4844/validator.md

@@ -10,16 +10,21 @@
 
 - [Introduction](#introduction)
 - [Prerequisites](#prerequisites)
+- [Custom types](#custom-types)
+- [Containers](#containers)
+  - [`BlobsAndCommmitments`](#blobsandcommmitments)
+  - [`PolynomialAndCommitment`](#polynomialandcommitment)
 - [Helpers](#helpers)
   - [`is_data_available`](#is_data_available)
   - [`hash_to_bls_field`](#hash_to_bls_field)
   - [`compute_powers`](#compute_powers)
-  - [`vector_lincomb`](#vector_lincomb)
+  - [`compute_aggregated_poly_and_commitment`](#compute_aggregated_poly_and_commitment)
   - [`verify_blobs_sidecar`](#verify_blobs_sidecar)
+  - [`compute_proof_from_blobs`](#compute_proof_from_blobs)
 - [Beacon chain responsibilities](#beacon-chain-responsibilities)
   - [Block proposal](#block-proposal)
     - [Constructing the `BeaconBlockBody`](#constructing-the-beaconblockbody)
-      - [Blob commitments](#blob-commitments)
+      - [Blob KZG commitments](#blob-kzg-commitments)
   - [Beacon Block publishing time](#beacon-block-publishing-time)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
@@ -74,9 +79,10 @@ Without the sidecar the block may be processed further optimistically,
 but MUST NOT be considered valid until a valid `BlobsSidecar` has been downloaded.
 
 ```python
-def is_data_available(slot: Slot, beacon_block_root: Root, kzgs: Sequence[KZGCommitment]):
+def is_data_available(slot: Slot, beacon_block_root: Root, blob_kzg_commitments: Sequence[KZGCommitment]) -> bool:
-    sidecar = retrieve_blobs_sidecar(slot, beacon_block_root)  # implementation dependent, raises an exception if not available
+    # `retrieve_blobs_sidecar` is implementation dependent, raises an exception if not available.
-    verify_blobs_sidecar(slot, beacon_block_root, kzgs, sidecar)
+    sidecar = retrieve_blobs_sidecar(slot, beacon_block_root)
+    return verify_blobs_sidecar(slot, beacon_block_root, blob_kzg_commitments, sidecar)
 ```
 
 ### `hash_to_bls_field`