This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.
Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):
* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients
With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.
This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.
Changes compared to the current design include:
* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
* the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
* clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
In Altair, light client sync protocol exchanges `BeaconBlockHeader`
structures for tracking current progress. Wrapping `BeaconBlockHeader`
inside a `LightClientHeader` allows future extensions of this header,
e.g., to also track `ExecutionPayloadHeader`.
Note: This changes the JSON REST format by adding a `beacon` nesting.
For SSZ, the serialization format stays same (but overall root changes).
* Add @description decorator
* Unify test case naming style
* more clean ups
* Altair tests cleanup
* Clean up Altair and Bellatrix `process_deposit` tests
* Clean up Bellatrix tests
* Clean up Capella tests
* PR feedback from @ralexstokes
* Add comments on the deposit fork version tests
* Remove `test_incorrect_sig_other_version` since it is duplicate to `test_ineffective_deposit_with_bad_fork_version`
* Add `test_ineffective_deposit_with_current_fork_version`
While the light client sync protocol currently provides access to the
latest `BeaconBlockHeader`, obtaining the matching execution data needs
workarounds such as downloading the full block.
Having ready access to the EL state root simplifies use cases that need
a way to cross-check `eth_getProof` responses against LC data.
Access to `block_hash` unlocks scenarios where a CL light client drives
an EL without `engine_newPayload`. As of Altair, only the CL beacon
block root is available, but the EL block hash is needed for engine API.
Other fields in the `ExecutionPayloadHeader` such as `logs_bloom` may
allow light client applications to monitor blocks for local interest,
e.g. for transfers affecting a certain wallet. This enables to download
only the few relevant blocks instead of every single one.
A new `LightClientStore` is proposed into the Capella spec that may be
used to sync LC data that includes execution data. Existing pre-Capella
LC data will remain as is, but can be locally upgraded before feeding it
into the new `LightClientStore` so that light clients do not need to run
a potentially expensive fork transition at a specific time. This enables
the `LightClientStore` to be upgraded at a use case dependent timing at
any time before Capella hits. Smart contract and embedded deployments
benefit from reduced code size and do not need synchronization with the
beacon chain clock to perform the Capella fork.
Introduce `get_lc_beacon_slot` and `get_lc_beacon_root` accessors
similar to `get_current_slot(state)` to account for future extensions
to the light client header structure that may override how those fields
are accessed. Idea is to extend with execution accessors in the future.
Introduce `block_to_light_client_header` helper function to enable
future forks to override it with additional info (e.g., execution),
without having to change the general light client logic.
Likewise, update existing light client data creation flow to use
`block_to_light_client_header` and default-initialize empty fields.
Furthermore, generalize `create_update` helper to streamline test code
using `block_to_light_client_header`.
Note: In Altair spec, LC header is the same as `BeaconBlockHeader`.
however; future forks will extend it with more information.
* EIP4844: bytes_to_bls_field() must not accept values >= BLS_MODULUS
bytes_to_bls_field() will be used in the precompile and hence it should error out when provided with malicious inputs.
* EIP4844: Add hash_to_bls_field() for use in compute_challenges()
The previous commit made bytes_to_bls_field() be strict about its inputs. However in compute_challenges() we are
dealing with Fiat-Shamir and hash outputs that could be innocuously higher than the modulus. For this reason we add the
hash_to_bls_field() helper for use in compute_challenges().
* EIP4844: Further use of bytes_to_bls_field() // Fix executable spec
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
Jacek Sieka
Dankrad Feist
dankrad
Hsiao-Wei Wang
Danny Ryan
George Kadianakis
Jacek Sieka
Potuz
Paul Harris
Age Manning
protolambda
kevaundray
Ben Edgington
Pawan Dhananjay
Emilia Hane
Etan Kissling
terence tsao
GeemoCandama
henridf
Mikhail Kalinin
dapplion
Justin Traglia
Stefan Bratanov
Alex Stokes
Tim Beiko