Jacek Sieka 65d3c6aeb6
Free the blobs
This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.

Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):

* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients

With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.

This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.

Changes compared to the current design include:

* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
  * the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
  * clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
2023-02-07 11:05:51 +01:00
2023-01-10 10:23:04 +01:00
2023-02-07 11:05:51 +01:00
2021-05-28 18:13:22 -07:00
2022-11-12 23:36:34 -08:00
2022-07-13 13:14:05 +03:00
2019-03-12 11:59:08 +00:00
2022-11-28 20:01:50 +08:00
2023-01-05 13:53:29 +01:00
2023-01-10 13:43:15 +01:00
2021-08-30 16:29:41 -06:00
2023-01-10 13:43:15 +01:00

Ethereum Proof-of-Stake Consensus Specifications

Join the chat at https://discord.gg/qGpsxSA Join the chat at https://gitter.im/ethereum/sharding

To learn more about proof-of-stake and sharding, see the PoS documentation, sharding documentation and the research compendium.

This repository hosts the current Ethereum proof-of-stake specifications. Discussions about design rationale and proposed changes can be brought up and discussed as issues. Solidified, agreed-upon changes to the spec can be made through pull requests.

Specs

GitHub release PyPI version

Core specifications for Ethereum proof-of-stake clients can be found in specs. These are divided into features. Features are researched and developed in parallel, and then consolidated into sequential upgrades when ready.

Stable Specifications

Seq. Code Name Fork Epoch Specs
0 Phase0 0
1 Altair 74240
2 Bellatrix
("The Merge")
144896

In-development Specifications

Code Name or Topic Specs Notes
Capella (tentative)
EIP4844 (tentative)
Sharding (outdated)
Custody Game (outdated) Dependent on sharding
Data Availability Sampling (outdated)

Accompanying documents can be found in specs and include:

Additional specifications for client implementers

Additional specifications and standards outside of requisite client functionality can be found in the following repos:

Design goals

The following are the broad design goals for the Ethereum proof-of-stake consensus specifications:

  • to minimize complexity, even at the cost of some losses in efficiency
  • to remain live through major network partitions and when very large portions of nodes go offline
  • to select all components such that they are either quantum secure or can be easily swapped out for quantum secure counterparts when available
  • to utilize crypto and design techniques that allow for a large participation of validators in total and per unit time
  • to allow for a typical consumer laptop with O(C) resources to process/validate O(1) shards (including any system level validation such as the beacon chain)

Useful external resources

For spec contributors

Documentation on the different components used during spec writing can be found here:

Consensus spec tests

Conformance tests built from the executable python spec are available in the Ethereum Proof-of-Stake Consensus Spec Tests repo. Compressed tarballs are available in releases.

Description
Ethereum 2.0 Specifications
Readme CC0-1.0
Languages
Python 98%
Makefile 2%