Commit Graph

5109 Commits

Author SHA1 Message Date
Paul Harris 901303f14f
Replaced EIP4844 references with Deneb
Fixes #3207
2023-02-13 17:08:51 +00:00
Dankrad Feist 7b642a2884
compute_challenge takes blobs + linter 2023-02-13 14:57:04 +00:00
Dankrad Feist f36925ea96
Merge branch 'dev' into kzg_multi_verify 2023-02-13 14:38:19 +00:00
Dankrad Feist fc10714f42
Call compute_challenge with polynomial as argument 2023-02-13 14:32:50 +00:00
Potuz 86962b9437 Simplify commitee weight computation 2023-02-10 11:43:38 -03:00
Jacek Sieka 8bc19d99ae
fixes
* expand sidecar gossip conditions
* editing
* add spec text for `BlobSidecar` signatures
2023-02-10 11:16:51 +01:00
Jacek Sieka 20278d4841
Merge remote-tracking branch 'origin/dev' into back-to-the-decoupled-future
Deneb merge
2023-02-10 10:44:31 +01:00
Jacek Sieka ffc78e9928
fixes
* separate constant for blob requests
* pedantry
2023-02-10 10:40:43 +01:00
Hsiao-Wei Wang 7b5acbfd21
Fix Capella fork test assertions 2023-02-10 00:30:00 +08:00
protolambda f6b8827eca
eip4844: move excess data gas field to end of execution payload for merkle proof path compat 2023-02-10 00:29:58 +08:00
Paul Harris 078e1cc871 fix references to eip4844 2023-02-08 09:22:28 +10:00
Jacek Sieka deb82e2f26
fix member 2023-02-07 11:23:18 +01:00
Jacek Sieka 1e07685f74
doctoc 2023-02-07 11:14:59 +01:00
Jacek Sieka 65d3c6aeb6
Free the blobs
This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.

Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):

* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients

With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.

This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.

Changes compared to the current design include:

* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
  * the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
  * clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
2023-02-07 11:05:51 +01:00
Paul Harris 0bf006cc06 Merge remote-tracking branch 'upstream/dev' into 3207 2023-02-02 16:34:34 +10:00
Dankrad Feist 659c7f513f
Change blob verification fiat-shamir to single blob 2023-01-29 13:05:02 +00:00
Dankrad Feist d89e579089
Updater toc 2023-01-27 14:33:43 +00:00
Dankrad Feist 03f4b8fa4d
Add KZG multi verify function 2023-01-27 14:23:38 +00:00
Hsiao-Wei Wang 127edf3af1
Merge pull request #3221 from benjaminion/fc-on-attester-slashing
Add on_attester_slashing() to handlers list
2023-01-27 21:49:34 +08:00
George Kadianakis 6e397b195b
EIP4844: Public methods take bytes as input (explicit validation) (#3224) 2023-01-25 16:15:19 +01:00
Hsiao-Wei Wang aede132f49
Merge pull request #3220 from benjaminion/fc-genesis-block
Add genesis_block to get_forkchoice_store() params
2023-01-24 22:22:25 +08:00
kevaundray 5eca56901e
Update polynomial-commitments.md (#3223) 2023-01-24 15:14:25 +01:00
Ben Edgington 8a0de1c595 Add on_attester_slashing() to handlers list 2023-01-24 13:25:40 +00:00
Ben Edgington d54c87a5c0 Add genesis_block to get_forkchoice_store() params 2023-01-24 13:08:41 +00:00
George Kadianakis 2626126935
EIP4844: compute_kzg_proof() now takes bytes as input (#3219) 2023-01-24 14:02:22 +01:00
Danny Ryan dd5c5af117
Merge pull request #3217 from pawanjay176/remove-point-validation
Remove kzg point and blob validations in gossip
2023-01-24 10:38:03 +01:00
Paul Harris 317facbd64 Replaced EIP4844 references with Deneb
Fixes #3207
2023-01-23 15:08:34 +01:00
Pawan Dhananjay a0791712cd
Remove kzg point validations in gossip 2023-01-20 21:32:03 +05:30
Emilia Hane f1435d456d
Update p2p-interface.md 2023-01-19 21:26:21 +01:00
Hsiao-Wei Wang 04d8f28cf6
Merge pull request #3214 from asn-d6/barycentric_no_assert
EIP4844: Handle barycentric evaluation at roots of unity
2023-01-17 23:00:17 +08:00
Hsiao-Wei Wang 521de12d9b
Merge pull request #3151 from etan-status/lc-eph
Add `ExecutionPayloadHeader` to LC data
2023-01-17 21:33:35 +08:00
George Kadianakis 296ba921c9 EIP4844: Handle barycentric evaluations at roots of unity 2023-01-16 17:51:50 +02:00
Etan Kissling 514d4431ca
Use `bellatrix` in `upgrade_x_to_capella` helpers 2023-01-13 20:11:05 +01:00
Hsiao-Wei Wang 427a6e809f
Merge pull request #3206 from ethereum/btec-fork-version
MUST use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message
2023-01-13 23:29:21 +08:00
Etan Kissling a580f82c7d
Use `beacon` wrapper in `upgrade_lc_header_to_capella` 2023-01-13 14:59:40 +01:00
Etan Kissling ca32fe8347
Add docstrings to explain empty header 2023-01-13 11:19:53 +01:00
Hsiao-Wei Wang 6a19cf568a
PR feedback from @terencechain: re-use `current_epoch` definition 2023-01-13 16:49:24 +08:00
Etan Kissling 82d6267951
Merge branch 'dev' into lc-eph 2023-01-12 18:44:54 +01:00
Hsiao-Wei Wang 2b2ffb5229
Merge pull request #3190 from etan-status/lc-headerwrapper
Add `LightClientHeader` wrapper
2023-01-13 00:40:44 +08:00
Hsiao-Wei Wang 4af46e7e78
Ignore `bls_to_execution_change` messages before `CAPELLA_FORK_EPOCH` 2023-01-12 23:47:36 +08:00
Hsiao-Wei Wang 730a7afe49
Always use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message 2023-01-12 23:28:41 +08:00
Danny Ryan fc6bdade50
Merge pull request #3174 from terencechain/clarify-empty-blob-sidecar
EIP4844: Clarify ratelimit behavior for sidecar with zero blobs
2023-01-11 19:45:03 -07:00
Danny Ryan 8d841c864b
minor copy edit 2023-01-11 19:44:39 -07:00
Danny Ryan 71d66afc0a
Merge pull request #3201 from ethereum/fix-fork-historical-summaries
Add `historical_summaries` setup to `fork.md`
2023-01-11 13:24:55 -07:00
terence tsao 96a553615a Add rate limit suggestion 2023-01-10 10:28:19 -08:00
Etan Kissling 53a95f0007
Merge branch 'dev' into lc-eph 2023-01-10 13:43:15 +01:00
Hsiao-Wei Wang f057f530fe
Add `historical_summaries` setup to fork.md 2023-01-10 16:45:58 +08:00
GeemoCandama d23d0147c8
Update p2p-interface.md
fixed typo
2023-01-07 14:28:12 -06:00
Etan Kissling b951ce077a
Move helper 2023-01-06 20:01:02 +01:00
Etan Kissling 5230c1ba87
Add `is_valid_light_client_header` and comments 2023-01-06 19:58:23 +01:00
Etan Kissling 79a717c8f6
Merge branch 'dev' into lc-headerwrapper 2023-01-06 19:45:18 +01:00
Danny Ryan ed1c6be132
Merge pull request #3185 from ethereum/fc-is_data_available
Move `is_data_available` check to fork-choice `on_block`
2023-01-06 10:55:29 -05:00
Danny Ryan 4ba2266fd5
Update specs/eip4844/fork-choice.md 2023-01-06 08:45:20 -07:00
Hsiao-Wei Wang c9f8e4fef8
Move `BlobsSidecar` 2023-01-06 23:39:04 +08:00
Hsiao-Wei Wang e154b3414c
Move `validate_blobs_sidecar` to fork-choice and add spec into execution spec scope 2023-01-06 23:19:44 +08:00
Danny Ryan d5c24dc731
Merge pull request #3186 from ethereum/capella-note
Add notes for new state historical accumulators
2023-01-06 10:13:32 -05:00
Danny Ryan 75937e5b55
Merge pull request #3181 from terencechain/4844-enable-withdrawal
EIP4844: Enable withdrawal
2023-01-06 10:12:57 -05:00
Danny Ryan d679b2e80d
Update specs/capella/beacon-chain.md 2023-01-06 08:06:39 -07:00
Hsiao-Wei Wang e937e2abbc
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2023-01-06 13:08:32 +08:00
Etan Kissling 375436094c
Merge branch 'dev' into lc-eph 2023-01-05 13:53:29 +01:00
Etan Kissling 5c64a2047a
Add `LightClientHeader` wrapper
In Altair, light client sync protocol exchanges `BeaconBlockHeader`
structures for tracking current progress. Wrapping `BeaconBlockHeader`
inside a `LightClientHeader` allows future extensions of this header,
e.g., to also track `ExecutionPayloadHeader`.

Note: This changes the JSON REST format by adding a `beacon` nesting.
For SSZ, the serialization format stays same (but overall root changes).
2023-01-05 13:36:04 +01:00
Hsiao-Wei Wang 19cf15b838
Merge pull request #3188 from henridf/patch-2
Fix typo in eip4844/BeaconBlocksByRoot docs
2023-01-05 12:52:14 +08:00
Hsiao-Wei Wang 0777a52f55
Merge pull request #3149 from etan-status/lc-toheader
Add `block_to_light_client_header` helper
2023-01-05 12:12:36 +08:00
henridf 7d2482a025
Fix typo in eip4844/BeaconBlocksByRoot docs 2023-01-04 16:29:15 +01:00
Hsiao-Wei Wang 118daae6d9
Add notes for new state historical accumulators 2023-01-04 17:55:51 +08:00
Hsiao-Wei Wang bb3601dcfa
Merge branch 'dev' into pr3181 2023-01-04 01:07:30 +08:00
Hsiao-Wei Wang 0f07cdcf72
Merge pull request #3165 from arnetheduck/block-root-accumulator-4
Historical batches
2023-01-04 00:23:21 +08:00
Hsiao-Wei Wang 6b94aab3af
Move `is_data_available` check to fork-choice `on_block` 2023-01-04 00:15:15 +08:00
Etan Kissling b0471511e5
Merge branch 'dev' into lc-eph 2023-01-03 15:36:37 +01:00
Hsiao-Wei Wang dcacb7164f
Rename `block_batch_root` to `block_summary_root` and `state_batch_root` to `state_summary_root` 2023-01-03 21:50:06 +08:00
Hsiao-Wei Wang 1cfabcbe54
Rename `HistoricalBatchSummary` -> `HistoricalSummary` and `historical_batches` -> `historical_summaries` 2023-01-02 23:07:00 +08:00
Danny Ryan 280f6e0568
Merge pull request #3167 from ethereum/akzg-tp
Add `compute_kzg_proof` as a public method
2023-01-01 14:14:53 -06:00
Dankrad Feist 4c573ff6b9 Add compute_kzg_proof to public methods 2023-01-01 14:04:56 -06:00
terence tsao 2afb02ba30 Fix Toc 2022-12-28 18:49:02 -08:00
terence tsao 45b1eb7979 EIP4844: Enable withdrawal 2022-12-28 18:42:24 -08:00
terence tsao 7204b5c3aa Merge branch 'clarify-empty-blob-sidecar' of github.com:terencechain/eth2.0-specs into clarify-empty-blob-sidecar 2022-12-23 22:24:40 +08:00
terence tsao 491bb1d207 Clean ups 2022-12-23 22:21:51 +08:00
terencechain 518c49bbba
Update specs/eip4844/p2p-interface.md
Co-authored-by: g11tech <develop@g11tech.io>
2022-12-23 20:47:14 +08:00
terence tsao 42121adeb0 Clarify what empty means 2022-12-23 14:30:30 +08:00
terence tsao 5a75704e37 Clarify behavior for empty blobs sidecar 2022-12-23 14:02:33 +08:00
Danny Ryan 4f77493b0f
Merge pull request #3169 from ethereum/false-after-prune
Handle unavailable data outside of prune window
2022-12-20 17:15:17 -07:00
Danny Ryan e6f5ab9a16
Merge pull request #3154 from dapplion/blobs-byroot-errorcode
Clarify BeaconBlockAndBlobsSidecarByRoot no blob available
2022-12-20 17:14:39 -07:00
Jacek Sieka b6ac1fa099
Update specs/capella/beacon-chain.md
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2022-12-20 07:46:18 +01:00
djrtwo 4be8f7d669 clarify how to handle unavailable data outside of the prune window 2022-12-19 10:15:18 -07:00
Hsiao-Wei Wang 84592f101b
Add tests for `process_historical_batches_update` 2022-12-16 01:01:00 +08:00
Hsiao-Wei Wang c185f91e00
Merge branch 'dev' into pr2649-tests 2022-12-15 22:56:04 +08:00
dapplion fac4b49a9b Change range lower bound to EIP4844_FORK_EPOCH 2022-12-15 11:27:33 +07:00
dapplion eaa1268dc4 Place dominant factor first 2022-12-15 11:12:37 +07:00
dapplion 29d36d8a78 Merge two sentences + drop range 2022-12-15 11:11:15 +07:00
George Kadianakis 05c2eb8ace
Merge pull request #3159 from kevaundray/patch-3
EIP4844: Clarify that z should not be in the domain
2022-12-14 17:47:48 +02:00
Danny Ryan f7187a6657
Merge pull request #3156 from ethereum/kzg-trusted-setup
Update G2 trusted setup length to 65
2022-12-14 08:15:53 -07:00
kevaundray e467ab9556
clarify that z should not be in the domain 2022-12-14 11:01:40 +00:00
Justin Traglia 224b0f20cf Add missing qoute 2022-12-13 11:53:18 -06:00
Justin Traglia 26605ac724 Fix some nits in eip4844 specs 2022-12-13 10:07:37 -06:00
Etan Kissling 02abdc311f
Merge branch 'dev' into lc-eph 2022-12-13 15:33:28 +01:00
Etan Kissling 3bfac0e16e
Merge branch 'lc-toheader' into lc-eph 2022-12-13 12:31:12 +01:00
Etan Kissling ef2a8b319d
Avoid line continuation syntax 2022-12-13 12:30:14 +01:00
Hsiao-Wei Wang 2ac06c10d3
Update G2 trusted setup length to 65 2022-12-13 18:14:50 +08:00
Hsiao-Wei Wang da3f5af919
Test case naming clean up (#3143)
* Add @description decorator

* Unify test case naming style

* more clean ups

* Altair tests cleanup

* Clean up Altair and Bellatrix `process_deposit` tests

* Clean up Bellatrix tests

* Clean up Capella tests

* PR feedback from @ralexstokes

* Add comments on the deposit fork version tests

* Remove `test_incorrect_sig_other_version` since it is duplicate to `test_ineffective_deposit_with_bad_fork_version`

* Add `test_ineffective_deposit_with_current_fork_version`
2022-12-13 15:13:44 +08:00
Danny Ryan 06d6d3888f
Merge pull request #3095 from ralexstokes/bound-withdrawals-sweep
bound the maximum number of validators considered for withdrawals per sweep
2022-12-12 19:53:29 -07:00