Commit Graph

5221 Commits

Author SHA1 Message Date
Mikhail Kalinin b22c89244a Cosmetic renaming 2023-02-23 14:09:01 +06:00
Mikhail Kalinin 23c10cfd7f Remove state.deposit_receipt_next_index variable 2023-02-23 13:53:15 +06:00
Henri DF 970da9efd2 Clean up max request blobs constants
The spec currently defines `MAX_REQUEST_BLOB_SIDECARS` as the "maximum
number of blob sidecars in a single request", but then later in the
RPC description defines the max is `MAX_REQUEST_BLOB_SIDECARS *
MAX_BLOBS_PER_BLOCK`.

Clean this up by defining `MAX_REQUEST_BLOB_SIDECARS` to be the actual max.
2023-02-22 17:20:44 +01:00
Henri DF d8111d7d3b Refer to "sidecar" consistently 2023-02-22 16:51:56 +01:00
Henri DF 837233a1be Fix reference to block->sidecar
(This was probably a cut-n-paste from block validation)
2023-02-22 16:50:56 +01:00
Mikhail Kalinin 08c7287387 Get rid of pending_deposits queue 2023-02-22 18:33:05 +06:00
kasey 9391f3ccfc fix MAX_REQUEST_BLOBS_SIDECARS typo 2023-02-21 21:48:21 -06:00
Mikhail Kalinin d5c7474d4d Move EIP6110 to features 2023-02-21 17:31:27 +06:00
Mikhail Kalinin e64607f7db Merge branch 'dev' into deposits 2023-02-21 17:26:30 +06:00
Age Manning 95401cf6e4
Clarify context bytes in the RPC methods in 4844 2023-02-21 01:10:40 +08:00
Danny Ryan ad58bfc304
Merge pull request #3244 from arnetheduck/back-to-the-decoupled-future
EIP-4844: Free the blobs
2023-02-20 09:40:56 -07:00
Jacek Sieka ac0ec660d3
add parent validation requirement
sama as block
2023-02-20 16:35:52 +01:00
Hsiao-Wei Wang 7fd36f7ce0
Merge pull request #3262 from kevaundray/patch-7
EIP4844: Remove repeated computation in polynomial-commitments.md
2023-02-20 22:45:31 +08:00
Jacek Sieka 4e2a9920f1
Update specs/deneb/p2p-interface.md
Co-authored-by: g11tech <develop@g11tech.io>
2023-02-20 12:15:53 +01:00
kevaundray 83cf02f668
Remove repeated computation 2023-02-20 10:57:39 +00:00
kevaundray 0632a5a32c
Update specs/deneb/polynomial-commitments.md
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2023-02-20 10:54:16 +00:00
Jacek Sieka 54d2559eb5
remove producer reorg on multi-blob
* also, use root/index for uniqueness
2023-02-18 17:45:16 +01:00
kevaundray c1a2962b31
Update polynomial-commitments.md 2023-02-18 15:09:43 +00:00
Danny Ryan 9dd7d2ba2f
fix Blob pluralization in a few places 2023-02-17 11:59:56 -07:00
Hsiao-Wei Wang a562710fe6
Fix `compute_quotient_eval_within_domain` overflow 2023-02-17 01:22:11 +08:00
Hsiao-Wei Wang a7e45db9ac
Fix `verify_kzg_proof_batch` and the tests 2023-02-16 22:14:14 +08:00
Hsiao-Wei Wang f23ed0cdbc
Make linter happy 2023-02-16 21:11:18 +08:00
Hsiao-Wei Wang c2315c90a3
Merge branch 'dev' into pr3244 2023-02-16 21:07:36 +08:00
Hsiao-Wei Wang ab11a014ee
Merge pull request #3248 from ethereum/mv-features
Move experimental features to `specs/_features` folder
2023-02-16 17:03:19 +08:00
Jacek Sieka 5fe857b209
fixes 2023-02-16 09:20:40 +01:00
Jacek Sieka 24a19bb886
fixes
* fight the test suite
* clarify who orphans the block
* must supply all blobs of a block in range request
2023-02-16 09:12:34 +01:00
Jacek Sieka 639ff9b2b0
Update specs/deneb/p2p-interface.md
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-02-16 08:30:40 +01:00
Jacek Sieka c39fda19c6
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-02-16 08:18:52 +01:00
Dankrad Feist 078d62e6ff
Simplify compute_challenge 2023-02-15 19:48:58 +00:00
Dankrad Feist 48e7be7dd0
Fix doctoc 2023-02-15 18:23:04 +00:00
dankrad adb5134266
Merge branch 'dev' into kzg_multi_verify 2023-02-15 18:10:39 +00:00
Ben Edgington 7637158a2f Change get_latest_attesting_balances() to get_weight() 2023-02-15 11:39:33 +00:00
Jacek Sieka f0dc126602
doctoc 2023-02-15 09:10:31 +01:00
Jacek Sieka a5f61fc173
correct function 2023-02-15 08:57:23 +01:00
Jacek Sieka da34af97d4
simplify blob verification, range request
* validate blobs using raw types
* remove `BlobSidecars` and send flattened list of `BlobSidecar`
instances instead
2023-02-15 08:51:57 +01:00
Jacek Sieka 3a37c3c497
Allow clients to orphan blocks from spammy proposers
Proposers that spam the blob topic with multiple blob versions, some of
which are invalid, MAY see their block orphaned.
2023-02-15 08:00:39 +01:00
Dankrad Feist 4086a09d0f
multi -> batch 2023-02-14 21:21:46 +00:00
Dankrad Feist c3cb7fa773
Comment on compute_challenge, assert on verify_blob_kzg_proof_multi 2023-02-14 21:12:02 +00:00
dankrad d8509e42c6
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:59:41 +00:00
dankrad aafbd45a19
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:59:24 +00:00
Dankrad Feist 3a6fccd389
Remove double hashing 2023-02-14 20:17:25 +00:00
Dankrad Feist 855cf062f0
Remove additional function 2023-02-14 20:07:34 +00:00
dankrad c49a2c2855
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:00:58 +00:00
Hsiao-Wei Wang db5a168f3b
Move experimental features to `specs/_features` folder 2023-02-14 23:41:40 +08:00
Danny Ryan 89f2eae318
Merge pull request #3218 from protolambda/excess-data-gas-field-position
eip4844: move excess data gas field to end of execution payload for merkle proof path compat
2023-02-14 07:50:25 -07:00
Jacek Sieka 58207c1c05
Upper limit on indices 2023-02-14 14:18:29 +01:00
George Kadianakis fc4e1a9acf
EIP4844: compute_kzg_proof() can now create proofs within the domain (#3243)
This will be used by optimistic rollups to create proofs about past data
2023-02-14 14:50:44 +02:00
Jacek Sieka e6b8324e25
sidecar domain 2023-02-14 13:39:59 +01:00
Jacek Sieka c8719f8524
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2023-02-14 13:32:18 +01:00
Dankrad Feist 86d955ab7f
Call compute_challenge with polynomial as argument 2023-02-13 17:17:41 +00:00
Dankrad Feist 7f1748b3c8
Change blob verification fiat-shamir to single blob 2023-02-13 17:16:15 +00:00
Dankrad Feist b76ea49fec
Add KZG multi verify function 2023-02-13 17:12:51 +00:00
Potuz f91b9863ce
Simplify commitee weight computation 2023-02-13 17:08:52 +00:00
Paul Harris 470c1b14b3
fix references to eip4844 2023-02-13 17:08:51 +00:00
Paul Harris 901303f14f
Replaced EIP4844 references with Deneb
Fixes #3207
2023-02-13 17:08:51 +00:00
Dankrad Feist 7b642a2884
compute_challenge takes blobs + linter 2023-02-13 14:57:04 +00:00
Dankrad Feist f36925ea96
Merge branch 'dev' into kzg_multi_verify 2023-02-13 14:38:19 +00:00
Dankrad Feist fc10714f42
Call compute_challenge with polynomial as argument 2023-02-13 14:32:50 +00:00
Potuz 86962b9437 Simplify commitee weight computation 2023-02-10 11:43:38 -03:00
Jacek Sieka 8bc19d99ae
fixes
* expand sidecar gossip conditions
* editing
* add spec text for `BlobSidecar` signatures
2023-02-10 11:16:51 +01:00
Jacek Sieka 20278d4841
Merge remote-tracking branch 'origin/dev' into back-to-the-decoupled-future
Deneb merge
2023-02-10 10:44:31 +01:00
Jacek Sieka ffc78e9928
fixes
* separate constant for blob requests
* pedantry
2023-02-10 10:40:43 +01:00
Age Manning 368e70d9be
Remove sending empty blobs sidecar responses 2023-02-10 10:45:07 +08:00
Hsiao-Wei Wang 7b5acbfd21
Fix Capella fork test assertions 2023-02-10 00:30:00 +08:00
protolambda f6b8827eca
eip4844: move excess data gas field to end of execution payload for merkle proof path compat 2023-02-10 00:29:58 +08:00
Paul Harris 078e1cc871 fix references to eip4844 2023-02-08 09:22:28 +10:00
Jacek Sieka deb82e2f26
fix member 2023-02-07 11:23:18 +01:00
Jacek Sieka 1e07685f74
doctoc 2023-02-07 11:14:59 +01:00
Jacek Sieka 65d3c6aeb6
Free the blobs
This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.

Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):

* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients

With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.

This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.

Changes compared to the current design include:

* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
  * the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
  * clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
2023-02-07 11:05:51 +01:00
Paul Harris 0bf006cc06 Merge remote-tracking branch 'upstream/dev' into 3207 2023-02-02 16:34:34 +10:00
Dankrad Feist 659c7f513f
Change blob verification fiat-shamir to single blob 2023-01-29 13:05:02 +00:00
Dankrad Feist d89e579089
Updater toc 2023-01-27 14:33:43 +00:00
Dankrad Feist 03f4b8fa4d
Add KZG multi verify function 2023-01-27 14:23:38 +00:00
Hsiao-Wei Wang 127edf3af1
Merge pull request #3221 from benjaminion/fc-on-attester-slashing
Add on_attester_slashing() to handlers list
2023-01-27 21:49:34 +08:00
George Kadianakis 6e397b195b
EIP4844: Public methods take bytes as input (explicit validation) (#3224) 2023-01-25 16:15:19 +01:00
Hsiao-Wei Wang aede132f49
Merge pull request #3220 from benjaminion/fc-genesis-block
Add genesis_block to get_forkchoice_store() params
2023-01-24 22:22:25 +08:00
kevaundray 5eca56901e
Update polynomial-commitments.md (#3223) 2023-01-24 15:14:25 +01:00
Ben Edgington 8a0de1c595 Add on_attester_slashing() to handlers list 2023-01-24 13:25:40 +00:00
Ben Edgington d54c87a5c0 Add genesis_block to get_forkchoice_store() params 2023-01-24 13:08:41 +00:00
George Kadianakis 2626126935
EIP4844: compute_kzg_proof() now takes bytes as input (#3219) 2023-01-24 14:02:22 +01:00
Danny Ryan dd5c5af117
Merge pull request #3217 from pawanjay176/remove-point-validation
Remove kzg point and blob validations in gossip
2023-01-24 10:38:03 +01:00
Paul Harris 317facbd64 Replaced EIP4844 references with Deneb
Fixes #3207
2023-01-23 15:08:34 +01:00
Pawan Dhananjay a0791712cd
Remove kzg point validations in gossip 2023-01-20 21:32:03 +05:30
Emilia Hane f1435d456d
Update p2p-interface.md 2023-01-19 21:26:21 +01:00
Hsiao-Wei Wang 04d8f28cf6
Merge pull request #3214 from asn-d6/barycentric_no_assert
EIP4844: Handle barycentric evaluation at roots of unity
2023-01-17 23:00:17 +08:00
Hsiao-Wei Wang 521de12d9b
Merge pull request #3151 from etan-status/lc-eph
Add `ExecutionPayloadHeader` to LC data
2023-01-17 21:33:35 +08:00
George Kadianakis 296ba921c9 EIP4844: Handle barycentric evaluations at roots of unity 2023-01-16 17:51:50 +02:00
Etan Kissling 514d4431ca
Use `bellatrix` in `upgrade_x_to_capella` helpers 2023-01-13 20:11:05 +01:00
Hsiao-Wei Wang 427a6e809f
Merge pull request #3206 from ethereum/btec-fork-version
MUST use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message
2023-01-13 23:29:21 +08:00
Etan Kissling a580f82c7d
Use `beacon` wrapper in `upgrade_lc_header_to_capella` 2023-01-13 14:59:40 +01:00
Etan Kissling ca32fe8347
Add docstrings to explain empty header 2023-01-13 11:19:53 +01:00
Hsiao-Wei Wang 6a19cf568a
PR feedback from @terencechain: re-use `current_epoch` definition 2023-01-13 16:49:24 +08:00
Etan Kissling 82d6267951
Merge branch 'dev' into lc-eph 2023-01-12 18:44:54 +01:00
Hsiao-Wei Wang 2b2ffb5229
Merge pull request #3190 from etan-status/lc-headerwrapper
Add `LightClientHeader` wrapper
2023-01-13 00:40:44 +08:00
Hsiao-Wei Wang 4af46e7e78
Ignore `bls_to_execution_change` messages before `CAPELLA_FORK_EPOCH` 2023-01-12 23:47:36 +08:00
Hsiao-Wei Wang 730a7afe49
Always use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message 2023-01-12 23:28:41 +08:00
Danny Ryan fc6bdade50
Merge pull request #3174 from terencechain/clarify-empty-blob-sidecar
EIP4844: Clarify ratelimit behavior for sidecar with zero blobs
2023-01-11 19:45:03 -07:00
Danny Ryan 8d841c864b
minor copy edit 2023-01-11 19:44:39 -07:00
Danny Ryan 71d66afc0a
Merge pull request #3201 from ethereum/fix-fork-historical-summaries
Add `historical_summaries` setup to `fork.md`
2023-01-11 13:24:55 -07:00
terence tsao 96a553615a Add rate limit suggestion 2023-01-10 10:28:19 -08:00
Etan Kissling 53a95f0007
Merge branch 'dev' into lc-eph 2023-01-10 13:43:15 +01:00
Hsiao-Wei Wang f057f530fe
Add `historical_summaries` setup to fork.md 2023-01-10 16:45:58 +08:00
GeemoCandama d23d0147c8
Update p2p-interface.md
fixed typo
2023-01-07 14:28:12 -06:00
Etan Kissling b951ce077a
Move helper 2023-01-06 20:01:02 +01:00
Etan Kissling 5230c1ba87
Add `is_valid_light_client_header` and comments 2023-01-06 19:58:23 +01:00
Etan Kissling 79a717c8f6
Merge branch 'dev' into lc-headerwrapper 2023-01-06 19:45:18 +01:00
Danny Ryan ed1c6be132
Merge pull request #3185 from ethereum/fc-is_data_available
Move `is_data_available` check to fork-choice `on_block`
2023-01-06 10:55:29 -05:00
Danny Ryan 4ba2266fd5
Update specs/eip4844/fork-choice.md 2023-01-06 08:45:20 -07:00
Hsiao-Wei Wang c9f8e4fef8
Move `BlobsSidecar` 2023-01-06 23:39:04 +08:00
Hsiao-Wei Wang e154b3414c
Move `validate_blobs_sidecar` to fork-choice and add spec into execution spec scope 2023-01-06 23:19:44 +08:00
Danny Ryan d5c24dc731
Merge pull request #3186 from ethereum/capella-note
Add notes for new state historical accumulators
2023-01-06 10:13:32 -05:00
Danny Ryan 75937e5b55
Merge pull request #3181 from terencechain/4844-enable-withdrawal
EIP4844: Enable withdrawal
2023-01-06 10:12:57 -05:00
Danny Ryan d679b2e80d
Update specs/capella/beacon-chain.md 2023-01-06 08:06:39 -07:00
Hsiao-Wei Wang e937e2abbc
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2023-01-06 13:08:32 +08:00
Etan Kissling 375436094c
Merge branch 'dev' into lc-eph 2023-01-05 13:53:29 +01:00
Etan Kissling 5c64a2047a
Add `LightClientHeader` wrapper
In Altair, light client sync protocol exchanges `BeaconBlockHeader`
structures for tracking current progress. Wrapping `BeaconBlockHeader`
inside a `LightClientHeader` allows future extensions of this header,
e.g., to also track `ExecutionPayloadHeader`.

Note: This changes the JSON REST format by adding a `beacon` nesting.
For SSZ, the serialization format stays same (but overall root changes).
2023-01-05 13:36:04 +01:00
Hsiao-Wei Wang 19cf15b838
Merge pull request #3188 from henridf/patch-2
Fix typo in eip4844/BeaconBlocksByRoot docs
2023-01-05 12:52:14 +08:00
Hsiao-Wei Wang 0777a52f55
Merge pull request #3149 from etan-status/lc-toheader
Add `block_to_light_client_header` helper
2023-01-05 12:12:36 +08:00
henridf 7d2482a025
Fix typo in eip4844/BeaconBlocksByRoot docs 2023-01-04 16:29:15 +01:00
Hsiao-Wei Wang 118daae6d9
Add notes for new state historical accumulators 2023-01-04 17:55:51 +08:00
Hsiao-Wei Wang bb3601dcfa
Merge branch 'dev' into pr3181 2023-01-04 01:07:30 +08:00
Hsiao-Wei Wang 0f07cdcf72
Merge pull request #3165 from arnetheduck/block-root-accumulator-4
Historical batches
2023-01-04 00:23:21 +08:00
Hsiao-Wei Wang 6b94aab3af
Move `is_data_available` check to fork-choice `on_block` 2023-01-04 00:15:15 +08:00
Etan Kissling b0471511e5
Merge branch 'dev' into lc-eph 2023-01-03 15:36:37 +01:00
Hsiao-Wei Wang dcacb7164f
Rename `block_batch_root` to `block_summary_root` and `state_batch_root` to `state_summary_root` 2023-01-03 21:50:06 +08:00
Hsiao-Wei Wang 1cfabcbe54
Rename `HistoricalBatchSummary` -> `HistoricalSummary` and `historical_batches` -> `historical_summaries` 2023-01-02 23:07:00 +08:00
Danny Ryan 280f6e0568
Merge pull request #3167 from ethereum/akzg-tp
Add `compute_kzg_proof` as a public method
2023-01-01 14:14:53 -06:00
Dankrad Feist 4c573ff6b9 Add compute_kzg_proof to public methods 2023-01-01 14:04:56 -06:00
terence tsao 2afb02ba30 Fix Toc 2022-12-28 18:49:02 -08:00
terence tsao 45b1eb7979 EIP4844: Enable withdrawal 2022-12-28 18:42:24 -08:00
Mikhail Kalinin 6eb118d34a Fix finality check in deposit processing 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 48f120c90e Set a higher limit to deposit receipts list 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 9d2a8f7d63 Give deposits EIP a name 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 8cc293c869 Fix wording 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 5ea983ac33 Fix toc, add more comments 2022-12-24 09:40:15 +06:00
Mikhail Kalinin b3c771c46d Preserve deposits per epoch boundary 2022-12-24 09:40:15 +06:00
Mikhail Kalinin c53ab45e19 Add in-protocol deposit processing 2022-12-24 09:40:15 +06:00
terence tsao 7204b5c3aa Merge branch 'clarify-empty-blob-sidecar' of github.com:terencechain/eth2.0-specs into clarify-empty-blob-sidecar 2022-12-23 22:24:40 +08:00
terence tsao 491bb1d207 Clean ups 2022-12-23 22:21:51 +08:00
terencechain 518c49bbba
Update specs/eip4844/p2p-interface.md
Co-authored-by: g11tech <develop@g11tech.io>
2022-12-23 20:47:14 +08:00
terence tsao 42121adeb0 Clarify what empty means 2022-12-23 14:30:30 +08:00
terence tsao 5a75704e37 Clarify behavior for empty blobs sidecar 2022-12-23 14:02:33 +08:00
Danny Ryan 4f77493b0f
Merge pull request #3169 from ethereum/false-after-prune
Handle unavailable data outside of prune window
2022-12-20 17:15:17 -07:00
Danny Ryan e6f5ab9a16
Merge pull request #3154 from dapplion/blobs-byroot-errorcode
Clarify BeaconBlockAndBlobsSidecarByRoot no blob available
2022-12-20 17:14:39 -07:00
Jacek Sieka b6ac1fa099
Update specs/capella/beacon-chain.md
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2022-12-20 07:46:18 +01:00
djrtwo 4be8f7d669 clarify how to handle unavailable data outside of the prune window 2022-12-19 10:15:18 -07:00
Hsiao-Wei Wang 84592f101b
Add tests for `process_historical_batches_update` 2022-12-16 01:01:00 +08:00
Hsiao-Wei Wang c185f91e00
Merge branch 'dev' into pr2649-tests 2022-12-15 22:56:04 +08:00
dapplion fac4b49a9b Change range lower bound to EIP4844_FORK_EPOCH 2022-12-15 11:27:33 +07:00
dapplion eaa1268dc4 Place dominant factor first 2022-12-15 11:12:37 +07:00