Commit Graph

5523 Commits

Author SHA1 Message Date
Mikhail Kalinin b22c89244a Cosmetic renaming 2023-02-23 14:09:01 +06:00
Mikhail Kalinin 23c10cfd7f Remove state.deposit_receipt_next_index variable 2023-02-23 13:53:15 +06:00
Henri DF 970da9efd2 Clean up max request blobs constants
The spec currently defines `MAX_REQUEST_BLOB_SIDECARS` as the "maximum
number of blob sidecars in a single request", but then later in the
RPC description defines the max is `MAX_REQUEST_BLOB_SIDECARS *
MAX_BLOBS_PER_BLOCK`.

Clean this up by defining `MAX_REQUEST_BLOB_SIDECARS` to be the actual max.
2023-02-22 17:20:44 +01:00
Henri DF d8111d7d3b Refer to "sidecar" consistently 2023-02-22 16:51:56 +01:00
Henri DF 837233a1be Fix reference to block->sidecar
(This was probably a cut-n-paste from block validation)
2023-02-22 16:50:56 +01:00
Mikhail Kalinin 08c7287387 Get rid of pending_deposits queue 2023-02-22 18:33:05 +06:00
kasey 9391f3ccfc fix MAX_REQUEST_BLOBS_SIDECARS typo 2023-02-21 21:48:21 -06:00
Mikhail Kalinin d5c7474d4d Move EIP6110 to features 2023-02-21 17:31:27 +06:00
Mikhail Kalinin e64607f7db Merge branch 'dev' into deposits 2023-02-21 17:26:30 +06:00
Age Manning 95401cf6e4
Clarify context bytes in the RPC methods in 4844 2023-02-21 01:10:40 +08:00
Danny Ryan ad58bfc304
Merge pull request #3244 from arnetheduck/back-to-the-decoupled-future
EIP-4844: Free the blobs
2023-02-20 09:40:56 -07:00
Jacek Sieka ac0ec660d3
add parent validation requirement
sama as block
2023-02-20 16:35:52 +01:00
Hsiao-Wei Wang 7fd36f7ce0
Merge pull request #3262 from kevaundray/patch-7
EIP4844: Remove repeated computation in polynomial-commitments.md
2023-02-20 22:45:31 +08:00
Jacek Sieka 4e2a9920f1
Update specs/deneb/p2p-interface.md
Co-authored-by: g11tech <develop@g11tech.io>
2023-02-20 12:15:53 +01:00
kevaundray 83cf02f668
Remove repeated computation 2023-02-20 10:57:39 +00:00
kevaundray 0632a5a32c
Update specs/deneb/polynomial-commitments.md
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2023-02-20 10:54:16 +00:00
Jacek Sieka 54d2559eb5
remove producer reorg on multi-blob
* also, use root/index for uniqueness
2023-02-18 17:45:16 +01:00
kevaundray c1a2962b31
Update polynomial-commitments.md 2023-02-18 15:09:43 +00:00
Danny Ryan 9dd7d2ba2f
fix Blob pluralization in a few places 2023-02-17 11:59:56 -07:00
Hsiao-Wei Wang a562710fe6
Fix `compute_quotient_eval_within_domain` overflow 2023-02-17 01:22:11 +08:00
Hsiao-Wei Wang a7e45db9ac
Fix `verify_kzg_proof_batch` and the tests 2023-02-16 22:14:14 +08:00
Hsiao-Wei Wang f23ed0cdbc
Make linter happy 2023-02-16 21:11:18 +08:00
Hsiao-Wei Wang c2315c90a3
Merge branch 'dev' into pr3244 2023-02-16 21:07:36 +08:00
Hsiao-Wei Wang ab11a014ee
Merge pull request #3248 from ethereum/mv-features
Move experimental features to `specs/_features` folder
2023-02-16 17:03:19 +08:00
Jacek Sieka 5fe857b209
fixes 2023-02-16 09:20:40 +01:00
Jacek Sieka 24a19bb886
fixes
* fight the test suite
* clarify who orphans the block
* must supply all blobs of a block in range request
2023-02-16 09:12:34 +01:00
Jacek Sieka 639ff9b2b0
Update specs/deneb/p2p-interface.md
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-02-16 08:30:40 +01:00
Jacek Sieka c39fda19c6
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-02-16 08:18:52 +01:00
Dankrad Feist 078d62e6ff
Simplify compute_challenge 2023-02-15 19:48:58 +00:00
Dankrad Feist 48e7be7dd0
Fix doctoc 2023-02-15 18:23:04 +00:00
dankrad adb5134266
Merge branch 'dev' into kzg_multi_verify 2023-02-15 18:10:39 +00:00
Ben Edgington 7637158a2f Change get_latest_attesting_balances() to get_weight() 2023-02-15 11:39:33 +00:00
Jacek Sieka f0dc126602
doctoc 2023-02-15 09:10:31 +01:00
Jacek Sieka a5f61fc173
correct function 2023-02-15 08:57:23 +01:00
Jacek Sieka da34af97d4
simplify blob verification, range request
* validate blobs using raw types
* remove `BlobSidecars` and send flattened list of `BlobSidecar`
instances instead
2023-02-15 08:51:57 +01:00
Jacek Sieka 3a37c3c497
Allow clients to orphan blocks from spammy proposers
Proposers that spam the blob topic with multiple blob versions, some of
which are invalid, MAY see their block orphaned.
2023-02-15 08:00:39 +01:00
Dankrad Feist 4086a09d0f
multi -> batch 2023-02-14 21:21:46 +00:00
Dankrad Feist c3cb7fa773
Comment on compute_challenge, assert on verify_blob_kzg_proof_multi 2023-02-14 21:12:02 +00:00
dankrad d8509e42c6
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:59:41 +00:00
dankrad aafbd45a19
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:59:24 +00:00
Dankrad Feist 3a6fccd389
Remove double hashing 2023-02-14 20:17:25 +00:00
Dankrad Feist 855cf062f0
Remove additional function 2023-02-14 20:07:34 +00:00
dankrad c49a2c2855
Update specs/deneb/polynomial-commitments.md
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2023-02-14 20:00:58 +00:00
Hsiao-Wei Wang db5a168f3b
Move experimental features to `specs/_features` folder 2023-02-14 23:41:40 +08:00
Danny Ryan 89f2eae318
Merge pull request #3218 from protolambda/excess-data-gas-field-position
eip4844: move excess data gas field to end of execution payload for merkle proof path compat
2023-02-14 07:50:25 -07:00
Jacek Sieka 58207c1c05
Upper limit on indices 2023-02-14 14:18:29 +01:00
George Kadianakis fc4e1a9acf
EIP4844: compute_kzg_proof() can now create proofs within the domain (#3243)
This will be used by optimistic rollups to create proofs about past data
2023-02-14 14:50:44 +02:00
Jacek Sieka e6b8324e25
sidecar domain 2023-02-14 13:39:59 +01:00
Jacek Sieka c8719f8524
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2023-02-14 13:32:18 +01:00
Dankrad Feist 86d955ab7f
Call compute_challenge with polynomial as argument 2023-02-13 17:17:41 +00:00
Dankrad Feist 7f1748b3c8
Change blob verification fiat-shamir to single blob 2023-02-13 17:16:15 +00:00
Dankrad Feist b76ea49fec
Add KZG multi verify function 2023-02-13 17:12:51 +00:00
Potuz f91b9863ce
Simplify commitee weight computation 2023-02-13 17:08:52 +00:00
Paul Harris 470c1b14b3
fix references to eip4844 2023-02-13 17:08:51 +00:00
Paul Harris 901303f14f
Replaced EIP4844 references with Deneb
Fixes #3207
2023-02-13 17:08:51 +00:00
Dankrad Feist 7b642a2884
compute_challenge takes blobs + linter 2023-02-13 14:57:04 +00:00
Dankrad Feist f36925ea96
Merge branch 'dev' into kzg_multi_verify 2023-02-13 14:38:19 +00:00
Dankrad Feist fc10714f42
Call compute_challenge with polynomial as argument 2023-02-13 14:32:50 +00:00
Potuz 86962b9437 Simplify commitee weight computation 2023-02-10 11:43:38 -03:00
Jacek Sieka 8bc19d99ae
fixes
* expand sidecar gossip conditions
* editing
* add spec text for `BlobSidecar` signatures
2023-02-10 11:16:51 +01:00
Jacek Sieka 20278d4841
Merge remote-tracking branch 'origin/dev' into back-to-the-decoupled-future
Deneb merge
2023-02-10 10:44:31 +01:00
Jacek Sieka ffc78e9928
fixes
* separate constant for blob requests
* pedantry
2023-02-10 10:40:43 +01:00
Age Manning 368e70d9be
Remove sending empty blobs sidecar responses 2023-02-10 10:45:07 +08:00
Hsiao-Wei Wang 7b5acbfd21
Fix Capella fork test assertions 2023-02-10 00:30:00 +08:00
protolambda f6b8827eca
eip4844: move excess data gas field to end of execution payload for merkle proof path compat 2023-02-10 00:29:58 +08:00
Hsiao-Wei Wang 6cc5b0fc8b
Merge branch 'dev' into capella-fc-cleanup 2023-02-10 00:24:10 +08:00
Paul Harris 078e1cc871 fix references to eip4844 2023-02-08 09:22:28 +10:00
Jacek Sieka deb82e2f26
fix member 2023-02-07 11:23:18 +01:00
Jacek Sieka 1e07685f74
doctoc 2023-02-07 11:14:59 +01:00
Jacek Sieka 65d3c6aeb6
Free the blobs
This PR reintroduces and further decouples blocks and blobs in EIP-4844,
so as to improve network and processing performance.

Block and blob processing, for the purpose of gossip validation, are
independent: they can both be propagated and gossip-validated
in parallel - the decoupled design allows 4 important optimizations
(or, if you are so inclined, removes 4 unnecessary pessimizations):

* Blocks and blobs travel on independent meshes allowing for better
parallelization and utilization of high-bandwidth peers
* Re-broadcasting after validation can start earlier allowing more
efficient use of upload bandwidth - blocks for example can be
rebroadcast to peers while blobs are still being downloaded
* bandwidth-reduction techniques such as per-peer deduplication are more
efficient because of the smaller message size
* gossip verification happens independently for blocks and blobs,
allowing better sharing / use of CPU and I/O resources in clients

With growing block sizes and additional blob data to stream, the network
streaming time becomes a dominant factor in propagation times - on a
100mbit line, streaming 1mb to 8 peers takes ~1s - this process is
repeated for each hop in both incoming and outgoing directions.

This design in particular sends each blob on a separate subnet, thus
maximising the potential for parallelisation and providing a natural
path for growing the number of blobs per block should the network be
judged to be able to handle it.

Changes compared to the current design include:

* `BlobsSidecar` is split into individual `BlobSidecar` containers -
each container is signed individually by the proposer
  * the signature is used during gossip validation but later dropped.
* KZG commitment verification is moved out of the gossip pipeline and
instead done before fork choice addition, when both block and sidecars
have arrived
  * clients may verify individual blob commitments earlier
* more generally and similar to block verification, gossip propagation
is performed solely based on trivial consistency checks and proposer
signature verification
* by-root blob requests are done per-blob, so as to retain the ability
to fill in blobs one-by-one assuming clients generally receive blobs
from gossip
* by-range blob requests are done per-block, so as to simplify
historical sync
* range and root requests are limited to `128` entries for both blocks
and blobs - practically, the current higher limit of `1024` for blocks
does not get used and keeping the limits consistent simplifies
implementation - with the merge, block sizes have grown significantly
and clients generally fetch smaller chunks.
2023-02-07 11:05:51 +01:00
Paul Harris 0bf006cc06 Merge remote-tracking branch 'upstream/dev' into 3207 2023-02-02 16:34:34 +10:00
Dankrad Feist 659c7f513f
Change blob verification fiat-shamir to single blob 2023-01-29 13:05:02 +00:00
Dankrad Feist d89e579089
Updater toc 2023-01-27 14:33:43 +00:00
Dankrad Feist 03f4b8fa4d
Add KZG multi verify function 2023-01-27 14:23:38 +00:00
Hsiao-Wei Wang 127edf3af1
Merge pull request #3221 from benjaminion/fc-on-attester-slashing
Add on_attester_slashing() to handlers list
2023-01-27 21:49:34 +08:00
Hsiao-Wei Wang ce987c4a64
Delete `is_merge_transition_block` check from Capella 2023-01-27 09:12:39 +01:00
George Kadianakis 6e397b195b
EIP4844: Public methods take bytes as input (explicit validation) (#3224) 2023-01-25 16:15:19 +01:00
Hsiao-Wei Wang aede132f49
Merge pull request #3220 from benjaminion/fc-genesis-block
Add genesis_block to get_forkchoice_store() params
2023-01-24 22:22:25 +08:00
kevaundray 5eca56901e
Update polynomial-commitments.md (#3223) 2023-01-24 15:14:25 +01:00
Ben Edgington 8a0de1c595 Add on_attester_slashing() to handlers list 2023-01-24 13:25:40 +00:00
Ben Edgington d54c87a5c0 Add genesis_block to get_forkchoice_store() params 2023-01-24 13:08:41 +00:00
George Kadianakis 2626126935
EIP4844: compute_kzg_proof() now takes bytes as input (#3219) 2023-01-24 14:02:22 +01:00
Danny Ryan dd5c5af117
Merge pull request #3217 from pawanjay176/remove-point-validation
Remove kzg point and blob validations in gossip
2023-01-24 10:38:03 +01:00
Paul Harris 317facbd64 Replaced EIP4844 references with Deneb
Fixes #3207
2023-01-23 15:08:34 +01:00
Pawan Dhananjay a0791712cd
Remove kzg point validations in gossip 2023-01-20 21:32:03 +05:30
Emilia Hane f1435d456d
Update p2p-interface.md 2023-01-19 21:26:21 +01:00
Hsiao-Wei Wang 04d8f28cf6
Merge pull request #3214 from asn-d6/barycentric_no_assert
EIP4844: Handle barycentric evaluation at roots of unity
2023-01-17 23:00:17 +08:00
Hsiao-Wei Wang 521de12d9b
Merge pull request #3151 from etan-status/lc-eph
Add `ExecutionPayloadHeader` to LC data
2023-01-17 21:33:35 +08:00
George Kadianakis 296ba921c9 EIP4844: Handle barycentric evaluations at roots of unity 2023-01-16 17:51:50 +02:00
Etan Kissling 514d4431ca
Use `bellatrix` in `upgrade_x_to_capella` helpers 2023-01-13 20:11:05 +01:00
Hsiao-Wei Wang 427a6e809f
Merge pull request #3206 from ethereum/btec-fork-version
MUST use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message
2023-01-13 23:29:21 +08:00
Etan Kissling a580f82c7d
Use `beacon` wrapper in `upgrade_lc_header_to_capella` 2023-01-13 14:59:40 +01:00
Etan Kissling ca32fe8347
Add docstrings to explain empty header 2023-01-13 11:19:53 +01:00
Hsiao-Wei Wang 6a19cf568a
PR feedback from @terencechain: re-use `current_epoch` definition 2023-01-13 16:49:24 +08:00
Etan Kissling 82d6267951
Merge branch 'dev' into lc-eph 2023-01-12 18:44:54 +01:00
Hsiao-Wei Wang 2b2ffb5229
Merge pull request #3190 from etan-status/lc-headerwrapper
Add `LightClientHeader` wrapper
2023-01-13 00:40:44 +08:00
Hsiao-Wei Wang 4af46e7e78
Ignore `bls_to_execution_change` messages before `CAPELLA_FORK_EPOCH` 2023-01-12 23:47:36 +08:00
Hsiao-Wei Wang 730a7afe49
Always use `GENESIS_FORK_VERSION` to sign `BLSToExecutionChange` message 2023-01-12 23:28:41 +08:00
Danny Ryan fc6bdade50
Merge pull request #3174 from terencechain/clarify-empty-blob-sidecar
EIP4844: Clarify ratelimit behavior for sidecar with zero blobs
2023-01-11 19:45:03 -07:00
Danny Ryan 8d841c864b
minor copy edit 2023-01-11 19:44:39 -07:00
Danny Ryan 71d66afc0a
Merge pull request #3201 from ethereum/fix-fork-historical-summaries
Add `historical_summaries` setup to `fork.md`
2023-01-11 13:24:55 -07:00
terence tsao 96a553615a Add rate limit suggestion 2023-01-10 10:28:19 -08:00
Etan Kissling 53a95f0007
Merge branch 'dev' into lc-eph 2023-01-10 13:43:15 +01:00
Hsiao-Wei Wang f057f530fe
Add `historical_summaries` setup to fork.md 2023-01-10 16:45:58 +08:00
GeemoCandama d23d0147c8
Update p2p-interface.md
fixed typo
2023-01-07 14:28:12 -06:00
Etan Kissling b951ce077a
Move helper 2023-01-06 20:01:02 +01:00
Etan Kissling 5230c1ba87
Add `is_valid_light_client_header` and comments 2023-01-06 19:58:23 +01:00
Etan Kissling 79a717c8f6
Merge branch 'dev' into lc-headerwrapper 2023-01-06 19:45:18 +01:00
Danny Ryan ed1c6be132
Merge pull request #3185 from ethereum/fc-is_data_available
Move `is_data_available` check to fork-choice `on_block`
2023-01-06 10:55:29 -05:00
Danny Ryan 4ba2266fd5
Update specs/eip4844/fork-choice.md 2023-01-06 08:45:20 -07:00
Hsiao-Wei Wang c9f8e4fef8
Move `BlobsSidecar` 2023-01-06 23:39:04 +08:00
Hsiao-Wei Wang e154b3414c
Move `validate_blobs_sidecar` to fork-choice and add spec into execution spec scope 2023-01-06 23:19:44 +08:00
Danny Ryan d5c24dc731
Merge pull request #3186 from ethereum/capella-note
Add notes for new state historical accumulators
2023-01-06 10:13:32 -05:00
Danny Ryan 75937e5b55
Merge pull request #3181 from terencechain/4844-enable-withdrawal
EIP4844: Enable withdrawal
2023-01-06 10:12:57 -05:00
Danny Ryan d679b2e80d
Update specs/capella/beacon-chain.md 2023-01-06 08:06:39 -07:00
Hsiao-Wei Wang e937e2abbc
Apply suggestions from code review
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2023-01-06 13:08:32 +08:00
Etan Kissling 375436094c
Merge branch 'dev' into lc-eph 2023-01-05 13:53:29 +01:00
Etan Kissling 5c64a2047a
Add `LightClientHeader` wrapper
In Altair, light client sync protocol exchanges `BeaconBlockHeader`
structures for tracking current progress. Wrapping `BeaconBlockHeader`
inside a `LightClientHeader` allows future extensions of this header,
e.g., to also track `ExecutionPayloadHeader`.

Note: This changes the JSON REST format by adding a `beacon` nesting.
For SSZ, the serialization format stays same (but overall root changes).
2023-01-05 13:36:04 +01:00
Hsiao-Wei Wang 19cf15b838
Merge pull request #3188 from henridf/patch-2
Fix typo in eip4844/BeaconBlocksByRoot docs
2023-01-05 12:52:14 +08:00
Hsiao-Wei Wang 0777a52f55
Merge pull request #3149 from etan-status/lc-toheader
Add `block_to_light_client_header` helper
2023-01-05 12:12:36 +08:00
henridf 7d2482a025
Fix typo in eip4844/BeaconBlocksByRoot docs 2023-01-04 16:29:15 +01:00
Hsiao-Wei Wang 118daae6d9
Add notes for new state historical accumulators 2023-01-04 17:55:51 +08:00
Hsiao-Wei Wang bb3601dcfa
Merge branch 'dev' into pr3181 2023-01-04 01:07:30 +08:00
Hsiao-Wei Wang 0f07cdcf72
Merge pull request #3165 from arnetheduck/block-root-accumulator-4
Historical batches
2023-01-04 00:23:21 +08:00
Hsiao-Wei Wang 6b94aab3af
Move `is_data_available` check to fork-choice `on_block` 2023-01-04 00:15:15 +08:00
Etan Kissling b0471511e5
Merge branch 'dev' into lc-eph 2023-01-03 15:36:37 +01:00
Hsiao-Wei Wang dcacb7164f
Rename `block_batch_root` to `block_summary_root` and `state_batch_root` to `state_summary_root` 2023-01-03 21:50:06 +08:00
Hsiao-Wei Wang 1cfabcbe54
Rename `HistoricalBatchSummary` -> `HistoricalSummary` and `historical_batches` -> `historical_summaries` 2023-01-02 23:07:00 +08:00
Danny Ryan 280f6e0568
Merge pull request #3167 from ethereum/akzg-tp
Add `compute_kzg_proof` as a public method
2023-01-01 14:14:53 -06:00
Dankrad Feist 4c573ff6b9 Add compute_kzg_proof to public methods 2023-01-01 14:04:56 -06:00
terence tsao 2afb02ba30 Fix Toc 2022-12-28 18:49:02 -08:00
terence tsao 45b1eb7979 EIP4844: Enable withdrawal 2022-12-28 18:42:24 -08:00
Mikhail Kalinin 6eb118d34a Fix finality check in deposit processing 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 48f120c90e Set a higher limit to deposit receipts list 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 9d2a8f7d63 Give deposits EIP a name 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 8cc293c869 Fix wording 2022-12-24 09:40:15 +06:00
Mikhail Kalinin 5ea983ac33 Fix toc, add more comments 2022-12-24 09:40:15 +06:00
Mikhail Kalinin b3c771c46d Preserve deposits per epoch boundary 2022-12-24 09:40:15 +06:00
Mikhail Kalinin c53ab45e19 Add in-protocol deposit processing 2022-12-24 09:40:15 +06:00
terence tsao 7204b5c3aa Merge branch 'clarify-empty-blob-sidecar' of github.com:terencechain/eth2.0-specs into clarify-empty-blob-sidecar 2022-12-23 22:24:40 +08:00
terence tsao 491bb1d207 Clean ups 2022-12-23 22:21:51 +08:00
terencechain 518c49bbba
Update specs/eip4844/p2p-interface.md
Co-authored-by: g11tech <develop@g11tech.io>
2022-12-23 20:47:14 +08:00
terence tsao 42121adeb0 Clarify what empty means 2022-12-23 14:30:30 +08:00
terence tsao 5a75704e37 Clarify behavior for empty blobs sidecar 2022-12-23 14:02:33 +08:00
Danny Ryan 4f77493b0f
Merge pull request #3169 from ethereum/false-after-prune
Handle unavailable data outside of prune window
2022-12-20 17:15:17 -07:00
Danny Ryan e6f5ab9a16
Merge pull request #3154 from dapplion/blobs-byroot-errorcode
Clarify BeaconBlockAndBlobsSidecarByRoot no blob available
2022-12-20 17:14:39 -07:00
Jacek Sieka b6ac1fa099
Update specs/capella/beacon-chain.md
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2022-12-20 07:46:18 +01:00
djrtwo 4be8f7d669 clarify how to handle unavailable data outside of the prune window 2022-12-19 10:15:18 -07:00
Hsiao-Wei Wang 84592f101b
Add tests for `process_historical_batches_update` 2022-12-16 01:01:00 +08:00
Hsiao-Wei Wang c185f91e00
Merge branch 'dev' into pr2649-tests 2022-12-15 22:56:04 +08:00
dapplion fac4b49a9b Change range lower bound to EIP4844_FORK_EPOCH 2022-12-15 11:27:33 +07:00
dapplion eaa1268dc4 Place dominant factor first 2022-12-15 11:12:37 +07:00
dapplion 29d36d8a78 Merge two sentences + drop range 2022-12-15 11:11:15 +07:00
George Kadianakis 05c2eb8ace
Merge pull request #3159 from kevaundray/patch-3
EIP4844: Clarify that z should not be in the domain
2022-12-14 17:47:48 +02:00
Danny Ryan f7187a6657
Merge pull request #3156 from ethereum/kzg-trusted-setup
Update G2 trusted setup length to 65
2022-12-14 08:15:53 -07:00
kevaundray e467ab9556
clarify that z should not be in the domain 2022-12-14 11:01:40 +00:00
Justin Traglia 224b0f20cf Add missing qoute 2022-12-13 11:53:18 -06:00
Justin Traglia 26605ac724 Fix some nits in eip4844 specs 2022-12-13 10:07:37 -06:00
Etan Kissling 02abdc311f
Merge branch 'dev' into lc-eph 2022-12-13 15:33:28 +01:00
Etan Kissling 3bfac0e16e
Merge branch 'lc-toheader' into lc-eph 2022-12-13 12:31:12 +01:00
Etan Kissling ef2a8b319d
Avoid line continuation syntax 2022-12-13 12:30:14 +01:00
Hsiao-Wei Wang 2ac06c10d3
Update G2 trusted setup length to 65 2022-12-13 18:14:50 +08:00
Hsiao-Wei Wang da3f5af919
Test case naming clean up (#3143)
* Add @description decorator

* Unify test case naming style

* more clean ups

* Altair tests cleanup

* Clean up Altair and Bellatrix `process_deposit` tests

* Clean up Bellatrix tests

* Clean up Capella tests

* PR feedback from @ralexstokes

* Add comments on the deposit fork version tests

* Remove `test_incorrect_sig_other_version` since it is duplicate to `test_ineffective_deposit_with_bad_fork_version`

* Add `test_ineffective_deposit_with_current_fork_version`
2022-12-13 15:13:44 +08:00
Danny Ryan 06d6d3888f
Merge pull request #3095 from ralexstokes/bound-withdrawals-sweep
bound the maximum number of validators considered for withdrawals per sweep
2022-12-12 19:53:29 -07:00
dapplion d4b8ab1173 Clarify BeaconBlockAndBlobsSidecarByRoot no blob available 2022-12-13 09:29:28 +07:00
Danny Ryan 1f943f0d64
Update specs/capella/beacon-chain.md 2022-12-12 14:22:58 -07:00
Stefan Bratanov a8449fa4aa Fix typo in BlobsSidecarsByRange v1 2022-12-12 15:36:51 +02:00
Etan Kissling 700bef7a45
Fix `is_valid_light_client_header` 2022-12-12 13:39:14 +01:00
Etan Kissling 8ad6810a44
EIP4844 support (`excess_data_gas`), fork tests nyi 2022-12-12 13:11:49 +01:00
Etan Kissling e67ca3d309
Compute epoch only once for better readability 2022-12-12 13:09:18 +01:00
Etan Kissling 5028a806ad
Implicit init during fork transition 2022-12-12 12:45:43 +01:00
Etan Kissling 11d2a59948
Flip `is_valid_light_client_header` logic for extensibility 2022-12-12 12:33:58 +01:00
Etan Kissling 2e97af2627
Add `ExecutionPayloadHeader` to LC data
While the light client sync protocol currently provides access to the
latest `BeaconBlockHeader`, obtaining the matching execution data needs
workarounds such as downloading the full block.

Having ready access to the EL state root simplifies use cases that need
a way to cross-check `eth_getProof` responses against LC data.

Access to `block_hash` unlocks scenarios where a CL light client drives
an EL without `engine_newPayload`. As of Altair, only the CL beacon
block root is available, but the EL block hash is needed for engine API.

Other fields in the `ExecutionPayloadHeader` such as `logs_bloom` may
allow light client applications to monitor blocks for local interest,
e.g. for transfers affecting a certain wallet. This enables to download
only the few relevant blocks instead of every single one.

A new `LightClientStore` is proposed into the Capella spec that may be
used to sync LC data that includes execution data. Existing pre-Capella
LC data will remain as is, but can be locally upgraded before feeding it
into the new `LightClientStore` so that light clients do not need to run
a potentially expensive fork transition at a specific time. This enables
the `LightClientStore` to be upgraded at a use case dependent timing at
any time before Capella hits. Smart contract and embedded deployments
benefit from reduced code size and do not need synchronization with the
beacon chain clock to perform the Capella fork.
2022-12-12 00:48:40 +01:00
Etan Kissling 9ab22b3ef3
Merge branch 'lc-accessors' into lc-eph 2022-12-12 00:41:31 +01:00
Alex Stokes 37e504e784
bound the maximum number of validators considered for withdrawals per sweep 2022-12-09 10:51:25 -07:00
Etan Kissling 14fd937046
Fix 2022-12-06 23:21:06 +01:00
Etan Kissling 0fb0b26742
Add accessors for LC header
Introduce `get_lc_beacon_slot` and `get_lc_beacon_root` accessors
similar to `get_current_slot(state)` to account for future extensions
to the light client header structure that may override how those fields
are accessed. Idea is to extend with execution accessors in the future.
2022-12-06 22:05:19 +01:00
Etan Kissling 8bf801ecc6
Add `block_to_light_client_header` helper
Introduce `block_to_light_client_header` helper function to enable
future forks to override it with additional info (e.g., execution),
without having to change the general light client logic.

Likewise, update existing light client data creation flow to use
`block_to_light_client_header` and default-initialize empty fields.

Furthermore, generalize `create_update` helper to streamline test code
using `block_to_light_client_header`.

Note: In Altair spec, LC header is the same as `BeaconBlockHeader`.
however; future forks will extend it with more information.
2022-12-06 17:44:41 +01:00
Danny Ryan 9839ed4934
Merge pull request #3145 from terencechain/update-max-blobs
EIP4844: Update `MAX_BLOBS_PER_BLOCK` to 4
2022-12-06 09:37:05 -07:00
Danny Ryan 5498519945
Merge pull request #3125 from terencechain/rm-is-data-available
EIP4844: remove optimistic sync mentions
2022-12-06 09:29:34 -07:00
Etan Kissling f4dafcfebc
Merge branch 'dev' into lc-blockfuncs 2022-12-05 18:38:40 +01:00
George Kadianakis 23d3aeebba
EIP4844 precompile: Be strict when parsing field elements from network (#3138)
* EIP4844: bytes_to_bls_field() must not accept values >= BLS_MODULUS

bytes_to_bls_field() will be used in the precompile and hence it should error out when provided with malicious inputs.

* EIP4844: Add hash_to_bls_field() for use in compute_challenges()

The previous commit made bytes_to_bls_field() be strict about its inputs. However in compute_challenges() we are
dealing with Fiat-Shamir and hash outputs that could be innocuously higher than the modulus. For this reason we add the
hash_to_bls_field() helper for use in compute_challenges().

* EIP4844: Further use of bytes_to_bls_field() // Fix executable spec

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-12-01 14:59:00 +02:00
Hsiao-Wei Wang 602f846dc7
Merge pull request #3134 from ethereum/eip4844-lint
Add EIP4844 pylint and Mypy checks
2022-12-01 20:01:48 +08:00
Tim Beiko aae00a0bf6
Clarify that full withdrawals only apply to exited validators 2022-11-30 16:21:06 -08:00
terence tsao 5b0bfe7eb6 EIP4844: Update `MAX_BLOBS_PER_BLOCK` to 4 2022-11-30 14:34:09 -08:00
Hsiao-Wei Wang b62c9e8779
Merge pull request #3014 from tersec/deprecate-phase0-beacon-blocks-req-resp
deprecate phase 0-only versions of beacon_blocks_by_{range,root}
2022-11-29 22:09:13 +08:00
terence tsao 339b788839 Revert previous changes. Remove optimistic wording 2022-11-28 19:52:11 -08:00
Hsiao-Wei Wang b3a176689d
WIP. Fixing mypy errors 2022-11-28 21:01:41 +08:00
Hsiao-Wei Wang 4f32fd95d6
Enable EIP4844 lint and fix Pylint 2022-11-28 20:01:50 +08:00
Roberto Bayardo a1e46d1ae4
EIP4844: Fix return value of compute_challenges (#3129)
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2022-11-24 15:26:02 +02:00
kevaundray 3ec25a8755
Fix docstring for compute_powers and compute_challenges methods (#3130) 2022-11-24 15:11:46 +02:00
kevaundray f0ff15c14d
EIP4844: Refactor Fiat-Shamir logic to handle empty sidecars (#3093)
Additionally, it makes the Fiat-Shamir hashing logic more robust by making the challenges independent of each other. It also makes it more efficient to implement by moving both challenge computations to a single function needing a single transcript hash.

Co-authored-by: George Kadianakis <desnacked@riseup.net>
Co-authored-by: Dankrad Feist <mail@dankradfeist.de>
2022-11-23 16:52:47 +02:00
terence tsao 3624742ff5 Remove is_data_available 2022-11-22 09:15:17 -08:00
Dmitrii Shmatko 4e3b700b86 Fix initialization of VERSIONED_HASH_VERSION_KZG 2022-11-22 11:51:10 +04:00
Etan Kissling 0fb2447a95
Merge branch 'dev' into lc-blockfuncs 2022-11-19 12:20:18 +01:00
Ramana Kumar 19c8f21b63
Fix type annotation in verify_aggregate_kzg_proof 2022-11-19 09:17:11 +00:00
George Kadianakis a45627164d
Merge pull request #3097 from asn-d6/verify_kzg_proof_bytes_api
EIP4844: Refactor `verify_kzg_proof()` to receive bytes (used in precompile)
2022-11-18 20:01:18 +02:00
Hsiao-Wei Wang 46c0447662
Merge pull request #3110 from ethereum/eip4844-test-vectors
Add EIP4844 to testgen and add more Capella+EIP4844 random fork tests
2022-11-18 17:18:02 +08:00
Danny Ryan 30e975806f
Merge pull request #3108 from realbigsean/validate-blobs-sidecar-in-gossip
Make `validate_blobs_sidecar` a gossip condition
2022-11-17 16:31:14 -07:00
Hsiao-Wei Wang dd8b67075e
Add note comment 2022-11-18 04:51:19 +08:00
Hsiao-Wei Wang 438a7e55ea
Conver `roots_of_unity_brp` to integers 2022-11-18 03:20:25 +08:00
Hsiao-Wei Wang 78f0e0301e
Merge pull request #3052 from Inphi/inphi/eip4844-rebase
Rebase EIP-4844 on Capella
2022-11-18 03:16:58 +08:00
Danny Ryan de789e2d5b
Merge pull request #3090 from ethereum/4844-topic-dep
clarify deprecation of `beacon_block` gossip topic in 4844
2022-11-17 12:16:24 -07:00
Hsiao-Wei Wang ee0e2a03d2
Merge branch 'dev' into pr3052 2022-11-18 02:44:46 +08:00
Hsiao-Wei Wang 30dce7f232
Apply suggestions from code review
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-18 02:25:56 +08:00
Hsiao-Wei Wang f1d4c9047a
Merge branch 'dev' into pr3052 2022-11-18 02:12:07 +08:00
Enrico Del Fante 620943b275
Update specs/eip4844/fork.md
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-16 19:19:39 +01:00
realbigsean e8e860e1f3
make a gossip condition 2022-11-16 10:46:04 -05:00
Enrico Del Fante b737e5370f
fix state upgrade 2022-11-16 12:12:18 +01:00
Alex Stokes a5f8a15819
Clarify how to transition gossip 2022-11-15 10:29:03 -07:00
Hsiao-Wei Wang 11a037fd92
Merge pull request #3089 from terencechain/beacon-block-and-blobs-sidecar-by-root
EIP4844: Add block and sidecar retrival by root
2022-11-16 00:41:58 +08:00
terence tsao bbc3ec02ff Fix typos 2022-11-12 23:36:34 -08:00
George Kadianakis fcac0b5c69 Clarify which functions are public and provided by the KZG library 2022-11-12 01:47:39 +02:00
George Kadianakis ff34006807 Refactor `verify_kzg_proof()` to receive bytes (used in precompile)
This way, client devs don't need to convert to field elements themselves, and the KZG library takes care fo it.
2022-11-12 01:47:39 +02:00
inphi 3df1371897
Merge remote-tracking branch 'origin/dev' into inphi/eip4844-rebase 2022-11-11 17:32:06 -05:00
inphi cd1e1133a5
excess_data_gas uint256 2022-11-11 17:25:06 -05:00
inphi 6327ffa687
rename excess_blobs 2022-11-11 17:24:05 -05:00
Alex Stokes 38669d4b7a
Update specs/eip4844/p2p-interface.md
Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2022-11-11 11:26:48 -07:00
Alex Stokes 078bba2e3c
clarify deprecation of `beacon_block` gossip topic in 4844 2022-11-11 11:26:45 -07:00
Alex Stokes faf308349e
Merge pull request #3086 from michaelsproul/light-client-p2p-fix
Fix slot checks in light client p2p spec
2022-11-11 11:03:33 -07:00
terence tsao e2a2d68cae Added a comment to support pre-fork-epoch 2022-11-11 09:50:06 -08:00
Michael Sproul 897b97990f
Further clarify light client slot checks 2022-11-11 22:09:33 +11:00
inphi 104cba06a6
replace get_blobs_and_kzg_commitments 2022-11-11 00:21:56 -05:00
inphi bed1df00a2
Remove withdrawal_queue from BeaconState upgrade 2022-11-10 13:57:30 -05:00
Danny Ryan d4a9b97cd0
Merge pull request #3084 from ethereum/validator-note-for-bls-change
Add note to Capella validator guide about BLS to execution change
2022-11-10 11:32:04 -07:00
inphi a59dd37933
Merge remote-tracking branch 'origin/dev' into inphi/eip4844-rebase 2022-11-10 13:17:57 -05:00
Danny Ryan 4b573f82f6
Merge pull request #3091 from terencechain/patch-138
EIP4844: Explicitly mention old gossip validations for block
2022-11-10 11:16:18 -07:00
Danny Ryan 0e7d3555ea
minor copy edits 2022-11-10 11:12:33 -07:00
Danny Ryan 37996fc36c
Merge pull request #3068 from potuz/withdrawals_without_queues
Withdrawals without queues
2022-11-10 10:49:25 -07:00
Alex Stokes be85e9356a
Update specs/capella/validator.md
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-11-10 09:12:35 -05:00
Alex Stokes 0f93e9aad8
Update specs/capella/validator.md
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-11-10 09:12:10 -05:00
Alex Stokes be098b60c0
Update specs/eip4844/p2p-interface.md 2022-11-10 09:10:01 -05:00
Potuz 7f266bcb0f Use next_validator_withdrawal_index 2022-11-10 08:33:11 -03:00
terencechain 579f8fd1d2
EIP4844: Explicitly mention old gossip validations for block 2022-11-09 15:06:58 -08:00
Potuz 710b124cdc fix last commit 2022-11-08 21:51:54 -03:00
Potuz 8f42e485c7 Add extra % 2022-11-08 19:53:58 -03:00
Michael Sproul dae1011194
Fix typos
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-09 09:41:21 +11:00
terence tsao c447662ba0 Fix `BeaconBlocksByRootV2` for pre-4844 support 2022-11-08 12:55:43 -08:00
terence tsao cb9159fe58 EIP4844: Add block and sidecar retrival by root 2022-11-08 10:41:48 -08:00
inphi e3e73a8f54
Add Capella fork version 2022-11-08 09:06:38 -05:00
Michael Sproul 91de8a09f3
Fix slot checks in light client p2p spec 2022-11-08 17:06:07 +11:00
Alex Stokes 8bb1751f46
Add note to Capella validator guide about BLS to execution change 2022-11-07 15:22:26 -07:00
Potuz 99e2704c2c make tests pass 2022-11-07 17:56:01 -03:00
Potuz ac670e2c47 add phase0 sentence regarding process slots 2022-11-07 17:34:19 -03:00
Potuz 2f89f5096d Danny's review 1st pass 2022-11-07 17:29:56 -03:00
terence tsao f010de0871 Remove domain blobs sidecar 2022-11-07 07:16:42 -08:00
Hsiao-Wei Wang 95ee291f58
Merge branch 'dev' into pr3052 2022-11-07 10:10:16 -05:00
Hsiao-Wei Wang 544ae28721
Merge pull request #3016 from ethereum/gen_kzg_trusted_setups
Rework the testing KZG trusted setups presets
2022-11-07 22:29:54 +08:00
Hsiao-Wei Wang b7dfd5a37e
Merge pull request #3027 from ethereum/eip-4844-fee-ssz-fix
eip4844: update tx_peek_blob_versioned_hashes to match tx type from fee market update
2022-11-07 22:22:12 +08:00
Hsiao-Wei Wang dac756efee
Minor clean up 2022-11-07 09:09:09 -05:00
Etan Kissling 08ff71688f
Merge branch 'dev' into lc-blockfuncs 2022-11-07 15:03:20 +01:00
Hsiao-Wei Wang 72544d5307
Merge pull request #3081 from ethereum/add-capella-bls-to-exec-change-block-packing
add note about packing BLS to execution changes into a block
2022-11-07 06:22:04 +08:00
Hsiao-Wei Wang 6f0a7ef046
Merge pull request #3080 from ethereum/ralexstokes-patch-1
Fix typo in `bellatrix` p2p docs
2022-11-07 05:18:03 +08:00
Alex Stokes b5ecf44c91
add note about packing BLS to execution changes into a block 2022-11-04 16:41:07 -06:00
Alex Stokes e1e37deb56
Merge pull request #3075 from ethDreamer/gossip_bls_to_execution_change
Capella: Add Gossip Topic bls_to_execution_change
2022-11-04 16:22:43 -06:00
Alex Stokes c8d1614e7e
Apply suggestions from code review 2022-11-04 16:19:26 -06:00
ethDreamer 355ca296b8
Update specs/capella/p2p-interface.md
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-04 17:13:23 -05:00
ethDreamer 6c1fa0a485
Update specs/capella/p2p-interface.md
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-04 17:13:15 -05:00
ethDreamer 1f68c57a7b
Update specs/capella/p2p-interface.md
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-11-04 17:13:07 -05:00
Alex Stokes cd5567b735
Update p2p-interface.md 2022-11-04 15:54:30 -06:00
Potuz 8488fb79d9 Alex Stokes' review 2022-11-04 18:47:56 -03:00
Potuz e15b02d16f lint 2022-11-04 09:13:21 -03:00
Potuz 22f803a038 name change and increment operator 2022-11-04 07:20:26 -03:00
Potuz 6e913ecbd2 rename to latest_withdrawal_validator_index 2022-11-03 20:23:35 -03:00
Hsiao-Wei Wang 0b1f32ec25
Fix capella/block_processing tests 2022-11-03 16:08:05 -05:00
Potuz a09d617737 right ret 2022-11-03 15:07:03 -03:00
Potuz 329bafa6e2 dapplion's suggestions 2022-11-03 14:58:55 -03:00
Potuz 12404d0250 fix for loop 2022-11-03 14:56:38 -03:00
Hsiao-Wei Wang ad36548480
Fix auto-rebase errors 2022-11-03 11:46:07 -05:00
Potuz 39e6ec5d14
lint 2022-11-03 11:43:56 -05:00
Potuz a14479a70f
g11tech review 2022-11-03 11:43:56 -05:00
Potuz ff1dd90012
fix toc 2022-11-03 11:43:24 -05:00
Potuz 5b92eae08f
type annotation 2022-11-03 11:43:24 -05:00
Potuz 0f74ab5686
type annotation 2022-11-03 11:43:24 -05:00
Potuz c156ea6adc
linting and typo 2022-11-03 11:43:24 -05:00
Potuz 49a2519da2
lint 2022-11-03 11:43:24 -05:00
Potuz 7dbd50e958
Reviewers' comments
- Implemented many of Alex's comments including reinsertion of the
  withdrawal index in the BeaconState
- Implemented Sean's suggestion of separating the logic for block
  production so that one matches the list in the payload with what
`get_expected_withdrawals` returns
- Changed `get_expected_wihdrawals` to match the current behavior and
  moved it to `beacon-chain.md`
2022-11-03 11:42:31 -05:00
Potuz f506087af5
rebase on top of develop 2022-11-03 11:42:30 -05:00
Potuz d958ed70b4
Implement withdrawals without queues 2022-11-03 11:40:54 -05:00
George Kadianakis b2d72a18f0 Fix type error in the inputs to hash_to_bls_field() 2022-11-03 18:13:36 +02:00
Hsiao-Wei Wang 5a0d84ed01
int casting to avoid underflow 2022-11-03 10:21:09 -05:00
George Kadianakis 86e15764ad
EIP4844: Update cryptography API (#3038)
This commit changes the public API of the KZG library to the following high-level API:

```
- verify_kzg_proof()
- compute_aggregate_kzg_proof()
- verify_aggregate_kzg_proof()
- blob_to_kzg_commitment()
```

compared to the previous much more low-level API:

```
- compute_powers()
- matrix_lincomb()
- lincomb()
- bytes_to_bls_field()
- evaluate_polynomial_in_evaluation_form()
- verify_kzg_proof()
- compute_kzg_proof()
```

This means that all the cryptographic logic (including Fiat-Shamir) is now isolated and hidden in the KZG library and the `validator.md` file ends up being significantly simplified, only calling high-level KZG functions.

Some additional things that this commit does:

- Moves all EIP4844 cryptography into polynomial-commitments.md
- Improves the Fiat-Shamir stack by removing the need for SSZ and by introducing simple domain separators

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
Co-authored-by: Dankrad Feist <mail@dankradfeist.de>
2022-11-03 17:01:32 +02:00
Hsiao-Wei Wang 87dcb83827
Merge pull request #3063 from etan-status/lc-gossipdigests
Document how to derive fork context for LC gossip
2022-11-02 13:12:23 +08:00
Mark Mackey 826c20ffc2 Capella: Add Gossip Topic bls_to_execution_change 2022-11-01 12:54:33 -05:00
Hsiao-Wei Wang 498a5b0722
Merge pull request #3062 from terencechain/rm-signed-blobs
EIP4844: Remove signed blobs
2022-11-01 22:32:45 +08:00
Etan Kissling ff0e525ecc
Pass blocks to LC data creation functions
Future light client protocol extensions may include data from the block
in addition to data from the state, e.g., `ExecutionPayloadHeader`.
To prepare for this, also pass the block to the corresponding functions.
In practice, blocks access is easier than historic state access, meaning
there are no practical limitations due to this change.
2022-10-28 22:14:25 +02:00
Alex Stokes be3c774069
Merge pull request #3065 from etan-status/lc-branchrootdocs
Clarify where LC proofs are rooted
2022-10-28 08:51:26 -06:00
Etan Kissling 5b6eb8007a
Clarify where LC proofs are rooted
Add more detailed LC object documentation to explain that the various
merkle proofs are relative to the beacon block's state root.
Likewise, clarify that sync committees relate to the finalized header
(not to the optimistic header, which can be a period ahead).
2022-10-27 21:44:38 +02:00
Etan Kissling af54c97a6c
Consistently use `compute_sync_committee_period_at_slot`
A few LC functions were not yet updated to use a more concise function
for computing sync committee period for a slot. Updating to that func.
2022-10-27 21:36:55 +02:00
Etan Kissling c399e11aa7
Document how to derive fork context for LC gossip
For LC gossip, the documentation did not specify what slot number to use
for deriving the gossip objects. This missing documentation is now added
to document using `attested_header.slot`.
2022-10-27 21:23:09 +02:00
terence tsao 71775c6e87 EIP4844: Remove signed blobs 2022-10-27 08:10:32 -07:00
Jacek Sieka 4d1b487b21
move field last
avoids changing "header" fields in state
2022-10-27 10:38:09 +02:00
Jacek Sieka 1c8d57eeb7
Historical batches
This PR, a continuation of
replaces `historical_roots` with
`historical_block_roots`.

By keeping an accumulator of historical block roots in the state, it
becomes possible to validate the entire block history that led up to
that particular state without executing the transitions, and without
checking them one by one in backwards order using a parent chain.

This is interesting for archival purposes as well as when implementing
sync protocols that can verify chunks of blocks quickly, meaning they
can be downloaded in any order.

It's also useful as it provides a canonical hash by which such chunks of
blocks can be named, with a direct reference in the state.

In this PR, `historical_roots` is frozen at its current value and
`historical_batches` are computed from the merge epoch onwards.

After this PR, `block_batch_root` in the state can be used to verify an
era of blocks against the state with a simple root check.

The `historical_roots` values on the other hand can be used to verify
that a constant distributed with clients is valid for a particular
state, and therefore extends the block validation all the way back to
genesis without backfilling `block_batch_root` and without introducing
any new security assumptions in the client.

As far as naming goes, it's convenient to talk about an "era" being 8192
slots ~= 1.14 days. The 8192 number comes from the
SLOTS_PER_HISTORICAL_ROOT constant.

With multiple easily verifable blocks in a file, it becomes trivial to
offload block history to out-of-protocol transfer methods (bittorrent /
ftp / whatever) - including execution payloads, paving the way for a
future in which clients purge block history in p2p.

This PR can be applied along with the merge which simplifies payload
distribution from the get-go. Both execution and consensus clients
benefit because from the merge onwards, they both need to be able to
supply ranges of blocks in the sync protocol from what effectively is
"cold storage".

Another possibility is to include it in a future cleanup PR - this
complicates the "cold storage" mode above by not covering exection
payloads from start.
2022-10-27 10:33:38 +02:00
Hsiao-Wei Wang af8c5bf532
Merge pull request #3046 from terencechain/couple-block-sidecar
EIP4844: couple beacon block and blob sidecar for p2p
2022-10-26 23:51:23 +08:00
Hsiao-Wei Wang b7c72c3b42
Merge pull request #3047 from terencechain/reduce-min-epochs-blobs-reqs
EIP4844: modify `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` to 18 days
2022-10-26 23:26:45 +08:00
Inphi ca538f52a0
Update specs/eip4844/beacon-chain.md
Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
2022-10-25 12:06:07 -04:00
inphi 459310f744
Fix test_process_execution_payload 2022-10-24 16:25:50 -04:00
inphi 0488c0c26e
remove unchanged epoch processing section 2022-10-24 12:51:54 -04:00
inphi 2ac57c7e7b
Fix py setup
And remove the ENABLE_WITHDRAWALS feature-flag. The Testing section in
the spec has been updated to specify how withdrawals is to be disabled
2022-10-24 12:44:11 -04:00
Hsiao-Wei Wang 9844ea1f30
Merge pull request #3040 from ethereum/capella-test-vectors
Capella test vectors
2022-10-23 06:44:03 +08:00
terence tsao e2d0a4fa9a Fix ToC and test 2022-10-22 08:36:46 -07:00
terence tsao 9f4ae4c6ee Update validator spec with `SignedBeaconBlockAndBlobsSidecar` 2022-10-21 17:33:55 -07:00
Inphi f6f2474c87
Update specs/eip4844/beacon-chain.md
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-10-20 14:39:18 -04:00
inphi 242e1b73bb
Rebase Capella on EIP-4844
This also introduces an `ENABLE_WITHDRAWALS` feature-flag to allow
implementers test EIP-4844 without including Capella-specific state
changes.
2022-10-20 10:36:08 -04:00
Hsiao-Wei Wang 53b63cedc5
Merge branch 'dev' into pr3050 2022-10-19 10:21:32 -05:00
terence tsao dfa5ac8008 @protolambda's feedback, `SignedBlobsSidecar` -> `BlobsSidecar` 2022-10-18 11:34:14 -07:00
terencechain cab2e3ef9a
Describe sidecar and block are recieved together
Co-authored-by: protolambda <proto@protolambda.com>
2022-10-18 11:23:19 -07:00
terencechain 779d9be66a
Compare sidecar slot with block slot
Co-authored-by: protolambda <proto@protolambda.com>
2022-10-18 11:22:43 -07:00
Jim McDonald 0345b2284a
Add validator_index to Withdrawal. 2022-10-18 15:35:06 +01:00
terence tsao b128148a17 Modify `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` to 18 days 2022-10-17 16:24:19 -07:00
terence tsao 96b90200b7 Fix ToC 2022-10-17 16:08:59 -07:00
terence tsao eb5a2c2741 Couple beacon block and blob sidecar for p2p 2022-10-17 15:16:38 -07:00
Hsiao-Wei Wang b90436c988
Fix capella random & fork 2022-10-14 23:42:42 -05:00
ethosdev 24f8ec170b
Remove work-in-progress notes in Bellatrix specs (#3033)
* Remove the work-in-progress note in Bellatrix spec

Bellatrix is done and released.

* Remove work-in-progress notes in Bellatrix specs

* Remove work-in-progress notes in Bellatrix specs

* Remove work-in-progress notes in Bellatrix specs
2022-10-15 05:05:45 +08:00
Alex Stokes 092617ec1e
Remove duplicated definition of `Validator` 2022-10-09 17:02:50 -05:00
Hsiao-Wei Wang 3552e2f6e8
Merge pull request #3018 from ethereum/bump-dep
Bump dep packages version and fix lint issues
2022-10-07 00:21:51 +08:00
protolambda 63ab5996f4
eip4844: update tx_peek_blob_versioned_hashes to match tx type as defined in EIP PR 5707 (fee market update) 2022-10-06 18:13:24 +02:00
Alex Stokes 4e26813c81
Merge pull request #3023 from dapplion/patch-1
Typo in sync committee duties description
2022-10-05 20:10:12 -06:00
Lion - dapplion fda2a69ff7
Typo in sync committee duties description
I assume it refers to producing a signature for the last slot of phase0 fork
2022-10-02 12:19:55 +02:00
Ramana Kumar 807650e8a1
Fix some type annotations for blobs
blobs should not be Sequence[BLSFieldElement], it should be
Sequence[Sequence[BLSFieldElement]]. But we can be more specific and use
Sequence[Blob].
2022-10-01 15:25:47 +01:00
Danny Ryan 6b9e65fab6
Merge pull request #3019 from terencechain/fix-eip4844
EIP4844: various fixes
2022-09-30 10:51:42 -06:00
Danny Ryan 9dfb1897a4
Merge pull request #3004 from Inphi/inphi/eip4844-fee-mkt
EIP-4844: Update ExecutionPayloadHeader
2022-09-30 08:16:53 -06:00
terencechain 779e6e7b7e
Update specs/eip4844/p2p-interface.md
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
2022-09-29 13:43:28 -07:00
terence tsao 06c91e4843 Try a different format for epoch 2022-09-29 07:55:50 -07:00
terence tsao 93cb17dc7f Eip4844: various fixes 2022-09-29 07:36:39 -07:00
Hsiao-Wei Wang 44424c4080
Bump dep packages version and fix lint issues 2022-09-28 12:38:21 +08:00
Ramana Kumar 9ac605cc5e
Fix incorrect comment
bit-reversal permutation is not idempotent
2022-09-27 12:13:56 +01:00
George Kadianakis 6c2b46ae32
Merge pull request #3015 from asn-d6/bytes_to_bls_field
EIP4844: Introduce bytes_to_bls_field() helper
2022-09-26 23:55:16 +03:00
Danny Ryan eeebd42a95
Merge pull request #2998 from ethereum/remove-withdrawn-epoch
remove withdrawn_epoch
2022-09-26 14:30:25 -06:00
George Kadianakis d197ed1451 EIP4844: Introduce bytes_to_bls_field() helper
Improves separation between BLS cryptography and Ethereum SSZ logic.

Now the BLS library just implements bytes_to_bls_field(). Then hash_to_bls_field() does the Ethereum SSZ magic and
calls bytes_to_bls_field().
2022-09-26 19:01:01 +03:00
Dustin Brody b5bcec8d34
deprecate phase 0-only versions of beacon_blocks_by_{range,root} 2022-09-26 15:01:45 +00:00
George Kadianakis f4ba8b55ee
EIP4844: Implement reverse bit ordering in KZG commitments
Co-authored-by: Dankrad Feist <mail@dankradfeist.de>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-09-26 16:39:16 +03:00
Alex Stokes 189d61e386
Merge pull request #3008 from terencechain/patch-137
eip4844: signed_blobs_header -> signed_blobs_sidecar
2022-09-22 15:13:14 +02:00
George Kadianakis e4fdb8a0ad
Merge pull request #3005 from ethereum/eip4844-patch-dr
Fixing EIP-4844 function names
2022-09-22 12:59:42 +03:00
terencechain 1bb863b2a6
eip4844: signed_blobs_header -> signed_blobs_sidecar 2022-09-21 14:49:18 -07:00
Justin Traglia 5517729859 Fix TOC with check_toc 2022-09-21 16:41:33 -05:00
Justin Traglia 7066307a0f Fix section name for withdraw_balance 2022-09-21 16:36:27 -05:00
inphi ceee7bfbf1
EIP-4844: Fee market updates to execution 2022-09-21 09:14:46 -04:00
Justin Traglia 603e27f459 Fix state list lengths table 2022-09-20 15:24:14 -05:00
Justin Traglia 3bc7ff9f80 Fix a few things in Capella specs 2022-09-20 14:43:38 -05:00
Dankrad Feist b35155005b
Rename matrix_lincomb to vector_lincomb and lincomb to g1_lincomb 2022-09-19 20:16:19 +01:00
Dankrad Feist b63ed22588
Fix signature of compute_aggregated_poly_and_commitment 2022-09-19 20:10:48 +01:00
Danny Ryan 70f90c5296
rmove withdrawn_epoch 2022-09-19 11:39:46 -06:00
Ben Guidarelli d70dcd9926
Fix link to beacon-chain doc 2022-09-15 10:21:06 -04:00
Hsiao-Wei Wang 109250b963
Replace `hash_tree_root(x)` with `hash(ssz_serialize(x))` 2022-08-22 16:00:40 +08:00
Hsiao-Wei Wang d133dae471
Fix codespell errors (#2975) 2022-08-18 22:08:44 +08:00
terencechain cc9a5fe61e
EIP4844: fix format 2022-08-17 06:48:05 -07:00
Danny Ryan 5d0f4af063
Merge mainnet ttd and bellatrix values (#2969)
* merge mainnet ttd and bellatrix values

* Update configs/minimal.yaml

Co-authored-by: Paul Harris <paul.harris@consensys.net>

Co-authored-by: Paul Harris <paul.harris@consensys.net>
2022-08-15 08:00:14 -06:00
Danny Ryan 18dc61f4c5
Merge pull request #2961 from ethereum/execution_payload_tests
more execution payload tests and cleanup old ones
2022-08-04 07:44:53 -06:00
Danny Ryan 7e360240f9
more execution payload tests and cleanup old ones 2022-08-01 11:42:38 -06:00
Danny Ryan fb71509052
Merge pull request #2953 from potuz/overloaded-index
Do not overload index with WithdrawalIndex and ValidatorIndex
2022-08-01 11:15:37 -04:00
David Theodore c51e67a059 fixed typo in on_block() definition 2022-07-29 00:13:32 -07:00
Potuz 2c32fa6473 Do not overload index with WithdrawalIndex and ValidatorIndex 2022-07-26 17:34:02 -03:00
Etan Kissling 52a741f7c6
Improve grammar
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-25 15:46:29 +02:00
Etan Kissling 56363cd94a
Define libp2p protocol for light client sync
While the current Altair specs define structures to aid light client
development, one missing key aspect is the network protocol definition.

Certain implementations have started defining their own REST based APIs,
e.g., Lodestar at https://github.com/ChainSafe/lodestar/blob/master/packages/api/src/routes/lightclient.ts
While such APIs are useful, REST does not seem to be the ideomatic
choice as the sole API available at such a low level for Ethereum.

This patch introduces a libp2p based protocol to allow light clients to
sync to the latest `BeaconBlockHeader` in a trustless and decentralized
manner, building on top of prior work from:
- @hwwhww at https://github.com/ethereum/consensus-specs/pull/2267
- @jinfwhuang at https://github.com/ethereum/consensus-specs/pull/2786
- Lodestar's REST API (also has an endpoint to fetch merkle proofs!)
2022-07-22 17:56:41 +02:00
Etan Kissling 8ced2c3ae1
Remove newlines between items for visual consistency 2022-07-22 17:46:07 +02:00
Etan Kissling 78d336e28d
Describe light client sync process
Provides documentation about the process that a light client may use to
sync a local `LightClientStore`.
2022-07-22 17:12:57 +02:00
Hsiao-Wei Wang 0e82acfd61
Merge pull request #2945 from Inphi/eip4844-nit
Fix Polynomial container size
2022-07-22 19:48:02 +08:00
Hsiao-Wei Wang d717f7e4d1
Merge pull request #2947 from etan-status/lc-forceupdate
Manually trigger `LightClientStore` force updates
2022-07-22 19:42:55 +08:00
Etan Kissling 8643e28f12
`try_...` --> `process_...` 2022-07-22 11:49:07 +02:00
Michael de Hoog b2c647f2b2
Remove triple m 2022-07-21 11:11:30 -05:00
Etan Kissling f9d866eb28
Manually trigger `LightClientStore` force updates
Replaces `process_slot_for_light_client_store` which force updates the
`LightClientStore` automatically based on `finalized_header` age with
`try_light_client_store_force_update` which may be manually called based
on use case dependent heuristics if light client sync appears stuck.
Not all use cases share the same risk profile.
2022-07-21 16:06:46 +02:00
inphi 9016b92691
Fix Polynomial container size 2022-07-21 01:58:22 -04:00
Etan Kissling 95019bb9a9
Improve readability of intro text
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-20 17:50:18 +02:00
Etan Kissling 06d9fd8cd3
Add `LightClient(Finality|Optimistic)Update`
Introduces reduced `LightClientUpdate` structures to allow keeping track
of the latest `finalized_header` and `optimistic_header`. This may also
help in scheduling the next query for a full `LightClientUpdate` once
sync committee finality has been reached.
2022-07-20 13:14:58 +02:00
Hsiao-Wei Wang 1d2ef9f8cb
Merge pull request #2938 from etan-status/lc-testsuite
Add functions for deriving light client data
2022-07-20 14:47:05 +08:00
dankrad 02a2b71d64
WIP implementation of Danksharding (#2792)
* Rough structure

* Most of the KZG checks in

* Fixes suggested by Vitalik

* Add low degree check for commitments

* Remove -1 check from degree proof (not needed)

* Require block builders to be validators to simplify things

* remove verify_intermediate_block_bid_commitment

* Rename back to process_block

* Degree check formula corrections

* Updated TOC, bid processing corrections

* Link to latest sharding doc

* Add shard samples + P2P

* Add validator guide for attestations and reconstruction

* Update specs/sharding/beacon-chain.md

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: vbuterin <v@buterin.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: vbuterin <v@buterin.com>

* Refactor polynomial operations into separate file

* Add missing polynomial functions

* Fix polynomial commitment file toc levels

* Refactor the payload to make better use of unions

* Add reverse bit order convention

* Correct inequality in verify_degree_proof

* Small fix

* Fix polynomial evaluation

* Update specs/sharding/beacon-chain.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* MAX_BEACON_BLOCKS_BETWEEN_INTERMEDIATE_BLOCKS definition added

* Sample reconstruction estimate

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Fix return value of roots_of_unity()

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Intermediate block -> Builder block

* Some small omissions in intermediate -> builder

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Z1 -> inf_G1

* Fix degree proof bound

* SignedShardSample -> ShardSample

* Typo

* Don't allow 0 for `next_power_of_two`

* Remove unused `ROOT_OF_UNITY` constant

* Throwaway variable name

* Fix function documentation of `bls_modular_inverse`

* Builder block bid increase by at least 1%, no RANDAO processing in builder blocks

* Fix tocs

* Fix tocs

* Fix typo

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: vbuterin <v@buterin.com>
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2022-07-18 11:20:25 -06:00
Etan Kissling 0941114acf
Merge branch 'dev' into lc-testsuite 2022-07-18 18:19:07 +02:00
Hsiao-Wei Wang a1546fdf4f
Merge branch 'dev' into fix-capella-genesis 2022-07-18 14:48:02 +08:00
Hsiao-Wei Wang 647e9d0ec0
PR feedback from @ralexstokes 2022-07-18 14:45:00 +08:00
Etan Kissling c9f0927d8b
Merge branch 'dev' into lc-testsuite 2022-07-17 06:34:52 +02:00
Etan Kissling 63816debcc
Merge commit '9c798806355f67a32704a8cd5a98d90deccb8f69' into lc-testsuite 2022-07-15 21:38:18 +02:00
Hsiao-Wei Wang 185f51ec0e
Merge branch 'dev' into pr2937 2022-07-16 01:40:39 +08:00
Hsiao-Wei Wang ec980dae4c
Rework `blobs_sidecar` propagation guide 2022-07-16 01:13:31 +08:00
Hsiao-Wei Wang 400e1e54c1
Fix EIP4844 `create_genesis_state`, `EIP4844_FORK_VERSION`, and `config_fork_epoch_overrides`
`config_fork_epoch_overrides`: since Capella and EIP4844 are in parallel, need to check if the field exists

Update `compute_fork_version`
2022-07-16 01:13:25 +08:00
Hsiao-Wei Wang 808f9c7f7f
Use `validate_*` for the valdiation functions that return `None` 2022-07-16 00:54:01 +08:00
Hsiao-Wei Wang 4176781123
PR feedback from @djrtwo 2022-07-15 23:37:32 +08:00
Hsiao-Wei Wang a205e9314d
Make block proposal direction more clear 2022-07-15 23:33:31 +08:00
Hsiao-Wei Wang 9c79880635
Fix genesis testing tool and add Capella version `initialize_beacon_state_from_eth1` helper 2022-07-15 22:22:19 +08:00
Etan Kissling a126f22cd9
Sync tocs 2022-07-15 13:49:19 +02:00
Etan Kissling de15971ae8
Cleanups 2022-07-15 13:45:34 +02:00
Etan Kissling 88206117c6
Extract full node specifics to separate doc 2022-07-15 13:37:27 +02:00
Etan Kissling d06f44ef55
Move `sync-protocol.md` into `light-client` subdir 2022-07-15 13:22:14 +02:00
Etan Kissling 6146f01400
Flip order of `assert` and assignment
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-15 12:03:44 +02:00
Etan Kissling dbefbcf326
Explicitly mention optional `finalized_block` 2022-07-15 12:01:49 +02:00
Etan Kissling 95c07575b1
Hide `build_proof` and `get_backing` from spec 2022-07-15 11:54:47 +02:00
Etan Kissling 86fe93ca96
Add functions for deriving light client data
Adds `create_light_client_bootstrap` and `create_light_client_update`
functions as a reference implementation for serving light client data.
This also enables a new test harness to verify that light client data
gets applied to a `LightClientStore` as expected.
2022-07-14 09:36:09 +02:00
Etan Kissling 66325d0e9b
Use helper variables to make ifs more readable
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 14:08:51 +02:00
George Kadianakis 5b9bf41de6 EIP-4844: Further improvements on the spec
- Move constants around
- Implement missing functions to make the spec executable

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:13:30 +03:00
George Kadianakis 221f3687bd EIP-4844: Improve the cryptographic parts of the spec
- Move more code into polynomial-commitments.md
- Implement aggregated sidecar verification logic from PR #2915
- Rename `kzgs` to `kzg_commitments`

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:12:31 +03:00
Etan Kissling 6048f1abf6
More concise finality ranking 2022-07-12 07:37:12 +02:00
Etan Kissling 654970c605
Add `LightClientBootstrap`
Introduces a new `LightClientBootstrap` structure to allow setting up a
`LightClientStore` with the initial sync committee and block header from
a user-configured trusted block root.

This leads to new cases where the `LightClientStore` is only aware of
the current but not the next sync committee. As a side effect of these
new cases, the store's `finalized_header` may now  advance into the next
sync committee period before a corresponding `LightClientUpdate` with
the new sync committee is obtained, improving responsiveness.

Note that so far, `LightClientUpdate.attested_header.slot` needed to be
newer than `LightClientStore.finalized_header.slot`. However, it is now
necessary to also consider certain older updates to try and backfill the
`next_sync_committee`. The `is_better_update` helper is also updated to
improve `best_valid_update` tracking.
2022-07-12 07:27:00 +02:00
Etan Kissling b7a0e532e4
Prefix update `period` variables 2022-07-10 22:05:18 -07:00
Etan Kissling 2ddb3833ff
Extend documentation for forced updates 2022-07-08 22:25:33 -07:00
Etan Kissling 33f572c98d
More concise sync committee finality ranking
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:05:48 -07:00
Etan Kissling fa85648385
`==` on new line
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:04:05 -07:00
Etan Kissling 35f5010de7
More concise finality ranking
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:01:56 -07:00
Etan Kissling aa8caee2e3
Add `compute_sync_committee_period_at_slot` 2022-07-08 21:59:09 -07:00
Etan Kissling c67ad597f0
Reorder helpers to group ones operating on updates 2022-07-08 21:46:34 -07:00
Etan Kissling c4dbd241ac
Root `next_sync_committee` in `attested_header`
`LightClientUpdate` structures currently use different merkle proof root
depending on the presence of `finalized_header`. By always rooting it in
the same state (the `attested_header.state_root`), logic gets simpler.

Caveats:
- In periods of extended non-finality, `update.finalized_header` may now
  be outdated by several sync committee periods. The old implementation
  rejected such updates as the `next_sync_committee` in them was stale,
  but the new implementation can properly handle this case.
- The `next_sync_committee` can no longer be considered finalized based
  on `is_finality_update`. Instead, waiting until `finalized_header` is
  in the `attested_header`'s sync committee period is now necessary.
- Because `update.finalized_header > store.finalized_header` no longer
  holds (for updates with finality), an `is_better_update` helper is
  added to improve `best_valid_update` tracking (in the past, finalized
  updates with supermajority participation would always directly apply)

This PR builds on prior work from:
- @hwwhww at https://github.com/ethereum/consensus-specs/pull/2829
2022-07-01 14:49:24 -07:00
Etan Kissling f5f3031c5e
Allow passing redundant `next_sync_committee`
The producer of `LightClientUpdate` structures usually does not know how
far the `LightClientStore` on the client side has advanced. Updates are
currently rejected when including a redundant `next_sync_committee` not
advancing the `LightClientStore`. Behaviour is changed to allow this.
2022-06-29 13:16:03 +02:00
Hsiao-Wei Wang f810b6714b
Merge pull request #2924 from etan-status/lc-genesis
Allow `LightClientUpdate` with genesis finality
2022-06-29 13:12:58 +02:00
Etan Kissling 65cfedef22
Update finality update documentation 2022-06-27 22:36:55 +02:00
Etan Kissling 99e30328a8
Switch condition order for readability
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-06-27 22:26:35 +02:00
fradamt 67d62025d9
fix typo and specify something
- committee_index is used as an input to compute_subnet_for_attestation but it's not previously defined
- attestation.data.committee_index is incorrect, the field is "index"
2022-06-27 14:20:25 +02:00
Etan Kissling c10daf1709
Allow `LightClientUpdate` with genesis finality
When `state.finalized_checkpoint` references the genesis slot, it points
to an empty `root`, instead of the actual genesis block hash. This patch
updates the `LightClientUpdate` logic to allow including finality proofs
for genesis `finalized_checkpoint.root`, better supporting non-mainnet.
When including such a finality proof, the proof is for the empty `root`,
but `finalized_header` is kept zeroed out to signify this edge case.
2022-06-27 12:10:40 +02:00
Hsiao-Wei Wang a40a64440c
Merge pull request #2916 from etan-status/lc-forkversion 2022-06-27 06:38:11 +02:00
Hsiao-Wei Wang d4a2bdcea0
Merge pull request #2915 from asn-d6/consensus-4844-proofs-optimization
Optimizing EIP-4844 block validation (using KZG proofs)
2022-06-26 13:55:17 +02:00
Hsiao-Wei Wang a3339c7b7f
PR suggestion from @Inphi 2022-06-25 00:13:20 +08:00
Etan Kissling 840733ffd2
Move `compute_fork_version` to `fork.md` 2022-06-23 21:03:24 +02:00
Hsiao-Wei Wang 94da861d9c
Fix ToC 2022-06-23 18:44:37 +08:00
Hsiao-Wei Wang 0ab280d396
minor suggestions 2022-06-23 18:40:09 +08:00
George Kadianakis e7e520791d Use Python's pow() in bls_modular_inverse(). 2022-06-22 15:42:03 +03:00
George Kadianakis 534a9d1d95 Move EIP-4844 cryptography code to its own file 2022-06-22 15:42:03 +03:00
Hsiao-Wei Wang 8a70295a84
Fix ToC 2022-06-20 21:26:41 +08:00
Etan Kissling 0b8ab23bd4
Remove `fork_version` from `LightClientUpdate`
The `fork_version` field in `LightClientUpdate` can be derived from the
`update.signature_slot` value by consulting the locally configured fork
schedule. The light client already needs access to the fork schedule to
determine the `GeneralizedIndex` values used for merkle proofs, and the
memory layouts of the structures (including `LightClientUpdate`). The
`fork_version` itself is network dependent and doesn't reveal that info.
2022-06-16 17:17:24 +02:00
Hsiao-Wei Wang 8cc008d11c
Merge pull request #2805 from etan-status/lc-period
Allow light client to verify signatures at period boundary
2022-06-16 16:53:12 +02:00
George Kadianakis 4be0b9d6bc Add needed math/crypto functions to validate KZG aggregated proofs
All code pretty much straight up copied from https://github.com/ethereum/EIPs/pull/5088
2022-06-14 16:37:08 +03:00
George Kadianakis e4c75d67ce Introduce high-level logic of new efficient block validation 2022-06-14 16:37:08 +03:00
inphi dbac7b1582
EIP-4844: Remove shard field from BlobsSidecar 2022-06-13 16:32:18 -04:00
Danny Ryan 74489d5523
Partial withdrawals (#2862)
* t push base design for partial withdrawals

* moor tests

* clean up withdrawals naming

* make partial withdrawal randomized tests better

* Apply suggestions from code review

Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* fix mainnet brokn test

* name swap

* lint

Co-authored-by: Alex Stokes <r.alex.stokes@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-06-08 13:16:12 -06:00
Murphy Law a3128b8c57
EIP-4844: fix a typo (#2898) 2022-05-31 08:40:06 -06:00
Jacek Sieka 0e6a7cd39a
deprecate `BeaconBlocksByRange.step` (#2856)
* deprecate `BeaconBlocksByRange.step`

The `step` parameter has not seen much implementation in real life
clients which instead opt to request variations on a few epochs at a
time (instead of interleaving single blocks, entire epochs are
interleaved).

At the same time, supporting `step` on the server side brings several
complications: more complex bounds checking logic, more complex loading
of blocks from linear storage (which presumably stores all blocks and
not just certain increments).

This PR suggests that we deprecate the whole idea. Backwards
compatibility is kept by simply responding with a single block when
`step > 0` - this is allowed by the spec and should thus be handled
gracefully by requesting clients already, should there exist any that
use larger step counts.

Removing `step` now allows simplifying the EL-CL protocol for serving
execution data from the EL to avoid double storage.

* Update specs/phase0/p2p-interface.md

Co-authored-by: Danny Ryan <dannyjryan@gmail.com>

Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
2022-05-20 13:18:24 -06:00
Caspar Schwarz-Schilling 72f7b88f2a
Update PROPOSER_SCORE_BOOST to 40 percent (#2895) 2022-05-20 13:15:40 -06:00
Mikhail Kalinin d62ebaeaca
Run on_tick before picking the head for proposal (#2897) 2022-05-19 07:56:49 -06:00
Alex Stokes 5337da5dff
use python syntax for bitwise AND operation (#2894) 2022-05-17 07:14:19 -06:00
Jacek Sieka bab5e402df
Ignore subset aggregates (#2847)
* Ignore subset aggregates

When aggregates are propagated through the network, it is often the case
that a better aggregate has already been seen - in particular, this
happens when an aggregator has not been able to include itself in the
mesh and therefore publishes an aggregate with only its own
attestations.

This new ignore rule allows dropping all aggregates that are
(non-strict) subsets of aggregates that have already been seen on the
network. In particular, it does not mandate dropping aggregates where a
union of previous aggregates would cause it to become a subset).

The logic for allowing this is based on the premise that any aggregate
that has already been seen by a peer will also have been seen by its
neighbours - a subset aggregate (strict or not) brings no new value to
the aggregation algorithm, except in the extreme edge case where you
could combine several such sparse aggregates into a single, more dense
"combined" aggregate and thus use less block space.

Further, as a small benefit, computing the `hash_tree_root` of the full
aggregate is generally not done -however, `hash_tree_root(data)` is
already done for other purposes as this is used as index in the beacon
API.

* add subset ignore rule to sync contributions as well

* typo
2022-05-17 07:05:22 -06:00
Alex Stokes 20a90f1df7
Add new `DomainType` for application usage (#2884) 2022-05-16 08:08:15 -06:00
Danny Ryan af1fbab181
Merge pull request #2888 from casparschwa/update-proposer-score-boost
Update proposer score boost
2022-05-16 08:06:59 -06:00
terencechain ec7bb23697
bellatrix-p2p: fix a typo 2022-05-14 10:30:20 -07:00
Danny Ryan c0a8178fff
minor patch to bytes formatting in capella 2022-05-10 09:37:01 -06:00
Danny Ryan 242f286f6e
Merge pull request #2878 from michaelsproul/forkchoice-b4-propose
Require fork choice to run before proposal
2022-05-10 09:23:16 -06:00
Caspar Schwarz-Schilling 712a4f9cf1 Fix linting 2022-05-09 15:05:05 +02:00
Caspar Schwarz-Schilling f5303ca58e Update PROPOSER_SCORE_BOOST to 33 percent 2022-05-09 10:17:24 +02:00
Danny Ryan c1d0836936
Merge pull request #2855 from ethereum/00-to-01
00 to 01 cred change
2022-05-05 07:31:49 -06:00
Michael Sproul 3f76792ee3
Require fork choice to run before proposal 2022-05-05 11:21:55 +10:00
Etan Kissling 5653649ca8
Allow light client to verify signatures at period boundary
As the sync committee signs the previous block, the situation arises at
every sync committee period boundary, that the new sync committee signs
a block in the previous sync committee period. The light client cannot
reliably detect this condition (e.g., assume that this is the case when
it is currently on the last slot of a sync committee period), because
the last couple slots of a sync committee period may not have a block.

For example, when receiving a `LightClientUpdate` that is constructed
as in the following illustration, it is unknown whether `sync_aggregate`
was signed by the current or next sync committee at `attested_header`.

```

        slot N           N + 1   |            N + 2   (slot not sent!)
                                 |
  +-----------------+     \ /    |     +----------------+
  | attested_header | <--- X ----|---- | sync_aggregate |
  +-----------------+     / \    |     +----------------+
                        missed   |
                                 |
                          sync committee
                          period boundary
```

This patch addresses this edge case by including the slot at which the
`sync_aggregate` was created into the `LightClientUpdate` object.

Note that the `signature_slot` cannot be trusted beyond the purpose of
signature verification, as it could be manipulated to any other slot
within the same sync committee period and fork version, without making
the `sync_aggregate` invalid.
2022-05-02 13:08:20 +02:00
Etan Kissling f125538122
Fix param order in `prepare_execution_payload`
Documentation on how to call `prepare_execution_payload` had the params
for `safe_block_hash` and `finalized_block` hash flipped compared to the
function definition. Also updated tests for consistency.
2022-04-25 16:49:00 +02:00
Hsiao-Wei Wang 83ac38c183
Merge pull request #2872 from benjaminion/patch-1 2022-04-17 08:00:33 -07:00
Ben Edgington eca7237b27
Correct typo: BLSCommitment -> KZGCommitment
The consensus chain counterpart to https://github.com/ethereum/EIPs/pull/4992. The same comment from there applies here as well.
2022-04-13 15:04:55 +01:00
Hsiao-Wei Wang dcf41ef5e2 Merge branch 'dev' into eip4844-typo 2022-04-12 22:14:37 +08:00
Hsiao-Wei Wang 5978c86f3a
Typo fix: `ksg_to_version_hash` -> `kzg_to_versioned_hash` 2022-04-10 18:41:49 +08:00
Hsiao-Wei Wang 6576cab612 Fix Capella `prepare_execution_payload` 2022-04-06 18:13:39 +08:00
Danny Ryan 1bfefe301d
Merge pull request #2858 from mkalinin/justified-to-safe-block
Bellatrix: pass justified as a safe block
2022-03-30 12:19:21 -06:00
Danny Ryan c87fc48053
minor typo 2022-03-30 12:04:23 -06:00
Mikhail Kalinin b13a9f0f8a Apply suggestions as per review 2022-03-30 15:36:01 +06:00
terence tsao eb26a1d74c
EIP4844: fix a minor typo 2022-03-26 08:26:52 -07:00
Danny Ryan 19613ac7cd
Merge pull request #2854 from ralexstokes/update-0x01-creds
Update 0x01 withdrawal credentials.
2022-03-25 16:27:41 -06:00
Mikhail Kalinin bd66114f4a Clarify names and move get_safe_block_hash to safe-block.md 2022-03-26 02:02:38 +06:00
Mikhail Kalinin c97cc6f4dc Add separate get_safe_beacon_block function 2022-03-24 17:22:11 +06:00