Commit Graph

7542 Commits

Author SHA1 Message Date
dankrad 02a2b71d64
WIP implementation of Danksharding (#2792)
* Rough structure

* Most of the KZG checks in

* Fixes suggested by Vitalik

* Add low degree check for commitments

* Remove -1 check from degree proof (not needed)

* Require block builders to be validators to simplify things

* remove verify_intermediate_block_bid_commitment

* Rename back to process_block

* Degree check formula corrections

* Updated TOC, bid processing corrections

* Link to latest sharding doc

* Add shard samples + P2P

* Add validator guide for attestations and reconstruction

* Update specs/sharding/beacon-chain.md

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: vbuterin <v@buterin.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: vbuterin <v@buterin.com>

* Refactor polynomial operations into separate file

* Add missing polynomial functions

* Fix polynomial commitment file toc levels

* Refactor the payload to make better use of unions

* Add reverse bit order convention

* Correct inequality in verify_degree_proof

* Small fix

* Fix polynomial evaluation

* Update specs/sharding/beacon-chain.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* MAX_BEACON_BLOCKS_BETWEEN_INTERMEDIATE_BLOCKS definition added

* Sample reconstruction estimate

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Fix return value of roots_of_unity()

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/beacon-chain.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Intermediate block -> Builder block

* Some small omissions in intermediate -> builder

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>

* Z1 -> inf_G1

* Fix degree proof bound

* SignedShardSample -> ShardSample

* Typo

* Don't allow 0 for `next_power_of_two`

* Remove unused `ROOT_OF_UNITY` constant

* Throwaway variable name

* Fix function documentation of `bls_modular_inverse`

* Builder block bid increase by at least 1%, no RANDAO processing in builder blocks

* Fix tocs

* Fix tocs

* Fix typo

* Update specs/sharding/polynomial-commitments.md

Co-authored-by: George Kadianakis <desnacked@riseup.net>

Co-authored-by: Danny Ryan <dannyjryan@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: vbuterin <v@buterin.com>
Co-authored-by: George Kadianakis <desnacked@riseup.net>
2022-07-18 11:20:25 -06:00
Danny Ryan fed037fe89
Merge pull request #2940 from ethereum/fix-capella-genesis
Fix Capella genesis testing tool
2022-07-18 08:06:26 -06:00
Hsiao-Wei Wang a1546fdf4f
Merge branch 'dev' into fix-capella-genesis 2022-07-18 14:48:02 +08:00
Hsiao-Wei Wang 647e9d0ec0
PR feedback from @ralexstokes 2022-07-18 14:45:00 +08:00
George Kadianakis 9e4f386324
Merge pull request #2937 from asn-d6/eip4844-exe-squashed-and-rebased
Executable EIP-4844 specs (PR #2901 squashed and rebased)
2022-07-16 16:59:20 +03:00
Hsiao-Wei Wang ec7c7c8589
Remove leftover print 2022-07-16 01:41:56 +08:00
Hsiao-Wei Wang 185f51ec0e
Merge branch 'dev' into pr2937 2022-07-16 01:40:39 +08:00
Hsiao-Wei Wang 5356fee282
Fix `test_override_config_fork_epoch` 2022-07-16 01:28:19 +08:00
Hsiao-Wei Wang ec980dae4c
Rework `blobs_sidecar` propagation guide 2022-07-16 01:13:31 +08:00
Hsiao-Wei Wang 400e1e54c1
Fix EIP4844 `create_genesis_state`, `EIP4844_FORK_VERSION`, and `config_fork_epoch_overrides`
`config_fork_epoch_overrides`: since Capella and EIP4844 are in parallel, need to check if the field exists

Update `compute_fork_version`
2022-07-16 01:13:25 +08:00
Hsiao-Wei Wang 808f9c7f7f
Use `validate_*` for the valdiation functions that return `None` 2022-07-16 00:54:01 +08:00
Hsiao-Wei Wang 4176781123
PR feedback from @djrtwo 2022-07-15 23:37:32 +08:00
Hsiao-Wei Wang a205e9314d
Make block proposal direction more clear 2022-07-15 23:33:31 +08:00
Danny Ryan d49c98c7aa
Merge pull request #2926 from ralexstokes/spec-diagnostis
Add summary diagnostics to spec test gen
2022-07-15 09:01:32 -06:00
Hsiao-Wei Wang 9c79880635
Fix genesis testing tool and add Capella version `initialize_beacon_state_from_eth1` helper 2022-07-15 22:22:19 +08:00
Hsiao-Wei Wang 3b6222d74e
Merge pull request #2935 from etan-status/lc-bootstrap
Add `LightClientBootstrap`
2022-07-14 14:01:07 +08:00
Etan Kissling 66325d0e9b
Use helper variables to make ifs more readable
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 14:08:51 +02:00
George Kadianakis 2558282e6d EIP-4844: Introduce KZG and blob verification unittests
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:14:31 +03:00
George Kadianakis 567a25f883 EIP-4844: Make the spec executable
- Implement all the required glue code to make things executable
- Implement a dummy KZG trusted setup

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:14:05 +03:00
George Kadianakis 5b9bf41de6 EIP-4844: Further improvements on the spec
- Move constants around
- Implement missing functions to make the spec executable

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:13:30 +03:00
George Kadianakis 221f3687bd EIP-4844: Improve the cryptographic parts of the spec
- Move more code into polynomial-commitments.md
- Implement aggregated sidecar verification logic from PR #2915
- Rename `kzgs` to `kzg_commitments`

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:12:31 +03:00
Etan Kissling 6048f1abf6
More concise finality ranking 2022-07-12 07:37:12 +02:00
Etan Kissling 654970c605
Add `LightClientBootstrap`
Introduces a new `LightClientBootstrap` structure to allow setting up a
`LightClientStore` with the initial sync committee and block header from
a user-configured trusted block root.

This leads to new cases where the `LightClientStore` is only aware of
the current but not the next sync committee. As a side effect of these
new cases, the store's `finalized_header` may now  advance into the next
sync committee period before a corresponding `LightClientUpdate` with
the new sync committee is obtained, improving responsiveness.

Note that so far, `LightClientUpdate.attested_header.slot` needed to be
newer than `LightClientStore.finalized_header.slot`. However, it is now
necessary to also consider certain older updates to try and backfill the
`next_sync_committee`. The `is_better_update` helper is also updated to
improve `best_valid_update` tracking.
2022-07-12 07:27:00 +02:00
Hsiao-Wei Wang 739587b9c2
Merge pull request #2932 from etan-status/lc-attestedroot
Root `next_sync_committee` in `attested_header`
2022-07-12 13:19:04 +08:00
Etan Kissling b7a0e532e4
Prefix update `period` variables 2022-07-10 22:05:18 -07:00
Etan Kissling 2ddb3833ff
Extend documentation for forced updates 2022-07-08 22:25:33 -07:00
Etan Kissling 33f572c98d
More concise sync committee finality ranking
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:05:48 -07:00
Etan Kissling fa85648385
`==` on new line
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:04:05 -07:00
Etan Kissling 35f5010de7
More concise finality ranking
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-08 22:01:56 -07:00
Etan Kissling aa8caee2e3
Add `compute_sync_committee_period_at_slot` 2022-07-08 21:59:09 -07:00
Etan Kissling c67ad597f0
Reorder helpers to group ones operating on updates 2022-07-08 21:46:34 -07:00
Alex Stokes 4c54bddb6c
Merge pull request #2933 from ethereum/fix-test-reports-dir
Fix Makefile: use the correct `TEST_REPORT_DIR`
2022-07-06 12:37:46 -06:00
Hsiao-Wei Wang 55202a4f00
Use the correct `TEST_REPORT_DIR` 2022-07-04 23:06:42 +08:00
Etan Kissling c4dbd241ac
Root `next_sync_committee` in `attested_header`
`LightClientUpdate` structures currently use different merkle proof root
depending on the presence of `finalized_header`. By always rooting it in
the same state (the `attested_header.state_root`), logic gets simpler.

Caveats:
- In periods of extended non-finality, `update.finalized_header` may now
  be outdated by several sync committee periods. The old implementation
  rejected such updates as the `next_sync_committee` in them was stale,
  but the new implementation can properly handle this case.
- The `next_sync_committee` can no longer be considered finalized based
  on `is_finality_update`. Instead, waiting until `finalized_header` is
  in the `attested_header`'s sync committee period is now necessary.
- Because `update.finalized_header > store.finalized_header` no longer
  holds (for updates with finality), an `is_better_update` helper is
  added to improve `best_valid_update` tracking (in the past, finalized
  updates with supermajority participation would always directly apply)

This PR builds on prior work from:
- @hwwhww at https://github.com/ethereum/consensus-specs/pull/2829
2022-07-01 14:49:24 -07:00
Hsiao-Wei Wang bfa048a449
Merge pull request #2929 from etan-status/lc-testterms
Cleanup light client tests
2022-07-02 02:32:18 +08:00
Hsiao-Wei Wang e9bc03a223
Merge pull request #2930 from ethereum/bellatrix-p2p-pyspec
Include Bellatrix p2p interface specs into pyspec
2022-07-02 02:31:57 +08:00
Hsiao-Wei Wang 4b5a4ad51f
Include Bellatrix p2p interface specs into pyspec 2022-06-30 22:12:09 +08:00
Hsiao-Wei Wang d2a1f57e09
Merge pull request #2928 from ralexstokes/test-gen-patch-up
remove data logging from BLS test gen
2022-06-30 17:10:08 +08:00
Etan Kissling e269b305b9
Cleanup light client tests
Various cleanups and minor fixes:
- Consistent terminology:
  - `signed_block` -> `attested_block`
  - `finalized_block_header` -> `finalized_header`
  - `snapshot_period` -> `store_period`
- Use correct block in finality test (`blocks[-1]` instead of new one)
- Add `signed_block_header` func to get header from `SignedBeaconBlock`
- Remove `block_header` from `get_sync_aggregate` helper arguments
- Use `state_transition_with_full_block` as shortcut for multiple calls
- Have `finalized_header` actually be header instead of full block body
- Consistent ordering of `assert` to match structure definition
2022-06-30 11:09:49 +02:00
Hsiao-Wei Wang abcaccaea4
Merge pull request #2927 from etan-status/lc-redundantnext
Allow passing redundant `next_sync_committee`
2022-06-30 17:07:39 +08:00
Alex Stokes 529ce301b8
write test generator diagnostics to a summary file 2022-06-29 12:09:56 -06:00
Alex Stokes 2dd9a52fe4
remove data logging from BLS test gen 2022-06-29 10:24:53 -06:00
Etan Kissling f5f3031c5e
Allow passing redundant `next_sync_committee`
The producer of `LightClientUpdate` structures usually does not know how
far the `LightClientStore` on the client side has advanced. Updates are
currently rejected when including a redundant `next_sync_committee` not
advancing the `LightClientStore`. Behaviour is changed to allow this.
2022-06-29 13:16:03 +02:00
Hsiao-Wei Wang f810b6714b
Merge pull request #2924 from etan-status/lc-genesis
Allow `LightClientUpdate` with genesis finality
2022-06-29 13:12:58 +02:00
Etan Kissling 65cfedef22
Update finality update documentation 2022-06-27 22:36:55 +02:00
Etan Kissling 99e30328a8
Switch condition order for readability
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-06-27 22:26:35 +02:00
Hsiao-Wei Wang 0ba5b3b5c5
Merge pull request #2925 from fradamt/patch-3
fix typo and specify something
2022-06-27 16:18:34 +02:00
fradamt 67d62025d9
fix typo and specify something
- committee_index is used as an input to compute_subnet_for_attestation but it's not previously defined
- attestation.data.committee_index is incorrect, the field is "index"
2022-06-27 14:20:25 +02:00
Etan Kissling c10daf1709
Allow `LightClientUpdate` with genesis finality
When `state.finalized_checkpoint` references the genesis slot, it points
to an empty `root`, instead of the actual genesis block hash. This patch
updates the `LightClientUpdate` logic to allow including finality proofs
for genesis `finalized_checkpoint.root`, better supporting non-mainnet.
When including such a finality proof, the proof is for the empty `root`,
but `finalized_header` is kept zeroed out to signify this edge case.
2022-06-27 12:10:40 +02:00
Hsiao-Wei Wang a40a64440c
Merge pull request #2916 from etan-status/lc-forkversion 2022-06-27 06:38:11 +02:00