Commit Graph

309 Commits

Author SHA1 Message Date
Hsiao-Wei Wang aeccd20fd1
Try multiprocessing 2023-05-06 16:34:14 +08:00
Hsiao-Wei Wang 5cb2733ed5
Add custom types `NodeID` and `SubnetID` and constant `NODE_ID_BITS` 2023-05-04 18:50:13 +08:00
Hsiao-Wei Wang 057517526e
Set python_requires=">=3.9" (#2964) 2023-05-03 12:25:04 +03:00
Hsiao-Wei Wang e117ff4d71
Merge branch 'dev' into eip6110-tests 2023-05-02 21:24:02 +08:00
Hsiao-Wei Wang e31fcbd6a9
Add `GetPayloadResponse` for `get_payload` API 2023-04-28 23:25:13 +08:00
Hsiao-Wei Wang 21d4370fd5
Add docs about how to add a new feature proposal in consensus-specs and online viewer support (#3239)
* Add docs

* update link to template

* Add more info

* Try mkdocs

* Create docs.yml

* Update docs.yml

* update

* update

* update

* Try mkdocs

* Add "B: Make it executable for pytest and test generator" section

* Use mkdocs-material

* Use `mkdocs-awesome-pages-plugin` to create custom specs order

* Add toc permalink

* Update site_url

* Add .pages files for navigations. Update highlight style

* Dark theme

* Fix list indent
2023-04-18 18:31:34 +08:00
Mikhail Kalinin 108f1eed86 Rebase EIP6110 to Deneb 2023-04-06 16:53:31 +06:00
Hsiao-Wei Wang 7476c1e0c9
Fix wrong doc path 2023-03-28 15:34:07 +08:00
Hsiao-Wei Wang cd7783e59d
EIP-6110: Fix `compute_fork_version` and add light client specs 2023-03-28 15:08:03 +08:00
Danny Ryan b53392b463
Merge pull request #3177 from mkalinin/deposits
In-protocol deposits flow (no queue approach)
2023-03-14 11:33:36 -06:00
dankrad 86fb82b221
Test generators for kzg-4844 libraries (#3274)
Arkworks integration and test generators for kzg-4844 libraries
2023-03-02 20:49:10 +00:00
Hsiao-Wei Wang a1daac098c
Make EIP-6110 executable and fix linter errors 2023-02-23 22:34:32 +08:00
Hsiao-Wei Wang f23ed0cdbc
Make linter happy 2023-02-16 21:11:18 +08:00
Hsiao-Wei Wang c2315c90a3
Merge branch 'dev' into pr3244 2023-02-16 21:07:36 +08:00
Jacek Sieka 5fe857b209
fixes 2023-02-16 09:20:40 +01:00
Jacek Sieka 24a19bb886
fixes
* fight the test suite
* clarify who orphans the block
* must supply all blobs of a block in range request
2023-02-16 09:12:34 +01:00
protolambda 902a9c9967
remerkleable: fix container dict key hashing 2023-02-10 00:29:59 +08:00
protolambda 832e96412c
fix container fork check with remerkleable v0.1.26 2023-02-10 00:29:59 +08:00
Paul Harris 317facbd64 Replaced EIP4844 references with Deneb
Fixes #3207
2023-01-23 15:08:34 +01:00
Etan Kissling 53a95f0007
Merge branch 'dev' into lc-eph 2023-01-10 13:43:15 +01:00
Danny Ryan ed1c6be132
Merge pull request #3185 from ethereum/fc-is_data_available
Move `is_data_available` check to fork-choice `on_block`
2023-01-06 10:55:29 -05:00
Hsiao-Wei Wang e154b3414c
Move `validate_blobs_sidecar` to fork-choice and add spec into execution spec scope 2023-01-06 23:19:44 +08:00
Etan Kissling 375436094c
Merge branch 'dev' into lc-eph 2023-01-05 13:53:29 +01:00
Hsiao-Wei Wang dba75eece9
Enable Capella feature in eip4844 fork
Fix lint
2023-01-04 01:40:46 +08:00
Hsiao-Wei Wang 1cfabcbe54
Rename `HistoricalBatchSummary` -> `HistoricalSummary` and `historical_batches` -> `historical_summaries` 2023-01-02 23:07:00 +08:00
Hsiao-Wei Wang 84592f101b
Add tests for `process_historical_batches_update` 2022-12-16 01:01:00 +08:00
Etan Kissling af785068d8
Merge branch 'el-rlp' into lc-eph 2022-12-13 12:27:59 +01:00
Etan Kissling 8ad6810a44
EIP4844 support (`excess_data_gas`), fork tests nyi 2022-12-12 13:11:49 +01:00
Etan Kissling 2e97af2627
Add `ExecutionPayloadHeader` to LC data
While the light client sync protocol currently provides access to the
latest `BeaconBlockHeader`, obtaining the matching execution data needs
workarounds such as downloading the full block.

Having ready access to the EL state root simplifies use cases that need
a way to cross-check `eth_getProof` responses against LC data.

Access to `block_hash` unlocks scenarios where a CL light client drives
an EL without `engine_newPayload`. As of Altair, only the CL beacon
block root is available, but the EL block hash is needed for engine API.

Other fields in the `ExecutionPayloadHeader` such as `logs_bloom` may
allow light client applications to monitor blocks for local interest,
e.g. for transfers affecting a certain wallet. This enables to download
only the few relevant blocks instead of every single one.

A new `LightClientStore` is proposed into the Capella spec that may be
used to sync LC data that includes execution data. Existing pre-Capella
LC data will remain as is, but can be locally upgraded before feeding it
into the new `LightClientStore` so that light clients do not need to run
a potentially expensive fork transition at a specific time. This enables
the `LightClientStore` to be upgraded at a use case dependent timing at
any time before Capella hits. Smart contract and embedded deployments
benefit from reduced code size and do not need synchronization with the
beacon chain clock to perform the Capella fork.
2022-12-12 00:48:40 +01:00
Etan Kissling 79c2099c9c
Merge branch 'dev' into el-rlp 2022-12-05 17:47:20 +01:00
Hsiao-Wei Wang edde563b3a
Workaround: ignore Invalid base class "ByteVector" error 2022-11-28 21:06:22 +08:00
Hsiao-Wei Wang b3a176689d
WIP. Fixing mypy errors 2022-11-28 21:01:41 +08:00
Hsiao-Wei Wang 4f32fd95d6
Enable EIP4844 lint and fix Pylint 2022-11-28 20:01:50 +08:00
Etan Kissling d51345c6b3
proper EL block hash implementation in tests
Replaces the "FAKE RLP HASH" in `ExecutionPayload.block_hash` of all
generated tests with the correct value.
2022-11-22 21:58:29 +01:00
Hsiao-Wei Wang 2f0157afad
Fix auto-rebase leftover 2022-11-18 03:20:25 +08:00
inphi a04f06bd11
Fix merge conflict 2022-11-10 13:29:33 -05:00
Hsiao-Wei Wang 95ee291f58
Merge branch 'dev' into pr3052 2022-11-07 10:10:16 -05:00
Hsiao-Wei Wang 3a49d6225e
Refactor `get_spec` 2022-11-03 10:22:43 -05:00
Hsiao-Wei Wang 7c016f3236
Add testing KZG trusted setups generator tool and load the file during building pyspec 2022-11-03 10:17:42 -05:00
George Kadianakis 86e15764ad
EIP4844: Update cryptography API (#3038)
This commit changes the public API of the KZG library to the following high-level API:

```
- verify_kzg_proof()
- compute_aggregate_kzg_proof()
- verify_aggregate_kzg_proof()
- blob_to_kzg_commitment()
```

compared to the previous much more low-level API:

```
- compute_powers()
- matrix_lincomb()
- lincomb()
- bytes_to_bls_field()
- evaluate_polynomial_in_evaluation_form()
- verify_kzg_proof()
- compute_kzg_proof()
```

This means that all the cryptographic logic (including Fiat-Shamir) is now isolated and hidden in the KZG library and the `validator.md` file ends up being significantly simplified, only calling high-level KZG functions.

Some additional things that this commit does:

- Moves all EIP4844 cryptography into polynomial-commitments.md
- Improves the Fiat-Shamir stack by removing the need for SSZ and by introducing simple domain separators

Co-authored-by: Kevaundray Wedderburn <kevtheappdev@gmail.com>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
Co-authored-by: Dankrad Feist <mail@dankradfeist.de>
2022-11-03 17:01:32 +02:00
Etan Kissling 9dc1a17b2d
Update `remerkleable` to 0.1.25
`remerkleable` was updated to address potentially incorrect computation
of `hash_tree_root` against default-initialized `Vector` objects.
Switching to the fixed version.
2022-10-28 20:40:21 +02:00
Hsiao-Wei Wang 317209591d
Make pyspec disable withdrawal-functions in EIP4844 2022-10-26 15:06:39 -05:00
inphi 2ac57c7e7b
Fix py setup
And remove the ENABLE_WITHDRAWALS feature-flag. The Testing section in
the spec has been updated to specify how withdrawals is to be disabled
2022-10-24 12:44:11 -04:00
Hsiao-Wei Wang 44424c4080
Bump dep packages version and fix lint issues 2022-09-28 12:38:21 +08:00
George Kadianakis f4ba8b55ee
EIP4844: Implement reverse bit ordering in KZG commitments
Co-authored-by: Dankrad Feist <mail@dankradfeist.de>
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-09-26 16:39:16 +03:00
Hsiao-Wei Wang 109250b963
Replace `hash_tree_root(x)` with `hash(ssz_serialize(x))` 2022-08-22 16:00:40 +08:00
Etan Kissling 56363cd94a
Define libp2p protocol for light client sync
While the current Altair specs define structures to aid light client
development, one missing key aspect is the network protocol definition.

Certain implementations have started defining their own REST based APIs,
e.g., Lodestar at https://github.com/ChainSafe/lodestar/blob/master/packages/api/src/routes/lightclient.ts
While such APIs are useful, REST does not seem to be the ideomatic
choice as the sole API available at such a low level for Ethereum.

This patch introduces a libp2p based protocol to allow light clients to
sync to the latest `BeaconBlockHeader` in a trustless and decentralized
manner, building on top of prior work from:
- @hwwhww at https://github.com/ethereum/consensus-specs/pull/2267
- @jinfwhuang at https://github.com/ethereum/consensus-specs/pull/2786
- Lodestar's REST API (also has an endpoint to fetch merkle proofs!)
2022-07-22 17:56:41 +02:00
Etan Kissling 78d336e28d
Describe light client sync process
Provides documentation about the process that a light client may use to
sync a local `LightClientStore`.
2022-07-22 17:12:57 +02:00
Etan Kissling c9f0927d8b
Merge branch 'dev' into lc-testsuite 2022-07-17 06:34:52 +02:00
Etan Kissling 9d41c2306f
Align format of override `config.yaml` with base cfg 2022-07-15 21:37:18 +02:00
Hsiao-Wei Wang 185f51ec0e
Merge branch 'dev' into pr2937 2022-07-16 01:40:39 +08:00
Hsiao-Wei Wang 4176781123
PR feedback from @djrtwo 2022-07-15 23:37:32 +08:00
Danny Ryan d49c98c7aa
Merge pull request #2926 from ralexstokes/spec-diagnostis
Add summary diagnostics to spec test gen
2022-07-15 09:01:32 -06:00
Etan Kissling 88206117c6
Extract full node specifics to separate doc 2022-07-15 13:37:27 +02:00
Etan Kissling d06f44ef55
Move `sync-protocol.md` into `light-client` subdir 2022-07-15 13:22:14 +02:00
Etan Kissling 95c07575b1
Hide `build_proof` and `get_backing` from spec 2022-07-15 11:54:47 +02:00
Etan Kissling 86fe93ca96
Add functions for deriving light client data
Adds `create_light_client_bootstrap` and `create_light_client_update`
functions as a reference implementation for serving light client data.
This also enables a new test harness to verify that light client data
gets applied to a `LightClientStore` as expected.
2022-07-14 09:36:09 +02:00
George Kadianakis 567a25f883 EIP-4844: Make the spec executable
- Implement all the required glue code to make things executable
- Implement a dummy KZG trusted setup

Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:14:05 +03:00
Etan Kissling 654970c605
Add `LightClientBootstrap`
Introduces a new `LightClientBootstrap` structure to allow setting up a
`LightClientStore` with the initial sync committee and block header from
a user-configured trusted block root.

This leads to new cases where the `LightClientStore` is only aware of
the current but not the next sync committee. As a side effect of these
new cases, the store's `finalized_header` may now  advance into the next
sync committee period before a corresponding `LightClientUpdate` with
the new sync committee is obtained, improving responsiveness.

Note that so far, `LightClientUpdate.attested_header.slot` needed to be
newer than `LightClientStore.finalized_header.slot`. However, it is now
necessary to also consider certain older updates to try and backfill the
`next_sync_committee`. The `is_better_update` helper is also updated to
improve `best_valid_update` tracking.
2022-07-12 07:27:00 +02:00
Hsiao-Wei Wang 4b5a4ad51f
Include Bellatrix p2p interface specs into pyspec 2022-06-30 22:12:09 +08:00
Alex Stokes 529ce301b8
write test generator diagnostics to a summary file 2022-06-29 12:09:56 -06:00
ChihChengLiang 87991c8fa2 bump milagro bls version 2022-05-14 19:47:46 +02:00
Danny Ryan c10d219664
Merge branch 'dev' into withdrawals-push 2022-03-22 08:11:09 -06:00
Danny Ryan 255e942f64
add validator and fork-choice mods for withdrawals in capella 2022-03-16 12:01:42 -06:00
Mikhail Kalinin 84ab086d94 Add safe_block_hash to notify_fc_updated 2022-03-14 19:09:21 +06:00
Hsiao-Wei Wang eef2dbb2b6 Ensure that the SSZ classes are ordered 2022-02-25 01:22:02 +08:00
Danny Ryan 2026103bb9
Merge branch 'dev' into withdrawals-push 2022-02-23 15:08:29 -07:00
Mikhail Kalinin eb4497fae7 Bellatrix: Rename execute_payload to notify_new_payload 2022-01-27 15:24:53 +06:00
Danny Ryan d5e4828aec
Merge pull request #2770 from sigp/opt-sync-2
Optimistic Sync
2022-01-25 10:26:29 -07:00
Danny Ryan a5b3c91f25
build opimistic sync file and fix a minor lint/typing issue 2022-01-20 09:25:02 -07:00
Alex Stokes 10603b71c0
indicate unused argument in utility function 2022-01-13 11:39:37 -08:00
Alex Stokes fd27d93898
add `pylint` to catch unused args 2022-01-13 11:27:11 -08:00
Jacek Sieka 46bc206740
Add `CONFIG_NAME` to configs
Runtime configurations apply to a certain network and the name of that
network is useful for humans such that they can talk about it.

Some of the existing configs already include a `CONFIG_NAME` toggle -
might as well add it here as well and avoid some confusion - this name
above all becomes useful in the beacon API.

By extension, the `CONFIG_NAME` config will appear in the beacon api as
a result of being defined here.
2021-12-29 13:02:36 +01:00
Hsiao-Wei Wang 171a9ccc81
Resolve conflicts 2021-12-27 18:30:12 +08:00
Hsiao-Wei Wang 8facc0a095
Merge -> Bellatrix 2021-12-23 14:25:43 +08:00
Danny Ryan 365b1f46f9
remove unnecessary definitions from CapellaSpecBuilder 2021-12-02 12:33:58 -07:00
Danny Ryan 4f24fe455b
basic capella withdrawal in place 2021-12-01 11:37:30 -07:00
Hsiao-Wei Wang 7de1495a42
Fix lint (#2750) 2021-11-30 06:38:42 -06:00
Hsiao-Wei Wang 79f1f16adf
Clean `PowBlock.difficulty` leftover 2021-11-16 22:08:00 +08:00
Mikhail Kalinin 0ad344bf72 Make notify_forkchoice_updated return payload id value 2021-11-04 15:58:48 +06:00
Alex Stokes 43da796b29
Update setup.py
Reflect changes to max transaction size in #2686.
2021-10-31 06:05:02 -07:00
Hsiao-Wei Wang 5b9652bf93
Merge branch 'dev' into get_pow_block-not-found 2021-10-26 21:37:08 +08:00
Danny Ryan 8a27a7cb5d
remove union type for eecution-payload txs 2021-10-19 15:46:18 -06:00
Danny Ryan 4bf88ba89e
Merge pull request #2683 from ethereum/remove-prepare-payload
remove prepare_payload
2021-10-19 09:41:44 -06:00
Hsiao-Wei Wang 34335e0334
Remove `prepare_payload` leftover 2021-10-19 15:30:49 +08:00
Hsiao-Wei Wang ba582b3e3a
Fix setup.py parser and rename `TBH_ACTIVATION_EPOCH` -> `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH` 2021-10-19 11:26:53 +08:00
Danny Ryan d5be6b5d68
remove prepare_payload in favor of a unification with notify_forkchoice_updated 2021-10-18 18:12:11 -06:00
Hsiao-Wei Wang cd5cf60deb
Clarify `get_pow_block` block-not-found case 2021-10-18 14:05:40 +08:00
protolambda 7e080c18f7
update remerkleable 2021-10-16 01:15:31 +02:00
Danny Ryan 130b81bf55
remove notify_consensus_validated in accordance with POS_CONSENSUS_VALIDATED removal from eip 3675 2021-10-10 10:47:59 -04:00
Jacek Sieka 38cf59a68c
show where code error happened, on error 2021-10-05 18:01:12 +03:00
Fredrik Svantes 0b808b5c5a
pip no longer invoked by an old script wrapper.
while debugging an error I encountered while running setup.py I also saw pip.main was used to installed packages. installing packages through pip.main (pip.main(["install", RUAMEL_YAML_VERSION])) is not recommended, and will be removed from future versions of pip.

https://github.com/pypa/pip/issues/5599
https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program
2021-09-24 16:33:14 +02:00
Danny Ryan 940d6b1cc5
Merge pull request #2613 from mkalinin/execution-engine-update
Update execution engine calls
2021-09-24 05:57:27 -06:00
Fredrik Svantes e0eec44e34
No longer importing "Paragraph"
removed Paragraph from being imported from marko.ext.gfm.elements as it is not being used
2021-09-24 12:45:37 +02:00
Mikhail Kalinin 41e06aec1b Introduce PayloadId type 2021-09-23 15:03:43 +06:00
Mikhail Kalinin 7d577ed422 Introduce ExecutionAddress type 2021-09-23 14:35:55 +06:00
Mikhail Kalinin 7ef938da77 Add notify_ prefix to EE functions sending notifications 2021-09-23 13:37:52 +06:00
Mikhail Kalinin 24bacafeee Update consensus_validated description 2021-09-22 23:52:15 +06:00
Mikhail Kalinin b530e01761 Update calls to execution_engine 2021-09-20 20:57:45 +06:00
Mikhail Kalinin 960a49afc9 Verify terminal PoW block after call to state_transition 2021-09-10 16:07:26 +06:00