56363cd94a
Define libp2p protocol for light client sync
...
While the current Altair specs define structures to aid light client
development, one missing key aspect is the network protocol definition.
Certain implementations have started defining their own REST based APIs,
e.g., Lodestar at https://github.com/ChainSafe/lodestar/blob/master/packages/api/src/routes/lightclient.ts
While such APIs are useful, REST does not seem to be the ideomatic
choice as the sole API available at such a low level for Ethereum.
This patch introduces a libp2p based protocol to allow light clients to
sync to the latest `BeaconBlockHeader` in a trustless and decentralized
manner, building on top of prior work from:
- @hwwhww at https://github.com/ethereum/consensus-specs/pull/2267
- @jinfwhuang at https://github.com/ethereum/consensus-specs/pull/2786
- Lodestar's REST API (also has an endpoint to fetch merkle proofs!)
2022-07-22 17:56:41 +02:00
78d336e28d
Describe light client sync process
...
Provides documentation about the process that a light client may use to
sync a local `LightClientStore`.
2022-07-22 17:12:57 +02:00
c9f0927d8b
Merge branch 'dev' into lc-testsuite
2022-07-17 06:34:52 +02:00
9d41c2306f
Align format of override `config.yaml` with base cfg
2022-07-15 21:37:18 +02:00
185f51ec0e
Merge branch 'dev' into pr2937
2022-07-16 01:40:39 +08:00
4176781123
PR feedback from @djrtwo
2022-07-15 23:37:32 +08:00
d49c98c7aa
Merge pull request #2926 from ralexstokes/spec-diagnostis
...
Add summary diagnostics to spec test gen
2022-07-15 09:01:32 -06:00
88206117c6
Extract full node specifics to separate doc
2022-07-15 13:37:27 +02:00
d06f44ef55
Move `sync-protocol.md` into `light-client` subdir
2022-07-15 13:22:14 +02:00
95c07575b1
Hide `build_proof` and `get_backing` from spec
2022-07-15 11:54:47 +02:00
86fe93ca96
Add functions for deriving light client data
...
Adds `create_light_client_bootstrap` and `create_light_client_update`
functions as a reference implementation for serving light client data.
This also enables a new test harness to verify that light client data
gets applied to a `LightClientStore` as expected.
2022-07-14 09:36:09 +02:00
567a25f883
EIP-4844: Make the spec executable
...
- Implement all the required glue code to make things executable
- Implement a dummy KZG trusted setup
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
2022-07-13 13:14:05 +03:00
654970c605
Add `LightClientBootstrap`
...
Introduces a new `LightClientBootstrap` structure to allow setting up a
`LightClientStore` with the initial sync committee and block header from
a user-configured trusted block root.
This leads to new cases where the `LightClientStore` is only aware of
the current but not the next sync committee. As a side effect of these
new cases, the store's `finalized_header` may now advance into the next
sync committee period before a corresponding `LightClientUpdate` with
the new sync committee is obtained, improving responsiveness.
Note that so far, `LightClientUpdate.attested_header.slot` needed to be
newer than `LightClientStore.finalized_header.slot`. However, it is now
necessary to also consider certain older updates to try and backfill the
`next_sync_committee`. The `is_better_update` helper is also updated to
improve `best_valid_update` tracking.
2022-07-12 07:27:00 +02:00
4b5a4ad51f
Include Bellatrix p2p interface specs into pyspec
2022-06-30 22:12:09 +08:00
529ce301b8
write test generator diagnostics to a summary file
2022-06-29 12:09:56 -06:00
87991c8fa2
bump milagro bls version
2022-05-14 19:47:46 +02:00
c10d219664
Merge branch 'dev' into withdrawals-push
2022-03-22 08:11:09 -06:00
255e942f64
add validator and fork-choice mods for withdrawals in capella
2022-03-16 12:01:42 -06:00
84ab086d94
Add safe_block_hash to notify_fc_updated
2022-03-14 19:09:21 +06:00
eef2dbb2b6
Ensure that the SSZ classes are ordered
2022-02-25 01:22:02 +08:00
2026103bb9
Merge branch 'dev' into withdrawals-push
2022-02-23 15:08:29 -07:00
eb4497fae7
Bellatrix: Rename execute_payload to notify_new_payload
2022-01-27 15:24:53 +06:00
d5e4828aec
Merge pull request #2770 from sigp/opt-sync-2
...
Optimistic Sync
2022-01-25 10:26:29 -07:00
a5b3c91f25
build opimistic sync file and fix a minor lint/typing issue
2022-01-20 09:25:02 -07:00
10603b71c0
indicate unused argument in utility function
2022-01-13 11:39:37 -08:00
fd27d93898
add `pylint` to catch unused args
2022-01-13 11:27:11 -08:00
46bc206740
Add `CONFIG_NAME` to configs
...
Runtime configurations apply to a certain network and the name of that
network is useful for humans such that they can talk about it.
Some of the existing configs already include a `CONFIG_NAME` toggle -
might as well add it here as well and avoid some confusion - this name
above all becomes useful in the beacon API.
By extension, the `CONFIG_NAME` config will appear in the beacon api as
a result of being defined here.
2021-12-29 13:02:36 +01:00
171a9ccc81
Resolve conflicts
2021-12-27 18:30:12 +08:00
8facc0a095
Merge -> Bellatrix
2021-12-23 14:25:43 +08:00
365b1f46f9
remove unnecessary definitions from CapellaSpecBuilder
2021-12-02 12:33:58 -07:00
4f24fe455b
basic capella withdrawal in place
2021-12-01 11:37:30 -07:00
7de1495a42
Fix lint ( #2750 )
2021-11-30 06:38:42 -06:00
79f1f16adf
Clean `PowBlock.difficulty` leftover
2021-11-16 22:08:00 +08:00
0ad344bf72
Make notify_forkchoice_updated return payload id value
2021-11-04 15:58:48 +06:00
43da796b29
Update setup.py
...
Reflect changes to max transaction size in #2686 .
2021-10-31 06:05:02 -07:00
5b9652bf93
Merge branch 'dev' into get_pow_block-not-found
2021-10-26 21:37:08 +08:00
8a27a7cb5d
remove union type for eecution-payload txs
2021-10-19 15:46:18 -06:00
4bf88ba89e
Merge pull request #2683 from ethereum/remove-prepare-payload
...
remove prepare_payload
2021-10-19 09:41:44 -06:00
34335e0334
Remove `prepare_payload` leftover
2021-10-19 15:30:49 +08:00
ba582b3e3a
Fix setup.py parser and rename `TBH_ACTIVATION_EPOCH` -> `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH`
2021-10-19 11:26:53 +08:00
d5be6b5d68
remove prepare_payload in favor of a unification with notify_forkchoice_updated
2021-10-18 18:12:11 -06:00
cd5cf60deb
Clarify `get_pow_block` block-not-found case
2021-10-18 14:05:40 +08:00
7e080c18f7
update remerkleable
2021-10-16 01:15:31 +02:00
130b81bf55
remove notify_consensus_validated in accordance with POS_CONSENSUS_VALIDATED removal from eip 3675
2021-10-10 10:47:59 -04:00
38cf59a68c
show where code error happened, on error
2021-10-05 18:01:12 +03:00
0b808b5c5a
pip no longer invoked by an old script wrapper.
...
while debugging an error I encountered while running setup.py I also saw pip.main was used to installed packages. installing packages through pip.main (pip.main(["install", RUAMEL_YAML_VERSION])) is not recommended, and will be removed from future versions of pip.
https://github.com/pypa/pip/issues/5599
https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program
2021-09-24 16:33:14 +02:00
940d6b1cc5
Merge pull request #2613 from mkalinin/execution-engine-update
...
Update execution engine calls
2021-09-24 05:57:27 -06:00
e0eec44e34
No longer importing "Paragraph"
...
removed Paragraph from being imported from marko.ext.gfm.elements as it is not being used
2021-09-24 12:45:37 +02:00
41e06aec1b
Introduce PayloadId type
2021-09-23 15:03:43 +06:00
7d577ed422
Introduce ExecutionAddress type
2021-09-23 14:35:55 +06:00
7ef938da77
Add notify_ prefix to EE functions sending notifications
2021-09-23 13:37:52 +06:00
24bacafeee
Update consensus_validated description
2021-09-22 23:52:15 +06:00
b530e01761
Update calls to execution_engine
2021-09-20 20:57:45 +06:00
960a49afc9
Verify terminal PoW block after call to state_transition
2021-09-10 16:07:26 +06:00
0bd3e0286b
Clean up startswith method in setup.py
...
`value.startswith(("uint", "Bytes", "ByteList", "Union"))`
2021-08-27 10:12:19 +09:00
b262854bb1
Rename the `eth2_*` functions to `eth_`
2021-08-03 21:51:02 +08:00
17fad2dea2
rebase The Merge onto Altair base functionality
2021-07-22 16:36:41 +02:00
947ca9082c
Merge pull request #2522 from mkalinin/strict-terminal-pow-block
...
Enforce terminal PoW block to be on the cusp
2021-07-20 08:28:46 -06:00
65f6aa1b33
Replace PyUnion with Optional
2021-07-20 17:37:52 +06:00
5ad36bd3d5
Update remerkleable to v0.1.22: list lookup speed improvement
2021-07-17 15:55:38 +02:00
add00ad2e0
Replace get_pow_chain_head with get_pow_block_at_total_difficulty in validator.md
2021-07-17 12:33:06 +06:00
a4a050e97a
Enforce terminal PoW block to be on the cusp
2021-07-16 19:16:32 +06:00
199d6aee58
Merge remote-tracking branch 'origin/dev' into randao_in_execution_payload
2021-06-19 00:28:38 +06:00
878b15df6a
polish merge/beacon-chain.md ( #2472 )
...
Polish `merge/beacon-chain.md` with mostly non-substantive changes.
**Non-substantive changes**
* rename `MAX_EXECUTION_TRANSACTIONS` to `MAX_TRANSACTIONS_PER_PAYLOAD`
- rename "execution transaction" to just "transaction" as per discussion with Danny
* rename `compute_time_at_slot` to `compute_timestamp_at_slot`
- the function returns a Unix timestamp
- "timestamp" matches `execution_payload.timestamp`
* be explicit about `ExecutionEngine.execution_state` for clarity
* rename `ExecutionPayload.number` to `ExecutionPayload.block_number`
- more specific ("number" is pretty vague)
- consistent with `ExecutionPayload.block_hash`
* rename `new_block` to `on_payload`
- the `on_` prefix is consistent with other event handlers (e.g. see `on_tick`, `on_block`, `on_attestation` [here](https://github.com/ethereum/eth2.0-specs/blob/dev/specs/phase0/fork-choice.md#handlers ))
- the `_payload` suffix is more to the point given the function accepts an `execution_payload`
- avoids conflict with `on_block` which is already used in the fork choice
* rework the table of contents for consistency
* order `is_execution_enabled` after `is_transition_completed` and `is_transition_block`
- `is_execution_enabled` refers to `is_transition_completed` and `is_transition_block`
* rename "transition" to "merge"
- "transition" is a bit vague—we will have other transitions at future hard forks
- there is no need for two words to refer to the same concept
* add a bunch of inline comments, e.g. in `process_execution_payload`
* make the `process_execution_payload` signature consistent with the other `process_` functions in `process_block` which take as arguments `state` and `block.body`
* remove `TRANSITION_TOTAL_DIFFICULTY`
- to be put in `merge/fork-choice.md` where it is used
* various misc cleanups
**Substantive changes**
* reorder `ExecutionPayload` fields
- for consistency with yellow paper and Eth1
- same for `ExecutionPayloadHeader`
- added comments separating out the execution block header fields from the extra fields (cosmetic)
2021-06-18 11:09:30 +01:00
6e86d8a696
Rename randao->random, other fixes as per review
2021-06-17 21:20:17 +06:00
cc20b80103
Fix linter, add more test helpers
2021-06-15 14:55:06 +06:00
5d9d786499
Merge pull request #2462 from mkalinin/merge-transition-with-dynamic-ttd
...
Merge transition process with computed transition total difficulty
2021-06-08 09:55:10 -06:00
6350e27366
Add difficulty field to PowBlock
2021-06-08 17:56:07 +06:00
8ba740a93a
Merge branch 'dev' into invariant-checks
2021-06-07 08:12:27 -06:00
30f2a07637
Merge pull request #2438 from ralexstokes/add-aggregate-pks-fn
...
Move BLS extensions to new file in Altair
2021-06-07 07:18:31 -06:00
049c18b6c4
Bump mypy to v0.812 (latest)
2021-06-03 20:49:29 +08:00
989d33f6ec
Bump mypy to v0.800. This release adds Python 3.9 support.
2021-06-03 17:30:43 +08:00
6226be9e1e
Implement TransitionStore and transition total difficulty computation
2021-06-01 16:28:30 +06:00
fd4369dc7c
Add merge/fork.md with upgrade_to_merge definition
2021-05-31 19:30:54 +06:00
8f005c18e5
when an untyped var is not the last config var, it needs a comma
2021-05-31 15:20:50 +02:00
d11586122f
update remerkleable, union support
2021-05-28 01:02:08 +02:00
6ecbc5e357
Add new file to spec infra
2021-05-27 12:21:04 -07:00
7a14e93c65
PR feedback
2021-05-27 12:02:44 -07:00
69f2a31406
Add some config invariant checks
2021-05-26 01:10:40 +08:00
74761db7a3
Fix lint error
2021-05-25 21:40:10 +08:00
fc1af1cff3
[pyspec] Use mainnet.py as the default spec
2021-05-25 21:13:12 +08:00
41ca149429
Explicitly define `bls.AggregatePKs`
2021-05-24 11:56:10 -07:00
1636a1df58
reduce duplicate combination code, and no in-place updates
2021-05-19 19:18:42 +02:00
291168e1e3
clarify config/preset separation, address review comments by @djrtwo
2021-05-19 17:15:34 +02:00
f5c647b47b
switch configuration to named tuple for reliable hashing, add test for config override functionality
2021-05-18 16:08:30 +02:00
e8b0c46138
misc test fixes for new config approach
2021-05-18 14:45:56 +02:00
0894125bf7
update test util
2021-05-18 13:59:26 +02:00
ccc6679e21
config as dataclass
2021-05-18 12:48:42 +02:00
6f68913e11
implement pyspec build targets
2021-05-18 12:12:01 +02:00
79d0fa037f
updated config util
2021-05-18 09:55:25 +02:00
4cd9a6c710
Merge pull request #2385 from ethereum/execution-payload-testing
...
Execution payload tests
2021-05-14 06:40:40 -06:00
0390ab819a
Protocols pyspec support + execution payload tests cleanup
2021-05-12 02:40:23 +02:00
2539d4e11d
Merge pull request #2380 from ethereum/merge-test-vectors
...
Merge test vectors: enable phase0 tests for Merge + start on new testing
2021-05-10 07:30:21 -06:00
e78e045847
Implement execution payload tests
2021-05-10 16:12:23 +06:00
4b27b076f6
add missing comma
2021-05-06 12:36:08 -06:00
953b0278a1
whitespace
2021-05-06 10:52:48 -07:00
4c73fec88e
convert `LightClientStore` to python object instead of SSZ object
...
this avoids the type overhead of having to define a max size for
the object's data and skips the overhead of serialization/consensus
for a type that does not need it
2021-05-06 10:52:48 -07:00
cc11328f74
fix merge forkchoice tests with mock get_pow_block
2021-05-05 16:26:41 +02:00
58e768392e
Simplify spec comment processing
2021-04-23 08:03:49 -07:00
a9e3ecabbd
demo spec comment feature
2021-04-21 18:49:51 -07:00
de3ac15d9c
introduce alternate spec parser with spec comments
2021-04-21 18:49:51 -07:00
f2df4bf5c8
Merge branch 'dev' into refactor-spec-builder2
2021-04-19 23:14:30 +08:00
9e4f5c1879
changed alphabet string to string module.
...
changed alphabet string to string module.
2021-04-19 22:41:45 +09:00
5349645a8f
Rename `SpecAdjustment` to `SpecBuilder` and add `build_spec` interface
2021-04-16 11:29:10 +08:00
e8c3d28a86
Merge branch 'dev' into refactor-spec-builder2
2021-04-16 11:03:57 +08:00
2ac19be198
Merge pull request #2331 from BenSchZA/BenSchZA-patch-1
...
Update variable name ALTAIR_INVAIANT_CHECKS to INVARIANT in `setup.py`
2021-04-15 07:20:14 -05:00
2baeee463d
Merge pull request #2322 from mkalinin/merge-cleanup
...
Merge cleanups and improvements
2021-04-15 07:17:55 -05:00
35fea7a11b
Update copy INVAIANT to INVARIANT
2021-04-14 21:37:25 +02:00
320172fb08
fix lint
2021-04-14 08:02:09 -05:00
18fc7c0498
Merge pull request #2311 from ralexstokes/add-altair-val-tests
...
Add some basic tests for Altair validator guide
2021-04-12 15:19:44 -07:00
4bf6331c21
update remerkleable
2021-04-12 15:11:21 -07:00
79230c5f68
Update pyspec builder
2021-04-10 03:04:27 +08:00
3320ebb865
Fix typo and add `SpecAdjustment.fork` property
2021-04-09 22:39:12 +08:00
ceb352be12
Refactor `imports_and_predefinitions` into `imports` and `preparations`
2021-04-09 22:17:01 +08:00
1ffa436836
Update `imports_and_predefinitions`
2021-04-09 21:28:58 +08:00
658ede2191
Refactor pyspec builder with `SpecAdjustment` classes
2021-04-09 21:12:05 +08:00
af9ea0efb6
Merge branch 'dev' into merge-exe
2021-04-09 07:21:51 -05:00
6e8b4b3ea9
Add eth2spec.merge.spec
2021-04-08 05:57:16 +08:00
0bbef67efe
Add ALTAIR_INVAIANT_CHECKS, checks the weight configurations
2021-04-06 00:08:30 +08:00
1acb1d6f70
disable phase1 pyspec build
2021-03-26 22:37:32 +01:00
2ce389216d
Executable Altair validator guide
2021-03-19 23:38:59 +08:00
32d0c8087f
Fix `NEXT_SYNC_COMMITTEE_INDEX`
2021-03-15 23:17:59 +08:00
a2268a432c
Fix missing --cov-report
2021-03-15 19:20:36 +08:00
310301236f
Fix typo
2021-03-12 20:29:28 +08:00
22fe06829b
Refactor `objects_to_spec`
2021-03-12 20:26:26 +08:00
c34eae33d0
Merge branch 'dev' into lightclient-sync-exe
2021-03-12 00:49:29 +08:00
b44e576e95
LIGHTCLIENT_PATCH/HF1 -> ALTAIR
2021-03-11 21:32:00 +08:00
c36106e630
put snappy in 'generator' extra dep build and use for generator builds
2021-03-10 17:18:11 -07:00
813608928c
Merge branch 'dev' into tests-compression
2021-03-10 16:43:02 -07:00
36b6f8c3bf
Merge branch 'dev' into lightclient-sync-exe
2021-03-10 16:25:01 -07:00
fab27d17f0
Merge pull request #2223 from ethereum/bump-py_ecc
...
Bump py_ecc to v5.2.0
2021-03-10 16:19:14 -07:00
91f6956b3a
Bump py_ecc to 5.2.0
2021-03-10 21:55:50 +08:00
6bd1efc73b
rename fork files
2021-03-08 17:16:13 -07:00
6df3de8506
Merge pull request #2199 from ethereum/turn_off_phase1_testgen
...
[testgen] turn off phase 1, turn on HF1, and refactoring
2021-02-22 13:28:19 -06:00
a28b2d7349
Bump Milagro dependency for M1 support
2021-02-20 15:29:34 +00:00
e58dcb40ac
Clean up and kick the cache
2021-02-18 17:51:01 +08:00
844c879f1c
Bump `py_ecc` and `milagro_bls_binding` ( #2169 )
...
* Bump py_ecc to v5.1.0 and milagro_bls_binding to v1.6.2
* python3.8 -> python3 for py39 compatibility
* fix
* Try python:3.9
* Revert: using Python3.8 in CI now
2021-01-05 21:00:52 +08:00
d01a4ad823
Fix depth calculation...again(!) and add unittests
2020-12-16 15:01:22 +08:00
bcde37c39f
Make `sync-protocol.md` pass the linter
2020-12-15 13:24:43 +08:00
acfe49e3f3
executable light client patch: beacon-chain.md ( #2141 )
...
* Bump remerkleable to 0.1.18
* Disable `sync-protocol.md` for now. Make linter pass
* Enable lightclient tests
* Use *new* `optional_fast_aggregate_verify`
* Fix ToC and codespell
* Do not run phase1 tests with Lightclient patch
* Fix the Eth1Data casting bug. Add a workaround.
* Fix `run_on_attestation` testing helper
* Revert
* Rename `optional_fast_aggregate_verify` to `eth2_fast_aggregate_verify`
* Apply Proto's suggestion
* Apply Danny's suggestion
* Fixing tests
* Fix after rebasing
* Rename `LIGHTCLIENT` -> `LIGHTCLIENT_PATCH`
* New doctoc
* Add lightclient patch configs
* fix gitignore light client patch generator output
* Upgrade state for light client patch
* Add `lightclient-fork.md` to deal the fork boundary and fix
`process_block_header`
* Misc cleanups
1) Add a summary note for every function that is changed.
2) Avoid changing `process_block` (instead only change `process_block_header`).
3) Rename `G2_INFINITY_POINT_SIG` to `G2_POINT_AT_INFINITY` to avoid `SIG` contraction.
4) Misc cleanups
* Update block.py
* Update beacon-chain.md
* Fix typo "minimal" -> "mainnet"
Co-authored-by: Marin Petrunić <mpetrunic@users.noreply.github.com>
* Use the new `BeaconBlockHeader` instead of phase 0 version
* Update config files
* Move `sync_committee_bits` and `sync_committee_signature` back to `BeaconBlockBody`
Co-authored-by: protolambda <proto@protolambda.com>
Co-authored-by: Justin <drakefjustin@gmail.com>
Co-authored-by: Marin Petrunić <mpetrunic@users.noreply.github.com>
2020-12-15 13:18:20 +08:00
4df3547edf
Make `lightclient` patch pass the linter ( #2133 )
...
* Make `lightclient` an executable patch fork
* fix conflicts
* Fix ToC
* Lightclient -> Light client
* Try protolambda/remerkleable#8
* Fix sync-protocol.md ToC
* Build lightclient/sync-protocol
* Fix typo
Co-authored-by: vbuterin <v@buterin.com>
Co-authored-by: vbuterin <v@buterin.com>
2020-11-17 10:41:26 +08:00
00a19e583e
Bump milagro_bls_binding to 1.5.0
2020-10-06 16:17:29 +08:00
4d3ac72473
Bump milagro_bls_binding to `1.4.0`, handle the exception cases
2020-10-05 21:55:49 +08:00
4613c6b333
Bump py_ecc to 5.0.0
2020-10-01 00:44:47 +08:00
68bcc19e09
Merge pull request #1957 from ethereum/testgenphase1
...
Enable test generation for phase1
2020-09-18 08:44:38 -06:00
177f88962e
Merge pull request #2073 from ethereum/weak-subjectivity-guide
...
Weak subjectivity guide
2020-09-17 12:07:19 -06:00
8bb05827ab
add weak-subjectivity.md to setup.py build and do final lint cleanups
2020-09-17 12:01:25 -06:00
5374890da0
Merge branch 'dev' into testgenphase1
2020-09-17 21:11:49 +08:00
28137a6176
Merge branch 'dev' into testgenphase1
2020-09-09 23:43:26 +02:00
6e900f4fcc
Add more typings, no substantive change
2020-09-03 19:27:16 +08:00
Etan Kissling
Hsiao-Wei Wang
Danny Ryan
George Kadianakis
Alex Stokes
ChihChengLiang
Mikhail Kalinin
Jacek Sieka
protolambda
Fredrik Svantes
Sam.An
Justin
BenSchZA
Hsiao-Wei Wang