Commit Graph

882 Commits

Author SHA1 Message Date
Jacek Sieka bf6ad41d7d add drop and sync committee metrics
* use storeBlock for processing API blocks
* avoid double block dump
* count all gossip metrics at the same spot
* simplify block broadcast
2021-10-20 18:20:12 +03:00
tersec c0a2f1c98e
refactor executionPayload tests; reduce HashSet creation (#3003) 2021-10-20 13:36:38 +02:00
zah 44d762119c
Client definitions for the sync committee REST APIs (#3002) 2021-10-19 19:44:05 +02:00
tersec a6ddb80779
refactor EF operations fixtures (#2998)
* refactor EF operations fixtures

* for phase0/altair, check completeness of operations directories
2021-10-19 16:38:41 +02:00
Jacek Sieka c40cc6cec1 clean up fork enum and field names
* single naming strategy
* simplify some fork code
* simplify forked block production
2021-10-19 11:06:38 +03:00
tersec ecf2dbbcf3
add Altair and Merge rewards tests (#2995) 2021-10-18 16:54:17 +02:00
Jacek Sieka 5b33783898
load altair state from database on startup (#2994)
* fixes replay from last phase0 block on startup
* better forked block loading
2021-10-18 14:32:54 +02:00
Eugene Kabanov 75a8fe3b2c
Increase REST server default limits. (#2991)
* Increase REST server default limits.

* Fix compilation error.

* Fix tests.
2021-10-18 09:14:44 +02:00
tersec b66e7c8573
v1.1.3 tests; re-enable minimal-preset altair transition tests (#2993) 2021-10-16 07:17:07 +00:00
Eugene Kabanov 7319f150e8
Number of REST fixes. (#2987)
* Initial commit.

* Fix path.

* Add validator keys to indices cache mechanism.
Move syncComitteeParticipants to common place.

* Fix sync participants order issue.

* Fix error code when state could not be found.
Refactor `state/validators` to use keysToIndices mechanism.

* Fix RestValidatorIndex to ValidatorIndex conversion TODOs.

* Address review comments.

* Fix REST test rules.
2021-10-14 12:38:38 +02:00
Etan Kissling 05908859b7
support merkle multiproof verification (#2980)
This introduces spec-compatible support to verify merkle multiproofs for
use, e.g., in light client syncing.
2021-10-14 10:28:22 +02:00
tersec 6cc8757930
update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990) 2021-10-14 06:30:21 +00:00
tersec 10981639f1
update 27 spec URLs to v1.1.2 (#2989) 2021-10-13 16:49:06 +00:00
Jacek Sieka f90b2b8b1f
reward accounting for altair+ (#2981)
Similar to the existing `RewardInfo`, this PR adds the infrastructure
needed to export epoch processing information from altair+. Because
accounting is done somewhat differently, the PR uses a fork-specific
object to extrct the information in order to make the cost on the spec
side low.

* RewardInfo -> EpochInfo, ForkedEpochInfo
* use array for computing new sync committee
* avoid repeated total active balance computations in block processing
* simplify proposer index check
* simplify epoch transition tests
* pre-compute base increment and reuse in epoch processing, and a few
other small optimizations

This PR introduces the type and does the heavy lifting in terms of
refactoring - the tools that use the accounting will need separate PR:s
(as well as refinements to the exportred information)
2021-10-13 16:24:36 +02:00
tersec 2eb9a608a4
add payloadId; add merge vector test script; remove consensusValidated (#2982) 2021-10-13 16:08:50 +02:00
Jacek Sieka 1da4192f3f
json-rpc: fix crashes on altair+ blocks in certain operations (#2983)
For the v2 interfaces, REST is recommeded instead
2021-10-13 13:20:18 +03:00
tersec 1f51331dd3
use specified execution_payload when constructing blocks; update Geth launch script (#2975)
* use specified execution_payload when constructing blocks; update Geth launch script

* revert DepositContractState change
2021-10-12 13:37:40 +02:00
Jacek Sieka 0c4bfb1911
test fixture cleanups (#2976)
* "official" -> "scenarios", like the submodule
* fewer test binaries - various compile hacks have been improved over
time, test suite should follow
* remove obsolete bls tests - there are better test vectors in
nim-blscurve
* remove obsolete mentions of `ssz_testing`
* remove obsolete comments about proc vs globals, unittest2 already uses
proc's
2021-10-12 13:36:52 +02:00
tersec 2ad1b7366a
update 62 spec URLs to v1.1.2 (#2979) 2021-10-12 10:17:37 +00:00
tersec 0ae736f397
update 67 spec URLs to v1.1.2 (#2977) 2021-10-12 08:09:59 +00:00
tersec 3c14e5728d
use v1.1.2 test vectors (#2968) 2021-10-08 11:36:35 +00:00
Etan Kissling ff5d9bfbfb
update defaultRuntimeConfig for v1.1.1 (#2958)
This synchronizes defaultRuntimeConfig with v1.1.1 of the spec.
- Introduces `TERMINAL_TOTAL_DIFFICULTY` and `TERMINAL_BLOCK_HASH`.
- Sets `ALTAIR_FORK_EPOCH` in `mainnet` preset.
2021-10-08 00:08:12 +03:00
Jacek Sieka fabec894dd
harden sync sub committee selection (#2965)
* harden sync sub committee selection

also turn it into an iterator

* fix test and warning
2021-10-07 13:19:47 +00:00
Etan Kissling 9ee134324b
allow `withXxx` to access fork-specific fields (#2943)
So far, `withState` and `withBlck` templates could only be used to have
convenience access to fork-agnostic BeaconState and BeaconBlock fields.
This patch:
- injects an additional `stateFork` constant that allows to use
  `when` expressions to also access Altair and Merge-specific fields.
- introduces a `withStateAndBlck` template to support operating on both
  a `BeaconState` and `BeaconBlock` at a time.
- makes sync committee related functions Merge aware.
- changes a couple if-else trees for forks into case statements so that
  forgotten future forks are promoted to compile-time errors.
2021-10-06 20:05:06 +03:00
Jacek Sieka a017e4a817
fix forkDigest 2021-10-06 18:47:11 +03:00
Eugene Kabanov 0fffdaf96d
Number of REST fixes. (#2961)
* Fix /eth/v1/config/spec response.
Cache /eth/v1/node/version response.

* Fix REST test rule.

* Workaround for `state_id` issue.

* Fix /eth/v1/events `head`, `chain_reorg` and `finalized_checkpoint` responses.

* Fix client configuration object.
2021-10-06 15:43:03 +02:00
Etan Kissling 55c2553193
enable uint128 / uint256 SSZ serialization tests (#2960)
So far, SSZ tests for `uint128` and `uint256` integers were disabled
due to "compile-time issues". The code has been adjusted to support
those additional test vectors, and the corresponding tests have been
enabled.
2021-10-06 06:14:52 +00:00
tersec 5cc2acb44e
use v1.1.1 test vectors (#2954) 2021-10-05 02:26:56 +00:00
Eugene Kabanov 65257b82f8
Validator key management API (#2755)
Implements https://github.com/ethereum/beacon-APIs/pull/151
2021-10-04 22:08:31 +03:00
Etan Kissling f10a2b18bb port EF light client tests
EF has a number of unit tests related to light client functionality, see
https://github.com/ethereum/consensus-specs/blob/v1.1.0/tests/core/pyspec/eth2spec/test/altair/unittests/test_sync_protocol.py
The tests have been ported over and integrated into our test suite.
2021-10-04 15:49:34 +03:00
Etan Kissling 2bbffbde10
abort compile when fork epoch is forgotten (#2939)
There are a few locations in the code that compare the current epoch to
the various FORK_EPOCH constants and branch off into fork-specific code.
When a new fork is introduced, it is sometimes forgotten to update all
of those branch locations. This patch introduces a compile-time check
that ensures that all branches need to be covered exhaustively. This is
done by replacing if-elif structures with case expressions.
2021-10-04 08:31:21 +00:00
Etan Kissling 2e9fa87f8b
use `SyncAggregate.init()` everywhere (#2932)
The initialization of a `SyncAggregate` to its default value is not very
intuitive. There is an `init` function in `sync_committee_msg_pool` that
provides a convenience wrapper. This patch exports that initializer so
that the rest of the code base can also take advantage of it.
2021-09-30 13:56:07 +00:00
tersec 6b3bf7eb7b
merge hardfork database support (#2911)
* merge hardfork database support

* working block_sim

* recreate state transition changes
2021-09-30 01:07:24 +00:00
tersec 47a6045b05
update a dozen spec URLs (#2929) 2021-09-29 23:43:28 +00:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
Etan Kissling 711c08f804
add function to build merkle proofs (#2874)
Adds a function that constructs a Merkle proof for a generalized index.
This will be used during light client sync to update light clients with
a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 15:02:34 +02:00
Etan Kissling e243ba2c0b
revise `makeBeaconBlock` overloads (#2879)
The phase0 and altair overloads of `makeBeaconBlock` slightly differ in
their signatures which makes using them unnecessarily verbose.
- A placeholder `sync_aggregate` argument similar to `executionPayload`
  is added to the phase0 overload to match the altair signature.
- A wrapper operating on `ForkedHashedBeaconState` is introduced.
2021-09-29 12:10:44 +00:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
Etan Kissling 6aff2cd19a add remaining REST spec references
This adds spec references for a few APIs that were missing them so far,
in line with how spec references already exist for the other REST APIs.
2021-09-29 14:14:56 +03:00
Tanguy 06df40454f
Improve discovery during fork (#2913) 2021-09-29 14:06:16 +03:00
tersec 1dc94aa36f
update 40 spec URLs to v1.1.0 (#2918) 2021-09-28 18:23:15 +00:00
Jacek Sieka e5346e4e95
simplify state fork access pattern (#2912)
* simplify state fork access pattern

* fixes

* unsafeAddr needs to be dereferenced outside of case for best effect
* remove hash_tree_root of ForkedXxx (doesn't make sense)
* simplify state transition

* fix vc

* readd hash_tree_root(forkedbeaconblock)

* readd htr(fhbs) as well

...and add some protections to not hash the wrong items elsewhere
2021-09-28 20:08:03 +02:00
Etan Kissling c510ffb16a
use `allSyncCommittees` everywhere (#2917)
There are still a few cases with manual loops through sync subcommittees
even though an `allSyncCommittees` iterator exists. Adjusted the few
remaining instances to also use the iterator instead.
2021-09-28 18:02:01 +00:00
tersec ca4c6b4c5c
update 30 spec URLs to v1.1.0 (#2914) 2021-09-28 14:01:46 +00:00
tersec aec5cf2b1b
update 31 spec reference URLs to v1.1.0 (#2910) 2021-09-28 07:46:06 +00:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
Eugene Kabanov 9d34d01cbd
Client SSZ API revisited. (#2907)
* Initial commit.

* Add SSZ getBlock().

* Automatic types detection for SSZ encoded objects.

* Change SSZ.decode() to readSszBytes().
2021-09-27 20:31:11 +02:00
tersec 2b2846b468
implement forked merge state/block support (#2890)
* implement forked state/block support

* merge support for containsOrphan; import cleanup; 80-column lines

* add merge block header operations and slot sanity fixture

* add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot()

* implement process_execution_payload() and add merge deposit operations tests

* add merge block sanity tests

* add merge case to syncCommitteeParticipants

* v1.1.0-beta.5 updates

* reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support

* add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
Etan Kissling 5bfff43785
make preset header comments consistent (#2905)
Currently there is a mix of comment formats in the various preset files.
Altair presets have a leading header line, Phase0 presets don't.
Furthermore, the Altair `minimal` header refers to `mainnet` (#2710).
Updated all of the header lines to match the spec.
2021-09-27 09:31:13 +00:00
Jacek Sieka e47a8cbe42
fixes (#2901)
* export kvstore from beacon_chain_db
* fix rest HashList deserialization
* fix asTrusted
2021-09-27 11:24:58 +02:00