Commit Graph

2115 Commits

Author SHA1 Message Date
Etan Kissling f10a2b18bb port EF light client tests
EF has a number of unit tests related to light client functionality, see
https://github.com/ethereum/consensus-specs/blob/v1.1.0/tests/core/pyspec/eth2spec/test/altair/unittests/test_sync_protocol.py
The tests have been ported over and integrated into our test suite.
2021-10-04 15:49:34 +03:00
Etan Kissling 2bbffbde10
abort compile when fork epoch is forgotten (#2939)
There are a few locations in the code that compare the current epoch to
the various FORK_EPOCH constants and branch off into fork-specific code.
When a new fork is introduced, it is sometimes forgotten to update all
of those branch locations. This patch introduces a compile-time check
that ensures that all branches need to be covered exhaustively. This is
done by replacing if-elif structures with case expressions.
2021-10-04 08:31:21 +00:00
Etan Kissling b217150f1d
use forked `getAttestationsForBlock` everywhere (#2937)
There are a number of locations in the code that get attestations on a
forked beacon state. For attestation pools test, a convenience wrapper
was available to reduce clutter. This patch integrates that wrapper into
the core component so that it can also take advantage of the wrapper.
2021-10-01 01:29:32 +00:00
Etan Kissling ba84a55699
fix `makeBeaconBlockForHeadAndSlot` for merge (#2934)
This fixes an if-else structure that was not aware of the merge phase in
`makeBeaconBlockForHeadAndSlot`, avoiding a potential crash.
2021-09-30 16:27:17 +00:00
Etan Kissling 2e9fa87f8b
use `SyncAggregate.init()` everywhere (#2932)
The initialization of a `SyncAggregate` to its default value is not very
intuitive. There is an `init` function in `sync_committee_msg_pool` that
provides a convenience wrapper. This patch exports that initializer so
that the rest of the code base can also take advantage of it.
2021-09-30 13:56:07 +00:00
tersec 6b3bf7eb7b
merge hardfork database support (#2911)
* merge hardfork database support

* working block_sim

* recreate state transition changes
2021-09-30 01:07:24 +00:00
tersec 47a6045b05
update a dozen spec URLs (#2929) 2021-09-29 23:43:28 +00:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
zah 554a31d20d
Implement the merge networking req/resp changes (#2922)
The other part of this change was already implemented in the `sendResponseChunk` function.
2021-09-29 16:44:43 +00:00
Etan Kissling 711c08f804
add function to build merkle proofs (#2874)
Adds a function that constructs a Merkle proof for a generalized index.
This will be used during light client sync to update light clients with
a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 15:02:34 +02:00
Etan Kissling e243ba2c0b
revise `makeBeaconBlock` overloads (#2879)
The phase0 and altair overloads of `makeBeaconBlock` slightly differ in
their signatures which makes using them unnecessarily verbose.
- A placeholder `sync_aggregate` argument similar to `executionPayload`
  is added to the phase0 overload to match the altair signature.
- A wrapper operating on `ForkedHashedBeaconState` is introduced.
2021-09-29 12:10:44 +00:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
Etan Kissling 4743807079 use errReject template everywhere
There were still a few instances that used the expansion of `errReject`
instead of using the template itself. It seems that those cases were
forgotten as part of other cleanups in #2809. Done now for readability.
2021-09-29 14:16:09 +03:00
Etan Kissling 6aff2cd19a add remaining REST spec references
This adds spec references for a few APIs that were missing them so far,
in line with how spec references already exist for the other REST APIs.
2021-09-29 14:14:56 +03:00
Tanguy 06df40454f
Improve discovery during fork (#2913) 2021-09-29 14:06:16 +03:00
tersec 1dc94aa36f
update 40 spec URLs to v1.1.0 (#2918) 2021-09-28 18:23:15 +00:00
Eugene Kabanov 1db6a92c4f
Fix double "data" enclosure in /eth/v1/beacon/states/{state_id}/sync_committees response. (#2908) 2021-09-28 20:08:23 +02:00
Jacek Sieka e5346e4e95
simplify state fork access pattern (#2912)
* simplify state fork access pattern

* fixes

* unsafeAddr needs to be dereferenced outside of case for best effect
* remove hash_tree_root of ForkedXxx (doesn't make sense)
* simplify state transition

* fix vc

* readd hash_tree_root(forkedbeaconblock)

* readd htr(fhbs) as well

...and add some protections to not hash the wrong items elsewhere
2021-09-28 20:08:03 +02:00
Etan Kissling c510ffb16a
use `allSyncCommittees` everywhere (#2917)
There are still a few cases with manual loops through sync subcommittees
even though an `allSyncCommittees` iterator exists. Adjusted the few
remaining instances to also use the iterator instead.
2021-09-28 18:02:01 +00:00
tersec ca4c6b4c5c
update 30 spec URLs to v1.1.0 (#2914) 2021-09-28 14:01:46 +00:00
Tanguy dc46559706
Improve peer cycling (#2888)
* Improve peer cycling

* Change metadata timeout values

* fix bad commit

* Log score when kicking peer

* remove unhealthySubnetCount
2021-09-28 09:58:03 +02:00
tersec aec5cf2b1b
update 31 spec reference URLs to v1.1.0 (#2910) 2021-09-28 07:46:06 +00:00
Etan Kissling 01a9b275ec
handle duplicate pubkeys in sync committee (#2902)
When sync committee message handling was introduced in #2830, the edge
case of the same validator being selected multiple times as part of a
sync subcommittee was not covered. Not handling that edge case makes
sync contributions have a lower-than-expected participation rate as each
sync validator is only counted up through once per subcommittee.
This patch ensures that this edge case is properly covered.
2021-09-28 07:44:20 +00:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
Eugene Kabanov 9d34d01cbd
Client SSZ API revisited. (#2907)
* Initial commit.

* Add SSZ getBlock().

* Automatic types detection for SSZ encoded objects.

* Change SSZ.decode() to readSszBytes().
2021-09-27 20:31:11 +02:00
Etan Kissling ba3884f449
ignore instead of reject duplicate sync msgs (#2903)
The P2P spec defines how certain error classes should be handled through
either IGNORE or REJECT verdicts. For sync committee message, the spec
defines that only the first message from each validator per subcommittee
and slot shall be accepted, the rest is ignored. However, current code
rejects those messages instead of ignoring them. Fixed to match spec.
2021-09-27 14:36:28 +00:00
tersec 2b2846b468
implement forked merge state/block support (#2890)
* implement forked state/block support

* merge support for containsOrphan; import cleanup; 80-column lines

* add merge block header operations and slot sanity fixture

* add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot()

* implement process_execution_payload() and add merge deposit operations tests

* add merge block sanity tests

* add merge case to syncCommitteeParticipants

* v1.1.0-beta.5 updates

* reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support

* add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
Etan Kissling cc30bf63b4
update gossip_processing and attestation docs (#2860)
The README file explaining gossip_processing, and the attestation_flow
docs were no longer accurate, as attestations and aggregates no longer
go through a queue (pending batching). This patch updates the docs
accordingly. It also improves some grammar and fixes some typos.
2021-09-27 15:11:10 +02:00
Etan Kissling 5bfff43785
make preset header comments consistent (#2905)
Currently there is a mix of comment formats in the various preset files.
Altair presets have a leading header line, Phase0 presets don't.
Furthermore, the Altair `minimal` header refers to `mainnet` (#2710).
Updated all of the header lines to match the spec.
2021-09-27 09:31:13 +00:00
Jacek Sieka e47a8cbe42
fixes (#2901)
* export kvstore from beacon_chain_db
* fix rest HashList deserialization
* fix asTrusted
2021-09-27 11:24:58 +02:00
Eugene Kabanov 71da905629
Fix sync_committee_bits serialization. (#2899)
* Fix REST `sync_committee_bits` serialization issue.

* Fix error value.
2021-09-27 10:14:43 +02:00
tersec edad6e5e83
use v1.1.0-beta.5 test vectors (#2900) 2021-09-25 03:07:07 +00:00
tersec 6f391691d9
disable sync committee subnets in simulation mode (#2897)
* disable sync committee subnets in simulation mode

* also gate getSyncCommitteeContributionAndProofTopic()
2021-09-24 14:43:53 +00:00
tersec bef98f6f50
sync committees don't exist in phase 0 (#2894) 2021-09-24 07:39:35 +00:00
Eugene Kabanov 0c635334a2
Sync committee related REST API implementation. (#2856) 2021-09-24 01:13:25 +03:00
tersec 5670d58155
test for newest fork first (#2891) 2021-09-23 06:53:36 +00:00
Etan Kissling c95d4f31ed
improve getStateField compile checks (#2889)
The current `getStateField` implementation fails at run-time when called
on a post-Altair state. This is improved by replacing the `if` structure
with a `case` expression, which is checked for exhaustive coverage at
compile time. Care is taken to preserve the `unsafeAddr` optimization.
2021-09-22 20:06:50 +00:00
Zahary Karadjov 9b41bb64da
Register the Altair topics in the valid topics list 2021-09-22 16:28:48 +03:00
Eugene Kabanov b566d4657f
REST /eth/v1/events API call implementation. (#2878)
* Placing callbacks into strategic places.

* Initial events call implementation.

* Post rebase fixes.

* Change addSyncContribution() implementation.

* Add `attestation-sent` event.
Remove gcsafe, raises from callbacks implementations.
Move `attestation-received` fire at the end of attestation processing.

* Address review comments.
2021-09-22 14:17:15 +02:00
Zahary Karadjov 3d37e08085
Sort the results of queryRandom in best-to-worst order 2021-09-22 14:22:21 +03:00
Zahary Karadjov 02372849f1 The peer cycling takes into accounts the syncnets ENR fields 2021-09-22 09:38:48 +03:00
Tanguy b9e9483c66
Fallback ping requests on metadata V2 (#2875)
Some clients disable their metadata V1 after the Altair fork, so we are unable to ping them correctly.

This PR adds a fallback to try metadata V2 if V1 fails.
2021-09-17 07:56:30 +02:00
Mamy Ratsimbazafy d1cb5b7220
Parallel attestation verification (#2718)
* Add parallel attestation verification

* Update tests, batchVerify doesn't use the threadpool with only single core (nim-blscurve update)

* bump nim-blscurve

* Debug info for failing eth2 test vectors

* remove submodule eth2-testnets

* verbose debugging of make failure on Windows (libbacktrace?)

* Remove CI debug mode

* initialization convention

* Fix new altair tests
2021-09-17 03:13:52 +03:00
Eugene Kabanov 7ab1856c04
SSZ encoded responses for REST API calls. (#2851) 2021-09-16 16:32:32 +03:00
Dustin Brody 6638476b5f discard putative blocks from invalid hardforks 2021-09-16 16:18:40 +03:00
Etan Kissling 7ccaeed4a8 remove forked message accessor
The template `message` extracts fork specific data from a forked block.
However, its current implementation does not compile due to each fork
returning a different type, i.e., the generated `case` code attempts to
return different types from each branch. As the template is unused, it
is safely removed for now.
2021-09-15 13:58:35 +03:00
Zahary Karadjov a71de3feda Address review comments 2021-09-14 12:12:49 +03:00
Zahary Karadjov 3e4faaa213 Add .editorconfig file and remove trailing newlines 2021-09-14 12:12:49 +03:00
Zahary Karadjov add7daa30c Light client sync spec functions 2021-09-14 12:12:49 +03:00
tersec 97b0070774
add merge SSZ consensus object tests (#2858)
* add merge SSZ consensus object tests

* add merge attestations, attester slashing, proposer slashing, and voluntary exit test fixtures

* throw catchable exception rather than assert on invalid SingleMemberUnion selector

* hash_tree_root can assert, because it's trusted data by that point

* re-assert on writing, since also trusted data

* beta.4 update

* implement upgrade_to_merge()
2021-09-11 10:01:05 +02:00