Commit Graph

3904 Commits

Author SHA1 Message Date
Ștefan Talpalaru d3890f0be2 better help text for --num-threads 2021-10-04 19:05:48 +03:00
Zahary Karadjov 22346f3e32 Don't look for syncnets peers before the Altair transition 2021-10-04 18:43:43 +03:00
Etan Kissling f10a2b18bb port EF light client tests
EF has a number of unit tests related to light client functionality, see
https://github.com/ethereum/consensus-specs/blob/v1.1.0/tests/core/pyspec/eth2spec/test/altair/unittests/test_sync_protocol.py
The tests have been ported over and integrated into our test suite.
2021-10-04 15:49:34 +03:00
Zahary Karadjov fcbc54bc08 Fixed typos (thanks to @gxbag) 2021-10-04 15:34:13 +03:00
zah 653fc72b2e
Document that the REST API is out of beta (#2947) 2021-10-04 13:55:02 +02:00
Ștefan Talpalaru 1c50cd224e
Grafana: add 3 more panels (#2946)
(and delete old tool)
2021-10-04 12:04:33 +02:00
Ștefan Talpalaru b67e43e9f1
remove unused Docker test (#2945) 2021-10-04 08:32:29 +00:00
Etan Kissling 2bbffbde10
abort compile when fork epoch is forgotten (#2939)
There are a few locations in the code that compare the current epoch to
the various FORK_EPOCH constants and branch off into fork-specific code.
When a new fork is introduced, it is sometimes forgotten to update all
of those branch locations. This patch introduces a compile-time check
that ensures that all branches need to be covered exhaustively. This is
done by replacing if-elif structures with case expressions.
2021-10-04 08:31:21 +00:00
Ștefan Talpalaru 4a7a62c072
dist: cross-build ARM64 on Debian-10.10 (#2944)
instead of Ubuntu-18.04. This fixes a crash when stress-testing the
metrics HTTP server. We think it was due to gcc-7.4.0 miscompiling
something. The new distro/version gives us gcc-8.3.0, but with the same
old glibc-2.28 we need on Raspbian.
2021-10-03 16:41:35 +02:00
Jacek Sieka eb706f89a5
Merge remote-tracking branch 'origin/stable' into unstable 2021-10-01 16:55:50 +02:00
Ștefan Talpalaru 9b2b3bf1ab
enable libp2p agent metrics (#2933) 2021-10-01 16:15:08 +02:00
Etan Kissling f8e9b1ff9d
remove privkey from mock withdrawal credentials (#2936)
In tests, the private key was put into the validator deposit's withdraw
credentials so that it can be recovered later. This leads to problems
when creating the validators through other means that do not put the key
there. In general, mock private keys only depend on the validator index,
though, and because it is clear what the index of a validator is, it is
not actually needed to put the key into the credentials.
2021-10-01 13:35:16 +02:00
Ștefan Talpalaru 744c0d2def update docker-compose.yml examples 2021-10-01 14:28:23 +03:00
Etan Kissling b217150f1d
use forked `getAttestationsForBlock` everywhere (#2937)
There are a number of locations in the code that get attestations on a
forked beacon state. For attestation pools test, a convenience wrapper
was available to reduce clutter. This patch integrates that wrapper into
the core component so that it can also take advantage of the wrapper.
2021-10-01 01:29:32 +00:00
Etan Kissling 6fc1511866
avoid `-Present` in copyrights (#2935)
Replaced non-conventional `-Present` form in copyright headers with the
more appropriate `-2021` version that is already used by the majority of
the code base. Copyright header should indicate the years during which
the specific file was modified.
2021-09-30 16:28:02 +00:00
Etan Kissling ba84a55699
fix `makeBeaconBlockForHeadAndSlot` for merge (#2934)
This fixes an if-else structure that was not aware of the merge phase in
`makeBeaconBlockForHeadAndSlot`, avoiding a potential crash.
2021-09-30 16:27:17 +00:00
Etan Kissling 9ee6c3e7ef
always use static keys in tests (#2928)
There are multiple ways how states can be processed during tests. One is
through tests/mocking, the other through tests/xxxutil. Both methods can
create genesis states with a set of deposits from mock validators. The
private keys for those validators are created differently, though. While
tests/mocking generates random keys, testblockutil generates fixed keys.
This is a problem when mixing both tests/mocking and testblockutil in a
single test case, as they assume that the deposits were generated using
the same method. This patch aligns the key generation across tests.
2021-09-30 17:14:03 +02:00
Etan Kissling 2e9fa87f8b
use `SyncAggregate.init()` everywhere (#2932)
The initialization of a `SyncAggregate` to its default value is not very
intuitive. There is an `init` function in `sync_committee_msg_pool` that
provides a convenience wrapper. This patch exports that initializer so
that the rest of the code base can also take advantage of it.
2021-09-30 13:56:07 +00:00
Jacek Sieka 850bec6ae1
speed up `validatorDb`/`validatorPerf` (#2885)
avoid needless hashing and use bigger txn:s
2021-09-30 15:21:06 +02:00
Jakub a924a4a366
grafana: hide slowest graphs in a folded row (#2931)
* grafana: update metrics.status.im.json

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* grafana: hide slowest graphs in a folded row

This includes following graphs:

- `received attestation delay`
- `received aggregate delay`
- `received beacon block delay`
- `storeBlock() duration`

Hiding those makes the whole dashboard load much faster.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-09-30 15:04:54 +02:00
Kim De Mey 0b1e0772de
Bump nim-eth to latest on master (#2927) 2021-09-30 09:00:47 +02:00
Dustin Brody 4a26e48226
add initial merge test vector scripts 2021-09-30 03:00:25 +00:00
tersec 6b3bf7eb7b
merge hardfork database support (#2911)
* merge hardfork database support

* working block_sim

* recreate state transition changes
2021-09-30 01:07:24 +00:00
tersec 47a6045b05
update a dozen spec URLs (#2929) 2021-09-29 23:43:28 +00:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
kdeme 933879e298 Bump nim-eth to allow for tcp/udp ports to always be configured in ENR 2021-09-29 20:15:24 +03:00
zah 554a31d20d
Implement the merge networking req/resp changes (#2922)
The other part of this change was already implemented in the `sendResponseChunk` function.
2021-09-29 16:44:43 +00:00
Mamy Ratsimbazafy 4aed54200d
fix multithreading memory leak (#2920)
* fix multithreading memory leak

* remove destructors for Nim 1.2 taskpool
2021-09-29 18:09:53 +02:00
Etan Kissling 711c08f804
add function to build merkle proofs (#2874)
Adds a function that constructs a Merkle proof for a generalized index.
This will be used during light client sync to update light clients with
a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 15:02:34 +02:00
Etan Kissling e243ba2c0b
revise `makeBeaconBlock` overloads (#2879)
The phase0 and altair overloads of `makeBeaconBlock` slightly differ in
their signatures which makes using them unnecessarily verbose.
- A placeholder `sync_aggregate` argument similar to `executionPayload`
  is added to the phase0 overload to match the altair signature.
- A wrapper operating on `ForkedHashedBeaconState` is introduced.
2021-09-29 12:10:44 +00:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
Etan Kissling 4743807079 use errReject template everywhere
There were still a few instances that used the expansion of `errReject`
instead of using the template itself. It seems that those cases were
forgotten as part of other cleanups in #2809. Done now for readability.
2021-09-29 14:16:09 +03:00
Etan Kissling 6aff2cd19a add remaining REST spec references
This adds spec references for a few APIs that were missing them so far,
in line with how spec references already exist for the other REST APIs.
2021-09-29 14:14:56 +03:00
Tanguy 06df40454f
Improve discovery during fork (#2913) 2021-09-29 14:06:16 +03:00
tersec 1dc94aa36f
update 40 spec URLs to v1.1.0 (#2918) 2021-09-28 18:23:15 +00:00
Eugene Kabanov 1db6a92c4f
Fix double "data" enclosure in /eth/v1/beacon/states/{state_id}/sync_committees response. (#2908) 2021-09-28 20:08:23 +02:00
Jacek Sieka e5346e4e95
simplify state fork access pattern (#2912)
* simplify state fork access pattern

* fixes

* unsafeAddr needs to be dereferenced outside of case for best effect
* remove hash_tree_root of ForkedXxx (doesn't make sense)
* simplify state transition

* fix vc

* readd hash_tree_root(forkedbeaconblock)

* readd htr(fhbs) as well

...and add some protections to not hash the wrong items elsewhere
2021-09-28 20:08:03 +02:00
Etan Kissling c510ffb16a
use `allSyncCommittees` everywhere (#2917)
There are still a few cases with manual loops through sync subcommittees
even though an `allSyncCommittees` iterator exists. Adjusted the few
remaining instances to also use the iterator instead.
2021-09-28 18:02:01 +00:00
Etan Kissling a0e7463732
fix `mockBlock` for current slot (#2915)
In #2867 a regression was introduced that broke `mockBlock` when used to
compute a block for the current state. This patch fixes the regression.
2021-09-28 16:27:44 +00:00
tersec ca4c6b4c5c
update 30 spec URLs to v1.1.0 (#2914) 2021-09-28 14:01:46 +00:00
Tanguy dc46559706
Improve peer cycling (#2888)
* Improve peer cycling

* Change metadata timeout values

* fix bad commit

* Log score when kicking peer

* remove unhealthySubnetCount
2021-09-28 09:58:03 +02:00
tersec aec5cf2b1b
update 31 spec reference URLs to v1.1.0 (#2910) 2021-09-28 07:46:06 +00:00
Etan Kissling 01a9b275ec
handle duplicate pubkeys in sync committee (#2902)
When sync committee message handling was introduced in #2830, the edge
case of the same validator being selected multiple times as part of a
sync subcommittee was not covered. Not handling that edge case makes
sync contributions have a lower-than-expected participation rate as each
sync validator is only counted up through once per subcommittee.
This patch ensures that this edge case is properly covered.
2021-09-28 07:44:20 +00:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
Eugene Kabanov 9d34d01cbd
Client SSZ API revisited. (#2907)
* Initial commit.

* Add SSZ getBlock().

* Automatic types detection for SSZ encoded objects.

* Change SSZ.decode() to readSszBytes().
2021-09-27 20:31:11 +02:00
Etan Kissling ba3884f449
ignore instead of reject duplicate sync msgs (#2903)
The P2P spec defines how certain error classes should be handled through
either IGNORE or REJECT verdicts. For sync committee message, the spec
defines that only the first message from each validator per subcommittee
and slot shall be accepted, the rest is ignored. However, current code
rejects those messages instead of ignoring them. Fixed to match spec.
2021-09-27 14:36:28 +00:00
tersec 2b2846b468
implement forked merge state/block support (#2890)
* implement forked state/block support

* merge support for containsOrphan; import cleanup; 80-column lines

* add merge block header operations and slot sanity fixture

* add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot()

* implement process_execution_payload() and add merge deposit operations tests

* add merge block sanity tests

* add merge case to syncCommitteeParticipants

* v1.1.0-beta.5 updates

* reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support

* add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
Etan Kissling cc30bf63b4
update gossip_processing and attestation docs (#2860)
The README file explaining gossip_processing, and the attestation_flow
docs were no longer accurate, as attestations and aggregates no longer
go through a queue (pending batching). This patch updates the docs
accordingly. It also improves some grammar and fixes some typos.
2021-09-27 15:11:10 +02:00
Etan Kissling 5bfff43785
make preset header comments consistent (#2905)
Currently there is a mix of comment formats in the various preset files.
Altair presets have a leading header line, Phase0 presets don't.
Furthermore, the Altair `minimal` header refers to `mainnet` (#2710).
Updated all of the header lines to match the spec.
2021-09-27 09:31:13 +00:00
Jacek Sieka e47a8cbe42
fixes (#2901)
* export kvstore from beacon_chain_db
* fix rest HashList deserialization
* fix asTrusted
2021-09-27 11:24:58 +02:00