Commit Graph

3940 Commits

Author SHA1 Message Date
Etan Kissling 6fc1511866
avoid `-Present` in copyrights (#2935)
Replaced non-conventional `-Present` form in copyright headers with the
more appropriate `-2021` version that is already used by the majority of
the code base. Copyright header should indicate the years during which
the specific file was modified.
2021-09-30 16:28:02 +00:00
Etan Kissling ba84a55699
fix `makeBeaconBlockForHeadAndSlot` for merge (#2934)
This fixes an if-else structure that was not aware of the merge phase in
`makeBeaconBlockForHeadAndSlot`, avoiding a potential crash.
2021-09-30 16:27:17 +00:00
Etan Kissling 9ee6c3e7ef
always use static keys in tests (#2928)
There are multiple ways how states can be processed during tests. One is
through tests/mocking, the other through tests/xxxutil. Both methods can
create genesis states with a set of deposits from mock validators. The
private keys for those validators are created differently, though. While
tests/mocking generates random keys, testblockutil generates fixed keys.
This is a problem when mixing both tests/mocking and testblockutil in a
single test case, as they assume that the deposits were generated using
the same method. This patch aligns the key generation across tests.
2021-09-30 17:14:03 +02:00
Etan Kissling 2e9fa87f8b
use `SyncAggregate.init()` everywhere (#2932)
The initialization of a `SyncAggregate` to its default value is not very
intuitive. There is an `init` function in `sync_committee_msg_pool` that
provides a convenience wrapper. This patch exports that initializer so
that the rest of the code base can also take advantage of it.
2021-09-30 13:56:07 +00:00
Jacek Sieka 850bec6ae1
speed up `validatorDb`/`validatorPerf` (#2885)
avoid needless hashing and use bigger txn:s
2021-09-30 15:21:06 +02:00
Jakub a924a4a366
grafana: hide slowest graphs in a folded row (#2931)
* grafana: update metrics.status.im.json

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* grafana: hide slowest graphs in a folded row

This includes following graphs:

- `received attestation delay`
- `received aggregate delay`
- `received beacon block delay`
- `storeBlock() duration`

Hiding those makes the whole dashboard load much faster.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-09-30 15:04:54 +02:00
Kim De Mey 0b1e0772de
Bump nim-eth to latest on master (#2927) 2021-09-30 09:00:47 +02:00
Dustin Brody 4a26e48226
add initial merge test vector scripts 2021-09-30 03:00:25 +00:00
tersec 6b3bf7eb7b
merge hardfork database support (#2911)
* merge hardfork database support

* working block_sim

* recreate state transition changes
2021-09-30 01:07:24 +00:00
tersec 47a6045b05
update a dozen spec URLs (#2929) 2021-09-29 23:43:28 +00:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
kdeme 933879e298 Bump nim-eth to allow for tcp/udp ports to always be configured in ENR 2021-09-29 20:15:24 +03:00
zah 554a31d20d
Implement the merge networking req/resp changes (#2922)
The other part of this change was already implemented in the `sendResponseChunk` function.
2021-09-29 16:44:43 +00:00
Mamy Ratsimbazafy 4aed54200d
fix multithreading memory leak (#2920)
* fix multithreading memory leak

* remove destructors for Nim 1.2 taskpool
2021-09-29 18:09:53 +02:00
Etan Kissling 711c08f804
add function to build merkle proofs (#2874)
Adds a function that constructs a Merkle proof for a generalized index.
This will be used during light client sync to update light clients with
a new state (see NEXT_SYNC_COMMITTEE_INDEX / FINALIZED_ROOT_INDEX).
2021-09-29 15:02:34 +02:00
Etan Kissling e243ba2c0b
revise `makeBeaconBlock` overloads (#2879)
The phase0 and altair overloads of `makeBeaconBlock` slightly differ in
their signatures which makes using them unnecessarily verbose.
- A placeholder `sync_aggregate` argument similar to `executionPayload`
  is added to the phase0 overload to match the altair signature.
- A wrapper operating on `ForkedHashedBeaconState` is introduced.
2021-09-29 12:10:44 +00:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
Etan Kissling 4743807079 use errReject template everywhere
There were still a few instances that used the expansion of `errReject`
instead of using the template itself. It seems that those cases were
forgotten as part of other cleanups in #2809. Done now for readability.
2021-09-29 14:16:09 +03:00
Etan Kissling 6aff2cd19a add remaining REST spec references
This adds spec references for a few APIs that were missing them so far,
in line with how spec references already exist for the other REST APIs.
2021-09-29 14:14:56 +03:00
Tanguy 06df40454f
Improve discovery during fork (#2913) 2021-09-29 14:06:16 +03:00
tersec 1dc94aa36f
update 40 spec URLs to v1.1.0 (#2918) 2021-09-28 18:23:15 +00:00
Eugene Kabanov 1db6a92c4f
Fix double "data" enclosure in /eth/v1/beacon/states/{state_id}/sync_committees response. (#2908) 2021-09-28 20:08:23 +02:00
Jacek Sieka e5346e4e95
simplify state fork access pattern (#2912)
* simplify state fork access pattern

* fixes

* unsafeAddr needs to be dereferenced outside of case for best effect
* remove hash_tree_root of ForkedXxx (doesn't make sense)
* simplify state transition

* fix vc

* readd hash_tree_root(forkedbeaconblock)

* readd htr(fhbs) as well

...and add some protections to not hash the wrong items elsewhere
2021-09-28 20:08:03 +02:00
Etan Kissling c510ffb16a
use `allSyncCommittees` everywhere (#2917)
There are still a few cases with manual loops through sync subcommittees
even though an `allSyncCommittees` iterator exists. Adjusted the few
remaining instances to also use the iterator instead.
2021-09-28 18:02:01 +00:00
Etan Kissling a0e7463732
fix `mockBlock` for current slot (#2915)
In #2867 a regression was introduced that broke `mockBlock` when used to
compute a block for the current state. This patch fixes the regression.
2021-09-28 16:27:44 +00:00
tersec ca4c6b4c5c
update 30 spec URLs to v1.1.0 (#2914) 2021-09-28 14:01:46 +00:00
Tanguy dc46559706
Improve peer cycling (#2888)
* Improve peer cycling

* Change metadata timeout values

* fix bad commit

* Log score when kicking peer

* remove unhealthySubnetCount
2021-09-28 09:58:03 +02:00
tersec aec5cf2b1b
update 31 spec reference URLs to v1.1.0 (#2910) 2021-09-28 07:46:06 +00:00
Etan Kissling 01a9b275ec
handle duplicate pubkeys in sync committee (#2902)
When sync committee message handling was introduced in #2830, the edge
case of the same validator being selected multiple times as part of a
sync subcommittee was not covered. Not handling that edge case makes
sync contributions have a lower-than-expected participation rate as each
sync validator is only counted up through once per subcommittee.
This patch ensures that this edge case is properly covered.
2021-09-28 07:44:20 +00:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
Eugene Kabanov 9d34d01cbd
Client SSZ API revisited. (#2907)
* Initial commit.

* Add SSZ getBlock().

* Automatic types detection for SSZ encoded objects.

* Change SSZ.decode() to readSszBytes().
2021-09-27 20:31:11 +02:00
Etan Kissling ba3884f449
ignore instead of reject duplicate sync msgs (#2903)
The P2P spec defines how certain error classes should be handled through
either IGNORE or REJECT verdicts. For sync committee message, the spec
defines that only the first message from each validator per subcommittee
and slot shall be accepted, the rest is ignored. However, current code
rejects those messages instead of ignoring them. Fixed to match spec.
2021-09-27 14:36:28 +00:00
tersec 2b2846b468
implement forked merge state/block support (#2890)
* implement forked state/block support

* merge support for containsOrphan; import cleanup; 80-column lines

* add merge block header operations and slot sanity fixture

* add epoch state transition tests; implement is_valid_gas_limit(), is_merge_block(), is_execution_enabled(), and compute_timestamp_at_slot()

* implement process_execution_payload() and add merge deposit operations tests

* add merge block sanity tests

* add merge case to syncCommitteeParticipants

* v1.1.0-beta.5 updates

* reduce getTestStates-based memory usage; don't try to REST-serialize ExecutionPayload transactions without underlying support

* add execution payload tests; switch var to let in tests/official/
2021-09-27 14:22:58 +00:00
Etan Kissling cc30bf63b4
update gossip_processing and attestation docs (#2860)
The README file explaining gossip_processing, and the attestation_flow
docs were no longer accurate, as attestations and aggregates no longer
go through a queue (pending batching). This patch updates the docs
accordingly. It also improves some grammar and fixes some typos.
2021-09-27 15:11:10 +02:00
Etan Kissling 5bfff43785
make preset header comments consistent (#2905)
Currently there is a mix of comment formats in the various preset files.
Altair presets have a leading header line, Phase0 presets don't.
Furthermore, the Altair `minimal` header refers to `mainnet` (#2710).
Updated all of the header lines to match the spec.
2021-09-27 09:31:13 +00:00
Jacek Sieka e47a8cbe42
fixes (#2901)
* export kvstore from beacon_chain_db
* fix rest HashList deserialization
* fix asTrusted
2021-09-27 11:24:58 +02:00
Eugene Kabanov 71da905629
Fix sync_committee_bits serialization. (#2899)
* Fix REST `sync_committee_bits` serialization issue.

* Fix error value.
2021-09-27 10:14:43 +02:00
tersec edad6e5e83
use v1.1.0-beta.5 test vectors (#2900) 2021-09-25 03:07:07 +00:00
tersec 6f391691d9
disable sync committee subnets in simulation mode (#2897)
* disable sync committee subnets in simulation mode

* also gate getSyncCommitteeContributionAndProofTopic()
2021-09-24 14:43:53 +00:00
Etan Kissling aaaae89445
bump nimbus-build-system (#2898)
Changes:
- QUICK_AND_DIRTY_NIMBLE
- fix build output color on macOS
2021-09-24 15:34:23 +02:00
tersec bef98f6f50
sync committees don't exist in phase 0 (#2894) 2021-09-24 07:39:35 +00:00
Eugene Kabanov 0c635334a2
Sync committee related REST API implementation. (#2856) 2021-09-24 01:13:25 +03:00
tersec 5670d58155
test for newest fork first (#2891) 2021-09-23 06:53:36 +00:00
Etan Kissling c95d4f31ed
improve getStateField compile checks (#2889)
The current `getStateField` implementation fails at run-time when called
on a post-Altair state. This is improved by replacing the `if` structure
with a `case` expression, which is checked for exhaustive coverage at
compile time. Care is taken to preserve the `unsafeAddr` optimization.
2021-09-22 20:06:50 +00:00
Etan Kissling 4491259c52
allow testing with custom cfg (#2884)
This extends some test utilities to allow simulating states that use
a custom RuntimeConfig, e.g., to test with an earlier ALTAIR_FORK_EPOCH.
2021-09-22 17:50:10 +00:00
sacha 2ec34e6761
Book edits (#2887)
* cp

* cp
2021-09-22 15:52:42 +02:00
Zahary Karadjov 9b41bb64da
Register the Altair topics in the valid topics list 2021-09-22 16:28:48 +03:00
Eugene Kabanov b566d4657f
REST /eth/v1/events API call implementation. (#2878)
* Placing callbacks into strategic places.

* Initial events call implementation.

* Post rebase fixes.

* Change addSyncContribution() implementation.

* Add `attestation-sent` event.
Remove gcsafe, raises from callbacks implementations.
Move `attestation-received` fire at the end of attestation processing.

* Address review comments.
2021-09-22 14:17:15 +02:00
Jacek Sieka 0b21ebfe74
readme: update toc 2021-09-22 14:16:36 +02:00
Zahary Karadjov 3d37e08085
Sort the results of queryRandom in best-to-worst order 2021-09-22 14:22:21 +03:00