3864 Commits

Author SHA1 Message Date
Zahary Karadjov
cfa0268d89
Version 23.9.1
Changes:

* Updated network metadata for Holesky
* Use hash_tree_root instead of SHA256 when verifying the Holesky
  genesis state
2023-09-25 21:24:25 +01:00
tersec
710f26745d
small attestation pool clean up (#5458) 2023-09-24 08:50:48 +00:00
Eugene Kabanov
98e08a9e02
Fix cancellation deprecate warnings. (#5454)
Add noCancel in exception handlers.
Refactor pending cancellations code.
2023-09-24 09:28:09 +02:00
Eugene Kabanov
e3fe762ec9
Fix unknown fields should be ignored when parsing liveness responses. (#5418) 2023-09-23 08:44:54 +02:00
Etan Kissling
47639ef89c
replace Eth2NetworkMetadata case object with Result (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
Etan Kissling
5150505e66
bump nim-ssz-serialization to a5767c1b4992dca3e5bd72557aadd4df0f183bf8 (#5457)
- nimble/ci updates
- use common super-type in `raisesssz` pragma
2023-09-22 23:44:57 +02:00
Eugene Kabanov
caaad4777c
BN+LC+SN: Fix cancellation deprecate warnings. (#5455)
* Fix deprecation cancel() warnings in SN, BN, LC.

* Fix missing import.
2023-09-22 11:06:27 +00:00
Eugene Kabanov
88565e6da4
Make KeystoreCache working with KeystoreManagement API. (#5443) 2023-09-21 20:07:02 +02:00
tersec
2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Jacek Sieka
e64b31986e
Increase default file limit on startup (#5436)
We use file descriptors for validators and sockets and might run out of
either on high-validator setups - increasing the limit here is harmless
and avoids a common limiting factor in setup

Co-authored-by: Etan Kissling <etan@status.im>
2023-09-21 16:08:13 +02:00
Etan Kissling
e7bc41e005
blck --> forkyBlck when using withBlck / withStateAndBlck (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
tersec
351472a2d4
per payload-builder payload builder validator registration (#5431) 2023-09-20 17:00:37 +00:00
Etan Kissling
ee57a07411
implement Confirmation rule prerequisite - fork choice filter change (#5450)
To support confirmation rule via beacon-APIs as described in spec PR,
add `--debug-fork-choice-version=pr3431` option and enable when Deneb
fork is scheduled. To opt-out, `--debug-fork-choice-version=stable`,
or don't schedule Deneb.

- https://github.com/ethereum/consensus-specs/pull/3431
- https://github.com/ethereum/consensus-specs/issues/3466

"will bundle this with deneb release":

- https://github.com/ethereum/pm/issues/844#issuecomment-1673359012
2023-09-20 18:27:55 +02:00
Etan Kissling
273f1d34cd
implement EIP-7514 for Deneb: Add Max Epoch Churn Limit (#5442)
* implement EIP-7514 for Deneb: Add Max Epoch Churn Limit

Cap activations per epoch according to EIP-7514:

- https://eips.ethereum.org/EIPS/eip-7514
- https://github.com/ethereum/consensus-specs/pull/3499

* apply proposer boost to first block in case of equivocation

Implement spec changes to fork choice; this only affects equivocation
when multiple blocks are signed for the same slot. Regular operation
is not changed.

- https://github.com/ethereum/consensus-specs/pull/3352

* bump test vectors to v1.4.0-beta.2-hotfix

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-09-20 13:39:56 +02:00
Etan Kissling
1640c45a95
reject invalid proofs when validating blobs (#5445)
Currently, passing `0xc00000...` proof seems to pass `verifyProofs`.
Unsure why such a check is not necessary in spec, and also unsure
whether it is correct to reject proof at infinity, or if it could
occur, e.g., for a blob containing all 0 bytes. Weird overall...

* proper fix
2023-09-19 22:13:58 +00:00
Etan Kissling
c1b43d166b
avoid repeated total active balance computation in fork choice (#5437)
We currently compute `justified_total_active_balance` inside
`calculateProposerBoost`, despite that sum already being known
in the `EpochRef` cache. Tracking `justified_total_active_balance`
whenever the justified checkpoint updates allows replacing the
repeated computation with a lookup, at minimal memory cost.
2023-09-19 22:04:55 +02:00
tersec
ee75d45a8b
fix ATTESTATION_PROPAGATION_SLOT_RANGE unknown constant warning (#5433)
* fix ATTESTATION_PROPAGATION_SLOT_RANGE unknown constant warning

* verify underflow not an issue
2023-09-19 06:54:58 +00:00
Tanguy
4918a4e2e0
Fix direct peers (#5427)
* Fix direct peers

* Support ENRs in DP, use DP in local testnet

* fix docs

* bump libp2p
2023-09-15 18:45:55 +00:00
Eugene Kabanov
12ef43bcc0
Fix issue when VC could not be interrupted while its in pre-genesis period. (#5424) 2023-09-15 12:39:26 +02:00
tersec
325d5a6afa
log builder unblinding requests (#5434) 2023-09-15 09:27:38 +00:00
tersec
dae4beb4a2
log engine getPayload requests (#5432) 2023-09-15 08:56:32 +00:00
Eugene Kabanov
aec953e4da
VC: Fix VC must not crash if beacon node address could not be resolved. (#5388)
* Fix VC should not crash, if beacon node URL could not be resolved.

* Bump chronos.

* Update .gitmodules.
2023-09-15 07:34:32 +02:00
tersec
5a29ad7e4f
update some consensus-spec URLs to v1.4.0-beta.1 (#5429) 2023-09-14 18:23:59 +00:00
tersec
7a001d4896
remove 4 more missing constants in preset handling warning (#5428) 2023-09-14 12:37:35 +00:00
Etan Kissling
176ea09c2b
cleanup OnBlockAdded usage (#5426)
Reduce repetitiveness when using forked `OnBlockAdded` callbacks by
introducing a template to obtain appropriate cb from `ConsensusFork`.
2023-09-13 17:57:54 +00:00
Etan Kissling
d7aaf1b809
only request nextPeriod sync duties close to end of currentPeriod (#5409)
Currently we always request duties for current and next sync period.
As sync periods are quite long (~27 hrs on Mainnet), having access to
the duties so early doesn't help too much. To avoid running into errors
when the BN does not have the duties available around period boundary,
delay requesting them until the current period is close to finish.

`SYNC_COMMITTEE_SUBNET_COUNT` epochs are what the spec says should be
the lookahead timing of starting to subscribe to sync committee gossip.
Reusing the constant here for consistency.

This fixes these warning messages in the first slot of a new period.

```
rocketpool_validator   | WRN 2023-09-07 20:19:35.439+00:00 Beacon node has incompatible configuration reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
rocketpool_validator   | WRN 2023-09-07 20:19:35.440+00:00 Unable to get sync committee duties        period=889 epoch=227584 reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" service=duties_service
rocketpool_validator   | NOT 2023-09-07 20:19:35.441+00:00 Beacon node is in sync                     head_slot=7274495 sync_distance=1 is_optimistic=false node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
```
2023-09-13 08:58:28 +02:00
Etan Kissling
4700030231
fix sync contribution validation for first slot per period (#5408)
Sync committee duties are performed by the sync committee as determined
by slot + 1. We did it correctly for individual messages, but selected
the incorrect participants for aggregate contributions for the very
first slot per period (roughly 1 per ~27 hrs on Mainnet). The faulty
participants selection code was originally introduced in #2925.
2023-09-13 03:23:18 +02:00
andri lim
5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
Etan Kissling
8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
Zahary Karadjov
6b1ab81046
Support both snappy-encoded and pure SSZ genesis states 2023-09-12 01:52:15 +03:00
tersec
2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Etan Kissling
ff176a1569
fix syncCommitteeDuties log in Slot end message (#5416)
In `Slot end`, sync duties are reported one slot too late,
because sync committee duties are determined by the next slot
instead of the current one. Address that by taking account of this.
2023-09-11 10:51:55 +00:00
Eugene Kabanov
3c3c4e8edf
Fix /eth/v1/validator/liveness/{epoch} call names and links in code (#5392)
* Fix getLiveness() call links and names.

* Add getLiveness() decoding test.

* Remove BN state updates from getLiveness() call.

* Fix AllTests.

* Reduce number of tests.
2023-09-11 12:07:34 +02:00
tersec
10ec7be686
optimize epoch registry processing (#5412) 2023-09-11 11:21:50 +02:00
tersec
cc13e0b7e0
restore full test coverage for process_rewards_and_penalties (#5407)
* restore full test coverage for process_rewards_and_penalties

* adjust ncli_db to use new iterator
2023-09-08 16:42:18 +00:00
Zahary Karadjov
f0bb1cbd4e
Version 23.9.0 2023-09-08 13:33:51 +03:00
Zahary Karadjov
c7db7d0f2c
Add CLI options --genesis-state and --genesis-state-url 2023-09-08 13:10:18 +03:00
zah
53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
tersec
f53422c181
optimize epoch transition via get_flag_index_deltas() and get_inactivity_penalty_deltas() (#5404)
* optimize epoch transition via get_flag_index_deltas() and get_inactivity_penalty_deltas()

* directly mutate Altair info in {get,update}_flag_and_inactivity_deltas
2023-09-07 11:14:52 +00:00
tersec
fbcb476ff9
allow archiving blobs (#5391) 2023-09-07 08:19:33 +02:00
zah
2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Etan Kissling
89d133d0e5
further reduce stack size of LC helpers for Nim 2.0 (#5401)
The `is_next_sync_committee_known` helper allocates a fresh
`SyncCommittee` in the caller and `nimZeroMem`s it on each use.
Use `static` to compare against a compile-time zeroed copy instead.
This also should help reduce stack size far enough to link with Nim 2.0.
2023-09-06 16:05:12 +02:00
Etan Kissling
ac3b2b4233
rename U to maxLen in statediff for Nim 2.0 (#5396)
Nim 2.0 gets confused when compiling `all_tests`:

```
Error: undeclared identifier: 'maxLen'
candidates (edit distance, scope distance); see '--spellSuggest':
 (3, 7): 'Table'
 (3, 7): 'len'
 (3, 7): 'max'
```

Renaming the generic parameter `U` to `maxLen` fixes this somehow.
It also increases readability to use the same name consistently.
2023-09-06 02:39:21 +00:00
Eugene Kabanov
4ce8e77b56
Address issue #5362. (#5370)
* Address issue #5362.

* Remove RestSpec object declaration and getSpec() API call declaration.

* Address review comments.
2023-09-05 23:39:01 +03:00
Etan Kissling
8ffb80e954
avoid Taskpool.spawn within {.async.} for Nim 2.0 (#5393)
In Nim 2.0, attempting to use `Taskpool.spawn` inside `{.async.}` `proc`
leads to `Error: cannot generate destructor for generic type: Isolated`.

Add an intermediate wrapper `proc` that performs the `spawn` operation
to workaround the problem.
2023-09-05 21:36:35 +02:00
Etan Kissling
dbb1a63ca9
collapse partialBeaconBlock templates into one (#5386)
Have a single `Forky` template for `partialBeaconBlock` production
rather than have several copies that are mostly identical.
2023-09-05 15:59:17 +02:00
tersec
0987628c0b
simplify beacon validator sync committee production for blocks slightly (#5364) 2023-09-05 12:31:33 +02:00
Eugene Kabanov
afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
Etan Kissling
be9ecfa1c9
add helpers for processing withdrawals to libnimbus_lc.a (#5374)
Similar to the existing helpers for processing transactions / receipts,
extend `libnimbus_lc.a` with support for processing withdrawals as well.
2023-09-04 20:44:03 +02:00
tersec
41ce581149
invalidate only required htr cache portion for immutable validators (#5381) 2023-09-04 19:48:31 +02:00