Commit Graph

148 Commits

Author SHA1 Message Date
Etan Kissling c3fe055bd4
fix incorrect config validation regression from #5959 (#5966)
During refactoring of #5959, some implicit `return` were overlooked,
resulting in spurious `err()` being returned without message.

```
{"lvl":"WRN","ts":"2024-02-26 10:12:20.469+00:00","msg":"Beacon nodes report different configuration values","reason":"","service":"fallback_service","node":"http://127.0.0.1:9303[Nimbus/v24.2.1-4e9bc7-stateofus]","node_index":0,"node_roles":"AGBSDT"}
```

Correcting the helpers to return explicit result in all exhaustive
cases so that this cannot happen anymore by accident.
2024-02-26 13:32:01 +01:00
Etan Kissling 4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
Etan Kissling e04e95167d
avoid `read`/`readError` in favor of `value`/`error` (#5904)
In VC logic, bump 3 remaining uses of `readError`/`read` to use
`error`/`value` instead. The surrounding logic guarantees success.
2024-02-19 10:52:35 +01:00
Etan Kissling 4fc1550d0f
add `{.push raises: [].}` to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
Eugene Kabanov 464ff68658
Address issues #5675 and #5681. (#5846) 2024-02-07 19:51:36 +00:00
Eugene Kabanov 21efe7e060
VC: Use produceBlockV3 when its available. (#5842)
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
Etan Kissling 61cb7fafdf
clear `BrokenClock` status if Nimbus extensions no longer supported (#5827)
When BN clock is out of sync, VC sets BN status to `BrokenClock`. It is
only reset to `Offline` after restoring time sync. However, if VC fails
encounters an error while checking time, Nimbus extensions are assumed
to be unavailable and the BN is no longer checked for having a synced
clock. This means it is never reset back to `Offline` if errors start
occurring _after_ BN is already set to `BrokenClock`. This could be
because BN is changed from Nimbus to an alternative implementation,
or due to intermittent connection issues.

Ensure that BN status is reset back to `Offline` when Nimbus extensions
are disabled to ensure eventual connection recovery.
2024-01-25 11:52:25 +01:00
tersec d8a2690a92
update builder API spec reference URLs to v0.4.0 (#5812) 2024-01-22 08:36:46 +01:00
tersec d669eef97b
rm unused code; fix a Deprecated warning; proc to func (#5807) 2024-01-20 21:36:01 +00:00
Eugene Kabanov 3648df7d4c
Fix VC not always be able to obtain feeRecipient value. (#5781)
Use state's validator value to obtain feeRecipient value.
Make feeRecipient and gasLimit calculation equal for BN and VC.
2024-01-19 14:36:04 +00:00
Jacek Sieka d5785677a8
split out eth1chain into its own module (#5768)
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
tersec cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 508f3b6368
add missing `std/` prefix to more imports (#5696)
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
tersec cafa1f3d3d
rm unused RPC signatures replaced by Plain versions (#5685) 2024-01-03 05:58:34 +00:00
tersec 0b9c632b69
fix some `Name` and `XDeclaredButNotUsed` hints (#5670) 2023-12-15 02:00:14 +01:00
tersec 61e355639a
fix VC builder API registration expiry check sense (#5663)
* fix VC builder API registration expiry check sense

* avoid pointlessly recomputing cached registrations

* rm unused variable
2023-12-11 22:54:46 +01:00
tersec c36d2aa103
fix XDeclaredButNotUsed warnings (#5648) 2023-12-05 11:45:47 +00:00
tersec 1791b0ee0a
fix XDeclaredButNotUsed warnings (#5638) 2023-12-04 21:14:35 +00:00
tersec 23983c34ac
rm debugRaiseAssert (#5629) 2023-11-26 19:40:10 +01:00
Jacek Sieka e1e809eeb7
Batch slashing protection registration (#5604)
This PR brings down the time to send 100 attestations from ~1s to
~100ms, making it feasible to run 10k validators on a single node (which
regularly send 300 attestations / slot).

This is done by batching the slashing protection database write in a
single transaction thus avoiding a slow fsync for every signature -
effects will be more pronounced on slow drives.

The benefit applies both to beacon and client validators.
2023-11-19 14:08:07 +01:00
Eugene Kabanov 9889b840ce
VC: new scoring functions. (#5447)
* Initial commit.

* Fix issues and tests.

* Fix test compilation issue.

* Update AllTests.

* Change the most poor score name from <lowest> to <bad>.
Split sync committee message score in range, so lexicographic scores will not intersect with normal one.
Lexicographic scores should be below to normal scores.

* Address review comments.
Fix aggregated attestation scoring to use MAX_VALIDATORS_PER_COMMITTEE.
Fix sync committee contributions to use SYNC_SUBCOMMITTEE_SIZE.
Add getUniqueVotes test vectors.

* Post-rebase fixes.

* Address review comments.

* Return back score calculation based on actual bits length.

* AllTests modification.
2023-11-14 12:13:26 +01:00
Eugene Kabanov 6bc038e8d5
VC: Obol middleware support (#5375) 2023-11-08 14:03:51 +02:00
Eugene Kabanov 49c851109e
VC: Add pruning slashing database. (#5551)
* Add slashing database pruning to VC.
Fix GetBlockHeaderResponse object declaration (spec has been changed).

* Switch to getFinalizedBlockHeader instead.

* Fix proper sign.
Add statements.
Show pruning log statement only when pruning happens.

* Optimize and remove debugging helpers.
2023-11-06 16:40:44 +02:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Eugene Kabanov 8cec3af61c
VC: Obtain randao signature before slot proposal. (#5490)
* Randao calculation caching for VC implementation.

* Add time monitoring for randao signatures process.

* Add delay calculation.

* Address review comments.

* Address review comments.
2023-11-04 08:14:14 +01:00
zah 35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
Eugene Kabanov 98e08a9e02
Fix cancellation deprecate warnings. (#5454)
Add noCancel in exception handlers.
Refactor pending cancellations code.
2023-09-24 09:28:09 +02:00
Etan Kissling e7bc41e005
`blck` --> `forkyBlck` when using `withBlck` / `withStateAndBlck` (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
Eugene Kabanov aec953e4da
VC: Fix VC must not crash if beacon node address could not be resolved. (#5388)
* Fix VC should not crash, if beacon node URL could not be resolved.

* Bump chronos.

* Update .gitmodules.
2023-09-15 07:34:32 +02:00
Etan Kissling d7aaf1b809
only request `nextPeriod` sync duties close to end of `currentPeriod` (#5409)
Currently we always request duties for current and next sync period.
As sync periods are quite long (~27 hrs on Mainnet), having access to
the duties so early doesn't help too much. To avoid running into errors
when the BN does not have the duties available around period boundary,
delay requesting them until the current period is close to finish.

`SYNC_COMMITTEE_SUBNET_COUNT` epochs are what the spec says should be
the lookahead timing of starting to subscribe to sync committee gossip.
Reusing the constant here for consistency.

This fixes these warning messages in the first slot of a new period.

```
rocketpool_validator   | WRN 2023-09-07 20:19:35.439+00:00 Beacon node has incompatible configuration reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
rocketpool_validator   | WRN 2023-09-07 20:19:35.440+00:00 Unable to get sync committee duties        period=889 epoch=227584 reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" service=duties_service
rocketpool_validator   | NOT 2023-09-07 20:19:35.441+00:00 Beacon node is in sync                     head_slot=7274495 sync_distance=1 is_optimistic=false node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
```
2023-09-13 08:58:28 +02:00
Eugene Kabanov 3c3c4e8edf
Fix /eth/v1/validator/liveness/{epoch} call names and links in code (#5392)
* Fix getLiveness() call links and names.

* Add getLiveness() decoding test.

* Remove BN state updates from getLiveness() call.

* Fix AllTests.

* Reduce number of tests.
2023-09-11 12:07:34 +02:00
zah 2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Eugene Kabanov afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Etan Kissling 13f436bc24
explicit `int` array indices for Nim 2.0 (#5306)
In Nim, when using a `distinct` capacity for an array, `int` lookups
don't work. Therefore, define the array with `int` capacity.
2023-08-17 17:26:08 +00:00
Eugene Kabanov 4a84457376
VC: Fix API timeouts issue. (#5297)
* Fix VC timeouts issue.
Add tests.

* Update AllTests.

* Remove assertion, which could crash VC when API call was interrupted by some higher logic.
2023-08-16 16:14:39 +00:00
Eugene Kabanov 025b0e3f04
VC: Refactor sync committee duties and selection proofs calculation. (#5184)
* Initial commit.

* Fix issues in fill*() procedures.

* Enable sync committee processing.

* Address review comments.

* Address remaining review comments.

* Add sync committee selection proof pruning mechanism.

* Fix recursion issue.
2023-08-15 19:00:27 +02:00
tersec 85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
Eugene Kabanov ac1b02698a
VC: Use scoring function to select best attestation data when using multiple BNs. (#5101)
* Initial commit.

* Move score selection log statement to debug level.

* Fix proper float64 log format.

* Cleanup imports and legacy code.

* Address review comments.

* Address review comments.

* Fix scoring function.

* Address review comments.

* Address review comments 2.
Fix registerBlock post-rebase issues.

* Simplify innerLoop decision making.

* Make getAttestationDataScore() more testable.
Add tests for getAttestationDataScore().

* Add modified AllTests copy.
2023-07-06 11:14:22 +02:00
Eugene Kabanov 5e1a0eac85
VC: Runtime config (#5130)
* Make VC able to understand any type of `/eth/v1/config/spec` response without any changes in source code.
Update compatibility checking.
Now VC is able to obtain any constant from `spec` call.

* Remove RestSpecVC declaration.
2023-06-28 15:33:38 +02:00
Eugene Kabanov c534a285b9
Fix polling block monitor from deadlocks when connection has been lost. (#5109)
Fix events block monitor from losing connection forever.
Add node to block received log statement.
2023-06-28 15:33:21 +02:00
Eugene Kabanov c2c5d80a4f
VC: Remote BN clock offset monitoring. (#4846)
* Initial commit.

* Add algorithm in comment.
Remove delays.
Fix logging statement issues.
Change update from epoch to slot.

* Obtain timestamp earlier.

* Add processing delays into algorithm.

* Fix time offset logging to produce integers instead of strings.

* Address review comments.

* Fix copyright year.
Fix updateStatus().

* Remove fields from Slot start log statement.
Fix issues when BN do not support Nimbus Extensions.
Rename metric name and type change.

* Add beacon role to disable time offset check manually.
2023-06-28 15:33:07 +02:00
tersec 614202e30d
automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
henridf fa212515f5
Implement validor client proposals for Deneb (#5094) 2023-06-23 09:30:16 +00:00
Eugene Kabanov e2f7065432
Fix iteration issue. (#5105)
Add elapsed_time to log statement.
2023-06-20 16:41:20 +00:00
henridf 0f8866d672
Add Deneb beacon API types (#5060)
* Add Deneb beacon API types

- Introduce the `ProduceBlockResponseV2` type for handling responses to `GET /eth/v2/validator/blocks/{slot}` (https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/produceBlockV2)

- Introduce the `RestPublishedSignedBlockContents` type for handling the request body to `POST /eth/v1/beacon/blocks` (https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/ValidatorRequiredApi/publishBlock)

And a few other small changes towards Deneb block production support.

* Address review feedback

* Post-rebase fixes
2023-06-19 08:56:52 +00:00
Etan Kissling e9262ab6a3
ensure sync duties for next epoch are registered in time (#5084)
* ensure sync duties for next epoch are registered in time

For attestations, VC queries duties for current and next epoch.
For sync messages, VC queries for current and next period (if soon).
This means that for sync messages we don't actually have the duties for
next epoch in all situations, leading to situation where VC may miss
sync duties in the final slot of an epoch when using. As duties remain
same within a sync committee period, simply copy them over to next epoch
to avoid this situation without adding network latency.

* Update beacon_chain/validator_client/duties_service.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-06-16 08:42:57 +00:00