Commit Graph

495 Commits

Author SHA1 Message Date
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
Etan Kissling 29fe958908
allow to pass trusted node sync options during BN startup (#5545)
Using trusted node sync currently requires to run two commands -
first the `trustedNodeSync` command to initialize the database,
followed by the regular startup command to continue syncing.

The `trustedNodeSync` options are now also available during regular
startup, and are used when the database is empty to simplify setting up
a new Nimbus beacon node. This also aligns behaviour closer with other
Ethereum consensus implementations.

The new logic only applies if the database has not yet been initialized;
same as before. Also, the database needs to be removed under the same
conditions as before when a fresh sync is desired.
2023-11-03 16:07:49 +01:00
tersec ed446b1d5a
replace ValidIpAddress with IpAddress in configurations (#5536) 2023-11-01 08:33:00 +01:00
Etan Kissling 173582ad70
initialize KZG crypto before starting network (#5543)
Move KZG trusted setup initialization before `BeaconNode.init` to avoid
edge case where network message is received and processed before crypto
library has been properly initialized. Followup from #4870.
2023-10-31 13:43:46 +01:00
Etan Kissling d289da8cd4
extract trusted node sync trigger into separate function (#5544)
To prepare for calling trusted node sync from the main Nimbus startup
logic, extract the trusted node sync trigger into a separate function.
Further allow passing a pre-opened database, as that will be needed to
check whether trusted node sync needs to be called during regular start.
2023-10-31 01:56:52 +01:00
tersec 4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
zah 35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec 447786518f
ShufflingRef approach to next-epoch validator duty calculation/prediction (#5414)
* ShufflingRef approach to next-epoch validator duty calculation/prediction

* refactor action_tracker.updateActions to take ShufflingRef + beacon_proposers; refactor maybeUpdateActionTrackerNextEpoch to be separate and reused function; add actual fallback logic

* document one possible set of conditions

* check epoch participation flags and inactivity scores to ensure no penalties and MAX_EFFECTIVE_BALANCE to ensure rewards don't matter

* correctly (un)shuffle each proposer index

* remove debugging assertion
2023-10-10 00:02:07 +00:00
Zahary Karadjov 77d6bc5f42
Merge stable into unstable 2023-09-28 11:32:54 +01:00
Etan Kissling 7c45b8f98d
convert remaining `toFork` -> `kind` for consistency (#5462)
We currently have multiple ways to obtain `ConsensusFork` or
`LcDataFork` from a forky type. Rename `toFork` to `kind`
for a consistent API naming.
2023-09-27 15:10:28 +00:00
Zahary Karadjov cfa0268d89
Version 23.9.1
Changes:

* Updated network metadata for Holesky
* Use hash_tree_root instead of SHA256 when verifying the Holesky
  genesis state
2023-09-25 21:24:25 +01:00
Eugene Kabanov 88565e6da4
Make KeystoreCache working with KeystoreManagement API. (#5443) 2023-09-21 20:07:02 +02:00
tersec 2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Jacek Sieka e64b31986e
Increase default file limit on startup (#5436)
We use file descriptors for validators and sockets and might run out of
either on high-validator setups - increasing the limit here is harmless
and avoids a common limiting factor in setup

Co-authored-by: Etan Kissling <etan@status.im>
2023-09-21 16:08:13 +02:00
Etan Kissling e7bc41e005
`blck` --> `forkyBlck` when using `withBlck` / `withStateAndBlck` (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
Etan Kissling ee57a07411
implement Confirmation rule prerequisite - fork choice filter change (#5450)
To support confirmation rule via beacon-APIs as described in spec PR,
add `--debug-fork-choice-version=pr3431` option and enable when Deneb
fork is scheduled. To opt-out, `--debug-fork-choice-version=stable`,
or don't schedule Deneb.

- https://github.com/ethereum/consensus-specs/pull/3431
- https://github.com/ethereum/consensus-specs/issues/3466

"will bundle this with deneb release":

- https://github.com/ethereum/pm/issues/844#issuecomment-1673359012
2023-09-20 18:27:55 +02:00
Etan Kissling 8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
Zahary Karadjov 6b1ab81046
Support both snappy-encoded and pure SSZ genesis states 2023-09-12 01:52:15 +03:00
tersec 2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Etan Kissling ff176a1569
fix `syncCommitteeDuties` log in `Slot end` message (#5416)
In `Slot end`, sync duties are reported one slot too late,
because sync committee duties are determined by the next slot
instead of the current one. Address that by taking account of this.
2023-09-11 10:51:55 +00:00
Zahary Karadjov c7db7d0f2c
Add CLI options `--genesis-state` and `--genesis-state-url` 2023-09-08 13:10:18 +03:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
tersec fbcb476ff9
allow archiving blobs (#5391) 2023-09-07 08:19:33 +02:00
zah 2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Eugene Kabanov afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
tersec a8c56b1660
update some consensus-spec URLs to v1.4.0-beta.1 (#5379) 2023-09-01 09:31:52 +00:00
Jacek Sieka 778a209d09
move ctrl-c handler out of exported function (#5347)
otherwise it gets set also for unit tests that callit which interferes
with gdb.
2023-08-28 10:40:23 +02:00
tersec db6f4e8090
update some consensus-spec URLs to v1.4.0-beta.1 (#5357) 2023-08-25 15:58:44 +00:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
tersec ef184c8959
subscribe to Deneb blob subnets (#5342) 2023-08-23 19:31:41 +00:00
Jacek Sieka a5151bc546
`validator_duties`->`beacon_validators` (#5333)
This PR renames the existing `validator_duties` to `beacon_validators`
and in doing so, names validators running inside the beacon node process
"beacon validators" while those running the VC can be referred to as
"client validators" to disambiguate the two.

The existing `validator_duties` instead takes on a new responsibility:
as a home for logic shared between beacon and client validators - ie
code that provides consistency in implementation and behavior between
the two modes of operation.

Not only does this simplify reasoning about where to put code -it also
reduces the number of dependencies the validator client has from ~5000
to ~3000 modules (!) according to `nim genDepend` significantly reducing
compile times.
2023-08-23 19:39:57 +03:00
Jacek Sieka 49729e1ef3
prevent concurrent `storeBlock` calls (fixes #5285) (#5295)
When a block is introduced to the system both via REST and gossip at the
same time, we will call `storeBlock` from two locations leading to a
dupliace check race condition as we wait for the EL.

This issue may manifest in particular when using an external block
builder that itself publishes the block onto the gossip network.

* refactor enqueue flow
* simplify calling `addBlock`
* complete request manager verifier future for blobless blocks
* re-verify parent conditions before adding block

among other things, it might have gone stale or finalized between one
call and the other
2023-08-17 15:12:37 +02:00
tersec d171303133
update some consensus spec URLs to v1.4.0-beta.1 (#5287) 2023-08-12 10:38:06 +00:00
henridf 9ceed40090
Prune blobs (#5283)
* Prune blobs

* Address review feedback
2023-08-11 00:15:34 +00:00
tersec 85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
tersec 1c55ef7345
note summary data about invalid baked-in states (#5247) 2023-08-04 07:36:42 +02:00
henridf 1da82a5dcd
Remove async blob queue from Request Manager (#5198)
* Remove async blob queue from Request Manager

* Review feedback

* Review feedback
2023-08-01 22:39:14 +02:00
Etan Kissling dd35d2d309
delay `onSlotEnd` if there are duties (#5196)
We currently call `onSlotEnd` whenever all in-BN validator duties are
completed. VC validator duties are not awaited. When `onSlotEnd` is
processed close to the slot start, a VC may therefore miss duties.
Adding a delay before `onSlotEnd` improves this situation.
The logic can be optimized further if `ActionTracker` would track
`knownValidators` from REST separately from in-process ones.
2023-07-18 20:55:36 +02:00
Jacek Sieka ca1775f725
Fetch by-root request directly from quarantine (#5167)
When the requestmanager is busy fetching blocks, the queue might get
filled with multiple entries of the same root - since there is no
deduplication, requests containing the same root multiple times will be
sent out.

Also, because the items sit in the queue for a long time potentially,
the request might be stale by the time that the manager is ready with
the previous request.

This PR removes the queue and directly fetches the blocks to download
from the quarantine which solves both problems (the quarantine already
de-duplicates and is clean of stale information).

Removing the queue for blobs is left for a future PR.

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 18:22:02 +02:00
tersec 49056c3e39
use v1.3.0 stability subnet behavior when appropriate (#5156) 2023-07-07 04:16:12 +00:00
tersec 0be95571a7
update some consensus spec URLs to v1.4.0-beta.0 (#5150) 2023-06-30 16:12:54 +00:00
tersec ba94dc849f
opt-out support for v1.4.0 attestation stability subnets (#5128)
* opt-out support for v1.4.0 attestation stability subnets

* document debug-prefix-flag-policy and remove temporary from flag name
2023-06-30 05:23:15 +00:00
henridf 1234900065
Add blob handling to message router (#5106)
* Add blob handling to message router

* address review feedback

* Fix typos
2023-06-28 17:55:31 +00:00
Eugene Kabanov c2c5d80a4f
VC: Remote BN clock offset monitoring. (#4846)
* Initial commit.

* Add algorithm in comment.
Remove delays.
Fix logging statement issues.
Change update from epoch to slot.

* Obtain timestamp earlier.

* Add processing delays into algorithm.

* Fix time offset logging to produce integers instead of strings.

* Address review comments.

* Fix copyright year.
Fix updateStatus().

* Remove fields from Slot start log statement.
Fix issues when BN do not support Nimbus Extensions.
Rename metric name and type change.

* Add beacon role to disable time offset check manually.
2023-06-28 15:33:07 +02:00
tersec ba597ef0a2
per-validator payload builder configuration (#5062) 2023-06-25 12:00:17 +00:00
tersec 614202e30d
automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
Etan Kissling 2722778ce5
reduce `nim-eth` dependencies just for RNG (#5099)
We have several modules that import `nim-eth` for the sole purpose of
its `keys.newRng` function. This function is meanwhile a simple wrapper
around `nim-bearssl`'s `HmacDrbgContext.new()`, so the import doesn't
really serve a use anymore. Replace `keys.newRng` with the direct call
to reduce `nim-eth` imports.
2023-06-19 22:43:50 +00:00