Commit Graph

6008 Commits

Author SHA1 Message Date
Etan Kissling 7c45b8f98d
convert remaining `toFork` -> `kind` for consistency (#5462)
We currently have multiple ways to obtain `ConsensusFork` or
`LcDataFork` from a forky type. Rename `toFork` to `kind`
for a consistent API naming.
2023-09-27 15:10:28 +00:00
Eugene Kabanov 4fb95d000d
REST server fixes and improvements. (#5422)
* Move from Option[T] to Opt[T] usage.

* Add `finalized` flag.

* Fix compilation issue.

* Http415 error code for some REST API calls.
Introduce more comprehensive error reporting for block calls.
Deprecate decodeEthConsensusVersion() function.

* Bump http-utils.

* Fix copyright year.

* Fix serialization issue.

* Address review comments.

* Post rebase fixes.
2023-09-27 16:45:33 +02:00
Zahary Karadjov cfa0268d89
Version 23.9.1
Changes:

* Updated network metadata for Holesky
* Use hash_tree_root instead of SHA256 when verifying the Holesky
  genesis state
2023-09-25 21:24:25 +01:00
Jacek Sieka 96e05e4f37
stint: bump (#5459)
fixes 32-bit compilation
2023-09-25 21:08:48 +02:00
tersec 710f26745d
small attestation pool clean up (#5458) 2023-09-24 08:50:48 +00:00
Eugene Kabanov 98e08a9e02
Fix cancellation deprecate warnings. (#5454)
Add noCancel in exception handlers.
Refactor pending cancellations code.
2023-09-24 09:28:09 +02:00
Eugene Kabanov e3fe762ec9
Fix unknown fields should be ignored when parsing liveness responses. (#5418) 2023-09-23 08:44:54 +02:00
Etan Kissling 47639ef89c
replace `Eth2NetworkMetadata` case object with `Result` (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
Etan Kissling 5150505e66
bump `nim-ssz-serialization` to `a5767c1b4992dca3e5bd72557aadd4df0f183bf8` (#5457)
- nimble/ci updates
- use common super-type in `raisesssz` pragma
2023-09-22 23:44:57 +02:00
Eugene Kabanov caaad4777c
BN+LC+SN: Fix cancellation deprecate warnings. (#5455)
* Fix deprecation cancel() warnings in SN, BN, LC.

* Fix missing import.
2023-09-22 11:06:27 +00:00
Eugene Kabanov 88565e6da4
Make KeystoreCache working with KeystoreManagement API. (#5443) 2023-09-21 20:07:02 +02:00
tersec 2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Jacek Sieka e64b31986e
Increase default file limit on startup (#5436)
We use file descriptors for validators and sockets and might run out of
either on high-validator setups - increasing the limit here is harmless
and avoids a common limiting factor in setup

Co-authored-by: Etan Kissling <etan@status.im>
2023-09-21 16:08:13 +02:00
Etan Kissling e7bc41e005
`blck` --> `forkyBlck` when using `withBlck` / `withStateAndBlck` (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
tersec 351472a2d4
per payload-builder payload builder validator registration (#5431) 2023-09-20 17:00:37 +00:00
Etan Kissling ee57a07411
implement Confirmation rule prerequisite - fork choice filter change (#5450)
To support confirmation rule via beacon-APIs as described in spec PR,
add `--debug-fork-choice-version=pr3431` option and enable when Deneb
fork is scheduled. To opt-out, `--debug-fork-choice-version=stable`,
or don't schedule Deneb.

- https://github.com/ethereum/consensus-specs/pull/3431
- https://github.com/ethereum/consensus-specs/issues/3466

"will bundle this with deneb release":

- https://github.com/ethereum/pm/issues/844#issuecomment-1673359012
2023-09-20 18:27:55 +02:00
Etan Kissling 273f1d34cd
implement EIP-7514 for Deneb: Add Max Epoch Churn Limit (#5442)
* implement EIP-7514 for Deneb: Add Max Epoch Churn Limit

Cap activations per epoch according to EIP-7514:

- https://eips.ethereum.org/EIPS/eip-7514
- https://github.com/ethereum/consensus-specs/pull/3499

* apply proposer boost to first block in case of equivocation

Implement spec changes to fork choice; this only affects equivocation
when multiple blocks are signed for the same slot. Regular operation
is not changed.

- https://github.com/ethereum/consensus-specs/pull/3352

* bump test vectors to v1.4.0-beta.2-hotfix

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-09-20 13:39:56 +02:00
Etan Kissling aacb853a0b
add blob validation to `test_fixture_fork_choice` (#5441)
Preparation for processing new tests from:

- https://github.com/ethereum/consensus-specs/pull/3463
2023-09-20 01:14:49 +00:00
Etan Kissling eadd0d69ce
bump `nim-kzg4844` to `6c406c70ad8cde6ab56320158dd969beb45322ef` (#5448)
- bump `csources` to `v0.3.1`
2023-09-19 23:29:50 +00:00
Etan Kissling 1640c45a95
reject invalid proofs when validating blobs (#5445)
Currently, passing `0xc00000...` proof seems to pass `verifyProofs`.
Unsure why such a check is not necessary in spec, and also unsure
whether it is correct to reject proof at infinity, or if it could
occur, e.g., for a blob containing all 0 bytes. Weird overall...

* proper fix
2023-09-19 22:13:58 +00:00
Etan Kissling c1b43d166b
avoid repeated total active balance computation in fork choice (#5437)
We currently compute `justified_total_active_balance` inside
`calculateProposerBoost`, despite that sum already being known
in the `EpochRef` cache. Tracking `justified_total_active_balance`
whenever the justified checkpoint updates allows replacing the
repeated computation with a lookup, at minimal memory cost.
2023-09-19 22:04:55 +02:00
tersec ee75d45a8b
fix `ATTESTATION_PROPAGATION_SLOT_RANGE` unknown constant warning (#5433)
* fix ATTESTATION_PROPAGATION_SLOT_RANGE unknown constant warning

* verify underflow not an issue
2023-09-19 06:54:58 +00:00
Eugene Kabanov 521e7f535c
Bump chronos & presto. (#5394)
* Bump chronos & presto.

* Increase finalization timeouts for 10 minutes.

* Return Jenkins timeouts back to normal.
Bump chronos.

* Bump presto.
2023-09-19 08:01:58 +02:00
Miran 7eaa8f832b
Fix #2013, add images to the repo (#5438)
* add img for health.md

* add imgs for metrics.md

* add imgs for email.md

* add img for developers.md

* add imgs for grafana/readme.md

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-09-19 08:00:48 +02:00
Miran 244f930f42
update Holesky docs (#5435)
* update Holesky docs

* Update docs/the_nimbus_book/src/holesky.md

Co-authored-by: Jacek Sieka <jacek@status.im>

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: Etan Kissling <etan@status.im>
2023-09-19 08:00:27 +02:00
Etan Kissling a261384efc
exclude `*.service` files from lint check (#5440)
These `*.service` files are installed on user installations, so
referring to a license file outside their installation doesn't make
too much sense.
2023-09-18 21:54:06 +00:00
Etan Kissling 63440766ef
exclude 'public-keys/all.txt' file from checkout (#5439)
The 'vendor/holesky/public-keys/all.txt' file is not used by Nimbus,
but sometimes fails to download due to payment issues in `@eth-clients`.
Skipping the download of that file should reduce that risk.

```
Downloading public-keys/all.txt (240 MB)
Error downloading object: public-keys/all.txt (ba977b5): Smudge error: Error downloading public-keys/all.txt (ba977b5b1da180914c934623fce187860e1b54cff939e6208533b2cb5f589e07): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
```

* bump nbs: Patch libnatpmp.a rule to support llvm-mingw in targets.mk

* bump nbs: Allow passing config to submodule initialization in `update-common`
2023-09-18 20:24:13 +02:00
Tanguy 4918a4e2e0
Fix direct peers (#5427)
* Fix direct peers

* Support ENRs in DP, use DP in local testnet

* fix docs

* bump libp2p
2023-09-15 18:45:55 +00:00
Jacek Sieka b4159416c4
grafana: update dashboard
* new mesh efficiency board for measuring IDONTWANT
* fix scaling of several message/slot boards
2023-09-15 16:50:36 +02:00
tersec 20fea37735
update Docker base images to include `git-lfs` (#5430)
* update Docker base images to include git-lfs

* add copyright years
2023-09-15 10:45:49 +00:00
Eugene Kabanov 12ef43bcc0
Fix issue when VC could not be interrupted while its in pre-genesis period. (#5424) 2023-09-15 12:39:26 +02:00
tersec 325d5a6afa
log builder unblinding requests (#5434) 2023-09-15 09:27:38 +00:00
tersec dae4beb4a2
log engine getPayload requests (#5432) 2023-09-15 08:56:32 +00:00
Eugene Kabanov aec953e4da
VC: Fix VC must not crash if beacon node address could not be resolved. (#5388)
* Fix VC should not crash, if beacon node URL could not be resolved.

* Bump chronos.

* Update .gitmodules.
2023-09-15 07:34:32 +02:00
tersec 5a29ad7e4f
update some consensus-spec URLs to v1.4.0-beta.1 (#5429) 2023-09-14 18:23:59 +00:00
tersec 7a001d4896
remove 4 more missing constants in preset handling warning (#5428) 2023-09-14 12:37:35 +00:00
Etan Kissling bb8584c58a
provide hint when Git LFS is not installed (#5425)
Add a hint message during git clone to direct the user to install
Git LFS if it is missing. It is required to clone Holesky submodule.

---------

Co-authored-by: Jakub <jakub@status.im>
2023-09-14 12:09:02 +02:00
Etan Kissling 176ea09c2b
cleanup `OnBlockAdded` usage (#5426)
Reduce repetitiveness when using forked `OnBlockAdded` callbacks by
introducing a template to obtain appropriate cb from `ConsensusFork`.
2023-09-13 17:57:54 +00:00
Etan Kissling d7aaf1b809
only request `nextPeriod` sync duties close to end of `currentPeriod` (#5409)
Currently we always request duties for current and next sync period.
As sync periods are quite long (~27 hrs on Mainnet), having access to
the duties so early doesn't help too much. To avoid running into errors
when the BN does not have the duties available around period boundary,
delay requesting them until the current period is close to finish.

`SYNC_COMMITTEE_SUBNET_COUNT` epochs are what the spec says should be
the lookahead timing of starting to subscribe to sync committee gossip.
Reusing the constant here for consistency.

This fixes these warning messages in the first slot of a new period.

```
rocketpool_validator   | WRN 2023-09-07 20:19:35.439+00:00 Beacon node has incompatible configuration reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
rocketpool_validator   | WRN 2023-09-07 20:19:35.440+00:00 Unable to get sync committee duties        period=889 epoch=227584 reason="Epoch value is far from the future;400;getSyncCommitteeDuties(first);invalid-request" service=duties_service
rocketpool_validator   | NOT 2023-09-07 20:19:35.441+00:00 Beacon node is in sync                     head_slot=7274495 sync_distance=1 is_optimistic=false node=http://eth2:5052[Nimbus/v23.8.0-872b19-stateofus] node_index=0 node_roles=AGBSDT
```
2023-09-13 08:58:28 +02:00
Etan Kissling cd68c71c6c
add gossip tests for period boundary (#5423)
Test `validateSyncCommitteeMessage` and `validateContribution`
around sync committee period boundary to cover edge cases.
2023-09-13 08:32:11 +02:00
Etan Kissling 4700030231
fix sync contribution validation for first slot per period (#5408)
Sync committee duties are performed by the sync committee as determined
by slot + 1. We did it correctly for individual messages, but selected
the incorrect participants for aggregate contributions for the very
first slot per period (roughly 1 per ~27 hrs on Mainnet). The faulty
participants selection code was originally introduced in #2925.
2023-09-13 03:23:18 +02:00
andri lim 5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
tersec d4543239d7
rm state_sim WASM wrapper/demo (#5421) 2023-09-12 15:03:18 +00:00
Miran 73ab2e5f39
styling improvements for holesky.md (#5420)
- compact syntax for different options (as in the rest of the guide)
- correct numeration of the list (continuation vs starting from 1)
- add links to useful resources
- a note stating that Prater is deprecated
2023-09-12 13:53:55 +03:00
Etan Kissling 8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
tersec a06c0c550f
enable Windows GitHub Actions with Nim 20 (#5413) 2023-09-12 04:57:20 +00:00
Zahary Karadjov de8d52eb1a
Add Holesky section to the guide 2023-09-12 03:50:29 +03:00
Zahary Karadjov 568e1fbb8c
Final Release Notes for v23.9.0 2023-09-12 02:00:38 +03:00
Zahary Karadjov 6b1ab81046
Support both snappy-encoded and pure SSZ genesis states 2023-09-12 01:52:15 +03:00
Etan Kissling 672c69b0bf
increase Windows test stack size for Nim 2.0 (#5419)
Nim 2.0 has trouble running the `test_libnimbus_lc` test with the MinGW
default main thread stack size of 2 MB. Bump test stack size to 8 MB.
2023-09-11 20:59:07 +00:00