6472 Commits

Author SHA1 Message Date
tersec
2b91b66679
rm REST Capella builder API support (#5981) 2024-02-28 01:10:19 +00:00
Etan Kissling
1f48795e91
bump nim-taskpools to d4c43137c0590cb47f893a66ca8cb027fa6c217e (#5978)
- test refc in CI in Nim 2.0 and later
2024-02-27 23:55:18 +00:00
Etan Kissling
80ed64d2ff
bump nim-stint to 7bb0ccc7842577243edc3b8d61f75bf498551a7a (#5977)
- Upgrade github actions to v4
2024-02-27 22:49:28 +00:00
Etan Kissling
170036af18
bump nim-snappy to 984bdad602e977ea6c461c28201422fdeeccfb7c (#5975)
- use non-EOL macOS version for GitHub Actions CI
2024-02-27 20:46:31 +00:00
Etan Kissling
3a74821d1e
bump nim-results to dfeef8ba705d7bab3e2073bb44549f0a2562df70 (#5974)
- Formatted with nph v0.4-0-g537f1ce
- Add 592e0dcba157da84e2cced2309fa27dc67b667f3 to .git-blame-ignore-revs
- restore 1.2 compat in tests
2024-02-27 21:06:42 +01:00
Etan Kissling
a38e17cd8d
bump nim-presto to 223aadeb82d35b57e6dae99f0b325ec6345ce7ff (#5973)
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
- Add implementation of PRESTO middleware.
2024-02-27 19:36:37 +00:00
Etan Kissling
0356eda0db
bump nim-nat-traversal to d423ff9e436b8a8219926f4e3350b09108e42847 (#5972)
- bump `miniupnp` to `miniupnpc_2_2_6`
2024-02-27 18:02:23 +00:00
Etan Kissling
9d6be476cf
bump nim-kzg4844 to 39bee1b4c91a476ddcce04cd31e095063e993c2c (#5970)
- Upgrade github actions to v4
2024-02-27 17:48:56 +00:00
Etan Kissling
78b1514745
bump nim-blscurve to d091a579a2e7c4668140e675a6fb2c78b8c6dc57 (#5969)
- remove MIRACL Core backend
2024-02-27 16:50:10 +00:00
Etan Kissling
f1076bacaa
bump geth to v1.13.14 (#5968)
- https://github.com/ethereum/go-ethereum/releases/tag/v1.13.14
2024-02-27 15:57:18 +00:00
Jacek Sieka
2ea97e34bc
Merge remote-tracking branch 'origin/stable' into unstable 2024-02-27 14:03:41 +01:00
Eugene Kabanov
95280e6766
Fix stint parser to raise ValueError instead of RangeDefect. (#5967) 2024-02-27 03:46:12 +01:00
Zahary Karadjov
fc9c72f0eb
Version v24.2.2 v24.2.2 2024-02-27 03:24:30 +02:00
Zahary Karadjov
4a8d271c35
Bump nim-stint to e639ba700cb83a6b22e5b5a1053bea2820c8b4f6
Prevents RangeDefects when parsing invalid numeric values
2024-02-27 03:06:52 +02:00
Etan Kissling
26d0281e54
fix incorrect config validation regression from #5959 (#5966)
During refactoring of #5959, some implicit `return` were overlooked,
resulting in spurious `err()` being returned without message.

```
{"lvl":"WRN","ts":"2024-02-26 10:12:20.469+00:00","msg":"Beacon nodes report different configuration values","reason":"","service":"fallback_service","node":"http://127.0.0.1:9303[Nimbus/v24.2.1-4e9bc7-stateofus]","node_index":0,"node_roles":"AGBSDT"}
```

Correcting the helpers to return explicit result in all exhaustive
cases so that this cannot happen anymore by accident.
2024-02-26 16:25:12 +02:00
Etan Kissling
4fcfed2f1f
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 16:25:12 +02:00
Etan Kissling
c8adfb2336
use correct signing fork after Deneb when using deposits exit command (#5954)
In #5120, the `nimbus_beacon_node deposits exit` command was updated for
compatibility with EIP-7044, which forces signatures to be made using
`CAPELLA_FORK_VERSION` regardless of the `VoluntaryExit`'s `epoch` after
Deneb is activated.

This update had a regression, as an older mechanism was used to fetch
`RuntimeConfig`, resulting in an encoding issue (#5362). This was then
fixed in #5370, restoring general `deposits exit` functionality.

However, the logic from #5120 has another flaw, as it uses an incorrect
fork version based on the pre-Deneb logic even after Deneb and EIP-7044
are activated. Fix this now, so that `deposits exit` continues to work
correctly after Deneb activates.
2024-02-26 16:25:12 +02:00
Etan Kissling
48a4955e50
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-26 16:25:12 +02:00
Etan Kissling
c3fe055bd4
fix incorrect config validation regression from #5959 (#5966)
During refactoring of #5959, some implicit `return` were overlooked,
resulting in spurious `err()` being returned without message.

```
{"lvl":"WRN","ts":"2024-02-26 10:12:20.469+00:00","msg":"Beacon nodes report different configuration values","reason":"","service":"fallback_service","node":"http://127.0.0.1:9303[Nimbus/v24.2.1-4e9bc7-stateofus]","node_index":0,"node_roles":"AGBSDT"}
```

Correcting the helpers to return explicit result in all exhaustive
cases so that this cannot happen anymore by accident.
2024-02-26 13:32:01 +01:00
Etan Kissling
4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
Etan Kissling
00510a9d2f
fix Eth-Consensus-Block-Value reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00
tersec
fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec
f65c1121d2
add Electra overloads for spec functions; add Electra block processing (#5963) 2024-02-26 02:38:21 +00:00
Etan Kissling
acb1eb1ac6
extend notes on random Jenkins aarch64 test failures (#5962)
The weird `let` bug from #5757 appeared again :-) Document findings.
2024-02-26 03:02:03 +01:00
Etan Kissling
457e68eab3
use correct signing fork after Deneb when using deposits exit command (#5954)
In #5120, the `nimbus_beacon_node deposits exit` command was updated for
compatibility with EIP-7044, which forces signatures to be made using
`CAPELLA_FORK_VERSION` regardless of the `VoluntaryExit`'s `epoch` after
Deneb is activated.

This update had a regression, as an older mechanism was used to fetch
`RuntimeConfig`, resulting in an encoding issue (#5362). This was then
fixed in #5370, restoring general `deposits exit` functionality.

However, the logic from #5120 has another flaw, as it uses an incorrect
fork version based on the pre-Deneb logic even after Deneb and EIP-7044
are activated. Fix this now, so that `deposits exit` continues to work
correctly after Deneb activates.
2024-02-25 22:09:23 +01:00
tersec
a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587bc4f0c91b8e2f58884665a0ae80e34.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec
33ce235d47
document git-lfs requirement (#5957) 2024-02-25 18:19:05 +00:00
Etan Kissling
f54fa083b4
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-25 15:25:26 +01:00
tersec
d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec
feec45ba76
export electra definitions via forks (#5951) 2024-02-24 13:43:58 +00:00
tersec
de8ac999c7
add sanity checks for ELECTRA_FORK_VERSION and ELECTRA_FORK_EPOCH; add support for Electra to ForkyFoo (#5950) 2024-02-24 12:14:45 +01:00
tersec
e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
tersec
bf6b5a316f
add push raises to tests; rm unused test code (#5948) 2024-02-24 04:41:58 +01:00
tersec
95f4b7ddaa
add electra datatypes (#5947) 2024-02-24 02:43:10 +01:00
Etan Kissling
a7b9efe4d6
remove MIRACL Core support (#5941)
EIP-4844 requires BLST via `nim-kzg4844`; MIRACL Core is not supported.
Therefore, remove support for the MIRACL Core backend.

- https://github.com/status-im/nim-blscurve/pull/170
2024-02-22 16:13:31 +00:00
Etan Kissling
97527e2ca5
bump nim-web3 to 15325bd844a38234e4f705b3399d0bbfa3b63a12 (#5938)
- Replace eth_getBlockReceipts return type with Opt[T] instead of Option[T]
- use non-EOL macOS version and fix spelling
- test refc in CI in Nim 2.0 and later
2024-02-22 13:09:25 +00:00
Etan Kissling
1cf9d87137
bump nim-nat-traversal to a48100fa23b3f07895683f3ca30c6be2bac22ca0 (#5934)
- bump `miniupnp` to `miniupnpc_2_2_5`
- require minimum of Nim 1.6
2024-02-22 13:02:48 +00:00
Etan Kissling
fd54423a0a
bump nim-serialization to afae13adac25b6fa98bacf4b9f38458dc64317b0 (#5935)
- Upgrade to github actions v4
- Silence the infamous Conversion from itself warnings
2024-02-22 12:28:42 +00:00
Etan Kissling
c5cb0ea3f6
bump nim-testutils to ae476c67314ac4b294d21040315e0f716189a70e (#5936)
- Resolve ambiguity between chronicles and macros on nim v2
- Upgrade github action to v3
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 11:46:45 +00:00
Etan Kissling
cce8f25263
bump nim-unittest2 to b5028a44710f0317c5e710c80cec90127aa7e77c (#5937)
- unittest2.nim: ensure the testTeardownIMPL is performed at the end
- Add test for PR #35
- only run `teardown` if `setup` completed
- Fix allow break in test body regression
- Upgrade github actions to v4
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 10:36:25 +00:00
Etan Kissling
702f4c6a45
add {.push raises: [].} to more files (#5943)
Extend `{.push raises: [].}` to all files with spec references that are
periodically updated.

- https://github.com/status-im/nimbus-eth2/pull/5942
2024-02-22 11:03:09 +01:00
Etan Kissling
7642b0c04f
bump nim-zxcvbn to d0fb557f74a28912ff129ce0ba24008b00b1019b (#5939)
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 08:49:48 +00:00
tersec
0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
Etan Kissling
3863373575
bump geth to v1.13.13 (#5944)
- https://github.com/ethereum/go-ethereum/releases/tag/v1.13.13
2024-02-21 21:06:31 +00:00
tersec
c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Jacek Sieka
1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
Etan Kissling
86aee03bcf
bump nim-libbacktrace to 008de2cd425cbca19f7d922e4e8b209d92313752 (#5931)
- bump `whereami` to `c5f6b015c36964790493841b9b4623184c0f7c3f`
- require minimum of Nim 1.6
2024-02-21 14:26:18 +00:00
Etan Kissling
0c745e2f92
bump nim-http-utils to be57dbc902d36f37540897e98c69aa80f868cb45 (#5928)
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
2024-02-21 14:16:03 +00:00
Etan Kissling
e5ba18c55d
bump nim-zlib to 45b06fca15ce0f09586067d950da30c10227865a (#5933)
- bump `csources` to `v1.3.1`
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
2024-02-21 12:06:51 +00:00
Etan Kissling
c4535f5bb6
bump nim-metrics to 0e768ca7fb7df4798649145a403e7db65cae1f8b (#5932)
- Upgrade to github actions v4
- Fix 'value' template compilation error
- fix missing import
- Partial instantiation of 'value' template
2024-02-21 11:15:00 +00:00