Commit Graph

6702 Commits

Author SHA1 Message Date
Etan Kissling 00510a9d2f
fix `Eth-Consensus-Block-Value` reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00
tersec fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec f65c1121d2
add Electra overloads for spec functions; add Electra block processing (#5963) 2024-02-26 02:38:21 +00:00
Etan Kissling acb1eb1ac6
extend notes on random Jenkins aarch64 test failures (#5962)
The weird `let` bug from #5757 appeared again :-) Document findings.
2024-02-26 03:02:03 +01:00
Etan Kissling 457e68eab3
use correct signing fork after Deneb when using `deposits exit` command (#5954)
In #5120, the `nimbus_beacon_node deposits exit` command was updated for
compatibility with EIP-7044, which forces signatures to be made using
`CAPELLA_FORK_VERSION` regardless of the `VoluntaryExit`'s `epoch` after
Deneb is activated.

This update had a regression, as an older mechanism was used to fetch
`RuntimeConfig`, resulting in an encoding issue (#5362). This was then
fixed in #5370, restoring general `deposits exit` functionality.

However, the logic from #5120 has another flaw, as it uses an incorrect
fork version based on the pre-Deneb logic even after Deneb and EIP-7044
are activated. Fix this now, so that `deposits exit` continues to work
correctly after Deneb activates.
2024-02-25 22:09:23 +01:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec 33ce235d47
document git-lfs requirement (#5957) 2024-02-25 18:19:05 +00:00
Etan Kissling f54fa083b4
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-25 15:25:26 +01:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec feec45ba76
export electra definitions via forks (#5951) 2024-02-24 13:43:58 +00:00
tersec de8ac999c7
add sanity checks for ELECTRA_FORK_VERSION and ELECTRA_FORK_EPOCH; add support for Electra to ForkyFoo (#5950) 2024-02-24 12:14:45 +01:00
tersec e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
tersec bf6b5a316f
add push raises to tests; rm unused test code (#5948) 2024-02-24 04:41:58 +01:00
tersec 95f4b7ddaa
add electra datatypes (#5947) 2024-02-24 02:43:10 +01:00
Etan Kissling a7b9efe4d6
remove MIRACL Core support (#5941)
EIP-4844 requires BLST via `nim-kzg4844`; MIRACL Core is not supported.
Therefore, remove support for the MIRACL Core backend.

- https://github.com/status-im/nim-blscurve/pull/170
2024-02-22 16:13:31 +00:00
Etan Kissling 97527e2ca5
bump `nim-web3` to `15325bd844a38234e4f705b3399d0bbfa3b63a12` (#5938)
- Replace eth_getBlockReceipts return type with Opt[T] instead of Option[T]
- use non-EOL macOS version and fix spelling
- test refc in CI in Nim 2.0 and later
2024-02-22 13:09:25 +00:00
Etan Kissling 1cf9d87137
bump `nim-nat-traversal` to `a48100fa23b3f07895683f3ca30c6be2bac22ca0` (#5934)
- bump `miniupnp` to `miniupnpc_2_2_5`
- require minimum of Nim 1.6
2024-02-22 13:02:48 +00:00
Etan Kissling fd54423a0a
bump `nim-serialization` to `afae13adac25b6fa98bacf4b9f38458dc64317b0` (#5935)
- Upgrade to github actions v4
- Silence the infamous Conversion from itself warnings
2024-02-22 12:28:42 +00:00
Etan Kissling c5cb0ea3f6
bump `nim-testutils` to `ae476c67314ac4b294d21040315e0f716189a70e` (#5936)
- Resolve ambiguity between chronicles and macros on nim v2
- Upgrade github action to v3
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 11:46:45 +00:00
Etan Kissling cce8f25263
bump `nim-unittest2` to `b5028a44710f0317c5e710c80cec90127aa7e77c` (#5937)
- unittest2.nim: ensure the testTeardownIMPL is performed at the end
- Add test for PR #35
- only run `teardown` if `setup` completed
- Fix allow break in test body regression
- Upgrade github actions to v4
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 10:36:25 +00:00
Etan Kissling 702f4c6a45
add `{.push raises: [].}` to more files (#5943)
Extend `{.push raises: [].}` to all files with spec references that are
periodically updated.

- https://github.com/status-im/nimbus-eth2/pull/5942
2024-02-22 11:03:09 +01:00
Etan Kissling 7642b0c04f
bump `nim-zxcvbn` to `d0fb557f74a28912ff129ce0ba24008b00b1019b` (#5939)
- use non-EOL macOS version for GitHub Actions CI
2024-02-22 08:49:48 +00:00
tersec 0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
Etan Kissling 3863373575
bump `geth` to `v1.13.13` (#5944)
- https://github.com/ethereum/go-ethereum/releases/tag/v1.13.13
2024-02-21 21:06:31 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Jacek Sieka 1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
Etan Kissling 86aee03bcf
bump `nim-libbacktrace` to `008de2cd425cbca19f7d922e4e8b209d92313752` (#5931)
- bump `whereami` to `c5f6b015c36964790493841b9b4623184c0f7c3f`
- require minimum of Nim 1.6
2024-02-21 14:26:18 +00:00
Etan Kissling 0c745e2f92
bump `nim-http-utils` to `be57dbc902d36f37540897e98c69aa80f868cb45` (#5928)
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
2024-02-21 14:16:03 +00:00
Etan Kissling e5ba18c55d
bump `nim-zlib` to `45b06fca15ce0f09586067d950da30c10227865a` (#5933)
- bump `csources` to `v1.3.1`
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
2024-02-21 12:06:51 +00:00
Etan Kissling c4535f5bb6
bump `nim-metrics` to `0e768ca7fb7df4798649145a403e7db65cae1f8b` (#5932)
- Upgrade to github actions v4
- Fix 'value' template compilation error
- fix missing import
- Partial instantiation of 'value' template
2024-02-21 11:15:00 +00:00
Etan Kissling d1bce17109
bump `nim-json-rpc` to `8682bb6c858fbcff2f0070c241ce6aae030753a3` (#5930)
- align maximum body size between Websock and HTTP transports
2024-02-21 08:04:36 +00:00
Etan Kissling 1beb603c67
bump `nim-faststreams` to `8ff78919aa5a23aecaf4a647e23cfa8ecda6736a` (#5927)
- test refc in CI in Nim 2.0 and later
- use supported version of macOS
- stop testing Nim C++ backend in GitHub Actions
2024-02-21 05:40:25 +00:00
Etan Kissling 3e2c320b70
bump `nim-eth` to `d8209f623f837d14c43a9e3fd464b0e199c5d180` (#5926)
- Refactor p2pProtocol internals
- Fix chronos related deprecated warnings in discv5 and uTP
- normalise nimble, ci
- Fix chronos related deprecated warnings in uTP code part II
- Remove usage of stew/shim/net
- Upgrade github actions to v4
- Fix two bugs in Receipts RLP encoding/decoding
- Move Ethereum specific RLP encoding tests under tests/common
- Reduce compiler warnings in rlp
- test refc in CI in Nim 2.0 and later
- test Nim 2.0 in CI and use non-EOL macOS version
- Use asyncraises in p2p
- Don't auto write p2pProtocol macro expansion to file
- Fix improper yield usage in rlpx and refine exception handling
2024-02-21 04:31:07 +00:00
Etan Kissling 6d741467f0
bump `nim-confutils` to `0adf3b7db70736061bf12fa23c2fc51f395b289e` (#5925)
- Dynlib fix for status-go integration
- Add ORC testing for the devel branch
- Upgrade github actions to v4
- Add copyright to source code
- Add push raises
- add missing `{.raises.}` to `addConfigFileContent`
2024-02-21 03:54:43 +00:00
Etan Kissling 50872b2fd4
bump `nimcrypto` to `0c6ddab03a99805239b7875f71d2ca95fbed6f85` (#5924)
- use non-EOL macOS version for GitHub Actions CI and rm AppVeyor and Travis configs
2024-02-21 02:44:24 +00:00
Etan Kissling 2a1bf9944d
bump `nim-json-rpc` to `9c65df305390ab01a13788af5aea78cd62dceb2a` (#5923)
- Client also handle error message if id is null
- Client pass meaningful error to newFut when processMessage failed
- Refactoring: extract rpc handler from HTTP and WebSocket server
- Use pragma push/pop pair to disable warning
- http server better exception handling
- Fix CI badge url
- Upgrade github actions to v4
- Revert "Fix CI badge url"
- HttpAuthHook use async raises
- Move CancelledError handling to outer try/except of RpcWebsocketServer
- Implement RPC batch call both in servers and clients
- v0.4.0
- Should compile if chronicles log turned on
- Add framework to support more optional types
- v0.4.2
- test refc in CI in Nim 2.0 and later
- use non-EOL macOS version for GitHub Actions CI
- avoid failing uninitialized `Future`
- Improve batch call example and wrapper comments
- Fix ws and socket client error handling and add test to #212
- Add build test with chronicles to json enabled
2024-02-20 21:27:06 +01:00
Etan Kissling eefe8841ef
bump `nim-stew` to `1662762c0144854db60632e4115fe596ffa67fca` (#5921)
- Add push raises to shims/macros module
- rename `RangeError` -> `RangeDefect` in test name
2024-02-20 21:21:09 +01:00
Etan Kissling 88045a91cd
rename new timing metrics, as `_total` suffix is implicit (#5917)
* track latest duration instead of total in new timing metrics

Change `db_checkpoint_seconds` and `state_replay_seconds` metrics to
record the latest duration instead of the total. `nim-metrics` already
synthesizes a `_total` metric from these implicitly.

* still have to use inc, metrics only synthesizes the name not the sum

* prefix with `beacon_dag`
2024-02-20 20:34:41 +01:00
Etan Kissling 9286eb6795
bump `nim-kzg4844` to `5eba921b1079376403b8b8e1b44c70248eabbc70` (#5922)
- bump `csources` to `v0.4.1`
- test refc in CI in Nim 2.0 and later
- bump `csources` to `v0.4.2`
2024-02-20 18:45:00 +00:00
Etan Kissling 6fe183f0de
bump `nim-websock` to `ed89f58624ba6482225bb6a3169cefd7e632f498` (#5920)
- Reduce compiler warnings when using new chronos
- Upgrade github action to v3
- Fix autobahn tests
- Reenable tests
- Restrict autobahn tests run to merged PR only
- Upgrade github action setup-python to v3 and up/dn artifact to v3
- Reduce compiler warnings
- Fix github actions badge url
- Reduce compiler warnings
- Upgrade to github actions v4
- Revert "Fix github actions badge url"
- Downgrade upload artifact action to v3
- use non-EOL macOS version for GitHub Actions CI
2024-02-20 17:12:09 +00:00
Etan Kissling 8b494c2f26
bump `nim-chronicles` to `ab3ab545be0b550cca1c2529f7e97fbebf5eba81` (#5918)
- Upgrade to github actions v4
- Silence declared but unused warnings when there is no loggging code generated
2024-02-20 15:21:28 +01:00
Etan Kissling 74eeb0b93c
bump `nim-blscurve` to `1002533ad205fa71d60d6fd8f8c283b9352a5314` (#5916)
- update instructions for tracking upstream MIRACL Core
- bump `bls12-381-tests` to `v0.1.2`
- bump `miracl-core` to `0f67878bee7c4108405deb2b0b5e4e58d1ae30fc`
- test refc in CI in Nim 2.0 and later
- rename `milagro.nims` -> `miracl.nims`
- rename `milagro.nim` -> `miracl.nim`
- rename `milagro(Path|_func)` -> `miracl(Path|_func)`
- rename `milagro` references -> `miracl` in documentation
2024-02-20 13:35:22 +01:00
Etan Kissling ba7549a66e
bump `nim-bearssl` to `7640cc6aa859f6d4ad657c892f14c76dc1d34bd5` (#5915)
- test refc in CI in Nim 2.0 and later
2024-02-20 13:31:36 +01:00
tersec ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations (#5910) 2024-02-20 05:14:52 +00:00
Jacek Sieka 8d465a7d8c
vmon: Missed block metric (#5913)
Validator monitoring gained 2 new metrics for tracking when blocks are
included or not on the head chain.

Similar to attestations, if the block is produced in epoch N, reporting
will use the state when switching to epoch N+2 to do the reporting (so
as to reasonably stabilise the block inclusion in the face of reorgs).
2024-02-20 06:40:18 +02:00
tersec 87ae60f780
search for validator indices backwards while processing deposits (#5914) 2024-02-20 06:34:57 +02:00
Zahary Karadjov 7fe43fc204
Version v24.2.1 2024-02-20 05:49:56 +02:00
Etan Kissling ae12b71ce0
bump `holesky` to `60997dd7e876a67775a7f2d53b852e75bbb2a630` (#5912)
- Add Lodestar holesky bootnode ENRs
2024-02-19 19:27:10 +01:00
tersec 28f69ccc0a
add Prater/Goerli deprecation notice (#5898) 2024-02-19 10:09:39 +00:00
Etan Kissling 698a802168
extend linter to enforce exception tracking is turned on (#5909)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Add a CI task to ensure exceptions keep getting properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-19 11:02:36 +01:00