Commit Graph

1694 Commits

Author SHA1 Message Date
Etan Kissling 5d42859176
make `Gwei` `distinct` (#6090)
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
Etan Kissling 1dd2c939ac
bump `nim-web3` to `80c7aa6de2a26c57fa1f06ad47f3ac6058e6545b` (#6088)
- Add writeValue for BlockNumber
- make `BlockNumber` `distinct`
2024-03-19 14:21:47 +01:00
Jacek Sieka ed1ef19bf4
use `assign` for forky state assignment (#6055) 2024-03-19 09:50:25 +01:00
Etan Kissling ef2411e1a0
use correct `INACTIVITY_SCORE_RECOVERY_RATE` if overridden from default (#6091)
When a config defines a different `INACTIVITY_SCORE_RECOVERY_RATE` than
the default, `process_inactivity_updates` uses an incorrect rate ever
since #2710 when `INACTIVITY_SCORE_RECOVERY_RATE` became configurable.
2024-03-17 13:32:30 +01:00
tersec 0a6d189161
automated consensus spec URL updating to v1.4.0 (#6074) 2024-03-14 07:26:36 +01:00
Eugene Kabanov 72c844534f
Add Keymanager API graffiti endpoints. (#6054)
* Initial commit.

* Add more tests.

* Fix API mistypes.

* Fix mistypes in tests.

* Fix one more mistype.

* Fix affected tests because of error code 401.

* Add GetGraffitiResponse object.

* Add more tests.

* Fix compilation errors.

* Recover old behavior.

* Recover old behavior.

* Fix mistype.

* Test could not know default graffiti value.

* Make VC use adopted graffiti settings.

* Make BN use adopted graffiti settings.

* Update Alltests.

* Fix test.

* Revert "Fix test."

This reverts commit c735f855d3cb9c4a1c8e8af29d3f4438d068e31f.

* Workaround {.push raises.} requirement.

* Fix comment.

* Update Alltests.
2024-03-14 03:44:00 +00:00
tersec c3016a9bc5
use v1.4.0 consensus spec test vectors (#6072) 2024-03-13 21:42:40 +01:00
Etan Kissling 8bd8ffe2bb
align default `syncHorizon` computation logic across networks (#6066)
The `syncHorizon` describes the number of empty slots before the beacon
node considers itself to be out of sync. There are two places where we
currently set this to 50 slots, but it makes more sense to base it on
wall time, e.g., the 10 minutes that the default 50 are derived from.
2024-03-12 21:51:18 +01:00
tersec fd8429da74
rm Web3Signer Bellatrix block support (#6061) 2024-03-11 22:09:31 +00:00
Eugene Kabanov f088e5f57b
Consensus block value calculation for produceBlockV3 API call. (#5873)
* allow specifying get_proposer_reward block root at state.slot

* Add consensus_block_value calculation.

* Address review comments.

* Post-rebase adjustments.

* Use proper state to calculate consensus block value.

* Revert "allow specifying get_proposer_reward block root at state.slot"

This reverts commit 9fef9a8199f63056060527ac2531acc3b0ed8dcb.

* Fix post-revert problems.
Return back to Gwei.

* Adding test which is not working.

* Do not use test suite if it does not have post-state.

* Add debug logging.

* Increase logging to track sources of balance changes.

* Fix sync committee rewards/penalties calculation.

* Revert "Increase logging to track sources of balance changes."

This reverts commit 32feb20f2fdb66521401710866cd59ecc9951ef8.

* Adopt new vision to block rewards.

* Add block produce logging to VC.

* Remove rewards.nim.

* Eliminate toWei changes.

* Improve UInt256 shortLog.

* Fix conversion procedure.

* Address review comments.

* Fix test.

* Revert "Fix test."

This reverts commit 4948b2c1ec.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
Co-authored-by: Etan Kissling <etan@status.im>
2024-03-11 14:18:50 +00:00
Jacek Sieka 2d519ac715
remove some unused rest serialization code (#6056) 2024-03-09 11:38:24 +01:00
Etan Kissling a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 (#6038)
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
tersec 816361ed8c
Bloom filter acceleration for deposit processing (#5982) 2024-03-07 23:38:24 +00:00
Etan Kissling 50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` (#6036)
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
Etan Kissling 2c924bcc8c
add test runner for LC data collection tests (#6021)
Introduce a test runner for upcoming EF test suites related to canonical
light client data collection.

- https://github.com/ethereum/consensus-specs/pull/3553
2024-03-05 15:41:09 +01:00
tersec 8222d62500
rm unused phase0 and altair remote signing test code (#6027) 2024-03-05 02:38:34 +01:00
tersec 2a13c09615
add proposer reward accounting to block transitions (#6022)
* add proposer reward accounting to block transitions

* Update beacon_chain/spec/state_transition_block.nim

Co-authored-by: Etan Kissling <etan@status.im>

---------

Co-authored-by: Etan Kissling <etan@status.im>
2024-03-04 17:00:46 +00:00
Etan Kissling 542854e743
use `ConsensusFork.init` for parsing lowercase version (#6015)
Instead of maintaining string to `ConsensusFork` version inline,
use shared helper that also comes with static assertion checks
for completion.
2024-03-02 23:45:20 +01:00
tersec b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
tersec 8015c0bb47
rm spec/forks capella_mev usage which had been for remote signer (#6009) 2024-03-02 01:12:48 +01:00
tersec e12f5e0bba
rm more Capella builder API remote signer support (#6012) 2024-03-01 22:29:47 +00:00
tersec 0d48535f9a
refactor get_flag_and_inactivity_deltas() loop body to use per-validator get_flag_and_inactivity_delta() (#6007) 2024-03-01 16:31:25 +00:00
tersec 84034c0379
rm Capella builder API-related remote signer support (#6003) 2024-03-01 05:30:09 +00:00
tersec f076502e25
rm Capella builder API bid types and blinded block construction (#6002) 2024-03-01 00:02:13 +00:00
tersec 5da2bcd243
rm Capella builder API REST calls (#5997) 2024-02-29 12:37:08 +00:00
Etan Kissling 728f65080f
remove an unused metric in `rest_remote_signer_calls` (#5999)
The `nbc_remote_signer_communication_errors` metric is unused. Remove to
avoid verbose `XDeclaredButNotUsed` hint.
2024-02-29 12:00:15 +00:00
tersec 84b752c7a1
rm REST blinded forked Capella block support (#5994) 2024-02-28 18:27:26 +00:00
tersec 2b91b66679
rm REST Capella builder API support (#5981) 2024-02-28 01:10:19 +00:00
Etan Kissling 4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
Etan Kissling 00510a9d2f
fix `Eth-Consensus-Block-Value` reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00
tersec fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec f65c1121d2
add Electra overloads for spec functions; add Electra block processing (#5963) 2024-02-26 02:38:21 +00:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
Etan Kissling f54fa083b4
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-25 15:25:26 +01:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec feec45ba76
export electra definitions via forks (#5951) 2024-02-24 13:43:58 +00:00
tersec de8ac999c7
add sanity checks for ELECTRA_FORK_VERSION and ELECTRA_FORK_EPOCH; add support for Electra to ForkyFoo (#5950) 2024-02-24 12:14:45 +01:00
tersec e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
tersec 95f4b7ddaa
add electra datatypes (#5947) 2024-02-24 02:43:10 +01:00
Etan Kissling a7b9efe4d6
remove MIRACL Core support (#5941)
EIP-4844 requires BLST via `nim-kzg4844`; MIRACL Core is not supported.
Therefore, remove support for the MIRACL Core backend.

- https://github.com/status-im/nim-blscurve/pull/170
2024-02-22 16:13:31 +00:00
tersec 0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Jacek Sieka 1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
tersec ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations (#5910) 2024-02-20 05:14:52 +00:00
tersec 87ae60f780
search for validator indices backwards while processing deposits (#5914) 2024-02-20 06:34:57 +02:00
Etan Kissling 4fc1550d0f
add `{.push raises: [].}` to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
tersec e410fe0052
https://github.com/ethereum/consensus-specs/pull/3600 (#5896) 2024-02-17 09:02:50 +00:00
tersec ea29e0afc8
use 1.4.0-beta.7-hotfix consensus spec test vectors (#5894) 2024-02-16 04:49:18 +00:00
Jacek Sieka 9aabca6a64
Clean up debug/heads v2 types (#5859) 2024-02-07 17:51:12 +01:00
Jacek Sieka 47704bde14
raises for beacon validators & router (#5826)
Changes here are more significant because of some good old tech debt in
block production which has grown quite hairy - the reduction in
exception handling at least provides some steps in the right direction.
2024-02-07 12:26:04 +01:00