Commit Graph

1686 Commits

Author SHA1 Message Date
tersec fd8429da74
rm Web3Signer Bellatrix block support (#6061) 2024-03-11 22:09:31 +00:00
Eugene Kabanov f088e5f57b
Consensus block value calculation for produceBlockV3 API call. (#5873)
* allow specifying get_proposer_reward block root at state.slot

* Add consensus_block_value calculation.

* Address review comments.

* Post-rebase adjustments.

* Use proper state to calculate consensus block value.

* Revert "allow specifying get_proposer_reward block root at state.slot"

This reverts commit 9fef9a8199f63056060527ac2531acc3b0ed8dcb.

* Fix post-revert problems.
Return back to Gwei.

* Adding test which is not working.

* Do not use test suite if it does not have post-state.

* Add debug logging.

* Increase logging to track sources of balance changes.

* Fix sync committee rewards/penalties calculation.

* Revert "Increase logging to track sources of balance changes."

This reverts commit 32feb20f2fdb66521401710866cd59ecc9951ef8.

* Adopt new vision to block rewards.

* Add block produce logging to VC.

* Remove rewards.nim.

* Eliminate toWei changes.

* Improve UInt256 shortLog.

* Fix conversion procedure.

* Address review comments.

* Fix test.

* Revert "Fix test."

This reverts commit 4948b2c1ec.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
Co-authored-by: Etan Kissling <etan@status.im>
2024-03-11 14:18:50 +00:00
Jacek Sieka 2d519ac715
remove some unused rest serialization code (#6056) 2024-03-09 11:38:24 +01:00
Etan Kissling a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 (#6038)
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
tersec 816361ed8c
Bloom filter acceleration for deposit processing (#5982) 2024-03-07 23:38:24 +00:00
Etan Kissling 50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` (#6036)
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
Etan Kissling 2c924bcc8c
add test runner for LC data collection tests (#6021)
Introduce a test runner for upcoming EF test suites related to canonical
light client data collection.

- https://github.com/ethereum/consensus-specs/pull/3553
2024-03-05 15:41:09 +01:00
tersec 8222d62500
rm unused phase0 and altair remote signing test code (#6027) 2024-03-05 02:38:34 +01:00
tersec 2a13c09615
add proposer reward accounting to block transitions (#6022)
* add proposer reward accounting to block transitions

* Update beacon_chain/spec/state_transition_block.nim

Co-authored-by: Etan Kissling <etan@status.im>

---------

Co-authored-by: Etan Kissling <etan@status.im>
2024-03-04 17:00:46 +00:00
Etan Kissling 542854e743
use `ConsensusFork.init` for parsing lowercase version (#6015)
Instead of maintaining string to `ConsensusFork` version inline,
use shared helper that also comes with static assertion checks
for completion.
2024-03-02 23:45:20 +01:00
tersec b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
tersec 8015c0bb47
rm spec/forks capella_mev usage which had been for remote signer (#6009) 2024-03-02 01:12:48 +01:00
tersec e12f5e0bba
rm more Capella builder API remote signer support (#6012) 2024-03-01 22:29:47 +00:00
tersec 0d48535f9a
refactor get_flag_and_inactivity_deltas() loop body to use per-validator get_flag_and_inactivity_delta() (#6007) 2024-03-01 16:31:25 +00:00
tersec 84034c0379
rm Capella builder API-related remote signer support (#6003) 2024-03-01 05:30:09 +00:00
tersec f076502e25
rm Capella builder API bid types and blinded block construction (#6002) 2024-03-01 00:02:13 +00:00
tersec 5da2bcd243
rm Capella builder API REST calls (#5997) 2024-02-29 12:37:08 +00:00
Etan Kissling 728f65080f
remove an unused metric in `rest_remote_signer_calls` (#5999)
The `nbc_remote_signer_communication_errors` metric is unused. Remove to
avoid verbose `XDeclaredButNotUsed` hint.
2024-02-29 12:00:15 +00:00
tersec 84b752c7a1
rm REST blinded forked Capella block support (#5994) 2024-02-28 18:27:26 +00:00
tersec 2b91b66679
rm REST Capella builder API support (#5981) 2024-02-28 01:10:19 +00:00
Etan Kissling 4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
Etan Kissling 00510a9d2f
fix `Eth-Consensus-Block-Value` reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00
tersec fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec f65c1121d2
add Electra overloads for spec functions; add Electra block processing (#5963) 2024-02-26 02:38:21 +00:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
Etan Kissling f54fa083b4
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-25 15:25:26 +01:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec feec45ba76
export electra definitions via forks (#5951) 2024-02-24 13:43:58 +00:00
tersec de8ac999c7
add sanity checks for ELECTRA_FORK_VERSION and ELECTRA_FORK_EPOCH; add support for Electra to ForkyFoo (#5950) 2024-02-24 12:14:45 +01:00
tersec e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
tersec 95f4b7ddaa
add electra datatypes (#5947) 2024-02-24 02:43:10 +01:00
Etan Kissling a7b9efe4d6
remove MIRACL Core support (#5941)
EIP-4844 requires BLST via `nim-kzg4844`; MIRACL Core is not supported.
Therefore, remove support for the MIRACL Core backend.

- https://github.com/status-im/nim-blscurve/pull/170
2024-02-22 16:13:31 +00:00
tersec 0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Jacek Sieka 1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
tersec ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations (#5910) 2024-02-20 05:14:52 +00:00
tersec 87ae60f780
search for validator indices backwards while processing deposits (#5914) 2024-02-20 06:34:57 +02:00
Etan Kissling 4fc1550d0f
add `{.push raises: [].}` to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
tersec e410fe0052
https://github.com/ethereum/consensus-specs/pull/3600 (#5896) 2024-02-17 09:02:50 +00:00
tersec ea29e0afc8
use 1.4.0-beta.7-hotfix consensus spec test vectors (#5894) 2024-02-16 04:49:18 +00:00
Jacek Sieka 9aabca6a64
Clean up debug/heads v2 types (#5859) 2024-02-07 17:51:12 +01:00
Jacek Sieka 47704bde14
raises for beacon validators & router (#5826)
Changes here are more significant because of some good old tech debt in
block production which has grown quite hairy - the reduction in
exception handling at least provides some steps in the right direction.
2024-02-07 12:26:04 +01:00
Etan Kissling 94a65c2a9e
log `extra_data` instead of `extra_data_len` for `ExecutionPayload` (#5851)
Add more details to execution payload logs, reusing the same facilities
that we already use for `GraffitiBytes`.
2024-02-07 10:09:25 +01:00
Etan Kissling 41403022bb
prevent accidentally hashing `BeaconState`/`BeaconBlock` in Deneb (#5852)
Extend protection against accidentally calling computationally expensive
functions when a cache is available to Deneb, as done for earlier forks.
2024-02-06 19:57:53 +01:00
Eugene Kabanov 21efe7e060
VC: Use produceBlockV3 when its available. (#5842)
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
tersec 8b261dd3e0
fix blob_sidecar SSE versioned_hash field to be 0x-prefixed hex (#5844) 2024-01-31 04:50:24 +01:00
tersec 87052eba4e
implement getBlindedBlock REST API (#5829) 2024-01-31 03:18:55 +00:00
tersec 128834a8eb
use `RestPlainResponse` to improve builder API rerror reporting (#5819) 2024-01-24 23:27:22 +00:00
tersec d8a2690a92
update builder API spec reference URLs to v0.4.0 (#5812) 2024-01-22 08:36:46 +01:00
tersec 4ec36e0670
Revert "use `RestPlainResponse` to improve builder API rerror reporting" (#5811)
* Revert "use `RestPlainResponse` to improve builder API rerror reporting"

* Update rest_deneb_mev_calls.nim

copyright year linting

* Update rest_capella_mev_calls.nim

more copyright year linting
2024-01-21 22:39:45 +00:00