364 Commits

Author SHA1 Message Date
tersec
27921406e9
remove some debugRaiseAsserts and fill in Electra functionality (#6179) 2024-04-06 15:11:47 +02:00
tersec
0d534224b3
remove some debugRaiseAsserts and fill in Electra functionality (#6178) 2024-04-06 07:46:02 +00:00
tersec
0b6f75b88b
remove some debugRaiseAsserts and fill in actual Electra functionality (#6172) 2024-04-04 05:17:31 +02:00
tersec
7fa32b7f02
add Electra to ConsensusFork enum (#6169)
* add Electra to ConsensusFork enum

* fix gnosis check
2024-04-03 16:43:43 +02:00
Etan Kissling
5f4fa9ae69
avoid code repetition across forks for signed block contents (#6150)
Use forks sugar to make `RestPublishedSignedBlockContents` more concise.
2024-04-03 02:05:29 +02:00
Etan Kissling
5d42859176
make Gwei distinct (#6090)
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
Etan Kissling
1dd2c939ac
bump nim-web3 to 80c7aa6de2a26c57fa1f06ad47f3ac6058e6545b (#6088)
- Add writeValue for BlockNumber
- make `BlockNumber` `distinct`
2024-03-19 14:21:47 +01:00
Eugene Kabanov
72c844534f
Add Keymanager API graffiti endpoints. (#6054)
* Initial commit.

* Add more tests.

* Fix API mistypes.

* Fix mistypes in tests.

* Fix one more mistype.

* Fix affected tests because of error code 401.

* Add GetGraffitiResponse object.

* Add more tests.

* Fix compilation errors.

* Recover old behavior.

* Recover old behavior.

* Fix mistype.

* Test could not know default graffiti value.

* Make VC use adopted graffiti settings.

* Make BN use adopted graffiti settings.

* Update Alltests.

* Fix test.

* Revert "Fix test."

This reverts commit c735f855d3cb9c4a1c8e8af29d3f4438d068e31f.

* Workaround {.push raises.} requirement.

* Fix comment.

* Update Alltests.
2024-03-14 03:44:00 +00:00
Jacek Sieka
2d519ac715
remove some unused rest serialization code (#6056) 2024-03-09 11:38:24 +01:00
Etan Kissling
a0bc3fff86
fix /eth/v1/beacon/deposit_snapshot for EIP-4881 (#6038)
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
Etan Kissling
542854e743
use ConsensusFork.init for parsing lowercase version (#6015)
Instead of maintaining string to `ConsensusFork` version inline,
use shared helper that also comes with static assertion checks
for completion.
2024-03-02 23:45:20 +01:00
tersec
b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
tersec
84034c0379
rm Capella builder API-related remote signer support (#6003) 2024-03-01 05:30:09 +00:00
tersec
f076502e25
rm Capella builder API bid types and blinded block construction (#6002) 2024-03-01 00:02:13 +00:00
tersec
5da2bcd243
rm Capella builder API REST calls (#5997) 2024-02-29 12:37:08 +00:00
Etan Kissling
728f65080f
remove an unused metric in rest_remote_signer_calls (#5999)
The `nbc_remote_signer_communication_errors` metric is unused. Remove to
avoid verbose `XDeclaredButNotUsed` hint.
2024-02-29 12:00:15 +00:00
tersec
84b752c7a1
rm REST blinded forked Capella block support (#5994) 2024-02-28 18:27:26 +00:00
Etan Kissling
4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
Etan Kissling
00510a9d2f
fix Eth-Consensus-Block-Value reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00
tersec
fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec
a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587bc4f0c91b8e2f58884665a0ae80e34.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec
d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
Jacek Sieka
1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
Etan Kissling
4fc1550d0f
add {.push raises: [].} to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
Jacek Sieka
9aabca6a64
Clean up debug/heads v2 types (#5859) 2024-02-07 17:51:12 +01:00
Jacek Sieka
47704bde14
raises for beacon validators & router (#5826)
Changes here are more significant because of some good old tech debt in
block production which has grown quite hairy - the reduction in
exception handling at least provides some steps in the right direction.
2024-02-07 12:26:04 +01:00
Eugene Kabanov
21efe7e060
VC: Use produceBlockV3 when its available. (#5842)
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
tersec
87052eba4e
implement getBlindedBlock REST API (#5829) 2024-01-31 03:18:55 +00:00
tersec
36545e1d84
remove expensive logging from function called in prepareBeaconProposer inner loop (#5776) 2024-01-17 22:58:46 +00:00
tersec
cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
Jacek Sieka
b98f46c04d
Avoid global in p2p macro (fixes #4578) (#5719)
* Avoid global in p2p macro (fixes #4578)

* copy p2p macro to this repo and start de-crufting it
* make protocol registration dynamic, removing light client hacks et al
* split out light client protocol into its own file

* cleanups

* Option -> Opt
* remove more cruft

* further split beacon_sync

this allows the light client to respond to peer metadata messages
without exposing the block sync protocol

* better protocol init

* "constant" protocol index

* avoid casts

* copyright

* move some discovery code to discovery

* avoid extraneous data copy when sending chunks

* remove redundant forkdigest field

* document how to connect to a specific peer
2024-01-13 11:54:24 +02:00
tersec
69af8f943e
implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
Jacek Sieka
62cbdeefc5
verify genesis_time more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
tersec
cafa1f3d3d
rm unused RPC signatures replaced by Plain versions (#5685) 2024-01-03 05:58:34 +00:00
Etan Kissling
583782a061
emit proposer_slashing/attester_slashing SSE on beacon-API (#5678)
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling
a2081521f6
emit bls_to_execution_change SSE on beacon-API (#5677)
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
zah
29b29e1945
Use the new strict Json flavors mechanism for RestJson (#5674)
This requires all object types to be explicitly white-listed for
default serialization. The PR makes the minimal changes, although
a number of similar mechanisms in eth2_rest_serialization can now
be removed.
2023-12-19 17:44:27 +02:00
andri lim
15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
Eugene Kabanov
26bcb7057c
REST: Add implementation of postStateValidators and postStateValidatorBalances. (#5632)
* Add implementation POST versions of /eth/v1/beacon/states/{state_id}/validators and /eth/v1/beacon/states/{state_id}/validator_balances.
Add tests.

* Address review comments.
Fix toList() issue.

* Fix tests.

* Address review comments 2.

* Address review comments 3.
Fix unique check for validator identifiers.

* Address review comments.

* Fix constant value.
2023-11-29 12:05:03 +00:00
Eugene Kabanov
e2e4912645
REST API produceBlockV3 implementation (#5474)
Co-authored-by: Etan Kissling <etan@status.im>
Co-authored-by: Jacek Sieka <jacek@status.im>
2023-11-29 00:30:14 +01:00
tersec
115ffa70eb
rm unused code (#5623) 2023-11-25 12:09:18 +00:00
Etan Kissling
98e969084d
update Deneb for latest builder-specs flow (#5598)
The `BlobSidecar` construction has been moved to the relay and is no
longer done by the BN / VC in blinded flow. Builder bid contents have
been shrinked from full `BlindedBlobBundle` to `blob_kzg_commitments`.

- https://github.com/ethereum/builder-specs/pull/90
- https://github.com/ethereum/beacon-APIs/pull/369
2023-11-15 16:20:13 -08:00
Eugene Kabanov
9889b840ce
VC: new scoring functions. (#5447)
* Initial commit.

* Fix issues and tests.

* Fix test compilation issue.

* Update AllTests.

* Change the most poor score name from <lowest> to <bad>.
Split sync committee message score in range, so lexicographic scores will not intersect with normal one.
Lexicographic scores should be below to normal scores.

* Address review comments.
Fix aggregated attestation scoring to use MAX_VALIDATORS_PER_COMMITTEE.
Fix sync committee contributions to use SYNC_SUBCOMMITTEE_SIZE.
Add getUniqueVotes test vectors.

* Post-rebase fixes.

* Address review comments.

* Return back score calculation based on actual bits length.

* AllTests modification.
2023-11-14 12:13:26 +01:00
Eugene Kabanov
6bc038e8d5
VC: Obol middleware support (#5375) 2023-11-08 14:03:51 +02:00
Eugene Kabanov
49c851109e
VC: Add pruning slashing database. (#5551)
* Add slashing database pruning to VC.
Fix GetBlockHeaderResponse object declaration (spec has been changed).

* Switch to getFinalizedBlockHeader instead.

* Fix proper sign.
Add statements.
Show pruning log statement only when pruning happens.

* Optimize and remove debugging helpers.
2023-11-06 16:40:44 +02:00
Etan Kissling
d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Eugene Kabanov
e1f47bb27a
Address #5539. (#5548) 2023-11-01 08:32:41 +01:00
Eugene Kabanov
1a5bcb479e
Fix broadcast_validation handling in publishBlockV2 (fixes #5531) (#5547)
* Address issue #5531.

* Add more tests.

* Add to resttest ability to check values.
Fix tests.
2023-11-01 08:31:18 +01:00
tersec
41dfc9ed91
add KZGProof and Blob REST JSON serializations (#5526)
* add KZGProof and Blob REST JSON serializations

* error summary updates

* copyright notice

* consistent integer types for Nim 2.0
2023-10-25 23:50:59 +02:00
tersec
5aa5095e17
Fix REST JSON KzgCommitment encoding/decoding to use hex strings per spec (#5522) 2023-10-24 15:22:59 +00:00