Commit Graph

1694 Commits

Author SHA1 Message Date
Etan Kissling 94a65c2a9e
log `extra_data` instead of `extra_data_len` for `ExecutionPayload` (#5851)
Add more details to execution payload logs, reusing the same facilities
that we already use for `GraffitiBytes`.
2024-02-07 10:09:25 +01:00
Etan Kissling 41403022bb
prevent accidentally hashing `BeaconState`/`BeaconBlock` in Deneb (#5852)
Extend protection against accidentally calling computationally expensive
functions when a cache is available to Deneb, as done for earlier forks.
2024-02-06 19:57:53 +01:00
Eugene Kabanov 21efe7e060
VC: Use produceBlockV3 when its available. (#5842)
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
tersec 8b261dd3e0
fix blob_sidecar SSE versioned_hash field to be 0x-prefixed hex (#5844) 2024-01-31 04:50:24 +01:00
tersec 87052eba4e
implement getBlindedBlock REST API (#5829) 2024-01-31 03:18:55 +00:00
tersec 128834a8eb
use `RestPlainResponse` to improve builder API rerror reporting (#5819) 2024-01-24 23:27:22 +00:00
tersec d8a2690a92
update builder API spec reference URLs to v0.4.0 (#5812) 2024-01-22 08:36:46 +01:00
tersec 4ec36e0670
Revert "use `RestPlainResponse` to improve builder API rerror reporting" (#5811)
* Revert "use `RestPlainResponse` to improve builder API rerror reporting"

* Update rest_deneb_mev_calls.nim

copyright year linting

* Update rest_capella_mev_calls.nim

more copyright year linting
2024-01-21 22:39:45 +00:00
tersec 00cd032f7d
rm duplicate imports (#5810) 2024-01-21 10:21:01 +00:00
tersec 195a7525fa
revert non-PR commit 2024-01-21 07:06:09 +00:00
tersec 172374580d
rm duplicate imports 2024-01-21 07:03:42 +00:00
tersec 6c53dc1e11
automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
tersec 545fb17649
use RestPlainResponse to improve builder API rerror reporting (#5777) 2024-01-19 03:20:47 +00:00
Etan Kissling 006fa59ec2
use consensus spec v1.4.0-beta.6 test vectors (#5783) 2024-01-18 18:14:44 +00:00
Etan Kissling 62ee92a094
bolster `BlobSidecar` syncing on incomplete responses (#5766)
Avoid marking blocks invalid when corresponding `blobSidecarsByRange`
returns an incomplete / incorrect response while syncing. The block
itself may still be valid in that scenario.
2024-01-18 15:45:10 +01:00
tersec 36545e1d84
remove expensive logging from function called in prepareBeaconProposer inner loop (#5776) 2024-01-17 22:58:46 +00:00
Jacek Sieka d5785677a8
split out eth1chain into its own module (#5768)
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
Etan Kissling 11ebbe0cac
fix LC header production for Deneb (#5763)
The `blob_gas_used` field was not properly populated when constructing
Deneb light client data. This is due to #5026 not applying the change to
the entire codebase when the new field got introduced, and due to #5350
not catching that oversight in other modules. Also reviewed codebase and
discovered that `shortLog` for Deneb execution payloads has same bug.
2024-01-16 22:54:20 +00:00
tersec cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
Etan Kissling 7443a4ac08
load Ethereum mainnet KZG setup on Gnosis networks (#5756)
Gnosis networks re-use the trusted setup from Ethereum mainnet.
Load it to support Deneb.
2024-01-16 01:58:07 +01:00
tersec 3541cfe020
remove extraneous length checks in KZG batch proofs (#5744)
* remove extraneous length checks in KZG batch proofs

* re-add winservice import but only for Windows, to avoid UnusedImport warning

* also uses establishWindowsService
2024-01-15 16:53:34 +01:00
Jacek Sieka b98f46c04d
Avoid global in p2p macro (fixes #4578) (#5719)
* Avoid global in p2p macro (fixes #4578)

* copy p2p macro to this repo and start de-crufting it
* make protocol registration dynamic, removing light client hacks et al
* split out light client protocol into its own file

* cleanups

* Option -> Opt
* remove more cruft

* further split beacon_sync

this allows the light client to respond to peer metadata messages
without exposing the block sync protocol

* better protocol init

* "constant" protocol index

* avoid casts

* copyright

* move some discovery code to discovery

* avoid extraneous data copy when sending chunks

* remove redundant forkdigest field

* document how to connect to a specific peer
2024-01-13 11:54:24 +02:00
tersec 69af8f943e
implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
tersec 1559a09184
add more logging of KZG commitments, proofs, and blobs (#5735) 2024-01-13 07:44:40 +01:00
Etan Kissling 16256a5230
display current fork (+ next fork if applicable) in slot start / status (#5731)
Extend slot start message and default status bar with information about
current head fork and the next fork transition (corresponding to head).
This is useful to know whether a synced client is aware of a future fork
and can also be useful when syncing from old forks to follow progress
across the various forks.

```
 peers: 8 ❯ finalized: 741c2ce2:230474 ❯ head: b330f58b:230477:20 ❯ fork: Capella (next: Deneb:231680) ❯ time: 230599:24 (7379192) ❯ sync: 00h24m (99.63%) 2.6492slots/s (QwQUwQPQDQ:7375263)/opt
```

```
INF 2024-01-12 12:18:00.001+01:00 Slot start                                 topics="beacnde" slot=7379190 epoch=230599 fork="Capella (next: Deneb:231680)" sync="--h--m (99.62%) 0.0000slots/s (wwwwwwwwww:7375167)/opt" peers=0 head=741c2ce2:7375168 finalized=230472:723abe7e delay=1ms861us
```
2024-01-12 21:40:34 +01:00
Jacek Sieka e4a1ae67df
ssz: bump (#5717)
height-based merkleizer
2024-01-11 18:34:44 +01:00
tersec 07455e67a9
`proc` to `func` in engine authentication (#5718) 2024-01-08 21:12:58 +00:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 508f3b6368
add missing `std/` prefix to more imports (#5696)
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
Etan Kissling 7db95f047b
track latest `LightClientUpdate` only once fork choice selects it (#5691)
Instead of tracking the latest `LightClientUpdate` across all branches,
track the latest one on the current branch as selected by fork choice.
2024-01-03 23:36:05 +01:00
tersec cafa1f3d3d
rm unused RPC signatures replaced by Plain versions (#5685) 2024-01-03 05:58:34 +00:00
Etan Kissling 583782a061
emit `proposer_slashing`/`attester_slashing` SSE on beacon-API (#5678)
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling a2081521f6
emit `bls_to_execution_change` SSE on beacon-API (#5677)
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
zah 29b29e1945
Use the new strict Json flavors mechanism for RestJson (#5674)
This requires all object types to be explicitly white-listed for
default serialization. The PR makes the minimal changes, although
a number of similar mechanisms in eth2_rest_serialization can now
be removed.
2023-12-19 17:44:27 +02:00
tersec 9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates (#5672) 2023-12-16 03:27:06 +01:00
andri lim 15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
tersec df902fd00f
log execution payload (header) block_hash and parent_hash in block shortLogs (#5660) 2023-12-10 01:25:51 +01:00
Etan Kissling d8728b7b28
re-add comment for `PayloadAttributesV2` usage (#5658)
As followup from #5654, ensure that we still keep the comment around
referring to the correct `forkchoiceUpdated` to use being driven by
the fork schedule.
2023-12-08 03:50:38 +01:00
Etan Kissling 0a5d9ee027
use `PayloadAttributesV3` in `nimbus_light_client` for Deneb (#5654)
* use `PayloadAttributesV3` in `nimbus_light_client` for Deneb

From Deneb onward, `forkchoiceUpdated` requires `PayloadAttributesV3`.
In `nimbus_light_client` we still used `PayloadAttributesV2`.

Also clean up two other locations that were already correctly using
`PayloadAttributesV3`, to reduce code duplication.

* fix letter case
2023-12-07 17:10:22 +00:00
tersec 4776fecc33
consensus spec v1.4.0-beta.5 URL updates (#5655) 2023-12-06 22:16:55 +00:00
tersec 9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 (#5647) 2023-12-05 03:34:45 +01:00
tersec 1791b0ee0a
fix XDeclaredButNotUsed warnings (#5638) 2023-12-04 21:14:35 +00:00
tersec 6a07602aca
use consensus spec test vectors v1.4.0-beta.5 (#5645) 2023-12-04 22:00:27 +01:00
tersec 2fc43c9ba7
track block/blob matching/quarantines using both indices and commitments (#5621) 2023-12-01 18:58:46 +00:00
tersec 6a9d522705
Apply EIP-7044 to block signature batch verification (#5637) 2023-12-01 14:44:45 +00:00
Eugene Kabanov 26bcb7057c
REST: Add implementation of postStateValidators and postStateValidatorBalances. (#5632)
* Add implementation POST versions of /eth/v1/beacon/states/{state_id}/validators and /eth/v1/beacon/states/{state_id}/validator_balances.
Add tests.

* Address review comments.
Fix toList() issue.

* Fix tests.

* Address review comments 2.

* Address review comments 3.
Fix unique check for validator identifiers.

* Address review comments.

* Fix constant value.
2023-11-29 12:05:03 +00:00
Eugene Kabanov e2e4912645
REST API produceBlockV3 implementation (#5474)
Co-authored-by: Etan Kissling <etan@status.im>
Co-authored-by: Jacek Sieka <jacek@status.im>
2023-11-29 00:30:14 +01:00
tersec ab5343d1bc
update some consensus spec URLs to v1.4.0-beta.4 (#5631) 2023-11-27 19:56:34 +01:00
tersec 6dee4d59fe
switch engine authentication from `Option` to `Opt` (#5624)
* switch engine authentication from Option to Opt

* el_conf updates

* LC and el_conf tests
2023-11-27 14:48:29 +00:00
tersec 23983c34ac
rm debugRaiseAssert (#5629) 2023-11-26 19:40:10 +01:00
tersec 115ffa70eb
rm unused code (#5623) 2023-11-25 12:09:18 +00:00
Etan Kissling c33dd2c170
restrict best LC update collection to canonical blocks (#5613)
Simplify best `LightClientUpdate` collection by tracking only canonical
data instead of tracking the best update across all branches within the
sync committee period.

- https://github.com/ethereum/consensus-specs/pull/3553
2023-11-21 23:51:05 +01:00
Etan Kissling b179cb71a0
clean up `XDeclaredButNotUsed` for light client (#5606)
Handle remaining `XDeclaredButNotUsed` warnings for light client.
2023-11-18 00:03:27 +01:00
Etan Kissling b9a693d5fe
send LC finality update on event stream on supermajority (#5602)
When new finality is reached without supermajority sync committee
support, trigger another event push on beacon-API and libp2p once
the finality gains supermajority support.

- https://github.com/ethereum/consensus-specs/pull/3549
2023-11-16 19:57:15 -08:00
Etan Kissling 98e969084d
update Deneb for latest builder-specs flow (#5598)
The `BlobSidecar` construction has been moved to the relay and is no
longer done by the BN / VC in blinded flow. Builder bid contents have
been shrinked from full `BlindedBlobBundle` to `blob_kzg_commitments`.

- https://github.com/ethereum/builder-specs/pull/90
- https://github.com/ethereum/beacon-APIs/pull/369
2023-11-15 16:20:13 -08:00
Etan Kissling 0919ff05c9
reduce code duplication when collecting LC data (#5601)
Replace sections that need to be maintained with every `ConsensusFork`
related to LC data collection with a generic logic that keeps working
when unrelated parts of Ethereum change.
2023-11-15 03:21:03 +01:00
Eugene Kabanov 9889b840ce
VC: new scoring functions. (#5447)
* Initial commit.

* Fix issues and tests.

* Fix test compilation issue.

* Update AllTests.

* Change the most poor score name from <lowest> to <bad>.
Split sync committee message score in range, so lexicographic scores will not intersect with normal one.
Lexicographic scores should be below to normal scores.

* Address review comments.
Fix aggregated attestation scoring to use MAX_VALIDATORS_PER_COMMITTEE.
Fix sync committee contributions to use SYNC_SUBCOMMITTEE_SIZE.
Add getUniqueVotes test vectors.

* Post-rebase fixes.

* Address review comments.

* Return back score calculation based on actual bits length.

* AllTests modification.
2023-11-14 12:13:26 +01:00
tersec 54bdda13b4
rm unused code (#5596) 2023-11-11 11:49:34 +03:00
tersec 1ceac20ea6
disallow htr on Opt[Foo] (#5593) 2023-11-11 05:33:54 +00:00
tersec c96163dbca
update some consensus spec URLs to v1.4.0-beta.4 (#5594) 2023-11-11 06:27:53 +01:00
Etan Kissling 9ec7b700a8
bump `gnosis-chain-configs` to `b3fc927da3e790d989edb71bec99c1db00529307` (#5589)
- Ensure all config vars are present
- Remove preset files
- ensure config is synced
2023-11-10 18:04:41 +01:00
Etan Kissling eb35039704
allow higher `MIN_EPOCHS_FOR_BLOCK_REQUESTS` than safe minimum (#5590)
Gnosis uses `MIN_EPOCHS_FOR_BLOCK_REQUESTS` = 33024, but the computed
safe minimum (that Nimbus was using) is 2304. Relax the compatibility
check to allow `MIN_EPOCHS_FOR_BLOCK_REQUESTS` above the safe minimum
and honor `config.yaml` preferences for `MIN_EPOCHS_FOR_BLOCK_REQUESTS`.
2023-11-10 15:04:55 +00:00
Etan Kissling f388d76e49
bump `gnosis-chain-configs` to `b8ae3091439131949a994d638d730b5a5cb60f7a` (#5581)
- fix(ci): specify enr-cli version
- Update deneb config vars
2023-11-10 10:16:30 +01:00
Etan Kissling 101b723f52
make `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` configurable (#5582)
* make `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` configurable

Gnosis uses custom `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` to account
for the faster slot timing, so that blobs still remain available for
roughly the same amount of real time.

Also extend REST config endpoint with full config form `v1.4.0-beta.4`,
and extend compatibility checks when loading configs to reduce warnings.
2023-11-09 23:41:17 +03:00
Etan Kissling d9031d97dc
rename `_INDEX` generalized index constants to `_GINDEX` (#5583)
Rename constants for consistency with upcoming spec adjustment:

- https://github.com/ethereum/consensus-specs/pull/3537
2023-11-09 12:54:44 +00:00
Eugene Kabanov 6bc038e8d5
VC: Obol middleware support (#5375) 2023-11-08 14:03:51 +02:00
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Eugene Kabanov 49c851109e
VC: Add pruning slashing database. (#5551)
* Add slashing database pruning to VC.
Fix GetBlockHeaderResponse object declaration (spec has been changed).

* Switch to getFinalizedBlockHeader instead.

* Fix proper sign.
Add statements.
Show pruning log statement only when pruning happens.

* Optimize and remove debugging helpers.
2023-11-06 16:40:44 +02:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
Etan Kissling a05278e263
wrap `kzgs`/`proofs`/`blobs` fields as `BlobsBundle` (#5562)
Less type conversion / copying by keeping the `BlobsBundle` together.
2023-11-04 13:49:58 +00:00
Etan Kissling fb18f09e55
align `Blobs` limit for SSZ transport (#5558)
Capacity should be set to theoretical limit to ensure correct hash root.
Actual length may be shorter. Only use is `ExecutionPayloadForSigning`
so it doesn't matter yet in practice, but still worth fixing.
2023-11-04 10:25:06 +01:00
Etan Kissling c95f9feec4
remove unused Deneb code (#5556)
The `process_blob_kzg_commitments` step was removed in `v1.4.0-alpha.1`,
but we haven't deleted the now unused functions. Do that now.
2023-11-03 14:37:44 +00:00
Etan Kissling baf81595a7
add `KZG_COMMITMENT_INCLUSION_PROOF_DEPTH` preset constant (#5555)
`v1.4.0-beta.4` added `KZG_COMMITMENT_INCLUSION_PROOF_DEPTH` to preset:

- Spec PR: https://github.com/ethereum/consensus-specs/pull/3531
- Gnosis PR: https://github.com/gnosischain/configs/pull/17
2023-11-03 13:54:21 +00:00
tersec ab89e96265
fill in transactions root and KZG commitments from deneb.BlindedBeaconBlock in state transition (#5553) 2023-11-02 04:56:04 +01:00
tersec 657de2721a
fill blindedblockbody KZG commitments from deneb blinded header (#5550) 2023-11-01 14:00:21 +01:00
Eugene Kabanov e1f47bb27a
Address #5539. (#5548) 2023-11-01 08:32:41 +01:00
Eugene Kabanov 1a5bcb479e
Fix `broadcast_validation` handling in `publishBlockV2` (fixes #5531) (#5547)
* Address issue #5531.

* Add more tests.

* Add to resttest ability to check values.
Fix tests.
2023-11-01 08:31:18 +01:00
tersec 556d5e7114
rm unused code (#5538) 2023-11-01 05:53:09 +01:00
tersec 62d59daaa7
consensus-spec URL updates to v1.4.0-beta.3 (#5541) 2023-10-30 06:44:43 +00:00
tersec 34eade3a99
sign blinded blob sidecars during builder API proposals (#5537) 2023-10-28 05:10:23 +00:00
tersec 41dfc9ed91
add `KZGProof` and `Blob` REST JSON serializations (#5526)
* add KZGProof and Blob REST JSON serializations

* error summary updates

* copyright notice

* consistent integer types for Nim 2.0
2023-10-25 23:50:59 +02:00
tersec 5aa5095e17
Fix REST JSON KzgCommitment encoding/decoding to use hex strings per spec (#5522) 2023-10-24 15:22:59 +00:00
tersec 513c9d9637
fix Deneb RestPublishedSignedBlockContents REST JSON decoding (#5520) 2023-10-24 05:50:32 +00:00
tersec fd21882e24
improve test coverage of post-Bellatrix initialize_beacon_state_from_eth1 (#5517) 2023-10-24 01:58:52 +00:00
tersec b2e81de6db
remove unnecessary Builder API status endpoint usage (#5518) 2023-10-23 23:01:29 +00:00
tersec a986a1bcd8
using deneb_mev for Deneb is less incorrect than using capella_mev (#5516) 2023-10-20 01:39:47 +00:00
tersec 4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
tersec 9fd52ebe9f
use consensus spec v1.4.0-beta.3 test vectors (#5512) 2023-10-19 05:14:02 +00:00
tersec 6eb7d481fa
fix Deneb builder API block body type; introduce and use ExecutionPayloadHeaderAndBlindedBlobsBundle (#5509) 2023-10-18 02:54:02 +00:00
andri lim 0f9b52933e
Bump nim-eth: Change block timestamp from std.Time to distinct uint64 (#5510)
* Bump nim-eth: Change block timestamp from std.Time to distinct uint64

Also change tx.maxFeePerBlobGas from GasInt to UInt256
following Cancun latest spec

* Fix EthTime.now from func to proc due to sideeffects
2023-10-18 07:37:57 +07:00
zah 35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
tersec eeac00d08e
add RPC signature; add type constraints on proposeBlockMEV (#5498) 2023-10-12 11:49:48 +00:00
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec edc5c03230
consistently use SignedBlindedBeaconBlockContents; remove more Bellatrix Builder API remnants (#5493) 2023-10-11 11:48:35 +00:00
tersec 48197e4d55
add/update Deneb Builder API types and RPC signatures (#5492) 2023-10-11 04:25:59 +00:00
tersec 447786518f
ShufflingRef approach to next-epoch validator duty calculation/prediction (#5414)
* ShufflingRef approach to next-epoch validator duty calculation/prediction

* refactor action_tracker.updateActions to take ShufflingRef + beacon_proposers; refactor maybeUpdateActionTrackerNextEpoch to be separate and reused function; add actual fallback logic

* document one possible set of conditions

* check epoch participation flags and inactivity scores to ensure no penalties and MAX_EFFECTIVE_BALANCE to ensure rewards don't matter

* correctly (un)shuffle each proposer index

* remove debugging assertion
2023-10-10 00:02:07 +00:00
Etan Kissling dd1f362536
align `Beacon(Block(Body)?|State)Type` with other fork sugar (#5483)
The templates for `BeaconBlock`, `BeaconBlockBody` and `BeaconState`
are the only ones using a `macro` mechanism for code generation.
This prevents using the dot-syntax style `consensusFork.BeaconFoo`
in some situations, and also tends to trigger naming conflicts,
requiring the `Type` suffix. Furthermore, the `macro` only works
for types that are re-defined in every single `ConsensusFork`.

Replacing with the simpler but more verbose approach used for other
types for consistency and to avoid the downsides of the `macro`.

Furthermore, simplify `test_fixture_sanity_blocks` to use `forks` sugar.
2023-10-05 14:01:40 +02:00
Etan Kissling 297c768816
reduce redundant zero initialization for LC data objects (#5479)
Directly initialize `ForkedLightClientObj` instead of separately first
 setting the `kind` (initializing everything to zero) and then assigning
the forky data after that.
2023-10-04 18:11:45 +02:00
Etan Kissling 8aaea1a14e
simplify fork choice test (#5467)
Reduce overly verbose fork branching logic in fork choice test with
generic implementation based on `consensusFork`.
2023-09-29 13:26:34 +00:00