Commit Graph

1758 Commits

Author SHA1 Message Date
tersec de8ac999c7
add sanity checks for ELECTRA_FORK_VERSION and ELECTRA_FORK_EPOCH; add support for Electra to ForkyFoo (#5950) 2024-02-24 12:14:45 +01:00
tersec e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH (#5949) 2024-02-24 05:08:22 +00:00
tersec 95f4b7ddaa
add electra datatypes (#5947) 2024-02-24 02:43:10 +01:00
Etan Kissling a7b9efe4d6
remove MIRACL Core support (#5941)
EIP-4844 requires BLST via `nim-kzg4844`; MIRACL Core is not supported.
Therefore, remove support for the MIRACL Core backend.

- https://github.com/status-im/nim-blscurve/pull/170
2024-02-22 16:13:31 +00:00
tersec 0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Jacek Sieka 1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
tersec ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations (#5910) 2024-02-20 05:14:52 +00:00
tersec 87ae60f780
search for validator indices backwards while processing deposits (#5914) 2024-02-20 06:34:57 +02:00
Etan Kissling 4fc1550d0f
add `{.push raises: [].}` to recently modified files (#5908)
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
tersec e410fe0052
https://github.com/ethereum/consensus-specs/pull/3600 (#5896) 2024-02-17 09:02:50 +00:00
tersec ea29e0afc8
use 1.4.0-beta.7-hotfix consensus spec test vectors (#5894) 2024-02-16 04:49:18 +00:00
Jacek Sieka 9aabca6a64
Clean up debug/heads v2 types (#5859) 2024-02-07 17:51:12 +01:00
Jacek Sieka 47704bde14
raises for beacon validators & router (#5826)
Changes here are more significant because of some good old tech debt in
block production which has grown quite hairy - the reduction in
exception handling at least provides some steps in the right direction.
2024-02-07 12:26:04 +01:00
Etan Kissling 94a65c2a9e
log `extra_data` instead of `extra_data_len` for `ExecutionPayload` (#5851)
Add more details to execution payload logs, reusing the same facilities
that we already use for `GraffitiBytes`.
2024-02-07 10:09:25 +01:00
Etan Kissling 41403022bb
prevent accidentally hashing `BeaconState`/`BeaconBlock` in Deneb (#5852)
Extend protection against accidentally calling computationally expensive
functions when a cache is available to Deneb, as done for earlier forks.
2024-02-06 19:57:53 +01:00
Eugene Kabanov 21efe7e060
VC: Use produceBlockV3 when its available. (#5842)
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
tersec 8b261dd3e0
fix blob_sidecar SSE versioned_hash field to be 0x-prefixed hex (#5844) 2024-01-31 04:50:24 +01:00
tersec 87052eba4e
implement getBlindedBlock REST API (#5829) 2024-01-31 03:18:55 +00:00
tersec 128834a8eb
use `RestPlainResponse` to improve builder API rerror reporting (#5819) 2024-01-24 23:27:22 +00:00
tersec d8a2690a92
update builder API spec reference URLs to v0.4.0 (#5812) 2024-01-22 08:36:46 +01:00
tersec 4ec36e0670
Revert "use `RestPlainResponse` to improve builder API rerror reporting" (#5811)
* Revert "use `RestPlainResponse` to improve builder API rerror reporting"

* Update rest_deneb_mev_calls.nim

copyright year linting

* Update rest_capella_mev_calls.nim

more copyright year linting
2024-01-21 22:39:45 +00:00
tersec 00cd032f7d
rm duplicate imports (#5810) 2024-01-21 10:21:01 +00:00
tersec 195a7525fa
revert non-PR commit 2024-01-21 07:06:09 +00:00
tersec 172374580d
rm duplicate imports 2024-01-21 07:03:42 +00:00
tersec 6c53dc1e11
automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
tersec 545fb17649
use RestPlainResponse to improve builder API rerror reporting (#5777) 2024-01-19 03:20:47 +00:00
Etan Kissling 006fa59ec2
use consensus spec v1.4.0-beta.6 test vectors (#5783) 2024-01-18 18:14:44 +00:00
Etan Kissling 62ee92a094
bolster `BlobSidecar` syncing on incomplete responses (#5766)
Avoid marking blocks invalid when corresponding `blobSidecarsByRange`
returns an incomplete / incorrect response while syncing. The block
itself may still be valid in that scenario.
2024-01-18 15:45:10 +01:00
tersec 36545e1d84
remove expensive logging from function called in prepareBeaconProposer inner loop (#5776) 2024-01-17 22:58:46 +00:00
Jacek Sieka d5785677a8
split out eth1chain into its own module (#5768)
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
Etan Kissling 11ebbe0cac
fix LC header production for Deneb (#5763)
The `blob_gas_used` field was not properly populated when constructing
Deneb light client data. This is due to #5026 not applying the change to
the entire codebase when the new field got introduced, and due to #5350
not catching that oversight in other modules. Also reviewed codebase and
discovered that `shortLog` for Deneb execution payloads has same bug.
2024-01-16 22:54:20 +00:00
tersec cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
Etan Kissling 7443a4ac08
load Ethereum mainnet KZG setup on Gnosis networks (#5756)
Gnosis networks re-use the trusted setup from Ethereum mainnet.
Load it to support Deneb.
2024-01-16 01:58:07 +01:00
tersec 3541cfe020
remove extraneous length checks in KZG batch proofs (#5744)
* remove extraneous length checks in KZG batch proofs

* re-add winservice import but only for Windows, to avoid UnusedImport warning

* also uses establishWindowsService
2024-01-15 16:53:34 +01:00
Jacek Sieka b98f46c04d
Avoid global in p2p macro (fixes #4578) (#5719)
* Avoid global in p2p macro (fixes #4578)

* copy p2p macro to this repo and start de-crufting it
* make protocol registration dynamic, removing light client hacks et al
* split out light client protocol into its own file

* cleanups

* Option -> Opt
* remove more cruft

* further split beacon_sync

this allows the light client to respond to peer metadata messages
without exposing the block sync protocol

* better protocol init

* "constant" protocol index

* avoid casts

* copyright

* move some discovery code to discovery

* avoid extraneous data copy when sending chunks

* remove redundant forkdigest field

* document how to connect to a specific peer
2024-01-13 11:54:24 +02:00
tersec 69af8f943e
implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
tersec 1559a09184
add more logging of KZG commitments, proofs, and blobs (#5735) 2024-01-13 07:44:40 +01:00
Etan Kissling 16256a5230
display current fork (+ next fork if applicable) in slot start / status (#5731)
Extend slot start message and default status bar with information about
current head fork and the next fork transition (corresponding to head).
This is useful to know whether a synced client is aware of a future fork
and can also be useful when syncing from old forks to follow progress
across the various forks.

```
 peers: 8 ❯ finalized: 741c2ce2:230474 ❯ head: b330f58b:230477:20 ❯ fork: Capella (next: Deneb:231680) ❯ time: 230599:24 (7379192) ❯ sync: 00h24m (99.63%) 2.6492slots/s (QwQUwQPQDQ:7375263)/opt
```

```
INF 2024-01-12 12:18:00.001+01:00 Slot start                                 topics="beacnde" slot=7379190 epoch=230599 fork="Capella (next: Deneb:231680)" sync="--h--m (99.62%) 0.0000slots/s (wwwwwwwwww:7375167)/opt" peers=0 head=741c2ce2:7375168 finalized=230472:723abe7e delay=1ms861us
```
2024-01-12 21:40:34 +01:00
Jacek Sieka e4a1ae67df
ssz: bump (#5717)
height-based merkleizer
2024-01-11 18:34:44 +01:00
tersec 07455e67a9
`proc` to `func` in engine authentication (#5718) 2024-01-08 21:12:58 +00:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 508f3b6368
add missing `std/` prefix to more imports (#5696)
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
Etan Kissling 7db95f047b
track latest `LightClientUpdate` only once fork choice selects it (#5691)
Instead of tracking the latest `LightClientUpdate` across all branches,
track the latest one on the current branch as selected by fork choice.
2024-01-03 23:36:05 +01:00
tersec cafa1f3d3d
rm unused RPC signatures replaced by Plain versions (#5685) 2024-01-03 05:58:34 +00:00
Etan Kissling 583782a061
emit `proposer_slashing`/`attester_slashing` SSE on beacon-API (#5678)
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling a2081521f6
emit `bls_to_execution_change` SSE on beacon-API (#5677)
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
zah 29b29e1945
Use the new strict Json flavors mechanism for RestJson (#5674)
This requires all object types to be explicitly white-listed for
default serialization. The PR makes the minimal changes, although
a number of similar mechanisms in eth2_rest_serialization can now
be removed.
2023-12-19 17:44:27 +02:00
tersec 9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates (#5672) 2023-12-16 03:27:06 +01:00
andri lim 15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
tersec df902fd00f
log execution payload (header) block_hash and parent_hash in block shortLogs (#5660) 2023-12-10 01:25:51 +01:00
Etan Kissling d8728b7b28
re-add comment for `PayloadAttributesV2` usage (#5658)
As followup from #5654, ensure that we still keep the comment around
referring to the correct `forkchoiceUpdated` to use being driven by
the fork schedule.
2023-12-08 03:50:38 +01:00
Etan Kissling 0a5d9ee027
use `PayloadAttributesV3` in `nimbus_light_client` for Deneb (#5654)
* use `PayloadAttributesV3` in `nimbus_light_client` for Deneb

From Deneb onward, `forkchoiceUpdated` requires `PayloadAttributesV3`.
In `nimbus_light_client` we still used `PayloadAttributesV2`.

Also clean up two other locations that were already correctly using
`PayloadAttributesV3`, to reduce code duplication.

* fix letter case
2023-12-07 17:10:22 +00:00
tersec 4776fecc33
consensus spec v1.4.0-beta.5 URL updates (#5655) 2023-12-06 22:16:55 +00:00
tersec 9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 (#5647) 2023-12-05 03:34:45 +01:00
tersec 1791b0ee0a
fix XDeclaredButNotUsed warnings (#5638) 2023-12-04 21:14:35 +00:00
tersec 6a07602aca
use consensus spec test vectors v1.4.0-beta.5 (#5645) 2023-12-04 22:00:27 +01:00
tersec 2fc43c9ba7
track block/blob matching/quarantines using both indices and commitments (#5621) 2023-12-01 18:58:46 +00:00
tersec 6a9d522705
Apply EIP-7044 to block signature batch verification (#5637) 2023-12-01 14:44:45 +00:00
Eugene Kabanov 26bcb7057c
REST: Add implementation of postStateValidators and postStateValidatorBalances. (#5632)
* Add implementation POST versions of /eth/v1/beacon/states/{state_id}/validators and /eth/v1/beacon/states/{state_id}/validator_balances.
Add tests.

* Address review comments.
Fix toList() issue.

* Fix tests.

* Address review comments 2.

* Address review comments 3.
Fix unique check for validator identifiers.

* Address review comments.

* Fix constant value.
2023-11-29 12:05:03 +00:00
Eugene Kabanov e2e4912645
REST API produceBlockV3 implementation (#5474)
Co-authored-by: Etan Kissling <etan@status.im>
Co-authored-by: Jacek Sieka <jacek@status.im>
2023-11-29 00:30:14 +01:00
tersec ab5343d1bc
update some consensus spec URLs to v1.4.0-beta.4 (#5631) 2023-11-27 19:56:34 +01:00
tersec 6dee4d59fe
switch engine authentication from `Option` to `Opt` (#5624)
* switch engine authentication from Option to Opt

* el_conf updates

* LC and el_conf tests
2023-11-27 14:48:29 +00:00
tersec 23983c34ac
rm debugRaiseAssert (#5629) 2023-11-26 19:40:10 +01:00
tersec 115ffa70eb
rm unused code (#5623) 2023-11-25 12:09:18 +00:00
Etan Kissling c33dd2c170
restrict best LC update collection to canonical blocks (#5613)
Simplify best `LightClientUpdate` collection by tracking only canonical
data instead of tracking the best update across all branches within the
sync committee period.

- https://github.com/ethereum/consensus-specs/pull/3553
2023-11-21 23:51:05 +01:00
Etan Kissling b179cb71a0
clean up `XDeclaredButNotUsed` for light client (#5606)
Handle remaining `XDeclaredButNotUsed` warnings for light client.
2023-11-18 00:03:27 +01:00
Etan Kissling b9a693d5fe
send LC finality update on event stream on supermajority (#5602)
When new finality is reached without supermajority sync committee
support, trigger another event push on beacon-API and libp2p once
the finality gains supermajority support.

- https://github.com/ethereum/consensus-specs/pull/3549
2023-11-16 19:57:15 -08:00
Etan Kissling 98e969084d
update Deneb for latest builder-specs flow (#5598)
The `BlobSidecar` construction has been moved to the relay and is no
longer done by the BN / VC in blinded flow. Builder bid contents have
been shrinked from full `BlindedBlobBundle` to `blob_kzg_commitments`.

- https://github.com/ethereum/builder-specs/pull/90
- https://github.com/ethereum/beacon-APIs/pull/369
2023-11-15 16:20:13 -08:00
Etan Kissling 0919ff05c9
reduce code duplication when collecting LC data (#5601)
Replace sections that need to be maintained with every `ConsensusFork`
related to LC data collection with a generic logic that keeps working
when unrelated parts of Ethereum change.
2023-11-15 03:21:03 +01:00
Eugene Kabanov 9889b840ce
VC: new scoring functions. (#5447)
* Initial commit.

* Fix issues and tests.

* Fix test compilation issue.

* Update AllTests.

* Change the most poor score name from <lowest> to <bad>.
Split sync committee message score in range, so lexicographic scores will not intersect with normal one.
Lexicographic scores should be below to normal scores.

* Address review comments.
Fix aggregated attestation scoring to use MAX_VALIDATORS_PER_COMMITTEE.
Fix sync committee contributions to use SYNC_SUBCOMMITTEE_SIZE.
Add getUniqueVotes test vectors.

* Post-rebase fixes.

* Address review comments.

* Return back score calculation based on actual bits length.

* AllTests modification.
2023-11-14 12:13:26 +01:00
tersec 54bdda13b4
rm unused code (#5596) 2023-11-11 11:49:34 +03:00
tersec 1ceac20ea6
disallow htr on Opt[Foo] (#5593) 2023-11-11 05:33:54 +00:00
tersec c96163dbca
update some consensus spec URLs to v1.4.0-beta.4 (#5594) 2023-11-11 06:27:53 +01:00
Etan Kissling 9ec7b700a8
bump `gnosis-chain-configs` to `b3fc927da3e790d989edb71bec99c1db00529307` (#5589)
- Ensure all config vars are present
- Remove preset files
- ensure config is synced
2023-11-10 18:04:41 +01:00
Etan Kissling eb35039704
allow higher `MIN_EPOCHS_FOR_BLOCK_REQUESTS` than safe minimum (#5590)
Gnosis uses `MIN_EPOCHS_FOR_BLOCK_REQUESTS` = 33024, but the computed
safe minimum (that Nimbus was using) is 2304. Relax the compatibility
check to allow `MIN_EPOCHS_FOR_BLOCK_REQUESTS` above the safe minimum
and honor `config.yaml` preferences for `MIN_EPOCHS_FOR_BLOCK_REQUESTS`.
2023-11-10 15:04:55 +00:00
Etan Kissling f388d76e49
bump `gnosis-chain-configs` to `b8ae3091439131949a994d638d730b5a5cb60f7a` (#5581)
- fix(ci): specify enr-cli version
- Update deneb config vars
2023-11-10 10:16:30 +01:00
Etan Kissling 101b723f52
make `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` configurable (#5582)
* make `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` configurable

Gnosis uses custom `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` to account
for the faster slot timing, so that blobs still remain available for
roughly the same amount of real time.

Also extend REST config endpoint with full config form `v1.4.0-beta.4`,
and extend compatibility checks when loading configs to reduce warnings.
2023-11-09 23:41:17 +03:00
Etan Kissling d9031d97dc
rename `_INDEX` generalized index constants to `_GINDEX` (#5583)
Rename constants for consistency with upcoming spec adjustment:

- https://github.com/ethereum/consensus-specs/pull/3537
2023-11-09 12:54:44 +00:00
Eugene Kabanov 6bc038e8d5
VC: Obol middleware support (#5375) 2023-11-08 14:03:51 +02:00
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Eugene Kabanov 49c851109e
VC: Add pruning slashing database. (#5551)
* Add slashing database pruning to VC.
Fix GetBlockHeaderResponse object declaration (spec has been changed).

* Switch to getFinalizedBlockHeader instead.

* Fix proper sign.
Add statements.
Show pruning log statement only when pruning happens.

* Optimize and remove debugging helpers.
2023-11-06 16:40:44 +02:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
Etan Kissling a05278e263
wrap `kzgs`/`proofs`/`blobs` fields as `BlobsBundle` (#5562)
Less type conversion / copying by keeping the `BlobsBundle` together.
2023-11-04 13:49:58 +00:00
Etan Kissling fb18f09e55
align `Blobs` limit for SSZ transport (#5558)
Capacity should be set to theoretical limit to ensure correct hash root.
Actual length may be shorter. Only use is `ExecutionPayloadForSigning`
so it doesn't matter yet in practice, but still worth fixing.
2023-11-04 10:25:06 +01:00
Etan Kissling c95f9feec4
remove unused Deneb code (#5556)
The `process_blob_kzg_commitments` step was removed in `v1.4.0-alpha.1`,
but we haven't deleted the now unused functions. Do that now.
2023-11-03 14:37:44 +00:00
Etan Kissling baf81595a7
add `KZG_COMMITMENT_INCLUSION_PROOF_DEPTH` preset constant (#5555)
`v1.4.0-beta.4` added `KZG_COMMITMENT_INCLUSION_PROOF_DEPTH` to preset:

- Spec PR: https://github.com/ethereum/consensus-specs/pull/3531
- Gnosis PR: https://github.com/gnosischain/configs/pull/17
2023-11-03 13:54:21 +00:00
tersec ab89e96265
fill in transactions root and KZG commitments from deneb.BlindedBeaconBlock in state transition (#5553) 2023-11-02 04:56:04 +01:00
tersec 657de2721a
fill blindedblockbody KZG commitments from deneb blinded header (#5550) 2023-11-01 14:00:21 +01:00
Eugene Kabanov e1f47bb27a
Address #5539. (#5548) 2023-11-01 08:32:41 +01:00
Eugene Kabanov 1a5bcb479e
Fix `broadcast_validation` handling in `publishBlockV2` (fixes #5531) (#5547)
* Address issue #5531.

* Add more tests.

* Add to resttest ability to check values.
Fix tests.
2023-11-01 08:31:18 +01:00
tersec 556d5e7114
rm unused code (#5538) 2023-11-01 05:53:09 +01:00
tersec 62d59daaa7
consensus-spec URL updates to v1.4.0-beta.3 (#5541) 2023-10-30 06:44:43 +00:00
tersec 34eade3a99
sign blinded blob sidecars during builder API proposals (#5537) 2023-10-28 05:10:23 +00:00
tersec 41dfc9ed91
add `KZGProof` and `Blob` REST JSON serializations (#5526)
* add KZGProof and Blob REST JSON serializations

* error summary updates

* copyright notice

* consistent integer types for Nim 2.0
2023-10-25 23:50:59 +02:00
tersec 5aa5095e17
Fix REST JSON KzgCommitment encoding/decoding to use hex strings per spec (#5522) 2023-10-24 15:22:59 +00:00
tersec 513c9d9637
fix Deneb RestPublishedSignedBlockContents REST JSON decoding (#5520) 2023-10-24 05:50:32 +00:00
tersec fd21882e24
improve test coverage of post-Bellatrix initialize_beacon_state_from_eth1 (#5517) 2023-10-24 01:58:52 +00:00
tersec b2e81de6db
remove unnecessary Builder API status endpoint usage (#5518) 2023-10-23 23:01:29 +00:00
tersec a986a1bcd8
using deneb_mev for Deneb is less incorrect than using capella_mev (#5516) 2023-10-20 01:39:47 +00:00
tersec 4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
tersec 9fd52ebe9f
use consensus spec v1.4.0-beta.3 test vectors (#5512) 2023-10-19 05:14:02 +00:00
tersec 6eb7d481fa
fix Deneb builder API block body type; introduce and use ExecutionPayloadHeaderAndBlindedBlobsBundle (#5509) 2023-10-18 02:54:02 +00:00
andri lim 0f9b52933e
Bump nim-eth: Change block timestamp from std.Time to distinct uint64 (#5510)
* Bump nim-eth: Change block timestamp from std.Time to distinct uint64

Also change tx.maxFeePerBlobGas from GasInt to UInt256
following Cancun latest spec

* Fix EthTime.now from func to proc due to sideeffects
2023-10-18 07:37:57 +07:00
zah 35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
tersec eeac00d08e
add RPC signature; add type constraints on proposeBlockMEV (#5498) 2023-10-12 11:49:48 +00:00
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec edc5c03230
consistently use SignedBlindedBeaconBlockContents; remove more Bellatrix Builder API remnants (#5493) 2023-10-11 11:48:35 +00:00
tersec 48197e4d55
add/update Deneb Builder API types and RPC signatures (#5492) 2023-10-11 04:25:59 +00:00
tersec 447786518f
ShufflingRef approach to next-epoch validator duty calculation/prediction (#5414)
* ShufflingRef approach to next-epoch validator duty calculation/prediction

* refactor action_tracker.updateActions to take ShufflingRef + beacon_proposers; refactor maybeUpdateActionTrackerNextEpoch to be separate and reused function; add actual fallback logic

* document one possible set of conditions

* check epoch participation flags and inactivity scores to ensure no penalties and MAX_EFFECTIVE_BALANCE to ensure rewards don't matter

* correctly (un)shuffle each proposer index

* remove debugging assertion
2023-10-10 00:02:07 +00:00
Etan Kissling dd1f362536
align `Beacon(Block(Body)?|State)Type` with other fork sugar (#5483)
The templates for `BeaconBlock`, `BeaconBlockBody` and `BeaconState`
are the only ones using a `macro` mechanism for code generation.
This prevents using the dot-syntax style `consensusFork.BeaconFoo`
in some situations, and also tends to trigger naming conflicts,
requiring the `Type` suffix. Furthermore, the `macro` only works
for types that are re-defined in every single `ConsensusFork`.

Replacing with the simpler but more verbose approach used for other
types for consistency and to avoid the downsides of the `macro`.

Furthermore, simplify `test_fixture_sanity_blocks` to use `forks` sugar.
2023-10-05 14:01:40 +02:00
Etan Kissling 297c768816
reduce redundant zero initialization for LC data objects (#5479)
Directly initialize `ForkedLightClientObj` instead of separately first
 setting the `kind` (initializing everything to zero) and then assigning
the forky data after that.
2023-10-04 18:11:45 +02:00
Etan Kissling 8aaea1a14e
simplify fork choice test (#5467)
Reduce overly verbose fork branching logic in fork choice test with
generic implementation based on `consensusFork`.
2023-09-29 13:26:34 +00:00
Jacek Sieka 252473d2d1
ssz: bump (#5463)
more `zeroMem` avoidance, value semantics for merkleizer
2023-09-28 11:59:33 +00:00
Etan Kissling 7c45b8f98d
convert remaining `toFork` -> `kind` for consistency (#5462)
We currently have multiple ways to obtain `ConsensusFork` or
`LcDataFork` from a forky type. Rename `toFork` to `kind`
for a consistent API naming.
2023-09-27 15:10:28 +00:00
Eugene Kabanov 4fb95d000d
REST server fixes and improvements. (#5422)
* Move from Option[T] to Opt[T] usage.

* Add `finalized` flag.

* Fix compilation issue.

* Http415 error code for some REST API calls.
Introduce more comprehensive error reporting for block calls.
Deprecate decodeEthConsensusVersion() function.

* Bump http-utils.

* Fix copyright year.

* Fix serialization issue.

* Address review comments.

* Post rebase fixes.
2023-09-27 16:45:33 +02:00
tersec 710f26745d
small attestation pool clean up (#5458) 2023-09-24 08:50:48 +00:00
Eugene Kabanov e3fe762ec9
Fix unknown fields should be ignored when parsing liveness responses. (#5418) 2023-09-23 08:44:54 +02:00
Etan Kissling 47639ef89c
replace `Eth2NetworkMetadata` case object with `Result` (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
Etan Kissling 5150505e66
bump `nim-ssz-serialization` to `a5767c1b4992dca3e5bd72557aadd4df0f183bf8` (#5457)
- nimble/ci updates
- use common super-type in `raisesssz` pragma
2023-09-22 23:44:57 +02:00
tersec 2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Etan Kissling e7bc41e005
`blck` --> `forkyBlck` when using `withBlck` / `withStateAndBlck` (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
Etan Kissling 273f1d34cd
implement EIP-7514 for Deneb: Add Max Epoch Churn Limit (#5442)
* implement EIP-7514 for Deneb: Add Max Epoch Churn Limit

Cap activations per epoch according to EIP-7514:

- https://eips.ethereum.org/EIPS/eip-7514
- https://github.com/ethereum/consensus-specs/pull/3499

* apply proposer boost to first block in case of equivocation

Implement spec changes to fork choice; this only affects equivocation
when multiple blocks are signed for the same slot. Regular operation
is not changed.

- https://github.com/ethereum/consensus-specs/pull/3352

* bump test vectors to v1.4.0-beta.2-hotfix

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-09-20 13:39:56 +02:00
Etan Kissling 1640c45a95
reject invalid proofs when validating blobs (#5445)
Currently, passing `0xc00000...` proof seems to pass `verifyProofs`.
Unsure why such a check is not necessary in spec, and also unsure
whether it is correct to reject proof at infinity, or if it could
occur, e.g., for a blob containing all 0 bytes. Weird overall...

* proper fix
2023-09-19 22:13:58 +00:00
tersec ee75d45a8b
fix `ATTESTATION_PROPAGATION_SLOT_RANGE` unknown constant warning (#5433)
* fix ATTESTATION_PROPAGATION_SLOT_RANGE unknown constant warning

* verify underflow not an issue
2023-09-19 06:54:58 +00:00
tersec 5a29ad7e4f
update some consensus-spec URLs to v1.4.0-beta.1 (#5429) 2023-09-14 18:23:59 +00:00
tersec 7a001d4896
remove 4 more missing constants in preset handling warning (#5428) 2023-09-14 12:37:35 +00:00
andri lim 5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
Etan Kissling 8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
tersec 2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Eugene Kabanov 3c3c4e8edf
Fix /eth/v1/validator/liveness/{epoch} call names and links in code (#5392)
* Fix getLiveness() call links and names.

* Add getLiveness() decoding test.

* Remove BN state updates from getLiveness() call.

* Fix AllTests.

* Reduce number of tests.
2023-09-11 12:07:34 +02:00
tersec 10ec7be686
optimize epoch registry processing (#5412) 2023-09-11 11:21:50 +02:00
tersec cc13e0b7e0
restore full test coverage for process_rewards_and_penalties (#5407)
* restore full test coverage for process_rewards_and_penalties

* adjust ncli_db to use new iterator
2023-09-08 16:42:18 +00:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
tersec f53422c181
optimize epoch transition via get_flag_index_deltas() and get_inactivity_penalty_deltas() (#5404)
* optimize epoch transition via get_flag_index_deltas() and get_inactivity_penalty_deltas()

* directly mutate Altair info in {get,update}_flag_and_inactivity_deltas
2023-09-07 11:14:52 +00:00
Etan Kissling 89d133d0e5
further reduce stack size of LC helpers for Nim 2.0 (#5401)
The `is_next_sync_committee_known` helper allocates a fresh
`SyncCommittee` in the caller and `nimZeroMem`s it on each use.
Use `static` to compare against a compile-time zeroed copy instead.
This also should help reduce stack size far enough to link with Nim 2.0.
2023-09-06 16:05:12 +02:00
Eugene Kabanov 4ce8e77b56
Address issue #5362. (#5370)
* Address issue #5362.

* Remove RestSpec object declaration and getSpec() API call declaration.

* Address review comments.
2023-09-05 23:39:01 +03:00
Etan Kissling dbb1a63ca9
collapse `partialBeaconBlock` templates into one (#5386)
Have a single `Forky` template for `partialBeaconBlock` production
rather than have several copies that are mostly identical.
2023-09-05 15:59:17 +02:00
Etan Kissling be9ecfa1c9
add helpers for processing withdrawals to `libnimbus_lc.a` (#5374)
Similar to the existing helpers for processing transactions / receipts,
extend `libnimbus_lc.a` with support for processing withdrawals as well.
2023-09-04 20:44:03 +02:00
tersec a8c56b1660
update some consensus-spec URLs to v1.4.0-beta.1 (#5379) 2023-09-01 09:31:52 +00:00
Eugene Kabanov 757328372a
Dynamic validators set. (#5366)
* Initial commit.

* Fix argument to be optional.

* Adopt options.md.
2023-08-31 15:16:15 +03:00
tersec 6c42cfa79e
use correct decimal or hex encoding/decoding for REST GetSpec (#5371) 2023-08-30 12:43:25 +03:00
Etan Kissling 09020ebd2f
add helpers for processing receipts to `libnimbus_lc.a` (#5360)
Similar to the existing helpers for processing transactions,
extend `libnimbus_lc.a` with support for processing receipts as well.
2023-08-28 15:56:40 +00:00
tersec db6f4e8090
update some consensus-spec URLs to v1.4.0-beta.1 (#5357) 2023-08-25 15:58:44 +00:00
Etan Kissling 8ff0f623b4
change unused template to unused func (#5355)
The intentionally unused `toSszType` template may trigger warnings
as it is unused. Replacing it with an `{.error.}` `func` prevents that.
2023-08-25 11:09:13 +00:00
Etan Kissling d8f44ed905
fix LC header validation in Deneb (#5350)
`blob_gas_used` must be 0 before Deneb.
2023-08-25 10:36:10 +00:00
Etan Kissling 991c31f42b
add helpers for processing transactions to `libnimbus_lc.a` (#5269)
It is useful to verify transactions data against `transactionsRoot`.
Add corresponding functionality to the light client library.
2023-08-25 11:29:39 +02:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Etan Kissling 9637d3f8d0
remove unused `type` in `makeLimitedUInt` (#5354)
The `type L` definition inside `makeLimitedUInt` is unused. Remove it.
2023-08-25 11:28:52 +02:00