Commit Graph

4317 Commits

Author SHA1 Message Date
Eugene Kabanov fc2b232b9e
Fix indefinite loop when when validators are present, but VC does not know indices yet. (#6615)
Fix block pruning when no validators seen.
2024-10-09 03:00:06 +02:00
Jacek Sieka 7990cc2d1c
Ordered trie for trie root computations (#6610)
Gives a 100x speed boost for this operation which happens during
optimistic sync when computing the block hash.

Co-authored-by: Etan Kissling <etan@nimbus.team>
2024-10-08 22:10:50 +02:00
Eugene Kabanov a25afaed5a
Increase number of web3 signer requests before signaling failure to 4. (#6609)
Decrease number of pre-computed slots from 32 to 2.
Add validator field to some log statements.
2024-10-08 05:37:00 +02:00
Miran 3f37caa0da
fix deprecated warnings (#6606) 2024-10-08 01:54:02 +00:00
Eugene Kabanov 5d11c5229b
Add Web3signer electra support (#6607)
* Add Electra support to nimbus_signing_node.
Add tests.
Annotate nimbus_signing_node with asyncraises.

* Update AllTests.
2024-10-07 23:31:16 +00:00
tersec b370112fe0
some manual updates to EF consensus-specs v1.5.0-alpha.7 URLs (#6604) 2024-10-07 12:30:09 +02:00
tersec 50eed81dcf
electra BN attestation aggregating (#6596) 2024-10-07 07:53:38 +00:00
tersec 17d66118e8
automated consensus spec URL updating to v1.5.0-alpha.7 (#6601) 2024-10-06 14:03:56 +00:00
tersec 9c9f3707a1
use EF consensus spec v1.5.0-alpha.7 test vectors (#6600) 2024-10-06 13:15:56 +02:00
Pedro Miranda ff0cc8e62e
Electra - Filter aggregated attestations by root/committee (#6594)
* Filter aggregated attestations by root/committee

* fixed validator api documentation link

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-10-04 05:15:48 +00:00
tersec eb30b741d9
bump nimbus-build-system to use Nim v2.0.10 (#6592)
* bump nimbus-build-system to use Nim v2.0.10

* fix `CatchableError` leak on Windows

* define ERROR_INVALID_ACCESS

* trigger finding correct hashes for nix build

* check checksums checksum

* update nix nimble checksum

* csources didn't change
2024-10-03 17:13:44 +02:00
Jacek Sieka 8f0d910422
web: bump to new nim-eth types (#6590) 2024-10-02 15:26:11 +02:00
tersec 7abf899a66
Merge remote-tracking branch 'origin/stable' into unstable 2024-09-30 12:37:04 +00:00
tersec 3bf96a5923
version v24.9.0 2024-09-30 03:14:58 +00:00
Jacek Sieka b20f1ba8b3
Update nim-eth types (#6583)
Minimal changes needed for compatiblity with
https://github.com/status-im/nim-eth/pull/733 which aligns core types
with execution spec.
2024-09-29 13:32:55 +02:00
tersec 2e64fdfa67
centralize ConsensusFork <=> string handling in spec/forks (#6587) 2024-09-28 09:23:45 +02:00
Pedro Miranda 4beb890385
new pool attester slashings endpoint version (V2) for electra (#6585)
* new pool attester slashings endpoint version (V2) for electra

* formatting

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-28 00:00:26 +00:00
tersec a441695c51
plumb through Electra attestation callbacks (#6586) 2024-09-27 13:30:26 +02:00
tersec fa8b7e380f
add Electra beacon chain database state tests (#6584) 2024-09-27 07:53:58 +02:00
Pedro Miranda 744cc00984
replace default word usage (#6580)
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-26 19:49:03 +00:00
tersec 28b20936e9
add Electra attester slashing gossip support (#6582) 2024-09-26 08:11:58 +02:00
Pedro Miranda daf7f899c2
Attestation API updates for Electra (#6557)
* new V2 endpoint for beacon getBlockAttestations

* nnew GET endpoint version (V2) for getPoolAttestations

* new POST endpoint version (V2) for submitPoolAttestations

* remove premature ncli tests

* review improvements

* review comments and increased test coverage

* small improvements

* documentation typos

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-25 12:33:58 +00:00
tersec f2d6166099
remove block fork guessing/inference from REST JSON decoding (#6552)
* remove block fork guessing/inference from REST JSON decoding

* use template to avoid repetitive per-fork code

* consolidate RestPublishedSignedBeaconBlock and RestPublishedSignedBlockContents parsing fork handling
2024-09-25 02:58:53 +00:00
Agnish Ghosh 31b5c3e30e
add: inclusion proof and dc sidecar verification (#6577)
* add: inclusion proof and dc sidecar verification

* review 1
2024-09-24 14:46:05 +02:00
tersec d4e441e694
add Electra attester slashing pool (#6579) 2024-09-24 11:01:40 +02:00
Eugene Kabanov 841904bd53
Do not cache resolved hostnames in REST client. (#6570) 2024-09-24 04:23:38 +00:00
Eugene Kabanov 77e835153c
Bump nimcrypto to fix nim22 issues. (#6578)
* Bump nimcrypto to fix nim22 issues.

* Fix compilation issue.
2024-09-24 02:55:16 +00:00
Agnish Ghosh 85d7109065
add: `get_data_column_sidecars` (#6566)
* add: get_data_column_sidecars

* remove redundant type conversions

* remove redundant for loop

* review 2

* review 1
2024-09-22 23:35:57 +02:00
tersec ff9d32d30d
some v1.5.0-alpha.6 consensus spec URL updates (#6572) 2024-09-21 06:53:34 +02:00
tersec 256f6b8b89
automated consensus spec URL updating to v1.5.0-alpha.6 (#6571) 2024-09-21 03:22:41 +02:00
tersec 193a982d24
use EF consensus spec v1.5.0-alpha.6 test vectors (#6568)
* use EF consensus spec v1.5.0-alpha.6 test vectors

* LC changes are good as is, no need for the debug comment

* Fix indentation

* Use `execution_requests` for requests root

* fill in spec reference URLs for ExecutionPayload(Header)

* copy `execution_requests` when blidning blocks for Electra and add test

* fix `test_engine_api_conversions`

---------

Co-authored-by: Etan Kissling <etan@nimbus.team>
2024-09-20 21:32:16 +02:00
Etan Kissling 9df3d075de
fix blinded block conversion for Deneb and Electra (#6569)
`blob_gas_used` and `excess_blob_gas` were not copied on blinding
signed beacon block.
2024-09-20 16:02:24 +02:00
Eugene Kabanov 20bb20f75d
Add `ResolveAlways` flag for web3signer requests. (#6567) 2024-09-20 13:10:25 +02:00
tersec e94417c500
speed up electra attestation tests by 5x (#6563) 2024-09-19 07:46:38 +02:00
Eugene Kabanov e3fcd8b031
REST: getBlockRewards() and getSyncCommitteeRewards() implementation (#6556)
* Initial commit.

* Use temporary state instead of clearance.

* Attempt to fix `finalized`.

* Fix `genesis` response.

* Pre-calculate genesis block rewards response.

* Add implementation for sync committee rewards.

* Add total active balance calculation.

* Add genesis special case.

* Fix negative reward values.

* Address review comments.

* Fix isGenesis implementation and add REST test rules for both calls.
2024-09-18 23:34:32 +00:00
Etan Kissling de1552625f
bump nim-web3 to `0cc03e67dda6c4d9be0f9d9abde7508e46156fd9` (#6560)
- Add Prague (EIP-7685, 6110, 7002, 7251, and 7702) fields
2024-09-18 18:47:35 +02:00
Agnish Ghosh 3f0d58492e
add: ssz consensus objs for peerdas (#6555) 2024-09-17 22:57:48 +00:00
Agnish Ghosh 1feeff4f92
add: recover matrix (#6553) 2024-09-17 10:21:55 +00:00
Agnish Ghosh b3c8c71776
add: compute matrix (#6550)
* add: compute matrix

* rm unsused imports

* rm unused imports 2

* rm bin

* rm unused line

* improve doc

* rid space on proc sig

* review 1

* added seeded PRNG

* randomize blob count as well, with smaller seed

* review 2
2024-09-16 11:19:34 +00:00
tersec 72b5c2bd6e
add some electra support to REST JSON decoding; rm unused code (#6549) 2024-09-15 13:33:13 +00:00
tersec bd47f62f07
tweak indentation (#6548) 2024-09-13 23:31:45 +00:00
Pedro Miranda 7b6caebf6e
Add publishAggregateAndProofsV2 (#6546)
* added new endpoint version(v2) for validator aggregate and proof

* review improvements

* introduced nim template for proof executiion

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-13 21:11:09 +00:00
tersec ed2422112c
resolve several outstanding items for Electra support; add debug-foo to light client yamux (#6547) 2024-09-13 10:07:06 +02:00
Agnish Ghosh f53b621818
add: `get_extended_sample_count` with test (#6544)
* add: get_extended_sample_count with test

* drop return

* reviews

* review fix

* fixed

* fix doc

* hooked to all_tests

* rm bin

* add updated test file

* early return, maybe need results?

* refactor function intricacies

* drop columnsCount
2024-09-12 17:55:16 +02:00
tersec 16c21e1c1e
rm unused code/type conversoins; re-enable Electra block gossip verification (#6545) 2024-09-12 11:11:08 +02:00
Agnish Ghosh a2a7cab76d
Rid result from `get_custody_column_subnets` (#6543)
* ditch results in get custody column subnets

* fix other part
2024-09-11 12:49:01 +02:00
tersec 1503c7dffe
rm some unused code/imports (#6541) 2024-09-11 05:45:37 +02:00
Pedro Miranda 1ac9b851b9
Add GetAggregateAttestation V2 endpoint version (#6511)
* attestation pool support and tests

* REST endpoints changes

* initial ncli and validator client support

* updated tests file

* fixed typos

* review improvements

* remove V1 endpoint

* revert v1 removal

* V2 endpoint version available to pre electra

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-06 12:15:30 +00:00
Agnish Ghosh 444d1dd093
add: `get_custody_columns` for das_core (#6532)
* add: get_custody_columnns in das core specs with tests

* apply review changes

* review changes 2

* review 3
2024-09-04 16:05:18 +02:00
Agnish Ghosh 5a44e39e31
bump nim-kzg4844, enable ckzg2.0.0, fix whatever needed (#6502) 2024-09-02 11:47:00 +00:00
tersec 1e70b0d2a8
Merge remote-tracking branch 'origin/stable' into unstable 2024-08-30 02:21:05 +00:00
tersec bf4abf8b9e
version v24.8.0 2024-08-29 14:29:58 +00:00
Etan Kissling 73be2af0d9
more concise RLP decoding syntax in EL helpers (#6514)
The comment in `readExecutionTransaction` is not actually relevant when
the `rlp.decode` template is called from a non-generic function.
The function still cannot be simply inlined; `is_valid_versioned_hashes`
is itself generic and would require an `export eth_types_rlp` to work
properly (generic sandwich issue). The non-generic function wrapper
seems simpler and avoids injecting RLP into importing modules.
2024-08-28 14:21:28 +02:00
Pedro Miranda f9e44b2a3b
Improved electra attestations packing (#6498)
* test disjoint comittee bits

* test for aggregated attestations with disjoint bits

* improved attestation pool on-chain collecting and packing

* addressed format issues

* improved sorting method

* updated AllTests-mainnet file

* updated AllTests-mainnet file

* review corrections

* review corrections

* removed old phase0 artifacts

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-08-23 13:26:35 +02:00
tersec 8c2e8f2b0f
some v1.5.0-alpha.5 consensus spec URL updates (#6509) 2024-08-23 12:43:54 +02:00
Jacek Sieka 42c4ea109c
Revert "stop testing broken upstream version-2-0" (#6506)
* Revert "stop testing broken upstream version-2-0 (#6499)"

This reverts commit 2be7eba25a.

* fix field warning template
2024-08-23 12:18:06 +02:00
tersec e4f18cba09
stop tracking phase0 spec artifact of current/prev target epoch for attestation block packing (#6508) 2024-08-22 20:28:21 +02:00
tersec 44cc72c104
refactor engine API conversions out of EL manager (#6507) 2024-08-22 16:12:03 +02:00
Etan Kissling bd09e4d864
inhibit LC sync while DAG is synced (#6505)
Normally, running LC and DAG sync at same time is fine, but on tiny
devnet where some peer may not support the LC data, we can end up in
situation where peer gets disconnected when DAG is in sync, because
DAG sync never uses any req/resp on local devnet (perfect nw conditions)
so the LC sync over minutes removes the peer as sync is stuck.

We don't need to actively sync LC from network if DAG is already synced,
preventing this specific low peer devnet issue (there are others still).
LC is still locally updated when DAG finalized checkpoint advances.
2024-08-22 06:13:47 +00:00
Etan Kissling a597fe95fa
check blob versioned hashes when no EL is connected (#6501)
* check blob versioned hashes when no EL is connected

When no EL is conencted, we have to at the very least ensure that the
data in the beacon block is consistent with the execution payload.
We already do this for the block hash, but also have to do it for the
`blob_kzg_commitments`. To validate that they are linked with the
execution payload, we have to RLP decode all EIP-4844 blob transactions
and compare their blob versioned hashes with the hashed commitments.

* simplify loop in case where `blob_versioned_hashes` doesn't exist

* skip blob transaction parsing pre Deneb
2024-08-22 06:04:03 +00:00
tersec 21aeeaf561
automated consensus spec URL updating to v1.5.0-alpha.5 (#6504) 2024-08-21 14:25:19 +02:00
tersec 485ed833e0
use EF consensus-specs v1.5.0-alpha.5 test vectors (#6503) 2024-08-21 12:18:33 +02:00
Etan Kissling b511f3eeb7
Use Pippenger multiplication for combining multiple sigs of same msg (#6484)
Newer `blst` releases expose multiscalar Pippenger multiplication that
allows accelerated verification of signatures pertaining to same msg.

- https://gist.github.com/wemeetagain/d52fc4b077f80db6e423935244c2afb2
2024-08-16 23:42:46 +02:00
Etan Kissling 6f7c4fffe7
Synchronously check all `transactions` to have non-zero length (#6491)
Reject blocks with zero length transactions early when no EL connected.

- https://github.com/ethereum/consensus-specs/pull/3885
2024-08-16 22:52:35 +02:00
tersec 2895df128c
increase TNS state downloading timeout to 120 seconds (#6487) 2024-08-12 03:49:10 +02:00
tersec f258cba816
some v1.5.0-alpha.4 consensus spec URL updates (#6485) 2024-08-10 07:09:37 +02:00
tersec 904318cf83
automated consensus spec URL updating to v1.5.0-alpha.4 (#6483) 2024-08-09 18:24:49 +02:00
tersec 2e40a401b0
use EF consensus-specs v1.5.0-alpha.4 test vectors (#6482) 2024-08-09 15:02:04 +02:00
tersec d2c8561fcd
add some Electra config constants to beacon API config endpoint (#6479) 2024-08-07 18:15:29 +02:00
Etan Kissling 3375875e05
remove option to select Capella fork choice algo (#6478)
* remove option to select Capella fork choice algo

With Deneb having run stable for quite a while now, it's time to remove
the option to select the prior fork choice algo from Capella.

* also remove usage from test
2024-08-07 14:48:05 +00:00
tersec 535819263d
clear old electra attestations from attestation pool 2024-08-07 11:14:57 +00:00
Etan Kissling 51ca617d9f
fix LC header upgrade to Electra (#6473)
Followup on incorrect upgrade procedure in #6375 where `blob_gas_used`
was accidentally copied into `excess_blob_gas` when running Electra
`LightClientStore` with earlier `LightClient(Bootstrap|Update)`.
2024-08-06 15:48:15 +00:00
tersec c0fc0f41dd
calculate next slot's withdrawals properly even across epoch boundary (#6470) 2024-08-06 13:33:16 +02:00
tersec 8333365848
replace validator Bloom filter with partial bucket sort (#6469) 2024-08-02 07:30:22 +02:00
tersec 6a576978ba
Merge remote-tracking branch 'origin/stable' into unstable 2024-07-29 03:47:56 +00:00
tersec 0a4d3ac9c3
version v24.7.0 2024-07-29 00:45:51 +00:00
Etan Kissling ea16edd886
add test for shuffled attestation signatures (#6459)
Followup of #3212 to test proper signature verification. Also document
possible further optimization based on blst `v0.3.13`.
2024-07-26 23:31:05 +00:00
Eugene Kabanov b345c4fa72
Fix: Avoid downloading genesis multiple times, for holesky testnet. (#6452) 2024-07-26 14:54:03 +00:00
Etan Kissling 09f3882672
remove unsued `lcDataForkAtEpoch` helper (#6449)
`lcDataForkAtEpoch(.)` is not used anywhere and can be removed.
`lcDataForkAtConsensusFork(consensusForkAtEpoch(.))` saves same purpose.
2024-07-25 16:27:19 +02:00
tersec 949758ed74
fix some UnusedImport warnings (#6448) 2024-07-25 08:49:26 +00:00
tersec 15cca5f93e
hypergeometric distribution CDF for PeerDAS (#6447) 2024-07-25 01:59:24 +00:00
Eugene Kabanov fd4398d4c5
Fix EL requests should do some sleep before repeating request again. (#6441) 2024-07-23 22:49:52 +00:00
Etan Kissling 20ede0ab35
use correct fork digest when broadcasting blob sidecars and sync msgs (#6440)
The fork digest determines the underlying data type on libp2p gossip,
so it's important to use the matching fork digest instead of picking
whatever wall epoch happens to be.
2024-07-23 20:30:44 +00:00
Etan Kissling 9f21182646
construct info object for `BlobSidecar` earlier (#6445)
To avoid "forked" types creeping into `BlobSidecar`, move the reduction
to `BlobSidecarInfoObject` to the sole caller. The info object is fork
agnostic, so does not need "forked" if `BlobSidecar` ever updates.
2024-07-23 20:22:29 +00:00
Etan Kissling a769550db5
rename LC gindex constants to match spec (#6444)
Use `_ELECTRA` suffix for gindex constants to match consensus-specs.
2024-07-23 20:20:15 +00:00
Etan Kissling ae0a1488b5
Fix blob syncing for Electra (#6438)
* Fix blob syncing for Electra

`BlobSidecar` requests on libp2p have a context prefix based on:

> The `<context-bytes>` field is calculated as context =
>     `compute_fork_digest(fork_version, genesis_validators_root)`

We currently only process blobs if that indicates Deneb, meaning that
on Electra we incorrectly report `InvalidContextBytes` and refuse to
process the blob response data.

Fix this, and also ensure that the code no longer needs maintenance
with every fork unrelated to blobs.

* fix
2024-07-23 03:10:41 +00:00
tersec 6fa852400d
pause deployment of long-range sync option default (#6436) 2024-07-21 07:10:26 +02:00
tersec ebd0217a10
fix is_eligible_for_activation_queue for electra (#6427) 2024-07-15 21:08:20 +02:00
Eugene Kabanov 7853bd2878
BN: User agent recognition and error codes decoding. (#6414)
* Initial commit.

* Force update remote agent right before disconnect.

* Add handling errors from unknown remote agents.

* Address review comments.
2024-07-15 16:53:41 +02:00
Eugene Kabanov f751fde417
Use Nimbus specific agent string instead of nim-presto defaults. (#6424)
* Use Nimbus specific agent string instead of nim-presto defaults.
Use response(code) overload.

* Fix REST API test.
2024-07-15 02:27:23 +00:00
Kim De Mey ca15c4e433
Replace deprecated ENR functions (#6419) 2024-07-12 17:18:24 +02:00
Eugene Kabanov 031033a6f9
Fix REST /eth/v1/node/identity should return proper MultiAddresses (version 2). (#6422)
* Initial commit.

* Bump nim-libp2p with dualstack fixes.

* Pass announcedAddresses to the `p2p_addresses` list.
2024-07-11 17:39:38 +02:00
tersec befcf3f56a
bump sepolia for new bootnodes (#6421) 2024-07-11 07:13:37 +02:00
Etan Kissling dc007e3977
nullability annotations for `libnimbus_lc.h` (#6417)
Annotate functions that may return `NULL` with `_Nullable` to properly
bridge into Swift.
2024-07-09 12:38:16 +02:00
tersec 3db571d182
allow individual calculation of validator balances across epoch boundaries (#6416) 2024-07-07 00:32:50 +02:00
Jhett Black 3f051e9ab0
Add beacon node count metrics. (#6371) 2024-07-06 21:07:56 +00:00
andri lim 0b276315d2
Bump nim-eth: Convert GasInt to uint64 (#6415)
* Bump nim-eth: Convert GasInt to uint64

* Fixes

* Fix libnimbus_lc
2024-07-06 20:25:31 +00:00
Etan Kissling 7f59e80aaa
validate EL block hash in Electra when no EL is connected (#6407)
When no EL is connected, it is still required to validate the block hash
of `ExecutionPayload` to prevent attacks that trick us into attesting to
a circular chain with invalid in-between block hashes. This is already
done through Deneb but was still missing in Electra to be rectified now.
2024-07-05 10:18:50 +02:00
Eugene Kabanov c59bb71916
Add implementation of publishBlindedBlockV2() REST API endpoint. (#6413) 2024-07-04 19:48:12 +00:00
Etan Kissling abf818a9f3
bump nim-eth to `d8fda55c79dd48ba564f3cb540b968f4a1c1aae6` (#6405)
* bump nim-eth to `d8fda55c79dd48ba564f3cb540b968f4a1c1aae6`

- Overhaul of ENR implementation - part I
- Rework of ENR decoding code
- Update discv5 to use non deprecated ENR calls and simplify code
- simplify .nimble file
- avoid warnings when processing `GasInt` for RLP
- define Electra types and RLP encoding

* explicitly indicate consensus types over nim-eth types in EL manager
2024-07-04 18:40:11 +00:00