Commit Graph

534 Commits

Author SHA1 Message Date
Etan Kissling cf9dc4a79b
Merge branch 'dev/etan/df-forkedblobs' into feat_eip-7688 2024-12-04 14:18:18 +01:00
Etan Kissling 13df492133
Merge branch 'stable' into dev/etan/df-forkedblobs 2024-12-03 14:38:06 +01:00
tersec 5579c5234f
automated consensus spec URL updating to v1.5.0-alpha.9 (#6725) 2024-11-24 13:16:09 +01:00
Agnish Ghosh a157760643
Init Fulu fork (#6677)
* fulu

* fix ci

* fix ncli

* attempt to fix ci

* fix wss_sim build

* fix vc api

* updated tests

* resolve merge conflicts

* fix tests

* fix check

* attempt to fix minimal test in CI

* addressed reviews

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-11-13 10:29:14 +07:00
Eugene Kabanov 18409a69e1
Light forward sync mechanism (#6515)
* Initial commit.

* Add hybrid syncing.

* Compilation fixes.

* Cast custom event for our purposes.

* Instantiate AsyncEventQueue properly.

* Fix mistype.

* Further research on optimistic updates.

* Fixing circular deps.

* Add backfilling.

* Add block download feature.

* Add block store.

* Update backfill information before storing block.

* Use custom block verifier for backfilling sync.

* Skip signature verification in backfilling.

* Add one more generic reload to storeBackfillBlock().

* Add block verification debugging statements.

* Add more debugging

* Do not use database for backfilling, part 1.

* Fix for stash.

* Stash fixes part 2.

* Prepare for testing.

* Fix assertion.

* Fix post-restart syncing process.

* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.

* Add handling of Duplicates.

* Fix store duration and block backfilled log statements.

* Add proper syncing state log statement.

* Add snappy compression to beaconchain_file.
Format syncing speed properly.

* Add blobs verification.

* Add `slot` number to file structure for easy navigation over stream of compressed objects.

* Change database filename.

* Fix structure size.

* Add more consistency properties.

* Fix checkRepair() issues.

* Preparation to state rebuild process.

* Add plain & compressed size.

* Debugging snappy encode process.

* Add one more debugging line.

* Dump blocks.

* One more filedump.

* Fix chunk corruption code.

* Fix detection issue.

* Some fixes in state rebuilding process.

* Add more clearance steps.

* Move updateHead() back to block_processor.

* Fix compilation issues.

* Make code more async friendly.

* Fix async issues.
Add more information when proposer verification failed.

* Fix 8192 slots issue.

* Fix Future double completion issue.

* Pass updateFlags to some of the core procedures.

* Fix tests.

* Improve initial sync handling mechanism.

* Fix checkStateTransition() performance improvements.

* Add some performance tuning and meters.

* Light client performance tuning.

* Remove debugging statement.

* Use single file descriptor for blockchain file.

* Attempt to fix LC.

* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.

* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.

* Address review comments.

* Address review comments part 2.

* Address review comments part 1.

* Rebase and fix the issues.

* Address review comments part 3.

* Add tests and fix some issues in auto-repair mechanism.

* Add tests to all_tests.

* Rename binary test file to pass restrictions.

* Add `bin` extension to excluded list.
Recover binary test data.

* Rename fixture file to .bin again.

* Update AllTests.

* Address review comments part 4.

* Address review comments part 5 and fix tests.

* Address review comments part 6.

* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.

* Fix forgotten place.

* Post rebase fixes.

* Add unique slots tests.

* Optimize updateHead() code.

* Add forgotten changes.

* Address review comments on state as argument.
2024-10-30 05:38:53 +00:00
Eugene Kabanov fcf72a6e8e
VC: Electra fixes. (#6631)
* Initial commit.

* Add aggregated attestation processing.

* Add missing presets file.

* Fix compilation error.

* Fix post-rebase compilation error.

* Satisfy push raises requirement.

* Fix sync committee duties retrieval process.

* Fix forks configuration management.

* Fix deposits to use new fork configuration scheme.

* Fix /eth/v2/validator/aggregate_attestation implementation.

* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.

* Simplification of RANDAO fix.

* Fix typo.

* Address review comments and fix tests.

* Fix incorrect status codes in REST test.

* Rework attestation and aggregated attestations processing code.

* Address review comments.

* Fill committee_index in RegisteredAttestation construction code.

* Address review comments part 2.

* Address review comments part 3.

* use Deneb fork epoch

* Add transition from Deneb to Electra into CI finalization test.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-10-16 17:20:39 +00:00
Etan Kissling 91f47a382d
Merge branch 'dev/etan/df-forkedblobs' into feat_eip-7688 2024-10-11 15:52:10 +02:00
Etan Kissling a6311884d1
Merge branch 'unstable' into dev/etan/df-forkedblobs 2024-10-10 00:55:37 +02:00
tersec 3868365ac0
automated consensus spec URL updating to v1.5.0-alpha.8 (#6617) 2024-10-09 08:37:35 +02:00
tersec 50eed81dcf
electra BN attestation aggregating (#6596) 2024-10-07 07:53:38 +00:00
tersec 17d66118e8
automated consensus spec URL updating to v1.5.0-alpha.7 (#6601) 2024-10-06 14:03:56 +00:00
Etan Kissling d976a9313e
Merge branch 'stable' into dev/etan/df-forkedblobs 2024-10-04 16:14:58 +02:00
Pedro Miranda ff0cc8e62e
Electra - Filter aggregated attestations by root/committee (#6594)
* Filter aggregated attestations by root/committee

* fixed validator api documentation link

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-10-04 05:15:48 +00:00
Pedro Miranda 4beb890385
new pool attester slashings endpoint version (V2) for electra (#6585)
* new pool attester slashings endpoint version (V2) for electra

* formatting

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-28 00:00:26 +00:00
Pedro Miranda daf7f899c2
Attestation API updates for Electra (#6557)
* new V2 endpoint for beacon getBlockAttestations

* nnew GET endpoint version (V2) for getPoolAttestations

* new POST endpoint version (V2) for submitPoolAttestations

* remove premature ncli tests

* review improvements

* review comments and increased test coverage

* small improvements

* documentation typos

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-25 12:33:58 +00:00
tersec f2d6166099
remove block fork guessing/inference from REST JSON decoding (#6552)
* remove block fork guessing/inference from REST JSON decoding

* use template to avoid repetitive per-fork code

* consolidate RestPublishedSignedBeaconBlock and RestPublishedSignedBlockContents parsing fork handling
2024-09-25 02:58:53 +00:00
tersec d4e441e694
add Electra attester slashing pool (#6579) 2024-09-24 11:01:40 +02:00
Etan Kissling af75741f2a
Merge branch 'unstable' into feat_eip-7688 2024-09-21 14:44:53 +02:00
tersec 256f6b8b89
automated consensus spec URL updating to v1.5.0-alpha.6 (#6571) 2024-09-21 03:22:41 +02:00
tersec e94417c500
speed up electra attestation tests by 5x (#6563) 2024-09-19 07:46:38 +02:00
Eugene Kabanov e3fcd8b031
REST: getBlockRewards() and getSyncCommitteeRewards() implementation (#6556)
* Initial commit.

* Use temporary state instead of clearance.

* Attempt to fix `finalized`.

* Fix `genesis` response.

* Pre-calculate genesis block rewards response.

* Add implementation for sync committee rewards.

* Add total active balance calculation.

* Add genesis special case.

* Fix negative reward values.

* Address review comments.

* Fix isGenesis implementation and add REST test rules for both calls.
2024-09-18 23:34:32 +00:00
tersec bd47f62f07
tweak indentation (#6548) 2024-09-13 23:31:45 +00:00
Pedro Miranda 7b6caebf6e
Add publishAggregateAndProofsV2 (#6546)
* added new endpoint version(v2) for validator aggregate and proof

* review improvements

* introduced nim template for proof executiion

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-13 21:11:09 +00:00
Pedro Miranda 1ac9b851b9
Add GetAggregateAttestation V2 endpoint version (#6511)
* attestation pool support and tests

* REST endpoints changes

* initial ncli and validator client support

* updated tests file

* fixed typos

* review improvements

* remove V1 endpoint

* revert v1 removal

* V2 endpoint version available to pre electra

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-06 12:15:30 +00:00
Etan Kissling 7ac4756e73
Merge branch 'dev/etan/df-forkedblobs' into feat_eip-7688 2024-09-01 15:48:31 +02:00
Etan Kissling 18af8dcd9f
Merge branch 'stable' into dev/etan/df-forkedblobs 2024-08-30 14:54:33 +02:00
tersec 21aeeaf561
automated consensus spec URL updating to v1.5.0-alpha.5 (#6504) 2024-08-21 14:25:19 +02:00
tersec 904318cf83
automated consensus spec URL updating to v1.5.0-alpha.4 (#6483) 2024-08-09 18:24:49 +02:00
tersec d2c8561fcd
add some Electra config constants to beacon API config endpoint (#6479) 2024-08-07 18:15:29 +02:00
Etan Kissling 3375875e05
remove option to select Capella fork choice algo (#6478)
* remove option to select Capella fork choice algo

With Deneb having run stable for quite a while now, it's time to remove
the option to select the prior fork choice algo from Capella.

* also remove usage from test
2024-08-07 14:48:05 +00:00
Etan Kissling 6e80561080
Electra `BlobSidecar` for EIP-7688 2024-07-26 04:03:42 +02:00
Etan Kissling c2d5a55d95
cleanup pre-deneb fallback 2024-07-26 00:03:47 +02:00
Etan Kissling eff1acd1dc
pre-Deneb case for VC block publishing (ff) 2024-07-25 22:53:07 +02:00
Etan Kissling 365b567350
pre-Deneb case for VC block publishing 2024-07-25 22:45:36 +02:00
Etan Kissling d028baea2a
introduce `ForkedBlobSidecar` for EIP-7688 Electra period before PeerDAS
On `ELECTRA_FORK_EPOCH`, PeerDAS is not yet activated, hence the current
mechanism based on `BlobSidecar` is still in use. With EIP-7688, the
generalized indices of `BeaconBlockBody` get reindexed, changing the
length of the inclusion proof within the `BlobSidecar`. Because network
Req/Resp operations allow responses across fork boundaries, this creates
the need for a `ForkedBlobSidecar` in that layer, same as already done
for `ForkedSignedBeaconBock` for similar reasons.

Note: This PR is only needed if PeerDAS is adopted _after_ EIP-7688.
If PeerDAS is adopted _before_ EIP-7688, a similar PR may be needed for
forked columns. Coincidental `Forked` jank can only be fully avoided if
both features activate at the same epoch, actual changes to blobs aside.
Delaying EIP-7688 for sole purpose of epoch alignemnt is not worth it.
2024-07-25 18:54:38 +02:00
Eugene Kabanov f751fde417
Use Nimbus specific agent string instead of nim-presto defaults. (#6424)
* Use Nimbus specific agent string instead of nim-presto defaults.
Use response(code) overload.

* Fix REST API test.
2024-07-15 02:27:23 +00:00
Kim De Mey ca15c4e433
Replace deprecated ENR functions (#6419) 2024-07-12 17:18:24 +02:00
Eugene Kabanov 031033a6f9
Fix REST /eth/v1/node/identity should return proper MultiAddresses (version 2). (#6422)
* Initial commit.

* Bump nim-libp2p with dualstack fixes.

* Pass announcedAddresses to the `p2p_addresses` list.
2024-07-11 17:39:38 +02:00
Eugene Kabanov c59bb71916
Add implementation of publishBlindedBlockV2() REST API endpoint. (#6413) 2024-07-04 19:48:12 +00:00
Eugene Kabanov 190dbfd481
Fix empty status string handling, address #6175. (#6391) 2024-06-28 05:54:51 +02:00
tersec 88fa8b17ea
automated consensus spec URL updating to v1.5.0-alpha.3 (#6364) 2024-06-16 02:59:25 +02:00
Eugene Kabanov 27664291c4
VC/NIMBUS-BN validators protection. (#6329)
* Beacon node side implementation.

* Validator client side implementation.

* Address review comments and fix the test.

* Only 400 errors could be IndexedErrorMessage, 500 errors are always ErrorMessage.

* Remove VC shutdown functionality.

* Remove magic constants.

* Make arguments more visible and disable default values.

* Address review comments.
2024-06-11 19:38:16 +00:00
tersec 77d926525f
stop using deprecated stew/results (#6343)
* stop using deprecated stew/results

* exception tracking linting
2024-06-11 02:34:56 +00:00
tersec b56a671122
fix most ConvFromXtoItselfNotNeeded hints and unhide remaining ones (#6307) 2024-05-22 13:56:37 +02:00
Eugene Kabanov d7c5bc0397
[VC] Add builderBoostFactor support. (#6294)
* Initial commit.

* Replace localBlockValueBoost with builderBoostFactor.

* Add test.

* Update AllTests.

* Update options.md

* Recover `localBlockValueBoost` for BN-only mode.

* Address review comments.
2024-05-19 04:49:43 +03:00
Eugene Kabanov 0b8cb4efc4
Add metrics to all RPC {state_id} endpoints. (#6282) 2024-05-15 09:52:57 +03:00
tersec 3f972a2ca5
add some support for electra aggregated attestations (#6283) 2024-05-14 06:12:35 +02:00
tersec 6b8061b5d6
automated consensus spec URL updating to v1.5.0-alpha.2 (#6279) 2024-05-09 05:03:10 +00:00
Eugene Kabanov a6f68d3edc
Fix POST to /eth/v2/beacon/blocks unable to verify correct block signature. (#6261) 2024-05-06 04:15:17 +00:00
tersec d139c92df9
explicitly scope AttesterSlashing and IndexedAttestation types to phase0 (#6224) 2024-04-21 05:49:11 +00:00