1637 Commits

Author SHA1 Message Date
tersec
d791299262
rm support for deprecated reqStep in req/resp (#6857)
* rm support for deprecated reqStep in req/resp

* work around / character in platform
2025-01-18 05:11:06 +01:00
tersec
d04df8a091
sync manager should allow for electra blob counts (#6855)
* sync manager should allow for electra blob counts

* upload-artifact@v3 deprecation
2025-01-17 12:53:41 +00:00
Agnish Ghosh
bc55a3a0c7
add: compute_columns_for_custody_group tests (#6838)
* add new fixture networking tests

* update all tests

* fix test

* avoid extra variables

* other fixes

* oops

* updated tests file
2025-01-12 05:04:55 +00:00
Etan Kissling
a94ba8cbb5
Add test to compare SSZ bounds against Teku impl (#6836)
* Add test to compare SSZ bounds against Teku impl

Ensure that our SSZ bounds are computed the same way as in Teku by
comparing the bounds for `deneb.SignedBeaconBlock`.

* Lint

* Make platform agnostic regarding how Git checks out newlines
2025-01-10 22:55:42 +00:00
Etan Kissling
c63862cf88
validate EL block hash when running consensus block tests (#6406)
* validate EL block hash when running consensus block tests

We currently don't have an easy way to test EL block hash computation.
As the EL block hash in consensus-spec-tests is computed correctly,
update the test runners that load block from test files to also verify
the EL block hash. This increases missing test coverage.

Requires https://github.com/ethereum/consensus-specs/pull/3829

* fix

* resolve merge conflicts

* fix genesis case, and deal with `incorrect_block_hash` test

* add missing export marker

* fix import

* htr mutates underlying data, messing with differ, create copy in test

* Handle payloads with empty tx (unsupported in ordered trie tool)

* Update copyright years

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2025-01-10 13:34:49 +00:00
tersec
adda9738a4
automated consensus spec URL updating to v1.5.0-beta.0 (#6833) 2025-01-10 13:10:49 +01:00
tersec
046c37903d
use EF consensus spec v1.5.0-beta.0 test vectors (#6832) 2025-01-10 10:42:30 +01:00
tersec
4cca50873e
fix /eth/v1/beacon/rewards/block/{block_id} sync_aggregate value (#6829)
* fix /eth/v1/beacon/rewards/block/{block_id} sync_aggregate value

* adjust tests

* copyright year linting
2025-01-09 06:03:23 +01:00
tersec
bb22476175
some manual consensus spec URL updates to v1.5.0-alpha.10 (#6827)
* some manual consensus spec URL updates to v1.5.0-alpha.10

* copyright year linting
2025-01-08 06:47:22 +00:00
Agnish Ghosh
0a2da6897c
clean up eip7594 naming, useful links, test suite naming (#6820)
* clean up eip7594 naming, useful links, test suite naming

* fix copyright year

* updated tests
2025-01-07 22:49:22 +00:00
tersec
fb1c3ba718
add support for Electra blob subnets and IDs (#6806)
* add support for Electra blob subnets and IDs

* fix tests
2024-12-31 18:12:43 +01:00
Agnish Ghosh
c0108c2f2a
introduced custody groups, and renamed csc to cgc (#6789)
* introduced custody groups, and renamed csc to cgc

* accomodate tests

* revert to naming columns

* applied review changes

* updated all tests file

* addressed review 2

* merged in typo fixes by airdop farmers/other spam PRs

* handle lint ci

* shift to iterators, avoid redundant copying
2024-12-28 22:07:12 +01:00
tersec
9c9cbed987
implement SingleAttestation (#6783)
* implement SingleAttestation

* improve SingleAttestation shortLog

* avoid double ShuffleRef lookups in majority/happy case

* restore full electra check_aggregation_count, separately from phase0 one
2024-12-23 16:32:29 +00:00
tersec
561d845b26
add tests for Electra and Fulu gossip transitions (#6784) 2024-12-22 20:28:18 +01:00
Agnish Ghosh
06cf78af99
make getTargetGossipState aware of Fulu + other fixes (#6780)
* make getTargetGossipState aware of Fulu, make it aware of electra as well in LC

* fix indent

* minonr fix
2024-12-20 18:39:49 +01:00
Agnish Ghosh
c728d8754c
custody subnet count decoding during discovery (#6777)
* renamed eip7594_helpers to peerdas_helpers, added csc subnet decoding during discovery

* added link
2024-12-20 11:23:46 +01:00
tersec
232cf2a538
automated consensus spec URL updating to v1.5.0-alpha.10 (#6762) 2024-12-15 17:36:50 +01:00
tersec
7cb0a61471
use EF consensus spec v1.5.0-alpha.10 test vectors (#6761) 2024-12-15 14:18:52 +01:00
Jacek Sieka
031d24ff41
taskpools: bump (#6757)
* taskpools: bump

* better raises effects
* get rid of custom naming

* bump

* oops

* bump stable
2024-12-11 11:12:56 +00:00
Etan Kissling
f7e2507d14
Run LC data collection tests (#6737)
Add test runner for future tests / ensure it gets compiled.
2024-11-29 13:21:00 +00:00
tersec
5579c5234f
automated consensus spec URL updating to v1.5.0-alpha.9 (#6725) 2024-11-24 13:16:09 +01:00
tersec
710f16f510
use EF consensus spec test vectors v1.5.0-alpha.9 (#6724) 2024-11-24 09:47:36 +01:00
Agnish Ghosh
99bb845227
add column support to beacon chain db (#6660)
* add column support to beacon chain db

* slight fix

* fix db util

* fix import issues

* gate fulu columns if fulu fork epoch is far future epoch

* fixed all tests file

* fix comment
2024-11-19 06:53:13 +01:00
Eugene Kabanov
bd04dcc3dc
Fix syncing problem introduced by light forward syncing PR. (#6714)
* Fix blobs check response algorithm and add more logging information.

* Add MAX_BLOBS_PER_BLOCK check.
Add tests.

* Adopt AllTests.

* Address review comments.

* One more missed lenu64.
2024-11-19 06:35:09 +01:00
Agnish Ghosh
a157760643
Init Fulu fork (#6677)
* fulu

* fix ci

* fix ncli

* attempt to fix ci

* fix wss_sim build

* fix vc api

* updated tests

* resolve merge conflicts

* fix tests

* fix check

* attempt to fix minimal test in CI

* addressed reviews

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-11-13 10:29:14 +07:00
tersec
b7ea6a627e
fix some deprecation warnings (#6702) 2024-11-04 12:19:43 +01:00
Eugene Kabanov
18409a69e1
Light forward sync mechanism (#6515)
* Initial commit.

* Add hybrid syncing.

* Compilation fixes.

* Cast custom event for our purposes.

* Instantiate AsyncEventQueue properly.

* Fix mistype.

* Further research on optimistic updates.

* Fixing circular deps.

* Add backfilling.

* Add block download feature.

* Add block store.

* Update backfill information before storing block.

* Use custom block verifier for backfilling sync.

* Skip signature verification in backfilling.

* Add one more generic reload to storeBackfillBlock().

* Add block verification debugging statements.

* Add more debugging

* Do not use database for backfilling, part 1.

* Fix for stash.

* Stash fixes part 2.

* Prepare for testing.

* Fix assertion.

* Fix post-restart syncing process.

* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.

* Add handling of Duplicates.

* Fix store duration and block backfilled log statements.

* Add proper syncing state log statement.

* Add snappy compression to beaconchain_file.
Format syncing speed properly.

* Add blobs verification.

* Add `slot` number to file structure for easy navigation over stream of compressed objects.

* Change database filename.

* Fix structure size.

* Add more consistency properties.

* Fix checkRepair() issues.

* Preparation to state rebuild process.

* Add plain & compressed size.

* Debugging snappy encode process.

* Add one more debugging line.

* Dump blocks.

* One more filedump.

* Fix chunk corruption code.

* Fix detection issue.

* Some fixes in state rebuilding process.

* Add more clearance steps.

* Move updateHead() back to block_processor.

* Fix compilation issues.

* Make code more async friendly.

* Fix async issues.
Add more information when proposer verification failed.

* Fix 8192 slots issue.

* Fix Future double completion issue.

* Pass updateFlags to some of the core procedures.

* Fix tests.

* Improve initial sync handling mechanism.

* Fix checkStateTransition() performance improvements.

* Add some performance tuning and meters.

* Light client performance tuning.

* Remove debugging statement.

* Use single file descriptor for blockchain file.

* Attempt to fix LC.

* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.

* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.

* Address review comments.

* Address review comments part 2.

* Address review comments part 1.

* Rebase and fix the issues.

* Address review comments part 3.

* Add tests and fix some issues in auto-repair mechanism.

* Add tests to all_tests.

* Rename binary test file to pass restrictions.

* Add `bin` extension to excluded list.
Recover binary test data.

* Rename fixture file to .bin again.

* Update AllTests.

* Address review comments part 4.

* Address review comments part 5 and fix tests.

* Address review comments part 6.

* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.

* Fix forgotten place.

* Post rebase fixes.

* Add unique slots tests.

* Optimize updateHead() code.

* Add forgotten changes.

* Address review comments on state as argument.
2024-10-30 05:38:53 +00:00
tersec
03fe86f448
devel-branch-compatibility tweaks for Nim in test_validator_client (#6686) 2024-10-30 05:21:20 +01:00
tersec
f85521230f
stop checking for irrelevant unsigned wraparound in Electra deposit codepath (#6683) 2024-10-28 17:18:01 +01:00
tersec
58a34e00a1
fix inconsistent aggregation bits len in Electra (#6679) 2024-10-25 16:04:35 +02:00
tersec
84eaa57f82
don't use {.noinit.} with case objects (#6662) 2024-10-21 01:22:16 +02:00
Eugene Kabanov
fcf72a6e8e
VC: Electra fixes. (#6631)
* Initial commit.

* Add aggregated attestation processing.

* Add missing presets file.

* Fix compilation error.

* Fix post-rebase compilation error.

* Satisfy push raises requirement.

* Fix sync committee duties retrieval process.

* Fix forks configuration management.

* Fix deposits to use new fork configuration scheme.

* Fix /eth/v2/validator/aggregate_attestation implementation.

* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.

* Simplification of RANDAO fix.

* Fix typo.

* Address review comments and fix tests.

* Fix incorrect status codes in REST test.

* Rework attestation and aggregated attestations processing code.

* Address review comments.

* Fill committee_index in RegisteredAttestation construction code.

* Address review comments part 2.

* Address review comments part 3.

* use Deneb fork epoch

* Add transition from Deneb to Electra into CI finalization test.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-10-16 17:20:39 +00:00
tersec
acad28d05b
avoid combining multiple aggregates from same committee (#6642) 2024-10-15 06:02:36 +02:00
tersec
c01cf6601a
attestation and aggregated attestation BN fixes for Electra (#6641)
* attestation and aggregated attestation BN fixes for Electra

* disable electra transition for this PR

* use cleaned-up verification function everywhere
2024-10-14 14:20:26 +00:00
tersec
8a91d91b0d
rm ExecutionPayloadV4/electra.ExecutionPayload roundtrip tests (#6632) 2024-10-10 10:31:45 +00:00
tersec
3868365ac0
automated consensus spec URL updating to v1.5.0-alpha.8 (#6617) 2024-10-09 08:37:35 +02:00
tersec
b8a424991d
use EF consensus spec v1.5.0-alpha.8 test vectors (#6616) 2024-10-09 04:33:17 +00:00
Eugene Kabanov
5d11c5229b
Add Web3signer electra support (#6607)
* Add Electra support to nimbus_signing_node.
Add tests.
Annotate nimbus_signing_node with asyncraises.

* Update AllTests.
2024-10-07 23:31:16 +00:00
tersec
50eed81dcf
electra BN attestation aggregating (#6596) 2024-10-07 07:53:38 +00:00
tersec
17d66118e8
automated consensus spec URL updating to v1.5.0-alpha.7 (#6601) 2024-10-06 14:03:56 +00:00
tersec
9c9f3707a1
use EF consensus spec v1.5.0-alpha.7 test vectors (#6600) 2024-10-06 13:15:56 +02:00
Pedro Miranda
ff0cc8e62e
Electra - Filter aggregated attestations by root/committee (#6594)
* Filter aggregated attestations by root/committee

* fixed validator api documentation link

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-10-04 05:15:48 +00:00
tersec
fa8b7e380f
add Electra beacon chain database state tests (#6584) 2024-09-27 07:53:58 +02:00
tersec
f2d6166099
remove block fork guessing/inference from REST JSON decoding (#6552)
* remove block fork guessing/inference from REST JSON decoding

* use template to avoid repetitive per-fork code

* consolidate RestPublishedSignedBeaconBlock and RestPublishedSignedBlockContents parsing fork handling
2024-09-25 02:58:53 +00:00
tersec
d4e441e694
add Electra attester slashing pool (#6579) 2024-09-24 11:01:40 +02:00
Etan Kissling
ec831f1c99
clean up config loading in light client tests (#6574)
The consensus-spec-tests routinely add random experimental features
to their config.yaml which can be ignored.
2024-09-21 12:54:27 +00:00
tersec
256f6b8b89
automated consensus spec URL updating to v1.5.0-alpha.6 (#6571) 2024-09-21 03:22:41 +02:00
tersec
193a982d24
use EF consensus spec v1.5.0-alpha.6 test vectors (#6568)
* use EF consensus spec v1.5.0-alpha.6 test vectors

* LC changes are good as is, no need for the debug comment

* Fix indentation

* Use `execution_requests` for requests root

* fill in spec reference URLs for ExecutionPayload(Header)

* copy `execution_requests` when blidning blocks for Electra and add test

* fix `test_engine_api_conversions`

---------

Co-authored-by: Etan Kissling <etan@nimbus.team>
2024-09-20 21:32:16 +02:00
Etan Kissling
9df3d075de
fix blinded block conversion for Deneb and Electra (#6569)
`blob_gas_used` and `excess_blob_gas` were not copied on blinding
signed beacon block.
2024-09-20 16:02:24 +02:00
tersec
e94417c500
speed up electra attestation tests by 5x (#6563) 2024-09-19 07:46:38 +02:00