99bb845227
add column support to beacon chain db ( #6660 )
...
* add column support to beacon chain db
* slight fix
* fix db util
* fix import issues
* gate fulu columns if fulu fork epoch is far future epoch
* fixed all tests file
* fix comment
2024-11-19 06:53:13 +01:00
bd04dcc3dc
Fix syncing problem introduced by light forward syncing PR. ( #6714 )
...
* Fix blobs check response algorithm and add more logging information.
* Add MAX_BLOBS_PER_BLOCK check.
Add tests.
* Adopt AllTests.
* Address review comments.
* One more missed lenu64.
2024-11-19 06:35:09 +01:00
a157760643
Init Fulu fork ( #6677 )
...
* fulu
* fix ci
* fix ncli
* attempt to fix ci
* fix wss_sim build
* fix vc api
* updated tests
* resolve merge conflicts
* fix tests
* fix check
* attempt to fix minimal test in CI
* addressed reviews
---------
Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-11-13 10:29:14 +07:00
b7ea6a627e
fix some deprecation warnings ( #6702 )
2024-11-04 12:19:43 +01:00
18409a69e1
Light forward sync mechanism ( #6515 )
...
* Initial commit.
* Add hybrid syncing.
* Compilation fixes.
* Cast custom event for our purposes.
* Instantiate AsyncEventQueue properly.
* Fix mistype.
* Further research on optimistic updates.
* Fixing circular deps.
* Add backfilling.
* Add block download feature.
* Add block store.
* Update backfill information before storing block.
* Use custom block verifier for backfilling sync.
* Skip signature verification in backfilling.
* Add one more generic reload to storeBackfillBlock().
* Add block verification debugging statements.
* Add more debugging
* Do not use database for backfilling, part 1.
* Fix for stash.
* Stash fixes part 2.
* Prepare for testing.
* Fix assertion.
* Fix post-restart syncing process.
* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.
* Add handling of Duplicates.
* Fix store duration and block backfilled log statements.
* Add proper syncing state log statement.
* Add snappy compression to beaconchain_file.
Format syncing speed properly.
* Add blobs verification.
* Add `slot` number to file structure for easy navigation over stream of compressed objects.
* Change database filename.
* Fix structure size.
* Add more consistency properties.
* Fix checkRepair() issues.
* Preparation to state rebuild process.
* Add plain & compressed size.
* Debugging snappy encode process.
* Add one more debugging line.
* Dump blocks.
* One more filedump.
* Fix chunk corruption code.
* Fix detection issue.
* Some fixes in state rebuilding process.
* Add more clearance steps.
* Move updateHead() back to block_processor.
* Fix compilation issues.
* Make code more async friendly.
* Fix async issues.
Add more information when proposer verification failed.
* Fix 8192 slots issue.
* Fix Future double completion issue.
* Pass updateFlags to some of the core procedures.
* Fix tests.
* Improve initial sync handling mechanism.
* Fix checkStateTransition() performance improvements.
* Add some performance tuning and meters.
* Light client performance tuning.
* Remove debugging statement.
* Use single file descriptor for blockchain file.
* Attempt to fix LC.
* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.
* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.
* Address review comments.
* Address review comments part 2.
* Address review comments part 1.
* Rebase and fix the issues.
* Address review comments part 3.
* Add tests and fix some issues in auto-repair mechanism.
* Add tests to all_tests.
* Rename binary test file to pass restrictions.
* Add `bin` extension to excluded list.
Recover binary test data.
* Rename fixture file to .bin again.
* Update AllTests.
* Address review comments part 4.
* Address review comments part 5 and fix tests.
* Address review comments part 6.
* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.
* Fix forgotten place.
* Post rebase fixes.
* Add unique slots tests.
* Optimize updateHead() code.
* Add forgotten changes.
* Address review comments on state as argument.
2024-10-30 05:38:53 +00:00
03fe86f448
devel-branch-compatibility tweaks for Nim in test_validator_client ( #6686 )
2024-10-30 05:21:20 +01:00
f85521230f
stop checking for irrelevant unsigned wraparound in Electra deposit codepath ( #6683 )
2024-10-28 17:18:01 +01:00
58a34e00a1
fix inconsistent aggregation bits len in Electra ( #6679 )
2024-10-25 16:04:35 +02:00
84eaa57f82
don't use {.noinit.} with case objects ( #6662 )
2024-10-21 01:22:16 +02:00
fcf72a6e8e
VC: Electra fixes. ( #6631 )
...
* Initial commit.
* Add aggregated attestation processing.
* Add missing presets file.
* Fix compilation error.
* Fix post-rebase compilation error.
* Satisfy push raises requirement.
* Fix sync committee duties retrieval process.
* Fix forks configuration management.
* Fix deposits to use new fork configuration scheme.
* Fix /eth/v2/validator/aggregate_attestation implementation.
* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.
* Simplification of RANDAO fix.
* Fix typo.
* Address review comments and fix tests.
* Fix incorrect status codes in REST test.
* Rework attestation and aggregated attestations processing code.
* Address review comments.
* Fill committee_index in RegisteredAttestation construction code.
* Address review comments part 2.
* Address review comments part 3.
* use Deneb fork epoch
* Add transition from Deneb to Electra into CI finalization test.
---------
Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-10-16 17:20:39 +00:00
acad28d05b
avoid combining multiple aggregates from same committee ( #6642 )
2024-10-15 06:02:36 +02:00
c01cf6601a
attestation and aggregated attestation BN fixes for Electra ( #6641 )
...
* attestation and aggregated attestation BN fixes for Electra
* disable electra transition for this PR
* use cleaned-up verification function everywhere
2024-10-14 14:20:26 +00:00
8a91d91b0d
rm ExecutionPayloadV4/electra.ExecutionPayload roundtrip tests ( #6632 )
2024-10-10 10:31:45 +00:00
3868365ac0
automated consensus spec URL updating to v1.5.0-alpha.8 ( #6617 )
2024-10-09 08:37:35 +02:00
b8a424991d
use EF consensus spec v1.5.0-alpha.8 test vectors ( #6616 )
2024-10-09 04:33:17 +00:00
5d11c5229b
Add Web3signer electra support ( #6607 )
...
* Add Electra support to nimbus_signing_node.
Add tests.
Annotate nimbus_signing_node with asyncraises.
* Update AllTests.
2024-10-07 23:31:16 +00:00
50eed81dcf
electra BN attestation aggregating ( #6596 )
2024-10-07 07:53:38 +00:00
17d66118e8
automated consensus spec URL updating to v1.5.0-alpha.7 ( #6601 )
2024-10-06 14:03:56 +00:00
9c9f3707a1
use EF consensus spec v1.5.0-alpha.7 test vectors ( #6600 )
2024-10-06 13:15:56 +02:00
ff0cc8e62e
Electra - Filter aggregated attestations by root/committee ( #6594 )
...
* Filter aggregated attestations by root/committee
* fixed validator api documentation link
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-10-04 05:15:48 +00:00
fa8b7e380f
add Electra beacon chain database state tests ( #6584 )
2024-09-27 07:53:58 +02:00
f2d6166099
remove block fork guessing/inference from REST JSON decoding ( #6552 )
...
* remove block fork guessing/inference from REST JSON decoding
* use template to avoid repetitive per-fork code
* consolidate RestPublishedSignedBeaconBlock and RestPublishedSignedBlockContents parsing fork handling
2024-09-25 02:58:53 +00:00
d4e441e694
add Electra attester slashing pool ( #6579 )
2024-09-24 11:01:40 +02:00
ec831f1c99
clean up config loading in light client tests ( #6574 )
...
The consensus-spec-tests routinely add random experimental features
to their config.yaml which can be ignored.
2024-09-21 12:54:27 +00:00
256f6b8b89
automated consensus spec URL updating to v1.5.0-alpha.6 ( #6571 )
2024-09-21 03:22:41 +02:00
193a982d24
use EF consensus spec v1.5.0-alpha.6 test vectors ( #6568 )
...
* use EF consensus spec v1.5.0-alpha.6 test vectors
* LC changes are good as is, no need for the debug comment
* Fix indentation
* Use `execution_requests` for requests root
* fill in spec reference URLs for ExecutionPayload(Header)
* copy `execution_requests` when blidning blocks for Electra and add test
* fix `test_engine_api_conversions`
---------
Co-authored-by: Etan Kissling <etan@nimbus.team>
2024-09-20 21:32:16 +02:00
9df3d075de
fix blinded block conversion for Deneb and Electra ( #6569 )
...
`blob_gas_used` and `excess_blob_gas` were not copied on blinding
signed beacon block.
2024-09-20 16:02:24 +02:00
e94417c500
speed up electra attestation tests by 5x ( #6563 )
2024-09-19 07:46:38 +02:00
3f0d58492e
add: ssz consensus objs for peerdas ( #6555 )
2024-09-17 22:57:48 +00:00
1feeff4f92
add: recover matrix ( #6553 )
2024-09-17 10:21:55 +00:00
b3c8c71776
add: compute matrix ( #6550 )
...
* add: compute matrix
* rm unsused imports
* rm unused imports 2
* rm bin
* rm unused line
* improve doc
* rid space on proc sig
* review 1
* added seeded PRNG
* randomize blob count as well, with smaller seed
* review 2
2024-09-16 11:19:34 +00:00
f53b621818
add: `get_extended_sample_count` with test ( #6544 )
...
* add: get_extended_sample_count with test
* drop return
* reviews
* review fix
* fixed
* fix doc
* hooked to all_tests
* rm bin
* add updated test file
* early return, maybe need results?
* refactor function intricacies
* drop columnsCount
2024-09-12 17:55:16 +02:00
16c21e1c1e
rm unused code/type conversoins; re-enable Electra block gossip verification ( #6545 )
2024-09-12 11:11:08 +02:00
1ac9b851b9
Add GetAggregateAttestation V2 endpoint version ( #6511 )
...
* attestation pool support and tests
* REST endpoints changes
* initial ncli and validator client support
* updated tests file
* fixed typos
* review improvements
* remove V1 endpoint
* revert v1 removal
* V2 endpoint version available to pre electra
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-06 12:15:30 +00:00
444d1dd093
add: `get_custody_columns` for das_core ( #6532 )
...
* add: get_custody_columnns in das core specs with tests
* apply review changes
* review changes 2
* review 3
2024-09-04 16:05:18 +02:00
8306ad58bd
add: EF test vectors as per alpha 5 for peerdas ( #6531 )
...
* added: EF tests for peerdas kzg crypto
* keep links to alpha 5 (latest)
2024-09-02 17:23:06 +02:00
5a44e39e31
bump nim-kzg4844, enable ckzg2.0.0, fix whatever needed ( #6502 )
2024-09-02 11:47:00 +00:00
f9e44b2a3b
Improved electra attestations packing ( #6498 )
...
* test disjoint comittee bits
* test for aggregated attestations with disjoint bits
* improved attestation pool on-chain collecting and packing
* addressed format issues
* improved sorting method
* updated AllTests-mainnet file
* updated AllTests-mainnet file
* review corrections
* review corrections
* removed old phase0 artifacts
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-08-23 13:26:35 +02:00
44cc72c104
refactor engine API conversions out of EL manager ( #6507 )
2024-08-22 16:12:03 +02:00
21aeeaf561
automated consensus spec URL updating to v1.5.0-alpha.5 ( #6504 )
2024-08-21 14:25:19 +02:00
f258cba816
some v1.5.0-alpha.4 consensus spec URL updates ( #6485 )
2024-08-10 07:09:37 +02:00
904318cf83
automated consensus spec URL updating to v1.5.0-alpha.4 ( #6483 )
2024-08-09 18:24:49 +02:00
3375875e05
remove option to select Capella fork choice algo ( #6478 )
...
* remove option to select Capella fork choice algo
With Deneb having run stable for quite a while now, it's time to remove
the option to select the prior fork choice algo from Capella.
* also remove usage from test
2024-08-07 14:48:05 +00:00
c0fc0f41dd
calculate next slot's withdrawals properly even across epoch boundary ( #6470 )
2024-08-06 13:33:16 +02:00
8333365848
replace validator Bloom filter with partial bucket sort ( #6469 )
2024-08-02 07:30:22 +02:00
ea16edd886
add test for shuffled attestation signatures ( #6459 )
...
Followup of #3212 to test proper signature verification. Also document
possible further optimization based on blst `v0.3.13`.
2024-07-26 23:31:05 +00:00
c373e3ab35
reduce code repetition in `test_toblindedblock` ( #6453 )
...
Use `withAll` to generate the tests for various blinded block types
instead of copy pasting them for every fork.
2024-07-26 22:03:11 +00:00
15cca5f93e
hypergeometric distribution CDF for PeerDAS ( #6447 )
2024-07-25 01:59:24 +00:00
a769550db5
rename LC gindex constants to match spec ( #6444 )
...
Use `_ELECTRA` suffix for gindex constants to match consensus-specs.
2024-07-23 20:20:15 +00:00
cd008ba7ef
Bump NimYAML + bump related changes ( #6431 )
2024-07-17 21:50:29 +02:00
Agnish Ghosh
Eugene Kabanov
tersec
Pedro Miranda
Etan Kissling
Pedro Miranda
Etan Kissling
Kim De Mey