18409a69e1
Light forward sync mechanism ( #6515 )
...
* Initial commit.
* Add hybrid syncing.
* Compilation fixes.
* Cast custom event for our purposes.
* Instantiate AsyncEventQueue properly.
* Fix mistype.
* Further research on optimistic updates.
* Fixing circular deps.
* Add backfilling.
* Add block download feature.
* Add block store.
* Update backfill information before storing block.
* Use custom block verifier for backfilling sync.
* Skip signature verification in backfilling.
* Add one more generic reload to storeBackfillBlock().
* Add block verification debugging statements.
* Add more debugging
* Do not use database for backfilling, part 1.
* Fix for stash.
* Stash fixes part 2.
* Prepare for testing.
* Fix assertion.
* Fix post-restart syncing process.
* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.
* Add handling of Duplicates.
* Fix store duration and block backfilled log statements.
* Add proper syncing state log statement.
* Add snappy compression to beaconchain_file.
Format syncing speed properly.
* Add blobs verification.
* Add `slot` number to file structure for easy navigation over stream of compressed objects.
* Change database filename.
* Fix structure size.
* Add more consistency properties.
* Fix checkRepair() issues.
* Preparation to state rebuild process.
* Add plain & compressed size.
* Debugging snappy encode process.
* Add one more debugging line.
* Dump blocks.
* One more filedump.
* Fix chunk corruption code.
* Fix detection issue.
* Some fixes in state rebuilding process.
* Add more clearance steps.
* Move updateHead() back to block_processor.
* Fix compilation issues.
* Make code more async friendly.
* Fix async issues.
Add more information when proposer verification failed.
* Fix 8192 slots issue.
* Fix Future double completion issue.
* Pass updateFlags to some of the core procedures.
* Fix tests.
* Improve initial sync handling mechanism.
* Fix checkStateTransition() performance improvements.
* Add some performance tuning and meters.
* Light client performance tuning.
* Remove debugging statement.
* Use single file descriptor for blockchain file.
* Attempt to fix LC.
* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.
* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.
* Address review comments.
* Address review comments part 2.
* Address review comments part 1.
* Rebase and fix the issues.
* Address review comments part 3.
* Add tests and fix some issues in auto-repair mechanism.
* Add tests to all_tests.
* Rename binary test file to pass restrictions.
* Add `bin` extension to excluded list.
Recover binary test data.
* Rename fixture file to .bin again.
* Update AllTests.
* Address review comments part 4.
* Address review comments part 5 and fix tests.
* Address review comments part 6.
* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.
* Fix forgotten place.
* Post rebase fixes.
* Add unique slots tests.
* Optimize updateHead() code.
* Add forgotten changes.
* Address review comments on state as argument.
2024-10-30 05:38:53 +00:00
fcf72a6e8e
VC: Electra fixes. ( #6631 )
...
* Initial commit.
* Add aggregated attestation processing.
* Add missing presets file.
* Fix compilation error.
* Fix post-rebase compilation error.
* Satisfy push raises requirement.
* Fix sync committee duties retrieval process.
* Fix forks configuration management.
* Fix deposits to use new fork configuration scheme.
* Fix /eth/v2/validator/aggregate_attestation implementation.
* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.
* Simplification of RANDAO fix.
* Fix typo.
* Address review comments and fix tests.
* Fix incorrect status codes in REST test.
* Rework attestation and aggregated attestations processing code.
* Address review comments.
* Fill committee_index in RegisteredAttestation construction code.
* Address review comments part 2.
* Address review comments part 3.
* use Deneb fork epoch
* Add transition from Deneb to Electra into CI finalization test.
---------
Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-10-16 17:20:39 +00:00
0ecd2798e8
Deprecate logtrace tool. ( #6643 )
...
* Deprecate logtrace tool.
* Eliminate mentions of logtrace in Makefile and scripts.
2024-10-15 13:54:07 +00:00
bc51ba3077
styleCheck tweaks ( #6638 )
2024-10-12 00:51:28 +02:00
181f5f984d
use Deneb genesis for local testnet; add Electra forking support to local tesnet ( #6634 )
...
* use Deneb genesis for local testnet; add Electra forking support to local tesnet
* deneb -> electra in ncli_testnet help
2024-10-10 15:48:05 +00:00
6e610f64e0
rm unused/bitrotted ncli_query ( #6618 )
2024-10-10 05:22:44 +00:00
1258fd3592
Revert "stop testing with broken upstream/version-2-0" ( #6573 )
...
* Revert "stop testing with broken upstream/version-2-0 (#6554 )"
This reverts commit 205dff3378
2024-10-03 01:00:39 +00:00
b20f1ba8b3
Update nim-eth types ( #6583 )
...
Minimal changes needed for compatiblity with
https://github.com/status-im/nim-eth/pull/733 which aligns core types
with execution spec.
2024-09-29 13:32:55 +02:00
4beb890385
new pool attester slashings endpoint version (V2) for electra ( #6585 )
...
* new pool attester slashings endpoint version (V2) for electra
* formatting
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-28 00:00:26 +00:00
daf7f899c2
Attestation API updates for Electra ( #6557 )
...
* new V2 endpoint for beacon getBlockAttestations
* nnew GET endpoint version (V2) for getPoolAttestations
* new POST endpoint version (V2) for submitPoolAttestations
* remove premature ncli tests
* review improvements
* review comments and increased test coverage
* small improvements
* documentation typos
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-25 12:33:58 +00:00
71d8fb546d
update ncli_db to use Opt ( #6575 )
2024-09-23 10:02:09 +02:00
193a982d24
use EF consensus spec v1.5.0-alpha.6 test vectors ( #6568 )
...
* use EF consensus spec v1.5.0-alpha.6 test vectors
* LC changes are good as is, no need for the debug comment
* Fix indentation
* Use `execution_requests` for requests root
* fill in spec reference URLs for ExecutionPayload(Header)
* copy `execution_requests` when blidning blocks for Electra and add test
* fix `test_engine_api_conversions`
---------
Co-authored-by: Etan Kissling <etan@nimbus.team>
2024-09-20 21:32:16 +02:00
e3fcd8b031
REST: getBlockRewards() and getSyncCommitteeRewards() implementation ( #6556 )
...
* Initial commit.
* Use temporary state instead of clearance.
* Attempt to fix `finalized`.
* Fix `genesis` response.
* Pre-calculate genesis block rewards response.
* Add implementation for sync committee rewards.
* Add total active balance calculation.
* Add genesis special case.
* Fix negative reward values.
* Address review comments.
* Fix isGenesis implementation and add REST test rules for both calls.
2024-09-18 23:34:32 +00:00
16c21e1c1e
rm unused code/type conversoins; re-enable Electra block gossip verification ( #6545 )
2024-09-12 11:11:08 +02:00
1ac9b851b9
Add GetAggregateAttestation V2 endpoint version ( #6511 )
...
* attestation pool support and tests
* REST endpoints changes
* initial ncli and validator client support
* updated tests file
* fixed typos
* review improvements
* remove V1 endpoint
* revert v1 removal
* V2 endpoint version available to pre electra
---------
Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
2024-09-06 12:15:30 +00:00
44cc72c104
refactor engine API conversions out of EL manager ( #6507 )
2024-08-22 16:12:03 +02:00
d2c8561fcd
add some Electra config constants to beacon API config endpoint ( #6479 )
2024-08-07 18:15:29 +02:00
8333365848
replace validator Bloom filter with partial bucket sort ( #6469 )
2024-08-02 07:30:22 +02:00
f751fde417
Use Nimbus specific agent string instead of nim-presto defaults. ( #6424 )
...
* Use Nimbus specific agent string instead of nim-presto defaults.
Use response(code) overload.
* Fix REST API test.
2024-07-15 02:27:23 +00:00
13e766d4df
add database electra block tests ( #6394 )
2024-07-01 04:18:41 +00:00
7a8c1d818a
Bump nim-eth for Opt changes and make Opt related adjustments ( #6369 )
2024-06-18 23:46:51 +02:00
238589868c
Bump nim-web3 and nim-eth ( #6345 )
...
In nim-web3 all std.Option are replaced by results.Opt. The same goes in nim-eth, with additional fields name changes and GasInt changed from int64 to uint64.
2024-06-12 08:46:39 +07:00
f97703e6ae
Bump tornado from 6.3.3 to 6.4.1 in /ncli ( #6328 )
...
Bumps [tornado](https://github.com/tornadoweb/tornado ) from 6.3.3 to 6.4.1.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst )
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.3...v6.4.1 )
---
updated-dependencies:
- dependency-name: tornado
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-06 23:38:04 +00:00
b56a671122
fix most ConvFromXtoItselfNotNeeded hints and unhide remaining ones ( #6307 )
2024-05-22 13:56:37 +02:00
e0f8ea752b
era: speed up index reading ( #6305 )
2024-05-22 10:21:04 +02:00
dd452f71d3
rm outdated/semi-duplicate execution layer withdrawal request processing ( #6301 )
2024-05-22 01:59:11 +02:00
045c4cf185
electra attestation updates ( #6295 )
...
* electra attestation updates
In Electra, we have two attestation formats: on-chain and on-network -
the former combines all committees of a slot in a single committee bit
list.
This PR makes a number of cleanups to move towards fixing this -
attestation packing however still needs to be fixed as it currently
creates attestations with a single committee only which is very
inefficient.
* more attestations in the blocks
* signing and aggregation fixes
* tool fix
* test, import
2024-05-17 15:37:41 +03:00
72ac82ec0b
Bump jinja2 from 3.1.3 to 3.1.4 in /ncli ( #6268 )
...
Bumps [jinja2](https://github.com/pallets/jinja ) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases )
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst )
- [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4 )
---
updated-dependencies:
- dependency-name: jinja2
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-06 22:35:13 +00:00
6119389c3a
add EF consensus spec test Electra attestation operations fixture ( #6248 )
2024-04-28 00:52:14 +00:00
8c4ddd64c0
add EF consensus spec test attester slashing, proposer slashing, voluntary exit, and execution layer withdrawal request tests ( #6234 )
2024-04-25 08:41:17 +00:00
d139c92df9
explicitly scope AttesterSlashing and IndexedAttestation types to phase0 ( #6224 )
2024-04-21 05:49:11 +00:00
603c83522e
explicitly refer to phase0.{Attestation,TrustedAttestation} rather than sans module name ( #6214 )
2024-04-17 20:44:29 +00:00
0d534224b3
remove some debugRaiseAsserts and fill in Electra functionality ( #6178 )
2024-04-06 07:46:02 +00:00
06894dd2f6
remove some debugRaiseAsserts and fill in Electra functionality ( #6177 )
2024-04-05 21:30:06 +02:00
0b6f75b88b
remove some debugRaiseAsserts and fill in actual Electra functionality ( #6172 )
2024-04-04 05:17:31 +02:00
67ade106cc
Bump pillow from 10.2.0 to 10.3.0 in /ncli ( #6171 )
...
Bumps [pillow](https://github.com/python-pillow/Pillow ) from 10.2.0 to 10.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases )
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst )
- [Commits](https://github.com/python-pillow/Pillow/compare/10.2.0...10.3.0 )
---
updated-dependencies:
- dependency-name: pillow
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-03 18:56:42 +00:00
8d0a4a87e7
add function to import blobs to `ncli_db` ( #6166 )
...
* add function to import blobs to `ncli_db`
`ncli_db` does not support importing blobs, making it impossible to
complete the database solely from dumped files. Add the missing support.
* pass `cfg`
2024-04-03 18:55:53 +00:00
7fa32b7f02
add Electra to ConsensusFork enum ( #6169 )
...
* add Electra to ConsensusFork enum
* fix gnosis check
2024-04-03 16:43:43 +02:00
02ccfd488b
Bump black from 21.12b0 to 24.3.0 in /ncli ( #6111 )
...
Bumps [black](https://github.com/psf/black ) from 21.12b0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases )
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md )
- [Commits](https://github.com/psf/black/commits/24.3.0 )
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-20 21:54:09 +00:00
1fe6efcf53
Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1 ( #6106 )
...
Unify EthCall/EthSend into TransactionArgs (#138 )
2024-03-20 14:39:12 +07:00
5d42859176
make `Gwei` `distinct` ( #6090 )
...
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
f40083f1e5
annotate `validator_db_aggregator` with `{.raises.}` ( #6094 )
...
Show proper error message when `validator_db_aggregator` raises errors.
2024-03-17 16:17:07 +01:00
a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 ( #6038 )
...
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.
Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` ( #6036 )
...
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.
- https://eips.ethereum.org/EIPS/eip-4881#specification
Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.
That also explains the incorrect REST response reported in #5508 .
The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
b4db011daf
enable --styleCheck:error in ncli/ and research/ ( #6014 )
2024-03-02 06:25:17 +01:00
d7e9ff2a98
fix some stylecheck consistency Name hints in ncli/ and research/ ( #6004 )
2024-03-01 05:59:34 +01:00
f65c1121d2
add Electra overloads for spec functions; add Electra block processing ( #5963 )
2024-02-26 02:38:21 +00:00
a4f4a35845
Revert "initial Electra support skeleton" ( #5955 )
...
* Revert "initial Electra support skeleton (#5946 )"
This reverts commit d09bf3b587
2024-02-25 19:42:44 +00:00
d09bf3b587
initial Electra support skeleton ( #5946 )
2024-02-24 13:44:15 +00:00
e865817d44
add some push raises, ElectraBeaconStateNoImmutableValidators, and ELECTRA_FORK_VERSION/EPOCH ( #5949 )
2024-02-24 05:08:22 +00:00
Eugene Kabanov
tersec
Jacek Sieka
Pedro Miranda
Pedro Miranda
Kim De Mey
andri lim
dependabot[bot]
Etan Kissling