status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
7,049 Commits 113 Branches 114 Tags 213 MiB
Commit Graph

348 Commits

Author SHA1 Message Date
Agnish Ghosh a157760643
Init Fulu fork (#6677) 
* fulu

* fix ci

* fix ncli

* attempt to fix ci

* fix wss_sim build

* fix vc api

* updated tests

* resolve merge conflicts

* fix tests

* fix check

* attempt to fix minimal test in CI

* addressed reviews

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
 2024-11-13 10:29:14 +07:00
tersec b7ea6a627e
fix some deprecation warnings (#6702) 2024-11-04 12:19:43 +01:00
Eugene Kabanov 18409a69e1
Light forward sync mechanism (#6515) 
* Initial commit.

* Add hybrid syncing.

* Compilation fixes.

* Cast custom event for our purposes.

* Instantiate AsyncEventQueue properly.

* Fix mistype.

* Further research on optimistic updates.

* Fixing circular deps.

* Add backfilling.

* Add block download feature.

* Add block store.

* Update backfill information before storing block.

* Use custom block verifier for backfilling sync.

* Skip signature verification in backfilling.

* Add one more generic reload to storeBackfillBlock().

* Add block verification debugging statements.

* Add more debugging

* Do not use database for backfilling, part 1.

* Fix for stash.

* Stash fixes part 2.

* Prepare for testing.

* Fix assertion.

* Fix post-restart syncing process.

* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.

* Add handling of Duplicates.

* Fix store duration and block backfilled log statements.

* Add proper syncing state log statement.

* Add snappy compression to beaconchain_file.
Format syncing speed properly.

* Add blobs verification.

* Add `slot` number to file structure for easy navigation over stream of compressed objects.

* Change database filename.

* Fix structure size.

* Add more consistency properties.

* Fix checkRepair() issues.

* Preparation to state rebuild process.

* Add plain & compressed size.

* Debugging snappy encode process.

* Add one more debugging line.

* Dump blocks.

* One more filedump.

* Fix chunk corruption code.

* Fix detection issue.

* Some fixes in state rebuilding process.

* Add more clearance steps.

* Move updateHead() back to block_processor.

* Fix compilation issues.

* Make code more async friendly.

* Fix async issues.
Add more information when proposer verification failed.

* Fix 8192 slots issue.

* Fix Future double completion issue.

* Pass updateFlags to some of the core procedures.

* Fix tests.

* Improve initial sync handling mechanism.

* Fix checkStateTransition() performance improvements.

* Add some performance tuning and meters.

* Light client performance tuning.

* Remove debugging statement.

* Use single file descriptor for blockchain file.

* Attempt to fix LC.

* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.

* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.

* Address review comments.

* Address review comments part 2.

* Address review comments part 1.

* Rebase and fix the issues.

* Address review comments part 3.

* Add tests and fix some issues in auto-repair mechanism.

* Add tests to all_tests.

* Rename binary test file to pass restrictions.

* Add `bin` extension to excluded list.
Recover binary test data.

* Rename fixture file to .bin again.

* Update AllTests.

* Address review comments part 4.

* Address review comments part 5 and fix tests.

* Address review comments part 6.

* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.

* Fix forgotten place.

* Post rebase fixes.

* Add unique slots tests.

* Optimize updateHead() code.

* Add forgotten changes.

* Address review comments on state as argument.
 2024-10-30 05:38:53 +00:00
Eugene Kabanov fcf72a6e8e
VC: Electra fixes. (#6631) 
* Initial commit.

* Add aggregated attestation processing.

* Add missing presets file.

* Fix compilation error.

* Fix post-rebase compilation error.

* Satisfy push raises requirement.

* Fix sync committee duties retrieval process.

* Fix forks configuration management.

* Fix deposits to use new fork configuration scheme.

* Fix /eth/v2/validator/aggregate_attestation implementation.

* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.

* Simplification of RANDAO fix.

* Fix typo.

* Address review comments and fix tests.

* Fix incorrect status codes in REST test.

* Rework attestation and aggregated attestations processing code.

* Address review comments.

* Fill committee_index in RegisteredAttestation construction code.

* Address review comments part 2.

* Address review comments part 3.

* use Deneb fork epoch

* Add transition from Deneb to Electra into CI finalization test.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
 2024-10-16 17:20:39 +00:00
Eugene Kabanov 0ecd2798e8
Deprecate logtrace tool. (#6643) 
* Deprecate logtrace tool.

* Eliminate mentions of logtrace in Makefile and scripts.
 2024-10-15 13:54:07 +00:00
tersec bc51ba3077
styleCheck tweaks (#6638) 2024-10-12 00:51:28 +02:00
tersec 181f5f984d
use Deneb genesis for local testnet; add Electra forking support to local tesnet (#6634) 
* use Deneb genesis for local testnet; add Electra forking support to local tesnet

* deneb -> electra in ncli_testnet help
 2024-10-10 15:48:05 +00:00
tersec 6e610f64e0
rm unused/bitrotted ncli_query (#6618) 2024-10-10 05:22:44 +00:00
tersec 1258fd3592
Revert "stop testing with broken upstream/version-2-0" (#6573) 
* Revert "stop testing with broken upstream/version-2-0 (#6554)"

This reverts commit 205dff3378.

* stop using template which Nim 2.0.10 will apparently make recursive
 2024-10-03 01:00:39 +00:00
Jacek Sieka b20f1ba8b3
Update nim-eth types (#6583) 
Minimal changes needed for compatiblity with
https://github.com/status-im/nim-eth/pull/733 which aligns core types
with execution spec.
 2024-09-29 13:32:55 +02:00
Pedro Miranda 4beb890385
new pool attester slashings endpoint version (V2) for electra (#6585) 
* new pool attester slashings endpoint version (V2) for electra

* formatting

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
 2024-09-28 00:00:26 +00:00
Pedro Miranda daf7f899c2
Attestation API updates for Electra (#6557) 
* new V2 endpoint for beacon getBlockAttestations

* nnew GET endpoint version (V2) for getPoolAttestations

* new POST endpoint version (V2) for submitPoolAttestations

* remove premature ncli tests

* review improvements

* review comments and increased test coverage

* small improvements

* documentation typos

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
 2024-09-25 12:33:58 +00:00
tersec 71d8fb546d
update ncli_db to use Opt (#6575) 2024-09-23 10:02:09 +02:00
tersec 193a982d24
use EF consensus spec v1.5.0-alpha.6 test vectors (#6568) 
* use EF consensus spec v1.5.0-alpha.6 test vectors

* LC changes are good as is, no need for the debug comment

* Fix indentation

* Use `execution_requests` for requests root

* fill in spec reference URLs for ExecutionPayload(Header)

* copy `execution_requests` when blidning blocks for Electra and add test

* fix `test_engine_api_conversions`

---------

Co-authored-by: Etan Kissling <etan@nimbus.team>
 2024-09-20 21:32:16 +02:00
Eugene Kabanov e3fcd8b031
REST: getBlockRewards() and getSyncCommitteeRewards() implementation (#6556) 
* Initial commit.

* Use temporary state instead of clearance.

* Attempt to fix `finalized`.

* Fix `genesis` response.

* Pre-calculate genesis block rewards response.

* Add implementation for sync committee rewards.

* Add total active balance calculation.

* Add genesis special case.

* Fix negative reward values.

* Address review comments.

* Fix isGenesis implementation and add REST test rules for both calls.
 2024-09-18 23:34:32 +00:00
tersec 16c21e1c1e
rm unused code/type conversoins; re-enable Electra block gossip verification (#6545) 2024-09-12 11:11:08 +02:00
Pedro Miranda 1ac9b851b9
Add GetAggregateAttestation V2 endpoint version (#6511) 
* attestation pool support and tests

* REST endpoints changes

* initial ncli and validator client support

* updated tests file

* fixed typos

* review improvements

* remove V1 endpoint

* revert v1 removal

* V2 endpoint version available to pre electra

---------

Co-authored-by: Pedro Miranda <pedro.miranda@nimbus.team>
 2024-09-06 12:15:30 +00:00
tersec 44cc72c104
refactor engine API conversions out of EL manager (#6507) 2024-08-22 16:12:03 +02:00
tersec d2c8561fcd
add some Electra config constants to beacon API config endpoint (#6479) 2024-08-07 18:15:29 +02:00
tersec 8333365848
replace validator Bloom filter with partial bucket sort (#6469) 2024-08-02 07:30:22 +02:00
Eugene Kabanov f751fde417
Use Nimbus specific agent string instead of nim-presto defaults. (#6424) 
* Use Nimbus specific agent string instead of nim-presto defaults.
Use response(code) overload.

* Fix REST API test.
 2024-07-15 02:27:23 +00:00
tersec 13e766d4df
add database electra block tests (#6394) 2024-07-01 04:18:41 +00:00
Kim De Mey 7a8c1d818a
Bump nim-eth for Opt changes and make Opt related adjustments (#6369) 2024-06-18 23:46:51 +02:00
andri lim 238589868c
Bump nim-web3 and nim-eth (#6345) 
In nim-web3 all std.Option are replaced by results.Opt. The same goes in nim-eth, with additional fields name changes and GasInt changed from int64 to uint64.
 2024-06-12 08:46:39 +07:00
dependabot[bot] f97703e6ae
Bump tornado from 6.3.3 to 6.4.1 in /ncli (#6328) 
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.3 to 6.4.1.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.3...v6.4.1)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-06-06 23:38:04 +00:00
tersec b56a671122
fix most ConvFromXtoItselfNotNeeded hints and unhide remaining ones (#6307) 2024-05-22 13:56:37 +02:00
Jacek Sieka e0f8ea752b
era: speed up index reading (#6305) 2024-05-22 10:21:04 +02:00
tersec dd452f71d3
rm outdated/semi-duplicate execution layer withdrawal request processing (#6301) 2024-05-22 01:59:11 +02:00
Jacek Sieka 045c4cf185
electra attestation updates (#6295) 
* electra attestation updates

In Electra, we have two attestation formats: on-chain and on-network -
the former combines all committees of a slot in a single committee bit
list.

This PR makes a number of cleanups to move towards fixing this -
attestation packing however still needs to be fixed as it currently
creates attestations with a single committee only which is very
inefficient.

* more attestations in the blocks

* signing and aggregation fixes

* tool fix

* test, import
 2024-05-17 15:37:41 +03:00
dependabot[bot] 72ac82ec0b
Bump jinja2 from 3.1.3 to 3.1.4 in /ncli (#6268) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-05-06 22:35:13 +00:00
tersec 6119389c3a
add EF consensus spec test Electra attestation operations fixture (#6248) 2024-04-28 00:52:14 +00:00
tersec 8c4ddd64c0
add EF consensus spec test attester slashing, proposer slashing, voluntary exit, and execution layer withdrawal request tests (#6234) 2024-04-25 08:41:17 +00:00
tersec d139c92df9
explicitly scope AttesterSlashing and IndexedAttestation types to phase0 (#6224) 2024-04-21 05:49:11 +00:00
tersec 603c83522e
explicitly refer to phase0.{Attestation,TrustedAttestation} rather than sans module name (#6214) 2024-04-17 20:44:29 +00:00
tersec 0d534224b3
remove some debugRaiseAsserts and fill in Electra functionality (#6178) 2024-04-06 07:46:02 +00:00
tersec 06894dd2f6
remove some debugRaiseAsserts and fill in Electra functionality (#6177) 2024-04-05 21:30:06 +02:00
tersec 0b6f75b88b
remove some debugRaiseAsserts and fill in actual Electra functionality (#6172) 2024-04-04 05:17:31 +02:00
dependabot[bot] 67ade106cc
Bump pillow from 10.2.0 to 10.3.0 in /ncli (#6171) 
Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.2.0 to 10.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/10.2.0...10.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-04-03 18:56:42 +00:00
Etan Kissling 8d0a4a87e7
add function to import blobs to `ncli_db` (#6166) 
* add function to import blobs to `ncli_db`

`ncli_db` does not support importing blobs, making it impossible to
complete the database solely from dumped files. Add the missing support.

* pass `cfg`
 2024-04-03 18:55:53 +00:00
tersec 7fa32b7f02
add Electra to ConsensusFork enum (#6169) 
* add Electra to ConsensusFork enum

* fix gnosis check
 2024-04-03 16:43:43 +02:00
dependabot[bot] 02ccfd488b
Bump black from 21.12b0 to 24.3.0 in /ncli (#6111) 
Bumps [black](https://github.com/psf/black) from 21.12b0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/commits/24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-03-20 21:54:09 +00:00
andri lim 1fe6efcf53
Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1 (#6106) 
Unify EthCall/EthSend into TransactionArgs (#138)
 2024-03-20 14:39:12 +07:00
Etan Kissling 5d42859176
make `Gwei` `distinct` (#6090) 
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
 2024-03-19 14:22:07 +01:00
Etan Kissling f40083f1e5
annotate `validator_db_aggregator` with `{.raises.}` (#6094) 
Show proper error message when `validator_db_aggregator` raises errors.
 2024-03-17 16:17:07 +01:00
Etan Kissling a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 (#6038) 
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
 2024-03-08 14:22:03 +01:00
Etan Kissling 50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` (#6036) 
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
 2024-03-07 18:42:52 +01:00
tersec b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
tersec d7e9ff2a98
fix some stylecheck consistency Name hints in ncli/ and research/ (#6004) 2024-03-01 05:59:34 +01:00
tersec f65c1121d2
add Electra overloads for spec functions; add Electra block processing (#5963) 2024-02-26 02:38:21 +00:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955) 
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
 2024-02-25 19:42:44 +00:00