Commit Graph

7009 Commits

Author SHA1 Message Date
Agnish Ghosh c4d8ae40b0
fix (#6688) 2024-10-30 15:16:02 +01:00
Eugene Kabanov 18409a69e1
Light forward sync mechanism (#6515)
* Initial commit.

* Add hybrid syncing.

* Compilation fixes.

* Cast custom event for our purposes.

* Instantiate AsyncEventQueue properly.

* Fix mistype.

* Further research on optimistic updates.

* Fixing circular deps.

* Add backfilling.

* Add block download feature.

* Add block store.

* Update backfill information before storing block.

* Use custom block verifier for backfilling sync.

* Skip signature verification in backfilling.

* Add one more generic reload to storeBackfillBlock().

* Add block verification debugging statements.

* Add more debugging

* Do not use database for backfilling, part 1.

* Fix for stash.

* Stash fixes part 2.

* Prepare for testing.

* Fix assertion.

* Fix post-restart syncing process.

* Update backfill loading log statement.
Use proper backfill slot callback for sync manager.

* Add handling of Duplicates.

* Fix store duration and block backfilled log statements.

* Add proper syncing state log statement.

* Add snappy compression to beaconchain_file.
Format syncing speed properly.

* Add blobs verification.

* Add `slot` number to file structure for easy navigation over stream of compressed objects.

* Change database filename.

* Fix structure size.

* Add more consistency properties.

* Fix checkRepair() issues.

* Preparation to state rebuild process.

* Add plain & compressed size.

* Debugging snappy encode process.

* Add one more debugging line.

* Dump blocks.

* One more filedump.

* Fix chunk corruption code.

* Fix detection issue.

* Some fixes in state rebuilding process.

* Add more clearance steps.

* Move updateHead() back to block_processor.

* Fix compilation issues.

* Make code more async friendly.

* Fix async issues.
Add more information when proposer verification failed.

* Fix 8192 slots issue.

* Fix Future double completion issue.

* Pass updateFlags to some of the core procedures.

* Fix tests.

* Improve initial sync handling mechanism.

* Fix checkStateTransition() performance improvements.

* Add some performance tuning and meters.

* Light client performance tuning.

* Remove debugging statement.

* Use single file descriptor for blockchain file.

* Attempt to fix LC.

* Fix timeleft calculation when untrusted sync backfilling started right after LC block received.

* Workaround for `chronicles` + `results` `error` issue.
Remove some compilation warnings.
Fix `CatchableError` leaks on Windows.

* Address review comments.

* Address review comments part 2.

* Address review comments part 1.

* Rebase and fix the issues.

* Address review comments part 3.

* Add tests and fix some issues in auto-repair mechanism.

* Add tests to all_tests.

* Rename binary test file to pass restrictions.

* Add `bin` extension to excluded list.
Recover binary test data.

* Rename fixture file to .bin again.

* Update AllTests.

* Address review comments part 4.

* Address review comments part 5 and fix tests.

* Address review comments part 6.

* Eliminate foldl and combine from blobs processing.
Add some tests to ensure that checkResponse() also checks for correct order.

* Fix forgotten place.

* Post rebase fixes.

* Add unique slots tests.

* Optimize updateHead() code.

* Add forgotten changes.

* Address review comments on state as argument.
2024-10-30 05:38:53 +00:00
tersec 03fe86f448
devel-branch-compatibility tweaks for Nim in test_validator_client (#6686) 2024-10-30 05:21:20 +01:00
tersec a39dd8e874
remove remaining references to logtrace (#6684) 2024-10-29 11:02:44 +01:00
Jacek Sieka 6973c38de2
bump mdbook version req
* use same versions as style guide
* remove obsoletion notice from summary - this is not supported by
mdbook: https://github.com/rust-lang/mdBook/issues/1766
2024-10-29 08:57:11 +01:00
Jacek Sieka 012ff59ff1
eth1 docs (#6680)
Minimal quickstart for setting up the execution client

* bump mkdocs to resolve a strange cython incompatibility
2024-10-29 08:39:18 +01:00
tersec 00f276eacf
Merge remote-tracking branch 'origin/stable' into unstable 2024-10-29 07:29:04 +00:00
andri lim 4394e5eb1e
Bump nim-web3 and nim-serialization (#6676)
- No need to use derefType anymore when using useDefault***In #174 in nim-web3
  - dependency: nim-serialization
- Fix subscription handler #175 in nim-web3
2024-10-29 01:19:09 +00:00
Jacek Sieka d71309b943
json_rpc: bump (#6659)
* json_rpc: bump

* refactored error handling, logging and other fixes
* crash fix on overlong array in json

* bump
2024-10-29 00:40:26 +00:00
tersec c4037d6361
version v24.10.0 2024-10-29 00:27:43 +00:00
tersec f85521230f
stop checking for irrelevant unsigned wraparound in Electra deposit codepath (#6683) 2024-10-28 17:18:01 +01:00
tersec 4565c029e1
make Eth2Digest isZero 8x faster (#6682)
* make Eth2Digest isZero 8x faster

* use staticFor and copyMem

* branchless with or
2024-10-28 05:21:18 +00:00
tersec 58a34e00a1
fix inconsistent aggregation bits len in Electra (#6679) 2024-10-25 16:04:35 +02:00
tersec 8a6eab7af8
unhide ObservableStores and hide ImplicitTemplateRedefinition warnings (#6678) 2024-10-24 12:40:56 +02:00
Jacek Sieka 40f2acc93c
json-ser (#6674)
bump for overlong array crash fix
2024-10-23 17:12:42 +02:00
Eugene Kabanov a3d4a3ee5e
BN: Fix el_manager timeouts issue in block processing. (#6665)
* Fix el_manager + block_processor NEWPAYLOAD_TIMEOUT timeouts issue.
Use predefined array of exponential timeouts when all the requests to EL has been failed.

* Increase timeout value to (next_slot.start_time - 1.second)

* Address review comments.

* Do not repeat requests when node is optimistically synced.
2024-10-23 00:42:46 +00:00
Etan Kissling ead72deaa2
Bump nim-libbacktrace to `31d8d5c170441f11a730582ef353e506a3780b7a` (#6671)
- Replace `apt-fast` with `apt-get`
- Bump libbacktrace to `531aec7c52b66cd750a28a698f3c060f279b18b0`
2024-10-22 18:24:32 +02:00
Etan Kissling 117fecad99
Bump nim-web3 to `c76ddefea96e7763c3cc5b28ec5cbac1adefc4e7` (#6670)
- Fix AuthorizationObject fields name to v, r, s
2024-10-22 18:13:36 +02:00
andri lim 5d940b4c1e
Bump nim-eth to 66297c5c0a8c22ec3f16c899e902d79aa00df575 (#6669) 2024-10-22 14:15:33 +02:00
Agnish Ghosh 250a80eb0c
add gossip validation for dc, and data column quarantine strategy (#6581)
* add gossip validation for dc

* review 1

* rm callback

* review 2

* added custody columns as a global entity

* alpha 8

* few typosA
2024-10-22 05:19:34 +00:00
Etan Kissling 3cb7b9140a
Remove request details from LC (#6666)
Requests are no longer exposed by execution data, remove from LC API.
2024-10-22 03:41:01 +00:00
Etan Kissling 6f6de8d9dc
Implement latest EIP-7685 `requestsHash` method (#6667)
It's now this weird one-off hashing method so that EL can partially
forget in some modules what the data schema is.
2024-10-21 17:09:26 +02:00
tersec 7e5338deb3
disable optimistic Electra EL block hash verification (#6664) 2024-10-21 05:09:33 +02:00
Caleb Omoniyi eb993ffcee
Fix comment to reflect correct validator balance count transition (#6663) 2024-10-21 00:40:14 +00:00
tersec 84eaa57f82
don't use {.noinit.} with case objects (#6662) 2024-10-21 01:22:16 +02:00
tersec 2d3614ff69
update to devnet-4-compatible engine API (#6657) 2024-10-17 16:53:37 +02:00
tersec 6df8f92092
bump nimbus-build-system to always bundle checksums (#6656) 2024-10-17 05:37:52 +00:00
tersec f7ae15e107
rm experimental Yamux support (#6636) 2024-10-17 00:23:13 +00:00
Etan Kissling bd94c20325
fix migration of `blob_gas_used`/`excess_blob_gas` from Deneb to Electra (#6655)
`blob_gas_used` and `excess_blob_gas` were reset on fork transition
from Deneb to Electra, which is not according to spec. Fix it.
2024-10-17 00:15:34 +00:00
Etan Kissling 8f1eaade8c
bump nim-ssz-serialization to `3397088499cefc2e7aa5e73a56ad2d5b0f091735` (#6650)
- fix some compilation hints/warnings
- Replace `apt-fast` with `apt-get`
2024-10-16 21:55:33 +00:00
Etan Kissling 8061b6f9d2
bump nim-taskpools to `66585e2e960b7695e48ea60377fb3aeac96406e8` (#6651)
- update ci.yml to test Nim 2.2; also test gcc-14
2024-10-16 21:54:56 +00:00
Etan Kissling 14684394dd
bump nim-testutils to `4d37244f9f5e1acd8592a4ceb5c3fc47bc160181` (#6652)
- v0.5.3
2024-10-16 21:54:48 +00:00
Etan Kissling 310585855d
bump nim-toml-serialization to `fea85b27f0badcf617033ca1bc05444b5fd8aa7a` (#6653)
- v0.2.14
2024-10-16 21:54:36 +00:00
Etan Kissling c6cece359a
bump nimcrypto to `v0.6.2` (#6654)
- Improve burmMem / equalMemFull performance
- Bump version to 0.6.2
2024-10-16 21:54:29 +00:00
Etan Kissling b41c9a4561
bump nim-serialization to `08055d72f3ce0b29e799c191713e0ca2e903fe71` (#6649)
- v0.2.4
- Fix deprecated fields reader usage in generic_suite
2024-10-16 19:44:22 +00:00
Etan Kissling 7a486ff1f2
bump nim-json-serialization to `96fcb658b4d5ef81515b85bafdfffcc1eaf2cdf4` (#6648)
- fix stylecheck for BiggestUInt
- v0.2.8
- Fix import stew/results to pkg/results
2024-10-16 19:07:28 +00:00
Etan Kissling 0387342922
bump nim-json-rpc to `0408795be95c00d75e96eaef6eae8a9c734014f5` (#6647)
- fix stylecheck for UInt256
- v0.4.4
2024-10-16 18:36:56 +00:00
Eugene Kabanov fcf72a6e8e
VC: Electra fixes. (#6631)
* Initial commit.

* Add aggregated attestation processing.

* Add missing presets file.

* Fix compilation error.

* Fix post-rebase compilation error.

* Satisfy push raises requirement.

* Fix sync committee duties retrieval process.

* Fix forks configuration management.

* Fix deposits to use new fork configuration scheme.

* Fix /eth/v2/validator/aggregate_attestation implementation.

* Fix RANDAO preparation loop to handle blocks at epoch boundary properly.

* Simplification of RANDAO fix.

* Fix typo.

* Address review comments and fix tests.

* Fix incorrect status codes in REST test.

* Rework attestation and aggregated attestations processing code.

* Address review comments.

* Fill committee_index in RegisteredAttestation construction code.

* Address review comments part 2.

* Address review comments part 3.

* use Deneb fork epoch

* Add transition from Deneb to Electra into CI finalization test.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2024-10-16 17:20:39 +00:00
Etan Kissling ec0accf290
bump dependencies: Replace `apt-fast` with `apt-get` (#6646)
- nim-bearssl to `953a795b96d73a7401e7a1041587da9ffe6d78f6`
- nim-blscurve to `de2d3c79264bba18dbea469c8c5c4b3bb3c8bc55`
- nim-chronicles to `81a4a7a360c78be9c80c8f735c76b6d4a1517304`
- nim-confutils to `9bf293e5d29294848a9d2d79363d971d2abe8254`
- nim-faststreams to `c246d00eaa7d6f52019464b37da510a8be23e939`
- nim-http-utils to `8bb1acbaa4b86eb866145b0d468eff64a57d1897`
- nim-kzg4844 to `abef457e17c56901fd038bb1a7e11dfd5a4f919a`
- nim-metrics to `cacfdc12454a0804c65112b9f4f50d1375208dcd`
- nim-minilru to `c35304151ea39077330f225e3837450990d55e48`
- nim-nat-traversal to `5e4059746e9095e1731b02eeaecd62a70fbe664d`
- nim-presto to `bd410a26e0da62fdcf56d2be17bd53d84868a457`
- nim-secp256k1 to `0706e2c350266017dcd663ff2727eb2b03d2fa44`
- nim-stew to `a6e198132097fb544d04959aeb3b839e1408f942`
- nim-stint to `3236fa68394f1e3a06e2bc34218aacdd2d675923`
- nim-testutils to `30c03da87d501db3289f8598f7235e3af79952d6`
- nim-zlib to `91cf360b1aeb2e0c753ff8bac6de22a41c5ed8cd`
- nim-zxcvbn to `f31e4bd8fef867799bc42e47b073925a281349f3`
2024-10-15 21:47:40 +00:00
Etan Kissling c4be5f677a
bump nim-eth to `171531fbf1a308c1bf4d2f6b88c6ee6924657437` (#6645)
- rm Goerli bootnodes
- update Sepolia and Holesky bootnodes
- Fix Authorization fields name to v, r, s
- Add helper for recovering authority
2024-10-15 21:47:17 +00:00
Etan Kissling 03369d9c07
Replace `apt-fast` with `apt-get` (#6644)
`apt-fast` was removed from GitHub with Ubuntu 24.04:

- https://github.com/actions/runner-images/issues/10003

For compatibility, switch back to `apt-get`.
2024-10-15 21:45:28 +00:00
Eugene Kabanov 0ecd2798e8
Deprecate logtrace tool. (#6643)
* Deprecate logtrace tool.

* Eliminate mentions of logtrace in Makefile and scripts.
2024-10-15 13:54:07 +00:00
tersec acad28d05b
avoid combining multiple aggregates from same committee (#6642) 2024-10-15 06:02:36 +02:00
Miran 585deb8f90
use types from `eth/common/eth_types` (#6614)
* bump nim-eth

* use types from `eth/common/eth_types`
2024-10-14 14:55:56 +00:00
tersec c01cf6601a
attestation and aggregated attestation BN fixes for Electra (#6641)
* attestation and aggregated attestation BN fixes for Electra

* disable electra transition for this PR

* use cleaned-up verification function everywhere
2024-10-14 14:20:26 +00:00
tersec e9e0149d55
use Geth v1.14.11 (#6640) 2024-10-12 06:40:32 +02:00
tersec bc51ba3077
styleCheck tweaks (#6638) 2024-10-12 00:51:28 +02:00
tersec 5608a803d7
update beacon_validators to be more Nim 2.2-compatible (#6637) 2024-10-11 15:39:45 +02:00
tersec b4efaaa573
test with macOS ARM64 (#6635)
* test with macOS ARM64

* assign platform variable
2024-10-11 11:08:46 +02:00
tersec 181f5f984d
use Deneb genesis for local testnet; add Electra forking support to local tesnet (#6634)
* use Deneb genesis for local testnet; add Electra forking support to local tesnet

* deneb -> electra in ncli_testnet help
2024-10-10 15:48:05 +00:00