Commit Graph

322 Commits

Author SHA1 Message Date
Zahary Karadjov 05fb7ffff0
Don't require getopt to be installed by brew on macOS for Nix users
Other changes:

* Add/Update copyright banners
* Add post-review comment for https://github.com/status-im/nimbus-eth2/pull/5719
2024-01-13 12:37:41 +02:00
Jacek Sieka b98f46c04d
Avoid global in p2p macro (fixes #4578) (#5719)
* Avoid global in p2p macro (fixes #4578)

* copy p2p macro to this repo and start de-crufting it
* make protocol registration dynamic, removing light client hacks et al
* split out light client protocol into its own file

* cleanups

* Option -> Opt
* remove more cruft

* further split beacon_sync

this allows the light client to respond to peer metadata messages
without exposing the block sync protocol

* better protocol init

* "constant" protocol index

* avoid casts

* copyright

* move some discovery code to discovery

* avoid extraneous data copy when sending chunks

* remove redundant forkdigest field

* document how to connect to a specific peer
2024-01-13 11:54:24 +02:00
Etan Kissling b54bbdecae
bump `gnosis-chain-configs` to `ff5fbe9aecbf28df9beedb662920eeee537629b0` (#5725)
- Add deneb config vars to Chiado config
- Schedule Deneb on Chiado
2024-01-12 11:31:07 +01:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 508f3b6368
add missing `std/` prefix to more imports (#5696)
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
Jacek Sieka 3f525acb87
chronos: bump (#5684)
This PR causes a few new warnings to appear - these are harmless but
will need addressing separately as they span several libraries.

* new asyncraises syntax
* asyncraises support in several modules
* `sink` usage reduces spurious copying
* `?` compatiblity for `async` + `results`
* remove `-d:chronosStrictException` (obsolete)
2024-01-05 10:08:38 +01:00
Etan Kissling 571193d450
bump `goerli` to `6522ac6684693740cd4ddcc2a0662e03702aa4a1` (#5687)
- schedule dencun on goerli
2024-01-02 22:39:55 +01:00
Etan Kissling af63291df4
load Goerli metadata from `goerli` repo (#5680)
The `eth2-networks` repo often receives metadata updates with a delay.
Switch to `goerli` repo to obtain the latest config (Dencun scheduling)
when it is updated. This is in line with how Sepolia / Holesky work.

- https://github.com/eth-clients/goerli/pull/178
2024-01-02 16:15:44 +01:00
tersec 9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates (#5672) 2023-12-16 03:27:06 +01:00
andri lim 15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
tersec 4776fecc33
consensus spec v1.4.0-beta.5 URL updates (#5655) 2023-12-06 22:16:55 +00:00
tersec 91029ce6d6
fix XDeclaredButNotUsed hints (#5652) 2023-12-06 17:23:45 +01:00
tersec 9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 (#5647) 2023-12-05 03:34:45 +01:00
tersec ab5343d1bc
update some consensus spec URLs to v1.4.0-beta.4 (#5631) 2023-11-27 19:56:34 +01:00
Etan Kissling 946ebe54cf
simplify `chunkMaxSize[T]` helper implementation (#5614)
There is no longer a `MAX_CHUNK_SIZE_BELLATRIX` since #5284, and we have
not used separate limits since #4558. Clean up to reflect that.
2023-11-21 23:08:31 +01:00
tersec c96163dbca
update some consensus spec URLs to v1.4.0-beta.4 (#5594) 2023-11-11 06:27:53 +01:00
tersec c7952ff778
bump nim-eth to remove ValidIpAddress and replace with IpAddress (#5587) 2023-11-10 16:58:48 +01:00
tersec 68a77926e8
rm ropsten and rinkleby chain IDs from EL manager (#5586) 2023-11-09 23:35:40 +03:00
Eugene Kabanov c279bd4474
Fix issue when peers was not disconnected on low score. (#5579) 2023-11-08 13:42:50 +00:00
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
tersec ed446b1d5a
replace ValidIpAddress with IpAddress in configurations (#5536) 2023-11-01 08:33:00 +01:00
tersec 556d5e7114
rm unused code (#5538) 2023-11-01 05:53:09 +01:00
tersec 62d59daaa7
consensus-spec URL updates to v1.4.0-beta.3 (#5541) 2023-10-30 06:44:43 +00:00
tersec 4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec a4cf203849
fix mainnet bootstrap node comment parsing; fix nimbus guide grammar (#5486) 2023-10-06 19:26:30 +00:00
Etan Kissling 24c68bbcf7
fix `ProveField` warning in `network_metadata` (#5476)
Spurious `ProveField` warning can be avoided by using `case` instead of
`if` on `metadata.genesis.kind`. Also suppress `GlobalVar` hints when
`incbinEnabled` is used, which has global `let` definitions.
2023-10-04 10:19:05 +02:00
Zahary Karadjov 77d6bc5f42
Merge stable into unstable 2023-09-28 11:32:54 +01:00
Zahary Karadjov cfa0268d89
Version 23.9.1
Changes:

* Updated network metadata for Holesky
* Use hash_tree_root instead of SHA256 when verifying the Holesky
  genesis state
2023-09-25 21:24:25 +01:00
Etan Kissling 47639ef89c
replace `Eth2NetworkMetadata` case object with `Result` (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
Eugene Kabanov caaad4777c
BN+LC+SN: Fix cancellation deprecate warnings. (#5455)
* Fix deprecation cancel() warnings in SN, BN, LC.

* Fix missing import.
2023-09-22 11:06:27 +00:00
Tanguy 4918a4e2e0
Fix direct peers (#5427)
* Fix direct peers

* Support ENRs in DP, use DP in local testnet

* fix docs

* bump libp2p
2023-09-15 18:45:55 +00:00
tersec 7a001d4896
remove 4 more missing constants in preset handling warning (#5428) 2023-09-14 12:37:35 +00:00
Zahary Karadjov 6b1ab81046
Support both snappy-encoded and pure SSZ genesis states 2023-09-12 01:52:15 +03:00
Zahary Karadjov c7db7d0f2c
Add CLI options `--genesis-state` and `--genesis-state-url` 2023-09-08 13:10:18 +03:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
tersec db6f4e8090
update some consensus-spec URLs to v1.4.0-beta.1 (#5357) 2023-08-25 15:58:44 +00:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Tanguy 36413c8f21
Bump libp2p for gossipsub improvements (#5229)
* Bump libp2p

* Add bandwidthEstimate param

* Fix typo and type

* Add config for light client

* Fix unit

* Add debug prefix

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: diegomrsantos <diegomrsantos@gmail.com>
Co-authored-by: Diego <diego@status.im>
2023-08-24 16:04:19 +02:00
Etan Kissling 2dae8ea26e
cleanup `{.raises.}` for `writeValue` (#5308)
- Remove unnecessary `Defect` references
- Remove spurious `SerializationError` references
- Remove duplicate `writeValue` template in `keystore.nim`;
  same implementation already exists a bit further above in same file.
2023-08-19 17:11:56 +02:00
Etan Kissling 8064f5bcce
explicitly mention which `Success` enum to use (#5309)
In Nim 2.0 it can no longer infer that we mean `ResponseCode.Success`
inside `eth2_network`. Explicitly refer to that type to work around.
2023-08-17 18:33:35 +00:00
tersec d171303133
update some consensus spec URLs to v1.4.0-beta.1 (#5287) 2023-08-12 10:38:06 +00:00
tersec 6cd8f3ba55
Update constants/presets for consistency with consensus-specs v1.4.0 (#5284) 2023-08-12 03:10:12 +00:00
tersec 85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
tersec 001fbd0970
Revert "Revert "correctly align genesis size (#5244)" (#5253)" (#5254)
This reverts commit 26dcee111b.
2023-08-04 20:49:12 +00:00
tersec 26dcee111b
Revert "correctly align genesis size (#5244)" (#5253)
This reverts commit 706ce01bb1.
2023-08-04 20:11:20 +00:00
Jacek Sieka 706ce01bb1
correctly align genesis size (#5244) 2023-08-04 18:08:15 +03:00
Jacek Sieka b8a32419b8
async batch verification (+40% sig verification throughput) (#5176)
* async batch verification

When batch verification is done, the main thread is blocked reducing
concurrency.

With this PR, the new thread signalling primitive in chronos is used to
offload the full batch verification process to a separate thread
allowing the main threads to continue async operations while the other
threads verify signatures.

Similar to previous behavior, the number of ongoing batch verifications
is capped to prevent runaway resource usage.

In addition to the asynchronous processing, 3 addition changes help
drive throughput:

* A loop is used for batch accumulation: this prevents a stampede of
small batches in eager mode where both the eager and the scheduled batch
runner would pick batches off the queue, prematurely picking "fresh"
batches off the queue
* An additional small wait is introduced for small batches - this helps
create slightly larger batches which make better used of the increased
concurrency
* Up to 2 batches are scheduled to the threadpool during high pressure,
reducing startup latency for the threads

Together, these changes increase attestation verification throughput
under load up to 30%.

* fixup

* Update submodules

* fix blst build issues (and a PIC warning)

* bump

---------

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2023-08-03 11:36:45 +03:00