7028 Commits

Author SHA1 Message Date
Etan Kissling
a2f85e4b67
bump nim-ssz-serialization to 947f39fd34f3e73dc5b9465600e84ab98e848a56 (#5327)
- remove SingleMemberUnion read/write support
- note upstream Nim tracking bug
- remove rest of SingleMemberUnion
- remove spurious `{.raises.}`
2023-08-20 01:26:13 +02:00
Etan Kissling
bee0342f35
fix branch check in CI (#5326)
Use explicit refs to avoid occasional issues with branch check reporting
incorrectly that commit is not an ancestor of upstream head.
2023-08-19 20:49:52 +00:00
Etan Kissling
ad60ba027d
bump nim-eth to 894ec07d9f977a5ef9487630b71764df9163982f (#5325)
- fix `{.raises.}` annotation to `writeValue`
2023-08-19 20:34:29 +00:00
Etan Kissling
2dae8ea26e
cleanup {.raises.} for writeValue (#5308)
- Remove unnecessary `Defect` references
- Remove spurious `SerializationError` references
- Remove duplicate `writeValue` template in `keystore.nim`;
  same implementation already exists a bit further above in same file.
2023-08-19 17:11:56 +02:00
Etan Kissling
82037fab54
bump nim-serialization to 4bdbc29e54fe54049950e352bb969aab97173b35 (#5322)
- add `{.raises.}` annotation to `writeValue`
2023-08-19 14:17:27 +00:00
Etan Kissling
8de71bee83
allow filtering block processor logs by topic (#5321)
Add separate log topic for `block_processor` messages.

Topic named similar to the other `_processor` modules:

- `eth2_processor` --> `gossip_eth2`
- `light_client_processor` --> `gossip_lc`
- `optimistic_processor` --> `gossip_opt`
2023-08-19 11:17:29 +00:00
Etan Kissling
8e500aa62f
allow filtering fee recipient logs by topic (#5320)
Add separate log topic for `Refreshing fee recipient` messages.
2023-08-19 11:16:30 +00:00
tersec
6fd4983f6b
EIP-4788/parentBeaconBlockRoot fixes (#5318) 2023-08-19 08:38:17 +00:00
6162e735dd
launch_local_testnet: add debugging for port conflicts (#5317)
We have been seeing some port conflicts like:
```
[2023-08-15T00:31:47.625Z] Geth 0 failed to start
```
```
$ tail -n1 local-testnet-mainnet/logs/geth.?.txt
==> local-testnet-mainnet/logs/geth.0.txt <==
Fatal: Error starting protocol stack: listen tcp :6801: bind: address already in use

==> local-testnet-mainnet/logs/geth.1.txt <==
Fatal: Error starting protocol stack: listen tcp :6806: bind: address already in use

==> local-testnet-mainnet/logs/geth.2.txt <==
Fatal: Error starting protocol stack: listen tcp :6811: bind: address already in use
```
In order to debug this we'll need to add printing of some extra info
into `unstable` so feature branches include it.

Related: https://github.com/status-im/nimbus-eth2/issues/4575

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-18 22:24:40 +00:00
tersec
2d2d5bd549
fix blob sent/not sent logging (#5316) 2023-08-18 19:21:40 +00:00
Etan Kissling
ff037f1375
remove unnecessary mixin (#5310)
Nim 2.0 gets confused with the `mixin self, id` in `validator_monitor`.
It looks like this is actually not needed, so removing it.
2023-08-18 15:55:06 +02:00
Jacek Sieka
8208419f08
run minimal test thread-free (#5315)
This increases the end-to-end testing for single-threaded mode which
otherwise might run into problems like
https://github.com/status-im/nimbus-eth2/pull/5303
2023-08-18 13:54:24 +02:00
Etan Kissling
9a4a5d0f88
ensure LC runs exchangeTransitionConfiguration loop (#5312)
In #4465, a regression was introduced by deleting the periodic call to
`engine_exchangeTransitionConfiguration` in the Nimbus light client.
This led to the light client never treating the EL as online and,
subsequently, not sending `engine_newPayload` requests for new blocks.
Strangely, `engine_forkchoiceUpdated` requests still make it through :)

Geth still requires both `engine_newPayload` and `fcU` to be called.
By restoring the `exchangeTransitionConfiguration` loop, `newPayload`
requests are once more issued to the EL.
2023-08-18 11:30:02 +02:00
tersec
38b1bafd87
switch Nim issue reference to still-open non-dup of issue (#5314) 2023-08-18 08:13:31 +00:00
Jacek Sieka
5b879e3bc2
blscurve: bump for cleanups (#5311) 2023-08-18 01:01:03 +00:00
Etan Kissling
f70dbf1f53
use different argname than _ (#5307)
In Nim 2.0, if an argument is named `_` it can't be used in the impl.
Rename the `_` arg in `testblockutil` to something else.
2023-08-17 19:11:37 +00:00
Etan Kissling
8064f5bcce
explicitly mention which Success enum to use (#5309)
In Nim 2.0 it can no longer infer that we mean `ResponseCode.Success`
inside `eth2_network`. Explicitly refer to that type to work around.
2023-08-17 18:33:35 +00:00
Etan Kissling
072ec1f0d2
explicitly indicate type of nil in LC db disposeSafe (#5305)
In Nim 2.0, compiler fails to infer the type of `nil` within the
`disposeSafe` template. Avoid that problem by explicitly indicating it.
2023-08-17 17:46:58 +00:00
Etan Kissling
13f436bc24
explicit int array indices for Nim 2.0 (#5306)
In Nim, when using a `distinct` capacity for an array, `int` lookups
don't work. Therefore, define the array with `int` capacity.
2023-08-17 17:26:08 +00:00
Jacek Sieka
49729e1ef3
prevent concurrent storeBlock calls (fixes #5285) (#5295)
When a block is introduced to the system both via REST and gossip at the
same time, we will call `storeBlock` from two locations leading to a
dupliace check race condition as we wait for the EL.

This issue may manifest in particular when using an external block
builder that itself publishes the block onto the gossip network.

* refactor enqueue flow
* simplify calling `addBlock`
* complete request manager verifier future for blobless blocks
* re-verify parent conditions before adding block

among other things, it might have gone stale or finalized between one
call and the other
2023-08-17 15:12:37 +02:00
Jacek Sieka
5d936c24e4
fix batch validation when --num-threads=1 (#5303)
without a dedicated taskpools thread, validation will never happen!
2023-08-17 10:58:13 +02:00
tersec
0d1614326d
16-byte align ValidatorSig and TrustedSig (#5301) 2023-08-17 06:27:26 +00:00
tersec
22b7b721d0
adjust digest hashing not to care about alignment; increase ValidatorPubKey alignment (#5296) 2023-08-17 03:06:21 +00:00
Jacek Sieka
a150bc93a6
blst: 0.3.11 (#5300)
```
Essential changes:

    security bugfix in Go bindings
    run-time switch in portable build
    no-std support [exercised with limited stack size of 56K]
    serde support
blst_miller_loop_n interface, with parallelized Rust and Go
counterparts
    optional blst_fr_pentaroot subroutine
    Emscripten bindings
    hardened security on non-asm platforms
```
2023-08-16 18:13:58 +00:00
Etan Kissling
1fa3941aa5
set WITH_UBSAN correctly when running on GA (#5299)
GitHub actions script was not setting `WITH_UBSAN` correctly. Fix it.
2023-08-16 16:56:05 +00:00
Eugene Kabanov
4a84457376
VC: Fix API timeouts issue. (#5297)
* Fix VC timeouts issue.
Add tests.

* Update AllTests.

* Remove assertion, which could crash VC when API call was interrupted by some higher logic.
2023-08-16 16:14:39 +00:00
Jacek Sieka
ddddfc41aa
bearssl: bump to 0.2.1 (#5298) 2023-08-16 15:42:01 +00:00
tersec
230626306a
use fcU V3 in Deneb/Cancun (#5293) 2023-08-15 23:00:35 +00:00
tersec
c3584a9ea7
fix regression in remote signing REST schema (#5294) 2023-08-15 22:39:12 +00:00
Eugene Kabanov
025b0e3f04
VC: Refactor sync committee duties and selection proofs calculation. (#5184)
* Initial commit.

* Fix issues in fill*() procedures.

* Enable sync committee processing.

* Address review comments.

* Address remaining review comments.

* Add sync committee selection proof pruning mechanism.

* Fix recursion issue.
2023-08-15 19:00:27 +02:00
Jacek Sieka
b39e71335d
ssz: bump for 10% hash_tree_root perf increase, json loading fix (#5292)
* move file loading out of timed ncli section to focus benchmark on
deserialization rather than disk loading
* JSON loading fix included in bump, but does not practically happen
since we generally don't load hashlist from JSON
2023-08-15 13:28:33 +00:00
Eugene Kabanov
255131d331
Bump chronos. (#5281)
Add current/last connection query to the connection information.
2023-08-15 13:14:34 +02:00
tersec
af37a96dbd
don't send fcUs every block if in lc-opt regime and block putatively finalized (#5248) 2023-08-15 09:27:56 +00:00
dependabot[bot]
6de3c49512
Bump tornado from 6.3.2 to 6.3.3 in /ncli (#5291)
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-14 23:53:30 +00:00
Jacek Sieka
f77548310f
avoid quadratic attestation queue iteration (#5288)
every attestation is processed with a new wall time so we end up
iterating over all attestations for every attestation we queue - this is
4% of cpu time on a subscribe-all-subnets node

* remove redundant zero checks - block root must be an existing block
and therefore cannot be zero
* simplify "hasn't-voted" check to root only (isZeroMemory is dubiously
implemented for objects)
2023-08-14 16:48:30 +03:00
tersec
d171303133
update some consensus spec URLs to v1.4.0-beta.1 (#5287) 2023-08-12 10:38:06 +00:00
tersec
6cd8f3ba55
Update constants/presets for consistency with consensus-specs v1.4.0 (#5284) 2023-08-12 03:10:12 +00:00
henridf
9efd26c2e9
Add support for POST /eth/v2/beacon/blocks (#5214)
* Add support for POST /eth/v2/beacon/blocks

* More descriptive errors

* Address review feedback

* Return 500 (not 400) for a missing implementation case
2023-08-12 03:08:54 +00:00
henridf
9ceed40090
Prune blobs (#5283)
* Prune blobs

* Address review feedback
2023-08-11 00:15:34 +00:00
Jacek Sieka
3c9fc49411
reduce ncli/research imports (#5275)
* reduce ncli/research imports

avoids ssl, most of chronos etc, speeds up compiles

* copyright
2023-08-10 14:52:49 +02:00
Jacek Sieka
a154a42205
bump ssz (small speedup) (#5282) 2023-08-09 19:40:57 +00:00
Etan Kissling
973e9e9a98
speed up LC signature verification (#5268)
We know the aggregate publickey of a fully participating sync committee.
Because participation is typically very high (>95%), it is faster to
start from that aggregate publickey and subtract the individual keys of
non-participants, than summing up all the participating pubkeys.
2023-08-09 08:50:07 +02:00
tersec
8e398001fe
bump nimcrypto to f118a5f86b04fbfd388b584d749f93bb3795f88f (#5279) 2023-08-09 06:40:03 +00:00
tersec
85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
tersec
8c3d1cd3aa
ensure ValidatorPubKey int-compatibly aligned to fix UB (#5277)
* ensure ValidatorPubKey int-compatibly aligned

* use sizeof(Hash) explicitly
2023-08-08 19:01:32 +00:00
tersec
f8520c8b1c
use v1.4.0 stability subnets by default (#5276) 2023-08-08 19:01:04 +00:00
Etan Kissling
2b57a221de
bump nim-blscurve to 337ab727eb8a38a0a71725f6f01ce55563f355c0 (#5274)
- bump blst
- add sig aggregation bench
- allow subtracting from aggregate for when participation is high
- add `{.push: raises: [].}` annotation
2023-08-08 12:20:08 +00:00
tersec
18061f7991
Remove most remaining portions of Bellatrix Builder API (#5272) 2023-08-08 00:05:17 +00:00
tersec
3e2c0a64df
always use --mm:refc, even under Nim 2.0 (#5273) 2023-08-08 02:04:41 +02:00
Etan Kissling
41634f6385
avoid port 8301 on Jenkins (#5271)
Port 8301 is used by Consul: https://github.com/status-im/nimbus-eth2/issues/4575#issuecomment-1667490331

Avoid using it in the local testnet simulation when EXECUTOR_NUMBER == 2
2023-08-07 19:58:24 +00:00