Commit Graph

5977 Commits

Author SHA1 Message Date
Etan Kissling bee0342f35
fix branch check in CI (#5326)
Use explicit refs to avoid occasional issues with branch check reporting
incorrectly that commit is not an ancestor of upstream head.
2023-08-19 20:49:52 +00:00
Etan Kissling ad60ba027d
bump `nim-eth` to `894ec07d9f977a5ef9487630b71764df9163982f` (#5325)
- fix `{.raises.}` annotation to `writeValue`
2023-08-19 20:34:29 +00:00
Etan Kissling 2dae8ea26e
cleanup `{.raises.}` for `writeValue` (#5308)
- Remove unnecessary `Defect` references
- Remove spurious `SerializationError` references
- Remove duplicate `writeValue` template in `keystore.nim`;
  same implementation already exists a bit further above in same file.
2023-08-19 17:11:56 +02:00
Etan Kissling 82037fab54
bump `nim-serialization` to `4bdbc29e54fe54049950e352bb969aab97173b35` (#5322)
- add `{.raises.}` annotation to `writeValue`
2023-08-19 14:17:27 +00:00
Etan Kissling 8de71bee83
allow filtering block processor logs by topic (#5321)
Add separate log topic for `block_processor` messages.

Topic named similar to the other `_processor` modules:

- `eth2_processor` --> `gossip_eth2`
- `light_client_processor` --> `gossip_lc`
- `optimistic_processor` --> `gossip_opt`
2023-08-19 11:17:29 +00:00
Etan Kissling 8e500aa62f
allow filtering fee recipient logs by topic (#5320)
Add separate log topic for `Refreshing fee recipient` messages.
2023-08-19 11:16:30 +00:00
tersec 6fd4983f6b
EIP-4788/parentBeaconBlockRoot fixes (#5318) 2023-08-19 08:38:17 +00:00
Jakub Sokołowski 6162e735dd
launch_local_testnet: add debugging for port conflicts (#5317)
We have been seeing some port conflicts like:
```
[2023-08-15T00:31:47.625Z] Geth 0 failed to start
```
```
$ tail -n1 local-testnet-mainnet/logs/geth.?.txt
==> local-testnet-mainnet/logs/geth.0.txt <==
Fatal: Error starting protocol stack: listen tcp :6801: bind: address already in use

==> local-testnet-mainnet/logs/geth.1.txt <==
Fatal: Error starting protocol stack: listen tcp :6806: bind: address already in use

==> local-testnet-mainnet/logs/geth.2.txt <==
Fatal: Error starting protocol stack: listen tcp :6811: bind: address already in use
```
In order to debug this we'll need to add printing of some extra info
into `unstable` so feature branches include it.

Related: https://github.com/status-im/nimbus-eth2/issues/4575

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-18 22:24:40 +00:00
tersec 2d2d5bd549
fix blob sent/not sent logging (#5316) 2023-08-18 19:21:40 +00:00
Etan Kissling ff037f1375
remove unnecessary `mixin` (#5310)
Nim 2.0 gets confused with the `mixin self, id` in `validator_monitor`.
It looks like this is actually not needed, so removing it.
2023-08-18 15:55:06 +02:00
Jacek Sieka 8208419f08
run minimal test thread-free (#5315)
This increases the end-to-end testing for single-threaded mode which
otherwise might run into problems like
https://github.com/status-im/nimbus-eth2/pull/5303
2023-08-18 13:54:24 +02:00
Etan Kissling 9a4a5d0f88
ensure LC runs `exchangeTransitionConfiguration` loop (#5312)
In #4465, a regression was introduced by deleting the periodic call to
`engine_exchangeTransitionConfiguration` in the Nimbus light client.
This led to the light client never treating the EL as online and,
subsequently, not sending `engine_newPayload` requests for new blocks.
Strangely, `engine_forkchoiceUpdated` requests still make it through :)

Geth still requires both `engine_newPayload` and `fcU` to be called.
By restoring the `exchangeTransitionConfiguration` loop, `newPayload`
requests are once more issued to the EL.
2023-08-18 11:30:02 +02:00
tersec 38b1bafd87
switch Nim issue reference to still-open non-dup of issue (#5314) 2023-08-18 08:13:31 +00:00
Jacek Sieka 5b879e3bc2
blscurve: bump for cleanups (#5311) 2023-08-18 01:01:03 +00:00
Etan Kissling f70dbf1f53
use different argname than `_` (#5307)
In Nim 2.0, if an argument is named `_` it can't be used in the impl.
Rename the `_` arg in `testblockutil` to something else.
2023-08-17 19:11:37 +00:00
Etan Kissling 8064f5bcce
explicitly mention which `Success` enum to use (#5309)
In Nim 2.0 it can no longer infer that we mean `ResponseCode.Success`
inside `eth2_network`. Explicitly refer to that type to work around.
2023-08-17 18:33:35 +00:00
Etan Kissling 072ec1f0d2
explicitly indicate type of `nil` in LC db `disposeSafe` (#5305)
In Nim 2.0, compiler fails to infer the type of `nil` within the
`disposeSafe` template. Avoid that problem by explicitly indicating it.
2023-08-17 17:46:58 +00:00
Etan Kissling 13f436bc24
explicit `int` array indices for Nim 2.0 (#5306)
In Nim, when using a `distinct` capacity for an array, `int` lookups
don't work. Therefore, define the array with `int` capacity.
2023-08-17 17:26:08 +00:00
Jacek Sieka 49729e1ef3
prevent concurrent `storeBlock` calls (fixes #5285) (#5295)
When a block is introduced to the system both via REST and gossip at the
same time, we will call `storeBlock` from two locations leading to a
dupliace check race condition as we wait for the EL.

This issue may manifest in particular when using an external block
builder that itself publishes the block onto the gossip network.

* refactor enqueue flow
* simplify calling `addBlock`
* complete request manager verifier future for blobless blocks
* re-verify parent conditions before adding block

among other things, it might have gone stale or finalized between one
call and the other
2023-08-17 15:12:37 +02:00
Jacek Sieka 5d936c24e4
fix batch validation when `--num-threads=1` (#5303)
without a dedicated taskpools thread, validation will never happen!
2023-08-17 10:58:13 +02:00
tersec 0d1614326d
16-byte align ValidatorSig and TrustedSig (#5301) 2023-08-17 06:27:26 +00:00
tersec 22b7b721d0
adjust digest hashing not to care about alignment; increase ValidatorPubKey alignment (#5296) 2023-08-17 03:06:21 +00:00
Jacek Sieka a150bc93a6
blst: 0.3.11 (#5300)
```
Essential changes:

    security bugfix in Go bindings
    run-time switch in portable build
    no-std support [exercised with limited stack size of 56K]
    serde support
blst_miller_loop_n interface, with parallelized Rust and Go
counterparts
    optional blst_fr_pentaroot subroutine
    Emscripten bindings
    hardened security on non-asm platforms
```
2023-08-16 18:13:58 +00:00
Etan Kissling 1fa3941aa5
set `WITH_UBSAN` correctly when running on GA (#5299)
GitHub actions script was not setting `WITH_UBSAN` correctly. Fix it.
2023-08-16 16:56:05 +00:00
Eugene Kabanov 4a84457376
VC: Fix API timeouts issue. (#5297)
* Fix VC timeouts issue.
Add tests.

* Update AllTests.

* Remove assertion, which could crash VC when API call was interrupted by some higher logic.
2023-08-16 16:14:39 +00:00
Jacek Sieka ddddfc41aa
bearssl: bump to 0.2.1 (#5298) 2023-08-16 15:42:01 +00:00
tersec 230626306a
use fcU V3 in Deneb/Cancun (#5293) 2023-08-15 23:00:35 +00:00
tersec c3584a9ea7
fix regression in remote signing REST schema (#5294) 2023-08-15 22:39:12 +00:00
Eugene Kabanov 025b0e3f04
VC: Refactor sync committee duties and selection proofs calculation. (#5184)
* Initial commit.

* Fix issues in fill*() procedures.

* Enable sync committee processing.

* Address review comments.

* Address remaining review comments.

* Add sync committee selection proof pruning mechanism.

* Fix recursion issue.
2023-08-15 19:00:27 +02:00
Jacek Sieka b39e71335d
ssz: bump for 10% hash_tree_root perf increase, json loading fix (#5292)
* move file loading out of timed ncli section to focus benchmark on
deserialization rather than disk loading
* JSON loading fix included in bump, but does not practically happen
since we generally don't load hashlist from JSON
2023-08-15 13:28:33 +00:00
Eugene Kabanov 255131d331
Bump chronos. (#5281)
Add current/last connection query to the connection information.
2023-08-15 13:14:34 +02:00
tersec af37a96dbd
don't send fcUs every block if in lc-opt regime and block putatively finalized (#5248) 2023-08-15 09:27:56 +00:00
dependabot[bot] 6de3c49512
Bump tornado from 6.3.2 to 6.3.3 in /ncli (#5291)
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.3.2 to 6.3.3.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](https://github.com/tornadoweb/tornado/compare/v6.3.2...v6.3.3)

---
updated-dependencies:
- dependency-name: tornado
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-14 23:53:30 +00:00
Jacek Sieka f77548310f
avoid quadratic attestation queue iteration (#5288)
every attestation is processed with a new wall time so we end up
iterating over all attestations for every attestation we queue - this is
4% of cpu time on a subscribe-all-subnets node

* remove redundant zero checks - block root must be an existing block
and therefore cannot be zero
* simplify "hasn't-voted" check to root only (isZeroMemory is dubiously
implemented for objects)
2023-08-14 16:48:30 +03:00
tersec d171303133
update some consensus spec URLs to v1.4.0-beta.1 (#5287) 2023-08-12 10:38:06 +00:00
tersec 6cd8f3ba55
Update constants/presets for consistency with consensus-specs v1.4.0 (#5284) 2023-08-12 03:10:12 +00:00
henridf 9efd26c2e9
Add support for POST /eth/v2/beacon/blocks (#5214)
* Add support for POST /eth/v2/beacon/blocks

* More descriptive errors

* Address review feedback

* Return 500 (not 400) for a missing implementation case
2023-08-12 03:08:54 +00:00
henridf 9ceed40090
Prune blobs (#5283)
* Prune blobs

* Address review feedback
2023-08-11 00:15:34 +00:00
Jacek Sieka 3c9fc49411
reduce ncli/research imports (#5275)
* reduce ncli/research imports

avoids ssl, most of chronos etc, speeds up compiles

* copyright
2023-08-10 14:52:49 +02:00
Jacek Sieka a154a42205
bump ssz (small speedup) (#5282) 2023-08-09 19:40:57 +00:00
Etan Kissling 973e9e9a98
speed up LC signature verification (#5268)
We know the aggregate publickey of a fully participating sync committee.
Because participation is typically very high (>95%), it is faster to
start from that aggregate publickey and subtract the individual keys of
non-participants, than summing up all the participating pubkeys.
2023-08-09 08:50:07 +02:00
tersec 8e398001fe
bump nimcrypto to f118a5f86b04fbfd388b584d749f93bb3795f88f (#5279) 2023-08-09 06:40:03 +00:00
tersec 85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
tersec 8c3d1cd3aa
ensure `ValidatorPubKey` int-compatibly aligned to fix UB (#5277)
* ensure ValidatorPubKey int-compatibly aligned

* use sizeof(Hash) explicitly
2023-08-08 19:01:32 +00:00
tersec f8520c8b1c
use v1.4.0 stability subnets by default (#5276) 2023-08-08 19:01:04 +00:00
Etan Kissling 2b57a221de
bump `nim-blscurve` to `337ab727eb8a38a0a71725f6f01ce55563f355c0` (#5274)
- bump blst
- add sig aggregation bench
- allow subtracting from aggregate for when participation is high
- add `{.push: raises: [].}` annotation
2023-08-08 12:20:08 +00:00
tersec 18061f7991
Remove most remaining portions of Bellatrix Builder API (#5272) 2023-08-08 00:05:17 +00:00
tersec 3e2c0a64df
always use --mm:refc, even under Nim 2.0 (#5273) 2023-08-08 02:04:41 +02:00
Etan Kissling 41634f6385
avoid port 8301 on Jenkins (#5271)
Port 8301 is used by Consul: https://github.com/status-im/nimbus-eth2/issues/4575#issuecomment-1667490331

Avoid using it in the local testnet simulation when EXECUTOR_NUMBER == 2
2023-08-07 19:58:24 +00:00
Etan Kissling bb672a7e90
bump Jenkins build timeout (#5270)
With the gnosis binary, building all targets sometimes exceeds the 40m.
Bump timeout to 50m.

```
Cancelling nested steps due to timeout
Sending interrupt signal to process
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:134: /tmp/cciCtCWJ.ltrans44.ltrans.o] Error 1
make[2]: *** Waiting for unfinished jobs....
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:155: /tmp/cciCtCWJ.ltrans51.ltrans.o] Error 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[1]: *** [nimcache/release/nimbus_beacon_node_gnosis/nimbus_beacon_node_gnosis.makefile:3148: build] Error 1
make: *** [Makefile:699: gnosis-build] Error 2
```
2023-08-07 19:42:28 +00:00