Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.
Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
The `eth2-networks` repo often receives metadata updates with a delay.
Switch to `goerli` repo to obtain the latest config (Dencun scheduling)
when it is updated. This is in line with how Sepolia / Holesky work.
- https://github.com/eth-clients/goerli/pull/178
The 'vendor/holesky/public-keys/all.txt' file is not used by Nimbus,
but sometimes fails to download due to payment issues in `@eth-clients`.
Skipping the download of that file should reduce that risk.
```
Downloading public-keys/all.txt (240 MB)
Error downloading object: public-keys/all.txt (ba977b5): Smudge error: Error downloading public-keys/all.txt (ba977b5b1da180914c934623fce187860e1b54cff939e6208533b2cb5f589e07): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
```
* bump nbs: Patch libnatpmp.a rule to support llvm-mingw in targets.mk
* bump nbs: Allow passing config to submodule initialization in `update-common`
* Add metadata for the Holesky network
* Add copyright banner to the new Nim module
* Working version
* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
It occurs sometimes that a submodule is bumped to a PR commit instead of
the corresponding canonical branch (as registered in `.gitmodules`).
Because we typically use `squash`, that PR commit can subsequently
become unreachable, randomly breaking the build of `nimbus-eth2`.
Prevent these accidents by only allowing submodule bumps to commits
on the branch registered in `.gitmodules`. On private branches, simply
update `.gitmodules` to match the personal dev branch.
* Support for driving multiple EL nodes from a single Nimbus BN
Full list of changes:
* Eth1Monitor has been renamed to ELManager to match its current
responsibilities better.
* The ELManager is no longer optional in the code (it won't have
a nil value under any circumstances).
* The support for subscribing for headers was removed as it only
worked with WebSockets and contributed significant complexity
while bringing only a very minor advantage.
* The `--web3-url` parameter has been deprecated in favor of a
new `--el` parameter. The new parameter has a reasonable default
value and supports specifying a different JWT for each connection.
Each connection can also be configured with a different set of
responsibilities (e.g. download deposits, validate blocks and/or
produce blocks). On the command-line, these properties can be
configured through URL properties stored in the #anchor part of
the URL. In TOML files, they come with a very natural syntax
(althrough the URL scheme is also supported).
* The previously scattered EL-related state and logic is now moved
to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
in a follow-up commit). State is assigned properly either to the
`ELManager` or the to individual `ELConnection` objects where
appropriate.
The ELManager executes all Engine API requests against all attached
EL nodes, in parallel. It compares their results and if there is a
disagreement regarding the validity of a certain payload, this is
detected and the beacon node is protected from publishing a block
with a potential execution layer consensus bug in it.
The BN provides metrics per EL node for the number of successful or
failed requests for each type Engine API requests. If an EL node
goes offline and connectivity is resoted later, we report the
problem and the remedy in edge-triggered fashion.
* More progress towards implementing Deneb block production in the VC
and comparing the value of blocks produced by the EL and the builder
API.
* Adds a Makefile target for the zhejiang testnet
* Working Makefile targets for Capella devnet2
make capella-devnet-2
make clean-capella-devnet-2
You'll need to have https://github.com/tmuxinator/tmuxinator installed.
It's available as a regular package in most Linux distributions or through
Nix or Brew on macOS.
This commit also fixes the initial hang in the Eth1 monitor in the "find
TTD block" procedure through a fix to the network metadata files which
hasn't been upstreamed yet.
Other changes:
* Disabled Geth snap sync in the simulation
When all Geth nodes are configured to run with snap sync enabled, they all
start snap sync after the first forkchoiceUpdated which causes the BNs to
skip validator duties because the EL is syncing. The snap sync never completes
due to poor connectivity between the Geth nodes in the simulation.
`news` has a few open issues that are not present in `nim-websock`:
1. There is a 1 second delay between each MB of sent data.
2. Cancelling an ongoing `send` makes the entire WebSocket unusable.
3. Control packets do not have priority over ongoing message frames.
Using `news`, there are quite a few of these messages in Geth:
```
Previously seen beacon client is offline. Please ensure it is
operational to follow the chain!
```
It may take quite some time to reconnect when this happens.
Using `nim-websock`, this message still occurs because `eth1_monitor`
reconnects the EL connection when no new blocks occurred for 5 minutes,
but reconnecting is quick and the message is rarer.
rocksdb was never actually used in nimbus-eth2 and existed only to satisfy nim-eth dependencies for test running - these have since moved to nimbus-eth1.
* bump nim-eth
* Add parallel attestation verification
* Update tests, batchVerify doesn't use the threadpool with only single core (nim-blscurve update)
* bump nim-blscurve
* Debug info for failing eth2 test vectors
* remove submodule eth2-testnets
* verbose debugging of make failure on Windows (libbacktrace?)
* Remove CI debug mode
* initialization convention
* Fix new altair tests
This commit introduces `unittest2`, driven by the need to fix several
long-standing issues with `unittest`, in particular exception handling,
test isolation, timing and others.
Upgrading to `unittest2` generally requires updating the unittest import
and potentially fixing the code to support the new `one-proc-per-test`
setup.
* nim-unittest2 has been moved from Stefans repository to status-im -
https://github.com/status-im/nim-unittest2/pull/2 fixes some exception
handling issues and brings it in line with latest nim devel - a notable
difference to unittest is that each test is run inside a proc which
gives it a fresh stack and removes a lot of globals, meaning tests
should run under conditions more similar to "normal" code and interfere
with each other less - this is a crucial first step to improving the
testing experience in general, that stefan implemented for the
multithreading support
* the multithreading in unittest2 is optional, and disabled by default
* nim-testutils contains utilities for fuzzing, as well as other
"large:ish" features that have dependencies -
https://github.com/status-im/nim-testutils/pull/27 pulls in unittest2 so
unittest2 can be kept "low-dependency"
* nim-chronos needs the above unittest changes to test its own new,
stricter exception handling -
https://github.com/status-im/nim-chronos/pull/166 - the strictess
introduced is expected to increase with time
Etan Kissling
Jacek Sieka
diegomrsantos
Tanguy
Eugene Kabanov
zah
tersec
henridf
zah
Ștefan Talpalaru
Fredrik Svantes
kdeme
Mamy Ratsimbazafy