Commit Graph

876 Commits

Author SHA1 Message Date
Zahary Karadjov 46f48269ef
Backwards compatible handling of the web3-url parameter in TOML 2023-03-14 17:50:03 +02:00
tersec 2c0762013e
bump eth2-networks to include goerli/prater capella fork epoch (#4699) 2023-03-07 05:03:47 +00:00
zah 9cd30768e7
Bugfix: historic blocks violate the latest specs for the extraData field (fixes #4695) (#4700) 2023-03-07 04:43:27 +00:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
Jacek Sieka d9e938ef02
sqlite: bump to 3.40.1 (#4649)
Been a while since we updated

* fix Defect for 1.6 in kvstore
* bump nim-eth
2023-03-02 18:59:50 +00:00
Jacek Sieka c928dc981b
chronos: bump
async macro fixes and other cleanups
2023-02-28 11:10:26 +01:00
tersec 2dd2a782d9
disable BareExcept warnings (#4662)
* disable BareExcept warnings

* BareExcept only exists in >= 1.6.11
2023-02-24 19:50:46 +00:00
zah 7c83a7b48f
Bump NBS to fix nimbus-build-system.paths issues (#4659) 2023-02-23 23:17:23 +02:00
zah 6036f2e7d7
Local sim impovements (#4551)
* Local sim impovements

* Added support for running Capella and EIP-4844 simulations
  by downloading the correct version of Geth.

* Added support for using Nimbus remote signer and Web3Signer.
  Use 2 out of 3 threshold signing configuration in the mainnet
  configuration and regular remote signing in the minimal one.

* The local testnet simulation can now use a payload builder.
  This is currently not activated in CI due to lack of automated
  procedures for installing third-party relays or builders.

  You are adviced to use mergemock for now, but for most realistic
  results, we can create a simple builder based on the nimbus-eth1
  codebase that will be able to propose transactions from the regular
  network mempool.

* Start the simulation from a merged state. This would allow us
  to start removing pre-merge functionality such as the gossip
  subsciption logic. The commit also removes the merge-forcing
  hack installed after the TTD removal.

* Consolidate all the tools used in the local simulation into a
  single `ncli_testnet` binary.
2023-02-23 02:10:07 +00:00
tersec 68cb9fe7b1
use consensus spec v1.3.0-rc.3 test vectors (#4645) 2023-02-21 12:58:08 +00:00
Jacek Sieka 73702ad150
sqlite: bump (fix warning) (#4631) 2023-02-16 12:54:02 +00:00
zah 067ba13c52
Capella metadata for Sepolia (#4615)
Other changes:

Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as
this matches the latest spec and it's already used in the official
Sepolia config.
2023-02-15 14:44:09 +00:00
Philippe Schommers db91710ee2
chore: update gnosis chain configs (#4603) 2023-02-11 18:10:10 +01:00
tersec e902962771
nim-stew bump (#4610) 2023-02-10 23:59:00 +00:00
Eugene Kabanov e51095e2f8
Fix Windows MAX_PATH constraint issue in CI. (#4576)
* Fix MAX_PATH limitation in tests.

* Fix posix issues.

* Fix compilation issue.
2023-02-10 21:59:38 +01:00
tersec 9f0fb4742a
bump nim-stint to avoid compiler deprecation warnings (#4600) 2023-02-07 16:20:24 +00:00
Etan Kissling 171a185e4d
bump `merge-testnets` (#4598)
Updates `merge-testnets` with fix for Ropsten (#4593) and with correct
`CAPELLA_FORK_VERSION`.
2023-02-07 01:00:37 +01:00
tersec f5e302f973
use 1.3.0-rc.2-hotfix consensus spec test vectors (#4566) 2023-01-30 22:54:16 +00:00
tersec 29ea42ce1a
use v1.3.0-rc.2 consensus spec test vectors (#4564) 2023-01-29 11:28:04 +00:00
tersec 90e169869c
bump nim-stew for Results field access workaround (#4541) 2023-01-21 19:40:22 +00:00
henridf f8ee0def2b
Add stubs for EIP4844 engine API calls (#4536) 2023-01-21 00:47:38 +00:00
zah a2bc10e51c
Makefile targets for capella-devnet-3 (#4521) 2023-01-18 17:19:02 +02:00
tersec eb02415f00
execution/engine withdrawals amount in uint64 gwei (#4509) 2023-01-14 17:26:57 +00:00
tersec 68e08c6f15
use v1.3.0-rc.1 consensus spec test vectors (#4505) 2023-01-14 04:20:51 +00:00
zah 0f758c5f02
Working Makefile targets for Capella devnet2 (#4494)
* Working Makefile targets for Capella devnet2

make capella-devnet-2
make clean-capella-devnet-2

You'll need to have https://github.com/tmuxinator/tmuxinator installed.
It's available as a regular package in most Linux distributions or through
Nix or Brew on macOS.

This commit also fixes the initial hang in the Eth1 monitor in the "find
TTD block" procedure through a fix to the network metadata files which
hasn't been upstreamed yet.

Other changes:

* Disabled Geth snap sync in the simulation

When all Geth nodes are configured to run with snap sync enabled, they all
start snap sync after the first forkchoiceUpdated which causes the BNs to
skip validator duties because the EL is syncing. The snap sync never completes
due to poor connectivity between the Geth nodes in the simulation.
2023-01-13 12:21:58 +02:00
Zahary Karadjov b06502bf65
Gnosis const preset 2023-01-13 04:28:29 +02:00
henridf 309f8690de
Wire up engine_newPayloadV3 (#4482)
* Wire up eip4844's newPayloadV3

* Add eip4844 test

* Update AllTests-mainnet.md and fix typo
2023-01-11 18:21:19 +00:00
tersec 787703bd0c
use v1.3.0-rc.0 consensus spec test vectors (#4472) 2023-01-09 15:15:43 +01:00
Jacek Sieka 0ba9fc4ede
History pruning (fixes #4419) (#4445)
Introduce (optional) pruning of historical data - a pruned node will
continue to answer queries for historical data up to
`MIN_EPOCHS_FOR_BLOCK_REQUESTS` epochs, or roughly 5 months, capping
typical database usage at around 60-70gb.

To enable pruning, add `--history=prune` to the command line - on the
first start, old data will be cleared (which may take a while) - after
that, data is pruned continuously.

When pruning an existing database, the database will not shrink -
instead, the freed space is recycled as the node continues to run - to
free up space, perform a trusted node sync with a fresh database.

When switching on archive mode in a pruned node, history is retained
from that point onwards.

History pruning is scheduled to be enabled by default in a future
release.

In this PR, `minimal` mode from #4419 is not implemented meaning
retention periods for states and blocks are always the same - depending
on user demand, a future PR may implement `minimal` as well.
2023-01-07 10:02:15 +00:00
tersec d1b799eb64
bump nim-web3 for correct getPayloadV2 response signature (#4471) 2023-01-06 21:06:45 +00:00
Tanguy 75d0e0d4c8
Bump libp2p (#4470) 2023-01-06 19:33:14 +00:00
Etan Kissling 2184fd2322
bump `nim-eth`, extend empty block fallback for EIP4844 (#4425)
Implements `emptyPayloadToBlockHeader` for EIP-4844.
https://github.com/status-im/nim-eth/pull/570
2022-12-20 20:00:56 +01:00
zah 07d4160e00
Migrating the deposit contract snapshot can no longer fail on start-up (#4438)
The missing piece of data that had to be obtained previously from
the configured EL client is now part of the network metadata baked
into the binary.
2022-12-19 18:19:48 +01:00
tersec d269d82b71
use v1.3.0-alpha.2 EF consensus spec test vectors (#4430) 2022-12-15 01:51:06 +00:00
Jacek Sieka f3305189f1
bump & ci fixes (#4409) 2022-12-09 17:06:29 +01:00
Jacek Sieka 55a95bca3c
bump secp256k1 (#4388)
this time taking into account arm vs x86_64
2022-12-05 16:58:05 +00:00
tersec f1584eef2f
bump nim-eth to reduce undefined behavior from Nim (#4384) 2022-12-02 08:36:42 +01:00
zah 32039e95f0
Switch to Nim 1.6.11 (#4380) 2022-12-01 08:05:47 +02:00
Zahary Karadjov 7dd4718735
Version 22.11.0 2022-12-01 07:38:54 +02:00
Jacek Sieka cd160b5650
more strict read-only database mode (#4362)
* avoid creating pre-altair backwards compatibility tables
* allow running ncli_db era export without above tables present
* drop unused pre-altair backwards compatibility tables
* run benchmark on read-ronly database
* fix running benchmark from genesis
2022-11-28 23:21:58 +00:00
Etan Kissling c941dc801f
bump `nim-eth`, extend empty block fallback for Capella (#4357)
Implements `emptyPayloadToBlockHeader` for Capella.
https://github.com/status-im/nim-eth/pull/562
2022-11-28 14:41:25 +01:00
tersec b7cd0fe4a1
Revert "bump secp256k1 (#4356)" (#4359)
This reverts commit 60e01dcf78.
2022-11-25 13:07:16 +00:00
Jacek Sieka 60e01dcf78
bump secp256k1 (#4356)
See
eb5868e069
for changes
2022-11-25 10:51:20 +00:00
Jacek Sieka 04392c236b
bumps (#4352)
* build&style fixes
* use multithreading for LTO outside of make too
* blst 0.3.10 with no significant changes
2022-11-24 20:56:02 +00:00
tersec e965d5e0e9
bump nim-web3 to get withdrawal support in Engine API call sigs/types (#4348) 2022-11-24 09:52:45 +02:00
zah d07113767d
Bugfix: The VC was producing invalid sync committee contributions (#4343)
Since the sync committee duties are no longer updated on every slot
and previously the sync committee aggregators selection proofs were
generated during the duties update, this now resulted in the client
using stale selection proofs (they must be generated at each slot).

The fix consists of moving the selection proof generation logic in
a different function which is properly executed on each slot.

Other changes:

* The logtrace tool has been enhanced with a framework for adding
  new simpler log aggregation and analysis algorithms.
  The default CI testnet simulation will now ensure that the blocks
  in the network have reasonable sync committee participation.
2022-11-24 09:46:35 +02:00
tersec 1146470f7d
use v1.3.0-alpha.1 consensus spec test vectors (#4338) 2022-11-21 08:44:49 +01:00
Jacek Sieka cc1464a935
readd bearssl/cacert 2022-11-20 08:20:23 +01:00
Jacek Sieka 64322a3d45
prater: update CONFIG_NAME (#4322) 2022-11-15 17:35:48 +00:00
Jacek Sieka 367a13df34
stew: bump (#4323)
fix Opt constants, cleanups
2022-11-11 18:07:34 +00:00