Commit Graph

5201 Commits

Author SHA1 Message Date
tersec 8541674498
simplify ELMonitor fcU payload attributes handling (#4696) 2023-03-06 16:19:15 +00:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
tersec e3d96ef147
rename most eip4844Data to denebData (#4693) 2023-03-04 22:23:52 +00:00
tersec 3b41e6a0e7
rename ConsensusFork.EIP4844 to ConsensusFork.Deneb (#4692) 2023-03-04 13:35:39 +00:00
Eugene Kabanov c8b50765cf
Various fixes for VC and BN REST server. (#4673)
* Fix issue when VC unable to detect errors properly and act accordingly.
Switch all API functions used by VC to RestPlainResponse, this allows us to print errors returned by BN servers.

* Fix issue when prepareBeaconCommitteeSubnet() do not perform actions when BN is optimistically synced only.

* Fix Defect issue.

* Fix submit/publish returning `false` when operation was successful.

* Address review comments.

* Fix some client calls unable to receive `execution_optimistic` field, mark BN as OptSynced when such request has been made.

* Adjust warning levels.

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-03-03 20:20:01 +00:00
Jacek Sieka fec580c39c
doppelganger: guard against reordering (#4691) 2023-03-03 18:38:48 +01:00
Jacek Sieka 65aee9cf22
ncli_db: use fork from runtimeconfig when verifying era file (#4689)
Sepolia in particular goes through two hard forks during era 1, meaning
the phase0 fork is no longer part of the state, even though blocks that
belong to it still are phase0.
2023-03-03 13:15:11 +01:00
tersec ea060de6d4
more eip4844 -> deneb module references (#4690) 2023-03-02 21:09:24 +01:00
Jacek Sieka d9e938ef02
sqlite: bump to 3.40.1 (#4649)
Been a while since we updated

* fix Defect for 1.6 in kvstore
* bump nim-eth
2023-03-02 18:59:50 +00:00
Jacek Sieka daf8ee6c59
docs: add apt repo (#4644)
* docs: add apt repo

* Update docs/the_nimbus_book/src/keep-updated.md

Co-authored-by: tersec <tersec@users.noreply.github.com>

* more package manager integration

* fix admonition

---------

Co-authored-by: zah <zahary@status.im>
Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-03-02 17:22:07 +01:00
tersec d058aa09c8
more withdrowls (#4674) 2023-03-02 17:13:35 +01:00
Jacek Sieka 4d9eaafe9c
vc: fix missing attestations due to doppelganger in epoch 1 (#4688)
* log validator that triggers doppelganger
* move activity detection closer to where it's performed, record
aggregation as activity (since it's part of the liveness endpoint)
* vc: check for doppelgangers in epoch 0 also (so that activity in epoch
1 can happen)
* avoid some looping when processing activities
2023-03-02 16:55:45 +01:00
henridf 1de3cf5246
Remove SignedBeaconBlockAndBlobsSidecar (#4683)
This commit removes SignedBeaconBlockAndBlobsSidecar and all remaining
references.
2023-03-02 15:12:04 +01:00
tersec 88092bb411
don't try to validate execution block hashes of non-execution payloads (#4687) 2023-03-02 00:11:46 +00:00
henridf 5a9e63a029
Remove uses of beacon_block_and_blobs_sidecar topic (#4682)
* Remove use of beacon_block_and_blobs_sidecar topic

This topic goes away with decoupled blocks and blobs.

* remove use of getBeaconBlockAndBlobsSidecarTopic from test

* update nimbus_light_client.nim
2023-03-01 21:30:20 +01:00
Jakub Sokołowski 2f36f15b20
launch_local_testnet.sh: fix terminating Geth (#4684)
Also improve logging and list jobs after cleanup.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-01 18:27:55 +02:00
henridf 3681177cf4
Remove ForkySignedBeaconBlockMaybeBlobs (#4681)
This commit removes ForkySignedBeaconBlockMaybeBlobs and all
references. I tried to pull that thread only as little as was needed
to get rid of it. Left a placeholder BlobSidecar array (in lieu of
Opt[BlobsSidecar]) in a few places; this will be used as we rebuild
the decoupled implementation.
2023-02-28 11:36:17 +00:00
Jakub Sokołowski f46ed12f04
launch_local_testnet.sh: re-add targetting parent PID (#4680)
Otherwise we kill of other unrelated processes.

Fix for bug introduced most probably in:
https://github.com/status-im/nimbus-eth2/pull/4551

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-28 10:39:39 +00:00
Jakub Sokołowski f2a09802b1
launch_local_testnet.sh: print processes first
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-28 11:24:20 +01:00
Jakub Sokołowski 02437bb365
launch_local_testnet.sh: show which killed processes
First step in debugging issue most probably re-introduced by:
https://github.com/status-im/nimbus-eth2/pull/4551

Which causes the finalization tests script to kill other processes
unrelated to the given CI job.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-28 11:24:17 +01:00
Jacek Sieka c928dc981b
chronos: bump
async macro fixes and other cleanups
2023-02-28 11:10:26 +01:00
tersec fb1b7458f0
almost all remaining automated v1.3.0-rc.2 to rc.3 consensus spec URL updates (#4679) 2023-02-27 21:37:37 +00:00
tersec 8ee5e8271d
weaken overaggressive different-fork-next-slot check (#4665) 2023-02-27 21:23:00 +01:00
henridf 0a87889eaf
Add blobSidecar RPCs (#4677)
* Add blob_sidecars_by_range RPC

* Add blob_sidecars_by_root RPC
2023-02-27 21:16:59 +01:00
Kim De Mey 14f4100e1b
Fix Nimbus EL node index in local testnet script (#4671) 2023-02-27 20:39:50 +01:00
Jakub Sokołowski 60394ba114
ci: add VERBOSITY parameter to Jenkinsfile (#4676)
It will default to `V=0` since it's the first choice in the list.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-27 17:00:27 +01:00
henridf 144130e80e
Add DB support for BlobSidecar (#4675) 2023-02-27 14:02:37 +00:00
tersec adeaa9e6c4
build make all targets in debug mode on GitHub Actions CI (#4655) 2023-02-27 11:30:13 +00:00
henridf dede36fe86
Remove blobsSidecar from orphans table (#4670) 2023-02-27 06:10:22 +00:00
henridf a999da3361
Remove beacon_block_and_blobs_sidecar_by_root RPC (#4672)
Along with its use in the request manager.
2023-02-26 23:17:40 +00:00
tersec cdca07908b
no remote signer or threshold in default CI (#4667)
* no remote signer or threshold in default CI

* also no remote validators
2023-02-25 10:52:10 +00:00
tersec 73797fb911
revert that version of no-remote-signer 2023-02-25 07:17:01 +00:00
tersec 982d79f9a2
more eip4844 -> deneb changes (#4666) 2023-02-25 03:03:34 +02:00
tersec 3c542c3ae0
no remote signer in default CI 2023-02-25 00:18:34 +00:00
tersec 2dd2a782d9
disable BareExcept warnings (#4662)
* disable BareExcept warnings

* BareExcept only exists in >= 1.6.11
2023-02-24 19:50:46 +00:00
Eugene Kabanov e21742a325
Fix attester and proposer duties optimistic execution behavior. (#4657)
* Fix attester and proposer duties optimistic execution behavior.

* Address review comments.

* Address review comments more efficiently.
2023-02-23 23:13:17 +00:00
zah 7c83a7b48f
Bump NBS to fix nimbus-build-system.paths issues (#4659) 2023-02-23 23:17:23 +02:00
tersec 069d973fc6
use 'deneb' in 4 more modules (#4658) 2023-02-23 19:06:57 +01:00
Eugene Kabanov dd911a7ac6
Fix IndexError for reasons. (#4656)
* Fix IndexError for reasons.

* Deprecate specific Timeout reason.
Log exception message instead of single Timeout error.
2023-02-23 17:02:17 +01:00
tersec 8f269c92d7
rename eip48844ImplementationMissing to denebImplementationMissing (#4654) 2023-02-23 10:37:45 +00:00
tersec c6cefd8914
add success logging for being configured to use payload builder (#4653) 2023-02-23 11:46:18 +02:00
zah 6036f2e7d7
Local sim impovements (#4551)
* Local sim impovements

* Added support for running Capella and EIP-4844 simulations
  by downloading the correct version of Geth.

* Added support for using Nimbus remote signer and Web3Signer.
  Use 2 out of 3 threshold signing configuration in the mainnet
  configuration and regular remote signing in the minimal one.

* The local testnet simulation can now use a payload builder.
  This is currently not activated in CI due to lack of automated
  procedures for installing third-party relays or builders.

  You are adviced to use mergemock for now, but for most realistic
  results, we can create a simple builder based on the nimbus-eth1
  codebase that will be able to propose transactions from the regular
  network mempool.

* Start the simulation from a merged state. This would allow us
  to start removing pre-merge functionality such as the gossip
  subsciption logic. The commit also removes the merge-forcing
  hack installed after the TTD removal.

* Consolidate all the tools used in the local simulation into a
  single `ncli_testnet` binary.
2023-02-23 02:10:07 +00:00
Eugene Kabanov 08b6bb7a6b
VC: Use not-synced/opt-synced BNs. (#4635)
* Initial commit.

* Address review comments and recommendations.

* Fix too often `Execution client not in sync` messages in logs.

* Add failure reason for duties requests.

* Add more reasons to every place of ValidatorApiError.

* Address race condition issue.

* Remove `vc` argument for getFailureReason().
2023-02-23 00:11:00 +00:00
tersec 1dd07d5def
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651) 2023-02-22 14:10:00 +01:00
Jakub Sokołowski 1a0b2daa0a
beacon_chain: fix history flag help message (#4650)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-22 14:09:38 +01:00
tersec ffaa42b880
fix Deneb/EIP4844 phrasing (#4646) 2023-02-21 19:25:07 +01:00
tersec 29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
tersec 79eddcde40
capella builder API support (#4643)
* capella builder API support

* use capella EPH when appropriate

* fill in ExecutionPayload.withdrawals and sanity-check builder API withdrawals root
2023-02-21 14:21:38 +01:00
tersec 68cb9fe7b1
use consensus spec v1.3.0-rc.3 test vectors (#4645) 2023-02-21 12:58:08 +00:00
Zahary Karadjov ef619d618c
[Jenkins] Produce artifacts for the REST API test 2023-02-20 13:47:40 +02:00