Commit Graph

3174 Commits

Author SHA1 Message Date
tersec 8541674498
simplify ELMonitor fcU payload attributes handling (#4696) 2023-03-06 16:19:15 +00:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
tersec e3d96ef147
rename most eip4844Data to denebData (#4693) 2023-03-04 22:23:52 +00:00
tersec 3b41e6a0e7
rename ConsensusFork.EIP4844 to ConsensusFork.Deneb (#4692) 2023-03-04 13:35:39 +00:00
Eugene Kabanov c8b50765cf
Various fixes for VC and BN REST server. (#4673)
* Fix issue when VC unable to detect errors properly and act accordingly.
Switch all API functions used by VC to RestPlainResponse, this allows us to print errors returned by BN servers.

* Fix issue when prepareBeaconCommitteeSubnet() do not perform actions when BN is optimistically synced only.

* Fix Defect issue.

* Fix submit/publish returning `false` when operation was successful.

* Address review comments.

* Fix some client calls unable to receive `execution_optimistic` field, mark BN as OptSynced when such request has been made.

* Adjust warning levels.

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-03-03 20:20:01 +00:00
Jacek Sieka fec580c39c
doppelganger: guard against reordering (#4691) 2023-03-03 18:38:48 +01:00
Jacek Sieka 65aee9cf22
ncli_db: use fork from runtimeconfig when verifying era file (#4689)
Sepolia in particular goes through two hard forks during era 1, meaning
the phase0 fork is no longer part of the state, even though blocks that
belong to it still are phase0.
2023-03-03 13:15:11 +01:00
tersec ea060de6d4
more eip4844 -> deneb module references (#4690) 2023-03-02 21:09:24 +01:00
tersec d058aa09c8
more withdrowls (#4674) 2023-03-02 17:13:35 +01:00
Jacek Sieka 4d9eaafe9c
vc: fix missing attestations due to doppelganger in epoch 1 (#4688)
* log validator that triggers doppelganger
* move activity detection closer to where it's performed, record
aggregation as activity (since it's part of the liveness endpoint)
* vc: check for doppelgangers in epoch 0 also (so that activity in epoch
1 can happen)
* avoid some looping when processing activities
2023-03-02 16:55:45 +01:00
henridf 1de3cf5246
Remove SignedBeaconBlockAndBlobsSidecar (#4683)
This commit removes SignedBeaconBlockAndBlobsSidecar and all remaining
references.
2023-03-02 15:12:04 +01:00
tersec 88092bb411
don't try to validate execution block hashes of non-execution payloads (#4687) 2023-03-02 00:11:46 +00:00
henridf 5a9e63a029
Remove uses of beacon_block_and_blobs_sidecar topic (#4682)
* Remove use of beacon_block_and_blobs_sidecar topic

This topic goes away with decoupled blocks and blobs.

* remove use of getBeaconBlockAndBlobsSidecarTopic from test

* update nimbus_light_client.nim
2023-03-01 21:30:20 +01:00
henridf 3681177cf4
Remove ForkySignedBeaconBlockMaybeBlobs (#4681)
This commit removes ForkySignedBeaconBlockMaybeBlobs and all
references. I tried to pull that thread only as little as was needed
to get rid of it. Left a placeholder BlobSidecar array (in lieu of
Opt[BlobsSidecar]) in a few places; this will be used as we rebuild
the decoupled implementation.
2023-02-28 11:36:17 +00:00
tersec fb1b7458f0
almost all remaining automated v1.3.0-rc.2 to rc.3 consensus spec URL updates (#4679) 2023-02-27 21:37:37 +00:00
tersec 8ee5e8271d
weaken overaggressive different-fork-next-slot check (#4665) 2023-02-27 21:23:00 +01:00
henridf 0a87889eaf
Add blobSidecar RPCs (#4677)
* Add blob_sidecars_by_range RPC

* Add blob_sidecars_by_root RPC
2023-02-27 21:16:59 +01:00
henridf 144130e80e
Add DB support for BlobSidecar (#4675) 2023-02-27 14:02:37 +00:00
henridf dede36fe86
Remove blobsSidecar from orphans table (#4670) 2023-02-27 06:10:22 +00:00
henridf a999da3361
Remove beacon_block_and_blobs_sidecar_by_root RPC (#4672)
Along with its use in the request manager.
2023-02-26 23:17:40 +00:00
tersec 982d79f9a2
more eip4844 -> deneb changes (#4666) 2023-02-25 03:03:34 +02:00
tersec 2dd2a782d9
disable BareExcept warnings (#4662)
* disable BareExcept warnings

* BareExcept only exists in >= 1.6.11
2023-02-24 19:50:46 +00:00
Eugene Kabanov e21742a325
Fix attester and proposer duties optimistic execution behavior. (#4657)
* Fix attester and proposer duties optimistic execution behavior.

* Address review comments.

* Address review comments more efficiently.
2023-02-23 23:13:17 +00:00
tersec 069d973fc6
use 'deneb' in 4 more modules (#4658) 2023-02-23 19:06:57 +01:00
Eugene Kabanov dd911a7ac6
Fix IndexError for reasons. (#4656)
* Fix IndexError for reasons.

* Deprecate specific Timeout reason.
Log exception message instead of single Timeout error.
2023-02-23 17:02:17 +01:00
tersec 8f269c92d7
rename eip48844ImplementationMissing to denebImplementationMissing (#4654) 2023-02-23 10:37:45 +00:00
tersec c6cefd8914
add success logging for being configured to use payload builder (#4653) 2023-02-23 11:46:18 +02:00
zah 6036f2e7d7
Local sim impovements (#4551)
* Local sim impovements

* Added support for running Capella and EIP-4844 simulations
  by downloading the correct version of Geth.

* Added support for using Nimbus remote signer and Web3Signer.
  Use 2 out of 3 threshold signing configuration in the mainnet
  configuration and regular remote signing in the minimal one.

* The local testnet simulation can now use a payload builder.
  This is currently not activated in CI due to lack of automated
  procedures for installing third-party relays or builders.

  You are adviced to use mergemock for now, but for most realistic
  results, we can create a simple builder based on the nimbus-eth1
  codebase that will be able to propose transactions from the regular
  network mempool.

* Start the simulation from a merged state. This would allow us
  to start removing pre-merge functionality such as the gossip
  subsciption logic. The commit also removes the merge-forcing
  hack installed after the TTD removal.

* Consolidate all the tools used in the local simulation into a
  single `ncli_testnet` binary.
2023-02-23 02:10:07 +00:00
Eugene Kabanov 08b6bb7a6b
VC: Use not-synced/opt-synced BNs. (#4635)
* Initial commit.

* Address review comments and recommendations.

* Fix too often `Execution client not in sync` messages in logs.

* Add failure reason for duties requests.

* Add more reasons to every place of ValidatorApiError.

* Address race condition issue.

* Remove `vc` argument for getFailureReason().
2023-02-23 00:11:00 +00:00
tersec 1dd07d5def
switch 3 modules in beacon_chain/spec/ from eip4844 to deneb (#4651) 2023-02-22 14:10:00 +01:00
Jakub Sokołowski 1a0b2daa0a
beacon_chain: fix history flag help message (#4650)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-22 14:09:38 +01:00
tersec ffaa42b880
fix Deneb/EIP4844 phrasing (#4646) 2023-02-21 19:25:07 +01:00
tersec 29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
tersec 79eddcde40
capella builder API support (#4643)
* capella builder API support

* use capella EPH when appropriate

* fill in ExecutionPayload.withdrawals and sanity-check builder API withdrawals root
2023-02-21 14:21:38 +01:00
tersec 68cb9fe7b1
use consensus spec v1.3.0-rc.3 test vectors (#4645) 2023-02-21 12:58:08 +00:00
Jacek Sieka 83f9745df1
restore doppelganger check on connectivity loss (#4616)
* restore doppelganger check on connectivity loss

https://github.com/status-im/nimbus-eth2/pull/4398 introduced a
regression in functionality where doppelganger detection would not be
rerun during connectivity loss. This PR reintroduces this check and
makes some adjustments to the implementation to simplify the code flow
for both BN and VC.

* track when check was last performed for each validator (to deal with
late-added validators)
* track when we performed a doppel-detectable activity (attesting) so as
to avoid false positives
* remove nodeStart special case (this should be treated the same as
adding a validator dynamically just after startup)

* allow sync committee duties in doppelganger period

* don't trigger doppelganger when registering duties

* fix crash when expected index response is missing

* fix missing slashingSafe propagation
2023-02-20 13:28:56 +02:00
tersec b19f7001aa
log expected_withdrawals discrepancy (#4595)
* log expected_withdrawals discrepancy

* capella-only rather than not-bellatrix

* add one more (in theory unnecessary, but) guard against get(none)
2023-02-20 09:46:37 +01:00
tersec 72797cad09
switch some beacon_chain/spec/ from eip4844.Foo to deneb.Foo (#4642) 2023-02-20 09:45:49 +01:00
tersec 629b005c27
refactor batch validation not to require genesis_validators_root each time (#4640) 2023-02-20 09:26:22 +01:00
tersec 0cc0c7e6b7
builder spec v0.3.0 URL updates (#4641)
* builder spec v0.3.0 URL updates

* remove stray non-URL/comment change
2023-02-18 01:54:30 +01:00
tersec a382498cfe
batch-verify BLS to execution change messages (#4637) 2023-02-17 13:35:12 +00:00
tersec dc0bbe3a57
rm blockForkAtEpoch and switch callers to consensusForkAtEpoch (#4634) 2023-02-16 21:16:54 +01:00
Zahary Karadjov 8202d4a84e
Version 23.2.0 2023-02-16 20:19:22 +02:00
Eugene Kabanov e91415662b
Keystore cache implementation. (#4372) 2023-02-16 19:25:48 +02:00
Jacek Sieka d63179ab57
vc: avoid npe for unknown validator (#4632)
follow-up for #4590
2023-02-16 13:27:10 +00:00
tersec e342fdd97a
rename stateForkAtEpoch to consensusForkAtEpoch (#4627) 2023-02-16 11:32:12 +02:00
zah ff464e49cf
Implement the set of gas_limit end-points in the Keymanager API (#4612)
Fixes #3946
2023-02-15 15:10:31 +00:00
zah 067ba13c52
Capella metadata for Sepolia (#4615)
Other changes:

Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as
this matches the latest spec and it's already used in the official
Sepolia config.
2023-02-15 14:44:09 +00:00
tersec b7c3037f01
consensus spec ref URL updates to v1.3.0-rc.2 (#4626) 2023-02-15 14:43:51 +00:00
Eugene Kabanov 218ea42220
VC: Fix getBlockRoot() response with execution_optimistic. (#4622)
* Fix getStateRoot() and getBlockRoot() API functions which should obtain `execution_optimistic` field.
Fix sync committee service to check `execution_optimistic` field of getBlockRoot() response.

* 2nd part.

* Remove presets usage.
2023-02-15 15:09:31 +01:00