Commit Graph

287 Commits

Author SHA1 Message Date
tersec 2246a6ec95
Revert "include small dedup in block processor to handle blockByRoot blocks (#4814)" (#4840)
This reverts commit 8b3ffec0d5.

Syncing was broken with this: https://github.com/status-im/infra-nimbus/issues/132#issuecomment-1514465481
2023-04-19 19:16:27 +00:00
tersec 228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
tersec 8b3ffec0d5
include small dedup in block processor to handle blockByRoot blocks (#4814) 2023-04-17 19:36:15 +00:00
henridf 176c80a3d5
Fix incorrect naming and use of Quarantine.peekBlobless (#4826)
The 'peek' name was incorrect as it was actually removing from the
table. It was consequently used incorrectly in block processing: the
blobless block wasn't returned to the table when it should be.
2023-04-17 18:10:13 +02:00
tersec 75be7d267d
always use fcUV2 in shapella even for non-proposer fcUs (#4817)
* always use fcUV2 in shapella even for non-proposer fcUs

* avoid template/proc naming conflict with libp2p/signed_envelope.nim having a payload proc
2023-04-17 16:17:52 +02:00
henridf 4df851f4f5
Fix incorrect handling of blobless blocks (#4822)
We were passing along all blocks to local processing, including those
for which we were missing blobs. This commit fixes that.
2023-04-17 10:54:04 +00:00
henridf 29b431e312
Simplify block quarantine blobless (#4824)
* Simplify block quarantine blobless

The quarantine blobless table was initially keyed off of (Eth2Digest,
ValidatorSig). This was modelled off the orphan table. The presence of
the signature in the key is necessary for orphans, because we can't
verify the signature for an orphan. That is not the case for a
blobless block, where the signature can be verified.

So this PR changes the blobless block table to be keyed off a
Eth2Digest only. This simplifies the retrieval and handling of
blobless blocks.

* review feedback
2023-04-16 08:37:56 +00:00
henridf 57623af36a
Remove unnecessary field derefs in BlockProcessor.storeBlock (#4823) 2023-04-16 01:25:17 +00:00
henridf 021de18e06
Quarantine and reassembly of gossiped blobs and blocks (#4808) 2023-04-13 19:11:40 +00:00
tersec cd7da00d16
eliminate fcU/getPayload race condition causing missed proposals (#4800) 2023-04-12 12:33:21 +03:00
Etan Kissling c3d043c0e1
rename `loadExecutionBlockRoot` > `loadExecutionBlockHash` (#4807)
There are still some `executionBlockRoot` after this, separate rename.
2023-04-11 16:56:29 +00:00
Etan Kissling b7d08d0a38
do not report pre-Merge sync progress as `/opt` (#4801)
Before the merge, assume `payloadStatus == NewPaylodStatus.valid` to
avoid cases of sync progress being reported with `/opt` suffix.
2023-04-09 14:58:20 +00:00
henridf 04302081b4
Add blob sidecar gossip validation (#4785)
* Add blob gossip validation

* lint

* Add test for getBlobSidecarTopic

* Fix closure variable capture issue

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-04-04 15:12:34 +00:00
henridf 635a924e8c
Add nim-kzg4844 and use it in validate_blobs (#4732) 2023-03-23 09:47:04 +00:00
Tanguy 769ed00203
Add light client gossip metrics (#4745) 2023-03-21 08:55:48 +01:00
tersec 2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00
Etan Kissling 69013d153c
bump light client spec references to `v1.3.0-rc.3` (#4719) 2023-03-11 01:11:51 +00:00
Etan Kissling ad118cd354
rename `stateFork` > `consensusFork` (#4718)
Just the variable, not yet `lcDataForkAtStateFork` / `atStateFork`.

- Shorten comment in `light_client.nim` to keep line width
- Do not rename `stateFork` mention in `runProposalForkchoiceUpdated`.
- Do not rename `stateFork` in `getStateField(dag.headState, fork)`

Rest is just a mechanical mass replace
2023-03-11 00:35:52 +00:00
henridf f5612f2a77
Remove BlobsSidecar used in BeaconChainDB (#4710) 2023-03-10 12:51:36 +00:00
tersec a47f0b054e
finish eip4844 to deneb module rename (#4705) 2023-03-09 01:34:17 +01:00
henridf 90640cce05
Update sync to use post-decoupling RPC (#4701)
* Update sync to use post-decoupling RPCs

blob_sidecars_by_range returns a flat list of sidecars, which must
then be grouped per-slot.

* Add test for groupBlobs

* createBlobs: convert proc to func
2023-03-07 20:19:17 +00:00
tersec 63b1b0840f
5 more modules of eip4844.foo to deneb.foo renames (#4698) 2023-03-06 18:45:52 +00:00
tersec 8541674498
simplify ELMonitor fcU payload attributes handling (#4696) 2023-03-06 16:19:15 +00:00
zah 8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
tersec 3b41e6a0e7
rename ConsensusFork.EIP4844 to ConsensusFork.Deneb (#4692) 2023-03-04 13:35:39 +00:00
tersec d058aa09c8
more withdrowls (#4674) 2023-03-02 17:13:35 +01:00
henridf 1de3cf5246
Remove SignedBeaconBlockAndBlobsSidecar (#4683)
This commit removes SignedBeaconBlockAndBlobsSidecar and all remaining
references.
2023-03-02 15:12:04 +01:00
tersec 88092bb411
don't try to validate execution block hashes of non-execution payloads (#4687) 2023-03-02 00:11:46 +00:00
henridf 3681177cf4
Remove ForkySignedBeaconBlockMaybeBlobs (#4681)
This commit removes ForkySignedBeaconBlockMaybeBlobs and all
references. I tried to pull that thread only as little as was needed
to get rid of it. Left a placeholder BlobSidecar array (in lieu of
Opt[BlobsSidecar]) in a few places; this will be used as we rebuild
the decoupled implementation.
2023-02-28 11:36:17 +00:00
henridf dede36fe86
Remove blobsSidecar from orphans table (#4670) 2023-02-27 06:10:22 +00:00
tersec 29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
Jacek Sieka 83f9745df1
restore doppelganger check on connectivity loss (#4616)
* restore doppelganger check on connectivity loss

https://github.com/status-im/nimbus-eth2/pull/4398 introduced a
regression in functionality where doppelganger detection would not be
rerun during connectivity loss. This PR reintroduces this check and
makes some adjustments to the implementation to simplify the code flow
for both BN and VC.

* track when check was last performed for each validator (to deal with
late-added validators)
* track when we performed a doppel-detectable activity (attesting) so as
to avoid false positives
* remove nodeStart special case (this should be treated the same as
adding a validator dynamically just after startup)

* allow sync committee duties in doppelganger period

* don't trigger doppelganger when registering duties

* fix crash when expected index response is missing

* fix missing slashingSafe propagation
2023-02-20 13:28:56 +02:00
tersec 629b005c27
refactor batch validation not to require genesis_validators_root each time (#4640) 2023-02-20 09:26:22 +01:00
tersec a382498cfe
batch-verify BLS to execution change messages (#4637) 2023-02-17 13:35:12 +00:00
tersec cf551f10c4
don't fcU on blocks for which block processor received no newPayload reply (#4623) 2023-02-14 21:41:49 +01:00
tersec 3011d49946
refactor fcU sending and rename EL-side root to hash (#4614) 2023-02-14 07:48:39 +01:00
tersec aee19fec6b
block on forkchoiceUpdated EL calls due to doing fewer of them (#4609) 2023-02-13 12:13:52 +01:00
henridf 59e41dc65d
EIP4844 sync (#4581)
* EIP4844 Sync

* Pass eip4844 fork epoch rather than cfg to syncmanager

* Fix sync

* Update test

* map->mapIt
2023-02-11 20:48:35 +00:00
Jacek Sieka f3ddea6c86
Skip execution payload verification for finalized blocks (#4591)
While syncing the finalized portion of the chain, the execution client
cannot efficiently sync and most of the time returns `SYNCING` - in this
PR, we use CL-verified optmistic sync as long as the block is claimed to
be finalized, only occasionally updating the EL with progress.

Although a peer might lie about what is finalized and what isn't,
eventually we'll call the execution client - thus, all a dishonest
client can do is delay execution verification slightly. Gossip blocks in
particular are never assumed to be finalized.
2023-02-06 08:22:08 +01:00
tersec bca781b1b0
prioritize REST API-provided BLS to execution changes over gossip-received changes (#4580) 2023-02-03 16:28:28 +01:00
tersec 63ed5885ab
update engine API URLs to v1.0.0-beta.2 (#4579) 2023-02-01 18:49:36 +00:00
henridf 94837caa2a
Eip4844 sync fixes (#4577)
* fixes

* Make some log messages blob-aware

* remove redundant optBlobs()
2023-02-01 14:14:50 +00:00
tersec 58ed9308d2
automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4568) 2023-01-31 00:26:57 +01:00
tersec 0fb726c420
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560)
* `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork`

* revert unrelated change

* revert unrelated changes

* update test summaries
2023-01-28 19:53:41 +00:00
henridf 03b468c537
Pass correct block root to validate_blobs_sidecar (#4557) 2023-01-26 09:25:37 +00:00
henridf 7966ab6be2
Some EIP4844 fixes (#4549)
* debug log upon sidecar validation failure

* Fill in signature catch upon SignedBeaconBlockAndBlobsSidecar deser

* Always fill blobssidecar slot and root

* Skip lastFCU when eth1monitor is nil

* fix

* Use cached root
2023-01-25 18:35:46 +01:00
Etan Kissling efbd4e395a
avoid sending redundant LC finality updates (#4546)
When the epoch boundary block is missed, we incorrectly assume that the
next couple blocks improve finality, leading to repeated pushes of the
same light client finality update and incorrectly ignoring some gossip.
2023-01-24 17:44:55 +00:00
tersec fe1a57c220
use shortLog for execution payload logging (#4544) 2023-01-24 13:19:38 +00:00
Zahary Karadjov 285eec6512
Add metrics and debug logging for dropped BLS to execution change messages 2023-01-23 14:58:40 +01:00