303 Commits

Author SHA1 Message Date
Etan Kissling
40e89937c5
segregate sync committee messages by period / fork (#4953)
`SyncCommitteeMsgPool` grouped messages by their `beacon_block_root`.
This is problematic around sync committee period boundaries and forks.
Around sync committee period boundaries, members from both the current
and next sync committee may sign the same `beacon_block_root`; mixing
the signatures from both committees together is a mistake. Likewise,
around fork transitions, the `signing_root` changes, so those messages
also need to be segregated.
2023-05-17 07:55:55 +03:00
henridf
573228ffa0
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944) 2023-05-15 05:05:12 +00:00
tersec
2fcc01f516
modify newPayload failure logging (#4930) 2023-05-11 13:58:25 +03:00
henridf
be3f5b1eac
More blob tweaks/fixes from running in devnet (#4933)
* BeaconNode: don't call fetchMissingblobs with empty list

* More logging

* BlockProcessor.checkBloblessSignature: Add missing return value
2023-05-11 00:36:35 +00:00
tersec
d3929cbb45
update some beacon API spec URLs; fix some Name and DuplicateModuleImport hints (#4929) 2023-05-10 10:20:55 +00:00
tersec
e503cb4f51
make invalid execution payloads more visible (#4906) 2023-05-10 07:17:15 +00:00
Etan Kissling
5b3c211285
checkedReject > errReject for CI failure (#4909)
The `SignedContributionAndProof: invalid contribution signature` check
is sometimes hit around fork boundaries when running local testnet.
To avoid failing CI, revert this isntance to a plain `errReject` until
the underlying problem is addressed.
2023-05-09 09:44:24 +00:00
henridf
53436c2b9b
Add blob validation condition (#4902) 2023-05-06 20:09:17 +00:00
Etan Kissling
297881edb7
bump gossip validation refs to 1.3.0 spec (#4895)
Updates gossip validation spec references to v1.3.0 and fixes an
incorrect reference to "signed_aggregate_and_proof" in sync contribution
documentation.
2023-05-05 22:48:33 +02:00
Etan Kissling
ecaf6c8e92
harden gossip strictVerification in local testnets (#4880)
Fail local testnets on any gossip REJECT, instead of just asserting some
of the attestation related checks. This now also ensures that blocks,
BLS to Execution changes, blob sidecars and LC messages are checked
when running in a local testnet environment (`--verify-finalization`).

https://github.com/status-im/nimbus-eth2/pull/2904#discussion_r719603935
2023-05-02 13:06:02 +02:00
Etan Kissling
bb62fee7d5
fix gossip validation for duplicate blob sidecars (#4881)
`SignedBlobSidecar: already have blob with valid signature` is IGNORE
in spec, but was implemented as REJECT. Align with spec.
2023-05-01 08:08:39 +02:00
henridf
ea8ef5799e
Fill in missing BlobSidecar gossip check (#4863) 2023-04-26 17:33:33 +00:00
tersec
1ccb36b272
include small dedup in block processor to handle blockByRoot blocks (#4850) 2023-04-26 07:00:03 +00:00
henridf
da6169bfe9
BlockProcessor.storeBlock: write blobs to DB (#4854) 2023-04-25 13:55:35 +03:00
tersec
b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
tersec
d3400ca11b
low attestations during epoch should instafail in CI; dbg -> warn level in newPayload log (#4830)
* low attestations during epoch should instafail in CI; dbg -> warn level on newPayload log

* improve newPayload warning message when no valid EL connected

* reduce potential spam; make log spelling more consistent; use fatal/quit
2023-04-19 19:42:30 +00:00
tersec
2246a6ec95
Revert "include small dedup in block processor to handle blockByRoot blocks (#4814)" (#4840)
This reverts commit 8b3ffec0d530f3bfaa83198de098cb199f81b921.

Syncing was broken with this: https://github.com/status-im/infra-nimbus/issues/132#issuecomment-1514465481
2023-04-19 19:16:27 +00:00
tersec
228e10f1d9
update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
tersec
8b3ffec0d5
include small dedup in block processor to handle blockByRoot blocks (#4814) 2023-04-17 19:36:15 +00:00
henridf
176c80a3d5
Fix incorrect naming and use of Quarantine.peekBlobless (#4826)
The 'peek' name was incorrect as it was actually removing from the
table. It was consequently used incorrectly in block processing: the
blobless block wasn't returned to the table when it should be.
2023-04-17 18:10:13 +02:00
tersec
75be7d267d
always use fcUV2 in shapella even for non-proposer fcUs (#4817)
* always use fcUV2 in shapella even for non-proposer fcUs

* avoid template/proc naming conflict with libp2p/signed_envelope.nim having a payload proc
2023-04-17 16:17:52 +02:00
henridf
4df851f4f5
Fix incorrect handling of blobless blocks (#4822)
We were passing along all blocks to local processing, including those
for which we were missing blobs. This commit fixes that.
2023-04-17 10:54:04 +00:00
henridf
29b431e312
Simplify block quarantine blobless (#4824)
* Simplify block quarantine blobless

The quarantine blobless table was initially keyed off of (Eth2Digest,
ValidatorSig). This was modelled off the orphan table. The presence of
the signature in the key is necessary for orphans, because we can't
verify the signature for an orphan. That is not the case for a
blobless block, where the signature can be verified.

So this PR changes the blobless block table to be keyed off a
Eth2Digest only. This simplifies the retrieval and handling of
blobless blocks.

* review feedback
2023-04-16 08:37:56 +00:00
henridf
57623af36a
Remove unnecessary field derefs in BlockProcessor.storeBlock (#4823) 2023-04-16 01:25:17 +00:00
henridf
021de18e06
Quarantine and reassembly of gossiped blobs and blocks (#4808) 2023-04-13 19:11:40 +00:00
tersec
cd7da00d16
eliminate fcU/getPayload race condition causing missed proposals (#4800) 2023-04-12 12:33:21 +03:00
Etan Kissling
c3d043c0e1
rename loadExecutionBlockRoot > loadExecutionBlockHash (#4807)
There are still some `executionBlockRoot` after this, separate rename.
2023-04-11 16:56:29 +00:00
Etan Kissling
b7d08d0a38
do not report pre-Merge sync progress as /opt (#4801)
Before the merge, assume `payloadStatus == NewPaylodStatus.valid` to
avoid cases of sync progress being reported with `/opt` suffix.
2023-04-09 14:58:20 +00:00
henridf
04302081b4
Add blob sidecar gossip validation (#4785)
* Add blob gossip validation

* lint

* Add test for getBlobSidecarTopic

* Fix closure variable capture issue

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-04-04 15:12:34 +00:00
henridf
635a924e8c
Add nim-kzg4844 and use it in validate_blobs (#4732) 2023-03-23 09:47:04 +00:00
Tanguy
769ed00203
Add light client gossip metrics (#4745) 2023-03-21 08:55:48 +01:00
tersec
2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec
ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00
Etan Kissling
69013d153c
bump light client spec references to v1.3.0-rc.3 (#4719) 2023-03-11 01:11:51 +00:00
Etan Kissling
ad118cd354
rename stateFork > consensusFork (#4718)
Just the variable, not yet `lcDataForkAtStateFork` / `atStateFork`.

- Shorten comment in `light_client.nim` to keep line width
- Do not rename `stateFork` mention in `runProposalForkchoiceUpdated`.
- Do not rename `stateFork` in `getStateField(dag.headState, fork)`

Rest is just a mechanical mass replace
2023-03-11 00:35:52 +00:00
henridf
f5612f2a77
Remove BlobsSidecar used in BeaconChainDB (#4710) 2023-03-10 12:51:36 +00:00
tersec
a47f0b054e
finish eip4844 to deneb module rename (#4705) 2023-03-09 01:34:17 +01:00
henridf
90640cce05
Update sync to use post-decoupling RPC (#4701)
* Update sync to use post-decoupling RPCs

blob_sidecars_by_range returns a flat list of sidecars, which must
then be grouped per-slot.

* Add test for groupBlobs

* createBlobs: convert proc to func
2023-03-07 20:19:17 +00:00
tersec
63b1b0840f
5 more modules of eip4844.foo to deneb.foo renames (#4698) 2023-03-06 18:45:52 +00:00
tersec
8541674498
simplify ELMonitor fcU payload attributes handling (#4696) 2023-03-06 16:19:15 +00:00
zah
8771e91d53
Support for driving multiple EL nodes from a single Nimbus BN (#4465)
* Support for driving multiple EL nodes from a single Nimbus BN

Full list of changes:

* Eth1Monitor has been renamed to ELManager to match its current
  responsibilities better.

* The ELManager is no longer optional in the code (it won't have
  a nil value under any circumstances).

* The support for subscribing for headers was removed as it only
  worked with WebSockets and contributed significant complexity
  while bringing only a very minor advantage.

* The `--web3-url` parameter has been deprecated in favor of a
  new `--el` parameter. The new parameter has a reasonable default
  value and supports specifying a different JWT for each connection.
  Each connection can also be configured with a different set of
  responsibilities (e.g. download deposits, validate blocks and/or
  produce blocks). On the command-line, these properties can be
  configured through URL properties stored in the #anchor part of
  the URL. In TOML files, they come with a very natural syntax
  (althrough the URL scheme is also supported).

* The previously scattered EL-related state and logic is now moved
  to `eth1_monitor.nim` (this module will be renamed to `el_manager.nim`
  in a follow-up commit). State is assigned properly either to the
  `ELManager` or the to individual `ELConnection` objects where
  appropriate.

  The ELManager executes all Engine API requests against all attached
  EL nodes, in parallel. It compares their results and if there is a
  disagreement regarding the validity of a certain payload, this is
  detected and the beacon node is protected from publishing a block
  with a potential execution layer consensus bug in it.

  The BN provides metrics per EL node for the number of successful or
  failed requests for each type Engine API requests. If an EL node
  goes offline and connectivity is resoted later, we report the
  problem and the remedy in edge-triggered fashion.

* More progress towards implementing Deneb block production in the VC
  and comparing the value of blocks produced by the EL and the builder
  API.

* Adds a Makefile target for the zhejiang testnet
2023-03-05 01:40:21 +00:00
tersec
3b41e6a0e7
rename ConsensusFork.EIP4844 to ConsensusFork.Deneb (#4692) 2023-03-04 13:35:39 +00:00
tersec
d058aa09c8
more withdrowls (#4674) 2023-03-02 17:13:35 +01:00
henridf
1de3cf5246
Remove SignedBeaconBlockAndBlobsSidecar (#4683)
This commit removes SignedBeaconBlockAndBlobsSidecar and all remaining
references.
2023-03-02 15:12:04 +01:00
tersec
88092bb411
don't try to validate execution block hashes of non-execution payloads (#4687) 2023-03-02 00:11:46 +00:00
henridf
3681177cf4
Remove ForkySignedBeaconBlockMaybeBlobs (#4681)
This commit removes ForkySignedBeaconBlockMaybeBlobs and all
references. I tried to pull that thread only as little as was needed
to get rid of it. Left a placeholder BlobSidecar array (in lieu of
Opt[BlobsSidecar]) in a few places; this will be used as we rebuild
the decoupled implementation.
2023-02-28 11:36:17 +00:00
henridf
dede36fe86
Remove blobsSidecar from orphans table (#4670) 2023-02-27 06:10:22 +00:00
tersec
29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
Jacek Sieka
83f9745df1
restore doppelganger check on connectivity loss (#4616)
* restore doppelganger check on connectivity loss

https://github.com/status-im/nimbus-eth2/pull/4398 introduced a
regression in functionality where doppelganger detection would not be
rerun during connectivity loss. This PR reintroduces this check and
makes some adjustments to the implementation to simplify the code flow
for both BN and VC.

* track when check was last performed for each validator (to deal with
late-added validators)
* track when we performed a doppel-detectable activity (attesting) so as
to avoid false positives
* remove nodeStart special case (this should be treated the same as
adding a validator dynamically just after startup)

* allow sync committee duties in doppelganger period

* don't trigger doppelganger when registering duties

* fix crash when expected index response is missing

* fix missing slashingSafe propagation
2023-02-20 13:28:56 +02:00
tersec
629b005c27
refactor batch validation not to require genesis_validators_root each time (#4640) 2023-02-20 09:26:22 +01:00