Commit Graph

498 Commits

Author SHA1 Message Date
Agnish Ghosh b32205de7c
upstream peerdas alpha3 related spec changes + fix upstream related issues (#6468)
* reworked some of the das core specs, pr'd to check whether whether the conflicting type issue is centric to my machine or not

* bumped nim-blscurve to 9c6e80c6109133c0af3025654f5a8820282cff05, same as unstable

* bumped nim-eth2-scenarios, nim-nat-traversal at par with unstable, added more pathches, made peerdas devnet branch backward compatible, peerdas passing new ssz tests as per alpha3, disabled electra fixture tests, as branch hasn't been rebased for a while

* refactor test fixture files

* rm: serializeDataColumn

* refactor: took data columns extracted from blobs during block proposal to the heap

* disable blob broadcast in pd devnet

* fix addBlock in message router

* fix: data column iterator

* added debug checkpoints to check CI

* refactor if else conditions

* add: updated das core specs to alpha 3, and unit tests pass
2024-08-05 19:27:39 +05:30
Agnish Ghosh e80bd36f76 add: verify data column kzg proof during storeBlock, added serializeDataColumn for better logs 2024-07-26 14:19:44 +05:30
Agnish Ghosh 7a891f13d6
rework on cell and proof 2024-07-07 04:35:06 +05:30
Agnish Ghosh 85db9ca99c
regressive fix 2024-07-06 00:33:15 +05:30
Agnish Ghosh fab427d65f
enable dc in gossip and message router 2024-07-05 14:34:51 +05:30
Agnish Ghosh 6cdc6bfa32
debug: verify data column kzg proofs via kurtosis 2024-07-04 20:24:11 +05:30
Agnish Ghosh 887a44a82f
revert gossip val 2024-07-03 21:59:54 +05:30
Agnish Ghosh 53f7175cd8
minor fix 2024-07-03 21:54:40 +05:30
Agnish Ghosh 9f42196c75
exp disable of some gossip conditions 2024-07-03 21:51:59 +05:30
Agnish Ghosh 152d276d78
added reconstruction logic 2024-07-03 21:50:52 +05:30
Agnish Ghosh d0722cd465
update constants 2024-07-02 20:33:09 +05:30
Agnish Ghosh d8e1beff45
debug 4 2024-07-02 19:56:41 +05:30
Agnish Ghosh 8d2c4893d8
debug3 2024-07-02 19:28:23 +05:30
Agnish Ghosh 77cc2ef358
debug2 2024-07-02 18:35:58 +05:30
Agnish Ghosh 75c3e0b6d3
debug 2024-07-02 18:05:39 +05:30
Agnish Ghosh 2f7a3d0ff2
reenable checkpoints to debug exception 2024-07-02 17:15:08 +05:30
Agnish Ghosh 67fe8aca0b
disable blob activity (exp), improve gossip validation 2024-07-02 14:36:44 +05:30
Agnish Ghosh 9e6cad4105
bit more disabling for kurtosis 2024-07-02 04:51:58 +05:30
Agnish Ghosh a8e2c3e9a2
exp: disable some gossip conditions 2024-07-02 04:45:23 +05:30
Agnish Ghosh 510d988933
disable subnet gossip condition, fixed inclusion proof 2024-06-30 12:16:17 +05:30
Agnish Ghosh c8d957a495
add: experimental checkpoints on gossip validation to localize failing conditions 2024-06-29 14:26:52 +05:30
Agnish Ghosh 0e01d2fb15
experimental disable for inclusion proofs 2024-06-28 18:01:14 +05:30
Agnish Ghosh 27b0705e0d
verify kzg disable 2024-06-28 15:29:57 +05:30
Agnish Ghosh 5bf1e021a7
initiate data column quarantine 2024-06-28 14:53:08 +05:30
Agnish Ghosh 3db92f8b26
add: data column reconstruction logic 2024-06-27 19:31:04 +05:30
Agnish Ghosh 325bdfd4a2
support for enqueueing whichever is activated blob/data_column 2024-06-25 18:08:00 +05:30
Agnish Ghosh 07d33b3abd
add dataColumns to db during forward syncing 2024-06-24 21:28:23 +05:30
Agnish Ghosh 791d2fb0d1
add: forward and backward syncing for data columns, broadcasting data columns created from blobs, added dc support to sync_queue 2024-06-24 17:32:06 +05:30
Agnish Ghosh 46d07b140d
add: data column support in sync_protocol, sync_manager, request_manager, fix: gossipValidation rules 2024-06-18 19:01:56 +05:30
Agnish Ghosh b4810fabeb
fix 2024-06-02 14:49:38 +05:30
Agnish Ghosh c35b5f1b4b
disable more 2024-06-02 10:52:13 +05:30
Agnish Ghosh caf5557d8b
oops 2024-06-02 10:47:28 +05:30
Agnish Ghosh 578a10693f
conditional disabling 2024-06-02 10:46:10 +05:30
Agnish Ghosh ca19e12136
experimental chenges 2024-06-02 10:39:45 +05:30
Agnish Ghosh 26519f68a0
feat: added kzg specs to gossip validation rules, fixed peerdas from C API 2024-05-29 14:02:56 +05:30
Agnish Ghosh aa2ce79770
fixed upto electra 2024-05-27 18:10:00 +05:30
Jacek Sieka d191b35e2e
electra: attestatoin sending (#6296) 2024-05-17 17:34:46 +02:00
Jacek Sieka 045c4cf185
electra attestation updates (#6295)
* electra attestation updates

In Electra, we have two attestation formats: on-chain and on-network -
the former combines all committees of a slot in a single committee bit
list.

This PR makes a number of cleanups to move towards fixing this -
attestation packing however still needs to be fixed as it currently
creates attestations with a single committee only which is very
inefficient.

* more attestations in the blocks

* signing and aggregation fixes

* tool fix

* test, import
2024-05-17 15:37:41 +03:00
tersec 826bf4c3ee
be more optimistic about electra 2024-05-17 07:13:30 +00:00
tersec 4355f81731
temporarily disable optimistic block hashing on electra 2024-05-16 11:17:31 +00:00
tersec c1b9e82502
electra attestation gossip plumbing (#6287) 2024-05-14 19:01:26 +03:00
tersec 3f972a2ca5
add some support for electra aggregated attestations (#6283) 2024-05-14 06:12:35 +02:00
tersec 6b8061b5d6
automated consensus spec URL updating to v1.5.0-alpha.2 (#6279) 2024-05-09 05:03:10 +00:00
tersec d139c92df9
explicitly scope AttesterSlashing and IndexedAttestation types to phase0 (#6224) 2024-04-21 05:49:11 +00:00
careworry 2ec82fe3ce
chore: remove repetitive words (#6219)
Signed-off-by: careworry <worrycare@outlook.com>
2024-04-19 11:31:15 +00:00
tersec 603c83522e
explicitly refer to phase0.{Attestation,TrustedAttestation} rather than sans module name (#6214) 2024-04-17 20:44:29 +00:00
tersec 867995acd1
some consensus spec v1.4.0 spec URL updates (#6208) 2024-04-17 05:51:16 +02:00
tersec e51c5ec783
add Electra blob support to block/blob quarantines, block processor, and request manager (#6201) 2024-04-11 09:31:39 +00:00
tersec 937cc62b85
block_sim runs electra (#6181) 2024-04-07 09:58:11 +02:00
tersec 7fa32b7f02
add Electra to ConsensusFork enum (#6169)
* add Electra to ConsensusFork enum

* fix gnosis check
2024-04-03 16:43:43 +02:00
tersec f9e5294802
dump EL-INVALID blocks if requested the same way as CL-INVALID blocks; optimize epoch transition validator exit (#6144) 2024-03-27 04:34:56 +01:00
Etan Kissling fc9bc1da3a
add branch discovery module for supporting chain stall situation (#6125)
In split view situation, the canonical chain may only be served by a
tiny amount of peers, and branches may span long durations. Minority
branches may still have a large weight from attestations and should
be discovered. To assist with that, add a branch discovery module that
assists in such a situation by specifically targeting peers with unknown
histories and downloading from them, in addition to sync manager work
which handles popular branches.
2024-03-24 08:41:47 +00:00
Etan Kissling 3765e8ac06
ensure blobs are quarantined when block is quarantined (#6127)
When quarantining a block from block processor, we should also keep a
copy of its blobs. Otherwise, this involves more network roundtrips
to obtain information we already have. This is in line with how blobs
arrive from gossip and request manager sources. The existing flow does
not work when applying blocks from quarantine, which is addressed here.
2024-03-24 04:56:30 +01:00
Etan Kissling 2d9586a5a8
enqueue missing parent block if stored in local DB (#6122)
When checking for `MissingParent`, it may be that the parent block was
already discovered as part of a prior run. In that case, it can be
loaded from storage and processed without having to rediscover the
entire branch from the network. This is similar to #6112 but for blocks
that are discovered via gossip / sync mgr instead of via request mgr.
2024-03-22 14:35:46 +01:00
Etan Kissling 6f466894ab
answer `RequestManager` queries from disk if possible (#6109)
When restarting beacon node, orphaned blocks remain in the database but
on startup, only the canonical chain as selected by fork choice loads.
When a new block is discovered that builds on top of an orphaned block,
the orphaned block is re-downloaded using sync/request manager, despite
it already being present on disk. Such queries can be answered locally
to improve discovery speed of alternate forks.
2024-03-21 18:37:31 +01:00
tersec 0a6d189161
automated consensus spec URL updating to v1.4.0 (#6074) 2024-03-14 07:26:36 +01:00
tersec fef831d92a
rm unused ForkedTrustedBeaconBlock; add some Electra overloads to consensus_object_pools; Electra BeaconBlock gossip support (#5965) 2024-02-26 06:49:12 +00:00
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
tersec 134774e00d
ensure reason field logging consistently uses string type (#5878) 2024-02-10 03:50:31 +01:00
Etan Kissling 9593ef74b8
do not cache zero block hash if block unavailable (#5865)
With checkpoint sync, the checkpoint block is typically unavailable at
the start, and only backfilled later. To avoid treating it as having
zero hash, execution disabled in some contexts, wrap the result of
`loadExecutionBlockHash` in `Opt` and handle block hash being unknown.

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2024-02-09 22:10:38 +00:00
Etan Kissling e398078abc
`...ExecutionPayloadHash` --> `...ExecutionBlockHash` (#5864)
Finish the rename started in #4809 to have a consistent naming.
`ExecutionPayloadHash` suggests hash over payload instead of block.
`BlockHash` is also the canonical name in engine API.
2024-02-08 01:24:55 +01:00
Etan Kissling b7026a683a
avoid marking blocks as unviable if `blobless` quarantine is full (#5858)
Full caches should not be used to mark blocks as unviable. The unviable
status is quite persistent and a block marked as such won't be processed
again once the cache empties. Problem originally introduced in #4808.
2024-02-07 13:38:20 +00:00
tersec 45b4b46041
use "reason" instead of "error"/"validatorError" to log gossip ignore/reject reasons (#5839) 2024-01-31 03:18:20 +00:00
tersec 3d7f634e70
unrevert more of https://github.com/status-im/nimbus-eth2/pull/5765 (#5834) 2024-01-29 08:35:16 +01:00
tersec 225ef5e69a
partially revert https://github.com/status-im/nimbus-eth2/pull/5765 (#5833) 2024-01-28 23:45:52 +01:00
Jacek Sieka 6328c77778
raises for gossip (#5808)
* raises for gossip

* fix light client
2024-01-22 17:34:54 +01:00
tersec 6c53dc1e11
automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
Jacek Sieka 3ff9b69bf1
simplify eth2_network error handling (#5765)
This PR gets rid of a bunch of redundant exception handling through
async raises guarantees.

More can be removed once libp2p gets properly annotated.
2024-01-19 21:05:52 +00:00
Etan Kissling be73ce2e9a
import finalized head LC bootstrap on launch (#5775)
If the initial state replays cover the finalized head, import matching
`LightClientBootstrap` into database.

This also addresses this error when light client requests bootstrap from
the genesis slot on networks that launch with Altair enabled.

```
{"lvl":"DBG","ts":"2023-10-04 11:17:49.665+00:00","msg":"LC bootstrap unavailable: Sync committee branch not cached","topics":"chaindag_lc","slot":0}
```
2024-01-18 22:51:26 +00:00
Etan Kissling 62ee92a094
bolster `BlobSidecar` syncing on incomplete responses (#5766)
Avoid marking blocks invalid when corresponding `blobSidecarsByRange`
returns an incomplete / incorrect response while syncing. The block
itself may still be valid in that scenario.
2024-01-18 15:45:10 +01:00
Etan Kissling e7794b6aec
log `delay` relative to deadline (#5769)
To better analyze logs across different `SECONDS_PER_SLOT`, log delays
for duties relative to their deadline instead of relative to slot start.
2024-01-17 15:52:19 +01:00
Etan Kissling f409ed97ae
align sync contribution logs with aggregate processing (#5770)
There are two conditions leading to `duplicate contribution` log.
Align the logs with the ones used for attestation aggregates,
so that the two conditions can be separated when reading logs.
2024-01-17 15:46:54 +01:00
Jacek Sieka d5785677a8
split out eth1chain into its own module (#5768)
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
tersec cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
tersec 69af8f943e
implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
tersec 1559a09184
add more logging of KZG commitments, proofs, and blobs (#5735) 2024-01-13 07:44:40 +01:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 030226148d
rename `exit_pool` > `validator_change_pool` (#5679)
The `ExitPool` was renamed to `ValidatorChangePool` with Capella, but
the files were still using the previous name. Rename for consistency.
2023-12-23 06:55:47 +01:00
Etan Kissling 583782a061
emit `proposer_slashing`/`attester_slashing` SSE on beacon-API (#5678)
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling a2081521f6
emit `bls_to_execution_change` SSE on beacon-API (#5677)
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
tersec 9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates (#5672) 2023-12-16 03:27:06 +01:00
Etan Kissling 0a5d9ee027
use `PayloadAttributesV3` in `nimbus_light_client` for Deneb (#5654)
* use `PayloadAttributesV3` in `nimbus_light_client` for Deneb

From Deneb onward, `forkchoiceUpdated` requires `PayloadAttributesV3`.
In `nimbus_light_client` we still used `PayloadAttributesV2`.

Also clean up two other locations that were already correctly using
`PayloadAttributesV3`, to reduce code duplication.

* fix letter case
2023-12-07 17:10:22 +00:00
tersec 4776fecc33
consensus spec v1.4.0-beta.5 URL updates (#5655) 2023-12-06 22:16:55 +00:00
tersec c36d2aa103
fix XDeclaredButNotUsed warnings (#5648) 2023-12-05 11:45:47 +00:00
tersec 9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 (#5647) 2023-12-05 03:34:45 +01:00
tersec 11bbc4010e
use correct pubkey for gossip validation of BLS to execution changes (#5646) 2023-12-04 22:10:13 +01:00
tersec 2fc43c9ba7
track block/blob matching/quarantines using both indices and commitments (#5621) 2023-12-01 18:58:46 +00:00
tersec 115ffa70eb
rm unused code (#5623) 2023-11-25 12:09:18 +00:00
Etan Kissling 375f276f31
reorder gossip validation checks (#5610)
* reorder gossip validation checks

Doing the coverage check only after the corresponding committee index is
known allows optimization by early rejecting invalid data.

* use same helper for individual attestations as well
2023-11-20 16:42:29 +02:00
Etan Kissling b9a693d5fe
send LC finality update on event stream on supermajority (#5602)
When new finality is reached without supermajority sync committee
support, trigger another event push on beacon-API and libp2p once
the finality gains supermajority support.

- https://github.com/ethereum/consensus-specs/pull/3549
2023-11-16 19:57:15 -08:00
tersec 54bdda13b4
rm unused code (#5596) 2023-11-11 11:49:34 +03:00
tersec c96163dbca
update some consensus spec URLs to v1.4.0-beta.4 (#5594) 2023-11-11 06:27:53 +01:00
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling e4dacc30c4
fix validation of blob index against gossip topic (#5569)
In `v1.4.0-alpha.0`, the blob index validation on gossip was changed to
use `compute_subnet_for_blob_sidecar` instead of having a separate topic
for each individual blob. We updated the spec reference in #5363 without
updating the code accordingly. Fixing this now, and also adding the new
`MAX_BLOBS_PER_BLOCK` check from `v1.4.0-beta.3` as it shares the theme.
2023-11-05 09:14:47 +01:00
Etan Kissling 28b84ff93b
extend `hasBlob` to all blocks from same proposer and slot (#5568)
`v1.4.0-beta.4` made the Gossip rules more strict and now requires to
ignore blobs from other branches if there are equivocating blocks.
Those blobs are only requestable via Req/Resp.
2023-11-05 09:13:57 +01:00
Etan Kissling f14389bb84
avoid perpetually sending blobs to peers (#5563)
Fix regression from #4808 where blobs that are already known are issued
ACCEPT verdict, propagating them to peers over and over again.

`validateBlobSidecar` contains the correct IGNORE logic. Moved it above
the expensive checks to retain the performance of the check.
2023-11-04 20:36:12 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00