Commit Graph

587 Commits

Author SHA1 Message Date
Agnish Ghosh 635eebbed1 fix response time 2024-08-19 13:17:33 +05:30
Agnish Ghosh 287bf72fbb simplify data column reconstruct and broadcast logic 2024-08-19 12:04:05 +05:30
Agnish Ghosh 80b3e70673 slight fix 2024-08-19 01:30:34 +05:30
Agnish Ghosh 15a302cea2 fixes related to reconstruction 2024-08-19 01:23:37 +05:30
Agnish Ghosh 75e1bb616b
handling reconstruction condition internally 2024-08-18 13:35:16 +05:30
Agnish Ghosh 121a3687b8
disable blob quarantine checks 2024-08-18 02:10:51 +05:30
Agnish Ghosh f3f61cbbff conditionally reconstruct and broadcast only when supernode 2024-08-16 01:03:53 +05:30
Agnish Ghosh 9be615dff9
add: data column reconstruction and broadcast (#6481)
* save commit, decouples reconstruction and broadcasting

* save progress

* add: reconstruction event loop, previous reconstruction related cleanups
2024-08-08 17:44:55 +05:30
Agnish Ghosh b32205de7c
upstream peerdas alpha3 related spec changes + fix upstream related issues (#6468)
* reworked some of the das core specs, pr'd to check whether whether the conflicting type issue is centric to my machine or not

* bumped nim-blscurve to 9c6e80c6109133c0af3025654f5a8820282cff05, same as unstable

* bumped nim-eth2-scenarios, nim-nat-traversal at par with unstable, added more pathches, made peerdas devnet branch backward compatible, peerdas passing new ssz tests as per alpha3, disabled electra fixture tests, as branch hasn't been rebased for a while

* refactor test fixture files

* rm: serializeDataColumn

* refactor: took data columns extracted from blobs during block proposal to the heap

* disable blob broadcast in pd devnet

* fix addBlock in message router

* fix: data column iterator

* added debug checkpoints to check CI

* refactor if else conditions

* add: updated das core specs to alpha 3, and unit tests pass
2024-08-05 19:27:39 +05:30
Agnish Ghosh b001499281 add: valid custody peer set to RequestManager 2024-07-23 12:36:42 +05:30
Agnish Ghosh 5265eeb6ea refactor: sync manager to range request only valid peers if not supernode 2024-07-22 19:04:04 +05:30
Agnish Ghosh fa5b15436c
add: logic constructing valid set of peers 2024-07-19 13:58:26 +05:30
Agnish Ghosh e034f30b5e
add: subscribeAllSubnets feature 2024-07-15 13:50:41 +05:30
Agnish Ghosh b33900bd35
added enr struct 2024-07-15 13:11:09 +05:30
Agnish Ghosh 152d276d78
added reconstruction logic 2024-07-03 21:50:52 +05:30
Agnish Ghosh 0e710da37b
update timings 2024-07-02 21:27:50 +05:30
Agnish Ghosh 8d2c4893d8
debug3 2024-07-02 19:28:23 +05:30
Agnish Ghosh 742669052a
dc quarantine activation, keeping blobs compatible 2024-06-29 14:07:09 +05:30
Agnish Ghosh 5bf1e021a7
initiate data column quarantine 2024-06-28 14:53:08 +05:30
Agnish Ghosh 87bc91febe
fix: message router 2024-06-26 02:38:04 +05:30
Agnish Ghosh 791d2fb0d1
add: forward and backward syncing for data columns, broadcasting data columns created from blobs, added dc support to sync_queue 2024-06-24 17:32:06 +05:30
Agnish Ghosh f0cae305d6
add: pruneDataColumns at the end of slot 2024-06-19 20:42:31 +05:30
Agnish Ghosh 51f189ef53
add: getMissingDataColumns, requestManagerDataColumnLoop 2024-06-19 03:46:03 +05:30
Agnish Ghosh eb0e40a5f0
added dataColumnSidecar on deneb fork 2024-05-27 18:15:41 +05:30
Agnish Ghosh aa2ce79770
fixed upto electra 2024-05-27 18:10:00 +05:30
tersec 69f13eaa51
add electra gossip transition 2024-05-15 13:30:49 +00:00
tersec 0ca5211f6a
enable nbn electra block processing (#6288) 2024-05-14 20:29:03 +02:00
tersec c1b9e82502
electra attestation gossip plumbing (#6287) 2024-05-14 19:01:26 +03:00
tersec 9c669cf57a
some debugRaiseAssert to debugComment; unblock some spec tests (#6284) 2024-05-14 12:19:24 +03:00
tersec 3f972a2ca5
add some support for electra aggregated attestations (#6283) 2024-05-14 06:12:35 +02:00
tersec 6b8061b5d6
automated consensus spec URL updating to v1.5.0-alpha.2 (#6279) 2024-05-09 05:03:10 +00:00
tersec abc8bbbf23
add EF consensus spec test Electra fork and transition fixtures (#6251) 2024-04-28 16:13:17 +02:00
tersec 87452374e4
add Electra SSZ object test fixture (#6225) 2024-04-22 09:00:38 +00:00
tersec d139c92df9
explicitly scope AttesterSlashing and IndexedAttestation types to phase0 (#6224) 2024-04-21 05:49:11 +00:00
tersec 603c83522e
explicitly refer to phase0.{Attestation,TrustedAttestation} rather than sans module name (#6214) 2024-04-17 20:44:29 +00:00
tersec 1bf4058e22
add Electra support to beacon API produceBlindedBlock and produceBlockV3 endpoints; add further VC publish(Blinded)Block Electra support (#6190) 2024-04-10 10:54:00 +02:00
tersec 7fa32b7f02
add Electra to ConsensusFork enum (#6169)
* add Electra to ConsensusFork enum

* fix gnosis check
2024-04-03 16:43:43 +02:00
Etan Kissling 2dbe24c740
move split view catchup to research branch (#6133)
Using a dedicated branch for researching the effectiveness of split view
scenario handling simplifies testing and avoids having partial work on
`unstable`. If we want, we can reintroduce it under a `--debug` flag at
a later time. But for now, Goerli is a rare opoprtunity to test this,
maybe just for another week or so.

- https://github.com/status-im/infra-nimbus/pull/179
2024-03-25 19:09:31 +01:00
Etan Kissling fc9bc1da3a
add branch discovery module for supporting chain stall situation (#6125)
In split view situation, the canonical chain may only be served by a
tiny amount of peers, and branches may span long durations. Minority
branches may still have a large weight from attestations and should
be discovered. To assist with that, add a branch discovery module that
assists in such a situation by specifically targeting peers with unknown
histories and downloading from them, in addition to sync manager work
which handles popular branches.
2024-03-24 08:41:47 +00:00
Etan Kissling 66a9304fea
use separate state when catching up to perform validator duties (#6131)
There are situations where all states in the `blockchain_dag` are
occupied and cannot be borrowed.

- headState: Many assumptions in the code that it cannot be advanced
- clearanceState: Resets every time a new block gets imported, including
  blocks from non-canonical branches
- epochRefState: Used even more frequently than clearanceState

This means that during the catch-up mechanic where the head state is
slowly advanced to wall clock to catch up on validator duties in the
situation where the canonical head is way behind non-canonical heads,
we cannot use any of the three existing states. In that situation,
Nimbus already consumes an increased amount of memory due to all the
`BlockRef`, fork choice states and so on, so experience is degraded.
It seems reasonable to allocate a fourth state temporarily during that
mechanic, until a new proposal could be made on the canonical chain.

Note that currently, on `unstable`, proposals _do_ happen every couple
hours because sync manager doesn't manage to discover additional heads
in a split-view scenario on Goerli. However, with the branch discovery
module, new blocks are discovered all the time, and the clearanceState
may no longer be borrowed as it is reset to different branch too often.

The extra state could also find other uses in the future, e.g., for
incremental computations as in reindexing the database, or online
collection of historical light client data.
2024-03-24 07:18:33 +01:00
Etan Kissling 6f466894ab
answer `RequestManager` queries from disk if possible (#6109)
When restarting beacon node, orphaned blocks remain in the database but
on startup, only the canonical chain as selected by fork choice loads.
When a new block is discovered that builds on top of an orphaned block,
the orphaned block is re-downloaded using sync/request manager, despite
it already being present on disk. Such queries can be answered locally
to improve discovery speed of alternate forks.
2024-03-21 18:37:31 +01:00
Etan Kissling de2d205f61
use PR-3431 style fork choice on all networks (#6110)
To start phasing out Capella fork choice logic, set default to PR 3431.
A subsequent release can remove the fallback option.
2024-03-20 19:12:33 +01:00
Etan Kissling 035ca015e6
continue validator duties if chain does not progress for a long time (#6101)
Nimbus currently stops performing validator duties if the blockchain
does not progress for `node.config.syncHorizon` slots. This means that
the chain won't recover because no new blocks are proposed. To fix that,
continue performing validator duties if no progress is registered for a
long time, and none of our peers is indicating any progress.
2024-03-20 03:23:53 +01:00
Etan Kissling 5d42859176
make `Gwei` `distinct` (#6090)
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
tersec 0a6d189161
automated consensus spec URL updating to v1.4.0 (#6074) 2024-03-14 07:26:36 +01:00
Eugene Kabanov 72c844534f
Add Keymanager API graffiti endpoints. (#6054)
* Initial commit.

* Add more tests.

* Fix API mistypes.

* Fix mistypes in tests.

* Fix one more mistype.

* Fix affected tests because of error code 401.

* Add GetGraffitiResponse object.

* Add more tests.

* Fix compilation errors.

* Recover old behavior.

* Recover old behavior.

* Fix mistype.

* Test could not know default graffiti value.

* Make VC use adopted graffiti settings.

* Make BN use adopted graffiti settings.

* Update Alltests.

* Fix test.

* Revert "Fix test."

This reverts commit c735f855d3cb9c4a1c8e8af29d3f4438d068e31f.

* Workaround {.push raises.} requirement.

* Fix comment.

* Update Alltests.
2024-03-14 03:44:00 +00:00
Etan Kissling 984d5e4631
fix regression when fetching genesis data in trustedNodeSync mode (#6047)
Corrects a regression from #5998 that led to crashes in #6046.
In `trustedNodeSync` mode, the config does not contain genesis keys,
so attempting to load from them is a `Defect`.
2024-03-08 14:52:54 +01:00
Etan Kissling a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 (#6038)
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
Etan Kissling 50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` (#6036)
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
Etan Kissling 3b4b1ccb89
metrics for proposer and sync duties (#6028)
To avoid restarting with scheduled proposals or sync duties, which are
rare but have high rewards, add metrics to simpify tracking them.
2024-03-07 15:08:48 +01:00