Commit Graph

187 Commits

Author SHA1 Message Date
Jacek Sieka 0660ffcd3e
fix debug logging 2023-07-11 21:48:27 +02:00
Jacek Sieka ca1775f725
Fetch by-root request directly from quarantine (#5167)
When the requestmanager is busy fetching blocks, the queue might get
filled with multiple entries of the same root - since there is no
deduplication, requests containing the same root multiple times will be
sent out.

Also, because the items sit in the queue for a long time potentially,
the request might be stale by the time that the manager is ready with
the previous request.

This PR removes the queue and directly fetches the blocks to download
from the quarantine which solves both problems (the quarantine already
de-duplicates and is clean of stale information).

Removing the queue for blobs is left for a future PR.

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 18:22:02 +02:00
tersec 614202e30d
automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
Etan Kissling 8a853fc4e4
extract LC sync task scheduling helper function (#5117)
The helper function to compute delay until next light client sync task
can be useful from more general purpose contexts. Move to helpers, and
change it to return `Duration` instead of `BeaconTime` for flexibility.
2023-06-23 06:19:51 +00:00
Etan Kissling bab6a77abc
optimize LC sync task scheduling (#5112)
- When syncing `LightClientUpdatesByRange`, and peer replies with
  fewer periods than requested, no need to delay next request.
- When `FinalityUpdate` / `OptimisticUpdate` sync fails,
  no need to retry immediately.
2023-06-21 17:44:49 +00:00
Etan Kissling 6ec6b99ef6
extract `nextLightClientSyncTask` helper (#5108)
The `nextLightClientSyncTask` helper is generic enough to be useful for
contexts besides libp2p based light client sync. Extract it.
2023-06-21 06:28:14 +00:00
Etan Kissling d670af5bf5
align style when calling LC libp2p API (#5104)
The `UpdatesByRange` API takes `startPeriod / count`, but is internally
called by `Slice`. Move the logic that converts from the `Slice` to the
caller to reduce complexity inside the used `doRequest` function.
2023-06-20 20:21:32 +00:00
tersec 788cdb7133
automated v1.4.0-alpha.2 to v1.4.0-alpha.3 consensus spec URL updates (#5065) 2023-06-13 14:03:49 +00:00
Etan Kissling a3229a6b9b
extract helper for validating `UpdatesByRange` consistency (#5053)
Reduce code duplication when checking response of `UpdatesByRange`.
2023-06-12 14:22:32 +02:00
tersec 22208836b1
automated v1.4.0-alpha.1 to v1.4.0-alpha.2 consensus spec URL updates (#5056) 2023-06-10 09:56:54 +00:00
Jacek Sieka fc833d40f6
bump chronos (#5032)
* ratelimiter crash fix
* implicit returns now available!
2023-06-05 18:52:16 +00:00
henridf c2aa8313c3
Sync: fix backfill blob iteration order (#4993)
* Sync: fix backfill blob iteration order

* Address review feedback

* Add comment clarifying reason for func top-level placement
2023-06-05 14:42:27 +00:00
tersec f86febc111
update consensus spec URLs to v1.4.0-alpha.1 (#5027) 2023-06-04 10:32:50 +00:00
tersec 1f535336da
automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates (#4996)
* automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates

* add copyright year headers
2023-05-26 00:14:28 +02:00
tersec cd087b9a43
replace `optimisticRoots` table with field in `BlockRef` (#4969)
* replace optimisticRoots table with field in BlockRef

* copyright year

* mark finalized blocks as verified on load

* Update beacon_chain/consensus_object_pools/block_dag.nim

Co-authored-by: Etan Kissling <etan@status.im>

* expand non-optimistic block checking to all pre-merge blocks; refactor markBlockVerified to use BlockRef rather than block root and remove superfluous caller in newPayload path replaced by addResolvedHeadBlock BlockRef construction

* don't treat finalized block specially; VALID status is sticky

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-05-20 12:18:51 +00:00
henridf 0044c18c01
Add reqCount to BlobSidecar range request done log (#4982) 2023-05-19 20:14:41 +00:00
henridf 1cf777c64b
Fix sync for blocks older than MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS (#4977)
When doing sync for blocks older than
MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS, we skip the blobs by range
request, but we then pass en empty blob sequence to
validation, which then fails.

To fix this: Use an Option[Blobsidecars] to allow expressing the
distinction between "empty blob sequence" and "blobs unavailable". Use
the latter for "old" blocks, and don't attempt to run blob validation.
2023-05-19 16:25:11 +00:00
Etan Kissling 22c298c07b
final portion of trivial v1.3.0 bumps (#4926)
* final portion of trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint
2023-05-11 09:54:29 +00:00
henridf be3f5b1eac
More blob tweaks/fixes from running in devnet (#4933)
* BeaconNode: don't call fetchMissingblobs with empty list

* More logging

* BlockProcessor.checkBloblessSignature: Add missing return value
2023-05-11 00:36:35 +00:00
Etan Kissling 322429b191
final portion of non-trivial v1.3.0 bumps (#4927)
* final portion of non-trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs,
and cleans up surrounding sections / syncs comments, and so on.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint

* linebreak
2023-05-10 16:04:48 +02:00
henridf 23adf15e5a
Blob handling sync fixes (#4888)
* Fix groupBlobs

* Fix getShortMap

* Fix blob handling in sync

* lint

* Add some blob-related logging
2023-05-06 08:58:50 +00:00
henridf ef0b95dfbc
RequestManager: add support for fetching Blobs (#4844)
* RequestManager: add support for fetching Blobs

* Review feedback

* Lint

* Change peekSortedBlobless -> peekBlobless
2023-04-28 12:57:35 +00:00
tersec b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
henridf 12d640b691
Request manager: Handle blobless blocks (#4833)
Post-Deneb, when the request manager receives a missing block from a
peer, it needs to check if the corresponding blobs are available, and
if so pass them along. If they aren't available, the newly-fetched
block must be put in blobless quarantine (while the blobs are
retrieved, coming in next commit).
2023-04-19 19:37:38 +03:00
henridf 8f2f40f0af
Remove blockBlobsVerifier (#4829)
Having a separate 'blockBlobsVerifier' function for post-Deneb blocks
is no longer of any benefit. This commit removes it.
2023-04-18 02:12:57 +02:00
Jacek Sieka 7df75d77fa
move spec constants to spec (#4825)
* move spec constants to spec

quite funny, this removes >_1000_ transitive imports when building
`nimbus_validator_client`.

`genDepend`:

```sh
[arnetheduck@praeceps beacon_chain]$ diff -u
nimbus_validator_client.dot.pre nimbus_validator_client.dot | wc
   1167    3507  103822
```

* imports

* better constants

* oops

* too many blobs

* reorder spec constants in fork order

* cleanup
2023-04-17 17:18:54 +00:00
tersec c31fbc3977
remove most std/options imports (#4778) 2023-03-31 20:46:47 +00:00
tersec 2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00
Etan Kissling fd8e86972d
rename `lcDataForkAtStateFork` > `lcDataForkAtConsensusFork` (#4726) 2023-03-11 20:09:21 +00:00
Etan Kissling d8f39dd24f
rename `stateForkForDigest` > `consensusForkForDigest` (#4727) 2023-03-11 16:58:48 +00:00
Etan Kissling 69013d153c
bump light client spec references to `v1.3.0-rc.3` (#4719) 2023-03-11 01:11:51 +00:00
Etan Kissling 969c6f73ae
misc local `EIP4844` > `Deneb` bumps (#4717)
* misc local `EIP4844` > `Deneb` bumps

* fix
2023-03-11 00:28:19 +00:00
Etan Kissling e4b5478fff
rename `ForkDigests.eip4844` > `ForkDigests.deneb` (#4715) 2023-03-10 17:13:40 +00:00
tersec a47f0b054e
finish eip4844 to deneb module rename (#4705) 2023-03-09 01:34:17 +01:00
henridf 90640cce05
Update sync to use post-decoupling RPC (#4701)
* Update sync to use post-decoupling RPCs

blob_sidecars_by_range returns a flat list of sidecars, which must
then be grouped per-slot.

* Add test for groupBlobs

* createBlobs: convert proc to func
2023-03-07 20:19:17 +00:00
henridf 1de3cf5246
Remove SignedBeaconBlockAndBlobsSidecar (#4683)
This commit removes SignedBeaconBlockAndBlobsSidecar and all remaining
references.
2023-03-02 15:12:04 +01:00
henridf 0a87889eaf
Add blobSidecar RPCs (#4677)
* Add blob_sidecars_by_range RPC

* Add blob_sidecars_by_root RPC
2023-02-27 21:16:59 +01:00
henridf a999da3361
Remove beacon_block_and_blobs_sidecar_by_root RPC (#4672)
Along with its use in the request manager.
2023-02-26 23:17:40 +00:00
tersec 29fb65a9db
automated update of v1.3.0-rc.2 to v1.3.0-rc.3 consensus spec URLs (#4647) 2023-02-21 16:43:21 +00:00
tersec e342fdd97a
rename stateForkAtEpoch to consensusForkAtEpoch (#4627) 2023-02-16 11:32:12 +02:00
zah 067ba13c52
Capella metadata for Sepolia (#4615)
Other changes:

Renamed the `EIP_4844_FORK_*` config constants to `DENEB_FORK_*` as
this matches the latest spec and it's already used in the official
Sepolia config.
2023-02-15 14:44:09 +00:00
tersec b7c3037f01
consensus spec ref URL updates to v1.3.0-rc.2 (#4626) 2023-02-15 14:43:51 +00:00
tersec f9022618f0
deprecate req/resp GetMetaData v1 (#4608) 2023-02-13 12:02:20 +01:00
henridf 59e41dc65d
EIP4844 sync (#4581)
* EIP4844 Sync

* Pass eip4844 fork epoch rather than cfg to syncmanager

* Fix sync

* Update test

* map->mapIt
2023-02-11 20:48:35 +00:00
Jacek Sieka f3ddea6c86
Skip execution payload verification for finalized blocks (#4591)
While syncing the finalized portion of the chain, the execution client
cannot efficiently sync and most of the time returns `SYNCING` - in this
PR, we use CL-verified optmistic sync as long as the block is claimed to
be finalized, only occasionally updating the EL with progress.

Although a peer might lie about what is finalized and what isn't,
eventually we'll call the execution client - thus, all a dishonest
client can do is delay execution verification slightly. Gossip blocks in
particular are never assumed to be finalized.
2023-02-06 08:22:08 +01:00
henridf 9233a52e4b
Don't pass RuntimeConfig to RequestManager (#4589) 2023-02-04 12:13:09 +01:00
henridf 94837caa2a
Eip4844 sync fixes (#4577)
* fixes

* Make some log messages blob-aware

* remove redundant optBlobs()
2023-02-01 14:14:50 +00:00
henridf d7316ac863
Wire up blob fetching with request manager (#4527)
* Wire up blob fetching and request manager

* Update beacon_chain/sync/request_manager.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

* Review feedback: remove redundant RequestManager field.

* Fix syntax

* Improve blob fetching logic

* fix previous commit

* rman.isBlobsTime(): Remove spurious parameter

* Review feedback

* expressionify an if

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-02-01 00:25:08 +01:00
tersec 58ed9308d2
automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4568) 2023-01-31 00:26:57 +01:00