Commit Graph

3098 Commits

Author SHA1 Message Date
Etan Kissling ea6a6b1acd
track slot as part of fork choice debug API (#4565)
Extends fork choice state to also track slot numbers to improve accuracy
of `/eth/v1/debug/fork_choice` endpoint. Autoenable this API on devnet,
and disable some extra checks on devnet to aid focused testing efforts.
Align fork choice pruning logic with API based on checkpoints vs root.
2023-01-31 12:35:01 +00:00
tersec 58ed9308d2
automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4568) 2023-01-31 00:26:57 +01:00
tersec f5e302f973
use 1.3.0-rc.2-hotfix consensus spec test vectors (#4566) 2023-01-30 22:54:16 +00:00
tersec b2f415cbe5
remove merge cruft (#4559) 2023-01-30 20:22:05 +01:00
tersec 8e2792bdd7
remove Nim 1.2 workarounds and `TODO`s (#4533)
* clean up some Nim 1.2 workarounds

* re-add notes about JS backend

* another proc/noSideEffect -> func

* revert ncli/ncli_common.nim changes; 19969 evidently wasn't backported to 1.6
2023-01-30 20:21:51 +01:00
tersec 29ea42ce1a
use v1.3.0-rc.2 consensus spec test vectors (#4564) 2023-01-29 11:28:04 +00:00
tersec 0fb726c420
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560)
* `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork`

* revert unrelated change

* revert unrelated changes

* update test summaries
2023-01-28 19:53:41 +00:00
henridf 67e2f854d5
Fix BeaconBlockAndBlobsSidecar encoding (#4562) 2023-01-28 12:30:38 +00:00
tersec 7635b47a4e
update/clarify Nim issue link after retesting/re-isolating 2023-01-27 10:41:19 +00:00
Etan Kissling 9f279e2933
cache LC headers and sync committees for bootstrap (#4499)
To allow LC data retention longer than the one for historic states,
introduce persistent DB caches for `current_sync_committee` and
`LightClientHeader` for finalized epoch boundary blocks.
This way, historic `LightClientBootstrap` requests may still be honored
even after pruning. Note that historic `LightClientUpdate` requests are
already answered using fully persisted objects, so don't need changes.

Sync committees and headers are cached on finalization of new data.
For existing data, info is lazily cached on first access.

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-01-27 10:44:57 +01:00
Zahary Karadjov daa99f43c3
Merge branch 'stable' into unstable 2023-01-26 16:47:26 +01:00
Eugene Kabanov 08ed8ad43e
Adopt BN and VC header sizes and requirements, to avoid users confusion with default configuration options. (#4556)
Add comments.
2023-01-26 16:00:10 +01:00
tersec c22384d7ba
always use Bellatrix message size limits (#4558) 2023-01-26 14:24:23 +00:00
Zahary Karadjov 1ea102b033
Version 23.1.1 2023-01-26 14:37:03 +01:00
Jacek Sieka 0c1686eb20
fix pruned backfill summary (fixes #4552)
When enabling pruned history mode, the backfill summary as it was
computed with full backfilling may be pruned - don't try to load it on
init
2023-01-26 14:23:12 +01:00
Jacek Sieka fa27ce76a3
vmon: set details mode to default off
The "on" default for validator monitor details incurs a heavy
performance penalty on large-validator setups - this may cause excess
memory usage or slowdowns when metrics are queried - this PR changes the
default to off, as was intended for the 23.1.0 release.
2023-01-26 14:18:52 +01:00
Jacek Sieka 968e27bc97
fix pruned backfill summary (fixes #4552) (#4554)
When enabling pruned history mode, the backfill summary as it was
computed with full backfilling may be pruned - don't try to load it on
init
2023-01-26 11:23:17 +01:00
henridf 03b468c537
Pass correct block root to validate_blobs_sidecar (#4557) 2023-01-26 09:25:37 +00:00
henridf 7966ab6be2
Some EIP4844 fixes (#4549)
* debug log upon sidecar validation failure

* Fill in signature catch upon SignedBeaconBlockAndBlobsSidecar deser

* Always fill blobssidecar slot and root

* Skip lastFCU when eth1monitor is nil

* fix

* Use cached root
2023-01-25 18:35:46 +01:00
henridf d349a4797f
Fix minimum_request_epoch in BeaconBlockAndBlobsSidecarByRoot (#4555) 2023-01-25 18:33:01 +01:00
tersec d30544f790
disallow filling BLS to execution change pool pre-capella (#4547)
* disallow filling BLS to execution change pool pre-capella

* use better BLS change epoch
2023-01-24 22:25:51 +01:00
Etan Kissling efbd4e395a
avoid sending redundant LC finality updates (#4546)
When the epoch boundary block is missed, we incorrectly assume that the
next couple blocks improve finality, leading to repeated pushes of the
same light client finality update and incorrectly ignoring some gossip.
2023-01-24 17:44:55 +00:00
tersec fe1a57c220
use shortLog for execution payload logging (#4544) 2023-01-24 13:19:38 +00:00
Jacek Sieka ba0496105a
ncli_db: fix readonly properly (#4543)
🐡

Co-authored-by: Etan Kissling <etan@status.im>
2023-01-24 02:52:08 +00:00
Etan Kissling 0b8bb11c50
combine common implementation of LC helpers (#4542)
* combine common implementation of LC helpers

Combine replicated helper code from Altair/Capella/EIP4844 into single
`Forky` based implementation. Also convert `template` to `func` to avoid
selection of incorrect overload.

* fix
2023-01-23 19:38:03 +00:00
Zahary Karadjov 285eec6512
Add metrics and debug logging for dropped BLS to execution change messages 2023-01-23 14:58:40 +01:00
henridf 349001b7fb
eip4844 beacon block proposals (#4540)
* eip4844 beacon block proposals

* Don't fetch blobs under minimal preset

@tersec's summary of the issue:

BlobsBundleV1 in the execution API spec assumes a mainnet preset blob
size, where the EIP4844 consensus spec defines
FIELD_ELEMENTS_PER_BLOB: 4 under the minimal preset, which leads to a
Blob having a length of 4 * 32, not 4096 * 32 which BlobsBundleV1
requires.

* Revert unintentional script change
2023-01-21 23:13:21 +00:00
tersec 0a93023f46
capella fcU usage updating duly ensured (#4539) 2023-01-21 13:45:23 +00:00
henridf 96dfe18a3d
make message_router.routeSignedBeaconBlock blob-aware (#4537)
* make message_router.outeSignedBeaconBlock blob-aware

* Update beacon_chain/spec/forks.nim

* Update beacon_chain/spec/forks.nim
2023-01-21 06:34:04 +00:00
Etan Kissling a57cec56cb
update EIP4844 related light client sync docs (#4538)
Further clarify why it is okay to drop blobs during LC based sync.
2023-01-21 06:23:28 +00:00
henridf f8ee0def2b
Add stubs for EIP4844 engine API calls (#4536) 2023-01-21 00:47:38 +00:00
tersec 8c2e01da24
consensus spec ref URL updates to v1.3.0-rc.1 (#4534) 2023-01-20 22:33:05 +00:00
tersec 6a64048e80
remove vestigial/unused BLSToExecutionChange parameters (#4535) 2023-01-20 20:51:54 +00:00
tersec aacc8d702d
remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528) 2023-01-20 14:14:37 +00:00
Etan Kissling a5596dc5e6
add documentation for `shouldSyncOptimistically` (#4530)
Clarify rationale behind the `shouldSyncOptimistically` workaround for
why we subscribe to blocks gossip when light client syncing.
2023-01-20 05:31:05 +01:00
tersec cd58613615
remaining automated rc.0 to rc.1 consensus spec ref URL changes (#4529) 2023-01-20 00:20:51 +00:00
Etan Kissling 65ca523482
support EIP4844 LC data format (#4520)
Implements the proposed light client data format for EIP4844:
- https://github.com/ethereum/consensus-specs/pull/3151
2023-01-19 23:53:37 +01:00
tersec 819e007689
exit/validatorchange pool includes BLS to execution messages; REST support for new pool (#4519)
* exit/validatorchange pool includes BLS to execution messages; REST
support for new pool

* catch failed individual futures

* increase BLS changes bound and keep BLS seen consistent with subpool

* deque capacities should be powers of 2
2023-01-19 22:00:40 +00:00
Zahary Karadjov ba7c0bc091
Version 23.1.0 2023-01-18 19:42:41 +02:00
zah ae301af291
Fix #4500 (#4523) 2023-01-18 19:20:28 +02:00
Etan Kissling 58f4c13e14
support Capella LC data format (#4514)
Implements the proposed light client data format for Capella:
- https://github.com/ethereum/consensus-specs/pull/3151
2023-01-18 15:32:57 +01:00
tersec aea7a0c8b8
remove TTD monitoring (#4486) 2023-01-18 16:01:49 +02:00
Etan Kissling 9cceb1b4a0
support `readOnly` backend in LC client/data dbs (#4518)
When accessing DB in `readOnly` mode that does not already have latest
schema, initial writes trigger `attempt to write a readonly database`.
Avoid that by only writing schema when DB is not `readOnly`, and provide
data from legacy tables if such are present.
2023-01-18 03:06:23 +01:00
tersec 073c544f0c
automated update from v1.3.0-rc.0 to v1.3.0-rc.1 consensus spec URLs (#4517) 2023-01-17 16:10:52 +00:00
tersec 9b0868033e
use GitHub Actions cache and upload-artifact v3 (#4516) 2023-01-17 15:45:04 +00:00
henridf 05a6c6280d
Add `beacon_block_and_blobs_sidecar_by_root` RPC (#4475)
* Implement beacon_block_and_blobs_sidecar_by_root

* review feedback

* more review feedback
2023-01-16 17:08:16 +00:00
henridf 727920a571
Refactor block/blobs types (#4491)
* Refactor block/blobs types

Use type system to enforce invariant that a pre-4844 block cannot have
a sidecar.

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

* review feedback

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-01-16 16:26:48 +00:00
Etan Kissling fda03548e3
use `ForkedLightClientStore` internally (#4512)
When running `nimbus_light_client`, we persist the latest header from
`LightClientStore.finalized_header` in a database across restarts.
Because the data format is derived from the latest `LightClientStore`,
this could lead to data being persisted in pre-release formats.

To enable us to test later `LightClientStore` versions on devnets,
transition to a `ForkedLightClientStore` internally that is only
migrated to newer forks on-demand (instead of starting at latest).
2023-01-16 16:53:45 +01:00
Jacek Sieka d8caab500d
Enable validator monitor by default (#4468)
By enabling the validator monitor, more precise information about the
lifecycle of an attestation is logged at the higher `NOTICE` log level
while current `sent` messages are logged at `INF` instead, since they
are less interesting.

In particular, missed attestations and those that vote for the wrong
head are now detected and logged at NOTICE.

In addition to logging, this feature enables rich metrics around
attestation and sync committee performance - by default, validators are
tracked in aggregate but a detailed mode exists as well

This feature has been available since early Nimbus days, but it has now
been tuned and optimised such that it is safe to enable by default, even
for large setups.

* enable automatic validator monitoring by default
* replace `--validator-monitor-totals` flag with
`--validator-monitor-details` - the detailed mode is disabled by default
* lower "sent" log level to `INF` for several messages - in particular
those that are traced by the validator monitor

This is a retake on #3531 which was later reverted in #3578.
2023-01-16 11:28:35 +01:00
Etan Kissling 609227559f
LC data fork cleanup (#4506)
Distinguish between those code locations that need to be updated on each
light client data format change, and those others that should generally
be fine, as long as a valid light client object is processed.

The former are tagged with static assert for `LightClientDataFork.high`.

The latter are changed to `lcDataFork > LightClientDataFork.None` to
indicate that they depend only on presence of any valid object.
Also bundled a few minor cleanups and fixes.

Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 22:19:50 +01:00