Commit Graph

569 Commits

Author SHA1 Message Date
tersec a4f4a35845
Revert "initial Electra support skeleton" (#5955)
* Revert "initial Electra support skeleton (#5946)"

This reverts commit d09bf3b587.

* Update test_signing_node.nim
2024-02-25 19:42:44 +00:00
Etan Kissling f54fa083b4
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-25 15:25:26 +01:00
tersec d09bf3b587
initial Electra support skeleton (#5946) 2024-02-24 13:44:15 +00:00
tersec 0f155ebf95
some consensus spec v1.4.0-beta.7 spec URL updates (#5945) 2024-02-22 02:42:57 +00:00
tersec c73d7c6f6f
automated consensus spec URL updating to v1.4.0-beta.7 (#5942) 2024-02-21 19:44:48 +00:00
Etan Kissling 88045a91cd
rename new timing metrics, as `_total` suffix is implicit (#5917)
* track latest duration instead of total in new timing metrics

Change `db_checkpoint_seconds` and `state_replay_seconds` metrics to
record the latest duration instead of the total. `nim-metrics` already
synthesizes a `_total` metric from these implicitly.

* still have to use inc, metrics only synthesizes the name not the sum

* prefix with `beacon_dag`
2024-02-20 20:34:41 +01:00
Jacek Sieka 8d465a7d8c
vmon: Missed block metric (#5913)
Validator monitoring gained 2 new metrics for tracking when blocks are
included or not on the head chain.

Similar to attestations, if the block is produced in epoch N, reporting
will use the state when switching to epoch N+2 to do the reporting (so
as to reasonably stabilise the block inclusion in the face of reorgs).
2024-02-20 06:40:18 +02:00
Etan Kissling 92197ce690
add metric for database checkpoint duration (#5897)
Database checkpointing can take seconds, e.g., while Geth is syncing.
Add a debug log + metric for it, and also info log if it takes longer
than 250ms, same as for the existing `State replayed` log. If the log
shows up for a user while the system is not overloaded, it may point
to slow disk speed or thermal issue.
2024-02-19 11:00:11 +01:00
Jacek Sieka afdfe302f3
state loading optimizations (#5881)
* compute post-merge randao mix without loading state
* avoid copying state on shuffling computation and compute epochref
* speed up state copy for block production
2024-02-12 15:58:55 +01:00
tersec a4680cb7fa
refactor addHeadBlock() to research/ and tests/ helper (#5874)
* refactor addHeadBlock() to research/ and tests/ helper

* rm now-dead code
2024-02-09 23:46:51 +00:00
Etan Kissling 9593ef74b8
do not cache zero block hash if block unavailable (#5865)
With checkpoint sync, the checkpoint block is typically unavailable at
the start, and only backfilled later. To avoid treating it as having
zero hash, execution disabled in some contexts, wrap the result of
`loadExecutionBlockHash` in `Opt` and handle block hash being unknown.

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2024-02-09 22:10:38 +00:00
Etan Kissling 7c53841cd8
Revert "Revert "fix checkpoint block potentially not getting backfilled into DB (#5863)" (#5871)" (#5875)
This reverts commit 1575478b72.
2024-02-09 20:44:54 +01:00
Etan Kissling f2d92729a2
reduce verbosity of `Got request for pre-backfill slot` (#5876)
When syncing, we log a notice each time someone asks us for a block that
we haven't backfilled yet. This is quite verbose and not unexpected,
because the status message does not allow indicating backfill progress.
2024-02-09 20:32:31 +01:00
tersec 1575478b72
Revert "fix checkpoint block potentially not getting backfilled into DB (#5863)" (#5871)
This reverts commit 65e6f892de.
2024-02-09 12:49:07 +00:00
Etan Kissling 65e6f892de
fix checkpoint block potentially not getting backfilled into DB (#5863)
When using checkpoint sync, only checkpoint state is available, block is
not downloaded and backfilled later.

`dag.backfill` tracks latest filled `slot`, and latest `parent_root` for
which no block has been synced yet.

In checkpoint sync, this assumption is broken, because there, the start
`dag.backfill.slot` is set based on checkpoint state slot, and the block
is also not available.

However, sync manager in backward mode also requests `dag.backfill.slot`
and `block_clearance` then backfills the checkpoint block once it is
synced. But, there is no guarantee that a peer ever sends us that block.
They could send us all parent blocks and solely omit the checkpoint
block itself. In that situation, we would accept the parent blocks and
advance `dag.backfill`, and subsequently never request the checkpoint
block again, resulting in gap inside blocks DB that is never filled.

To mitigate that, the assumption is restored that `dag.backfill.slot`
is the latest filled `slot`, and `dag.backfill.parent_root` is the next
block that needs to be synced. By setting `slot` to `tail.slot + 1` and
`parent_root` to `tail.root`, we put a fake summary into `dag.backfill`
so that `block_clearance` only proceeds once checkpoint block exists.
2024-02-09 11:20:36 +01:00
Etan Kissling 4266e16835
allow `getBlockIdAtSlot` to answer queries from available states (#5869)
After checkpoint sync, historical block IDs cannot yet be queried.
However, they are needed to compute dependent roots of `ShufflingRef`.
To allow lookup, enable `getBlockIdAtSlot` to answer from compatible
states in memory; as long as they descend from the finalized checkpoint
and the requested slot is sufficiently recent, `block_roots` contains
everything to recover `BlockSlotId` up to `SLOTS_PER_HISTORICAL_ROOT`.
This is similar to how `attester_dependent_root` etc. are computed.

This accelerates the first couple minutes of checkpoint sync on Mainnet,
especially the time until finality advances past the synced checkpoint.
2024-02-09 11:13:00 +01:00
Etan Kissling e398078abc
`...ExecutionPayloadHash` --> `...ExecutionBlockHash` (#5864)
Finish the rename started in #4809 to have a consistent naming.
`ExecutionPayloadHash` suggests hash over payload instead of block.
`BlockHash` is also the canonical name in engine API.
2024-02-08 01:24:55 +01:00
Etan Kissling 94ba0a9bd1
consider block availability when initializing LC data collector (#5860)
When using checkpoint sync, the initial block is missing in the DB.
Update the LC data collector initialization to account for that,
avoiding a spurious error message when it is incorrectly accessed:

```
ERR 2024-02-07 11:21:55.416+01:00 Block failed to load unexpectedly          topics="chaindag_lc" bid=d30517a7:8257504 tail=8257504
```

Also fixes a regression from #5691 that resulted in similar messages
while importing the first few blocks after checkpoint sync.

Thanks to @arnetheduck for reporting this.
2024-02-07 18:03:19 +00:00
Etan Kissling b7026a683a
avoid marking blocks as unviable if `blobless` quarantine is full (#5858)
Full caches should not be used to mark blocks as unviable. The unviable
status is quite persistent and a block marked as such won't be processed
again once the cache empties. Problem originally introduced in #4808.
2024-02-07 13:38:20 +00:00
Jacek Sieka 6328c77778
raises for gossip (#5808)
* raises for gossip

* fix light client
2024-01-22 17:34:54 +01:00
tersec 6c53dc1e11
automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
Eugene Kabanov 3648df7d4c
Fix VC not always be able to obtain feeRecipient value. (#5781)
Use state's validator value to obtain feeRecipient value.
Make feeRecipient and gasLimit calculation equal for BN and VC.
2024-01-19 14:36:04 +00:00
Etan Kissling be73ce2e9a
import finalized head LC bootstrap on launch (#5775)
If the initial state replays cover the finalized head, import matching
`LightClientBootstrap` into database.

This also addresses this error when light client requests bootstrap from
the genesis slot on networks that launch with Altair enabled.

```
{"lvl":"DBG","ts":"2023-10-04 11:17:49.665+00:00","msg":"LC bootstrap unavailable: Sync committee branch not cached","topics":"chaindag_lc","slot":0}
```
2024-01-18 22:51:26 +00:00
tersec cf1bec7670
update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
tersec 69af8f943e
implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
Jacek Sieka 62cbdeefc5
verify `genesis_time` more strictly (fixes #1667) (#5694)
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling 7db95f047b
track latest `LightClientUpdate` only once fork choice selects it (#5691)
Instead of tracking the latest `LightClientUpdate` across all branches,
track the latest one on the current branch as selected by fork choice.
2024-01-03 23:36:05 +01:00
Etan Kissling 030226148d
rename `exit_pool` > `validator_change_pool` (#5679)
The `ExitPool` was renamed to `ValidatorChangePool` with Capella, but
the files were still using the previous name. Rename for consistency.
2023-12-23 06:55:47 +01:00
Etan Kissling 583782a061
emit `proposer_slashing`/`attester_slashing` SSE on beacon-API (#5678)
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling a2081521f6
emit `bls_to_execution_change` SSE on beacon-API (#5677)
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
tersec 9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates (#5672) 2023-12-16 03:27:06 +01:00
andri lim 15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 (#5664)
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
Etan Kissling 0a5d9ee027
use `PayloadAttributesV3` in `nimbus_light_client` for Deneb (#5654)
* use `PayloadAttributesV3` in `nimbus_light_client` for Deneb

From Deneb onward, `forkchoiceUpdated` requires `PayloadAttributesV3`.
In `nimbus_light_client` we still used `PayloadAttributesV2`.

Also clean up two other locations that were already correctly using
`PayloadAttributesV3`, to reduce code duplication.

* fix letter case
2023-12-07 17:10:22 +00:00
tersec 4776fecc33
consensus spec v1.4.0-beta.5 URL updates (#5655) 2023-12-06 22:16:55 +00:00
Jacek Sieka 4a56faa579
era: fix verifier at empty slots (#5641)
* era: fix verifier at empty slots

* avoid returning zero-byte block data to REST/p2p when loading era
files

* fix local test
2023-12-05 07:55:25 +01:00
tersec 9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 (#5647) 2023-12-05 03:34:45 +01:00
tersec 2fc43c9ba7
track block/blob matching/quarantines using both indices and commitments (#5621) 2023-12-01 18:58:46 +00:00
tersec 6a9d522705
Apply EIP-7044 to block signature batch verification (#5637) 2023-12-01 14:44:45 +00:00
Etan Kissling beb915e308
remove emoji from function names (#5633)
Some editors seem to have issues with those. The ones in strings seem to
be alright, though. Rename the affected symbols.
2023-11-29 00:30:44 +00:00
tersec ab5343d1bc
update some consensus spec URLs to v1.4.0-beta.4 (#5631) 2023-11-27 19:56:34 +01:00
tersec 115ffa70eb
rm unused code (#5623) 2023-11-25 12:09:18 +00:00
tersec 5aa595fbc6
Option[deneb.SignedBeaconBlock] -> Opt[deneb.SignedBeaconBlock] (#5620) 2023-11-24 17:34:45 +01:00
Etan Kissling 8cea8af620
fix startup after BN exited between head and finalized blocks updates (#5617)
When the BN exits after writing new `head` to database, but before
completing the `updateFinalizedBlocks` call, the database is slightly
inconsistent due to the partial write. We currently fail to start up
after that. Fix that by catching up on partial `updateFinalizedBlocks`
tasks on start up, and add a test for this edge case.
2023-11-23 00:44:20 +01:00
Etan Kissling b0b8f21223
fix LC data import `--only-new` regression (#5616)
Corrects LC server startup logic for `LightClientDataImportMode.OnlyNew`
after regression in #5607.
2023-11-22 14:05:31 +01:00
Etan Kissling c33dd2c170
restrict best LC update collection to canonical blocks (#5613)
Simplify best `LightClientUpdate` collection by tracking only canonical
data instead of tracking the best update across all branches within the
sync committee period.

- https://github.com/ethereum/consensus-specs/pull/3553
2023-11-21 23:51:05 +01:00
Etan Kissling 375f276f31
reorder gossip validation checks (#5610)
* reorder gossip validation checks

Doing the coverage check only after the corresponding committee index is
known allows optimization by early rejecting invalid data.

* use same helper for individual attestations as well
2023-11-20 16:42:29 +02:00
Etan Kissling d8144c6de1
cache new LC data after creating new LC updates (#5607)
When creating new LC updates, information about the parent block's post
state must be available (cached), but information about current block's
post state is not yet required. Caching information about the current
block's post state can be delayed, simplifying the LC data collection
logic a bit and allowing more future flexibility with the cache design.
2023-11-18 01:17:26 +00:00
Etan Kissling b179cb71a0
clean up `XDeclaredButNotUsed` for light client (#5606)
Handle remaining `XDeclaredButNotUsed` warnings for light client.
2023-11-18 00:03:27 +01:00
Etan Kissling b9a693d5fe
send LC finality update on event stream on supermajority (#5602)
When new finality is reached without supermajority sync committee
support, trigger another event push on beacon-API and libp2p once
the finality gains supermajority support.

- https://github.com/ethereum/consensus-specs/pull/3549
2023-11-16 19:57:15 -08:00
Etan Kissling 0919ff05c9
reduce code duplication when collecting LC data (#5601)
Replace sections that need to be maintained with every `ConsensusFork`
related to LC data collection with a generic logic that keeps working
when unrelated parts of Ethereum change.
2023-11-15 03:21:03 +01:00