Commit Graph

2926 Commits

Author SHA1 Message Date
tersec b3f6be71d5
refactor `makeBeaconBlock`; some capella support for `ncli_db` and `wss_sim` (#4321) 2022-11-11 15:37:43 +01:00
Etan Kissling 93714756d7
update `sync_queue` docs w.r.t. joker's problem (#4317)
Explicitly mention in-line documentation within `sync_queue` relating to
older specification version to make rationale clearer.
2022-11-11 15:36:02 +01:00
Etan Kissling 94ff73af34
remove unused `SyncQueue.getFullMap` function (#4319)
The `getFullMap` function is unused and can be removed (verified with
`{.deprecated.}`).
2022-11-11 12:36:56 +00:00
Etan Kissling 2eb56f6e1b
rename `PeerScoreXyzBlocks` -> `PeerScoreXyzValues` (#4318)
The various `PeerScore` constants are used for both beacon blocks and
LC objects, and will likely also find use for EIP4844 blob sidecars.
Renaming them to use more generically applicable names not referring
to blocks explicitly aymore.
2022-11-11 11:34:28 +00:00
Etan Kissling 94b3df25f8
remove unused `sync_queue` types (#4316)
`SyncManagerError` and `sync_queue.BeaconBlocksRes` are unused and can
be removed for cleanup (verified with `{.deprecated.}`).
2022-11-11 10:41:30 +00:00
tersec 35b1104bea
`block_sim` runs capella by default (#4315) 2022-11-11 10:17:27 +00:00
tersec 04cbea754b
don't require attached validator for blinded block BN endpoint (#4313) 2022-11-10 20:18:08 +00:00
Etan Kissling 48994f67d3
rename `BlockError` -> `VerifierError` (#4310)
We currently use `BlockError` for both beacon blocks and LC objects.
In light of EIP4844, we will likely also use it for blob sidecars.
To avoid confusion, renaming it to a more generic `VerifierError`,
and update its documentation to be more generic.

To avoid long lines as a followup, also renaming the `block_processor`'s
`BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and
`BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
Jacek Sieka 09ade6d33d
Make trusted node sync era-aware (#4283)
This PR removes a bunch of code to make TNS aware of era files, avoiding
a duplicated backfill when era files are available.

* reuse chaindag for loading backfill state, replacing the TNS homebrew
* fix era block iteration to skip empty slots
* add tests for `can_advance_slots`
2022-11-10 10:44:47 +00:00
Jacek Sieka 1691d89695
3x key loading speedup via blst sha256 (refs #4277) (#4301)
* 3x key loading speedup via blst sha256 (refs #4277)

* readd sha2
2022-11-10 00:05:50 +02:00
tersec 90eb2ccb20
database and fork choice test runner support for capella (#4309) 2022-11-09 17:32:10 +00:00
tersec 0919b8689e
run capella fork transition tests in CI (#4307) 2022-11-09 12:28:34 +00:00
tersec 5c0ad1b5c8
automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4305) 2022-11-09 09:20:53 +00:00
Jacek Sieka 83c78c8002
use mainnet as default config (#4302)
Without this, tools like `ncli_db` require command line parameter to
process mainnet blocks, which is a regression since the merge - this is
a subset of #4147.
2022-11-08 22:53:02 +00:00
Jacek Sieka b170a09c47
remove `news` leftovers (#4299) 2022-11-08 20:06:54 +00:00
tersec a4d2637259
implement /eth/v1/beacon/blinded_blocks BN endpoint (#4286)
* implement /eth/v1/beacon/blinded_blocks BN endpoint

* avoid function names in non-debug logs

* callers log, so callee can just return err()
2022-11-08 18:08:43 +00:00
Etan Kissling 7ad610f6d7
set fee recipient in empty payload fallback (#4291)
When the EL/Builder fails to produce an execution payload, we fall back
to an empty `ExecutionPayload`. Even though it contains no transactions
it should refer to the configured fee recipient. This is useful for
privacy reasons (do not reveal the reason for the empty payload) and for
compliance with additional fee recipient rules by staking pools.
2022-11-08 14:19:56 +00:00
Jacek Sieka 8297b962cc
Fix VC-based sync subnet subscriptions (#4293)
* move duty tracking code to `ActionTracker`
* fix earlier duties overwriting later ones
* re-run subnet selection when new duty appears
* log upcoming duties as soon as they're known (vs 4 epochs before)
2022-11-08 12:43:38 +01:00
tersec 909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
Jacek Sieka fae3ed5067
log block number / fee recipient (#4294) 2022-11-07 22:16:47 +00:00
tersec 0a43c89cd2
run capella block sanity tests in CI (#4292) 2022-11-07 18:37:48 +00:00
Eugene Kabanov 8417b7e064
Sync committee subscription fixes. (#4281)
* Preparing code.

* Fix prepareXXX procedures to use `onceToAll` strategy only.

* Remove lighthouse like subscription code.

* Address review comments.
2022-11-03 20:23:33 +01:00
tersec 5b46f0b723
add Capella support to Forked* (#4276)
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
tersec cee5a73a31
v1/beacon/blocks/{block_id}, v1/debug/beacon/states/{state_id}, and v1/validator/blocks/{slot} deprecated (#4279)
* v1/beacon/blocks/{block_id}, v1/debug/beacon/states/{state_id}, and v1/validator/blocks/{slot} deprecated

* Update beacon_chain/rpc/rest_constants.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update beacon_chain/rpc/rest_constants.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-11-02 10:56:55 +00:00
Jacek Sieka fc724b21e8
Tighten libp2p request quotas (#4254)
To further tighten Nimbus against spam, this PR introduces a global
quota for block requests (shared between peers) as well as a general
per-peer request limit that applies to all libp2p requests.

* apply request quota before decoding message
* for high-bandwidth requests (blocks), apply a shared global quota
which helps manage bandwidth for high-peer setups
* add metrics
2022-11-02 10:46:53 +00:00
Jacek Sieka d839b9d07e
State-only checkpoint state startup (#4251)
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](https://github.com/ethereum/beacon-APIs/pull/226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
2022-11-02 10:02:38 +00:00
Etan Kissling aff9147c31
avoid packing attestations from other forks (#4273)
* avoid packing attestations from other forks

Revisit #3893 using method based on Lighthouse (less heavy computation).

* fix comment
2022-11-01 14:23:40 +02:00
tersec 3ef09ff596
Support `/eth/v1/validator/blinded_blocks` (#4272)
* Support BN endpoints for producing blinded blocks

* use correct endpoint version

* serve either JSON or SSZ versions of endpoint
2022-10-31 18:39:03 +01:00
Eugene Kabanov 00f083785d
VC: Fix for #4116 (external block builders support) (#4260) 2022-10-29 11:00:51 +02:00
tersec 0cfc1b776e
add all missing epoch transition tests (#4269) 2022-10-28 08:02:33 +00:00
Jacek Sieka 1572814a0c
Highlight execution client for optimistically synced node (#4266)
...such that the user knows where to look
2022-10-27 17:22:32 +00:00
Jacek Sieka 63a3f2b1ad
Tighten chunk decoding limits (#4264)
* cap maximum number of chunks to download from peer (fixes #1620)
* drop support for requesting blocks via v1 / phase0 protocol
* tighten bounds checking of fixed-size messages
2022-10-27 18:51:43 +02:00
Jacek Sieka 28fc70de6d
Add error to block production log 2022-10-27 16:46:49 +02:00
tersec 7dd5c49c4e
use v1.3.0-alpha.0 test vectors (#4263) 2022-10-27 11:54:39 +00:00
tersec 06ccf5b80c
capella test vector support (#4261) 2022-10-27 06:29:24 +00:00
tersec 69ed3a2fd6
fix false-positive warnings on expected VALID fcU status; adjust log levels (#4242)
* fix false-positive warnings on expected VALID fcU status; adjust log levels

* clearer info/warning message wording
2022-10-26 21:14:11 +00:00
Jacek Sieka b08d0ff2ab
Optimistic mode (#4262)
In optimistic mode, Nimbus will sync optimistically even when the
execution client is offline / not available.

An optimistic node is less secure because it has not validated block
transactions via the execution client and can thus not be used for
validation duties.
2022-10-26 20:44:45 +00:00
Jacek Sieka af52edaef7
Reduce fee recipient log spam (#4255)
Once is enough
2022-10-25 14:47:43 +02:00
tersec f9830836a9
deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten (#4241)
* deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten

* remove Makefile support for Ropsten
2022-10-24 23:32:52 +03:00
Eugene Kabanov 367e7052f4
VC: some fixes (#4240)
* Skip doppelganger protection for validators which activated just now or in future.

* Fix sync committee duties spam issue.

* Optimize sync committee duties logging statements.

* Fix missing lazyWait.

* Add short path.

* Address #4087.

* Add missing watch for crash.
2022-10-21 16:53:30 +02:00
Eugene Kabanov fa7b37a58a
VC: Remove BN network dependencies. (#4247)
* Remove one more and probably last one implicit `eth2_network` dependency from validator client.

* Add note comment.

* Fix tests issue.
2022-10-18 20:37:17 +02:00
Jacek Sieka 6c73015562
remove unused `is_valid_indexed_attestation` (#4248) 2022-10-18 17:02:58 +00:00
Eugene Kabanov 9fd4e7405e
Remove validator_duties dependency from validator_client. (#4244) 2022-10-17 13:42:43 +00:00
tersec fb6e6d9cf4
remove `newPayload` from block production flow (#4186)
* remove `newPayload` from block production flow

* refactor block_processor to run `newPayload` as part of `storeBlock`
2022-10-14 22:48:56 +03:00
Jacek Sieka 819442acc3
Allow chain dag without genesis / block (#4230)
* Allow chain dag without genesis / block

This PR enables the initialization of the dag without access to blocks
or genesis state - it is a prerequisite for implementing a number of
interesting features:

* checkpoint sync without any block download
* pruning of blocks and states

* backfill checkpoint block
2022-10-14 22:40:10 +03:00
Zahary Karadjov 89582c10db
Merge branch 'stable' into unstable 2022-10-14 22:24:55 +03:00
cheatfate b54a03ae07
Fix MEV builder file descriptor leaks. 2022-10-14 16:09:39 +03:00
Eugene Kabanov 805a12e467
VC: Fix doppelganger protection never allow attestations. (#4236)
* Fix doppelganger protection reorders validator indices in response issue.

* Add chronos metrics endpoint to nimbus REST API.

* Doppelganger protection now works on duties not on attestations.
Improve logging for doppelganger and indices.

* Improve doppelganger and indices logging.

* Add number of validators to logs.

* Move logging dumps from `debug` to `trace` level.
2022-10-14 14:19:17 +02:00
Etan Kissling 95aec9de12
update to v1 of LC REST API (#4232)
The LC REST API has been merged into the ethereum/beacon-APIs specs:
- https://github.com/ethereum/beacon-APIs/pull/247

Update URLs to v1 and update REST tests. Note that REST tests do not
start with Altair, so the tested BN will return empty / error responses.
2022-10-13 13:42:53 +03:00
Etan Kissling 4b7bb4796f
update for latest LC REST proposal (#4213)
Implements the latest proposal for providing LC data via REST, as of
https://github.com/ethereum/beacon-APIs/pull/247 with a v0 suffix.

Requests:
- `/eth/v0/beacon/light_client/bootstrap/{block_root}`
- `/eth/v0/beacon/light_client/updates?start_period={start_period}&count={count}`
- `/eth/v0/beacon/light_client/finality_update`
- `/eth/v0/beacon/light_client/optimistic_update`

HTTP Server-Sent Events (SSE):
- `light_client_finality_update_v0`
- `light_client_optimistic_update_v0`
2022-10-13 13:42:46 +03:00