Commit Graph

1166 Commits

Author SHA1 Message Date
Etan Kissling 2eb56f6e1b
rename `PeerScoreXyzBlocks` -> `PeerScoreXyzValues` (#4318)
The various `PeerScore` constants are used for both beacon blocks and
LC objects, and will likely also find use for EIP4844 blob sidecars.
Renaming them to use more generically applicable names not referring
to blocks explicitly aymore.
2022-11-11 11:34:28 +00:00
Etan Kissling 48994f67d3
rename `BlockError` -> `VerifierError` (#4310)
We currently use `BlockError` for both beacon blocks and LC objects.
In light of EIP4844, we will likely also use it for blob sidecars.
To avoid confusion, renaming it to a more generic `VerifierError`,
and update its documentation to be more generic.

To avoid long lines as a followup, also renaming the `block_processor`'s
`BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and
`BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
Jacek Sieka 09ade6d33d
Make trusted node sync era-aware (#4283)
This PR removes a bunch of code to make TNS aware of era files, avoiding
a duplicated backfill when era files are available.

* reuse chaindag for loading backfill state, replacing the TNS homebrew
* fix era block iteration to skip empty slots
* add tests for `can_advance_slots`
2022-11-10 10:44:47 +00:00
tersec a63ec842bc
add light client capella CI tests (#4311) 2022-11-10 10:23:46 +00:00
tersec 90eb2ccb20
database and fork choice test runner support for capella (#4309) 2022-11-09 17:32:10 +00:00
tersec 0919b8689e
run capella fork transition tests in CI (#4307) 2022-11-09 12:28:34 +00:00
Zahary Karadjov 651f01722c
Disable discovery in the Keymanager API tests 2022-11-09 13:01:54 +02:00
tersec 5c0ad1b5c8
automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4305) 2022-11-09 09:20:53 +00:00
Jacek Sieka b170a09c47
remove `news` leftovers (#4299) 2022-11-08 20:06:54 +00:00
Etan Kissling 7ad610f6d7
set fee recipient in empty payload fallback (#4291)
When the EL/Builder fails to produce an execution payload, we fall back
to an empty `ExecutionPayload`. Even though it contains no transactions
it should refer to the configured fee recipient. This is useful for
privacy reasons (do not reveal the reason for the empty payload) and for
compliance with additional fee recipient rules by staking pools.
2022-11-08 14:19:56 +00:00
Jacek Sieka 02b48fafad
remove unused / obsolete / unmaintained modules (#4298)
mostly this is chronicles-tail and its (extensive) dependencies along
with the simulation monitoring dashboard that is not maintained
2022-11-08 14:37:23 +01:00
Jacek Sieka 8297b962cc
Fix VC-based sync subnet subscriptions (#4293)
* move duty tracking code to `ActionTracker`
* fix earlier duties overwriting later ones
* re-run subnet selection when new duty appears
* log upcoming duties as soon as they're known (vs 4 epochs before)
2022-11-08 12:43:38 +01:00
tersec 909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
tersec 229b58b002
run capella slot sanity tests in CI (#4295) 2022-11-08 00:33:57 +00:00
tersec 0a43c89cd2
run capella block sanity tests in CI (#4292) 2022-11-07 18:37:48 +00:00
tersec 5b46f0b723
add Capella support to Forked* (#4276)
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
Jacek Sieka d839b9d07e
State-only checkpoint state startup (#4251)
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](https://github.com/ethereum/beacon-APIs/pull/226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
2022-11-02 10:02:38 +00:00
Etan Kissling aff9147c31
avoid packing attestations from other forks (#4273)
* avoid packing attestations from other forks

Revisit #3893 using method based on Lighthouse (less heavy computation).

* fix comment
2022-11-01 14:23:40 +02:00
tersec 0cfc1b776e
add all missing epoch transition tests (#4269) 2022-10-28 08:02:33 +00:00
tersec 48e351d672
run EF Capella consensus test fixtures (#4267) 2022-10-27 19:11:13 +00:00
tersec 7dd5c49c4e
use v1.3.0-alpha.0 test vectors (#4263) 2022-10-27 11:54:39 +00:00
tersec 06ccf5b80c
capella test vector support (#4261) 2022-10-27 06:29:24 +00:00
tersec f9830836a9
deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten (#4241)
* deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten

* remove Makefile support for Ropsten
2022-10-24 23:32:52 +03:00
tersec fb6e6d9cf4
remove `newPayload` from block production flow (#4186)
* remove `newPayload` from block production flow

* refactor block_processor to run `newPayload` as part of `storeBlock`
2022-10-14 22:48:56 +03:00
Jacek Sieka 819442acc3
Allow chain dag without genesis / block (#4230)
* Allow chain dag without genesis / block

This PR enables the initialization of the dag without access to blocks
or genesis state - it is a prerequisite for implementing a number of
interesting features:

* checkpoint sync without any block download
* pruning of blocks and states

* backfill checkpoint block
2022-10-14 22:40:10 +03:00
Jacek Sieka 40bed02f60
Build block in parallel with attestation packing (#4185)
* fix block proposal in first slot after checkpoint
2022-10-04 11:24:16 +00:00
tersec ad7541567c
move LVH handling to tests/; increase maximum fork choice retries (#4205) 2022-10-03 13:10:08 +00:00
tersec 0a4aa5fdb3
switch `withStateAndBlck` usage to `forkyState`; rm unused `tests/mocking/` modules (#4206) 2022-10-03 13:08:50 +00:00
Jacek Sieka af9ec577d0
nicer error message for failed backfill (#4188)
* nicer error message for failed backfill

Many checkpoint sources don't support block download

* RestGenericError -> RestErrorMessage

...and other assorted fixes to bring rest types closer to spec

* fix tests
2022-09-29 23:55:18 +03:00
Etan Kissling 5968ed586b
use LRU strategy for shuffling/epoch caches (#4196)
When EL `newPayload` is slow (e.g., Raspberry Pi with Besu), the epoch
and shuffling caches tend to fill up with multiple copies per epoch when
processing gossip and performing validator duties close to wall slot.
The old strategy of evicting oldest epoch led to the same item being
evicted over and over, leading to blocking of over 5 minutes in extreme
cases where alternate epochs/shuffling got loaded repeatedly.
Changing the cache eviction strategy to least-recently-used seems to
improve the situation drastically. A simple implementation was selected
based on single linked-list without a hashtable.
2022-09-29 14:55:58 +00:00
tersec c367b14ad9
deprecate `--safe-slots-to-import-optimistically` (#4182) 2022-09-29 06:29:49 +00:00
tersec 1819d79e07
avoid potential database inconsistency after fcU `INVALID`+crash (#4192)
* avoid database race-condition inconsistency after fcU `INVALID` then crash

* ensure head doesn't fall behind finalized; add more tests for head movement/reloading DAG
2022-09-28 21:07:31 +00:00
Eugene Kabanov 8778e1cf8d
Fix REST generic error parsing. (#4189)
* Fix REST generic error parser.
* Unescape test vectors.
* Fix RestGenericError writer and tests, to encode `code` as `Number`.
2022-09-28 18:47:15 +00:00
Jacek Sieka b1bc830a92
Harden EpochRef loading against bogus block root at tail (#4178)
* add more error information when things go wrong with database
* lower log level when reloading attestations from no-block epoch start
slot
2022-09-27 18:56:08 +02:00
tersec 0f6d19b4b3
implement v1.2.0 optimistic sync tests (#4174)
* implement v1.2.0 optimistic sync tests

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Etan Kissling <etan@status.im>

* `lvh` -> `latestValidHash` and only invalidate one specific block"

* `getEarliestInvalidRoot` -> `getEarliestInvalidBlockRoot`; `defaultEarliestInvalidRoot` -> `defaultEarliestInvalidBlockRoot`

Co-authored-by: Etan Kissling <etan@status.im>
2022-09-27 15:11:47 +03:00
Jacek Sieka 7f9af78ddb
test randao skippping (complements #3837) (#4179) 2022-09-27 09:22:24 +02:00
tersec 9750cd3a38
update state diffs to Bellatrix (#4177) 2022-09-26 19:13:50 +00:00
tersec 3c03ba86c1
update consensus spec ref URLs to v1.2.0 (#4164) 2022-09-23 07:56:06 +00:00
tersec 72e6b2021a
use v1.2.0 consensus spec test vectors (#4163) 2022-09-22 22:24:13 +00:00
zah ad63bba446
Support Prysm and Ethdo Keystores (Fixes #4107) (#4149) 2022-09-20 01:09:56 +03:00
Eugene Kabanov 174292b7e4
Sync gaps fix (#4090) 2022-09-19 12:37:42 +03:00
Eugene Kabanov ca871a5435
Fix HTTP/REST clients HTTP Content-Type header parsers. (#4139)
* Fix client HTTP content-type parsers.

* Fix tests.

* Address review comment and apply wildcard checks for generic decodeBytes.
2022-09-19 12:17:29 +03:00
Etan Kissling 9999362b11
detect mismatch of config and binary (#4132)
* detect mismatch of config and binary

When loading configuration that sets keys that Nimbus bakes into the
binary at compile-time, raise an error if the config is incompatible
instead of ignoring the conflicting value.
2022-09-19 12:07:46 +03:00
Jacek Sieka ef8bab58eb
load suggested fee recipient file also when keymanager is disabled (#4078)
Since these files may have been created in a previous run or manually,
we want to keep loading them even on nodes that don't enable the
keystore API (for example static setups)

Other changes:

* log keystore loading progressively (#3699)
* print initial fee recipient when loading validators
* log dynamic fee recipient updates
2022-09-17 08:30:07 +03:00
Etan Kissling 3ba016d75f
consistent peer scoring for missing non-finalized parent (#3381)
When the sync queue processes results for a blocks by range request,
and the requested range contained some slots that are already finalized,
`BlockError.MissingParent` currently leads to `PeerScoreBadBlocks` even
when the error occurs on a non-finalized slot in the requested range.
This patch changes the scoring in that case to `PeerScoreMissingBlocks`
for consistency with range requests solely covering non-finalized slots,
and, likewise, rewinds the sync queue to the next `rewindSlot`.
2022-09-16 21:45:53 +02:00
tersec 0410aec9d8
remove rest of `withState.state` usage (#4120)
* remove rest of `withState.state` usage

* remove scaffolding
2022-09-16 15:35:00 +02:00
tersec 5b0b48f6e9
implement /eth/v1/validator/register_validator (#4115) 2022-09-13 14:52:26 +03:00
tersec 8be964a152
update consensus layer spec ref URLs to v1.2.0-rc.3 (#4109) 2022-09-10 17:16:38 +00:00
tersec 19bf460a3b
more `withState` `state` -> `forkyState` (#4104) 2022-09-10 08:12:07 +02:00
tersec 1d620f0123
consensus spec URL updates to v1.2.0-rc.3 (#4105) 2022-09-09 21:56:06 +00:00