1166 Commits

Author SHA1 Message Date
Etan Kissling
2eb56f6e1b
rename PeerScoreXyzBlocks -> PeerScoreXyzValues (#4318)
The various `PeerScore` constants are used for both beacon blocks and
LC objects, and will likely also find use for EIP4844 blob sidecars.
Renaming them to use more generically applicable names not referring
to blocks explicitly aymore.
2022-11-11 11:34:28 +00:00
Etan Kissling
48994f67d3
rename BlockError -> VerifierError (#4310)
We currently use `BlockError` for both beacon blocks and LC objects.
In light of EIP4844, we will likely also use it for blob sidecars.
To avoid confusion, renaming it to a more generic `VerifierError`,
and update its documentation to be more generic.

To avoid long lines as a followup, also renaming the `block_processor`'s
`BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and
`BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
Jacek Sieka
09ade6d33d
Make trusted node sync era-aware (#4283)
This PR removes a bunch of code to make TNS aware of era files, avoiding
a duplicated backfill when era files are available.

* reuse chaindag for loading backfill state, replacing the TNS homebrew
* fix era block iteration to skip empty slots
* add tests for `can_advance_slots`
2022-11-10 10:44:47 +00:00
tersec
a63ec842bc
add light client capella CI tests (#4311) 2022-11-10 10:23:46 +00:00
tersec
90eb2ccb20
database and fork choice test runner support for capella (#4309) 2022-11-09 17:32:10 +00:00
tersec
0919b8689e
run capella fork transition tests in CI (#4307) 2022-11-09 12:28:34 +00:00
Zahary Karadjov
651f01722c
Disable discovery in the Keymanager API tests 2022-11-09 13:01:54 +02:00
tersec
5c0ad1b5c8
automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4305) 2022-11-09 09:20:53 +00:00
Jacek Sieka
b170a09c47
remove news leftovers (#4299) 2022-11-08 20:06:54 +00:00
Etan Kissling
7ad610f6d7
set fee recipient in empty payload fallback (#4291)
When the EL/Builder fails to produce an execution payload, we fall back
to an empty `ExecutionPayload`. Even though it contains no transactions
it should refer to the configured fee recipient. This is useful for
privacy reasons (do not reveal the reason for the empty payload) and for
compliance with additional fee recipient rules by staking pools.
2022-11-08 14:19:56 +00:00
Jacek Sieka
02b48fafad
remove unused / obsolete / unmaintained modules (#4298)
mostly this is chronicles-tail and its (extensive) dependencies along
with the simulation monitoring dashboard that is not maintained
2022-11-08 14:37:23 +01:00
Jacek Sieka
8297b962cc
Fix VC-based sync subnet subscriptions (#4293)
* move duty tracking code to `ActionTracker`
* fix earlier duties overwriting later ones
* re-run subnet selection when new duty appears
* log upcoming duties as soon as they're known (vs 4 epochs before)
2022-11-08 12:43:38 +01:00
tersec
909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
tersec
229b58b002
run capella slot sanity tests in CI (#4295) 2022-11-08 00:33:57 +00:00
tersec
0a43c89cd2
run capella block sanity tests in CI (#4292) 2022-11-07 18:37:48 +00:00
tersec
5b46f0b723
add Capella support to Forked* (#4276)
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
Jacek Sieka
d839b9d07e
State-only checkpoint state startup (#4251)
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](https://github.com/ethereum/beacon-APIs/pull/226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
2022-11-02 10:02:38 +00:00
Etan Kissling
aff9147c31
avoid packing attestations from other forks (#4273)
* avoid packing attestations from other forks

Revisit #3893 using method based on Lighthouse (less heavy computation).

* fix comment
2022-11-01 14:23:40 +02:00
tersec
0cfc1b776e
add all missing epoch transition tests (#4269) 2022-10-28 08:02:33 +00:00
tersec
48e351d672
run EF Capella consensus test fixtures (#4267) 2022-10-27 19:11:13 +00:00
tersec
7dd5c49c4e
use v1.3.0-alpha.0 test vectors (#4263) 2022-10-27 11:54:39 +00:00
tersec
06ccf5b80c
capella test vector support (#4261) 2022-10-27 06:29:24 +00:00
tersec
f9830836a9
deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten (#4241)
* deprecate --terminal-total-difficulty-override; remove launch script for deprecated ropsten

* remove Makefile support for Ropsten
2022-10-24 23:32:52 +03:00
tersec
fb6e6d9cf4
remove newPayload from block production flow (#4186)
* remove `newPayload` from block production flow

* refactor block_processor to run `newPayload` as part of `storeBlock`
2022-10-14 22:48:56 +03:00
Jacek Sieka
819442acc3
Allow chain dag without genesis / block (#4230)
* Allow chain dag without genesis / block

This PR enables the initialization of the dag without access to blocks
or genesis state - it is a prerequisite for implementing a number of
interesting features:

* checkpoint sync without any block download
* pruning of blocks and states

* backfill checkpoint block
2022-10-14 22:40:10 +03:00
Jacek Sieka
40bed02f60
Build block in parallel with attestation packing (#4185)
* fix block proposal in first slot after checkpoint
2022-10-04 11:24:16 +00:00
tersec
ad7541567c
move LVH handling to tests/; increase maximum fork choice retries (#4205) 2022-10-03 13:10:08 +00:00
tersec
0a4aa5fdb3
switch withStateAndBlck usage to forkyState; rm unused tests/mocking/ modules (#4206) 2022-10-03 13:08:50 +00:00
Jacek Sieka
af9ec577d0
nicer error message for failed backfill (#4188)
* nicer error message for failed backfill

Many checkpoint sources don't support block download

* RestGenericError -> RestErrorMessage

...and other assorted fixes to bring rest types closer to spec

* fix tests
2022-09-29 23:55:18 +03:00
Etan Kissling
5968ed586b
use LRU strategy for shuffling/epoch caches (#4196)
When EL `newPayload` is slow (e.g., Raspberry Pi with Besu), the epoch
and shuffling caches tend to fill up with multiple copies per epoch when
processing gossip and performing validator duties close to wall slot.
The old strategy of evicting oldest epoch led to the same item being
evicted over and over, leading to blocking of over 5 minutes in extreme
cases where alternate epochs/shuffling got loaded repeatedly.
Changing the cache eviction strategy to least-recently-used seems to
improve the situation drastically. A simple implementation was selected
based on single linked-list without a hashtable.
2022-09-29 14:55:58 +00:00
tersec
c367b14ad9
deprecate --safe-slots-to-import-optimistically (#4182) 2022-09-29 06:29:49 +00:00
tersec
1819d79e07
avoid potential database inconsistency after fcU INVALID+crash (#4192)
* avoid database race-condition inconsistency after fcU `INVALID` then crash

* ensure head doesn't fall behind finalized; add more tests for head movement/reloading DAG
2022-09-28 21:07:31 +00:00
Eugene Kabanov
8778e1cf8d
Fix REST generic error parsing. (#4189)
* Fix REST generic error parser.
* Unescape test vectors.
* Fix RestGenericError writer and tests, to encode `code` as `Number`.
2022-09-28 18:47:15 +00:00
Jacek Sieka
b1bc830a92
Harden EpochRef loading against bogus block root at tail (#4178)
* add more error information when things go wrong with database
* lower log level when reloading attestations from no-block epoch start
slot
2022-09-27 18:56:08 +02:00
tersec
0f6d19b4b3
implement v1.2.0 optimistic sync tests (#4174)
* implement v1.2.0 optimistic sync tests

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Etan Kissling <etan@status.im>

* `lvh` -> `latestValidHash` and only invalidate one specific block"

* `getEarliestInvalidRoot` -> `getEarliestInvalidBlockRoot`; `defaultEarliestInvalidRoot` -> `defaultEarliestInvalidBlockRoot`

Co-authored-by: Etan Kissling <etan@status.im>
2022-09-27 15:11:47 +03:00
Jacek Sieka
7f9af78ddb
test randao skippping (complements #3837) (#4179) 2022-09-27 09:22:24 +02:00
tersec
9750cd3a38
update state diffs to Bellatrix (#4177) 2022-09-26 19:13:50 +00:00
tersec
3c03ba86c1
update consensus spec ref URLs to v1.2.0 (#4164) 2022-09-23 07:56:06 +00:00
tersec
72e6b2021a
use v1.2.0 consensus spec test vectors (#4163) 2022-09-22 22:24:13 +00:00
zah
ad63bba446
Support Prysm and Ethdo Keystores (Fixes #4107) (#4149) 2022-09-20 01:09:56 +03:00
Eugene Kabanov
174292b7e4
Sync gaps fix (#4090) 2022-09-19 12:37:42 +03:00
Eugene Kabanov
ca871a5435
Fix HTTP/REST clients HTTP Content-Type header parsers. (#4139)
* Fix client HTTP content-type parsers.

* Fix tests.

* Address review comment and apply wildcard checks for generic decodeBytes.
2022-09-19 12:17:29 +03:00
Etan Kissling
9999362b11
detect mismatch of config and binary (#4132)
* detect mismatch of config and binary

When loading configuration that sets keys that Nimbus bakes into the
binary at compile-time, raise an error if the config is incompatible
instead of ignoring the conflicting value.
2022-09-19 12:07:46 +03:00
Jacek Sieka
ef8bab58eb
load suggested fee recipient file also when keymanager is disabled (#4078)
Since these files may have been created in a previous run or manually,
we want to keep loading them even on nodes that don't enable the
keystore API (for example static setups)

Other changes:

* log keystore loading progressively (#3699)
* print initial fee recipient when loading validators
* log dynamic fee recipient updates
2022-09-17 08:30:07 +03:00
Etan Kissling
3ba016d75f
consistent peer scoring for missing non-finalized parent (#3381)
When the sync queue processes results for a blocks by range request,
and the requested range contained some slots that are already finalized,
`BlockError.MissingParent` currently leads to `PeerScoreBadBlocks` even
when the error occurs on a non-finalized slot in the requested range.
This patch changes the scoring in that case to `PeerScoreMissingBlocks`
for consistency with range requests solely covering non-finalized slots,
and, likewise, rewinds the sync queue to the next `rewindSlot`.
2022-09-16 21:45:53 +02:00
tersec
0410aec9d8
remove rest of withState.state usage (#4120)
* remove rest of `withState.state` usage

* remove scaffolding
2022-09-16 15:35:00 +02:00
tersec
5b0b48f6e9
implement /eth/v1/validator/register_validator (#4115) 2022-09-13 14:52:26 +03:00
tersec
8be964a152
update consensus layer spec ref URLs to v1.2.0-rc.3 (#4109) 2022-09-10 17:16:38 +00:00
tersec
19bf460a3b
more withState state -> forkyState (#4104) 2022-09-10 08:12:07 +02:00
tersec
1d620f0123
consensus spec URL updates to v1.2.0-rc.3 (#4105) 2022-09-09 21:56:06 +00:00