4907 Commits

Author SHA1 Message Date
Jacek Sieka
0fbea67cb7
grafana: remove container 2022-11-16 12:50:03 +01:00
Jacek Sieka
4925776e66
export dashboards properly
using export function
2022-11-16 12:38:52 +01:00
Jacek Sieka
0bccef68bd
update dashboards 2022-11-16 11:54:53 +01:00
Etan Kissling
bee3c4d68e
properly stop EL instances from prior testnets (#4328)
Launching multiple local testnet simulation sequentially can lead to
existing EL processes from prior failed/aborted runs not being stopped
properly, subsequently leading to hard-to-debug CI test failures.
Fixing the cleanup logic addresses this problem.
2022-11-16 12:12:20 +02:00
Jacek Sieka
aac61165d5
ncli_db: better error message on missing history 2022-11-16 10:12:50 +01:00
Jacek Sieka
64322a3d45
prater: update CONFIG_NAME (#4322) 2022-11-15 17:35:48 +00:00
Jacek Sieka
1b90f65f7b
doc updates (#4314)
* optimise index / landing page real estate, add feature list
* fee recipient in quickstart
* recommend http over websockets  (fewer command line options needed)
* simplify PBS, fee recipient docs
* add VC role / sentry node docs (#4140)
* Update docs/the_nimbus_book/src/rest-api.md

Co-authored-by: tersec <tersec@users.noreply.github.com>
2022-11-13 07:59:55 +01:00
Jacek Sieka
367a13df34
stew: bump (#4323)
fix Opt constants, cleanups
2022-11-11 18:07:34 +00:00
tersec
b3f6be71d5
refactor makeBeaconBlock; some capella support for ncli_db and wss_sim (#4321) 2022-11-11 15:37:43 +01:00
Etan Kissling
93714756d7
update sync_queue docs w.r.t. joker's problem (#4317)
Explicitly mention in-line documentation within `sync_queue` relating to
older specification version to make rationale clearer.
2022-11-11 15:36:02 +01:00
Etan Kissling
94ff73af34
remove unused SyncQueue.getFullMap function (#4319)
The `getFullMap` function is unused and can be removed (verified with
`{.deprecated.}`).
2022-11-11 12:36:56 +00:00
Etan Kissling
2eb56f6e1b
rename PeerScoreXyzBlocks -> PeerScoreXyzValues (#4318)
The various `PeerScore` constants are used for both beacon blocks and
LC objects, and will likely also find use for EIP4844 blob sidecars.
Renaming them to use more generically applicable names not referring
to blocks explicitly aymore.
2022-11-11 11:34:28 +00:00
Etan Kissling
94b3df25f8
remove unused sync_queue types (#4316)
`SyncManagerError` and `sync_queue.BeaconBlocksRes` are unused and can
be removed for cleanup (verified with `{.deprecated.}`).
2022-11-11 10:41:30 +00:00
tersec
35b1104bea
block_sim runs capella by default (#4315) 2022-11-11 10:17:27 +00:00
tersec
04cbea754b
don't require attached validator for blinded block BN endpoint (#4313) 2022-11-10 20:18:08 +00:00
Etan Kissling
48994f67d3
rename BlockError -> VerifierError (#4310)
We currently use `BlockError` for both beacon blocks and LC objects.
In light of EIP4844, we will likely also use it for blob sidecars.
To avoid confusion, renaming it to a more generic `VerifierError`,
and update its documentation to be more generic.

To avoid long lines as a followup, also renaming the `block_processor`'s
`BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and
`BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
Kim De Mey
2256a22594
Bump nim-eth for prettier logs in discv5 (#4312) 2022-11-10 16:37:37 +01:00
Jacek Sieka
09ade6d33d
Make trusted node sync era-aware (#4283)
This PR removes a bunch of code to make TNS aware of era files, avoiding
a duplicated backfill when era files are available.

* reuse chaindag for loading backfill state, replacing the TNS homebrew
* fix era block iteration to skip empty slots
* add tests for `can_advance_slots`
2022-11-10 10:44:47 +00:00
tersec
a63ec842bc
add light client capella CI tests (#4311) 2022-11-10 10:23:46 +00:00
Jacek Sieka
1691d89695
3x key loading speedup via blst sha256 (refs #4277) (#4301)
* 3x key loading speedup via blst sha256 (refs #4277)

* readd sha2
2022-11-10 00:05:50 +02:00
tersec
90eb2ccb20
database and fork choice test runner support for capella (#4309) 2022-11-09 17:32:10 +00:00
tersec
0919b8689e
run capella fork transition tests in CI (#4307) 2022-11-09 12:28:34 +00:00
Zahary Karadjov
651f01722c
Disable discovery in the Keymanager API tests 2022-11-09 13:01:54 +02:00
tersec
5c0ad1b5c8
automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4305) 2022-11-09 09:20:53 +00:00
Jacek Sieka
83c78c8002
use mainnet as default config (#4302)
Without this, tools like `ncli_db` require command line parameter to
process mainnet blocks, which is a regression since the merge - this is
a subset of #4147.
2022-11-08 22:53:02 +00:00
Jacek Sieka
b170a09c47
remove news leftovers (#4299) 2022-11-08 20:06:54 +00:00
tersec
a4d2637259
implement /eth/v1/beacon/blinded_blocks BN endpoint (#4286)
* implement /eth/v1/beacon/blinded_blocks BN endpoint

* avoid function names in non-debug logs

* callers log, so callee can just return err()
2022-11-08 18:08:43 +00:00
Jacek Sieka
38651d1bfe
bump nim-eth (#4300)
* removal of eth1 code
* various discovery fixes
* nim-devel fixes
2022-11-08 16:45:56 +00:00
Etan Kissling
7ad610f6d7
set fee recipient in empty payload fallback (#4291)
When the EL/Builder fails to produce an execution payload, we fall back
to an empty `ExecutionPayload`. Even though it contains no transactions
it should refer to the configured fee recipient. This is useful for
privacy reasons (do not reveal the reason for the empty payload) and for
compliance with additional fee recipient rules by staking pools.
2022-11-08 14:19:56 +00:00
Jacek Sieka
02b48fafad
remove unused / obsolete / unmaintained modules (#4298)
mostly this is chronicles-tail and its (extensive) dependencies along
with the simulation monitoring dashboard that is not maintained
2022-11-08 14:37:23 +01:00
Jacek Sieka
8297b962cc
Fix VC-based sync subnet subscriptions (#4293)
* move duty tracking code to `ActionTracker`
* fix earlier duties overwriting later ones
* re-run subnet selection when new duty appears
* log upcoming duties as soon as they're known (vs 4 epochs before)
2022-11-08 12:43:38 +01:00
Jacek Sieka
e98cfa88cb
bumps (#4297)
* fix toml overflow handling
* random nim devel fixes
* unicode db update
2022-11-08 10:56:06 +00:00
tersec
909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
tersec
229b58b002
run capella slot sanity tests in CI (#4295) 2022-11-08 00:33:57 +00:00
Jacek Sieka
fae3ed5067
log block number / fee recipient (#4294) 2022-11-07 22:16:47 +00:00
tersec
0a43c89cd2
run capella block sanity tests in CI (#4292) 2022-11-07 18:37:48 +00:00
Zahary Karadjov
26fe3990b9
Show last 50 lines on test failure instead of last 10 2022-11-04 14:30:29 +02:00
zah
865637e8d8
Show last 10 lines from the test log file on test failure (#4284) 2022-11-04 01:11:11 +00:00
Eugene Kabanov
8417b7e064
Sync committee subscription fixes. (#4281)
* Preparing code.

* Fix prepareXXX procedures to use `onceToAll` strategy only.

* Remove lighthouse like subscription code.

* Address review comments.
2022-11-03 20:23:33 +01:00
Tanguy
865a930f65
Bump libp2p (#4282) 2022-11-03 20:22:53 +01:00
tersec
5b46f0b723
add Capella support to Forked* (#4276)
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
tersec
cee5a73a31
v1/beacon/blocks/{block_id}, v1/debug/beacon/states/{state_id}, and v1/validator/blocks/{slot} deprecated (#4279)
* v1/beacon/blocks/{block_id}, v1/debug/beacon/states/{state_id}, and v1/validator/blocks/{slot} deprecated

* Update beacon_chain/rpc/rest_constants.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update beacon_chain/rpc/rest_constants.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
2022-11-02 10:56:55 +00:00
Jacek Sieka
fc724b21e8
Tighten libp2p request quotas (#4254)
To further tighten Nimbus against spam, this PR introduces a global
quota for block requests (shared between peers) as well as a general
per-peer request limit that applies to all libp2p requests.

* apply request quota before decoding message
* for high-bandwidth requests (blocks), apply a shared global quota
which helps manage bandwidth for high-peer setups
* add metrics
2022-11-02 10:46:53 +00:00
Jacek Sieka
d839b9d07e
State-only checkpoint state startup (#4251)
Currently, we require genesis and a checkpoint block and state to start
from an arbitrary slot - this PR relaxes this requirement so that we can
start with a state alone.

The current trusted-node-sync algorithm works by first downloading
blocks until we find an epoch aligned non-empty slot, then downloads the
state via slot.

However, current
[proposals](https://github.com/ethereum/beacon-APIs/pull/226) for
checkpointing prefer finalized state as
the main reference - this allows more simple access control and caching
on the server side - in particular, this should help checkpoint-syncing
from sources that have a fast `finalized` state download (like infura
and teku) but are slow when accessing state via slot.

Earlier versions of Nimbus will not be able to read databases created
without a checkpoint block and genesis. In most cases, backfilling makes
the database compatible except where genesis is also missing (custom
networks).

* backfill checkpoint block from libp2p instead of checkpoint source,
when doing trusted node sync
* allow starting the client without genesis / checkpoint block
* perform epoch start slot lookahead when loading tail state, so as to
deal with the case where the epoch start slot does not have a block
* replace `--blockId` with `--state-id` in TNS command line
* when replaying, also look at the parent of the last-known-block (even
if we don't have the parent block data, we can still replay from a
"parent" state) - in particular, this clears the way for implementing
state pruning
* deprecate `--finalized-checkpoint-block` option (no longer needed)
2022-11-02 10:02:38 +00:00
Etan Kissling
aff9147c31
avoid packing attestations from other forks (#4273)
* avoid packing attestations from other forks

Revisit #3893 using method based on Lighthouse (less heavy computation).

* fix comment
2022-11-01 14:23:40 +02:00
tersec
3ef09ff596
Support /eth/v1/validator/blinded_blocks (#4272)
* Support BN endpoints for producing blinded blocks

* use correct endpoint version

* serve either JSON or SSZ versions of endpoint
2022-10-31 18:39:03 +01:00
Eugene Kabanov
00f083785d
VC: Fix for #4116 (external block builders support) (#4260) 2022-10-29 11:00:51 +02:00
tersec
0cfc1b776e
add all missing epoch transition tests (#4269) 2022-10-28 08:02:33 +00:00
tersec
48e351d672
run EF Capella consensus test fixtures (#4267) 2022-10-27 19:11:13 +00:00
Jacek Sieka
1572814a0c
Highlight execution client for optimistically synced node (#4266)
...such that the user knows where to look
2022-10-27 17:22:32 +00:00