Commit Graph

5161 Commits

Author SHA1 Message Date
tersec 9f0fb4742a
bump nim-stint to avoid compiler deprecation warnings (#4600) 2023-02-07 16:20:24 +00:00
Jacek Sieka 856fcea8d7
fix slow checking of unknown validators (#4590)
We do a linear scan of all pubkeys for each validator and slot - this
becomes expensive with large validator counts.

* normalise BN/VC validator startup logging
* fix crash when host cannot be resolved while adding remote validator
* silence repeated log spam for unknown validators
* print pubkey/index/activation mapping on startup/validator
identification
2023-02-07 14:53:36 +00:00
Jacek Sieka 98db005c70
seed sync committee state cache (#4592)
By pre-seeding the sync committee cache when applying blocks, we avoid a
significantly expensive validator set traversal / sync committee index
construction during sync / block application - 20-30% sync speedup
post-altair.

* also cache/reload total active balance for another cool 10%
2023-02-07 11:22:22 +01:00
Etan Kissling 171a185e4d
bump `merge-testnets` (#4598)
Updates `merge-testnets` with fix for Ropsten (#4593) and with correct
`CAPELLA_FORK_VERSION`.
2023-02-07 01:00:37 +01:00
tersec ebb45e10da
initial implementation of capella MEV (#4567) 2023-02-06 19:07:30 +01:00
Jakub Sokołowski b79d267b8f
drop dependency on lsb-release script (#4597)
The `/etc/os-release` file exists in most distributions and can be
easily read in Bash by sourcing it:
```
 > docker run --rm -it debian:bullseye
root@2f5d6e038738:/# grep '^ID=' /etc/os-release
ID=debian
```
```
 > docker run --rm -it ubuntu:22.04
root@316b572b6e4d:/# grep '^ID=' /etc/os-release
ID=ubuntu
```
The dependency on `lsb-release` tool
is unnecessary, and pulls in additional big dependencies like `python3`:
```
 # apt show lsb-release | grep Depends
Depends: python3:any, distro-info-data
```
Which if used in a Docker container would make it unnecessarily big.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-06 11:43:24 +01:00
Jakub Sokołowski c81352ce4e
ignore systemctl failure in after_install scripts (#4596)
Otherwise installation in Docker containers fails with:
```
...
Adding new user `nimbus' (UID 101) with group `nimbus' ...
Not creating home directory `/home/nimbus'.
/var/lib/dpkg/info/nimbus-beacon-node.postinst: 39: systemctl: not found
dpkg: error processing package nimbus-beacon-node (--configure):
 installed nimbus-beacon-node package post-installation script subprocess returned error exit status 127
Errors were encountered while processing:
 nimbus-beacon-node
E: Sub-process /usr/bin/dpkg returned an error code (1)
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-06 11:01:57 +01:00
Jacek Sieka f3ddea6c86
Skip execution payload verification for finalized blocks (#4591)
While syncing the finalized portion of the chain, the execution client
cannot efficiently sync and most of the time returns `SYNCING` - in this
PR, we use CL-verified optmistic sync as long as the block is claimed to
be finalized, only occasionally updating the EL with progress.

Although a peer might lie about what is finalized and what isn't,
eventually we'll call the execution client - thus, all a dishonest
client can do is delay execution verification slightly. Gossip blocks in
particular are never assumed to be finalized.
2023-02-06 08:22:08 +01:00
tersec 4c845b9749
note 8GB requirement is for building; fix macOS supported version (#4594) 2023-02-06 08:21:11 +01:00
henridf 9233a52e4b
Don't pass RuntimeConfig to RequestManager (#4589) 2023-02-04 12:13:09 +01:00
tersec bca781b1b0
prioritize REST API-provided BLS to execution changes over gossip-received changes (#4580) 2023-02-03 16:28:28 +01:00
tersec 1c62a5eb24
capella VC support (#4586) 2023-02-03 16:12:11 +01:00
tersec 956aee2d35
fill some capella/EIP4844 missing implementations (#4585) 2023-02-02 22:24:06 +00:00
tersec eb56762d41
some proc -> func in signatures_batch (#4582) 2023-02-02 19:05:30 +00:00
Andy Tudhope caa9eec394
Update hardware.md (#4584)
Due to increased requirements when building beacon nodes.

Though this is annoying, I still think you all are wonderful and magical beings.
2023-02-02 18:14:54 +01:00
tersec 63ed5885ab
update engine API URLs to v1.0.0-beta.2 (#4579) 2023-02-01 18:49:36 +00:00
tersec bcc9781cc7
rm obsolete interop module (#4570) 2023-02-01 16:29:55 +01:00
henridf 94837caa2a
Eip4844 sync fixes (#4577)
* fixes

* Make some log messages blob-aware

* remove redundant optBlobs()
2023-02-01 14:14:50 +00:00
henridf d7316ac863
Wire up blob fetching with request manager (#4527)
* Wire up blob fetching and request manager

* Update beacon_chain/sync/request_manager.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

* Review feedback: remove redundant RequestManager field.

* Fix syntax

* Improve blob fetching logic

* fix previous commit

* rman.isBlobsTime(): Remove spurious parameter

* Review feedback

* expressionify an if

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-02-01 00:25:08 +01:00
tersec cdf69b9360
more automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4572) 2023-01-31 22:49:17 +00:00
tersec c71bddb612
try not requiring brew install gnu-getopt (#4569) 2023-01-31 15:22:57 +02:00
Etan Kissling ea6a6b1acd
track slot as part of fork choice debug API (#4565)
Extends fork choice state to also track slot numbers to improve accuracy
of `/eth/v1/debug/fork_choice` endpoint. Autoenable this API on devnet,
and disable some extra checks on devnet to aid focused testing efforts.
Align fork choice pruning logic with API based on checkpoints vs root.
2023-01-31 12:35:01 +00:00
tersec 58ed9308d2
automated v1.3.0-rc.1 to v1.3.0-rc.2 consensus spec URL updates (#4568) 2023-01-31 00:26:57 +01:00
tersec f5e302f973
use 1.3.0-rc.2-hotfix consensus spec test vectors (#4566) 2023-01-30 22:54:16 +00:00
tersec b2f415cbe5
remove merge cruft (#4559) 2023-01-30 20:22:05 +01:00
tersec 8e2792bdd7
remove Nim 1.2 workarounds and `TODO`s (#4533)
* clean up some Nim 1.2 workarounds

* re-add notes about JS backend

* another proc/noSideEffect -> func

* revert ncli/ncli_common.nim changes; 19969 evidently wasn't backported to 1.6
2023-01-30 20:21:51 +01:00
tersec 29ea42ce1a
use v1.3.0-rc.2 consensus spec test vectors (#4564) 2023-01-29 11:28:04 +00:00
Etan Kissling 258f151594
prepare LC update ranking test runner for Capella (#4563)
* prepare LC update ranking test runner for Capella

Adds missing Capella/EIP4844 support to LC update ranking runner.

* fix uncommitted typo

* skip capella / eip4844 tests

* cleanup test skip
2023-01-29 03:34:26 +00:00
tersec 0fb726c420
`BeaconStateFork/BeaconBlockFork` -> `ConsensusFork` (#4560)
* `BeaconStateFork/BeaconBlockFork` -> `ConsensusFork`

* revert unrelated change

* revert unrelated changes

* update test summaries
2023-01-28 19:53:41 +00:00
henridf 67e2f854d5
Fix BeaconBlockAndBlobsSidecar encoding (#4562) 2023-01-28 12:30:38 +00:00
tersec 7635b47a4e
update/clarify Nim issue link after retesting/re-isolating 2023-01-27 10:41:19 +00:00
Etan Kissling 9f279e2933
cache LC headers and sync committees for bootstrap (#4499)
To allow LC data retention longer than the one for historic states,
introduce persistent DB caches for `current_sync_committee` and
`LightClientHeader` for finalized epoch boundary blocks.
This way, historic `LightClientBootstrap` requests may still be honored
even after pruning. Note that historic `LightClientUpdate` requests are
already answered using fully persisted objects, so don't need changes.

Sync committees and headers are cached on finalization of new data.
For existing data, info is lazily cached on first access.

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-01-27 10:44:57 +01:00
Zahary Karadjov daa99f43c3
Merge branch 'stable' into unstable 2023-01-26 16:47:26 +01:00
Eugene Kabanov 08ed8ad43e
Adopt BN and VC header sizes and requirements, to avoid users confusion with default configuration options. (#4556)
Add comments.
2023-01-26 16:00:10 +01:00
tersec c22384d7ba
always use Bellatrix message size limits (#4558) 2023-01-26 14:24:23 +00:00
Zahary Karadjov 1ea102b033
Version 23.1.1 2023-01-26 14:37:03 +01:00
Jacek Sieka 0c1686eb20
fix pruned backfill summary (fixes #4552)
When enabling pruned history mode, the backfill summary as it was
computed with full backfilling may be pruned - don't try to load it on
init
2023-01-26 14:23:12 +01:00
Jacek Sieka fa27ce76a3
vmon: set details mode to default off
The "on" default for validator monitor details incurs a heavy
performance penalty on large-validator setups - this may cause excess
memory usage or slowdowns when metrics are queried - this PR changes the
default to off, as was intended for the 23.1.0 release.
2023-01-26 14:18:52 +01:00
Jacek Sieka 968e27bc97
fix pruned backfill summary (fixes #4552) (#4554)
When enabling pruned history mode, the backfill summary as it was
computed with full backfilling may be pruned - don't try to load it on
init
2023-01-26 11:23:17 +01:00
henridf 03b468c537
Pass correct block root to validate_blobs_sidecar (#4557) 2023-01-26 09:25:37 +00:00
henridf 7966ab6be2
Some EIP4844 fixes (#4549)
* debug log upon sidecar validation failure

* Fill in signature catch upon SignedBeaconBlockAndBlobsSidecar deser

* Always fill blobssidecar slot and root

* Skip lastFCU when eth1monitor is nil

* fix

* Use cached root
2023-01-25 18:35:46 +01:00
henridf d349a4797f
Fix minimum_request_epoch in BeaconBlockAndBlobsSidecarByRoot (#4555) 2023-01-25 18:33:01 +01:00
tersec d30544f790
disallow filling BLS to execution change pool pre-capella (#4547)
* disallow filling BLS to execution change pool pre-capella

* use better BLS change epoch
2023-01-24 22:25:51 +01:00
Etan Kissling efbd4e395a
avoid sending redundant LC finality updates (#4546)
When the epoch boundary block is missed, we incorrectly assume that the
next couple blocks improve finality, leading to repeated pushes of the
same light client finality update and incorrectly ignoring some gossip.
2023-01-24 17:44:55 +00:00
tersec fe1a57c220
use shortLog for execution payload logging (#4544) 2023-01-24 13:19:38 +00:00
Jacek Sieka ba0496105a
ncli_db: fix readonly properly (#4543)
🐡

Co-authored-by: Etan Kissling <etan@status.im>
2023-01-24 02:52:08 +00:00
Etan Kissling 0b8bb11c50
combine common implementation of LC helpers (#4542)
* combine common implementation of LC helpers

Combine replicated helper code from Altair/Capella/EIP4844 into single
`Forky` based implementation. Also convert `template` to `func` to avoid
selection of incorrect overload.

* fix
2023-01-23 19:38:03 +00:00
Zahary Karadjov 285eec6512
Add metrics and debug logging for dropped BLS to execution change messages 2023-01-23 14:58:40 +01:00
henridf 349001b7fb
eip4844 beacon block proposals (#4540)
* eip4844 beacon block proposals

* Don't fetch blobs under minimal preset

@tersec's summary of the issue:

BlobsBundleV1 in the execution API spec assumes a mainnet preset blob
size, where the EIP4844 consensus spec defines
FIELD_ELEMENTS_PER_BLOB: 4 under the minimal preset, which leads to a
Blob having a length of 4 * 32, not 4096 * 32 which BlobsBundleV1
requires.

* Revert unintentional script change
2023-01-21 23:13:21 +00:00
tersec 90e169869c
bump nim-stew for Results field access workaround (#4541) 2023-01-21 19:40:22 +00:00