Commit Graph

6686 Commits

Author SHA1 Message Date
Etan Kissling 12a2f8c026
when adding duplicates to quarantine, schedule deepest missing parent (#6112)
During sync, sometimes the same block gets encountered and added to
quarantine multiple times. If its parent is already known, quarantine
incorrectly registers it as missing, leading to re-download. This can
be fixed by registering the parent's deepest missing parent recursively.

Also increase the stickiness of `missing`. We only perform 4 attempts
within ~16 seconds before giving up. Very frequently, this is not enough
and there is no progress until sync manager kicks in even on holesky.
2024-03-21 18:41:05 +00:00
Etan Kissling 6f466894ab
answer `RequestManager` queries from disk if possible (#6109)
When restarting beacon node, orphaned blocks remain in the database but
on startup, only the canonical chain as selected by fork choice loads.
When a new block is discovered that builds on top of an orphaned block,
the orphaned block is re-downloaded using sync/request manager, despite
it already being present on disk. Such queries can be answered locally
to improve discovery speed of alternate forks.
2024-03-21 18:37:31 +01:00
Etan Kissling 9256db2265
use `LPProtocol.new` instead of `LPProtocol()` (#6117)
Avoid potenial issue with https://github.com/vacp2p/nim-libp2p/pull/1064#discussion_r1534021691
in a future dependency bump.
2024-03-21 17:53:59 +01:00
tersec 213076e4cd
Revert "Set default localBlockValueBoost to 10 (#6103)" (#6118)
This reverts commit d66a769135.
2024-03-21 15:13:58 +00:00
Etan Kissling 3d45c0575a
avoid resetting chain stall detection on lag spike (#6115)
During lag spike, e.g., from state replays, peer count can temporarily
drop significantly. Should not have to wait another 60 minutes in that
situation just to be back where one started.
2024-03-21 04:55:29 +01:00
Fredrik Svantes d66a769135
Set default localBlockValueBoost to 10 (#6103)
* Set default localBlockValueBoost to 10

* Updated local-block-value-boost in documentation to say 10 as default
2024-03-21 02:06:03 +00:00
dependabot[bot] 02ccfd488b
Bump black from 21.12b0 to 24.3.0 in /ncli (#6111)
Bumps [black](https://github.com/psf/black) from 21.12b0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/commits/24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-20 21:54:09 +00:00
Etan Kissling de2d205f61
use PR-3431 style fork choice on all networks (#6110)
To start phasing out Capella fork choice logic, set default to PR 3431.
A subsequent release can remove the fallback option.
2024-03-20 19:12:33 +01:00
Etan Kissling eb5acdb7dd
make sure `clearanceState` builds on top of `headState` in chain stall (#6108)
The `clearanceState` points to the latest resolved block, regardless of
whether that block is canonical according to fork choice. If chain is
stalled and we want to prepare for resuming validator duties, we need
a recent state according to fork choice to avoid lag spikes and missing
slot timings.
2024-03-20 16:41:56 +01:00
Etan Kissling e75b209076
Revert `nim-libp2p` to `28609597d104a9be880ed5e1648e1ce18ca9dc38` (#6107)
* Revert "bump `nim-libp2p` to `49a92e564167c0ffdcc86838c5e45cc985665d96` (#6084)"

This reverts commit 78f3e03d53.

* Revert "bump `nim-libp2p` to `ae13a0d58301159e6b3bfc43fe23986c254c741a` (#6065)"

This reverts commit 4a6ed0323e.
2024-03-20 13:46:12 +01:00
andri lim 1fe6efcf53
Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1 (#6106)
Unify EthCall/EthSend into TransactionArgs (#138)
2024-03-20 14:39:12 +07:00
Jacek Sieka 032b91c631
extend seen ttl to cover 2 epochs (#6098)
this allows us to drop these useless messages earlier in the pipeline

https://github.com/ethereum/consensus-specs/pull/3627
2024-03-20 08:07:16 +01:00
Etan Kissling 4c0b9efb30
fix chain stall detection (#6105)
Used the incorrect count for `numPeers` and did not account for heads on
alternate branches in in chain stall detection.
2024-03-20 04:51:55 +01:00
Etan Kissling 035ca015e6
continue validator duties if chain does not progress for a long time (#6101)
Nimbus currently stops performing validator duties if the blockchain
does not progress for `node.config.syncHorizon` slots. This means that
the chain won't recover because no new blocks are proposed. To fix that,
continue performing validator duties if no progress is registered for a
long time, and none of our peers is indicating any progress.
2024-03-20 03:23:53 +01:00
Etan Kissling 8b604b59a7
fix `BlockNumber` serialization (#6102)
Correct formatting of `BlockNumber` in EL manager, regression from
its conversion to `distinct` in #6088.
2024-03-19 22:14:08 +01:00
Etan Kissling 8514e4a44c
bump `gnosis-chain-configs` to `14d8439235fa757dd39b9fb1c10a06a99a720989` (#6100)
- Add more Chiado bootnodes for GnosisDAO
2024-03-19 16:32:48 +00:00
Etan Kissling 5d42859176
make `Gwei` `distinct` (#6090)
#6087 introduced a subtle change to `nim-web3` resulting in `Gwei` to be
serialized differently than before. Using a `distinct` type for `Gwei`
improves type safety and avoids such problems in the future.
2024-03-19 14:22:07 +01:00
Etan Kissling 1dd2c939ac
bump `nim-web3` to `80c7aa6de2a26c57fa1f06ad47f3ac6058e6545b` (#6088)
- Add writeValue for BlockNumber
- make `BlockNumber` `distinct`
2024-03-19 14:21:47 +01:00
Etan Kissling 595d110b37
avoid blocking deep reorgs > 64 epochs (#6099)
On Goerli there are some instances of long streaks of empty epochs due
to different branches being built in parallel. They sometimes lead to
`Request for pruned historical state` logs requiring a BN restart to
resolve. Avoid that by trying to restore states from the entire non-
finalized history, to avoid losing sync in such situtions.
2024-03-19 14:21:25 +01:00
Miran e1aa9e6de5
fix commit hash when publishing book (#6097)
Before, the commit hash was taken from the `gh-pages` branch,
instead from the branch used to publish the book.
2024-03-19 09:56:47 +01:00
Jacek Sieka ed1ef19bf4
use `assign` for forky state assignment (#6055) 2024-03-19 09:50:25 +01:00
Etan Kissling d4d27164f9
bump `nim-sqlite3-abi` to `1453b19b1a3cac24002dead15e02bd978cb52355` (#6096)
- bump `sqlite-amalgamation` to `3.45.2`
2024-03-18 00:17:29 +01:00
Etan Kissling d22dfaed41
bump `nim-ssz-serialization` to `0fc5e49093fa8d3c07476738e3257d0d8e7999a3` (#6095)
- more fixes for `distinct` integer types
- avoid double testing `--mm:refc`
2024-03-18 00:14:19 +01:00
Etan Kissling f40083f1e5
annotate `validator_db_aggregator` with `{.raises.}` (#6094)
Show proper error message when `validator_db_aggregator` raises errors.
2024-03-17 16:17:07 +01:00
Etan Kissling ef2411e1a0
use correct `INACTIVITY_SCORE_RECOVERY_RATE` if overridden from default (#6091)
When a config defines a different `INACTIVITY_SCORE_RECOVERY_RATE` than
the default, `process_inactivity_updates` uses an incorrect rate ever
since #2710 when `INACTIVITY_SCORE_RECOVERY_RATE` became configurable.
2024-03-17 13:32:30 +01:00
Etan Kissling 4aea780320
bump `nim-ssz-serialization` to `9bb15468c64851e9300ccab662f16a15be6d833e` (#6089)
- use `toSszType` for elements of `HashList|HashArray|List|array`
2024-03-17 02:46:49 +01:00
Etan Kissling 30460aad9c
bump `nim-chronos` to `47cc17719f4293bf80a22ebe28e3bfc54b2a59a1` (#6083)
- print warning when calling failed
2024-03-16 15:38:17 +01:00
Etan Kissling 74a238460b
bump `nim-json-rpc` to `ad8721e0f3c6925597b5a93b6c53e040f26b5fb3` (#6086)
- Export errors for json-rpc clients
2024-03-16 04:05:44 +00:00
Etan Kissling 448e610f8a
bump `nim-presto` to `a9687dda1c3e20d5b066d42b33c2a63f018af93f` (#6085)
- Add examples
2024-03-16 03:46:37 +00:00
Etan Kissling 78f3e03d53
bump `nim-libp2p` to `49a92e564167c0ffdcc86838c5e45cc985665d96` (#6084)
- default `MultiAddress` param for `newStandardSwitch` does not raise
- clean up triple lookup and avoid `KeyError` when adding muxer
- `{.async: (raises).}` for `relay/utils.nim`
- `{.async: (raises).}` annotations for `protocols/secure`
- avoid pointless exception raising in `dcutr/server`
2024-03-16 02:25:40 +00:00
Etan Kissling 82b8c96f72
bump `nim-results` to `e2adf66b8bc2f41606e8469a5f0a850d1e545b55` (#6082)
- Formatted with nph v0.5.1-0-gde5cd48
- Update CI
- ci: Combine c/c++
- extend `optValue` support for `Result[void, E]`
- Document a few fixes
2024-03-16 02:15:03 +00:00
Etan Kissling 7a7c024534
bump `nim-libbacktrace` to `027570111c161d8378bca9e84b5f75500a8c38a3` (#6081)
- bump `libbacktrace` to `7ead8c1ea2f4aeafe9c5b9ef8a9461a9ba781aa8`
2024-03-16 02:22:20 +01:00
Etan Kissling 2d52016e5c
bump `nim-stew` to `a0c085a51fe4f2d82aa96173ac49b3bfe6043858` (#6079)
- strformat: compile-time format string parser (backport Nim 2.2)
2024-03-16 02:08:54 +01:00
Etan Kissling b3bce7ce79
bump `nim-stint` to `3c238df6cd4b9c1f37a9f103383e7d2bbd420c13` (#6078)
- fix noInit to noinit; use evergreen GitHub Actions image versions
2024-03-16 01:53:35 +01:00
Etan Kissling 4a6ed0323e
bump `nim-libp2p` to `ae13a0d58301159e6b3bfc43fe23986c254c741a` (#6065)
- Send priority with queue fix
2024-03-15 22:49:01 +01:00
Etan Kissling 1bd5819dad
cache head block eligibility for fork choice (#6076)
When there are long periods of non-finality, `nodeIsViableForHead` has
been observed to consume significant time as it repeatedly walks the
non-finalized check graph as part of determining what heads are eligible
for fork choice. Caching the result resolves that.

Overall, it may still be better to prune fork choice more aggressively
when finality advances, to fully avoid the case specced out using the
linear scan. The current implementation is very close to spec, though,
so such a change should not be introduced without thorough testing.

The simple cache should allow significantly better performance on Goerli
while the network is still supported (Mid April).
2024-03-15 22:48:18 +01:00
tersec 0a6d189161
automated consensus spec URL updating to v1.4.0 (#6074) 2024-03-14 07:26:36 +01:00
Eugene Kabanov 72c844534f
Add Keymanager API graffiti endpoints. (#6054)
* Initial commit.

* Add more tests.

* Fix API mistypes.

* Fix mistypes in tests.

* Fix one more mistype.

* Fix affected tests because of error code 401.

* Add GetGraffitiResponse object.

* Add more tests.

* Fix compilation errors.

* Recover old behavior.

* Recover old behavior.

* Fix mistype.

* Test could not know default graffiti value.

* Make VC use adopted graffiti settings.

* Make BN use adopted graffiti settings.

* Update Alltests.

* Fix test.

* Revert "Fix test."

This reverts commit c735f855d3cb9c4a1c8e8af29d3f4438d068e31f.

* Workaround {.push raises.} requirement.

* Fix comment.

* Update Alltests.
2024-03-14 03:44:00 +00:00
tersec c3016a9bc5
use v1.4.0 consensus spec test vectors (#6072) 2024-03-13 21:42:40 +01:00
Etan Kissling 4e2ffca44a
use fixed max depth for `BlockRef` (#6070)
In `block_dag` there is a max depth of 100 years configured to detect
internal inconsistencies, e.g., circular references. As `BlockRef` was
changed long ago to only reflect the non-finalized chain segment, the
theoretically supported max depth can be reduced and simplified.
2024-03-13 13:01:51 +01:00
Etan Kissling efb1971d7e
pass `cfg` to `ETHBeaconClockCreateFromState` LC API (#6071)
We don't need the `cfg` right now, but it makes sense to have the object
passed to the clock so that the API doesn't break if we want to support
configurable `SECONDS_PER_SLOT`. As the `libnimbus_lc` library is not
yet widely used, better to add the argument now than later.
2024-03-13 13:01:39 +01:00
Etan Kissling 8bd8ffe2bb
align default `syncHorizon` computation logic across networks (#6066)
The `syncHorizon` describes the number of empty slots before the beacon
node considers itself to be out of sync. There are two places where we
currently set this to 50 slots, but it makes more sense to base it on
wall time, e.g., the 10 minutes that the default 50 are derived from.
2024-03-12 21:51:18 +01:00
kaiserd e56190c7aa
chore(vendor): update nim-libp2p path (#6069)
Signed-off-by: ksr <kaiserd@users.noreply.github.com>
Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2024-03-12 20:57:21 +01:00
Etan Kissling 659be0b08e
bump `nim-kzg4844` to `057f7c653e1abe91cca9aac2f94832f39228ea98` (#6067)
- bump `c-kzg-4844` to `v1.0.0`
2024-03-12 18:32:04 +01:00
Etan Kissling 79b230828d
bump `nim-unittest2` to `e96f3215030cbfa13abc2f5827069b6f8ba87e38` (#6063)
- release `v0.2.2`
2024-03-12 03:35:37 +01:00
tersec 707409b8d3
make EL light client disk usage table consistent with hardware table (#6064) 2024-03-12 02:34:40 +00:00
Etan Kissling 1a9cafd3dd
bump `nim-libp2p` to `28609597d104a9be880ed5e1648e1ce18ca9dc38` (#6062)
- fix: allFuturesThrowing compilation issue on daily
- feat: Yamux timeout
- Fix empty path crash issue for MultiAddresses `unix`, `ip6zone`, `dns***`.
- fix: move transport interop tests to nim-libp2p repo
- use chronos 4.0.0
- fix typo in `ProtoMessage.toString()`
- increase tolerance of `simple heartbeat` test
- remove unused `MultiBase.encode(..., Cid)` function
- move `allFutureThrowing` helper to tests
- use new Chronos `trackCounter` APIs for leaks checks in tests
- remove unused `LPStreamError` types
- change `SecioError` and `NoiseError` to descendants of `LPStreamError`
- do not log yamux buffers without sanitization (trace log level)
- don't forget closing the stream when final `{Fin}` fails in yamux
- use `race` instead of `or` to avoid lockup
- in yamux, do not write `{Rst}` packet to stream that's in use
- avoid `KeyError` in edge case of yamux handler
- avoid triple lookup in `m.flushed` yamux table
- catch `WebSocketError` in `wstransport`
- define proper parent error type for `YamuxError`
- document known `--mm:orc` crash
- add `{.async: (raises).}` to `libp2p/stream` modules
2024-03-12 03:28:51 +01:00
tersec fd8429da74
rm Web3Signer Bellatrix block support (#6061) 2024-03-11 22:09:31 +00:00
Jacek Sieka 7d06edb7c9
Mention blob retention policy in docs (#6058) 2024-03-11 20:07:41 +00:00
Eugene Kabanov f088e5f57b
Consensus block value calculation for produceBlockV3 API call. (#5873)
* allow specifying get_proposer_reward block root at state.slot

* Add consensus_block_value calculation.

* Address review comments.

* Post-rebase adjustments.

* Use proper state to calculate consensus block value.

* Revert "allow specifying get_proposer_reward block root at state.slot"

This reverts commit 9fef9a8199f63056060527ac2531acc3b0ed8dcb.

* Fix post-revert problems.
Return back to Gwei.

* Adding test which is not working.

* Do not use test suite if it does not have post-state.

* Add debug logging.

* Increase logging to track sources of balance changes.

* Fix sync committee rewards/penalties calculation.

* Revert "Increase logging to track sources of balance changes."

This reverts commit 32feb20f2fdb66521401710866cd59ecc9951ef8.

* Adopt new vision to block rewards.

* Add block produce logging to VC.

* Remove rewards.nim.

* Eliminate toWei changes.

* Improve UInt256 shortLog.

* Fix conversion procedure.

* Address review comments.

* Fix test.

* Revert "Fix test."

This reverts commit 4948b2c1ec.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
Co-authored-by: Etan Kissling <etan@status.im>
2024-03-11 14:18:50 +00:00