Commit Graph

3986 Commits

Author SHA1 Message Date
Jacek Sieka ed1ef19bf4
use `assign` for forky state assignment (#6055) 2024-03-19 09:50:25 +01:00
Etan Kissling ef2411e1a0
use correct `INACTIVITY_SCORE_RECOVERY_RATE` if overridden from default (#6091)
When a config defines a different `INACTIVITY_SCORE_RECOVERY_RATE` than
the default, `process_inactivity_updates` uses an incorrect rate ever
since #2710 when `INACTIVITY_SCORE_RECOVERY_RATE` became configurable.
2024-03-17 13:32:30 +01:00
Etan Kissling 1bd5819dad
cache head block eligibility for fork choice (#6076)
When there are long periods of non-finality, `nodeIsViableForHead` has
been observed to consume significant time as it repeatedly walks the
non-finalized check graph as part of determining what heads are eligible
for fork choice. Caching the result resolves that.

Overall, it may still be better to prune fork choice more aggressively
when finality advances, to fully avoid the case specced out using the
linear scan. The current implementation is very close to spec, though,
so such a change should not be introduced without thorough testing.

The simple cache should allow significantly better performance on Goerli
while the network is still supported (Mid April).
2024-03-15 22:48:18 +01:00
tersec 0a6d189161
automated consensus spec URL updating to v1.4.0 (#6074) 2024-03-14 07:26:36 +01:00
Eugene Kabanov 72c844534f
Add Keymanager API graffiti endpoints. (#6054)
* Initial commit.

* Add more tests.

* Fix API mistypes.

* Fix mistypes in tests.

* Fix one more mistype.

* Fix affected tests because of error code 401.

* Add GetGraffitiResponse object.

* Add more tests.

* Fix compilation errors.

* Recover old behavior.

* Recover old behavior.

* Fix mistype.

* Test could not know default graffiti value.

* Make VC use adopted graffiti settings.

* Make BN use adopted graffiti settings.

* Update Alltests.

* Fix test.

* Revert "Fix test."

This reverts commit c735f855d3cb9c4a1c8e8af29d3f4438d068e31f.

* Workaround {.push raises.} requirement.

* Fix comment.

* Update Alltests.
2024-03-14 03:44:00 +00:00
tersec c3016a9bc5
use v1.4.0 consensus spec test vectors (#6072) 2024-03-13 21:42:40 +01:00
Etan Kissling 4e2ffca44a
use fixed max depth for `BlockRef` (#6070)
In `block_dag` there is a max depth of 100 years configured to detect
internal inconsistencies, e.g., circular references. As `BlockRef` was
changed long ago to only reflect the non-finalized chain segment, the
theoretically supported max depth can be reduced and simplified.
2024-03-13 13:01:51 +01:00
Etan Kissling efb1971d7e
pass `cfg` to `ETHBeaconClockCreateFromState` LC API (#6071)
We don't need the `cfg` right now, but it makes sense to have the object
passed to the clock so that the API doesn't break if we want to support
configurable `SECONDS_PER_SLOT`. As the `libnimbus_lc` library is not
yet widely used, better to add the argument now than later.
2024-03-13 13:01:39 +01:00
Etan Kissling 8bd8ffe2bb
align default `syncHorizon` computation logic across networks (#6066)
The `syncHorizon` describes the number of empty slots before the beacon
node considers itself to be out of sync. There are two places where we
currently set this to 50 slots, but it makes more sense to base it on
wall time, e.g., the 10 minutes that the default 50 are derived from.
2024-03-12 21:51:18 +01:00
tersec fd8429da74
rm Web3Signer Bellatrix block support (#6061) 2024-03-11 22:09:31 +00:00
Eugene Kabanov f088e5f57b
Consensus block value calculation for produceBlockV3 API call. (#5873)
* allow specifying get_proposer_reward block root at state.slot

* Add consensus_block_value calculation.

* Address review comments.

* Post-rebase adjustments.

* Use proper state to calculate consensus block value.

* Revert "allow specifying get_proposer_reward block root at state.slot"

This reverts commit 9fef9a8199f63056060527ac2531acc3b0ed8dcb.

* Fix post-revert problems.
Return back to Gwei.

* Adding test which is not working.

* Do not use test suite if it does not have post-state.

* Add debug logging.

* Increase logging to track sources of balance changes.

* Fix sync committee rewards/penalties calculation.

* Revert "Increase logging to track sources of balance changes."

This reverts commit 32feb20f2fdb66521401710866cd59ecc9951ef8.

* Adopt new vision to block rewards.

* Add block produce logging to VC.

* Remove rewards.nim.

* Eliminate toWei changes.

* Improve UInt256 shortLog.

* Fix conversion procedure.

* Address review comments.

* Fix test.

* Revert "Fix test."

This reverts commit 4948b2c1ec.

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
Co-authored-by: Etan Kissling <etan@status.im>
2024-03-11 14:18:50 +00:00
Jacek Sieka 2d519ac715
remove some unused rest serialization code (#6056) 2024-03-09 11:38:24 +01:00
tersec 55ba43592b
req/resp status message requires zero hash for genesis finalized epoch (#6053) 2024-03-08 21:33:59 +01:00
Etan Kissling 1cdd3c62fd
extend `syncEth1Chain tick` log (#6049)
Provide additional context in the `syncEth1Chain tick` debug log to aid
with understanding of flow when debugging on a more precise basis than
just having the metrics.
2024-03-08 19:27:45 +01:00
Etan Kissling 984d5e4631
fix regression when fetching genesis data in trustedNodeSync mode (#6047)
Corrects a regression from #5998 that led to crashes in #6046.
In `trustedNodeSync` mode, the config does not contain genesis keys,
so attempting to load from them is a `Defect`.
2024-03-08 14:52:54 +01:00
Etan Kissling a0bc3fff86
fix `/eth/v1/beacon/deposit_snapshot` for EIP-4881 (#6038)
Fix the `/eth/v1/beacon/deposit_snapshot` API to produce proper EIP-4881
compatible `DepositTreeSnapshot` responses. The endpoint used to expose
a Nimbus-specific database internal format.

Also fix trusted node sync to consume properly formatted EIP-4881 data
with `--with-deposit-snapshot`, and `--finalized-deposit-tree-snapshot`
beacon node launch option to use the EIP-4881 data. Further ensure that
`ncli_testnet` produces EIP-4881 formatted data for interoperability.
2024-03-08 14:22:03 +01:00
tersec 816361ed8c
Bloom filter acceleration for deposit processing (#5982) 2024-03-07 23:38:24 +00:00
tersec 11d522dd86
use debug- prefix for Yamux experimental support (#6010) 2024-03-07 19:52:13 +01:00
Etan Kissling 50a43f397f
rename `DepositTreeSnapshot` -> `DepositContractSnapshot` (#6036)
EIP-4881 was never correctly implemented, the `DepositTreeSnapshot`
structure has nothing to do with its actual definition. Reflect that
by renaming the type to a Nimbus-specific `DepositContractSnapshot`,
so that an actual EIP-4881 implementation can use the correct names.

- https://eips.ethereum.org/EIPS/eip-4881#specification

Notably, `DepositTreeSnapshot` contains a compressed sequence in
`finalized`, only containing the minimally required intermediate roots.

That also explains the incorrect REST response reported in #5508.

The non-canonical representation was introduced in #4303 and is also
persisted in the database. We'll have to maintain it for a while.
2024-03-07 18:42:52 +01:00
Etan Kissling 3b4b1ccb89
metrics for proposer and sync duties (#6028)
To avoid restarting with scheduled proposals or sync duties, which are
rare but have high rewards, add metrics to simpify tracking them.
2024-03-07 15:08:48 +01:00
Etan Kissling b7c52c9537
consider proposal slot in `Slot end`'s `nextActionWait` log/metric (#6026)
`nextActionWait` currently shows `n/a` if only proposal is scheduled
but no attestation, e.g., attestation was already made for current
epoch and validator is exiting next epoch so doesn't have another
attestation lined up. It's an edge case but it's still more correct
to also log `nextActionWait` if only proposal is scheduled.
2024-03-06 12:20:53 +01:00
Etan Kissling 89d9dc24bd
allow `--external-beacon-api-url` to fallback to genesis if viable (#5998)
When using `--external-beacon-api-url`, one has to accompany it with
either `--trusted-block-root` or `--trusted-state-root`. If neither is
specified, we can fallback to a deeply finalized noncontroversial block
root. For networks that started post Altair, e.g., Holesky, the genesis
block root fulfills that requirement, as in, it is implicitly trusted.
Therefore, if only `--external-beacon-api-url` is provided without any
`--trusted-block-root` or `--trusted-state-root`, use genesis block root
if it is a viable starting point (post-Altair).

```
build/nimbus_beacon_node \
    --network=holesky \
    --data-dir="$HOME/Downloads/nimbus/data/holesky" \
    "--external-beacon-api-url=http://unstable.holesky.beacon-api.nimbus.team" \
    --tcp-port=9010 --udp-port=9010 \
    --rest --log-level=DEBUG \
    --no-el
```
2024-03-05 15:41:22 +01:00
Etan Kissling 2c924bcc8c
add test runner for LC data collection tests (#6021)
Introduce a test runner for upcoming EF test suites related to canonical
light client data collection.

- https://github.com/ethereum/consensus-specs/pull/3553
2024-03-05 15:41:09 +01:00
tersec 8222d62500
rm unused phase0 and altair remote signing test code (#6027) 2024-03-05 02:38:34 +01:00
tersec 2a13c09615
add proposer reward accounting to block transitions (#6022)
* add proposer reward accounting to block transitions

* Update beacon_chain/spec/state_transition_block.nim

Co-authored-by: Etan Kissling <etan@status.im>

---------

Co-authored-by: Etan Kissling <etan@status.im>
2024-03-04 17:00:46 +00:00
Etan Kissling bb8c6cda18
bump `nim-web3` to `747001250a27278e0f0f1706fe1dec88ad3ede8d` (#6020)
- Remove error field from AccessListResult
- Add optional yParity field to TransactionObject
2024-03-03 17:17:06 +01:00
tersec 9d56f3920b
fix some styleCheck hints (#6016) 2024-03-03 02:04:45 +01:00
Etan Kissling 542854e743
use `ConsensusFork.init` for parsing lowercase version (#6015)
Instead of maintaining string to `ConsensusFork` version inline,
use shared helper that also comes with static assertion checks
for completion.
2024-03-02 23:45:20 +01:00
tersec b4db011daf
enable --styleCheck:error in ncli/ and research/ (#6014) 2024-03-02 06:25:17 +01:00
Etan Kissling d8b8aee7b7
avoid style check issue with `syncAggregate` (#6013)
Style check confuses `func syncAggregate` because it accesses some other
object's `sync_aggregate` member in the body. Rename the func to avoid.
2024-03-02 02:54:37 +01:00
tersec 8015c0bb47
rm spec/forks capella_mev usage which had been for remote signer (#6009) 2024-03-02 01:12:48 +01:00
tersec e12f5e0bba
rm more Capella builder API remote signer support (#6012) 2024-03-01 22:29:47 +00:00
tersec 0d48535f9a
refactor get_flag_and_inactivity_deltas() loop body to use per-validator get_flag_and_inactivity_delta() (#6007) 2024-03-01 16:31:25 +00:00
tersec 84034c0379
rm Capella builder API-related remote signer support (#6003) 2024-03-01 05:30:09 +00:00
tersec 684de046db
switch Builder API validator registration error to warning (#6005) 2024-03-01 06:25:29 +01:00
tersec f076502e25
rm Capella builder API bid types and blinded block construction (#6002) 2024-03-01 00:02:13 +00:00
tersec 5da2bcd243
rm Capella builder API REST calls (#5997) 2024-02-29 12:37:08 +00:00
Etan Kissling 728f65080f
remove an unused metric in `rest_remote_signer_calls` (#5999)
The `nbc_remote_signer_communication_errors` metric is unused. Remove to
avoid verbose `XDeclaredButNotUsed` hint.
2024-02-29 12:00:15 +00:00
tersec 84b752c7a1
rm REST blinded forked Capella block support (#5994) 2024-02-28 18:27:26 +00:00
Etan Kissling a91366734b
remove unnecessary exports in `eth2_discovery` (#5987)
The exports in `eth2_discovery` produce deprecation warnings as they
refer to `close`, `closeWait` and so on. Turns out that they are not
necessary at all. The `Eth2DiscoveryProtocol` is even already exported
two lines above using `*` marker...
2024-02-28 13:34:16 +00:00
tersec 2b91b66679
rm REST Capella builder API support (#5981) 2024-02-28 01:10:19 +00:00
Jacek Sieka 2ea97e34bc
Merge remote-tracking branch 'origin/stable' into unstable 2024-02-27 14:03:41 +01:00
Zahary Karadjov fc9c72f0eb
Version v24.2.2 2024-02-27 03:24:30 +02:00
Etan Kissling 26d0281e54
fix incorrect config validation regression from #5959 (#5966)
During refactoring of #5959, some implicit `return` were overlooked,
resulting in spurious `err()` being returned without message.

```
{"lvl":"WRN","ts":"2024-02-26 10:12:20.469+00:00","msg":"Beacon nodes report different configuration values","reason":"","service":"fallback_service","node":"http://127.0.0.1:9303[Nimbus/v24.2.1-4e9bc7-stateofus]","node_index":0,"node_roles":"AGBSDT"}
```

Correcting the helpers to return explicit result in all exhaustive
cases so that this cannot happen anymore by accident.
2024-02-26 16:25:12 +02:00
Etan Kissling 4fcfed2f1f
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 16:25:12 +02:00
Etan Kissling c8adfb2336
use correct signing fork after Deneb when using `deposits exit` command (#5954)
In #5120, the `nimbus_beacon_node deposits exit` command was updated for
compatibility with EIP-7044, which forces signatures to be made using
`CAPELLA_FORK_VERSION` regardless of the `VoluntaryExit`'s `epoch` after
Deneb is activated.

This update had a regression, as an older mechanism was used to fetch
`RuntimeConfig`, resulting in an encoding issue (#5362). This was then
fixed in #5370, restoring general `deposits exit` functionality.

However, the logic from #5120 has another flaw, as it uses an incorrect
fork version based on the pre-Deneb logic even after Deneb and EIP-7044
are activated. Fix this now, so that `deposits exit` continues to work
correctly after Deneb activates.
2024-02-26 16:25:12 +02:00
Etan Kissling 48a4955e50
fix EIP-7044 implementation when using batch verification (#5953)
In #5120, EIP-7044 support got added to the state transition function to
force `CAPELLA_FORK_VERSION` to be used when validiting `VoluntaryExit`
messages, irrespective of their `epoch`.

In #5637, similar logic was added when batch verifying BLS signatures,
which is used during gossip validation (libp2p gossipsub, and req/resp).
However, that logic did not match the one introduced in #5120, and only
uses `CAPELLA_FORK_VERSION` when a `VoluntaryExit`'s `epoch` was set to
a value `>= CAPELLA_FORK_EPOCH`. Otherwise, `BELLATRIX_FORK_VERSION`
would still be used when validating `VoluntaryExit`, e.g., with `epoch`
set to `0`, as is the case in this Holesky block:

- https://holesky.beaconcha.in/slot/1076985#voluntary-exits

Extracting the correct logic from #5120 into a function, and reusing it
when verifying BLS signatures fixes this issue, and also leverages the
exhaustive EF test suite that covers the (correct) #5120 logic.

This fix only affects networks that have EIP-7044 applied (post-Deneb).

Without the fix, Deneb blocks with a `VoluntaryExit` with `epoch` set to
`< CAPELLA_FORK_EPOCH` incorrectly fail to validate despite being valid.

Incorrect blocks that contain a malicious `VoluntaryExit` with `epoch`
set to `< CAPELLA_FORK_EPOCH` and signed using `BELLATRIX_FORK_VERSION`
_would_ pass the BLS verification stage, but subsequently fail the state
transition logic. Such blocks would still correctly be labeled invalid.
2024-02-26 16:25:12 +02:00
Etan Kissling c3fe055bd4
fix incorrect config validation regression from #5959 (#5966)
During refactoring of #5959, some implicit `return` were overlooked,
resulting in spurious `err()` being returned without message.

```
{"lvl":"WRN","ts":"2024-02-26 10:12:20.469+00:00","msg":"Beacon nodes report different configuration values","reason":"","service":"fallback_service","node":"http://127.0.0.1:9303[Nimbus/v24.2.1-4e9bc7-stateofus]","node_index":0,"node_roles":"AGBSDT"}
```

Correcting the helpers to return explicit result in all exhaustive
cases so that this cannot happen anymore by accident.
2024-02-26 13:32:01 +01:00
Etan Kissling 4e9bc7f570
add EIP-7044 support to keymanager API (#5959)
* add EIP-7044 support to keymanager API

When trying to sign `VoluntaryExit` via keymanager API, the logic is not
yet aware of EIP-7044 (part of Deneb). This patch adds missing EIP-7044
support to the keymanager API as well.

As part of this, the VC needs to become aware about:

- `CAPELLA_FORK_VERSION`: To correctly form the EIP-7044 signing domain.
  The fork schedule does not indicate which of the results, if any,
  corresponds to Capella.
- `CAPELLA_FORK_EPOCH`: To detect whether Capella was scheduled.
  If a BN does not have it in its config while other BNs have it,
  this leads to a log if Capella has not activated yet, or marks the BN
  as incompatible if Capella already activated.
- `DENEB_FORK_EPOCH`: To check whether EIP-7044 logic should be used.

Related PRs:

- #5120 added support for processing EIP-7044 `VoluntaryExit` messages
  as part of the state transition functions (tested by EF spec tests).
- #5953 synced the support from #5120 to gossip validation.
- #5954 added support to the `nimbus_beacon_node deposits exit` command.
- #5956 contains an alternative generic version of `VCForkConfig`.

* address reviewer feedback: letter case, module location, double lookup

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* Update beacon_chain/rpc/rest_constants.nim

* move `VCRuntimeConfig` back to `rest_types`

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>

* fix `getForkVersion` helper

---------

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2024-02-26 09:48:07 +01:00
Etan Kissling 00510a9d2f
fix `Eth-Consensus-Block-Value` reading (#5964)
Fix regression from #5842 where `Eth-Execution-Payload-Value` is parsed
into `consensusValue` instead of `Eth-consensus-Block-Value`. We don't
use those values for now, but fixing avoids hard-to-debug bugs later.
2024-02-26 09:41:41 +01:00